Big Data Processing for the AI Era

Sail: An open-source computation framework with a mission to unify batch processing, stream processing, and compute-intensive AI workloads.

Documentation

Apache Spark → Sail

Processing Speed

94% ↓

Hardware Costs

Code Changes

The Sail architecture is designed to shape the future of distributed processing.

Curious How Much You Could Save?

Let us show you. We offer a free, no-obligation comparison of your current Spark workloads against Sail — see exactly how much time and money you could be saving.

Let our engineers handle the setup.

Get Started with Sail

Sail features a drop-in replacement for Spark SQL and the Spark DataFrame API in both single-host and distributed settings.

bash

pip install "pysail[spark]"

bash

sail spark server --port 50051

python

from pysail.spark import SparkConnectServer

server = SparkConnectServer(port=50051)
server.start(background=False)

bash

kubectl apply -f sail.yaml
kubectl -n sail port-forward service/sail-spark-server 50051:50051

Once you have a running Sail server, you can connect to it in PySpark.
No changes are needed in your PySpark code!

python

from pyspark.sql import SparkSession

spark = SparkSession.builder.remote("sc://localhost:50051").getOrCreate()
spark.sql("SELECT 1 + 1").show()

Sail Support Options

LakeSail offers commercial support for Sail, with flexible coverage tailored to your needs. Get in touch for more details.

Community

Free

Public issue tracker
Public Slack channel
No guaranteed response time
SQL and DataFrame API support only

Enterprise

Negotiable

Private issue tracker
Private Slack channel
Guaranteed response time
Custom integrations
Flexible scope and support