We’re excited to ship Sail 0.5! This release introduces a redesigned control plane for distributed execution to support task region scheduling, unified shuffle, and failure recovery. A system catalog is also available to observe internal states via SQL.
AI workflows rely on Python, but Spark isolates Python behind an inter-process boundary, a constraint even accelerators like Photon can’t eliminate. Sail executes Python UDFs natively in-process for true high-performance distributed compute.
Sail adopts Apache DataFusion’s trusted query engine and extends it as part of a larger, distributed compute framework with Spark-compatible semantics.
The future of data is composable. Sail brings distributed computation into that vision—modular, Arrow-native, and Spark-compatible.
Sail 0.4 introduces native Apache Iceberg support and major improvements to Delta Lake integration.
Sail turns one! Celebrate with us as we reflect on our journey, and look ahead to the future of unified data and AI workloads.
Sail 0.3.2 brings native Delta Lake read/write support and expanded object storage integration with Azure, GCS, Cloudflare R2, and AWS S3.
Sail 0.3 adds support for Spark 4.0 while maintaining compatibility with Spark 3.5. This is the latest evidence of our long-term commitment to your data processing needs.
The LakeSail team is thrilled to announce the release of Sail 0.2.6! This version delivers a range of enhancements across temporal data handling, SQL compatibility, Parquet integration, and the MCP server.
With the Sail MCP server, data analytics in Spark is possible for both LLM agents and humans.
Join us for a close look at Sail’s new in-house SQL parser built using parser combinators and Rust procedural macros.
Rust is redefining big data infrastructure by offering superior performance, memory safety, and scalability over traditional JVM-based systems.
Join us in a brief tour of how the enhanced UDF support opens up possibilities to bridge the gap between traditional ETL workloads and AI.
Announcing Sail 0.2—the latest milestone in the evolution of distributed compute frameworks. Explore how advancements in programming languages and data infrastructure make it possible to unify batch, stream, and AI workloads into a seamless, high-performance framework.
We are thrilled to unveil the preview release of Sail 0.2, which introduces support for distributed processing on Kubernetes! Tune in for a detailed architectural deep dive and an overview of our increased support for Spark.
Discover how Sail Enterprise Support empowers your team with dedicated, flexible, and customizable solutions to meet the needs of your organization.
Deriving insights from your data shouldn’t cost you an arm, a leg, and a kidney. Check out this Sail recipe to learn how you can work directly with your data in Amazon S3, saving both time and money.
We are thrilled to announce the 0.1 release of Sail. Get started with the PySail package today, and check out the documentation site.
The preview of Sail is here! In the derived TPC-H benchmark, Sail achieves nearly 4x speed-up and 94% hardware cost reduction, with the same PySpark code.