How Sail Utilizes and Extends Apache DataFusion
Sail adopts Apache DataFusion’s trusted query engine and extends it as part of a larger, distributed compute framework with Spark-compatible semantics.
Shehab Amin
Co-Founder & CEO
Shehab leads LakeSail while actively contributing to open source projects that push the boundaries of big data infrastructure. Early in his career, Shehab developed a passion for data engineering and co-founded Tradery Capital, an autonomous AI-powered financial modeling trading fund. He grew up in the Bay Area and earned his bachelor’s degree in Computer Science from UC Berkeley.
Sail, the Last Piece of the Composable Data Stack
The future of data is composable. Sail brings distributed computation into that vision—modular, Arrow-native, and Spark-compatible.
Sail 0.4: Native Apache Iceberg Support
Sail 0.4 introduces native Apache Iceberg support and major improvements to Delta Lake integration.
Sail Turns One
Sail turns one! Celebrate with us as we reflect on our journey, and look ahead to the future of unified data and AI workloads.
Sail 0.3.2: Start the Journey from Your Lakehouse
Sail 0.3.2 brings native Delta Lake read/write support and expanded object storage integration with Azure, GCS, Cloudflare R2, and AWS S3.
Sail 0.3: Long Live Spark
Sail 0.3 adds support for Spark 4.0 while maintaining compatibility with Spark 3.5. This is the latest evidence of our long-term commitment to your data processing needs.
Announcing Sail 0.2.6
The LakeSail team is thrilled to announce the release of Sail 0.2.6! This version delivers a range of enhancements across temporal data handling, SQL compatibility, Parquet integration, and the MCP server.
Sail MCP Server: Spark Analytics for LLM Agents
With the Sail MCP server, data analytics in Spark is possible for both LLM agents and humans.
Writing a Rust SQL Parser in One Week
Join us for a close look at Sail’s new in-house SQL parser built using parser combinators and Rust procedural macros.
Beyond the JVM: How Rust is Redefining Big Data for the AI Era
Rust is redefining big data infrastructure by offering superior performance, memory safety, and scalability over traditional JVM-based systems.
Sail 0.2.1: Enhanced UDF Support and Steps towards Full Spark Parity
Join us in a brief tour of how the enhanced UDF support opens up possibilities to bridge the gap between traditional ETL workloads and AI.
Why It’s Possible Now: Sail 0.2 and the Evolution of Distributed Compute Frameworks
Announcing Sail 0.2—the latest milestone in the evolution of distributed compute frameworks. Explore how advancements in programming languages and data infrastructure make it possible to unify batch, stream, and AI workloads into a seamless, high-performance framework.
Sail 0.2 and the Future of Distributed Processing
We are thrilled to unveil the preview release of Sail 0.2, which introduces support for distributed processing on Kubernetes! Tune in for a detailed architectural deep dive and an overview of our increased support for Spark.
Introducing Sail Enterprise Support
Discover how Sail Enterprise Support empowers your team with dedicated, flexible, and customizable solutions to meet the needs of your organization.
A Sail Recipe: Tackling an Out-of-Control Redshift Bill
Deriving insights from your data shouldn't cost you an arm, a leg, and a kidney. Check out this Sail recipe to learn how you can work directly with your data in Amazon S3, saving both time and money.
The First PySail Release
We are thrilled to announce the 0.1 release of Sail. Get started with the PySail package today, and check out the documentation site.
Supercharge Spark: Quadruple Speed, Cut Costs by 94%
The preview of Sail is here! In the derived TPC-H benchmark, Sail achieves nearly 4x speed-up and 94% hardware cost reduction, with the same PySpark code.