Announcing Sail 0.2.6

The Sail 0.2.6 release is all about enhancing the user experience, with improvements in temporal data handling, more accurate Parquet file reading, and expanded functionality in the MCP server. Here’s a look at the key updates in this release.

Improved Temporal Data Casting and Display ​

Temporal data handling has been significantly improved to enhance usability and consistency. Users can now cast timestamps, dates, intervals, structs, and maps to strings, as well as cast strings to timestamps, dates, and intervals. Timestamps can also be cast to integers, following Spark’s behavior of truncating to whole seconds. The display of timestamp, date, and interval literals has been improved for better parity with Spark. This update also introduces rigorous, more consistent mapping between Spark interval types and Apache Arrow data types, resolving earlier inconsistencies.

Accurate Reading of Parquet INT96 Timestamps ​

Parquet INT96 data types are now read as timestamps with microsecond precision, aligning more closely with Spark’s behavior. While this marks a key improvement, full compatibility with Spark’s Parquet reader remains a work in progress. Future updates will address NTZ-to-LTZ conversion and support for the org.apache.spark.sql.parquet.row.metadata metadata key in Parquet files.

Support for Column Metadata in the Spark DataFrame API ​

Column metadata is now supported in the Spark DataFrame API, addressing previous issues where metadata was ignored and metadata value data types were not properly preserved.

Improved SQL Alias Support in GROUP BY and HAVING Clauses ​

It’s now possible to reference aliased expressions from the SELECT clause in GROUP BY and HAVING clauses. Ambiguities are properly handled, even when the same name appears in both the query input and the aggregated output.

Expanded MCP Server Capabilities ​

New endpoints have been added to the MCP server, including list_local_directories, list_s3_directories, create_csv_view, and create_json_view, alongside the existing create_parquet_view endpoint. These additions make it easier to explore data sources and create views from a broader range of formats.

Looking Forward ​

Sail 0.2.6 marks a significant step forward in improving data compatibility, usability, and performance across the framework. We’re excited to continue building on these improvements in future releases and encourage you to upgrade and explore the new capabilities. As always, your feedback is invaluable—let us know what features or improvements would make Sail even better for your workflows.

To try Sail or contribute, visit github.com/lakehq/sail. Join our growing community at lakesail.com/slack.