lakefs-spec: An fsspec backend for lakeFS¶
Early Adopters
We are seeking early adopters who would like to actively participate in our feedback process and shape the future of the library. If you are interested in using the library and want to get in touch with us, please reach out via Github Discussions.
Welcome to lakeFS-spec, a filesystem-spec backend implementation for the lakeFS data lake. Our primary goal is to streamline versioned data operations in lakeFS, enabling seamless integration with popular data science tools such as Pandas, Polars, and DuckDB directly from Python.
Highlights:
- High-level abstraction over basic lakeFS repository operations
- Seamless integration into the fsspec ecosystem
- Transaction support
- Zero-config option through config autodiscovery
- Automatic up-/download management to avoid unnecessary transfers for unchanged files