Skip to content

file

File-like objects returned by LakeFSFileSystem.open().

These are thin subclasses of the lakeFS SDK's ObjectReader and ObjectWriter, which translate lakeFS API errors into Python builtin exceptions.

LakeFSObjectReader

Bases: _ErrorTranslationMixin, ObjectReader, Generic[AnyStr]

A file-like object for reading from lakeFS, raising Python builtin exceptions on lakeFS API errors.

Returned by LakeFSFileSystem.open() in read mode. The type parameter is bytes for binary modes and str for text modes.

Source code in src/lakefs_spec/file.py 
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
class LakeFSObjectReader(_ErrorTranslationMixin, ObjectReader, Generic[AnyStr]):
    """
    A file-like object for reading from lakeFS, raising Python builtin exceptions
    on lakeFS API errors.

    Returned by ``LakeFSFileSystem.open()`` in read mode.
    The type parameter is ``bytes`` for binary modes and ``str`` for text modes.
    """

    def read(self, n: int | None = None) -> AnyStr:
        with self._translate_errors():
            return cast(AnyStr, super().read(n))

    def readline(self, limit: int = -1) -> AnyStr:
        with self._translate_errors():
            return cast(AnyStr, super().readline(limit))

    def __iter__(self) -> Iterator[AnyStr]:
        return self

    def __next__(self) -> AnyStr:
        return cast(AnyStr, super().__next__())

    def seek(self, offset: int, whence: int = 0) -> int:
        # Seeking from the end requires a stat call to the lakeFS server.
        with self._translate_errors():
            return super().seek(offset, whence)

LakeFSObjectWriter

Bases: _ErrorTranslationMixin, ObjectWriter, Generic[AnyStr]

A file-like object for writing to lakeFS, raising Python builtin exceptions on lakeFS API errors.

Returned by LakeFSFileSystem.open() in write mode. The type parameter is bytes for binary modes and str for text modes, mirroring typing.IO. Data is buffered locally and only uploaded to the lakeFS server on close().

Source code in src/lakefs_spec/file.py 
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
class LakeFSObjectWriter(_ErrorTranslationMixin, ObjectWriter, Generic[AnyStr]):
    """
    A file-like object for writing to lakeFS, raising Python builtin exceptions on lakeFS API errors.

    Returned by ``LakeFSFileSystem.open()`` in write mode. The type parameter is ``bytes`` for binary modes
    and ``str`` for text modes, mirroring ``typing.IO``.
    Data is buffered locally and only uploaded to the lakeFS server on ``close()``.
    """

    # The lakeFS SDK declares `write` with a method-scoped `AnyStr`,
    # i.e. every writer accepts both str and bytes.
    # Narrowing to the class type parameter is intentional:
    # writing str to a binary writer fails at upload time.
    def write(self, s: AnyStr) -> int:  # ty: ignore[invalid-method-override]
        return super().write(s)

    def close(self) -> None:
        with self._translate_errors():
            super().close()