Skip to content

Home / reference / virtual

icechunk.virtual#

Virtual chunk containers and specs for referencing external data.

icechunk.virtual #

Classes:

Name Description
VirtualChunkContainer

A virtual chunk container is a configuration that allows Icechunk to read virtual references from a storage backend.

VirtualChunkSpec

The specification for a virtual chunk reference.

VirtualChunkContainer #

A virtual chunk container is a configuration that allows Icechunk to read virtual references from a storage backend.

Attributes:

Name Type Description
name str | None

Optional name for this container. When set, chunks can use relative vcc://name/path locations instead of full absolute URLs.

url_prefix str

The prefix of urls that will use this containers configuration for reading virtual references.

store ObjectStoreConfig

The storage backend to use for the virtual chunk container.

Methods:

Name Description
__new__

Create a new VirtualChunkContainer object

Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
class VirtualChunkContainer:
    """A virtual chunk container is a configuration that allows Icechunk to read virtual references from a storage backend.

    Attributes
    ----------
    name: str | None
        Optional name for this container. When set, chunks can use relative ``vcc://name/path``
        locations instead of full absolute URLs.
    url_prefix: str
        The prefix of urls that will use this containers configuration for reading virtual references.
    store: ObjectStoreConfig
        The storage backend to use for the virtual chunk container.
    """

    name: str | None
    url_prefix: str
    store: ObjectStoreConfig

    def __new__(
        cls,
        url_prefix: str,
        store: _AnyObjectStoreConfig,
        name: str | None = None,
    ) -> VirtualChunkContainer:
        """
        Create a new `VirtualChunkContainer` object

        Parameters
        ----------
        url_prefix: str
            The prefix of urls that will use this containers configuration for reading virtual references.
        store: ObjectStoreConfig
            The storage backend to use for the virtual chunk container.
        name: str | None
            Optional name for this container. When set, chunks can use relative ``vcc://name/path``
            locations instead of full absolute URLs.
        """

    def __repr__(self) -> str: ...
    def __str__(self) -> str: ...
    def _repr_html_(self) -> str: ...

__new__ #

__new__(url_prefix, store, name=None)

Create a new VirtualChunkContainer object

Parameters:

Name Type Description Default
url_prefix str

The prefix of urls that will use this containers configuration for reading virtual references.

required
store _AnyObjectStoreConfig

The storage backend to use for the virtual chunk container.

required
name str | None

Optional name for this container. When set, chunks can use relative vcc://name/path locations instead of full absolute URLs.

None
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
def __new__(
    cls,
    url_prefix: str,
    store: _AnyObjectStoreConfig,
    name: str | None = None,
) -> VirtualChunkContainer:
    """
    Create a new `VirtualChunkContainer` object

    Parameters
    ----------
    url_prefix: str
        The prefix of urls that will use this containers configuration for reading virtual references.
    store: ObjectStoreConfig
        The storage backend to use for the virtual chunk container.
    name: str | None
        Optional name for this container. When set, chunks can use relative ``vcc://name/path``
        locations instead of full absolute URLs.
    """

VirtualChunkSpec #

The specification for a virtual chunk reference.

Attributes:

Name Type Description
etag_checksum str | None

Optional object store e-tag for the containing object.

index list[int]

The chunk index, in chunk coordinates space

last_updated_at_checksum datetime | None

Optional timestamp for the containing object.

length int

The length of the chunk in bytes

location str

The URL to the virtual chunk data, something like 's3://bucket/foo.nc'

offset int

The chunk offset within the pointed object, in bytes

Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
class VirtualChunkSpec:
    """The specification for a virtual chunk reference."""
    @property
    def index(self) -> list[int]:
        """The chunk index, in chunk coordinates space"""
        ...
    @property
    def location(self) -> str:
        """The URL to the virtual chunk data, something like 's3://bucket/foo.nc'"""
        ...
    @property
    def offset(self) -> int:
        """The chunk offset within the pointed object, in bytes"""
        ...
    @property
    def length(self) -> int:
        """The length of the chunk in bytes"""
        ...
    @property
    def etag_checksum(self) -> str | None:
        """Optional object store e-tag for the containing object.

        Icechunk will refuse to serve data from this chunk if the etag has changed.
        """
        ...
    @property
    def last_updated_at_checksum(self) -> datetime.datetime | None:
        """Optional timestamp for the containing object.

        Icechunk will refuse to serve data from this chunk if it has been modified in object store after this time.
        """
        ...

    def __new__(
        cls,
        index: list[int],
        location: str,
        offset: int,
        length: int,
        etag_checksum: str | None = None,
        last_updated_at_checksum: datetime.datetime | None = None,
    ) -> VirtualChunkSpec: ...

etag_checksum property #

etag_checksum

Optional object store e-tag for the containing object.

Icechunk will refuse to serve data from this chunk if the etag has changed.

index property #

index

The chunk index, in chunk coordinates space

last_updated_at_checksum property #

last_updated_at_checksum

Optional timestamp for the containing object.

Icechunk will refuse to serve data from this chunk if it has been modified in object store after this time.

length property #

length

The length of the chunk in bytes

location property #

location

The URL to the virtual chunk data, something like 's3://bucket/foo.nc'

offset property #

offset

The chunk offset within the pointed object, in bytes