Home / reference / credentials
icechunk.credentials#
Credential types and factory functions for S3, GCS, and Azure.
icechunk.credentials #
Classes:
| Name | Description |
|---|---|
AzureCredentials | Credentials for an azure storage backend |
AzureRefreshableCredential | A refreshable credential for Azure storage with optional expiration. |
AzureStaticCredentials | Credentials for an azure storage backend |
GcsBearerCredential | Credentials for a google cloud storage backend |
GcsCredentials | Credentials for a google cloud storage backend |
GcsStaticCredentials | Credentials for a google cloud storage backend |
S3Credentials | Credentials for an S3 storage backend |
S3StaticCredentials | Credentials for an S3 storage backend |
Functions:
| Name | Description |
|---|---|
azure_credentials | Create credentials Azure Blob Storage object store. |
azure_from_env_credentials | Instruct Azure Blob Storage object store to fetch credentials from the operative system environment. |
azure_refreshable_credentials | Create refreshable credentials for Azure Blob Storage object store. |
azure_static_credentials | Create static credentials Azure Blob Storage object store. |
containers_credentials | Build a map of credentials for virtual chunk containers. |
gcs_anonymous_credentials | Create anonymous credentials for Google Cloud Storage object store. |
gcs_credentials | Create credentials Google Cloud Storage object store. |
gcs_from_env_credentials | Instruct Google Cloud Storage object store to fetch credentials from the operative system environment. |
gcs_refreshable_credentials | Create refreshable credentials for Google Cloud Storage object store. |
gcs_static_credentials | Create static credentials Google Cloud Storage object store. |
s3_anonymous_credentials | Create no-signature credentials for S3 and S3 compatible object stores. |
s3_credentials | Create credentials for S3 and S3 compatible object stores. |
s3_from_env_credentials | Instruct S3 and S3 compatible object stores to gather credentials from the operative system environment. |
s3_refreshable_credentials | Create refreshable credentials for S3 and S3 compatible object stores. |
s3_static_credentials | Create static credentials for S3 and S3 compatible object stores. |
AzureCredentials #
Credentials for an azure storage backend
This can be used to authenticate with an azure storage backend.
Classes:
| Name | Description |
|---|---|
FromEnv | Uses credentials from environment variables |
Refreshable | Allows for an outside authority to pass in a function that can be used to provide credentials. |
Static | Uses azure credentials without expiration |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
FromEnv #
Refreshable #
Allows for an outside authority to pass in a function that can be used to provide credentials.
This is useful for credentials that have an expiration time, or are otherwise not known ahead of time.
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
AzureRefreshableCredential #
A refreshable credential for Azure storage with optional expiration.
Classes:
| Name | Description |
|---|---|
AccessKey | Refreshable access key credential. |
BearerToken | Refreshable bearer token credential. |
SasToken | Refreshable SAS token credential. |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
AccessKey #
Refreshable access key credential.
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
BearerToken #
Refreshable bearer token credential.
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
SasToken #
Refreshable SAS token credential.
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
AzureStaticCredentials #
Credentials for an azure storage backend
Classes:
| Name | Description |
|---|---|
AccessKey | Credentials for an azure storage backend using an access key |
BearerToken | Credentials for an azure storage backend using a bearer token |
SasToken | Credentials for an azure storage backend using a shared access signature token |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
AccessKey #
Credentials for an azure storage backend using an access key
Methods:
| Name | Description |
|---|---|
__new__ | Create access key credentials. |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
__new__ #
Create access key credentials.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key | str | The access key to use for authentication. | required |
BearerToken #
Credentials for an azure storage backend using a bearer token
Methods:
| Name | Description |
|---|---|
__new__ | Create bearer token credentials. |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
__new__ #
Create bearer token credentials.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
token | str | The bearer token to use for authentication. | required |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
SasToken #
Credentials for an azure storage backend using a shared access signature token
Methods:
| Name | Description |
|---|---|
__new__ | Create SAS token credentials. |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
__new__ #
Create SAS token credentials.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
token | str | The shared access signature token to use for authentication. | required |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
GcsBearerCredential #
Credentials for a google cloud storage backend
This is a bearer token that has an expiration time.
Methods:
| Name | Description |
|---|---|
__new__ | Create a GcsBearerCredential object |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
__new__ #
Create a GcsBearerCredential object
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
bearer | str | The bearer token to use for authentication. | required |
expires_after | datetime | None | The expiration time of the bearer token. | None |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
GcsCredentials #
Credentials for a google cloud storage backend
This can be used to authenticate with a google cloud storage backend.
Classes:
| Name | Description |
|---|---|
Anonymous | Uses anonymous credentials |
FromEnv | Uses credentials from environment variables |
Refreshable | Allows for an outside authority to pass in a function that can be used to provide credentials. |
Static | Uses gcs credentials without expiration |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
Anonymous #
FromEnv #
Refreshable #
Allows for an outside authority to pass in a function that can be used to provide credentials.
This is useful for credentials that have an expiration time, or are otherwise not known ahead of time.
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
GcsStaticCredentials #
Credentials for a google cloud storage backend
Classes:
| Name | Description |
|---|---|
ApplicationCredentials | Credentials for a google cloud storage backend using application default credentials |
BearerToken | Credentials for a google cloud storage backend using a bearer token |
ServiceAccount | Credentials for a google cloud storage backend using a service account json file |
ServiceAccountKey | Credentials for a google cloud storage backend using a serialized service account key |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
ApplicationCredentials #
Credentials for a google cloud storage backend using application default credentials
Methods:
| Name | Description |
|---|---|
__new__ | Create application default credentials. |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
__new__ #
Create application default credentials.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path | str | The path to the application default credentials (ADC) file. | required |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
BearerToken #
Credentials for a google cloud storage backend using a bearer token
Methods:
| Name | Description |
|---|---|
__new__ | Create bearer token credentials. |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
__new__ #
Create bearer token credentials.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
token | str | The bearer token to use for authentication. | required |
ServiceAccount #
Credentials for a google cloud storage backend using a service account json file
Methods:
| Name | Description |
|---|---|
__new__ | Create service account credentials. |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
__new__ #
Create service account credentials.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path | str | The path to the service account json file. | required |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
ServiceAccountKey #
Credentials for a google cloud storage backend using a serialized service account key
Methods:
| Name | Description |
|---|---|
__new__ | Create service account key credentials. |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
__new__ #
Create service account key credentials.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key | str | The serialized service account key. | required |
S3Credentials #
Credentials for an S3 storage backend
Classes:
| Name | Description |
|---|---|
Anonymous | Does not sign requests, useful for public buckets |
FromEnv | Uses credentials from environment variables |
Refreshable | Allows for an outside authority to pass in a function that can be used to provide credentials. |
Static | Uses s3 credentials without expiration |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
Anonymous #
FromEnv #
Refreshable #
Allows for an outside authority to pass in a function that can be used to provide credentials.
This is useful for credentials that have an expiration time, or are otherwise not known ahead of time.
Methods:
| Name | Description |
|---|---|
__new__ | Create refreshable S3 credentials. |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
__new__ #
Create refreshable S3 credentials.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
pickled_function | bytes | The pickled function to use to provide credentials. | required |
current | S3StaticCredentials | None | The initial credentials. They will be returned the first time credentials are requested and then deleted. | None |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
Static #
Uses s3 credentials without expiration
Methods:
| Name | Description |
|---|---|
__new__ | Create static S3 credentials. |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
__new__ #
Create static S3 credentials.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
credentials | S3StaticCredentials | The credentials to use for authentication. | required |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
S3StaticCredentials #
Credentials for an S3 storage backend
Attributes: access_key_id: str The access key ID to use for authentication. secret_access_key: str The secret access key to use for authentication. session_token: str | None The session token to use for authentication. expires_after: datetime.datetime | None Optional, the expiration time of the credentials.
Methods:
| Name | Description |
|---|---|
__new__ | Create a new |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
__new__ #
Create a new S3StaticCredentials object
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
access_key_id | str | The access key ID to use for authentication. | required |
secret_access_key | str | The secret access key to use for authentication. | required |
session_token | str | None | Optional, the session token to use for authentication. | None |
expires_after | datetime | None | Optional, the expiration time of the credentials. | None |
Source code in icechunk-python/python/icechunk/_icechunk_python.pyi
azure_credentials #
azure_credentials(
*,
access_key=None,
sas_token=None,
bearer_token=None,
from_env=None,
get_credentials=None,
scatter_initial_credentials=False,
)
Create credentials Azure Blob Storage object store.
If all arguments are None, credentials are fetched from the operative system environment.
Source code in icechunk-python/python/icechunk/credentials.py
azure_from_env_credentials #
Instruct Azure Blob Storage object store to fetch credentials from the operative system environment.
azure_refreshable_credentials #
Create refreshable credentials for Azure Blob Storage object store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
get_credentials | Callable[[], AzureRefreshableCredential] | Use this function to get and refresh the credentials. The function must be picklable. | required |
scatter_initial_credentials | bool | Immediately call and store the value returned by get_credentials. This is useful if the repo or session will be pickled to generate many copies. Passing scatter_initial_credentials=True will ensure all those copies don't need to call get_credentials immediately. After the initial set of credentials has expired, the cached value is no longer used. Notice that credentials obtained are stored, and they can be sent over the network if you pickle the session/repo. | False |
Source code in icechunk-python/python/icechunk/credentials.py
azure_static_credentials #
Create static credentials Azure Blob Storage object store.
Source code in icechunk-python/python/icechunk/credentials.py
containers_credentials #
Build a map of credentials for virtual chunk containers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
m | Mapping[str, AnyS3Credential | AnyGcsCredential | AnyAzureCredential | None] | A mapping from container url prefixes to credentials. | required |
Examples:
import icechunk as ic
config = ic.RepositoryConfig.default()
config.inline_chunk_threshold_bytes = 512
virtual_store_config = ic.s3_store(
region="us-east-1",
endpoint_url="http://localhost:4200",
allow_http=True,
s3_compatible=True,
force_path_style=True,
)
container = ic.VirtualChunkContainer("s3://somebucket", virtual_store_config)
config.set_virtual_chunk_container(container)
credentials = ic.containers_credentials(
{"s3://somebucket": ic.s3_credentials(access_key_id="ACCESS_KEY", secret_access_key="SECRET"}
)
repo = ic.Repository.create(
storage=ic.local_filesystem_storage(store_path),
config=config,
authorize_virtual_chunk_access=credentials,
)
Source code in icechunk-python/python/icechunk/credentials.py
gcs_anonymous_credentials #
Create anonymous credentials for Google Cloud Storage object store.
gcs_credentials #
gcs_credentials(
*,
service_account_file=None,
service_account_key=None,
application_credentials=None,
bearer_token=None,
from_env=None,
anonymous=None,
get_credentials=None,
scatter_initial_credentials=False,
)
Create credentials Google Cloud Storage object store.
If all arguments are None, credentials are fetched from the operative system environment.
Source code in icechunk-python/python/icechunk/credentials.py
gcs_from_env_credentials #
Instruct Google Cloud Storage object store to fetch credentials from the operative system environment.
gcs_refreshable_credentials #
Create refreshable credentials for Google Cloud Storage object store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
get_credentials | Callable[[], GcsBearerCredential] | Use this function to get and refresh the credentials. The function must be pickable. | required |
scatter_initial_credentials | bool | Immediately call and store the value returned by get_credentials. This is useful if the repo or session will be pickled to generate many copies. Passing scatter_initial_credentials=True will ensure all those copies don't need to call get_credentials immediately. After the initial set of credentials has expired, the cached value is no longer used. Notice that credentials obtained are stored, and they can be sent over the network if you pickle the session/repo. | False |
Source code in icechunk-python/python/icechunk/credentials.py
gcs_static_credentials #
gcs_static_credentials(
*,
service_account_file=None,
service_account_key=None,
application_credentials=None,
bearer_token=None,
)
Create static credentials Google Cloud Storage object store.
Source code in icechunk-python/python/icechunk/credentials.py
s3_anonymous_credentials #
Create no-signature credentials for S3 and S3 compatible object stores.
s3_credentials #
s3_credentials(
*,
access_key_id=None,
secret_access_key=None,
session_token=None,
expires_after=None,
anonymous=None,
from_env=None,
get_credentials=None,
scatter_initial_credentials=False,
)
Create credentials for S3 and S3 compatible object stores.
If all arguments are None, credentials are fetched from the environment.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
access_key_id | str | None | S3 credential access key | None |
secret_access_key | str | None | S3 credential secret access key | None |
session_token | str | None | Optional S3 credential session token | None |
expires_after | datetime | None | Optional expiration for the object store credentials | None |
anonymous | bool | None | If set to True requests to the object store will not be signed | None |
from_env | bool | None | Fetch credentials from the operative system environment | None |
get_credentials | Callable[[], S3StaticCredentials] | None | Use this function to get and refresh object store credentials | None |
scatter_initial_credentials | bool | Immediately call and store the value returned by get_credentials. This is useful if the repo or session will be pickled to generate many copies. Passing scatter_initial_credentials=True will ensure all those copies don't need to call get_credentials immediately. After the initial set of credentials has expired, the cached value is no longer used. Notice that credentials obtained are stored, and they can be sent over the network if you pickle the session/repo. | False |
Source code in icechunk-python/python/icechunk/credentials.py
147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 | |
s3_from_env_credentials #
Instruct S3 and S3 compatible object stores to gather credentials from the operative system environment.
s3_refreshable_credentials #
Create refreshable credentials for S3 and S3 compatible object stores.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
get_credentials | Callable[[], S3StaticCredentials] | Use this function to get and refresh the credentials. The function must be pickable. | required |
scatter_initial_credentials | bool | Immediately call and store the value returned by get_credentials. This is useful if the repo or session will be pickled to generate many copies. Passing scatter_initial_credentials=True will ensure all those copies don't need to call get_credentials immediately. After the initial set of credentials has expired, the cached value is no longer used. Notice that credentials obtained are stored, and they can be sent over the network if you pickle the session/repo. | False |
Source code in icechunk-python/python/icechunk/credentials.py
s3_static_credentials #
s3_static_credentials(
*,
access_key_id,
secret_access_key,
session_token=None,
expires_after=None,
)
Create static credentials for S3 and S3 compatible object stores.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
access_key_id | str | S3 credential access key | required |
secret_access_key | str | S3 credential secret access key | required |
session_token | str | None | Optional S3 credential session token | None |
expires_after | datetime | None | Optional expiration for the object store credentials | None |