ebonite.repository.dataset.artifact module

class ebonite.repository.dataset.artifact.DatasetReader

Bases: ebonite.repository.dataset.artifact.DatasetReader, pyjackson.decorators.SubtypeRegisterMixin

ABC for reading Dataset from files (artifacts) to use with ArtifactDatasetSource

type = 'pyjackson.decorators.DatasetReader'

class ebonite.repository.dataset.artifact.DatasetWriter

Bases: ebonite.repository.dataset.artifact.DatasetWriter, pyjackson.decorators.SubtypeRegisterMixin

ABC for writing Dataset to files (artifacts) to use with ArtifactDatasetSource

type = 'pyjackson.decorators.DatasetWriter'

class ebonite.repository.dataset.artifact.OneFileDatasetReader(dataset_type: ebonite.core.objects.dataset_type.DatasetType)

Bases: ebonite.repository.dataset.artifact.DatasetReader

read(artifacts: ebonite.core.objects.artifacts.ArtifactCollection) → ebonite.core.objects.dataset_source.Dataset

Method to read Dataset from artifacts

Parameters:artifacts – artifacts to read

convert(payload: bytes)

type = 'ebonite.repository.dataset.artifact.OneFileDatasetReader'

class ebonite.repository.dataset.artifact.OneFileDatasetWriter

Bases: ebonite.repository.dataset.artifact.DatasetWriter

FILENAME = 'data'

convert(instance) → bytes

write(dataset: ebonite.core.objects.dataset_source.Dataset) → Tuple[ebonite.repository.dataset.artifact.DatasetReader, ebonite.core.objects.artifacts.ArtifactCollection]

Method to write dataset to artifacts

Parameters:dataset – dataset to write Returns:tuple of DatasetReader and ArtifactCollection.

DatasetReader must produce the same dataset if used with same artifacts

type = 'ebonite.repository.dataset.artifact.OneFileDatasetWriter'

class ebonite.repository.dataset.artifact.PrimitiveDatasetReader(dataset_type: ebonite.core.objects.dataset_type.DatasetType)

Bases: ebonite.repository.dataset.artifact.OneFileDatasetReader

convert(payload: bytes)

type = 'ebonite.repository.dataset.artifact.PrimitiveDatasetReader'

class ebonite.repository.dataset.artifact.PrimitiveDatasetWriter

Bases: ebonite.repository.dataset.artifact.OneFileDatasetWriter

convert(instance) → bytes

type = 'ebonite.repository.dataset.artifact.PrimitiveDatasetWriter'

class ebonite.repository.dataset.artifact.PickleReader(dataset_type: ebonite.core.objects.dataset_type.DatasetType)

Bases: ebonite.repository.dataset.artifact.OneFileDatasetReader

convert(payload: bytes)

type = 'ebonite.repository.dataset.artifact.PickleReader'

class ebonite.repository.dataset.artifact.PickleWriter

Bases: ebonite.repository.dataset.artifact.OneFileDatasetWriter

convert(instance) → bytes

type = 'ebonite.repository.dataset.artifact.PickleWriter'

class ebonite.repository.dataset.artifact.ArtifactDatasetRepository(repo: ebonite.repository.artifact.base.ArtifactRepository)

Bases: ebonite.repository.dataset.base.DatasetRepository

DatasetRpository implementation that saves datasets as artifacts to ArtifactRepository

Parameters:repo – underlying ArtifactRepository

ARTIFACT_TYPE = 'datasets'

save(dataset_id: str, dataset: ebonite.core.objects.dataset_source.Dataset) → ebonite.core.objects.dataset_source.DatasetSource

Method to save dataset to this repository

Parameters:

• dataset_id – string identifier
• dataset – dataset to save

Returns:

DatasetSource that produces same Dataset

delete(dataset_id: str)

Method to delete dataset from this repository

Parameters:dataset_id – dataset identifier

class ebonite.repository.dataset.artifact.ArtifactDatasetSource(reader: ebonite.repository.dataset.artifact.DatasetReader, artifacts: ebonite.core.objects.artifacts.ArtifactCollection, dataset_type: ebonite.core.objects.dataset_type.DatasetType)

Bases: ebonite.core.objects.dataset_source.DatasetSource

DatasetSource for reading datasets from ArtifactDatasetRepository

Parameters:

• reader – DatasetReader for this dataset
• artifacts – ArtifactCollection with actual files
• dataset_type – DatasetType of contained dataset

read() → ebonite.core.objects.dataset_source.Dataset

Abstract method that must return produced Dataset instance

type = 'ebonite.repository.dataset.artifact.ArtifactDatasetSource'