ebonite.core.objects package

class ebonite.core.objects.Project(name: str, id: int = None, author: str = None, creation_date: datetime.datetime = None)[source]

Bases: ebonite.core.objects.core.EboniteObject

Project is a collection of tasks

Parameters:
  • id – project id
  • name – project name
  • author – user that created that project
  • creation_date – date when this project was created
delete(cascade: bool = False)[source]

Deletes project and(if required) all tasks associated with it from metadata repository

Parameters:cascade – whether should project be deleted with all associated tasks
Returns:Nothing
add_task(task: ebonite.core.objects.core.Task)[source]

Add task to project and save it to meta repo

Parameters:task – task to add
add_tasks(tasks: List[Task])[source]

Add multiple tasks and save them to meta repo

Parameters:tasks – tasks to add
delete_task(task: ebonite.core.objects.core.Task, cascade: bool = False)[source]

Remove task from this project and delete it from meta repo

Parameters:
  • cascade – whether task should be deleted with all nested objects
  • task – task to delete
save()[source]

Saves object state to metadata repository

has_children()[source]

Checks if object has existing relationship

class ebonite.core.objects.Requirements(requirements: List[ebonite.core.objects.requirements.Requirement] = None)[source]

Bases: ebonite.core.objects.base.EboniteParams

A collection of requirements

Parameters:requirements – list of Requirement instances
installable

List of installable requirements

custom

List of custom requirements

of_type(type_: Type[T]) → List[T][source]
Parameters:type – type of requirements
Returns:List of requirements of type type_
modules

List of module names

add(requirement: ebonite.core.objects.requirements.Requirement)[source]

Adds requirement to this collection

Parameters:requirement – Requirement instance to add
to_pip() → List[str][source]
Returns:list of pip installable packages
class ebonite.core.objects.Requirement[source]

Bases: ebonite.core.objects.requirements.Requirement, pyjackson.decorators.SubtypeRegisterMixin

Base class for python requirement

type = 'pyjackson.decorators.Requirement'
class ebonite.core.objects.ArtifactCollection[source]

Bases: ebonite.core.objects.artifacts.ArtifactCollection, pyjackson.decorators.SubtypeRegisterMixin

Base class for artifact collection. Artifact collection is a number of named artifacts, represented by Blob’s

Must be pyjackson-able

type = 'pyjackson.decorators.ArtifactCollection'
class ebonite.core.objects.ModelWrapper(io: ebonite.core.objects.wrapper.ModelIO)[source]

Bases: ebonite.core.objects.wrapper.ModelWrapper, pyjackson.decorators.SubtypeRegisterMixin

Base class for model wrapper. Wrapper is an object that can save, load and inference a model

Must be pyjackson-serializable

type = 'pyjackson.decorators.ModelWrapper'
class ebonite.core.objects.Task(name: str, id: int = None, project_id: int = None, author: str = None, creation_date: datetime.datetime = None, datasets: Dict[str, ebonite.core.objects.dataset_source.DatasetSource] = None, metrics: Dict[str, ebonite.core.objects.metric.Metric] = None, evaluation_sets: Dict[str, ebonite.core.objects.core.EvaluationSet] = None)[source]

Bases: ebonite.core.objects.core.EboniteObject, ebonite.core.objects.core.WithDatasetRepository

Task is a collection of models

Parameters:
  • id – task id
  • name – task name
  • project_id – parent project id for this task
  • author – user that created that task
  • creation_date – date when this task was created
project
delete(cascade: bool = False)[source]

Deletes task from metadata

Parameters:cascade – whether should task be deleted with all associated objects
Returns:Nothing
add_model(model: ebonite.core.objects.core.Model)[source]

Add model to task and save it to meta repo

Parameters:model – model to add
add_models(models: List[Model])[source]

Add multiple models and save them to meta repo

Parameters:models – models to add
delete_model(model: ebonite.core.objects.core.Model, force=False)[source]

Remove model from this task and delete it from meta repo

Parameters:
  • model – model to delete
  • force – whether model artifacts’ deletion errors should be ignored, default is false
create_and_push_model(model_object, input_data, model_name: str = None, **kwargs) → ebonite.core.objects.core.Model[source]

Create Model instance from model object and push it to repository

Parameters:
  • model_object – model object to build Model from
  • input_data – input data sample to determine structure of inputs and outputs for given model
  • model_name – name for model
  • kwargs – other create() arguments
Returns:

created Model
push_model(model: ebonite.core.objects.core.Model) → ebonite.core.objects.core.Model[source]

Push Model instance to task repository

Parameters:model – Model to push
Returns:same pushed Model
add_pipeline(pipeline: ebonite.core.objects.core.Pipeline)[source]

Add model to task and save it to meta repo

Parameters:pipeline – pipeline to add
add_pipelines(pipelines: List[Pipeline])[source]

Add multiple models and save them to meta repo

Parameters:pipelines – pipelines to add
delete_pipeline(pipeline: ebonite.core.objects.core.Pipeline)[source]

Remove model from this task and delete it from meta repo

Parameters:pipeline – pipeline to delete
add_image(image: ebonite.core.objects.core.Image)[source]

Add image for model and save it to meta repo

Parameters:image – image to add
add_images(images: List[Image])[source]

Add multiple images for model and save them to meta repo

Parameters:images – images to add
delete_image(image: ebonite.core.objects.core.Image, meta_only: bool = False, cascade: bool = False)[source]

Remove image from this model and delete it from meta repo

Parameters:
  • image – image to delete
  • meta_only – should image be deleted only from metadata
  • cascade – whether image should be deleted with all instances
save()[source]

Saves task to meta repository and pushes unsaved datasets to dataset repository

has_children()[source]

Checks if object has existing relationship

add_evaluation(name: str, data: Union[str, ebonite.core.objects.dataset_source.AbstractDataset, ebonite.core.objects.dataset_source.DatasetSource, Any], target: Union[str, ebonite.core.objects.dataset_source.AbstractDataset, ebonite.core.objects.dataset_source.DatasetSource, Any], metrics: Union[str, ebonite.core.objects.metric.Metric, Any, List[Union[str, ebonite.core.objects.metric.Metric, Any]]])[source]

Adds new evaluation set to this task

Parameters:
  • name – name of the evaluation set
  • data – input dataset for evaluation
  • target – ground truth for input data
  • metrics – one or more metrics to measure
delete_evaluation(name: str, save: bool = True)[source]

Deletes evaluation set from task

Parameters:
  • name – name of the evaluation to delete
  • save – also update task metadata in repo
add_dataset(name, dataset: Union[ebonite.core.objects.dataset_source.DatasetSource, ebonite.core.objects.dataset_source.AbstractDataset, Any])[source]

Adds new dataset to this task

Parameters:
  • name – name of the dataset
  • dataset – Dataset, DatasetSource or raw dataset object
push_datasets()[source]

Pushes all unsaved datasets to dataset repository

delete_dataset(name: str, force: bool = False, save: bool = True)[source]

Deletes dataset from task with artifacts

Parameters:
  • name – name of the dataset to delete
  • force – wheter to check evalsets that use this dataset and remove them or raise error
  • save – also update task metadata in repo
add_metric(name, metric: Union[ebonite.core.objects.metric.Metric, Any])[source]

Adds metric to this task

Parameters:
  • name – name of the metric
  • metric – Metric or raw metric object
delete_metric(name: str, force: bool = False, save: bool = True)[source]

Deletes metric from task

Parameters:
  • name – name of the metric to delete
  • force – wheter to check evalsets that use this metric and remove them or raise error
  • save – also update task metadata in repo
evaluate_all(force=False, save_result=True) → Dict[str, ebonite.core.objects.core.EvaluationResult][source]

Evaluates all viable pairs of evalsets and models/pipelines

Parameters:
  • force – force reevaluate already evaluated
  • save_result – save evaluation results to meta
class ebonite.core.objects.Image(name: Optional[str], source: ebonite.core.objects.core.Buildable, id: int = None, params: ebonite.core.objects.core.Image.Params = None, author: str = None, creation_date: datetime.datetime = None, task_id: int = None, environment_id: int = None)[source]

Bases: ebonite.core.objects.core._WithBuilder

Class that represents metadata for image built from Buildable Actual type of image depends on .params field type

Parameters:
  • name – name of the image
  • id – id of the image
  • author – author of the image
  • source – Buildable instance this image was built from
  • params – Image.Params instance
  • task_id – task.id this image belongs to
  • environment_id – environment.id this image belongs to
Parma creation_date:
 

creation date of the image
class Params[source]

Bases: ebonite.core.objects.core.Params, pyjackson.decorators.SubtypeRegisterMixin

Abstract class that represents different types of images

type = 'pyjackson.decorators.Params'
task
delete(meta_only: bool = False, cascade: bool = False)[source]

Deletes existing image from metadata repository and image provider

Parameters:
  • meta_only – should image be deleted only from metadata
  • cascade – whether to delete nested RuntimeInstances
bind_meta_repo(repo: ebonite.repository.metadata.base.MetadataRepository)[source]
is_built() → bool[source]

Checks if image was built and wasn’t removed

build(**kwargs)[source]

Build this image

Parameters:kwargs – additional params for builder.build_image (depends on builder implementation)
remove(**kwargs)[source]

remove this image (from environment, not from ebonite metadata)

save()[source]

Saves object state to metadata repository

has_children()[source]

Checks if object has existing relationship

class ebonite.core.objects.Model(name: str, wrapper_meta: Optional[dict] = None, artifact: Optional[ebonite.core.objects.artifacts.ArtifactCollection] = None, requirements: ebonite.core.objects.requirements.Requirements = None, params: Dict[str, Any] = None, description: str = None, id: int = None, task_id: int = None, author: str = None, creation_date: datetime.datetime = None, evaluations: Dict[str, Dict[str, ebonite.core.objects.core.EvaluationResultCollection]] = None)[source]

Bases: ebonite.core.objects.core._InTask

Model contains metadata for machine learning model

Parameters:
  • name – model name
  • wrapper_meta – ModelWrapper instance for this model
  • artifact – ArtifactCollection instance with model artifacts
  • requirements – Requirements instance with model requirements
  • params – dict with arbitrary parameters. Must be json-serializable
  • description – text description of this model
  • id – model id
  • task_id – parent task_id
  • author – user that created that model
  • creation_date – date when this model was created
PYTHON_VERSION = 'python_version'
load()[source]

Load model artifacts into wrapper

ensure_loaded()[source]

Ensure that wrapper has loaded model object

wrapper
with_wrapper(wrapper: ebonite.core.objects.wrapper.ModelWrapper)[source]

Bind wrapper instance to this Model

Parameters:wrapper – ModelWrapper instance
Returns:self
wrapper_meta

pyjackson representation of ModelWrapper for this model: e.g., this provides possibility to move a model between repositories without its dependencies being installed

Type:return
with_wrapper_meta(wrapper_meta: dict)[source]

Bind wrapper_meta dict to this Model

Parameters:wrapper_meta – dict with serialized ModelWrapper instance
Returns:self
artifact

persisted artifacts if any

Type:return
artifact_any

artifacts in any state (persisted or not)

Type:return
artifact_req_persisted

Similar to artifact but checks that no unpersisted artifacts are left

Returns:persisted artifacts if any
attach_artifact(artifact: ebonite.core.objects.artifacts.ArtifactCollection)[source]
Parameters:artifact – artifacts to attach to model in an unpersisted state
persist_artifacts(persister: Callable[[ArtifactCollection], ArtifactCollection])[source]

Model artifacts persisting workflow

Parameters:persister – external object which stores model artifacts
without_artifacts() → ebonite.core.objects.core.Model[source]
Returns:copy of the model with no artifacts attached
classmethod create(model_object, input_data, model_name: str = None, params: Dict[str, Any] = None, description: str = None, additional_artifacts: ebonite.core.objects.artifacts.ArtifactCollection = None, additional_requirements: Union[ebonite.core.objects.requirements.Requirements, ebonite.core.objects.requirements.Requirement, List[ebonite.core.objects.requirements.Requirement], str, List[str]] = None, custom_wrapper: ebonite.core.objects.wrapper.ModelWrapper = None, custom_artifact: ebonite.core.objects.artifacts.ArtifactCollection = None, custom_requirements: Union[ebonite.core.objects.requirements.Requirements, ebonite.core.objects.requirements.Requirement, List[ebonite.core.objects.requirements.Requirement], str, List[str]] = None) → ebonite.core.objects.core.Model[source]

Creates Model instance from arbitrary model objects and sample of input data

Parameters:
  • model_object – The model object to analyze.
  • input_data – Input data sample to determine structure of inputs and outputs for given model object.
  • model_name – The model name.
  • params – dict with arbitrary parameters. Must be json-serializable
  • description – text description of this model
  • additional_artifacts – Additional artifact.
  • additional_requirements – Additional requirements.
  • custom_wrapper – Custom model wrapper.
  • custom_artifact – Custom artifact collection to replace all other.
  • custom_requirements – Custom requirements to replace all other.
Returns:

Model
delete(force: bool = False)[source]

Deletes model from metadata and artifact repositories

Parameters:force – whether model artifacts’ deletion errors should be ignored, default is false
Returns:Nothing
push(task: ebonite.core.objects.core.Task = None) → ebonite.core.objects.core.Model[source]

Pushes Model instance into metadata and artifact repositories

Parameters:task – Task instance to save model to. Optional if model already has

task :return: same saved Model instance

as_pipeline(method_name=None) → ebonite.core.objects.core.Pipeline[source]

Create Pipeline that consists of this model’s single method

Parameters:method_name – name of the method. can be omitted if model has only one method
save()[source]

Saves model to metadata repo and pushes unpersisted artifacts

has_children()[source]

Checks if object has existing relationship

evaluate_set(evalset: Union[str, ebonite.core.objects.core.EvaluationSet], evaluation_name: str = None, method_name: str = None, timestamp=None, save=True, force=False, raise_on_error=False) → Optional[ebonite.core.objects.core.EvaluationResult][source]

Evaluates this model

Parameters:
  • evalset – evalset or it’s name
  • evaluation_name – name of this evaluation
  • method_name – name of wrapper method. If none, all methods with consistent datatypes will be evaluated
  • timestamp – time of the evaluation (defaults to now)
  • save – save results to meta
  • force – force reevalute
  • raise_on_error – raise error if datatypes are incorrect or just return
evaluate(input: ebonite.core.objects.dataset_source.DatasetSource, output: ebonite.core.objects.dataset_source.DatasetSource, metrics: Dict[str, ebonite.core.objects.metric.Metric], evaluation_name: str = None, method_name: str = None, timestamp=None, save=True, force=False, raise_on_error=False) → Union[ebonite.core.objects.core.EvaluationResult, Dict[str, ebonite.core.objects.core.EvaluationResult], None][source]

Evaluates this model

Parameters:
  • input – input data
  • output – target
  • metrics – dict of metrics to evaluate
  • evaluation_name – name of this evaluation
  • method_name – name of wrapper method. If none, all methods with consistent datatypes will be evaluated
  • timestamp – time of the evaluation (defaults to now)
  • save – save results to meta
  • force – force reevalute
  • raise_on_error – raise error if datatypes are incorrect or just return
class ebonite.core.objects.DatasetType[source]

Bases: ebonite.core.objects.dataset_type.DatasetType, pyjackson.decorators.SubtypeRegisterMixin

Base class for dataset type metadata. Children of this class must be both pyjackson-serializable and be a pyjackson serializer for it’s dataset type

type = 'pyjackson.generics.DatasetType'
class ebonite.core.objects.RuntimeEnvironment(name: str, id: int = None, params: ebonite.core.objects.core.RuntimeEnvironment.Params = None, author: str = None, creation_date: datetime.datetime = None)[source]

Bases: ebonite.core.objects.core.EboniteObject

Represents and environment where you can build and deploy your services Actual type of environment depends on .params field type

Parameters:
  • name – name of the environment
  • id – id of the environment
  • author – author of the enviroment
  • creation_date – creation date of the enviroment
  • params – RuntimeEnvironment.Params instance
class Params[source]

Bases: ebonite.core.objects.core.Params, pyjackson.decorators.SubtypeRegisterMixin

Abstract class that represents different types of environments

type = 'pyjackson.decorators.Params'
delete(meta_only: bool = False, cascade: bool = False)[source]

Deletes environment from metadata repository and(if required) stops associated instances

Parameters:
  • meta_only – wheter to only delete metadata
  • cascade – Whether should environment be deleted with all associated instances
Returns:

Nothing
save()[source]

Saves this env to metadata repository

has_children()[source]

Checks if object has existing relationship

class ebonite.core.objects.RuntimeInstance(name: Optional[str], id: int = None, image_id: int = None, environment_id: int = None, params: ebonite.core.objects.core.RuntimeInstance.Params = None, author: str = None, creation_date: datetime.datetime = None)[source]

Bases: ebonite.core.objects.core._WithRunner

Class that represents metadata for instance running in environment Actual type of instance depends on .params field type

Parameters:
  • name – name of the instance
  • id – id of the instance
  • author – author of the instance
  • image_id – id of base image for htis instance
  • params – RuntimeInstance.Params instance
Parma creation_date:
 

creation date of the instance
class Params[source]

Bases: ebonite.core.objects.core.Params, pyjackson.decorators.SubtypeRegisterMixin

Abstract class that represents different types of images

type = 'pyjackson.decorators.Params'
image
delete(meta_only: bool = False)[source]

Stops instance of model service and deletes it from repository

Parameters:meta_only – only remove from metadata, do not stop instance
Returns:nothing
run(**runner_kwargs) → ebonite.core.objects.core.RuntimeInstance[source]

Run this instance

Parameters:runner_kwargs – additional params for runner.run (depends on runner implementation)
logs(**kwargs)[source]

Get logs of this instance

Parameters:kwargs – parameters for runner logs method
Yields:str logs from running instance
is_running(**kwargs) → bool[source]

Checks whether instance is running

Parameters:kwargs – params for runner is_running method
Returns:“is running” flag
stop(**kwargs)[source]

Stops the instance

Parameters:kwargs – params for runner stop method
exists(**kwargs) → bool[source]

Checks if instance exists (it may be stopped)

Parameters:kwargs – params for runner instance_exists method
remove(**kwargs)[source]

Removes the instance from environment (not from metadata)

Parameters:kwargs – params for runner remove_instance method
save()[source]

Saves object state to metadata repository

has_children()[source]

Checks if object has existing relationship

class ebonite.core.objects.ModelIO[source]

Bases: ebonite.core.objects.wrapper.ModelIO, pyjackson.decorators.SubtypeRegisterMixin

Helps model wrapper with IO

Must be pyjackson-serializable

type = 'pyjackson.decorators.ModelIO'
class ebonite.core.objects.Pipeline(name: str, steps: List[ebonite.core.objects.core.PipelineStep], input_data: ebonite.core.objects.dataset_type.DatasetType, output_data: ebonite.core.objects.dataset_type.DatasetType, id: int = None, author: str = None, creation_date: datetime.datetime = None, task_id: int = None, evaluations: Dict[str, ebonite.core.objects.core.EvaluationResultCollection] = None)[source]

Bases: ebonite.core.objects.core._InTask

Pipeline is a class to represent a sequence of different Model’s methods. They can be used to reuse different models (for example, pre-processing functions) in different pipelines. Pipelines must have exact same in and out data types as tasks they are in

Parameters:
  • name – name of the pipeline
  • steps – sequence of :class:`.PipelineStep`s the pipeline consists of
  • input_data – datatype of input dataset
  • output_data – datatype of output datset
  • id – id of the pipeline
  • author – author of the pipeline
  • creation_date – date of creation
  • task_id – task.id of parent task
delete()[source]

Deletes pipeline from metadata

load()[source]
run(data)[source]

Applies sequence of pipeline steps to data

Parameters:data – data to apply pipeline to. must have type Pipeline.input_data
Returns:processed data of type Pipeline.output_data
append(model: Union[ebonite.core.objects.core.Model, ebonite.core.objects.core._WrapperMethodAccessor], method_name: str = None)[source]

Appends another Model to the sequence of this pipeline steps

Parameters:
  • model – either Model instance, or model method (as in model.method where method is method name)
  • method_name – if Model was provided in model, this should be method name.

can be omitted if model have only one method

save()[source]

Saves this pipeline to metadata repository

has_children()[source]

Checks if object has existing relationship

evaluate_set(evalset: Union[str, ebonite.core.objects.core.EvaluationSet], evaluation_name: str = None, timestamp=None, save=True, force=False, raise_on_error=False) → Optional[ebonite.core.objects.core.EvaluationResult][source]

Evaluates this pipeline

Parameters:
  • evalset – evalset or it’s name
  • evaluation_name – name of this evaluation
  • timestamp – time of the evaluation (defaults to now)
  • save – save results to meta
  • force – force reevalute
  • raise_on_error – raise error if datatypes are incorrect or just return
evaluate(input: ebonite.core.objects.dataset_source.DatasetSource, output: ebonite.core.objects.dataset_source.DatasetSource, metrics: Dict[str, ebonite.core.objects.metric.Metric], evaluation_name: str = None, timestamp=None, save=True, force=False, raise_on_error=False) → Optional[ebonite.core.objects.core.EvaluationResult][source]

Evaluates this pipeline

Parameters:
  • input – input data
  • output – target
  • metrics – dict of metrics to evaluate
  • evaluation_name – name of this evaluation
  • timestamp – time of the evaluation (defaults to now)
  • save – save results to meta
  • force – force reevalute
  • raise_on_error – raise error if datatypes are incorrect or just return
class ebonite.core.objects.PipelineStep(model_name: str, method_name: str)[source]

Bases: ebonite.core.objects.base.EboniteParams

A class to represent one step of a Pipeline - a Model with one of its’ methods name

Parameters:
  • model_name – name of the Model (in the same Task as Pipeline object)
  • method_name – name of the method in Model’s wrapper to use