core.contexts.processing.datalake.local_context

local_context

Classes:

Name	Description
LocalDatalakeProcessingContext	Context for local testing of processing jobs without real job execution on Picsellia.

Functions:

Name	Description
create_processing	Create a processing configuration in Picsellia.
get_processing	Get the ID of a processing by name.
launch_processing	Launch a processing job on a datalake.

LocalDatalakeProcessingContext(processing_parameters_cls, processing_parameters=None, api_token=None, host=None, organization_id=None, organization_name=None, job_type=None, input_datalake_id=None, output_datalake_id=None, model_version_id=None, target_id=None, inputs=None, offset=0, limit=100, use_id=True, working_dir=None)

Bases: PicselliaLocalProcessingContext, Generic[TParameters]

              flowchart TD
              picsellia_cv_engine.core.contexts.processing.datalake.local_context.LocalDatalakeProcessingContext[LocalDatalakeProcessingContext]
              picsellia_cv_engine.core.contexts.processing.common.local_picsellia_context.PicselliaLocalProcessingContext[PicselliaLocalProcessingContext]
              picsellia_cv_engine.core.contexts.common.picsellia_context.PicselliaContext[PicselliaContext]

                              picsellia_cv_engine.core.contexts.processing.common.local_picsellia_context.PicselliaLocalProcessingContext --> picsellia_cv_engine.core.contexts.processing.datalake.local_context.LocalDatalakeProcessingContext
                                picsellia_cv_engine.core.contexts.common.picsellia_context.PicselliaContext --> picsellia_cv_engine.core.contexts.processing.common.local_picsellia_context.PicselliaLocalProcessingContext
                



              click picsellia_cv_engine.core.contexts.processing.datalake.local_context.LocalDatalakeProcessingContext href "" "picsellia_cv_engine.core.contexts.processing.datalake.local_context.LocalDatalakeProcessingContext"
              click picsellia_cv_engine.core.contexts.processing.common.local_picsellia_context.PicselliaLocalProcessingContext href "" "picsellia_cv_engine.core.contexts.processing.common.local_picsellia_context.PicselliaLocalProcessingContext"
              click picsellia_cv_engine.core.contexts.common.picsellia_context.PicselliaContext href "" "picsellia_cv_engine.core.contexts.common.picsellia_context.PicselliaContext"
            

Context for local testing of processing jobs without real job execution on Picsellia.

Methods:

Name	Description
get_datalake
get_model_version
get_data_ids	List data IDs from a datalake with offset and limit.
to_dict	Convert context to a dictionary for logging or serialization.

Attributes:

Name	Type	Description
job_type
target
offset
limit
data_ids
api_token
host
organization_id
organization_name
client
working_dir	str
target_id
parameters	dict[str, Any]
inputs
use_id
processing_parameters	TParameters

job_type = job_type instance-attribute

target = self.client.get_datalake(id=(self.target_id)) instance-attribute

offset = offset instance-attribute

limit = limit instance-attribute

data_ids = None instance-attribute

api_token = api_token or os.getenv('api_token') instance-attribute

host = host or os.getenv('host', 'https://app.picsellia.com') instance-attribute

organization_id = organization_id or os.getenv('organization_id') instance-attribute

organization_name = organization_name or os.getenv('organization_name') instance-attribute

client = self._initialize_client() instance-attribute

working_dir property

target_id = target_id instance-attribute

parameters = parameters_dict if parameters_dict is not None else {} instance-attribute

inputs = inputs instance-attribute

use_id = use_id instance-attribute

processing_parameters = processing_parameters_cls(log_data=(self.parameters)) instance-attribute

get_datalake(datalake_id)

get_model_version()

get_data_ids(offset, limit)

List data IDs from a datalake with offset and limit.

to_dict()

Convert context to a dictionary for logging or serialization.

create_processing(client, name, type, default_cpu, default_gpu, default_parameters, docker_image, docker_tag, docker_flags=None)

Create a processing configuration in Picsellia.

Returns:

Name	Type	Description
str	str	ID of the created processing.

get_processing(client, name)

Get the ID of a processing by name.

Returns:

Name	Type	Description
str	str	ID of the found processing.

launch_processing(client, datalake, data_ids, model_version_id, processing_id, parameters, cpu, gpu, target_datalake_name=None)

Launch a processing job on a datalake.

Returns:

Name	Type	Description
Job	Job	The launched job object.