core.contexts.processing.datalake.picsellia_context¶
picsellia_context
¶
Classes:
| Name | Description |
|---|---|
PicselliaDatalakeProcessingContext |
Context for running Picsellia datalake processing jobs. |
PicselliaDatalakeProcessingContext(processing_parameters_cls, api_token=None, host=None, organization_id=None, organization_name=None, job_id=None, use_id=True, working_dir=None)
¶
Bases: PicselliaProcessingContext, Generic[TParameters]
flowchart TD
picsellia_cv_engine.core.contexts.processing.datalake.picsellia_context.PicselliaDatalakeProcessingContext[PicselliaDatalakeProcessingContext]
picsellia_cv_engine.core.contexts.processing.common.picsellia_context.PicselliaProcessingContext[PicselliaProcessingContext]
picsellia_cv_engine.core.contexts.common.picsellia_context.PicselliaContext[PicselliaContext]
picsellia_cv_engine.core.contexts.processing.common.picsellia_context.PicselliaProcessingContext --> picsellia_cv_engine.core.contexts.processing.datalake.picsellia_context.PicselliaDatalakeProcessingContext
picsellia_cv_engine.core.contexts.common.picsellia_context.PicselliaContext --> picsellia_cv_engine.core.contexts.processing.common.picsellia_context.PicselliaProcessingContext
click picsellia_cv_engine.core.contexts.processing.datalake.picsellia_context.PicselliaDatalakeProcessingContext href "" "picsellia_cv_engine.core.contexts.processing.datalake.picsellia_context.PicselliaDatalakeProcessingContext"
click picsellia_cv_engine.core.contexts.processing.common.picsellia_context.PicselliaProcessingContext href "" "picsellia_cv_engine.core.contexts.processing.common.picsellia_context.PicselliaProcessingContext"
click picsellia_cv_engine.core.contexts.common.picsellia_context.PicselliaContext href "" "picsellia_cv_engine.core.contexts.common.picsellia_context.PicselliaContext"
Context for running Picsellia datalake processing jobs.
Methods:
| Name | Description |
|---|---|
get_data_ids |
Retrieve data IDs from the job payload. |
to_dict |
|
get_datalake |
|
get_model_version |
|
Attributes:
| Name | Type | Description |
|---|---|---|
data_ids |
|
|
target |
|
|
api_token |
|
|
host |
|
|
organization_id |
|
|
organization_name |
|
|
client |
|
|
working_dir |
str
|
|
job_id |
|
|
job |
|
|
job_info |
dict[str, Any]
|
|
job_context |
dict[str, Any]
|
|
target_id |
|
|
parameters |
dict[str, Any]
|
|
inputs |
dict[str, Any]
|
|
payload_presigned_url |
str | None
|
|
use_id |
|
|
processing_parameters |
TParameters
|
|
processing_type |
|
data_ids = self.get_data_ids()
instance-attribute
¶
target = self.client.get_datalake(id=(self.target_id))
instance-attribute
¶
api_token = api_token or os.getenv('api_token')
instance-attribute
¶
host = host or os.getenv('host', 'https://app.picsellia.com')
instance-attribute
¶
organization_id = organization_id or os.getenv('organization_id')
instance-attribute
¶
organization_name = organization_name or os.getenv('organization_name')
instance-attribute
¶
client = self._initialize_client()
instance-attribute
¶
working_dir
property
¶
job_id = job_id or os.environ.get('job_id')
instance-attribute
¶
job = self._initialize_job()
instance-attribute
¶
job_info = self.job.sync()
instance-attribute
¶
job_context = self._initialize_job_context()
instance-attribute
¶
target_id = self.job_context.get('target_id')
instance-attribute
¶
parameters = self.job_context.get('parameters', {}) or {}
instance-attribute
¶
inputs = self.job_context.get('inputs', {}) or {}
instance-attribute
¶
payload_presigned_url = self.job_context.get('payload_presigned_url')
instance-attribute
¶
use_id = use_id
instance-attribute
¶
processing_parameters = processing_parameters_cls(log_data=(self.parameters))
instance-attribute
¶
processing_type = self.client.get_processing(name=processing_name).type
instance-attribute
¶
get_data_ids()
¶
Retrieve data IDs from the job payload.