pulsar.client.staging package¶
Submodules¶
pulsar.client.staging.down module¶
Code run on the client side for unstaging complete Pulsar jobs.
pulsar.client.staging.up module¶
Module contents¶
- class pulsar.client.staging.ClientInput(path, input_type, object_store_ref=None)[source]¶
Bases:
object
- property action_source¶
- class pulsar.client.staging.ClientInputs(client_inputs)[source]¶
Bases:
object
Abstraction describing input datasets for a job.
- class pulsar.client.staging.ClientJobDescription(command_line, tool=None, config_files=None, input_files=None, client_inputs=None, client_outputs=None, working_directory=None, metadata_directory=None, dependencies_description=None, env=[], arbitrary_files=None, job_directory_files=None, tool_directory_required_files=None, rewrite_paths=True, touch_outputs=None, container=None, remote_pulsar_app_config=None, guest_ports=None)[source]¶
Bases:
object
A description of how client views job - command_line, inputs, etc..
Parameters
- command_linestr
The local command line to execute, this will be rewritten for the remote server.
- config_fileslist
List of Galaxy ‘configfile’s produced for this job. These will be rewritten and sent to remote server.
- input_fileslist
List of input files used by job. These will be transferred and references rewritten.
- client_outputsClientOutputs
Description of outputs produced by job (at least output files along with optional version string and working directory outputs.
- tool_dirstr
Directory containing tool to execute (if a wrapper is used, it will be transferred to remote server).
- working_directorystr
Local path created by Galaxy for running this job (job_wrapper.tool_working_directory).
- metadata_directorystr
Local path created by Galaxy for running this job (job_wrapper.working_directory).
- dependencies_descriptionlist
galaxy.tools.deps.dependencies.DependencyDescription object describing tool dependency context for remote depenency resolution.
- env: list
List of dict object describing environment variables to populate.
- version_filestr
Path to version file expected on the client server
- arbitrary_filesdict()
Additional non-input, non-tool, non-config, non-working directory files to transfer before staging job. This is most likely data indices but can be anything. For now these are copied into staging working directory but this will be reworked to find a better, more robust location.
- rewrite_pathsboolean
Indicates whether paths should be rewritten in job inputs (command_line and config files) while staging files).
- property input_files¶
- property output_files¶
- property tool_dependencies¶
- property version_file¶
- class pulsar.client.staging.ClientOutputs(working_directory=None, output_files=[], work_dir_outputs=None, version_file=None, dynamic_outputs=None, metadata_directory=None, job_directory=None, dynamic_file_sources=None)[source]¶
Bases:
object
Abstraction describing the output datasets EXPECTED by the Galaxy job runner client.
- class pulsar.client.staging.DynamicFileSourceType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]¶
Bases:
str
,Enum
- galaxy = 'galaxy'¶
- legacy_galaxy = 'legacy_galaxy'¶
- class pulsar.client.staging.PulsarOutputs(working_directory_contents, output_directory_contents, metadata_directory_contents, job_directory_contents, remote_separator='/', realized_dynamic_file_sources=None)[source]¶
Bases:
object
Abstraction describing the output files PRODUCED by the remote Pulsar server.