jobcontrol.core

Objects responsible for JobControl core functionality.

Note

Important objects from this module should be imported in main __init___, in order to “abstract away” the namespace and have them in a more nicely accessible place.

class jobcontrol.core.JobControl(storage, config)[source]

The main JobControl class.

Parameters:
classmethod from_config_file(config_file)[source]

Initialize JobControl by loading configuration from a file. Will also initialize storage taking values from the configuration.

Parameters:config_file – Path to configuration file, or an open file descriptor (or file-like object).
Returns:a JobControl instance
classmethod from_config(config)[source]

Initialize JobControl from some configuration.

Parameters:config – Either a jobcontrol.config.JobControlConfig instance, or a dict to be passed as argument to that class constructor.
Returns:a JobControl instance
get_job(job_id)[source]

Get a job, by id.

Parameters:job_id – The job id
Returns:a JobInfo class instance associated with the requested job.
Raises:jobcontrol.exceptions.NotFound if a job with that id was not found in the configuration.
iter_jobs()[source]

Generator yielding all the jobs, one by one.

Yields:for each job, a JobInfo class instance associated with the job.
get_build(build_id)[source]

Get a build, by id.

Parameters:build_id – The build id
Returns:a BuildInfo instance associated with the build.
Raises:jobcontrol.exceptions.NotFound if a build with that id was not found in the configuration.
create_build(job_id)[source]

Create a build, from a job configuration.

Note

Currently, we require that all the dependencies have already been built; in the future, it will be possible to build them automatically.

Note

Also, current implementation doesn’t allow for customizations to either the job configuration nor the build one (pinning, dep/revdep building, ...).

Parameters:

job_id – Id of the job for which to start a build

Returns:

a BuildInfo instance associated with the newly created build.

Raises:
build_job(job_id)[source]

Create and run a new build for the specified job.

This is simply a shortcut that runs create_build() then run_build(). (Mostly for compatibility reasons).

Returns:a BuildInfo instance associated with the newly created build.
run_build(build_id)[source]

Actually run a build.

  • take the build configuration
  • make sure all the dependencies are built
  • take return values from the dependencies -> pass as arguments
  • run the build
  • build the reverse dependencies as well, if required to do so
Parameters:build_id – either a BuildInfo instance, or a build id
prune_logs(policy=None)[source]
report_progress(group_name, current, total, status_line='')[source]

Report progress for the currently running build.

Parameters:
  • group_name – The report “group name”: either a tuple representing the “path”, or None for the top-level.
  • current – Current progress
  • total – Total progress
  • status_line – An optional line of text, describing the currently running operation.
get_celery_app()[source]

Return the Celery application, configured with values from the current configuration.

Note

this is a bit hackish, as we are just updating configuration values in the global object with ones from the jobcontrol configuration, not replacing all the configuration at once.

class jobcontrol.core.JobExecutionContext(app, job_id, build_id)[source]

Class to hold “global” context during job execution.

This class can also act as a context manager for temporary context:

with JobExecutionContext(app, job_id, build_id):
    pass # do stuff in an execution context
Parameters:
  • app – The JobControl instance running jobs
  • job_id – Id of the currently running job
  • build_id – Id of the currently running build
push()[source]

Push this context in the global stack

pop()[source]

Pop this context from the global stack

current_app[source]

Returns the currently running app

current_job[source]

Returns a JobInfo instance associated with the currently running job.

current_build[source]

Returns a BuildInfo instance associated with the currently running build.

class jobcontrol.core.JobControlLogHandler[source]

Logging handler sending messages to the appropriate JobControl instance that will dispatch them to storage.

flush()[source]

No-op, as we don’t need to flush anything

emit(record)[source]

“Emit” the log record (if there is an execution context, store the log record appropriately; otherwise, just ignore it).

class jobcontrol.core.JobInfo(app, job_id, config)[source]

High-level interface to jobs.

Provides high-level methods for, eg, creating a build out of this job or iterating builds.

Configuration is stored in the config attribute (a BuildConfig instance).

id[source]
config[source]
get_deps()[source]

Iterate over jobs this job depends upon.

Yields:JobInfo instances
get_status()[source]

Return a label describing the current status of the job.

Returns:
  • 'not_built' the job has no builds
  • 'running' the job has running builds
  • 'success' the job has at least a successful build
  • 'failed' the job only has failed builds
  • 'outdated' the job has at least a successful build, but older than one dependency build
get_revdeps()[source]

Iterate over jobs depending on this one

Yields:JobInfo instances
iter_builds(*a, **kw)[source]

Iterate over builds for this job.

Accepts the same arguments as jobcontrol.interfaces.StorageBase.get_job_builds()

Yields:BuildInfo instances
get_builds(*a, **kw)[source]

DEPRECATED alias for iter_builds()

run()[source]

Trigger run for this job (will automatically create a build, etc.)

create_build()[source]
get_latest_successful_build()[source]

Get latest successful build for this job, if any. Otherwise, returns None.

get_docs()[source]

Get documentation for this job.

get_conf_as_yaml()[source]

Return the job configuration as serialized YAML, mostly for displaying on user interfaces.

has_builds()[source]

Check whether this job has any build.

has_successful_builds()[source]

Check whether this job has any successful build.

has_running_builds()[source]

Check whether this job has any running build.

is_outdated()[source]

Check whether any dependency has builds more recent than the newest build for this job.

can_be_built()[source]

Checks whether a job can be built, i.e.: whether all the dependencies have at least one successful build.

class jobcontrol.core.BuildInfo(app, build_id, info=None)[source]

High-level interface to builds.

Parameters:
  • app – The JobControl instance this build was retrieved from
  • build_id – The build id
  • info – Optionally, this can be used to pre-populate the build information (useful, eg. if we are retrieving a bunch of builds from the database at once).
app
build_id
id[source]

The build id

job_id[source]

The job id

info[source]

Property used to lazily access the build attributes.

Returns a dict with the following keys:

  • 'id'
  • 'job_id'
  • 'start_time'
  • 'end_time'
  • 'started'
  • 'finished'
  • 'success'
  • 'skipped'
  • 'config'
  • 'retval'
  • 'exception'
  • 'exception_tb'
config[source]
descriptive_status[source]

Return a label describing the current status of the build.

Returns:
  • 'CREATED' if the build was not started yet
  • 'RUNNING' if the build was started but did not finish
  • 'SUCCESSFUL' if the build run with success
  • 'SKIPPED' if the build was skipped
  • 'FAILED' if the build execution failed
refresh()[source]

Refresh the build status information from database

get_progress_info()[source]

Get information about the build progress

get_job()[source]

Get a JobInfo associated with this build’s job

delete()[source]

Delete all information related to this build from database

run()[source]

Calls run_build() on the main app for this build

iter_log_messages(**kw)[source]

Iterate over log messages for this build.

Keywords are passed directly to the underlying iter_log_messages() method of the storage.

Previous topic

jobcontrol.config

Next topic

jobcontrol.exceptions

This Page