jobcontrol.interfaces

Interfaces for NEW jobcontrol objects.

Data model:

Build   id SERIAL
-----   job_id TEXT
        start_time TIMESTAMP
        end_time TIMESTAMP
        started BOOLEAN
        finished BOOLEAN
        success BOOLEAN
        skipped BOOLEAN
        config BINARY (pickled)
            Copy of the job configuration whan the build was started,
            along with build-specific configuration (such as pinning)
        retval BINARY (pickled)
        exception BINARY (pickled)
            Pickled exception object (or None)
        exception_tb BINARY (pickled)
            Pickled TracebackInfo object

Build progress
--------------
        build_id INTEGER (references Build.id)
        group_name VARCHAR(128)
            Name of the "progress group" (separated by '::')
        current INTEGER
            Current progress value
        total INTEGER
            Total progress value
        status_line TEXT
            An optional line of text describing current state
        UNIQUE constraint on (build_id, group_name)

Log     id SERIAL
---     build_id INTEGER (references Build.id)
        created TIMESTAMP
        level INTEGER
        record BINARY (pickled)
            Pickled "custom" LogRecord object
        exception_tb BINARY
            Pickled TracebackInfo object

Job configuration:

The job configuration is stored as a YAML-serialized dict.

Recognised keys are:

  • function in module:function format, specify the function to be called
  • args a list of arguments to be passed to the function
  • kwargs a dict of keyword arguments to be passed to the function
  • title a descriptive title, to be shown on the interfaces
  • notes notes, to be shown in interfaces (in restructured text)
  • dependencies list of dependency job names

Additionally, args/kwargs may contain references to return value of dependency builds, by using the !retval <name> syntax.

Exception traceback serialization

To be used both in build records and associated with log messages containing an exception.

We want to include the following information:

  • Details about the call stack, as in normal tracebacks: filename, line number, function name, line of code (plus some context)
  • Local variables: we are not guaranteed we can safely pickle / unpickle arbitrary values; moreover this might result in huge fields, etc. So our better chance is to just store a dictionary mapping names to repr()s of the values (trimmed to a – large – maximum length, just to be on the safe side).
class jobcontrol.interfaces.StorageBase[source]
classmethod from_url(url)[source]
install()[source]
uninstall()[source]
get_job_builds(job_id, started=None, finished=None, success=None, skipped=None, order='asc', limit=100)[source]

Iterate over all the builds for a job, sorted by date, according to the order specified by order.

Parameters:
  • job_id – The job id
  • started – If set to a boolean, filter on the “started” field
  • finished – If set to a boolean, filter on the “finished” field
  • success – If set to a boolean, filter on the “success” field
  • skipped – If set to a boolean, filter on the “skipped” field
  • order – ‘asc’ (default) or ‘desc’
  • limit – only return the first limit builds
Yield:

Dictionaries representing build information

create_build(job_id, config=None)[source]

Create a build.

Parameters:
  • job_id – The job for which a build should be started
  • job_config – The job configuration (function, args, kwargs, ..) to be copied inside the object (we will use this from now on).
  • build_config

    Build configuration, containing things like dependency build pinning, etc.

    • dependency_builds: dict mapping job ids to build ids, or None to indicate “create a new build” for this job.
Returns:

the build id

get_build(build_id)[source]

Get information about a build.

Returns:the build information, as a dict
delete_build(build_id)[source]

Delete a build, by id.

start_build(build_id)[source]

Register a build execution start.

finish_build(build_id, success=None, skipped=None, retval=None, exception=None, exception_tb=None)[source]

Register a build execution end.

finish_build_with_exception(build_id)[source]
update_build_progress(build_id, current, total)[source]
report_build_progress(build_id, current, total, group_name='', status_line='')[source]

Report progress for a build.

Parameters:
  • build_id – The build id for which to report progress
  • current – The current number of “steps” done
  • total – The total amount of “steps”
  • group_name – Optionally, a name used to nest multiple progress “levels”. A tuple (or string separated by ‘::’ can be used to specify multiple “nesting” levels)
  • status_line – Optionally, a line of text indicating the current build status.
get_build_progress_info(build_id)[source]

Return progress information for a build.

Returns:a list of tuples: (name, current, total, status_line)
get_latest_successful_build(job_id)[source]

Helper method to retrieve the latest successful build for a given job. Calls get_job_builds() in the background.

Returns:information about the build, as a dict
log_message(build_id, record)[source]

Store a log record associated with a build.

prune_log_messages(job_id=None, build_id=None, max_age=None, level=None)[source]

Delete (old) log messages.

Parameters:
  • job_id – If specified, only delete messages for this job
  • build_id – If specified, only delete messages for this build
  • max_age – If specified, only delete log messages with an age greater than this one (in seconds)
  • level – If specified, only delete log messages with a level equal or minor to this one
iter_log_messages(build_id=None, max_date=None, min_date=None, min_level=None)[source]

Iterate over log messages, applying some filters.

Parameters:
  • build_id – If specified, only return messages for this build
  • max_date – If specified, only return messages newer than this date
  • min_date – If specified, only return messages older than this date
  • min_level – If specified, only return messages with a level at least equal to this one
pack(obj, safe=False)[source]
pack_exception(exception)[source]
unpack(obj, safe=False)[source]

Previous topic

jobcontrol.globals

Next topic

jobcontrol.utils

This Page