API reference

futureproof consists in two main objects: executors and task managers.

If you’re used to concurrent.futures you’ll note this is a departure from the single Executor entrypoint. This is intentional as we separate the scheduling of the tasks from the execution of the tasks.

Executors

futureproof’s executors are a thin wrapper around the ones exposed by concurrent.futures.

class futureproof.executors.ThreadPoolExecutor(*args, monitor_interval=2, **kwargs)

Wrapper around concurrent.futures ThreadPoolExecutor.

Arguments not specified below will be forwarded to the underlying executor.

Parameters

monitor_interval – Frequency in seconds for monitor logging, defaults to 2 seconds, set to 0 to disable.

class futureproof.executors.ProcessPoolExecutor(*args, monitor_interval=2, **kwargs)

Wrapper around concurrent.futures ProcessPoolExecutor.

Available only in Python 3.7 and above.

Arguments not specified below will be forwarded to the underlying executor.

Parameters

monitor_interval – Frequency in seconds for monitor logging, defaults to 2 seconds, set to 0 to disable.

In the executors is where futureproof adds the monitoring functionality by regularly checking for completed tasks and logging as required.

An important difference with concurrent.futures is that you don’t call them directly, they need to be passed as parameters to task managers.

Task managers

Task managers are the main point of interaction with futureproof. They encapsulate a queue of jobs to be executed by an executor, exposing methods to add jobs and retrieve results easily.

class futureproof.task_manager.TaskManager(executor: futureproof.executors._FutureProofExecutor, error_policy: Union[futureproof.task_manager.ErrorPolicyEnum, str] = ErrorPolicyEnum.RAISE)

Manages how tasks are created and placed in the queue.

Executors pull Tasks from the managers.

Parameters

• executor – The executor that will execute the submitted tasks.

• error_policy – Error policy indicating what should the behaviour be if an exception is raised. Defaults to raise, raising the exception as soon as it happens and stopping all execution. log will only log the exception but continue the execution of the remaining tasks. ignore will not do anything, note, however, that the exception will be set as the result of the task so users can re-raise them if they so choose.

as_completed() → Iterator[futureproof.task_manager.Task]

Start the manager and return an iterator of completed tasks.

When using the task manager as a context manager as_completed must be used inside the context, otherwise there will be no effect as the task manager will wait until all tasks are completed.

completed_tasks

List of completed Task objects

join() → None

Block until all tasks are completed.

Alternatively use the task manager as a context manager, upon exiting join will be called and results will be available

map(fn: Callable, iterable: Iterable) → None

Submit a set of tasks from a callable and a iterable of arguments

iterable may be any iterable of primitives or an iterable of argument tuples.

property results: List

List of results for completed tasks.

Note the contents may different as more tasks are completed.

run() → None

Start the manager and wait until all tasks are completed before shutting down.

submit(fn: Callable, *args: Any, **kwargs: Any) → futureproof.task_manager.Task

Submit a task for execution.

The newly created Task will be returned.

class futureproof.task_manager.Task(fn, args=(), kwargs={}, result=None, complete=False)

Tasks describe an execution with parameters and encapsulate the result.

When submitted a Future is added to it encapsulating the result.

class futureproof.task_manager.ErrorPolicyEnum(value)

Error policy options.