Futureproof - Bulletproof concurrent.futures#

Documentation Status Build Status Supported Python Versions PyPI codecov shoutouts.dev

concurrent.futures is amazing, but it’s got some sharp edges that have bit me many times in the past.

Futureproof is a thin wrapper around it addressing some of these problems and adding some usability features.

Features:#

  • Monitoring: a summary of recent completed tasks is logged by default.

  • Fail fast: errors cause the main thread to raise an exception and stop by default.

  • Error policy: the user can decide whether to raise, log or completely ignore errors on tasks.

  • Backpressure control: large collections of tasks are consumed lazily as the executor completes tasks, drastically reducing memory consumption and improving responsiveness in these situations.

Current status: Alpha#

The API is subject to change, any changes will be documented in the changelog.

Futureproof was designed to wrap ThreadPoolExecutor, however version 0.2+ includes limited support ProcessPoolExecutor but only for Python3.7+.