TheadPoolExecutor with a bounded queue in Python

For tasks that are I/O bound, such as network calls, parallelizing across multiple threads can greatly increase throughput. Python 3 comes with a ThreadPoolExecutor class that makes it easy to queue up units of work that will by run by a pool of threads. ThreadPoolExecutor has an unbounded queue though, and sometimes I want to submit only a certain number of tasks before blocking on the submit() method. For example, sending a large number of requests to a server as fast as possible without buffering all the requests in memory. The below BoundedExecutor class below accomplishes this. Note that it delegates to a ThreadPoolExecutor instance instead of subclassing ThreadPoolExecutor, this decouples it from the underlying implementation of ThreadPoolExecutor (see Effective Java Item #16: Favor composition over inheritance).

Continue reading “TheadPoolExecutor with a bounded queue in Python”