pysys.utils.threadpool module

exception pysys.utils.threadpool.NoResultsPending

Bases: Exception

All work requests have been processed.

exception pysys.utils.threadpool.NoWorkersAvailable

Bases: Exception

No worker threads available to process remaining requests.

class pysys.utils.threadpool.ThreadPool(num_workers, q_size=0, resq_size=0, poll_timeout=5)

Bases: object

Main pool to manage worker threads processing an internal request queue.

__init__(num_workers, q_size=0, resq_size=0, poll_timeout=5)

Class constructor.

Parameters:

• num_workers – The number of worker threads processing the queue
• q_size – The request queue size
• resq_size – The response queue size
• poll_timeout – The polling timeout of worker threads when getting requests from the queue

createWorkers(num_workers, poll_timeout=5)

Create additional threads on the workers stack.

Parameters:

• num_workers – The number of workers to add to the stack
• poll_timeout – The timeout of the threads when waiting for a request on the queue

dismissWorkers(num_workers, do_join=False)

Dismiss worker threads from the workers stack.

Stops a set number of workers in the workers list by popping the workers of the list stack.

Parameters:

• num_workers – The number of workers to dismiss
• do_join – If True wait for all threads to terminate before returning from the call

joinAllDismissedWorkers()

Join all dismissed workers.

Blocks until all dismissed worker threads terminate. Use when calling dismissWorkers with do_join = False.

poll(block=False)

Poll the request queue until the queue is empty.

Raises a NoResultsPending or NoWorkersAvailable exception if the results queue is initially empty, or there are no available workers. Otherwise processes the results queue and calls the request callback with the result of the request.

putRequest(request, block=True, timeout=0)

Place a WorkRequest on the request queue.

Parameters:

• request – The WorkRequest to place on the request queue
• block – If set to True, block queue operations until complete, otherwise use timeout
• timeout – The timeout to use for queue operations when block is set to False

wait()

Block until there are no request results pending on the queue.

Callbacks for work requests are executed by this method until all results have been dealt with.

class pysys.utils.threadpool.WorkRequest(callable_, args=None, kwds=None, requestID=None, callback=None, exc_callback=<function _handle_thread_exception>)

Bases: object

Holds the details of a request placed on the thread pool request queue.

__init__(callable_, args=None, kwds=None, requestID=None, callback=None, exc_callback=<function _handle_thread_exception>)

Class constructor.

Parameters:

• callable – The callable object or function
• args – The argument list to the callable object or function
• kwds – The keyword arguments to the callable object or function
• requestID – An ID for the request
• callback – A callback on completion of the request
• exc_callback – A callback when the request throws an excetion

class pysys.utils.threadpool.WorkerThread(requests_queue, results_queue, poll_timeout=5, **kwds)

Bases: threading.Thread

Thread to perform work requests managed by the thread pool object.

The thread polls the thread safe queue of the thread pool instance to retrieve work requests in the form of a callable reference with parameters. On completion of a work request the thread places the results on another thread safe queue of the thread pool and waits to get a new request.

__init__(requests_queue, results_queue, poll_timeout=5, **kwds)

Class constructor.

Parameters:

• requests_queue – Reference to the threadpool’s request queue
• results_queue – Reference to the threadpool’s results queue
• poll_timeout – The timeout when trying to obtain a request from the request queue
• kwds – Variable arguments to be passed to the threading.Thread constructor

dismiss()

Stop running of the worker thread.

run()

Start running the worker thread.