backend.SessionPool

class backend.SessionPool(session, func, processes=None, maxtasksperchild=None, context=None, **kwargs)

An implementation of a multiprocessing Pool which loads a Session object into each worker that is applied as the first argument to the workers’ function.

This should be used in a with statement as follows:

with SessionPool(self, worker_func, 4) as p:

result = p.map(p.runner, <iterable>)

Note that nested functions and labmdas can be used as the worker_function since we pickle them using the dill library

__init__(session, func, processes=None, maxtasksperchild=None, context=None, **kwargs)

Initialize the Pool with the given session, function, and number of processes. The session is saved to a temporary directory and is then loaded ino each worker process. The function is the global function you want the workers to run, the first argument of which should be the session

static init_pool(session_filename, dilled_function, kwargs)

This is the initialization method for multiprocessing pools, which takes as an argument the filename of the saved session and the function to run in the pool

static runner(*args, **kwargs)

This is the function to actually use when running a mp Pool