[Biopython-dev] Bio.Tools.MultiProc code

[Jeffrey Chang]

Hello everyone,

I just checked in a new package Bio.Tools.MultiProc that contains some
code for rudimentary multiprocessing.  There are 3 modules:
- copen.py
  Code that wraps a child process in a file-like object.  This
  basically gives you a file handle to a fork'd system command or
  python function.

- Task.py
  Partially implements the threading.Thread interface around copen, to
  give a common interface for multi-threaded and fork'd subprocesses.

- Scheduler.py
  Helps run multiple Thread (or Task) objects.


Here's a trivial example:

>>> from Bio.Tools.MultiProc import copen
>>> def add1(x):
...   return x+1
... 
>>> h = copen.copen_fn(add1, 5)   
>>> print h.read()
6
>>> 

add1 was actually computed as a separate process.


I've been using this code to help speed up a trivially parallelizable
computation.  Hopefully someone else may find this useful as well!

Jeff