[Biojava-dev] ExternalProcess class

[mark.schreiber at novartis.com]

Hi Martin -

More interesting stuff! BioJava already has two classes to launch external 
processes, ExecRunner and ProcessTools. From memory one is better than the 
other but I can't for the life of me remember which (can anyone help me 
out?). We should deprecate the lesser of the two.

Potentially yours is better than these two. Can you check out these two 
classes, if yours improves upon them then I'd be happy to check it in.

- Mark





"Martin Szugat" <Martin.Szugat at GMX.net>
Sent by: biojava-dev-bounces at portal.open-bio.org
09/06/2005 07:55 PM

 
        To:     <biojava-dev at biojava.org>
        cc:     (bcc: Mark Schreiber/GP/Novartis)
        Subject:        [Biojava-dev] ExternalProcess class


Hi!

It's me again. I've implemented an ExternalProcess class. It encapsulates
the necessity to run multi threaded input and output handlers when calling
an external process using Runtime.exec().

The STDERR and STDOUT outputs as well as the STDIN input must be
read/written in separate threads otherwise the calling application may 
hang
up. A problem occurs when running an external program multiple times, e.g.
running BLAST a thousand times. In this case for each iteration three
threads are generated. Under Linux threads are implemented as processes,
thus the process "java" is started three times for each iteration. However
the thread objects are not terminated by the garbage collector and thus 
the
threads/processes are not terminated. Even explicitly freeing the objects
does not work (I've tested this several times). This results in an
OutOfMemoryException after a few hundreds iterations, because the numbers 
of
processes is limited under Linux.

I've solved this problem by using BioJava's SimpleThreadPool. Output 
reading
and input writing is handled by Runnable input/output handlers, e.g. using 
a
StreamPipe objects it is possible to redirect the STDOUT of an external
process to the STDOUT of the calling process.

The usage of the ExternalProcess class is very simple, e.g. there are some
simple static methods that encapsulate the internal complexity. The class
also supports setting the environment variables or to inherit them from 
the
parent process. In addition the working directory can be set or can be
inherited. Finally there is a special feature: one can define variables 
for
the command line arguments, e.g. "program -c %PARAMETER%"---%PARAMETER% is
replaced by the value from a Properties object with the key "PARAMETER".

The classes and interfaces are fully documentated and there is a 
(repeated)
Unit-Test for the ExternalProcess class. I've attached this test, the
ExternalProcess class as well as the various handler classes.

I hope you'll find these classes useful and integrate them into BioJava.

Best regards

Martin

____________________
/   Martin Szugat    \
/ Author and Developer \
+--------------------------------+---------------+
|Phone: +49 (0)821 4206442       |Address:       |
|Fax:   +49 (0)821 4206443       |               |
|Mobil: +49 (0)179 7789714       |Zwerchgasse 6  |
|Email: Martin.Szugat at GMX.net    |86150 Augsburg |
|Web:   http://szugat.gmxhome.de |Germany        |
+--------------------------------+---------------+












_______________________________________________
biojava-dev mailing list
biojava-dev at biojava.org
http://biojava.org/mailman/listinfo/biojava-dev

[ Attachment ''READERWRITERPIPE.JAVA'' removed by Mark Schreiber ]
[ Attachment ''STREAMPIPE.JAVA'' removed by Mark Schreiber ]
[ Attachment ''EXTERNALPROCESS.JAVA'' removed by Mark Schreiber ]
[ Attachment ''EXTERNALPROCESSTEST.JAVA'' removed by Mark Schreiber ]
[ Attachment ''OUTPUTHANDLER.JAVA'' removed by Mark Schreiber ]
[ Attachment ''INPUTHANDLER.JAVA'' removed by Mark Schreiber ]
[ Attachment ''SIMPLEOUTPUTHANDLER.JAVA'' removed by Mark Schreiber ]
[ Attachment ''READERINPUTHANDLER.JAVA'' removed by Mark Schreiber ]
[ Attachment ''SIMPLEINPUTHANDLER.JAVA'' removed by Mark Schreiber ]
[ Attachment ''WRITEROUTPUTHANDLER.JAVA'' removed by Mark Schreiber ]