[Biojava-dev] ExternalProcess class

[mark.schreiber at novartis.com]

Hi Martin -

They are documented in the 1.4 API 
(http://www.biojava.org/docs/api14/index.html), but not the 1.3 API.

Looking at the code further the ExecRunner appears to be the better of 
those two. Your I/O pattern may be more useful though.

- Mark







"Martin Szugat" <Martin.Szugat at GMX.net>
09/19/2005 05:59 PM

 
        To:     Mark Schreiber/GP/Novartis at PH
        cc:     <biojava-dev at biojava.org>
        Subject:        RE: [Biojava-dev] ExternalProcess class


Hi Mark,

> More interesting stuff! BioJava already has two classes to launch 
external
> processes, ExecRunner and ProcessTools. From memory one is better than 
the
> other but I can't for the life of me remember which (can anyone help me
> out?). We should deprecate the lesser of the two.

That's interesting! I didn't find these classes because they are not
documented in the JavaDoc?!

> 
> Potentially yours is better than these two. Can you check out these two
> classes, if yours improves upon them then I'd be happy to check it in.

I don't know if it's better but different: You can provide your custom
(threaded) input and output handlers. I use it for parsing and writing 
BLAST
output/input on the fly.

I'll make a new package and send it you with the next mail.

Best regards

Martin

> 
> - Mark
> 
> 
> 
> 
> 
> "Martin Szugat" <Martin.Szugat at GMX.net>
> Sent by: biojava-dev-bounces at portal.open-bio.org
> 09/06/2005 07:55 PM
> 
> 
>         To:     <biojava-dev at biojava.org>
>         cc:     (bcc: Mark Schreiber/GP/Novartis)
>         Subject:        [Biojava-dev] ExternalProcess class
> 
> 
> Hi!
> 
> It's me again. I've implemented an ExternalProcess class. It 
encapsulates
> the necessity to run multi threaded input and output handlers when 
calling
> an external process using Runtime.exec().
> 
> The STDERR and STDOUT outputs as well as the STDIN input must be
> read/written in separate threads otherwise the calling application may
> hang
> up. A problem occurs when running an external program multiple times, 
e.g.
> running BLAST a thousand times. In this case for each iteration three
> threads are generated. Under Linux threads are implemented as processes,
> thus the process "java" is started three times for each iteration. 
However
> the thread objects are not terminated by the garbage collector and thus
> the
> threads/processes are not terminated. Even explicitly freeing the 
objects
> does not work (I've tested this several times). This results in an
> OutOfMemoryException after a few hundreds iterations, because the 
numbers
> of
> processes is limited under Linux.
> 
> I've solved this problem by using BioJava's SimpleThreadPool. Output
> reading
> and input writing is handled by Runnable input/output handlers, e.g. 
using
> a
> StreamPipe objects it is possible to redirect the STDOUT of an external
> process to the STDOUT of the calling process.
> 
> The usage of the ExternalProcess class is very simple, e.g. there are 
some
> simple static methods that encapsulate the internal complexity. The 
class
> also supports setting the environment variables or to inherit them from
> the
> parent process. In addition the working directory can be set or can be
> inherited. Finally there is a special feature: one can define variables
> for
> the command line arguments, e.g. "program -c %PARAMETER%"---%PARAMETER% 
is
> replaced by the value from a Properties object with the key "PARAMETER".
> 
> The classes and interfaces are fully documentated and there is a
> (repeated)
> Unit-Test for the ExternalProcess class. I've attached this test, the
> ExternalProcess class as well as the various handler classes.
> 
> I hope you'll find these classes useful and integrate them into BioJava.
> 
> Best regards
> 
> Martin
> 
> ____________________
> /   Martin Szugat    \
> / Author and Developer \
> +--------------------------------+---------------+
> |Phone: +49 (0)821 4206442       |Address:       |
> |Fax:   +49 (0)821 4206443       |               |
> |Mobil: +49 (0)179 7789714       |Zwerchgasse 6  |
> |Email: Martin.Szugat at GMX.net    |86150 Augsburg |
> |Web:   http://szugat.gmxhome.de |Germany        |
> +--------------------------------+---------------+
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> biojava-dev mailing list
> biojava-dev at biojava.org
> http://biojava.org/mailman/listinfo/biojava-dev
> 
> [ Attachment ''READERWRITERPIPE.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''STREAMPIPE.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''EXTERNALPROCESS.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''EXTERNALPROCESSTEST.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''OUTPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''INPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''SIMPLEOUTPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''READERINPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''SIMPLEINPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''WRITEROUTPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> 
> 
> _______________________________________________
> biojava-dev mailing list
> biojava-dev at biojava.org
> http://biojava.org/mailman/listinfo/biojava-dev