[Biojava-l] Job launching system

[Keith James]

Just recently I've been working on a system for loading our batch
queues in a trackable manner and I'm wondering whether there is any
interest in adding it as a biojava cvs module. In some ways it's a bit
like a simplified Ensembl pipeline minus all the application-specific
bits (e.g. program output parsing) but with support for jobs being
"owned" by more than one project. The idea was to decouple running
programs from parsing output and storing the data.

All the resource management is of course delegated to a batch system
like LSF or PBS, although without a batch system it can just run stuff
on the local host. The DB persistence is a bit naive at the moment (it
might need some caching).

Features:
 Job ownership
 Job dependencies (both && / || children) and retries on failure
 Java native interface to LSF for submission/queue monitoring
 Jobs implement standard Java MutableTreeNode interface (-> JTree GUIs)
 Command line generation with Velocity Template Engine
 Plugin job selection policies
 PostgreSQL and Oracle support
 Logging with log4j
 Email notifications

Anyway, let me know.

Keith

-- 

-= Keith James - kdj@sanger.ac.uk - http://www.sanger.ac.uk/Users/kdj =-
Pathogen Sequencing Unit, Wellcome Trust Sanger Institute, Cambridge, UK