[Bioperl-pipeline] pipelinemanager wish

[Elia Stupka]

> 	I would like to see this feature in the PipelineManager. Say we want to
> kill the PipelineManager, but in a nice controlled manner (not system
> failure) I want to have the option to do a nice shutdown/interrupt. In
> other words be able to do some rollback and shutdown. Things that need
> to be done:

I think these is the sort of thing where Arne was advocating that a
multi-threaded java live application would be so much easier, Frans and 
Juguang and Kiran, when/if you have time and deem it reasonable you
could look into a java pipelinemanager which would integrate the
monitor as well as a start button, a pause button, a shutdown button,
restart, etc. but anyway...

> Set all submitted jobs to fail. ( I don't see a suspend mode as being
> different from failing)

Except a)it increases your failed count, b)it will show as failed, and
make the user worried about why it failed. I think it would be nice to
distinguish it from FAILED, not in the internals (i.e. FAILED and
suspended issue the same commands everywhere) but for the external view of
it, e.g. in the monitor.

> Another shutdown mode would be a forced shutdown whereby you just kill
> all jobs, set everything not completed to failed. Of course, the state
> of your pipeline is unstable since you don't know how many outputs were
> already written for those in writing stage.

For the quick forced shutdown it could be decided by the user if all
running jobs should be set to failed, so they get rerun. In theory if the
output is in already it will (in the worst case) throw exceptions, or
write output twice (mild error), or be nice and not store it because it's
in again (best case)... if you don't do this, then the forced shutdown is
equal to not doing anthing and just switching off the
pipeline.... However, don't forget that you can issue a nice LSF suspend
which will stop all jobs and start running them again when you switch on
the machine again, that would be an alternative, not as quick as the
brutal shutdown, but faster than letting jobs finish.

> the interrupt mechanism maybe some  file or some table or have
> pipelinemanager reading a ctrl char from the terminal.

You can easily just write a shutdown script which suspends the
pipelinemanager process, shuts all things down, then kills the pipeline
manager process.

Elia

********************************
* http://www.fugu-sg.org/~elia *
* tel:    +65 6874 1467        *
* mobile: +65 9030 7613        *
* fax:    +65 6779 1117        *
********************************