[Bioperl-l] Remote blast fork errors / Process limit restrictions

[Chris Fields]

On Dec 17, 2009, at 2:23 PM, Robert Bradbury wrote:

> Oh, yes, in case it was not clear, the fork calls which fails is in
> DB/WebDBSeqI.pm: line 722
>     defined(my $pid = fork)
>          or $self->throw("'Couldn't fork: $!");

Okay, that's a bit more helpful.

> And of course that is because Linux has reached the process limits for the
> user (due to accumulated background processes which are uncollected).

Right, but again, we need to check this in a cross-platform compatible way.

> And they could be resolved by simply executing a simple waitpid call for
> prior orphaned children before forking [1] But such a succinct solution
> would violate "functional" programming rules -- clean up what you create --
> instead they would tend to fall into the OO camp -- "Oh don't worry the
> garbage collector will take care of it".  Green programming is a little less
> cavalier.
> 
> Robert
> 
> 1. IMO, a very very real problem with programming today is that there is no
> connection between programmers and the cost of their programs.  How many
> programmers know the instruction cycle time of their computers, what does an
> instruction cost in terms of W consumed, W wasted (heat generation),
> fruitless scanning over uncollected zombie processes, etc.  It may be that
> only that programmers who grew up in the era when CPU cycles were expensive
> (300 ns/cycle) who know what each instruction required in terms of cycles
> consider these perspectives.  Now things (cpu use, processor use, etc) tend
> to be swept under the rug and it appears that that is the case with the
> standard implementation of bioper.  The documentation does not clearly state
> that additional sub-processes may be created and need to be collected.  You
> are providing a utility that only works "this much".  And guess what -- I
> happen to have run into the "this".

Um, yeah.  Okay.

chris