Bioperl: Memory leak in BLAST modules

[Steve Chervitz]

Lincoln,

Yes, there are some cycles but they are *supposed* to be cleaned up
during destruction. I'm not sure I've taken the best approach
with the destructors for the relevant objects, however.

The cycles in the object refs to be aware of:

 * Bio::Tools::Blast.pm contains refs for Bio::Tools::Blast::Sbjct.pm objects.
 * Bio::Tools::Blast::Sbjct.pm contains refs for Bio::Tools::Blast::HSP.pm 
         objects and a ref to the parent Blast.pm object.
 * Bio::Tools::Blast::HSP.pm contains a ref to the parent Sbjct.pm object.

The key methods to look into are the DESTROY() methods of:

 * Blast.pm, Sbjct.pm, HSP.pm, and the objects they inherit from:
 * Bio::Tools::SeqAnal.pm (superclass of Blast.pm)
 * Bio::Root::Object.pm (superclass of SeqAnal.pm, Sbjct.pm, and HSP.pm)
        This object manages the reference to the reference to the
        parent object ('_parent') for all objects that know their parents.

I've tinkered around with this issue quite a bit. Any advice you could
offer would be greatly appreciated. 

Steve 

Lincoln Stein writes:
 > Unfortunately it's still leaking, although maybe not quite so fast.  I
 > don't think I'm going to be able to complete my analysis task at this
 > rate.
 > 
 > Are there any cycles in the object references that Blast.pm generates?
 > 
 > Lincoln
 > 
 > Steve Chervitz writes:
 >  > Lincoln,
 >  > 
 >  > I've been grappling with the memory leak for a while and have a new
 >  > version of Blast.pm that includes a fix for memory leaks when parsing
 >  > streams of reports. This is version 0.063 which you can get from:
 >  > 
 >  > http://genome-www.stanford.edu/perlOOP/bioperl/lib/Bio/Tools/Blast.pm
 >  > 
 >  > Memory use is still an issue, particularly if you are parsing huge
 >  > reports (on the order of 1 Mb or more) and not imposing significance
 >  > criteria. If you can include even a minimal cutoff (i.e., -signif =>
 >  > 0.01), that should improve memory usage compared to using no cutoff. 
 >  > 
 >  > Also, when creating individual Blast objects, it's a good idea to
 >  > explicitly destroy them when you're done processing each one
 >  > ($blast_obj->destroy). Using a single $blast_obj variable and
 >  > re-assigning it should accomplish the same thing (but calling
 >  > destroy() makes me feel better ;). 
 >  > 
 >  > If you (or others) have additional ideas for how to improve memory
 >  > usage, I'd be happy to hear them. Do you know of a good memory
 >  > management tool for Perl that can show reference counts etc. during
 >  > the course of an execution?
 >  > 
 >  > I haven't bundled the new version of Blast.pm into a distribution
 >  > package yet since there are other changes I want to include. But look
 >  > for it soon. 
 >  > 
 >  > Steve 
 >  > ___________________________________________________
 >  > Steve A. Chervitz               Neomorphic Software  
 >  > sac@neomorphic.com              2612b 8th Street
 >  > http://www.neomorphic.com       Berkeley, CA 94710 
 >  > 
 >  > 
 >  > Lincoln Stein writes:
 >  >  > Does anyone know of a memory leak in the Bio::Tools::Blast module?
 >  >  > I've got a script that creates and destroys several thousands of these 
 >  >  > objects and it is definitely leaking.  I don't want spend time
 >  >  > debugging the thing if there's a known problem.
 >  >  > 
 >  >  > Lincoln
 >  >  > 
 >  >  > -- 
 >  >  > ========================================================================
 >  >  > Lincoln D. Stein                           Cold Spring Harbor Laboratory
 >  >  > lstein@cshl.org			                  Cold Spring Harbor, NY
 >  >  > ========================================================================
 >  >  > =========== Bioperl Project Mailing List Message Footer =======
 >  >  > Project URL: http://bio.perl.org/
 >  >  > For info about how to (un)subscribe, where messages are archived, etc:
 >  >  > http://www.techfak.uni-bielefeld.de/bcd/Perl/Bio/vsns-bcd-perl.html
 >  >  > ====================================================================
 >  >  > 
 >  > =========== Bioperl Project Mailing List Message Footer =======
 >  > Project URL: http://bio.perl.org/
 >  > For info about how to (un)subscribe, where messages are archived, etc:
 >  > http://www.techfak.uni-bielefeld.de/bcd/Perl/Bio/vsns-bcd-perl.html
 >  > ====================================================================
 > -- 
 > ========================================================================
 > Lincoln D. Stein                           Cold Spring Harbor Laboratory
 > lstein@cshl.org			                  Cold Spring Harbor, NY
 > ========================================================================
 > 
=========== Bioperl Project Mailing List Message Footer =======
Project URL: http://bio.perl.org/
For info about how to (un)subscribe, where messages are archived, etc:
http://www.techfak.uni-bielefeld.de/bcd/Perl/Bio/vsns-bcd-perl.html
====================================================================