[Bioperl-l] load_seqdatabase.pl memory requirements unusually large

[Hilmar Lapp]

It could be, but that doesn't explain the test failures. They need not  
have the same cause, but they could, and the tests failing the way  
they do is certainly not right.

	-hilmar

On Feb 6, 2009, at 3:04 PM, Florent Angly wrote:

> Out of the blue, I'm going to ask: could it be some memory leak due  
> to a circular reference?
> Florent
>
> Hilmar Lapp wrote:
>> Something seems to cause Perl to be crapping out. It it were a  
>> programmatic exception you would see the message and the trace.
>>
>> Could you run these tests by themselves:
>>
>>    ./Build test --test-files t/11locuslink.t
>>
>> If that doesn't reveal the error, add a verbose=1 argument.
>>
>> Let us know what you find.
>>
>>    -hilmar
>>
>> On Feb 6, 2009, at 11:03 AM, Michael Muratet wrote:
>>
>>> Greetings
>>>
>>> I have use bioperl-db and load_seqdatabase.pl many times in the  
>>> past and it's worked pretty much out of the box.
>>>
>>> I have been trying to load fasta files from hg18. For chr1, the  
>>> virtual and resident memory quickly builds to 14 GB or so, then  
>>> starts using up the 2GB swap until it's full and then the system  
>>> hangs. The system is an 8-core Dell with 16GB of physical memory.  
>>> chr1.fa is ~242MB. All of the disk storage is network mounted on  
>>> an EMC system which (I am told) has a proprietary version of  
>>> something that's NFS-like.
>>>
>>> I loaded chrM (~17kb) and load_seqdatabase grew to over 4 GB  
>>> before it completed.
>>>
>>> I am using MySQL 5.0.51a-community, DBI 1.607, perl 5.85, bioperl  
>>> 1.6.0, bioperl-db 1.006900.
>>>
>>> I have the innodb engine enabled in MySQL and the buffers and  
>>> caches set for a 'large' system.
>>>
>>> I had some errors during the bioperl-db install:
>>>
>>> Test Summary Report
>>> -------------------
>>> t/01dbadaptor.t (Wstat: 256 Tests: 23 Failed: 1)
>>> Failed test:  23
>>> Non-zero exit status: 1
>>> t/10ensembl.t   (Wstat: 65280 Tests: 5 Failed: 0)
>>> Non-zero exit status: 255
>>> Parse errors: Bad plan.  You planned 18 tests but ran 5.
>>> t/11locuslink.t (Wstat: 65280 Tests: 7 Failed: 0)
>>> Non-zero exit status: 255
>>> Parse errors: Bad plan.  You planned 113 tests but ran 7.
>>> t/15cluster.t   (Wstat: 65280 Tests: 7 Failed: 0)
>>> Non-zero exit status: 255
>>> Parse errors: Bad plan.  You planned 162 tests but ran 7.
>>> Files=16, Tests=1205, 21 wallclock secs ( 0.51 usr  0.12 sys +  
>>> 14.90 cusr  2.69 csys = 18.22 CPU)
>>> Result: FAIL
>>> Failed 4/16 test programs. 1/1205 subtests failed.
>>>
>>> The most recent bioperl-db documentation says that a workable  
>>> version may be possible after some errors and so I went ahead with  
>>> the install.
>>>
>>> The mailing list archive has some discussion about throughput but  
>>> nothing really about filling up memory.
>>>
>>> Can anyone offer any clues about what's going on or where to start  
>>> looking?
>>>
>>> Thanks
>>>
>>> Mike
>>> _______________________________________________
>>> Bioperl-l mailing list
>>> Bioperl-l at lists.open-bio.org
>>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>>

-- 
===========================================================
: Hilmar Lapp  -:-  Durham, NC  -:-  hlapp at gmx dot net :
===========================================================