[Bioperl-l] load_seqdatabase.pl memory requirements unusually large

[Michael Muratet]

Greetings

I have use bioperl-db and load_seqdatabase.pl many times in the past  
and it's worked pretty much out of the box.

I have been trying to load fasta files from hg18. For chr1, the  
virtual and resident memory quickly builds to 14 GB or so, then starts  
using up the 2GB swap until it's full and then the system hangs. The  
system is an 8-core Dell with 16GB of physical memory. chr1.fa is  
~242MB. All of the disk storage is network mounted on an EMC system  
which (I am told) has a proprietary version of something that's NFS- 
like.

I loaded chrM (~17kb) and load_seqdatabase grew to over 4 GB before it  
completed.

I am using MySQL 5.0.51a-community, DBI 1.607, perl 5.85, bioperl  
1.6.0, bioperl-db 1.006900.

I have the innodb engine enabled in MySQL and the buffers and caches  
set for a 'large' system.

I had some errors during the bioperl-db install:

Test Summary Report
-------------------
t/01dbadaptor.t (Wstat: 256 Tests: 23 Failed: 1)
   Failed test:  23
   Non-zero exit status: 1
t/10ensembl.t   (Wstat: 65280 Tests: 5 Failed: 0)
   Non-zero exit status: 255
   Parse errors: Bad plan.  You planned 18 tests but ran 5.
t/11locuslink.t (Wstat: 65280 Tests: 7 Failed: 0)
   Non-zero exit status: 255
   Parse errors: Bad plan.  You planned 113 tests but ran 7.
t/15cluster.t   (Wstat: 65280 Tests: 7 Failed: 0)
   Non-zero exit status: 255
   Parse errors: Bad plan.  You planned 162 tests but ran 7.
Files=16, Tests=1205, 21 wallclock secs ( 0.51 usr  0.12 sys + 14.90  
cusr  2.69 csys = 18.22 CPU)
Result: FAIL
Failed 4/16 test programs. 1/1205 subtests failed.

The most recent bioperl-db documentation says that a workable version  
may be possible after some errors and so I went ahead with the install.

The mailing list archive has some discussion about throughput but  
nothing really about filling up memory.

Can anyone offer any clues about what's going on or where to start  
looking?

Thanks

Mike