[Bioperl-l] Error loading sequence with load_seqdatabase.pl
Hilmar Lapp
hlapp at gmx.net
Thu Jun 9 02:15:14 EDT 2005
First off, I wouldn't continue this thread on biosql-l, it's for schema
questions and this clearly isn't one. The script is a perl/bioperl
script and some people on the bioperl-l list may happen to be on a
platform similar to yours.
Second, you're saying now the script wouldn't run at all anymore? With
what error message? You might try to supply --debug as an option if the
script does at least something before it dies. The output will be
potentially extensive so be sure to capture in a file, then send it me.
If the script dies immediately I'm afraid I can't do anything. My gut
feeling is that there is something with your compiler, C runtime
library, or DBI/DBD compiled code that doesn't play well with your
perl. Did you do a binary install of perl or did you compile it from
source?
Really your best bet is to find someone who's on the same or a similar
platform and see whether he/she has had similar problems or none of
these.
-hilmar
On Jun 9, 2005, at 11:55 AM, Duangdaow Kanhasiri wrote:
> The system I use hase following configs:
>
> CPU: 2 @ AthlonXP2000
> OS: Rocks Cluster v 3.3
> Total Memory: 2 GB
> DBD::Pg version: 1.42
> DBI version: 1.48
>
> I've attached the out put of the top command (top.txt)
> with this mail. Unfortunately that the script
> load_seqdatabase.pl wouldn't run anymore, no matter
> how many time I tried running it, therefore, I
> couldn't measure how much it consumes the resource
> (cpu, memory) on the machine.
>
> Regards,
>
> Davina
>
>
> --- Hilmar Lapp <hlapp at gmx.net> wrote:
>
>> What OS are you running this on? How much memory
>> have you got on the
>> machine on which you run the script, and on the
>> machine on which you
>> run the database? Are these the same or not? Which
>> version of DBI and
>> DBD::Pg?
>>
>> This hasn't been reported by anyone else really so I
>> suspect it's
>> either due to too limited memory, or a problem in
>> the DBD driver or in
>> the DBI compiled code. Can you watch the process
>> (using, e.g., top) and
>> see how fast it increases in memory consumption?
>> Since you can continue
>> when you restart it's not something specific to one
>> sequence that would
>> trigger the problem; rather it appears whenever you
>> have run through a
>> certain number of entries the process dies.
>>
>> -hilmar
>>
>> On Jun 8, 2005, at 7:43 PM, Duangdaow Kanhasiri
>> wrote:
>>
>>> Hi,
>>>
>>> I've used the bioperl script load_seqdatabase.pl
>> (came
>>> with the biosql' scripts) to load the bacterial
>>> sequence in genbank format(*.gbk) into PostgreSQL
>> 8.0
>>> database on Linux machine as:
>>>
>>> $perl load_seqdatabase.pl /export/Bacteria/*/*.gbk
>> &
>>>
>>> Where under the /export/Bacteria/ path are the
>>> Bacteria's name path e.g. Acinetobacter_sp_ADP1
>> and
>>> the file name are like NC_006824.gbk.
>>>
>>> Previously it used to load some sequences in to
>> some
>>> tables in biosql database (count from table
>> bioentry)
>>>
>>> bioseq=# select count(*) from bioentry;
>>> count
>>> -------
>>> 33
>>> (1 row)
>>>
>>>
>>> However, after a while it then stopped with the
>> the
>>> error:
>>>
>>> [1]+ Segmentation fault perl
>> load_seqdatabase.pl
>>> /export/Bacteria/*/*.gbk &
>>>
>>> I then checked and removed the *.gbk file that
>> have
>>> already been loaded in to the table, leaving only
>> the
>>> unloaded ones and ran the scripted again. It
>>> continued to work for some times and stopped
>> again. I
>>> repeated the process several times until 173
>> sequences
>>> were loaded into the table:
>>>
>>> bioseq=# select count(*) from bioentry;
>>> count
>>> -------
>>> 173
>>> (1 row)
>>>
>>> The program then stopped again and this time it
>>> wouldn't run anymore even I tried with only on
>> file.
>>> The error is still the same like:
>>>
>>> $ perl load_seqdatabase.pl
>>>
>>
> /export/Bacteria/Lactobacillus_johnsonii_NCC_533/NC_005362.gbk
>>> Segmentation fault
>>> $
>>>
>>> Now I couldn't load the rest of my sequences into
>> the
>>> database anymore. I would be very apprecialed if
>> any
>>> one knows how to solve the "Segmentation fault"
>>> problem?
>>>
>>> Regards,
>>>
>>> Davina
>>>
>>>
>>>
>>> __________________________________
>>> Discover Yahoo!
>>> Have fun online with music videos, cool games, IM
>> and more. Check it
>>> out!
>>> http://discover.yahoo.com/
>>>
>>
> online.html<load_seqdatabase.pl>_______________________________________
>>
>>> ________
>>> Bioperl-l mailing list
>>> Bioperl-l at portal.open-bio.org
>>>
>>
> http://portal.open-bio.org/mailman/listinfo/bioperl-l
>> --
>>
> -------------------------------------------------------------
>> Hilmar Lapp email: lapp
>> at gnf.org
>> GNF, San Diego, Ca. 92121 phone:
>> +1-858-812-1757
>>
> -------------------------------------------------------------
>>
>>
>>
>
>
>
> __________________________________
> Discover Yahoo!
> Get on-the-go sports scores, stock quotes, news and more. Check it out!
> http://discover.yahoo.com/mobile.html[root@biogenome root]# top
> 10:31:14 up 27 days, 21:20, 5 users, load average: 0.00, 0.02, 0.03
> 193 processes: 192 sleeping, 1 running, 0 zombie, 0 stopped
> CPU states: cpu user nice system irq softirq iowait
> idle
> total 1.8% 0.0% 0.0% 0.0% 0.0% 0.0%
> 198.0%
> cpu00 1.9% 0.0% 0.0% 0.0% 0.0% 0.0%
> 98.0%
> cpu01 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
> 100.0%
> Mem: 2057220k av, 1556640k used, 500580k free, 0k shrd,
> 167096k buff
> 1101048k actv, 266692k in_d, 39936k in_c
> Swap: 4192956k av, 91620k used, 4101336k free
> 1196752k cached
>
> PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU
> COMMAND
> 16683 root 23 0 1288 1288 844 R 1.9 0.0 0:00 0 top
> 1 root 15 0 520 516 456 S 0.0 0.0 0:29 0 init
> 2 root RT 0 0 0 0 SW 0.0 0.0 0:00 0
> migration/0
> 3 root RT 0 0 0 0 SW 0.0 0.0 0:00 1
> migration/1
> 4 root 15 0 0 0 0 SW 0.0 0.0 0:00 1
> keventd
> 5 root 34 19 0 0 0 SWN 0.0 0.0 0:00 0
> ksoftirqd/0
> 6 root 34 19 0 0 0 SWN 0.0 0.0 0:00 1
> ksoftirqd/1
> 9 root 15 0 0 0 0 SW 0.0 0.0 0:00 0
> bdflush
> 7 root 15 0 0 0 0 SW 0.0 0.0 0:37 0
> kswapd
> 8 root 15 0 0 0 0 SW 0.0 0.0 0:24 0
> kscand
> 10 root 15 0 0 0 0 SW 0.0 0.0 0:19 0
> kupdated
> 11 root 25 0 0 0 0 SW 0.0 0.0 0:00 0
> mdrecoveryd
> 17 root 25 0 0 0 0 SW 0.0 0.0 0:00 1
> scsi_eh_0
> 18 root 25 0 0 0 0 SW 0.0 0.0 0:00 1
> aacraid
> 20 root 25 0 0 0 0 SW 0.0 0.0 0:00 1
> scsi_eh_0
> 23 root 15 0 0 0 0 SW 0.0 0.0 1:29 1
> kjournald
> 70 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 khubd
> 1165 root 15 0 0 0 0 SW 0.0 0.0 0:49 0
> kjournald
> 1418 root 15 0 0 0 0 SW 0.0 0.0 0:00 1 eth0
> 1543 root 15 0 620 608 524 S 0.0 0.0 0:59 0
> syslogd
> 1547 root 15 0 484 424 420 S 0.0 0.0 0:00 0 klogd
> 1557 root 15 0 456 448 392 S 0.0 0.0 2:04 0
> irqbalance
> 1565 rpc 15 0 572 548 500 S 0.0 0.0 0:00 0
> portmap
> 1584 rpcuser 25 0 716 632 628 S 0.0 0.0 0:00 1
> rpc.statd
> 1595 root 15 0 404 388 344 S 0.0 0.0 0:06 0 mdadm
> 1619 root RT 0 556 456 424 S 0.0 0.0 0:16 1
> auditd
> 1629 nobody 15 0 1180 1016 724 S 0.0 0.0 24:57 1
> gmetad
> 1658 root 15 0 472 424 400 S 0.0 0.0 0:01 0 pvfsd
>
>
> [root at biogenome DBD]# df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/sda1 5.8G 3.6G 1.9G 66% /
> /dev/sda3 125G 24G 95G 21% /export
> none 1005M 0 1005M 0% /dev/shm
> tmpfs 503M 3.5M 499M 1% /var/lib/ganglia/rrds
>
--
-------------------------------------------------------------
Hilmar Lapp email: lapp at gnf.org
GNF, San Diego, Ca. 92121 phone: +1-858-812-1757
-------------------------------------------------------------
More information about the Bioperl-l
mailing list