[MOBY-dev] XML parsing takes forever
Kenny Billiau
kenny.billiau at psb.ugent.be
Thu Oct 30 13:32:53 UTC 2008
Hi,
this is my first question, so I hope I'm sending this to the right
maillinglist. I recently installed the BioMoby packages and generated my
service using the moses-* perl scripts. So myCache is filled without problems
and I'm using the testing registry.
I'm using perl 5.8.5.
Distribution E/EK/EKAWAS/MOBY-1.05.tar.gz
Distribution E/EK/EKAWAS/MOBY-Client-1.02.tar.gz
Distribution E/EK/EKAWAS/MOSES-MOBY-0.86.tar.gz
I can provide the xml files I use to test these services too.
$ moses-generate-services.pl psb.ugent.be
$ moses-generate-services.pl -b psb.ugent.be
All works well.
I'm only interested in the BogasGetRegion and the BogasSaveRegion.
If I test BogasGetRegion with following command (1), all works fine:
$ moses-testing-service.pl -e
http://hal2000/sandbox/BOGAS_dev/trunk/www/ws/moby.pl BogasGetRegion
~/tmp/moby.xml
$ moses-testing-service.pl -e
http://hal2000/sandbox/BOGAS_dev/trunk/www/ws/moby.pl BogasSaveRegion
~/tmp/mobysave.xml
But If I test BogasSaveRegion (2), something goes seriously wrong. The command
as above is used, and the response from command (1) is fed to this one. Nothing
is done with the results in BogasSaveRegion, I simply generated the perl
modules and send the xml file to it.
This is the error message:
=====================================
--- TRANSPORT ERROR ---
500 read timeout
syntax error at line 1, column 0, byte 0 at
/usr/lib64/perl5/vendor_perl/5.8.5/x86_64-linux-thread-multi/XML/Parser.pm line
187
500 read timeout
=====================================
The timeout for apache is set to 10 minutes, but the transport error occures
after 180secs.
If I truncate the response from (1) enough, i get a response for (2) back
within those 180 secs. But still, as the XML file from (1) only is 250KB, it
seems unreasonably long.
If I look at what get's into to log, the script seems to do all it's number
crunching in the generated Base.pm (here called BogasSaveRegionBase.pm).
Generating the parser works, parsing itself goes slow beyong belief:
$self->default_throw_with_stack (0) unless $in_testing_mode;
my $in_package;
eval {
my $parser = $self->create_parser(
lowestKnownDataTypes => { 'locus_id' => 'String','date' =>
'DateTime','region' => 'Region', },
loadDataTypes => [ qw( String DateTime Region Boolean ) ]
) ;
# does not get beyond this point in a reasonable amount of time.
$in_package = $parser->parse (
method => $in_testing_mode ? 'file' : 'string',
data => $data
) ;
};
I tried to put the webservice through dprof, with these results (unfortunatly,
Apache::DProf seems to die on me, but invoking the mod_perl envirnment with
-d:DProf works):
[davebowman at hal2000 ~]$ dprofpp -F /tmp/tmon.out
XML:: LibXML::parse_string has 1 unstacked calls in outer
XML:: SAX::Base::end_element has 1 unstacked calls in outer
BOGAS::Moby::BogasSaveRegionBase::BogasSaveRegion has 1 unstacked calls in
outer
XML::SAX::Base::parse has 1 unstacked calls in outer
Exporter::Heavy::heavy_export has 16 unstacked calls in outer
Config::Simple::import_from has 1 unstacked calls in outer
SOAP::Server::handle has 1 unstacked calls in outer
MOSES::MOBY:: Parser::callMethod has 1 unstacked calls in outer
MOSES::MOBY:: Base::as_string has 1 unstacked calls in outer
Config::Simple::verbose has 1 unstacked calls in outer
utf8::AUTOLOAD has -1 unstacked calls in outer
HTTP::Message::__ANON__ has 3 unstacked calls in outer
Dumpvalue::unwrap has 8 unstacked calls in outer
XML::LibXML::_parse_sax_string has 1 unstacked calls in outer
MOSES::MOBY::Parser::parse has 1 unstacked calls in outer
XML::LibXML::SAX::_parse has 1 unstacked calls in outer
Config::Simple::vars has 1 unstacked calls in outer
MOSES::MOBY::Parser::end_element has 1 unstacked calls in outer
SOAP::Transport::HTTP::Server::handle has 1 unstacked calls in outer
AutoLoader::AUTOLOAD has -3 unstacked calls in outer
XML::SAX::Base::parse_string has 1 unstacked calls in outer
Fcntl::AUTOLOAD has -6 unstacked calls in outer
Dumpvalue:: dumpValue has 1 unstacked calls in outer
Dumpvalue:: DumpElem has 7 unstacked calls in outer
HTTP::Message::AUTOLOAD has -3 unstacked calls in outer
Fcntl::__ANON__ has 6 unstacked calls in outer
XML::SAX::Base::__ANON__ has 1 unstacked calls in outer
Exporter::export has -16 unstacked calls in outer
utf8::SWASHNEW has 1 unstacked calls in outer
SOAP::Transport::HTTP::CGI::handle has 1 unstacked calls in outer
XML::LibXML::SAX::_parse_string has 1 unstacked calls in outer
Faking 32 exit timestamp(s).
Total Elapsed Time = -34.8210 Seconds
User+System Time = 0 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c Name
0.00 140.6 140.64 486940 0.0003 0.0003 IO::Scalar::print
0.00 3.553 3.975 449046 0.0000 0.0000 Dumpvalue::stringify
0.00 2.551 2.551 574503 0.0000 0.0000 File::Spec::Unix::canonpath
0.00 2.005 147.07 232696 0.0000 0.0006 Dumpvalue::unwrap
0.00 1.177 1.177 423816 0.0000 0.0000 overload::mycan
0.00 1.019 1.019 743 0.0014 0.0014
MOSES::MOBY::Cache::Registries::in
it
0.00 0.959 3.885 742 0.0013 0.0052
MOSES::MOBY::Generators::Utils::fi
nd_file
0.00 0.581 2.989 286495 0.0000 0.0000 File::Spec::Unix::catfile
0.00 0.450 146.88 230026 0.0000 0.0006 Dumpvalue::DumpElem
0.00 0.367 0.607 56930 0.0000 0.0000 MOSES::MOBY::Base::__ANON__
0.00 0.257 1.876 1925 0.0001 0.0010
MOSES::MOBY::Cache::Central::get_d
atatype
0.00 0.246 0.246 1925 0.0001 0.0001
MOSES::MOBY::Cache::Central::_clea
n
0.00 0.234 155.05 1 0.2342 155.05 XML::LibXML::_parse_sax_string
0.00 0.218 0.218 11242 0.0000 0.0000 XML::LibXML::Node::_childNodes
0.00 0.143 0.143 108644 0.0000 0.0000 XML::LibXML::Node::nodeName
These results don't make me anymuch wiser. I tried switching xml parsers by
changing then in the mobyservices.cfg, with no result.
Does anyone know what the problem is?
Or how to debug this?
Am I missing something obvious here?
-Kenny
==================================================================
Kenny Billiau
Web Developer
Tel:+32 (0)9 331 36 95 fax:+32 (0)9 3313809
VIB Department of Plant Systems Biology, Ghent University
Technologiepark 927, 9052 Gent, BELGIUM
kenny.billiau at psb.ugent.be http://bioinformatics.psb.ugent.be
==================================================================
More information about the MOBY-dev
mailing list