[DAS] SOAP/XML: best way to marshall large objects
Tue, 8 Jan 2002 09:32:12 -0500
I have a question for Brian, Thomas, and anyone else who has good experience
I have just spent some time with the Perl SOAPLite modules, and am wondering
how best to marshall large data structures in SOAP. The case of concern is
the DAS features request, which in SOAPish form becomes the
This request returns a large number of features, typically thousands. In the
Perl SOAP API, there are two alternative ways to handle this. One processes
the entire response in one big swallow and returns an array of objects. The
other creates a getNextSequenceFeature iterator and calls the iterator
repeatedly to fetch each feature.
Neither method is particularly optimal. The first wastes a lot of time
parsing the XML stream and eating up memory before it returns a result that
the user code can work with. The second makes a network method call for each
feature, killing the transaction with latency delays.
Of course, I can also get the XML stream from the SOAP request, pass it to a
parser, and invoke callbacks, and perhaps this is the best way to do it. But
I wonder what experience and advice people who are experienced with the Java
APIs have on this before I jump to the wrong conclusions.
Lincoln D. Stein Cold Spring Harbor Laboratory
firstname.lastname@example.org Cold Spring Harbor, NY