[Bioperl-l] RE: NCBI eutils

[NIH Email Delivery]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_000_01C29B62.31D83B30
Content-Type: text/plain

This message was resent by the system. We are sorry for any inconvenience
this delay may have caused sender or recipient.



------_=_NextPart_000_01C29B62.31D83B30
Content-Type: message/rfc822

Message-ID: <C22C76D2A47CF24DA972C1963DC1A83302DF379A@nihexchange1.nih.gov>
From: "Ostell, Jim (NIH/NLM/NCBI)" <ostell@ncbi.nlm.nih.gov>
To: "'lstein@cshl.org'" <lstein@cshl.org>
Cc: "'bioperl-l@bioperl.org'" <bioperl-l@bioperl.org>
Subject: RE: NCBI eutils
Date: Tue, 3 Dec 2002 11:30:10 -0500 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2656.59)
Content-Type: text/plain;
	charset="iso-8859-1"

We looked into this and found that some newer binaries that we had in
testing (with
a slightly different Entrez link bar and some other changes) do not seem to
show this
problem. So it seems to be fixed, although we are a little uncertain since
we don't know
why it is fixed in this version. But it is hard to debug the old one..

So try it and let us know if you still see a problem now. If the problem
reappears, let us
know right away and we will see if we can track it down.

  Jim Ostell

-----Original Message-----
From: Lincoln Stein [mailto:lstein@cshl.org] 
Sent: Sunday, December 01, 2002 3:20 PM
To: Ostell, Jim (NIH/NLM/NCBI)
Cc: bioperl-l@bioperl.org
Subject: NCBI eutils


Hi Jim,

We're very nearly ready to release a version of bioperl that uses the NCBI 
eutils (esearch, epost and efetch) and enforces the 3s delay that you folks 
recommend.  The only problem is that in recent days equery/efetch have
become 
very unreliable.  The symptom is this:

	1) formulate a query and submit it to equery.
	2) recover the WebEnv and QueryKey fields
	3) use these fields in a request to efetch
	4) efetch returns an OK code, but empty content

Here is an example:

	GET
http://www.ncbi.nih.gov/entrez/eutils/esearch.fcgi?db=nucleotide&
		mindate=1%2F1%2F2002&maxdate=1%2F30%2F2002&datetype=mdat&
	
usehistory=y&tool=bioperl&term=Onchocerca+volvulus[Organism]&retmax=73

The response looks OK and returns:

<?xml version="1.0"?>
<!DOCTYPE eSearchResult PUBLIC "-//NLM//DTD eSearchResult, 11 May 2002//EN" 
"http://www.ncbi.nlm.nih.gov/entrez/query/DTD/eSearch_020511.dtd">
<eSearchResult>
	<Count>731</Count>
	<RetMax>100</RetMax>
	<RetStart>0</RetStart>
	<QueryKey>1</QueryKey>
	
<WebEnv>M%40uXFBO%5DBt%5E%5ECD_AN%3C%3C%3CcBD%3F%60%5C%3C%40%3Ch%40%5CHfC%3F
%3Do%3C%3EE%3FjBAeI%3EKF%5E%3CD</WebEnv>
	<IdList>
		<Id>5835443</Id>
		<Id>12005979</Id>
		<Id>12005977</Id>
		...
	</IdList>
	<TranslationSet>
		<Translation>
			<From>Onchocerca+volvulus%5BOrganism%5D</From>
			<To>%22Onchocerca+volvulus%22%5BOrganism%5D</To>
		</Translation>
	</TranslationSet>
	<TranslationStack>
		<TermSet>
			<Term>"Onchocerca volvulus"[Organism]</Term>
			<Field>Organism</Field>
			<Count>15432</Count>
			<Explode>Y</Explode>
		</TermSet>
		<TermSet>
			<Term>1/1/2002[mdat]</Term>
			<Field>mdat</Field>
			<Count>-1</Count>
			<Explode>Y</Explode>
		</TermSet>
		<TermSet>
			<Term>1/30/2002[mdat]</Term>
			<Field>mdat</Field>
			<Count>-1</Count>
			<Explode>Y</Explode>
		</TermSet>
		<OP>RANGE</OP>
		<OP>AND</OP>
	</TranslationStack>
</eSearchResult>

Now when I run the fetch on the indicated WebEnv, I get

GET 
'http://www.ncbi.nih.gov/entrez/eutils/efetch.fcgi?rettype=gb&db=nucleotide&
query_key=1&
	tool=bioperl&retmode=text&	
WebEnv=Oc[%40A%5ECE%5C]B_EJ%3CIiF_B%5CF%40dGheCJGYkka%40A]Kj%3F%5ECJIkJ%3ELK
Aa%3C%3D&
usehistory=y'

Connection: close
Date: Sun, 01 Dec 2002 20:18:38 GMT
Via: 1.1 www.ncbi.nih.gov
Server: Apache
Content-Type: text/plain
Client-Date: Sun, 01 Dec 2002 20:18:38 GMT
Client-Response-Num: 1
Client-Transfer-Encoding: chunked
Set-Cookie: WebEnv=p=KP_FeGB>ffADfECcNbIIcYchi^FIzb@G?`FkDCAj=DIcD>KFd\E; 
domain=.nlm.nih.gov; path=/; expires=Sun, 01-Dec-2002 21:18:41 GMT
X-Cache: MISS from www.ncbi.nih.gov

There is supposed to be some content following the headers, but it looks to
me 
as though the NCBI server crashed.

Note that this isn't always the case.  About one time out of five it works.

Last week, this was working 100% of the time, but it started to get flaky 
over thanksgiving.

Lincoln

-- 
Lincoln Stein
lstein@cshl.org

------_=_NextPart_000_01C29B62.31D83B30--