<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class="">Hi David,</div>
<div class=""><br class="">
</div>
<div class="">NCBI/Entrez queries are prone to this - I usually wrap the request in a structure that allows retries in case of errors, e.g. something like</div>
<div class=""><br class="">
</div>
<div class="">```</div>
<div class=""># Report last exception as string<br class="">
def last_exception():<br class="">
""" Returns last exception as a string, or use in logging."""<br class="">
exc_type, exc_value, exc_traceback = sys.exc_info()<br class="">
return ''.join(traceback.format_exception(exc_type, exc_value,<br class="">
exc_traceback))</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
# Retry Entrez requests (or any other function)<br class="">
def entrez_retry(fn, logger, *fnargs, **fnkwargs):<br class="">
"""Retries the passed function up to the number of times specified<br class="">
by args.retries<br class="">
"""<br class="">
tries, success = 0, False<br class="">
while not success and tries < args.retries:<br class="">
try:<br class="">
output = fn(*fnargs, **fnkwargs)<br class="">
success = True<br class="">
except:<br class="">
tries += 1<br class="">
logger.warning("Entrez query %s(%s, %s) failed (%d/%d)",<br class="">
fn, fnargs, fnkwargs, tries+1, args.retries)<br class="">
logger.warning(last_exception())<br class="">
if not success:<br class="">
logger.error("Too many Entrez failures (exiting)")<br class="">
sys.exit(1)<br class="">
return output</div>
<div class="">```</div>
<div class=""><br class="">
</div>
<div class="">The except: clause would be better if it caught IncompleteRead…</div>
<div class=""><br class="">
</div>
<div class="">Cheers,</div>
<div class=""><br class="">
</div>
<div class="">L.</div>
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On 2 Nov 2016, at 08:57, David Martin (Staff) <<a href="mailto:d.m.a.martin@dundee.ac.uk" class="">d.m.a.martin@dundee.ac.uk</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; font-family: Helvetica; font-size: 10px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I’m trying to retrieve a sequence from NCBI and am getting incomplete reads. Any hints or tips?<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
..d<span class="Apple-converted-space"> </span><o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
handle= Entrez.efetch(db="nuccore", id=gi, retmode='text', rettype='gb')<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
seq= handle.read()<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Traceback (most recent call last):<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
File "<ipython-input-11-f879be0f80fe>", line 2, in <module><o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
seq= handle.read()<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
File "C:\Anaconda3\lib\http\client.py", line 440, in read<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
return self._readall_chunked()<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
File "C:\Anaconda3\lib\http\client.py", line 550, in _readall_chunked<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
raise IncompleteRead(b''.join(value))<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
IncompleteRead: IncompleteRead(6841762 bytes read)<o:p class=""></o:p></div>
</div>
<br style="font-family: Helvetica; font-size: 10px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: 10pt;" class="">The
University of Dundee is a registered Scottish Charity, No: SC015096</span><span style="font-family: Helvetica; font-size: 10px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""><span class="Apple-converted-space"> </span>_______________________________________________</span><br style="font-family: Helvetica; font-size: 10px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 10px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Biopython
mailing list - </span><a href="mailto:Biopython@mailman.open-bio.org" style="color: rgb(149, 79, 114); text-decoration: underline; font-family: Helvetica; font-size: 10px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Biopython@mailman.open-bio.org</a><br style="font-family: Helvetica; font-size: 10px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<a href="http://mailman.open-bio.org/mailman/listinfo/biopython" style="color: rgb(149, 79, 114); text-decoration: underline; font-family: Helvetica; font-size: 10px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">http://mailman.open-bio.org/mailman/listinfo/biopython</a></div>
</blockquote>
</div>
<br class="">
<div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px;">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px;">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px;">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<span class="Apple-style-span" style="border-collapse: separate; orphans: 2; text-indent: 0px; widows: 2; border-spacing: 0px;">
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
--<br class="">
Dr Leighton Pritchard<br class="">
Information and Computing Sciences Group; Weeds, Pests and Diseases Theme<br class="">
DG31, James Hutton Institute (Dundee)<br class="">
Errol Road, Invergowrie, Perth and Kinross, Scotland, DD2 5DA<br class="">
e: <a href="mailto:leighton.pritchard@hutton.ac.uk" class="">leighton.pritchard@hutton.ac.uk</a> w:
<a href="http://www.hutton.ac.uk/staff/leighton-pritchard" class="">http://www.hutton.ac.uk/staff/leighton-pritchard</a><br class="">
gpg/pgp: 0xFEFC205C tel: +44(0)844 928 5428 x8827 or +44(0)1382 568827<br class="">
<br class="">
</div>
</span></div>
</span></div>
</span></div>
</span></div>
</div>
</div>
<br class="">
<br /><br />
<p>This email is from the James Hutton Institute, however the views expressed by the sender are not necessarily the views of the James Hutton Institute and its subsidiaries. This email and any attachments are confidential and
are intended solely for the use of the recipient(s) to whom they are addressed.</p>
<p>If you are not the intended recipient, you should not read, copy, disclose or rely on any information contained in this email, and we would ask you to contact the sender immediately and delete the email from your system. Although the James Hutton Institute has taken reasonable precautions to ensure no viruses are present in this email, neither the Institute nor the sender accepts any responsibility for any viruses, and it is your responsibility to scan the email and any attachments.</p>
The James Hutton Institute is a Scottish charitable company limited by guarantee.
<br />
Registered in Scotland No. SC374831
<br />
Registered Office: The James Hutton Institute, Invergowrie Dundee DD2 5DA.
<br />
Charity No. SC041796<p></p></body></html>