[Bioperl-l] [Fwd: Enquiry about Remoteblast.pm]

[Chris Fields]

On Mar 5, 2010, at 7:20 AM, Robert Bradbury wrote:

> On Fri, Mar 5, 2010 at 5:05 AM, Dave Messina <David.Messina at sbc.su.se>wrote:
> 
>> My apologies for jumping the gun on the email thing — that won't take
>> effect until June 1.
>> 
>> See full details here:
>> 
>> http://groups.google.com/group/bioperl-l/browse_thread/thread/979a35fb9e22e45d/e7c88e7f087ff42d
>> 
>> 
>> Looks like the problems with RemoteBlast (as Chris reported elsewhere in
>> this thread) is at NCBI's servers (and is probably temporary).
>> 
>> 
> I would not be at all surprised if any problems involving RemoteBlast were
> related to the recent changeovers to a Javascript requirement for all
> interfaces to NCBI databases (this took place around mid-February and I
> complained about this in a previous email to the BioPerl list).

Robert, according to Palani's recent response NCBI provided a perl script that worked, so I don't think it a Javascript issue.  My guess is a change in the returned page information that isn't caught by the current regex, a problem that has happened in the past.  I'll be looking into it today.

> I received a response back from Dr. Eric Sayers at NCBI on Feb. 26 that
> indicated that they were aware of the problem (involving a Javascript
> requirement) and indicated that NCBI developers were "investigating" ways to
> mitigate the problem.
> 
> I've looked briefly at the new Javascript code that one is required to run
> when using PubMed, etc. and it looks like they may have completely changed
> the external interfaces to NCBI databases -- so I'm not surprised if that
> broke some or all other external interfaces used by BioPerl (RemoteBlast,
> Eutils, etc.).  I'd suggest that you try to document the problems as best
> you can and submit them to the NCBI help desk (or info at ncbi.nlm.nih.gov).
> It may be worth noting that it took ~3 weeks for me to receive any response
> to my reports.

EUtilities works fine (both regular and SOAP); all regression tests are passing, so it's not affecting everything.

> Also note, that (a) to the best of my knowledge there has been no public
> discussion regarding these recent changes at NCBI; and (b) under the Jan.
> 21, 2009 Memorandum on Transparency and Open Government, and under the Dec
> 8, 2009 Open Government Directive, NCBI *should* be doing a better job
> working with its end users (and the taxpayers) -- and at least thus far,
> while NIH seems to be making an effort that doesn't seem to have filtered
> down to NCBI.
> 
> (For example, no open/public discussion regarding the email requirement for
> remote blasts...).
> 
> It is also worth noting that it should be possible to file FOI requests with
> NIH/NCBI to find out exactly what they are doing and why they are doing it.
> I haven't taken such steps yet but I have given consideration to doing so.
> 
> Robert

The email requirement has always been indicated, it was just never enforced.  B/c of increased spamming issues on the NCBI server they took up the initiative to require users provide an email address (and enforce it starting in June).   I just made a change to the BioPerl install that requests an email and bypasses Bio::DB::EUtilities tests if one is not provided, other tools will be following suit. 

I don't think there is anything insidious about this.  My guess is they will be using them merely to track server usage per user and IP, and take necessary measures (i.e. contact or block) if needed.  

Finally, I'm not sure where the hostility is coming from.  NCBI has provided a great service to the community for many years, even through many funding cuts, and they have had quite a few.  Frankly, if one doesn't like their service requirements, there are other databases that one can use.  

chris