[MOBY-dev] Dead services and how to figure them out

Tue Nov 11 08:05:42 UTC 2008

Hi Mark and others,

I tend to have strong opinions, but utter them softly (meaning both  
"hard to hear" and "gently" ;-)

I strongly lean towards keeping QoS separate from the core registry,  
many arguments have been given. But colocalization would make sense.  
We have been discussing failure redundancy for Moby Central, and a  
mirror plus QoS on the same server would make sense. If a client has  
no access to Central data, it doesn't need QoS. On the other hand, the  
central Central used for registration i.e. writing to the registry  
doesn't need QoS. That allows us to keep provider-provided metadata  
(like "this is the authoritative service for x" or "this is a mirror/ 
backup service for x") separate from real, externally measured QoS.

I actually might have a student that could put QoS awareness into our  
Jabba aggregator, which would make a lot of sense.

Greetings from Cologne in pouring rain,

HEiko

On 10.11.2008, at 20:02, Mark Wilkinson wrote:

> Hi Sergio!
>
> it looks like we're back to the recurring question:  what metadata  
> belongs IN the registry, and what metadata belongs elsewhere...
>
>    <<sigh>>
>
> I tend to have the opinion that only metadata necessary for  
> discovery of a service should be in the registry itself, and that  
> metadata about QoS, uptime, downtime, versioning, etc. all belongs  
> somewhere else...  by "somewhere else" I mean (a) in the hands of  
> the service provider themselves, or (b) in a third-party metadata  
> repository.  This was one of the primary driving motivations behind  
> our use of LSIDs in BioMoby, since the LSID allowed us to provide  
> this non-registry metadata through a straightforward, predictable  
> API that was independent of the registry API.  That way, we didn't  
> have to modify the registry or the registry API every time we had a  
> new kind of metadata!  I *still* believe this is the right choice...  
> though I understand that sometimes efficiency of query is a  
> compelling reason to have all of that metadata in the same place.   
> Nevertheless, I don't think it should be a function of the registry  
> to decide which services it will and will not show you!  The  
> registry should simply tell you everything it knows, and it should  
> be a client-side decision which of those services it displays to the  
> end-user (IMO).  As such, I don't necessarily see the motivation for  
> having this kind of metadata as part of the registry API...
>
> How do others feel about this?  Is there *anyone* (other than  
> Gbrowse Moby) who actually uses the LSID resolver at MOBY Central to  
> retrieve this third-party metadata?  If not, is there a reason why  
> not?
>
> Martin, perhaps you can weigh-in on this question, as you had once- 
> upon-a-time worked on the NetNanny project that would have monitored  
> QoS... do you have a strong opinion one way or the other about where  
> this metadata should live?  (LOL!  Asking Martin if he has a "strong  
> opinion" is perhaps redundant ;-)  LOL!)
>
> Best wishes all!
>
> Mark
>
>
>
>
>
>
>
>
> On Mon, 10 Nov 2008 10:37:25 -0800, Sergio Ramirez Ramirez <srramirez at uma.es 
> > wrote:
>
>> Thanks Mark for your answer.
>>
>> I respond "between lines":
>>
>> Mark Wilkinson wrote:
>>> On Mon, 10 Nov 2008 08:54:32 -0800, Sergio Ramirez Ramirez <srramirez at uma.es 
>>> > wrote:
>>>
>>>
>>>>  * Timeout: If the time limit is got when trying to connect to the
>>>>      service.
>>>>    * Disconnected: The connection with the service couldn't be  
>>>> established.
>>>
>>> What's the difference between these two?
>>>
>> Really disconnected is more general than Timeout. It can be used  
>> when is not possible to connect with the service but not exactly  
>> due a timeout.
>>>
>>>
>>>> As you can see Eddie, the proposal is compatible with the one you  
>>>> mention. The only difference is the way to query the status of  
>>>> the service, that know can  be accessible using the functions of  
>>>> the MOBY Central..
>>>
>>>
>>> Thanks Sergio!  As usual, the INB has put together a very clear  
>>> and thought-out proposal, and it is perfectly consistent with what  
>>> we had in mind!  fantastic!
>>>
>>> If I understand correctly, you currently keep this kind of  
>>> metadata in the INB extended MOBY Central, right?  How tightly  
>>> integrated is it with MOBY Central?  (i.e. have you changed the  
>>> MOBY Central API such that you query for this metadata inside of  
>>> the functions defined by the 'official' API, or is this data  
>>> accessible via new API functions?)
>> The idea is to make the information of the test done accessible  
>> thought the API. For that  we need to include the status and the  
>> last test done as new information of the registry. Related with the  
>> functions, the proposal was to extend the functions to recover the  
>> list of services so they returns also information about the status.  
>> Also was proposed that those services that don't pass the test  
>> weren't returned in the list by default so the users always got a  
>> list of working services
>>>
>>> We (my lab) are similarly interested in QoS issues, and where to  
>>> store this kind of QoSD metadata.  We're looking at repositories  
>>> such as BioCatalogue as a possibile store, since we don't think  
>>> that this is the kind of metadata that the registry itself should  
>>> be in control of (in fact, the registry may not even be capable of  
>>> knowing!)
>>> Do you or your colleagues at INB have strong opinions about where  
>>> to store this metadata, and what the API should be to query for it?
>> In my personal opinion the information have to be stored in  
>> accesible way (web-service, xml or similar) so can be handle  
>> programmability. The real format and protocol used for stored them  
>> is not the most important thing.
>>>
>>> best wishes!
>>>
>>> Mark
>>>
>>> _______________________________________________
>>> MOBY-dev mailing list
>>> MOBY-dev at lists.open-bio.org
>>> http://lists.open-bio.org/mailman/listinfo/moby-dev
>>>
>>
>>
>
>
>
> -- 
> Mark D Wilkinson, PI Bioinformatics
> Assistant Professor, Medical Genetics
> The James Hogg iCAPTURE Centre for Cardiovascular and Pulmonary  
> Research
> Providence Heart + Lung Institute
> University of British Columbia - St. Paul's Hospital
> Vancouver, BC, Canada
> _______________________________________________
> MOBY-dev mailing list
> MOBY-dev at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/moby-dev