[MOBY-dev] The Agent behaviour - discussion and questions to the MOBY community

Sat Aug 5 15:13:03 UTC 2006

Dear all,

Here is some feedback from the BioMOBY BOF at Fortaleza.  
Unfortunately Martin and I were the only Mobifiers present, but we  
had a good discussion and brainstorm session about these recent  
developments.

We feel the RDF Agent is a good tool to protect services from  
deregistration by third parties. Furthermore if it is possible to  
call the Agent to pay a visit immediately it will be just as  
convenient for developers to use the old way of service registration  
as it is to use the new way with a SignatureURL. Mark wrote: "MOBY  
Central is currently configured to trigger an Agent visit if you call  
the registerService method with your SignatureURL as the only  
parameter.  This allows you to *immediately* deregister or update a  
service". We were wondering though how this works if you have  
multiple services described in one RDF document. Can I make one call  
with only a SignatureURL and have all of them checked (updated/ 
deleted/appended)?

We are glad that registering a service with a SignatureURL is  
optional for now. At least for the near furture, as this  
functionality is not yet "production ready" and only partially  
documented. Therefore it is currently quite difficult for people like  
us who are running additional registries to implement this way of  
service registration. Personally I'm looking forward to test this  
once I get back!  And I'll try to contribute to the documentation  
where I can. (Unfortunately, the network at the conference is not  
very stable and way too slow for any serious networking.)

We also feel the need to have some functionality for cleaning  
registries, but we think the RDF agent is not suitable for this task.  
Using a SignatureURL prevents third parties from deleting or altering  
service registrations, but there is no way the RDF agent can figure  
out whether a service is "up" and whether it is correctly registered.  
You can put an RDF document online and register a service that even  
never existed. Furthermore it is possible to make mistakes in the RDF  
document and list incorrect inputs and/or outputs. So with or without  
RDF document there is still no way a client can figure out if the  
information from a registry is up-to-date.

Summarising, using SignatureURLs and the RDF Agent makes (de-) 
registration more secure, but it doesn't solve the issue of pollution  
of registries due to dead or mis-registered services.

Registry pollution seriously hampers service discovery and spoils the  
fun. The recently discussed BioMOBY "ping" would be a good candidate  
to check whether a service is up. I think we already agreed that the  
ping request will be an empty request. Hence, a ping request has no  
query alias mobyData tag and there is an empty mobyContent tag:

<moby:MOBY xmlns:moby='http://www.biomoby.org/moby'
               xmlns='http://www.biomoby.org/moby'>
	<moby:mobyContent moby:authority='www.myauthority.org' />
</moby:MOBY>

There was still some discussion though about what the ping response  
should look like. We suggest this will be the same as the input with  
the option to append serviceNotes. In the serviceNotes section we  
could optionally use exceptionCode 700 to signal everything is Ok,  
but just as with normal service invocation serviceNotes remain  
optional. So a minimal ping response would be something like this:

<moby:MOBY xmlns:moby='http://www.biomoby.org/moby'
               xmlns='http://www.biomoby.org/moby'>
	<moby:mobyContent moby:authority='www.myauthority.org' />
</moby:MOBY>

With serviceNotes it would look like this:

<moby:MOBY xmlns:moby='http://www.biomoby.org/moby'
               xmlns='http://www.biomoby.org/moby'>
	<moby:mobyContent moby:authority='www.myauthority.org'>
		<moby:serviceNotes>
			<moby:Notes>
				Some human readable information about this service...
			</moby:Notes>
			<moby:mobyException severity ="information">
				<moby:exceptionCode>700</moby:exceptionCode>
				<moby:exceptionMessage>Service is up</moby:exceptionMessage>
			<moby:/mobyException>
		</moby:serviceNotes>
	</moby:mobyContent>
</moby:MOBY>

The question that remains is who will use the ping? We think both a  
registry and the clients could take advantage of the ping. The  
registry could have a Ping Agent in addition to the RDF Agent to  
check whether services are up. And it would be fine if this Ping  
Agent removes dead services after a certain amount of unsuccessful  
attempts. Furthermore to get the most up-to-date information a client  
could use a ping to check whether services are up. Having an option  
to hide services which are down would make more sense to us compared  
to hiding services without a SignatureURL as a SignatureURL doesn't  
tell us anything about service availability nor about service  
quality. The SignatureURL only tells you something about registration  
security, which is probably not very useful for most clients.

Would this make everybody happy?

Cheers from Fortaleza,

Martin and Pieter

On 4-Aug-2006, at 1:50 PM, Mark Wilkinson wrote:

> Issue #1:
> -------------
> Well, along with the rest of you, I got an inbox full of threatening
> emails from the agent yesterday.  Ugh!  Sorry about that!
>
> So, the first-try of the agent was possibly a bit of a disaster w.r.t.
> public relations, but we did learn from it, and we're re-coding it now
> to have a less annoying behaviour!
>
> History:  The primary purpose of the agent is to enable a distributed
> and safe way to add/update/remove your services from the registry.  At
> present, services registered without a SignatureURL can be removed by
> anyone, which is quite a dangerous situation.  For this reason, we
> encourage people to now download their Service Signature RDF from the
> page that Eddie has set up at
> http://mobycentral.icapture.ubc.ca/servlets/forms/getSignatureForm and
> save the output to whatever URL you enter in the Signature URL box.
> This will protect your service from deregistration by third- 
> parties.  If
> you need to deregister your service, simply remove its description  
> from
> the RDF document, and the agent will remove it from the registry the
> next time it crawls.  [[More about this in Issue #2 below...]]
>
> Another behaviour of the agent is its (configurable) ability to
> "cleanse" the registry of any service that do not have a SignatureURL.
> The assumption was that "dead" services would likely not have
> SignatureURLs, and that any service that was intended to be  
> production-
> quality would have been registered with a SignatureURL so those  
> without
> could/should be removed.  This became a part of the draft policy
> statement for my instance of MOBY Central
> (http://mobycentral.icapture.ubc.ca/MOBYCentral_Policy.html).
>
> I've had complaints about this policy from several of the key MOBY
> partners, and they have convinced me to loosen this policy to be less
> "tyrannical", and in the process we have come up with an alternative
> proposal that I wish to put forward to the community.
>
> My concern with the way people use the main public registry has been
> that people tend to leave their "trash" lying around in there for, in
> some cases, years!  This clutters-up the search results of other  
> people
> and is generally quite a nuisance.  However, if we agree that services
> with a SignatureURL are "production quality" and that services  
> without a
> SignatureURL are "test", then it is still possible to filter-out these
> "test" services at the client-level, while allowing them to persist in
> the registry for people to use and experiment with until they are  
> ready
> to re-register them with a SignatureURL.
>
> i.e. the new policy proposal is that services without a  
> SignatureURL can
> remain in the main public registry; however it is up to the client
> whether or not these services are displayed in the search results.
>
> Does this sound more reasonable to everyone?
>
> If so, I will update the policy statement, and we can make a note of
> this "convention" in the API documentation as well.
>
> ===============
>
> Issue #2:  Immediate Deregistration/Updating
>
> The MOBY Central code does not allow you to deregister a service  
> with a
> Signature URL, nor can you update it.  This can be very annoying, I
> know!!  It's not sufficient to just wait for the agent to visit during
> the night... there must be a better way!
>
> It is undocumented because we're still experimenting to see if  
> there are
> any obvious down-sides, but MOBY Central is currently configured to
> trigger an Agent visit if you call the registerService method with  
> your
> SignatureURL as the only parameter.  This allows you to *immediately*
> deregister or update a service (based on a local edit of your  
> Signature
> RDF document) rather than waiting for the next agent crawl.
>
> I invite all developers to try this and let us know if this seems  
> like a
> sensible behaviour.  There may be better ways to accomplish this,  
> so if
> you have other ideas, or if you see any potential security-holes in  
> this
> approach, please let us know.
>
> ============
>
> Cheers all!
>
> Mark
>
> -- 
> Mark Wilkinson
> Asst. Professor, Dept. of Medical Genetics
> University of British Columbia
> PI in Bioinformatics, iCAPTURE Centre
> St. Paul's Hospital, Rm. 166, 1081 Burrard St.
> Vancouver, BC, V6Z 1Y6
> tel: 604 682 2344 x62129
> fax: 604 806 9274
>
> "Since the point of a definition is to explain the meaning of a  
> term to
>    someone who is unfamiliar with its proper application, the use of
> language that doesn't help such a person learn how to apply the  
> term is
>  pointless. Thus, "happiness is a warm puppy" may be a lovely thought,
>                      but it is a lousy definition."
>                                                                  
> Köhler et al, 2006
>
> _______________________________________________
> MOBY-dev mailing list
> MOBY-dev at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/moby-dev

Wageningen University and Research centre (WUR)
Laboratory of Bioinformatics
Transitorium (building 312) room 1034
Dreijenlaan 3
6703 HA Wageningen
The Netherlands
phone: 0317-483 060
fax: 0317-483 584
mobile: 06-143 66 783
pieter.neerincx at wur.nl