From senger at ebi.ac.uk Wed Jun 8 11:19:24 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed Jun 8 11:11:58 2005 Subject: [MOBY-dev] Notes from working group 'B' In-Reply-To: <429B1C9B.2060105@illuminae.com> Message-ID: I am sorry that I am late in this discussion but I was away for more than a week. Also I am reading from the back - so I may be barking on alredy non-existing tree... > > Yes! That would make life much easier. I am currently testing some > > stuff where I have an async service that has an additional help call. > > This service is a wrapper for a commandline tool. If you call the > > service with an empty Moby object it returns an object > > "ServiceOptions" which HASA String that contains the info which you > > would normally get form --help on the commandline along with some info > > about which databases are available and when they were updated. > > Putting this kind of info in the service signature would make more > > sense though. > I agree that such stuff belongs to metadata. Because if your service returns different output data for different input values (e.g. ServiceOptions for an empty input) - you will have trouble to make such service into workflows. This is why Moby is good: it limits you so the services can be more interoperable. Let's keep it like that. Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Wed Jun 8 11:55:03 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed Jun 8 11:46:55 2005 Subject: [moby] Re: [MOBY-dev] Notes from working group 'B' In-Reply-To: <1117467545.29183.35.camel@mobycentral.mrl.ubc.ca> Message-ID: > At the moment, and under the proposed API, there is no defined way for a > client to interact with the same (replicated) service when it calls an > async service as it does when it retrieves the result. > And I think that's okay. The method 2,3,4 in the async invocation must be treated by clients as one "atomic" operation. If any of them fails, client can start again using a new (replicated) URL. It, of course, does not preclude some low-level-grid-enabled implementations that shared resources between replicated services - and that are able to respond by another service instance to a request originally sent to a different service instance. But that would be rather an exceptional case I guess. BTW, I have still on my dest an action from the meeting to send a proposal how to do this async invocation (so it was me not the Spanich group who promised to do it). But it seems now that Mark put already everything on the table - thanks, Mark :-). Anyway, I will try to summarize it once again in one of the following emails. > multiple URLs for the same (authURI,serviceName) combination, then the > combination of (authURI,serviceName,URL) will uniquely identify a > service > This is an interesting point because it opens an issue which is not fully (as far as I can tell) explained in the Moby API: Does the service URL fully identify a service? Obviously not - I can see several services with the same URL - and the server engine "adds" the real service name to this URL. But is this a correct behaviour? Surely it works for Perl services - but for Java/Axis services (I think) we always add service name to the URL. I guess it works with Perl because SOAP::Lite takes the service name from the SOAPAction header. But I think that it should not... Well, whatever is right, it would be worth to discuss it here (or to point me to an existing explanation in the Moby API) - and then to explicitly tell in the API (a) what should be in the URL/endpoint, (b) what should be in the SOAPAction header, (c) what should be in the URI (this one is already fully documented in the Moby API). Regards, Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Wed Jun 8 12:03:28 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed Jun 8 11:55:18 2005 Subject: [MOBY-dev] Notes from working group 'B' In-Reply-To: <1117220757.30771.43.camel@mobycentral.mrl.ubc.ca> Message-ID: > I???m sure there are things I have missed??? please make your additions and > comments to the list. > I think that, regarding LSID metadata, we talked also about a need to define our small vocabulary (well, an ontology) for various basic predicates that would be used in the services metadata (QoS, sample input, asynchonocity etc.). It would be helpful to publish this vocabulary as part of the Moby API (as an optional part). Any idea who is going to prepare the first draft? Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Wed Jun 8 12:03:28 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed Jun 8 11:55:19 2005 Subject: [MOBY-dev] Notes from working group 'B' In-Reply-To: <1117220757.30771.43.camel@mobycentral.mrl.ubc.ca> Message-ID: > I???m sure there are things I have missed??? please make your additions and > comments to the list. > I think that, regarding LSID metadata, we talked also about a need to define our small vocabulary (well, an ontology) for various basic predicates that would be used in the services metadata (QoS, sample input, asynchonocity etc.). It would be helpful to publish this vocabulary as part of the Moby API (as an optional part). Any idea who is going to prepare the first draft? Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Fri Jun 10 14:38:07 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri Jun 10 14:29:52 2005 Subject: [MOBY-dev] going offline for a couple of weeks Message-ID: <1118428687.27504.60.camel@mobycentral.mrl.ubc.ca> Hi all, There are Genome Canada face-to-face reviews coming up next week, followed by ISMB. I also have a meeting in Rome starting Tuesday. As such, I anticipate being "unavailable" as of this afternoon until ~June 25th. I just want to let the other developers know this so that you can quickly jump in and respond to questions that get posted to the list. We've got a pretty good reputation for rapid response :-) :-) Cheers all! See you on the other side! M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From senger at ebi.ac.uk Sat Jun 11 14:07:41 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Sat Jun 11 13:59:28 2005 Subject: [MOBY-dev] BioMOBY Asynchronous Services In-Reply-To: <42A856DE.6030400@cnb.uam.es> Message-ID: Here is my "action" from Vancouver... With best regards, and cheers, (and sorry for a long email), Martin Asynchronous BioMoby services ============================= In Vancouver, we have started a discussion about the ways how to call BioMoby services asynchronously. I promised to summarize the experiences we have been using in other Web Services (Soaplab etc.) for this task - and this email does it. Meanwhile, however, David Gonz?lez Pisano (and other people) from Spanish National Institute of Bioinformatics (INB) created a detailed proposal how to introduce asynchronicity into BioMoby, based on a different approach (more BioMoby specific). I have also been presented at a meeting (incidentally, in a completely different, non-Moby related meeting at EBI) where this proposal was explained. Therefore, I can make some rudimentary comparison of the two proposed methods. But first, what they are? (I skip the introduction why we need an asynchronous behavior - this is already very well introduced in David's proposal). Almost any asynchronous invocations consists of: 1) Staring request: a) A client sends a request, (somehow) specifying that this request should be treated asynchronously. b) A service responds with an identifier assigned to this request (called also 'job handler' or 'ticket'). 2) A client uses this identifier to send a polling request. A service responds with a status indication. Polling requests are being sent until status indicates that job had finished. 3) Getting result request. Using still the same identifier, a client ask for a result and service provides it. (Sometimes this kind of request can be part of the last polling request from the previous step.) The INB's proposal achieves the above by adding specific attributes into the existing mobyData XML tag, and letting the service return an empty result until a real result is available. My proposal suggests to have several methods (whose names are derived by fixed suffixes), each of them would return a specific result (still defined as a Moby object), and one of them would return a real result. Both proposals are similar (almost identical) in their features, they differ only in a way how these new features are coded into used protocol. So how do they differ? 1) I must start with the one that INB's proposal claims as their advantage. The proposal says that is "able to support asynchronous calls without changing the underlying software". I do not buy it, however. I think that *both* proposals would need to change clients if they want to take advantage of the new features, and to change services in order to provide new features. But also, in *both* proposals, there is no need to change anything if you still use a synchronous call. The old clients will not break, the old services will continue to work. So this does not constitute any difference between the two proposals (IMHO, of course). 2) A real difference - and this one is in favor of the INB's proposal - is that specifying asynchronicity by attributes of the mobyData tag allows to ask for different treatment of individual mobyData parts. A client can sent two mobyData tags in one request, asking for the first one to be treated synchronously and the second one asynchronously. This cannot be achieved by having several method names. 3) Another real difference - this time in favor of my proposal - is related to the existing standards. BioMoby uses very proprietary way how it defines its data types and from that how its input and output data look like. But for the transport messages the BioMoby's behavior is so far quite standard. The same service always transforms data A into data B. And I think that this is a good thing. Introducing new attributes but keeping the same method name means that a service would return different things each time. But if we have fixed method names - we again know that always a service/method1 will transform data A into data C, and a service/method2 will transform data C into data B. The reason why I take this as an advantage is that there is a lot of general tools for Web services that could be still used for BioMoby services if we keep the behavior standard. Having more methods is perfectly easy to define in WSDL (that can be still a bit less useful for BioMoby data types - because they are by default non-standard - but in the same time good enough for many other tools). I would feel a bit uncomfortable to invent a proprietary solution if there is a standard one. Unless, of course, you feel that the point ad 2) above is significantly important that it justifies a proprietary solution. 4) A (perhaps slightly disputable) difference is about the returned objects from the starting and polling requests. The INB's proposal does not return any real object because the status is in a mobyData attributes. My proposal defines several real BioMoby data types (BioMoby objects) that can carry more complex information that just attributes. They can even be extended by inheritance like any other BioMoby object. The only difference comparing to other BioMoby data types (objects) is that you will never discover any service returning them - because a service provider does not register his/her service with such output types. This allows, btw, to use the same objects for a complex notification (if we have anything like that in the future) - see few comments on it in my proposal. 5) A small difference - in favor of the INB's proposal - is that their proposal allows to save one network request because the last polling request can become a request returning a real result. And that's it - I have not found any other differences (so far). Still, it may be useful to summarize what *neither* of two proposals solves: 1) A client does not know *in advance* whether a service supports asynchonicity or not - without some investigation. BioMoby strategy is to put/find this information in a service metadata using LSID resolution. I would say that such information is so crucial that it should be accessible easier - but allowing to have it in the service registration object, so clients can find/get it directly together with the service URL when they ask the registry. 2) A client must treat the full sequence of corresponding requests (starting, polling and resulting requests) as an "atomic" operation - which means that if any of these requests fails, the client needs to start again from the beginning. This solves the problem with having same services replicated but not always being able to take over each other in the middle of the job. 3) Introducing asynchonicity opens doors to the longer persistence of results. It is easy to imagine that once you have a job identifier (ticket, or whatever) you theoretically can ask for the same result several time without creating it again from the beginning. Of course, if a service supports it. And also a client could have an option of say "please remove the result, I am not anymore interested in it" (again, here both proposals can accommodate this feature either by having an additional (cleaning, reseting) attribute or a additional method). But BioMoby needs to document (or reject) such persistent behavior. Well, perhaps now I should say what is my proposal :-) a) It is based on an OMG standard "LSAE - Life Sciences Analysis Engine" (http://www.omg.org/cgi-bin/doc?dtc/05-04-01 and http://www.omg.org/cgi-bin/doc?dtc/05-04-08). b) If we have a service ABC and its service provider wishes to support asynchronous calls then the service will have the following methods (with the following input/output data types): RealResult ABC (RealInput input) This is a standard synchronous invocation, and the RealResult and RealInput are normal BioMoby data types. The service ABC is registered with them as input and output. JobHandler ABC_async (RealInput input) This is a "starting request". The JobHandler contains a ticket (job identifier) but may contain also other things (like a suggestion how often is worth to poll for results). This method call fails if the service does not support an asynchronous behavior. This does not break the old clients because they would never use this method. NotificationEvent ABC_status (JobHandler handler) This is a "polling request". NotificationEvent contains status of the invoked job (it can failed if the handler is not knows or expired). The ways how it represents a status can be different (from just a status, to a status including progress where progress can be simple hear-beating progress, percent progress, step progress or time progress). For a feel what it can contain look please at the picture taken from the LSAE model: http://www.ebi.ac.uk/~senger/LSAE-pictures.html. I would convert them into a proper BioMoby data types (objects) if/when is clearer that this proposal has chance to be accepted. RealResult ABC_result (JobHandler handler) This is a "result request" returning a real result. It can fail if the handler does not exist or if the method is called in an unappropriated moment/order. We can add also a method ABC_clean (JobHandler handler) if we decide to support persistence. -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Mon Jun 13 11:41:50 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon Jun 13 11:33:33 2005 Subject: [MOBY-dev] MOBY-DIC action item that has gone astray... Message-ID: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> Hi all, Who was it that volunteered to look up the use of LDAP or DNS-like synchronization? (please don't say it was me :-) ) M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From jmfernandez at cnb.uam.es Mon Jun 13 14:28:44 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez?=) Date: Mon Jun 13 14:27:56 2005 Subject: [MOBY-dev] MOBY-DIC action item that has gone astray... In-Reply-To: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> References: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> Message-ID: <42ADD05C.8090107@cnb.uam.es> Hi everybody, I think Heiko told he was going to ask some network managers he was working with (sorry, Heiko!). Best regards, Jos? Mar?a Mark Wilkinson wrote: > Hi all, > > Who was it that volunteered to look up the use of LDAP or DNS-like > synchronization? (please don't say it was me :-) ) > > M > > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez@cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From markw at illuminae.com Mon Jun 13 14:41:54 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon Jun 13 14:33:34 2005 Subject: [unclassified] Re: [MOBY-dev] MOBY-DIC action item that has gone astray... In-Reply-To: <42ADD05C.8090107@cnb.uam.es> References: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> <42ADD05C.8090107@cnb.uam.es> Message-ID: <1118688114.554.129.camel@mobycentral.mrl.ubc.ca> Ha! And he thought he was going to get away with hiding at the MPI ;-) Caught ya!!! M On Mon, 2005-06-13 at 20:28 +0200, Jos? Mar?a Fern?ndez wrote: > Hi everybody, > I think Heiko told he was going to ask some network managers he was > working with (sorry, Heiko!). > > Best regards, > Jos? Mar?a > > Mark Wilkinson wrote: > > Hi all, > > > > Who was it that volunteered to look up the use of LDAP or DNS-like > > synchronization? (please don't say it was me :-) ) > > > > M > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Wed Jun 29 11:25:05 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed Jun 29 11:17:53 2005 Subject: [MOBY-dev] MOBY Central hacked last night Message-ID: <1120058705.20181.26.camel@mobycentral.mrl.ubc.ca> Hi all, MOBY Central is down for at least a day due to a nasty hack last night. We may have to rebuild from scratch, so it may not be a rapid recovery. Speaks again to the need for a mirroring system! Hey, Heiko... how's the research going? ;-) ;-) You can run, but you can't hide! Sorry for the down-time... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Thu Jun 30 19:38:45 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Thu Jun 30 19:29:57 2005 Subject: [MOBY-dev] MOBY Central Registry up and running at a new location Message-ID: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> Hi all, The core of MOBY Central is up and running again at: URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl URI: http://mobycentral.icapture.ubc.ca/MOBY/Central No accessories are functioning, however. Just the registry. Also, none of my services will be working either. No data loss as far as I can tell at the moment, so that is good news! I will begin migrating the accessories over to this new server tomorrow. Hopefully we will have things running again by the end of the weekend, but for the moment at least the registry itself is functional. remember you can use this registry as your default for the Perl libraries by setting the environment variables: MOBY_SERVER= $URL (above) MOBY_URI = $URI (above) This will cause MOBY::Client::Central to connect to this server by default. I will make changes to the test harness in the CVS soon. It doesn't matter, really, because the version in the CVS is non-functional until we can get the Perl LSID resolver stack working }-/ Anyway, sorry for the down-time! What a disaster! (but I did learn a lot about what it takes to install MOBY Central now! So I have lots of good notes for our documentation drive...) M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From senger at ebi.ac.uk Wed Jun 8 11:19:24 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 8 Jun 2005 16:19:24 +0100 (BST) Subject: [MOBY-dev] Notes from working group 'B' In-Reply-To: <429B1C9B.2060105@illuminae.com> Message-ID: I am sorry that I am late in this discussion but I was away for more than a week. Also I am reading from the back - so I may be barking on alredy non-existing tree... > > Yes! That would make life much easier. I am currently testing some > > stuff where I have an async service that has an additional help call. > > This service is a wrapper for a commandline tool. If you call the > > service with an empty Moby object it returns an object > > "ServiceOptions" which HASA String that contains the info which you > > would normally get form --help on the commandline along with some info > > about which databases are available and when they were updated. > > Putting this kind of info in the service signature would make more > > sense though. > I agree that such stuff belongs to metadata. Because if your service returns different output data for different input values (e.g. ServiceOptions for an empty input) - you will have trouble to make such service into workflows. This is why Moby is good: it limits you so the services can be more interoperable. Let's keep it like that. Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Wed Jun 8 11:55:03 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 8 Jun 2005 16:55:03 +0100 (BST) Subject: [moby] Re: [MOBY-dev] Notes from working group 'B' In-Reply-To: <1117467545.29183.35.camel@mobycentral.mrl.ubc.ca> Message-ID: > At the moment, and under the proposed API, there is no defined way for a > client to interact with the same (replicated) service when it calls an > async service as it does when it retrieves the result. > And I think that's okay. The method 2,3,4 in the async invocation must be treated by clients as one "atomic" operation. If any of them fails, client can start again using a new (replicated) URL. It, of course, does not preclude some low-level-grid-enabled implementations that shared resources between replicated services - and that are able to respond by another service instance to a request originally sent to a different service instance. But that would be rather an exceptional case I guess. BTW, I have still on my dest an action from the meeting to send a proposal how to do this async invocation (so it was me not the Spanich group who promised to do it). But it seems now that Mark put already everything on the table - thanks, Mark :-). Anyway, I will try to summarize it once again in one of the following emails. > multiple URLs for the same (authURI,serviceName) combination, then the > combination of (authURI,serviceName,URL) will uniquely identify a > service > This is an interesting point because it opens an issue which is not fully (as far as I can tell) explained in the Moby API: Does the service URL fully identify a service? Obviously not - I can see several services with the same URL - and the server engine "adds" the real service name to this URL. But is this a correct behaviour? Surely it works for Perl services - but for Java/Axis services (I think) we always add service name to the URL. I guess it works with Perl because SOAP::Lite takes the service name from the SOAPAction header. But I think that it should not... Well, whatever is right, it would be worth to discuss it here (or to point me to an existing explanation in the Moby API) - and then to explicitly tell in the API (a) what should be in the URL/endpoint, (b) what should be in the SOAPAction header, (c) what should be in the URI (this one is already fully documented in the Moby API). Regards, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Wed Jun 8 12:03:28 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 8 Jun 2005 17:03:28 +0100 (BST) Subject: [MOBY-dev] Notes from working group 'B' In-Reply-To: <1117220757.30771.43.camel@mobycentral.mrl.ubc.ca> Message-ID: > I???m sure there are things I have missed??? please make your additions and > comments to the list. > I think that, regarding LSID metadata, we talked also about a need to define our small vocabulary (well, an ontology) for various basic predicates that would be used in the services metadata (QoS, sample input, asynchonocity etc.). It would be helpful to publish this vocabulary as part of the Moby API (as an optional part). Any idea who is going to prepare the first draft? Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Wed Jun 8 12:03:28 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 8 Jun 2005 17:03:28 +0100 (BST) Subject: [MOBY-dev] Notes from working group 'B' In-Reply-To: <1117220757.30771.43.camel@mobycentral.mrl.ubc.ca> Message-ID: > I???m sure there are things I have missed??? please make your additions and > comments to the list. > I think that, regarding LSID metadata, we talked also about a need to define our small vocabulary (well, an ontology) for various basic predicates that would be used in the services metadata (QoS, sample input, asynchonocity etc.). It would be helpful to publish this vocabulary as part of the Moby API (as an optional part). Any idea who is going to prepare the first draft? Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Fri Jun 10 14:38:07 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 10 Jun 2005 11:38:07 -0700 Subject: [MOBY-dev] going offline for a couple of weeks Message-ID: <1118428687.27504.60.camel@mobycentral.mrl.ubc.ca> Hi all, There are Genome Canada face-to-face reviews coming up next week, followed by ISMB. I also have a meeting in Rome starting Tuesday. As such, I anticipate being "unavailable" as of this afternoon until ~June 25th. I just want to let the other developers know this so that you can quickly jump in and respond to questions that get posted to the list. We've got a pretty good reputation for rapid response :-) :-) Cheers all! See you on the other side! M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From senger at ebi.ac.uk Sat Jun 11 14:07:41 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Sat, 11 Jun 2005 19:07:41 +0100 (BST) Subject: [MOBY-dev] BioMOBY Asynchronous Services In-Reply-To: <42A856DE.6030400@cnb.uam.es> Message-ID: Here is my "action" from Vancouver... With best regards, and cheers, (and sorry for a long email), Martin Asynchronous BioMoby services ============================= In Vancouver, we have started a discussion about the ways how to call BioMoby services asynchronously. I promised to summarize the experiences we have been using in other Web Services (Soaplab etc.) for this task - and this email does it. Meanwhile, however, David Gonz?lez Pisano (and other people) from Spanish National Institute of Bioinformatics (INB) created a detailed proposal how to introduce asynchronicity into BioMoby, based on a different approach (more BioMoby specific). I have also been presented at a meeting (incidentally, in a completely different, non-Moby related meeting at EBI) where this proposal was explained. Therefore, I can make some rudimentary comparison of the two proposed methods. But first, what they are? (I skip the introduction why we need an asynchronous behavior - this is already very well introduced in David's proposal). Almost any asynchronous invocations consists of: 1) Staring request: a) A client sends a request, (somehow) specifying that this request should be treated asynchronously. b) A service responds with an identifier assigned to this request (called also 'job handler' or 'ticket'). 2) A client uses this identifier to send a polling request. A service responds with a status indication. Polling requests are being sent until status indicates that job had finished. 3) Getting result request. Using still the same identifier, a client ask for a result and service provides it. (Sometimes this kind of request can be part of the last polling request from the previous step.) The INB's proposal achieves the above by adding specific attributes into the existing mobyData XML tag, and letting the service return an empty result until a real result is available. My proposal suggests to have several methods (whose names are derived by fixed suffixes), each of them would return a specific result (still defined as a Moby object), and one of them would return a real result. Both proposals are similar (almost identical) in their features, they differ only in a way how these new features are coded into used protocol. So how do they differ? 1) I must start with the one that INB's proposal claims as their advantage. The proposal says that is "able to support asynchronous calls without changing the underlying software". I do not buy it, however. I think that *both* proposals would need to change clients if they want to take advantage of the new features, and to change services in order to provide new features. But also, in *both* proposals, there is no need to change anything if you still use a synchronous call. The old clients will not break, the old services will continue to work. So this does not constitute any difference between the two proposals (IMHO, of course). 2) A real difference - and this one is in favor of the INB's proposal - is that specifying asynchronicity by attributes of the mobyData tag allows to ask for different treatment of individual mobyData parts. A client can sent two mobyData tags in one request, asking for the first one to be treated synchronously and the second one asynchronously. This cannot be achieved by having several method names. 3) Another real difference - this time in favor of my proposal - is related to the existing standards. BioMoby uses very proprietary way how it defines its data types and from that how its input and output data look like. But for the transport messages the BioMoby's behavior is so far quite standard. The same service always transforms data A into data B. And I think that this is a good thing. Introducing new attributes but keeping the same method name means that a service would return different things each time. But if we have fixed method names - we again know that always a service/method1 will transform data A into data C, and a service/method2 will transform data C into data B. The reason why I take this as an advantage is that there is a lot of general tools for Web services that could be still used for BioMoby services if we keep the behavior standard. Having more methods is perfectly easy to define in WSDL (that can be still a bit less useful for BioMoby data types - because they are by default non-standard - but in the same time good enough for many other tools). I would feel a bit uncomfortable to invent a proprietary solution if there is a standard one. Unless, of course, you feel that the point ad 2) above is significantly important that it justifies a proprietary solution. 4) A (perhaps slightly disputable) difference is about the returned objects from the starting and polling requests. The INB's proposal does not return any real object because the status is in a mobyData attributes. My proposal defines several real BioMoby data types (BioMoby objects) that can carry more complex information that just attributes. They can even be extended by inheritance like any other BioMoby object. The only difference comparing to other BioMoby data types (objects) is that you will never discover any service returning them - because a service provider does not register his/her service with such output types. This allows, btw, to use the same objects for a complex notification (if we have anything like that in the future) - see few comments on it in my proposal. 5) A small difference - in favor of the INB's proposal - is that their proposal allows to save one network request because the last polling request can become a request returning a real result. And that's it - I have not found any other differences (so far). Still, it may be useful to summarize what *neither* of two proposals solves: 1) A client does not know *in advance* whether a service supports asynchonicity or not - without some investigation. BioMoby strategy is to put/find this information in a service metadata using LSID resolution. I would say that such information is so crucial that it should be accessible easier - but allowing to have it in the service registration object, so clients can find/get it directly together with the service URL when they ask the registry. 2) A client must treat the full sequence of corresponding requests (starting, polling and resulting requests) as an "atomic" operation - which means that if any of these requests fails, the client needs to start again from the beginning. This solves the problem with having same services replicated but not always being able to take over each other in the middle of the job. 3) Introducing asynchonicity opens doors to the longer persistence of results. It is easy to imagine that once you have a job identifier (ticket, or whatever) you theoretically can ask for the same result several time without creating it again from the beginning. Of course, if a service supports it. And also a client could have an option of say "please remove the result, I am not anymore interested in it" (again, here both proposals can accommodate this feature either by having an additional (cleaning, reseting) attribute or a additional method). But BioMoby needs to document (or reject) such persistent behavior. Well, perhaps now I should say what is my proposal :-) a) It is based on an OMG standard "LSAE - Life Sciences Analysis Engine" (http://www.omg.org/cgi-bin/doc?dtc/05-04-01 and http://www.omg.org/cgi-bin/doc?dtc/05-04-08). b) If we have a service ABC and its service provider wishes to support asynchronous calls then the service will have the following methods (with the following input/output data types): RealResult ABC (RealInput input) This is a standard synchronous invocation, and the RealResult and RealInput are normal BioMoby data types. The service ABC is registered with them as input and output. JobHandler ABC_async (RealInput input) This is a "starting request". The JobHandler contains a ticket (job identifier) but may contain also other things (like a suggestion how often is worth to poll for results). This method call fails if the service does not support an asynchronous behavior. This does not break the old clients because they would never use this method. NotificationEvent ABC_status (JobHandler handler) This is a "polling request". NotificationEvent contains status of the invoked job (it can failed if the handler is not knows or expired). The ways how it represents a status can be different (from just a status, to a status including progress where progress can be simple hear-beating progress, percent progress, step progress or time progress). For a feel what it can contain look please at the picture taken from the LSAE model: http://www.ebi.ac.uk/~senger/LSAE-pictures.html. I would convert them into a proper BioMoby data types (objects) if/when is clearer that this proposal has chance to be accepted. RealResult ABC_result (JobHandler handler) This is a "result request" returning a real result. It can fail if the handler does not exist or if the method is called in an unappropriated moment/order. We can add also a method ABC_clean (JobHandler handler) if we decide to support persistence. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Mon Jun 13 11:41:50 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 13 Jun 2005 08:41:50 -0700 Subject: [MOBY-dev] MOBY-DIC action item that has gone astray... Message-ID: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> Hi all, Who was it that volunteered to look up the use of LDAP or DNS-like synchronization? (please don't say it was me :-) ) M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From jmfernandez at cnb.uam.es Mon Jun 13 14:28:44 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez?=) Date: Mon, 13 Jun 2005 20:28:44 +0200 Subject: [MOBY-dev] MOBY-DIC action item that has gone astray... In-Reply-To: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> References: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> Message-ID: <42ADD05C.8090107@cnb.uam.es> Hi everybody, I think Heiko told he was going to ask some network managers he was working with (sorry, Heiko!). Best regards, Jos? Mar?a Mark Wilkinson wrote: > Hi all, > > Who was it that volunteered to look up the use of LDAP or DNS-like > synchronization? (please don't say it was me :-) ) > > M > > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From markw at illuminae.com Mon Jun 13 14:41:54 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 13 Jun 2005 11:41:54 -0700 Subject: [unclassified] Re: [MOBY-dev] MOBY-DIC action item that has gone astray... In-Reply-To: <42ADD05C.8090107@cnb.uam.es> References: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> <42ADD05C.8090107@cnb.uam.es> Message-ID: <1118688114.554.129.camel@mobycentral.mrl.ubc.ca> Ha! And he thought he was going to get away with hiding at the MPI ;-) Caught ya!!! M On Mon, 2005-06-13 at 20:28 +0200, Jos? Mar?a Fern?ndez wrote: > Hi everybody, > I think Heiko told he was going to ask some network managers he was > working with (sorry, Heiko!). > > Best regards, > Jos? Mar?a > > Mark Wilkinson wrote: > > Hi all, > > > > Who was it that volunteered to look up the use of LDAP or DNS-like > > synchronization? (please don't say it was me :-) ) > > > > M > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Wed Jun 29 11:25:05 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 29 Jun 2005 08:25:05 -0700 Subject: [MOBY-dev] MOBY Central hacked last night Message-ID: <1120058705.20181.26.camel@mobycentral.mrl.ubc.ca> Hi all, MOBY Central is down for at least a day due to a nasty hack last night. We may have to rebuild from scratch, so it may not be a rapid recovery. Speaks again to the need for a mirroring system! Hey, Heiko... how's the research going? ;-) ;-) You can run, but you can't hide! Sorry for the down-time... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Thu Jun 30 19:38:45 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Thu, 30 Jun 2005 16:38:45 -0700 Subject: [MOBY-dev] MOBY Central Registry up and running at a new location Message-ID: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> Hi all, The core of MOBY Central is up and running again at: URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl URI: http://mobycentral.icapture.ubc.ca/MOBY/Central No accessories are functioning, however. Just the registry. Also, none of my services will be working either. No data loss as far as I can tell at the moment, so that is good news! I will begin migrating the accessories over to this new server tomorrow. Hopefully we will have things running again by the end of the weekend, but for the moment at least the registry itself is functional. remember you can use this registry as your default for the Perl libraries by setting the environment variables: MOBY_SERVER= $URL (above) MOBY_URI = $URI (above) This will cause MOBY::Client::Central to connect to this server by default. I will make changes to the test harness in the CVS soon. It doesn't matter, really, because the version in the CVS is non-functional until we can get the Perl LSID resolver stack working }-/ Anyway, sorry for the down-time! What a disaster! (but I did learn a lot about what it takes to install MOBY Central now! So I have lots of good notes for our documentation drive...) M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From senger at ebi.ac.uk Wed Jun 8 15:19:24 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 8 Jun 2005 16:19:24 +0100 (BST) Subject: [MOBY-dev] Notes from working group 'B' In-Reply-To: <429B1C9B.2060105@illuminae.com> Message-ID: I am sorry that I am late in this discussion but I was away for more than a week. Also I am reading from the back - so I may be barking on alredy non-existing tree... > > Yes! That would make life much easier. I am currently testing some > > stuff where I have an async service that has an additional help call. > > This service is a wrapper for a commandline tool. If you call the > > service with an empty Moby object it returns an object > > "ServiceOptions" which HASA String that contains the info which you > > would normally get form --help on the commandline along with some info > > about which databases are available and when they were updated. > > Putting this kind of info in the service signature would make more > > sense though. > I agree that such stuff belongs to metadata. Because if your service returns different output data for different input values (e.g. ServiceOptions for an empty input) - you will have trouble to make such service into workflows. This is why Moby is good: it limits you so the services can be more interoperable. Let's keep it like that. Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Wed Jun 8 15:55:03 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 8 Jun 2005 16:55:03 +0100 (BST) Subject: [moby] Re: [MOBY-dev] Notes from working group 'B' In-Reply-To: <1117467545.29183.35.camel@mobycentral.mrl.ubc.ca> Message-ID: > At the moment, and under the proposed API, there is no defined way for a > client to interact with the same (replicated) service when it calls an > async service as it does when it retrieves the result. > And I think that's okay. The method 2,3,4 in the async invocation must be treated by clients as one "atomic" operation. If any of them fails, client can start again using a new (replicated) URL. It, of course, does not preclude some low-level-grid-enabled implementations that shared resources between replicated services - and that are able to respond by another service instance to a request originally sent to a different service instance. But that would be rather an exceptional case I guess. BTW, I have still on my dest an action from the meeting to send a proposal how to do this async invocation (so it was me not the Spanich group who promised to do it). But it seems now that Mark put already everything on the table - thanks, Mark :-). Anyway, I will try to summarize it once again in one of the following emails. > multiple URLs for the same (authURI,serviceName) combination, then the > combination of (authURI,serviceName,URL) will uniquely identify a > service > This is an interesting point because it opens an issue which is not fully (as far as I can tell) explained in the Moby API: Does the service URL fully identify a service? Obviously not - I can see several services with the same URL - and the server engine "adds" the real service name to this URL. But is this a correct behaviour? Surely it works for Perl services - but for Java/Axis services (I think) we always add service name to the URL. I guess it works with Perl because SOAP::Lite takes the service name from the SOAPAction header. But I think that it should not... Well, whatever is right, it would be worth to discuss it here (or to point me to an existing explanation in the Moby API) - and then to explicitly tell in the API (a) what should be in the URL/endpoint, (b) what should be in the SOAPAction header, (c) what should be in the URI (this one is already fully documented in the Moby API). Regards, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Wed Jun 8 16:03:28 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 8 Jun 2005 17:03:28 +0100 (BST) Subject: [MOBY-dev] Notes from working group 'B' In-Reply-To: <1117220757.30771.43.camel@mobycentral.mrl.ubc.ca> Message-ID: > I???m sure there are things I have missed??? please make your additions and > comments to the list. > I think that, regarding LSID metadata, we talked also about a need to define our small vocabulary (well, an ontology) for various basic predicates that would be used in the services metadata (QoS, sample input, asynchonocity etc.). It would be helpful to publish this vocabulary as part of the Moby API (as an optional part). Any idea who is going to prepare the first draft? Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Wed Jun 8 16:03:28 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 8 Jun 2005 17:03:28 +0100 (BST) Subject: [MOBY-dev] Notes from working group 'B' In-Reply-To: <1117220757.30771.43.camel@mobycentral.mrl.ubc.ca> Message-ID: > I???m sure there are things I have missed??? please make your additions and > comments to the list. > I think that, regarding LSID metadata, we talked also about a need to define our small vocabulary (well, an ontology) for various basic predicates that would be used in the services metadata (QoS, sample input, asynchonocity etc.). It would be helpful to publish this vocabulary as part of the Moby API (as an optional part). Any idea who is going to prepare the first draft? Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Fri Jun 10 18:38:07 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 10 Jun 2005 11:38:07 -0700 Subject: [MOBY-dev] going offline for a couple of weeks Message-ID: <1118428687.27504.60.camel@mobycentral.mrl.ubc.ca> Hi all, There are Genome Canada face-to-face reviews coming up next week, followed by ISMB. I also have a meeting in Rome starting Tuesday. As such, I anticipate being "unavailable" as of this afternoon until ~June 25th. I just want to let the other developers know this so that you can quickly jump in and respond to questions that get posted to the list. We've got a pretty good reputation for rapid response :-) :-) Cheers all! See you on the other side! M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From senger at ebi.ac.uk Sat Jun 11 18:07:41 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Sat, 11 Jun 2005 19:07:41 +0100 (BST) Subject: [MOBY-dev] BioMOBY Asynchronous Services In-Reply-To: <42A856DE.6030400@cnb.uam.es> Message-ID: Here is my "action" from Vancouver... With best regards, and cheers, (and sorry for a long email), Martin Asynchronous BioMoby services ============================= In Vancouver, we have started a discussion about the ways how to call BioMoby services asynchronously. I promised to summarize the experiences we have been using in other Web Services (Soaplab etc.) for this task - and this email does it. Meanwhile, however, David Gonz?lez Pisano (and other people) from Spanish National Institute of Bioinformatics (INB) created a detailed proposal how to introduce asynchronicity into BioMoby, based on a different approach (more BioMoby specific). I have also been presented at a meeting (incidentally, in a completely different, non-Moby related meeting at EBI) where this proposal was explained. Therefore, I can make some rudimentary comparison of the two proposed methods. But first, what they are? (I skip the introduction why we need an asynchronous behavior - this is already very well introduced in David's proposal). Almost any asynchronous invocations consists of: 1) Staring request: a) A client sends a request, (somehow) specifying that this request should be treated asynchronously. b) A service responds with an identifier assigned to this request (called also 'job handler' or 'ticket'). 2) A client uses this identifier to send a polling request. A service responds with a status indication. Polling requests are being sent until status indicates that job had finished. 3) Getting result request. Using still the same identifier, a client ask for a result and service provides it. (Sometimes this kind of request can be part of the last polling request from the previous step.) The INB's proposal achieves the above by adding specific attributes into the existing mobyData XML tag, and letting the service return an empty result until a real result is available. My proposal suggests to have several methods (whose names are derived by fixed suffixes), each of them would return a specific result (still defined as a Moby object), and one of them would return a real result. Both proposals are similar (almost identical) in their features, they differ only in a way how these new features are coded into used protocol. So how do they differ? 1) I must start with the one that INB's proposal claims as their advantage. The proposal says that is "able to support asynchronous calls without changing the underlying software". I do not buy it, however. I think that *both* proposals would need to change clients if they want to take advantage of the new features, and to change services in order to provide new features. But also, in *both* proposals, there is no need to change anything if you still use a synchronous call. The old clients will not break, the old services will continue to work. So this does not constitute any difference between the two proposals (IMHO, of course). 2) A real difference - and this one is in favor of the INB's proposal - is that specifying asynchronicity by attributes of the mobyData tag allows to ask for different treatment of individual mobyData parts. A client can sent two mobyData tags in one request, asking for the first one to be treated synchronously and the second one asynchronously. This cannot be achieved by having several method names. 3) Another real difference - this time in favor of my proposal - is related to the existing standards. BioMoby uses very proprietary way how it defines its data types and from that how its input and output data look like. But for the transport messages the BioMoby's behavior is so far quite standard. The same service always transforms data A into data B. And I think that this is a good thing. Introducing new attributes but keeping the same method name means that a service would return different things each time. But if we have fixed method names - we again know that always a service/method1 will transform data A into data C, and a service/method2 will transform data C into data B. The reason why I take this as an advantage is that there is a lot of general tools for Web services that could be still used for BioMoby services if we keep the behavior standard. Having more methods is perfectly easy to define in WSDL (that can be still a bit less useful for BioMoby data types - because they are by default non-standard - but in the same time good enough for many other tools). I would feel a bit uncomfortable to invent a proprietary solution if there is a standard one. Unless, of course, you feel that the point ad 2) above is significantly important that it justifies a proprietary solution. 4) A (perhaps slightly disputable) difference is about the returned objects from the starting and polling requests. The INB's proposal does not return any real object because the status is in a mobyData attributes. My proposal defines several real BioMoby data types (BioMoby objects) that can carry more complex information that just attributes. They can even be extended by inheritance like any other BioMoby object. The only difference comparing to other BioMoby data types (objects) is that you will never discover any service returning them - because a service provider does not register his/her service with such output types. This allows, btw, to use the same objects for a complex notification (if we have anything like that in the future) - see few comments on it in my proposal. 5) A small difference - in favor of the INB's proposal - is that their proposal allows to save one network request because the last polling request can become a request returning a real result. And that's it - I have not found any other differences (so far). Still, it may be useful to summarize what *neither* of two proposals solves: 1) A client does not know *in advance* whether a service supports asynchonicity or not - without some investigation. BioMoby strategy is to put/find this information in a service metadata using LSID resolution. I would say that such information is so crucial that it should be accessible easier - but allowing to have it in the service registration object, so clients can find/get it directly together with the service URL when they ask the registry. 2) A client must treat the full sequence of corresponding requests (starting, polling and resulting requests) as an "atomic" operation - which means that if any of these requests fails, the client needs to start again from the beginning. This solves the problem with having same services replicated but not always being able to take over each other in the middle of the job. 3) Introducing asynchonicity opens doors to the longer persistence of results. It is easy to imagine that once you have a job identifier (ticket, or whatever) you theoretically can ask for the same result several time without creating it again from the beginning. Of course, if a service supports it. And also a client could have an option of say "please remove the result, I am not anymore interested in it" (again, here both proposals can accommodate this feature either by having an additional (cleaning, reseting) attribute or a additional method). But BioMoby needs to document (or reject) such persistent behavior. Well, perhaps now I should say what is my proposal :-) a) It is based on an OMG standard "LSAE - Life Sciences Analysis Engine" (http://www.omg.org/cgi-bin/doc?dtc/05-04-01 and http://www.omg.org/cgi-bin/doc?dtc/05-04-08). b) If we have a service ABC and its service provider wishes to support asynchronous calls then the service will have the following methods (with the following input/output data types): RealResult ABC (RealInput input) This is a standard synchronous invocation, and the RealResult and RealInput are normal BioMoby data types. The service ABC is registered with them as input and output. JobHandler ABC_async (RealInput input) This is a "starting request". The JobHandler contains a ticket (job identifier) but may contain also other things (like a suggestion how often is worth to poll for results). This method call fails if the service does not support an asynchronous behavior. This does not break the old clients because they would never use this method. NotificationEvent ABC_status (JobHandler handler) This is a "polling request". NotificationEvent contains status of the invoked job (it can failed if the handler is not knows or expired). The ways how it represents a status can be different (from just a status, to a status including progress where progress can be simple hear-beating progress, percent progress, step progress or time progress). For a feel what it can contain look please at the picture taken from the LSAE model: http://www.ebi.ac.uk/~senger/LSAE-pictures.html. I would convert them into a proper BioMoby data types (objects) if/when is clearer that this proposal has chance to be accepted. RealResult ABC_result (JobHandler handler) This is a "result request" returning a real result. It can fail if the handler does not exist or if the method is called in an unappropriated moment/order. We can add also a method ABC_clean (JobHandler handler) if we decide to support persistence. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Mon Jun 13 15:41:50 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 13 Jun 2005 08:41:50 -0700 Subject: [MOBY-dev] MOBY-DIC action item that has gone astray... Message-ID: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> Hi all, Who was it that volunteered to look up the use of LDAP or DNS-like synchronization? (please don't say it was me :-) ) M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From jmfernandez at cnb.uam.es Mon Jun 13 18:28:44 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez?=) Date: Mon, 13 Jun 2005 20:28:44 +0200 Subject: [MOBY-dev] MOBY-DIC action item that has gone astray... In-Reply-To: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> References: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> Message-ID: <42ADD05C.8090107@cnb.uam.es> Hi everybody, I think Heiko told he was going to ask some network managers he was working with (sorry, Heiko!). Best regards, Jos? Mar?a Mark Wilkinson wrote: > Hi all, > > Who was it that volunteered to look up the use of LDAP or DNS-like > synchronization? (please don't say it was me :-) ) > > M > > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From markw at illuminae.com Mon Jun 13 18:41:54 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 13 Jun 2005 11:41:54 -0700 Subject: [unclassified] Re: [MOBY-dev] MOBY-DIC action item that has gone astray... In-Reply-To: <42ADD05C.8090107@cnb.uam.es> References: <1118677310.554.85.camel@mobycentral.mrl.ubc.ca> <42ADD05C.8090107@cnb.uam.es> Message-ID: <1118688114.554.129.camel@mobycentral.mrl.ubc.ca> Ha! And he thought he was going to get away with hiding at the MPI ;-) Caught ya!!! M On Mon, 2005-06-13 at 20:28 +0200, Jos? Mar?a Fern?ndez wrote: > Hi everybody, > I think Heiko told he was going to ask some network managers he was > working with (sorry, Heiko!). > > Best regards, > Jos? Mar?a > > Mark Wilkinson wrote: > > Hi all, > > > > Who was it that volunteered to look up the use of LDAP or DNS-like > > synchronization? (please don't say it was me :-) ) > > > > M > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Wed Jun 29 15:25:05 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 29 Jun 2005 08:25:05 -0700 Subject: [MOBY-dev] MOBY Central hacked last night Message-ID: <1120058705.20181.26.camel@mobycentral.mrl.ubc.ca> Hi all, MOBY Central is down for at least a day due to a nasty hack last night. We may have to rebuild from scratch, so it may not be a rapid recovery. Speaks again to the need for a mirroring system! Hey, Heiko... how's the research going? ;-) ;-) You can run, but you can't hide! Sorry for the down-time... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Thu Jun 30 23:38:45 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Thu, 30 Jun 2005 16:38:45 -0700 Subject: [MOBY-dev] MOBY Central Registry up and running at a new location Message-ID: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> Hi all, The core of MOBY Central is up and running again at: URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl URI: http://mobycentral.icapture.ubc.ca/MOBY/Central No accessories are functioning, however. Just the registry. Also, none of my services will be working either. No data loss as far as I can tell at the moment, so that is good news! I will begin migrating the accessories over to this new server tomorrow. Hopefully we will have things running again by the end of the weekend, but for the moment at least the registry itself is functional. remember you can use this registry as your default for the Perl libraries by setting the environment variables: MOBY_SERVER= $URL (above) MOBY_URI = $URI (above) This will cause MOBY::Client::Central to connect to this server by default. I will make changes to the test harness in the CVS soon. It doesn't matter, really, because the version in the CVS is non-functional until we can get the Perl LSID resolver stack working }-/ Anyway, sorry for the down-time! What a disaster! (but I did learn a lot about what it takes to install MOBY Central now! So I have lots of good notes for our documentation drive...) M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC