From Pieter.Neerincx at wur.nl Wed Jul 6 10:50:42 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Wed Jul 6 10:41:49 2005 Subject: [MOBY-dev] Perl API: MOBY::Central::DUMP_MySQL() In-Reply-To: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> References: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> Message-ID: <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> Hi all, This message is for those who use Perl and have local MOBY Centrals. I had some problems with the way MOBY::Central::DUMP_MySQL() currently works. Getting a DUMP fails for example with the new central at: URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl URI: http://mobycentral.icapture.ubc.ca/MOBY/Central Currently DUMP_MySQL() has the mysql username, the path to mysqldump and the lack of a password, host and portnumber hard coded. Hence it ignores whatever you have in your mobycentral.config file. Therefore I propose the code below which uses MOBY::Config->new() to get the connection details from your mobycentral.config. The path to mysqldump is no longer hard coded. Hence if it's not in the path of webserver user, I think it's better to modify the path env var as compared to hard coding it in the perl modules:) Unless anyone has objections against the porposed code I'll merge this with the code in the CVS in a few days... Cheers, Pieter -------------------------------------------------- sub DUMP_MySQL { my ( $pkg ) = @_; my $config = MOBY::Config->new(); my @dbsections = ('mobycentral', 'mobyobject', 'mobyservice', 'mobynamespace', 'mobyrelationship'); my @response; foreach my $dbsection (@dbsections) { my $dbname = ${${$config}{$dbsection}}{'dbname'}; my $username = ${${$config}{$dbsection}}{'username'}; my $password = ${${$config}{$dbsection}}{'password'}; my $host = ${${$config}{$dbsection}}{'url'}; my $port = ${${$config}{$dbsection}}{'port'}; open( IN, "mysqldump -h $host -P $port -u $username --password=$password $dbname|" ) || die "can't open $dbname for dumping"; my @dbdump; while ( ) { push @dbdump, $_; } my $dbdump = ( join "", @dbdump ); push @response, $dbdump; } return [@response]; } ------------------------------------------------ Wageningen University and Research centre (WUR) Laboratory of Bioinformatics Transitorium (building 312) room 1038 Dreijenlaan 3 6703 HA Wageningen phone: 0317-484 706 fax: 0317-483 584 mobile: 06-143 66 783 pieter.neerincx@wur.nl From markw at illuminae.com Wed Jul 6 11:09:44 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed Jul 6 11:00:46 2005 Subject: [moby] [MOBY-dev] Perl API: MOBY::Central::DUMP_MySQL() In-Reply-To: <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> References: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> Message-ID: <1120662584.8869.38.camel@mobycentral.mrl.ubc.ca> Hi Pieter, I didn't realize that the calls were hard-coded! Ugh! Thanks for catching that error - please do go ahead and fix the code. We will be committing a major change to the organization of SQL in the codebase in the next few days, so once you have committed I am going to tag this version and make a release before the chaos ensues :-) M On Wed, 2005-07-06 at 16:50 +0200, Pieter Neerincx wrote: > Hi all, > > This message is for those who use Perl and have local MOBY Centrals. > I had some problems with the way MOBY::Central::DUMP_MySQL() > currently works. Getting a DUMP fails for example with the new > central at: > > URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl > URI: http://mobycentral.icapture.ubc.ca/MOBY/Central > > Currently DUMP_MySQL() has the mysql username, the path to mysqldump > and the lack of a password, host and portnumber hard coded. Hence it > ignores whatever you have in your mobycentral.config file. Therefore > I propose the code below which uses MOBY::Config->new() to get the > connection details from your mobycentral.config. The path to > mysqldump is no longer hard coded. Hence if it's not in the path of > webserver user, I think it's better to modify the path env var as > compared to hard coding it in the perl modules:) Unless anyone has > objections against the porposed code I'll merge this with the code in > the CVS in a few days... > > Cheers, > > Pieter > > -------------------------------------------------- > sub DUMP_MySQL { > my ( $pkg ) = @_; > my $config = MOBY::Config->new(); > my @dbsections = ('mobycentral', 'mobyobject', > 'mobyservice', 'mobynamespace', 'mobyrelationship'); > my @response; > > foreach my $dbsection (@dbsections) { > my $dbname = ${${$config}{$dbsection}}{'dbname'}; > my $username = ${${$config}{$dbsection}}{'username'}; > my $password = ${${$config}{$dbsection}}{'password'}; > my $host = ${${$config}{$dbsection}}{'url'}; > my $port = ${${$config}{$dbsection}}{'port'}; > open( IN, "mysqldump -h $host -P $port -u $username > --password=$password $dbname|" ) > || die "can't open $dbname for dumping"; > my @dbdump; > while ( ) { > push @dbdump, $_; > } > my $dbdump = ( join "", @dbdump ); > push @response, $dbdump; > } > return [@response]; > } > ------------------------------------------------ > > > > Wageningen University and Research centre (WUR) > Laboratory of Bioinformatics > Transitorium (building 312) room 1038 > Dreijenlaan 3 > 6703 HA Wageningen > phone: 0317-484 706 > fax: 0317-483 584 > mobile: 06-143 66 783 > pieter.neerincx@wur.nl > > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From Pieter.Neerincx at wur.nl Wed Jul 6 11:41:59 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Wed Jul 6 11:33:16 2005 Subject: [moby] [MOBY-dev] Perl API: MOBY::Central::DUMP_MySQL() In-Reply-To: <1120662584.8869.38.camel@mobycentral.mrl.ubc.ca> References: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> <1120662584.8869.38.camel@mobycentral.mrl.ubc.ca> Message-ID: Hi Mark, On 6-Jul-2005, at 5:09 PM, Mark Wilkinson wrote: > Hi Pieter, > > I didn't realize that the calls were hard-coded! Ugh! > > Thanks for catching that error - please do go ahead and fix the code. Ok, done. > > We will be committing a major change to the organization of SQL in the > codebase in the next few days, so once you have committed I am > going to > tag this version and make a release before the chaos ensues :-) Whoops, good luck! Cheers, Pieter > > M > > > On Wed, 2005-07-06 at 16:50 +0200, Pieter Neerincx wrote: > >> Hi all, >> >> This message is for those who use Perl and have local MOBY Centrals. >> I had some problems with the way MOBY::Central::DUMP_MySQL() >> currently works. Getting a DUMP fails for example with the new >> central at: >> >> URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl >> URI: http://mobycentral.icapture.ubc.ca/MOBY/Central >> >> Currently DUMP_MySQL() has the mysql username, the path to mysqldump >> and the lack of a password, host and portnumber hard coded. Hence it >> ignores whatever you have in your mobycentral.config file. Therefore >> I propose the code below which uses MOBY::Config->new() to get the >> connection details from your mobycentral.config. The path to >> mysqldump is no longer hard coded. Hence if it's not in the path of >> webserver user, I think it's better to modify the path env var as >> compared to hard coding it in the perl modules:) Unless anyone has >> objections against the porposed code I'll merge this with the code in >> the CVS in a few days... >> >> Cheers, >> >> Pieter >> >> -------------------------------------------------- >> sub DUMP_MySQL { >> my ( $pkg ) = @_; >> my $config = MOBY::Config->new(); >> my @dbsections = ('mobycentral', 'mobyobject', >> 'mobyservice', 'mobynamespace', 'mobyrelationship'); >> my @response; >> >> foreach my $dbsection (@dbsections) { >> my $dbname = ${${$config}{$dbsection}}{'dbname'}; >> my $username = ${${$config}{$dbsection}} >> {'username'}; >> my $password = ${${$config}{$dbsection}} >> {'password'}; >> my $host = ${${$config}{$dbsection}}{'url'}; >> my $port = ${${$config}{$dbsection}}{'port'}; >> open( IN, "mysqldump -h $host -P $port -u $username >> --password=$password $dbname|" ) >> || die "can't open $dbname for dumping"; >> my @dbdump; >> while ( ) { >> push @dbdump, $_; >> } >> my $dbdump = ( join "", @dbdump ); >> push @response, $dbdump; >> } >> return [@response]; >> } >> ------------------------------------------------ >> >> >> >> Wageningen University and Research centre (WUR) >> Laboratory of Bioinformatics >> Transitorium (building 312) room 1038 >> Dreijenlaan 3 >> 6703 HA Wageningen >> phone: 0317-484 706 >> fax: 0317-483 584 >> mobile: 06-143 66 783 >> pieter.neerincx@wur.nl >> >> >> _______________________________________________ >> MOBY-dev mailing list >> MOBY-dev@biomoby.org >> http://www.biomoby.org/mailman/listinfo/moby-dev >> > -- > Mark Wilkinson > Asst. Professor > Dept. of Medical Genetics > University of British Columbia > PI in Bioinformatics > iCAPTURE Centre > St. Paul's Hospital > Vancouver, BC > > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > Wageningen University and Research centre (WUR) Laboratory of Bioinformatics Transitorium (building 312) room 1038 Dreijenlaan 3 6703 HA Wageningen phone: 0317-484 706 fax: 0317-483 584 mobile: 06-143 66 783 pieter.neerincx@wur.nl From markw at illuminae.com Wed Jul 6 12:35:19 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed Jul 6 12:26:16 2005 Subject: [MOBY-dev] Bad documetation for Secondary parameters in MOBY Central Message-ID: <1120667719.8869.75.camel@mobycentral.mrl.ubc.ca> Hi all, I just got a heads-up from Eddie that the documentation for MOBY::Central was incorrect. It had enumerated the valid secondary data-types as STRING|INT|FLOAT, etc. They are, in fact, String, Integer, Float, DateTime (capitalization differences). This is tested at the DB level, so it causes some mis-registration if you aren't aware of the problem. We're fixing the code to throw a warning when it detects a mistaken parameter, but I wanted to let you know that this is an issue in case you have been wondering why you can't register secondaries :-/ M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From rebecca.ernst at gsf.de Thu Jul 7 09:45:35 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Thu Jul 7 09:37:08 2005 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <42CD31FF.6040803@gsf.de> Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de From markw at illuminae.com Thu Jul 7 09:56:16 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu Jul 7 09:47:25 2005 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1562815689-1120744590-cardhu_blackberry.rim.net-11062-@engine12-cell01> I guess it doesn't make you feel any better if I say "yes, we know..." :-) M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas@gmail.com, markw@illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 09:56:16 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu Jul 7 09:47:36 2005 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1562815689-1120744590-cardhu_blackberry.rim.net-11062-@engine12-cell01> I guess it doesn't make you feel any better if I say "yes, we know..." :-) M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas@gmail.com, markw@illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 09:58:53 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu Jul 7 09:50:03 2005 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> We really need to get write-access to the Taverna CVS so that we can fix these kinds of things in a timely way... Hint hint :-). Anyone from myGrid listening? M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas@gmail.com, markw@illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 09:58:53 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu Jul 7 09:50:16 2005 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> We really need to get write-access to the Taverna CVS so that we can fix these kinds of things in a timely way... Hint hint :-). Anyone from myGrid listening? M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas@gmail.com, markw@illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From schoof at mpiz-koeln.mpg.de Thu Jul 7 11:34:39 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Thu Jul 7 11:25:57 2005 Subject: [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CD31FF.6040803@gsf.de> References: <42CD31FF.6040803@gsf.de> Message-ID: <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> Please let's take a minute and review the use of collections in BioMoby, before we go off to create "workarounds"... I myself am confused about the use of collections. Originally I had in my mind that Collections were a construct to allow objects that inherently belong together to be "bagged". Example: A multiple alignment program that takes any number of sequences as input. Example2: A keyword search that takes any number of keywords and then does a query combining all of them. As opposed to: Inputing a list of keywords and executing the keyword search for each of them, which would require a separate moby:mobyData and queryID for each of the keywords. The confusion starts with the output of services. My understanding was that ONLY a service that is guaranteed to output exactly one object for each query (e.g. an averaging service that outputs the average of a list of inputs) is registered as outputting a Simple, all others have to output collections (as there must be exactly one mobyData matching the queryID of the input in the response, and a mobyData may contain multiple Simple elements only if wrapped by a Collection). This makes it impossible to distinguish between the situation where one query produces multiple unrelated results, versus one or more bags of related results. Imagine a "getnonoverlappingsubstrings" service: with input abc, it should output [a, bc], [ab, c] and [a, b, c]. Outputting [a, bc, ab, c, a, b, c] would not be useful. More biological example: Dirk has a service that returns sets of orthologous genes when given a set of species. For this, he requires collections both for input and for output. The ideal situation in my view would be: Input: Collection of Species Output: Many Collections, each Collection contains orthologous genes (one from each species). The Collection here defines a set of orthologs, and using a collection would be more elegant than having to define a Moby object "OrhologSet" has (2...n) Objects In practice, the Collection tag has been used to indicate when more than one Simple occurs, with no "semantic" meaning. This imho is not necessary; when more than one Simple occurs, why not put more than one Simple? It's easy enough for everyone to figure that out. Then, Collection could be used to actually transfer meaning ;-) This is a drastic change in the way the API is being interpreted, and will break code. So it needs calm thinking. But with the "big" API change coming up for the Primitives, it could be done. And it would make things clearer, also to e.g. the Taverna developers: Getting back many Simple articles in response to a query very intuitively indicates to continue on with each one individually, whereas getting back a Collection indicates to put the whole thing as input into the next service, which is what they implemented. Makes perfect sense, as there can and will be services that consume Collections. E.g., the ortholog set case above: Pipe it into a Multiple Alignment service, and you get all the alignments for each of the set of orthologs. Getting one massive alignment of everything wouldn't make sense. Maybe I've made myself clear, maybe not. Anyway, the Collection issue has led to quite some discussions between Rebecca, Dirk and myself, and we are all not happy with the way they are currently handled. Best, Heiko On 7. Jul 2005, at 15:45 Uhr, Rebecca Ernst wrote: Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de From tmo at ebi.ac.uk Thu Jul 7 11:50:31 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Thu Jul 7 11:41:26 2005 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> References: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> Message-ID: <42CD4F47.4030901@ebi.ac.uk> mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom From tmo at ebi.ac.uk Thu Jul 7 11:50:31 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Thu Jul 7 11:41:27 2005 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> References: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> Message-ID: <42CD4F47.4030901@ebi.ac.uk> mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom From markw at illuminae.com Thu Jul 7 12:25:13 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu Jul 7 12:16:41 2005 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1289629624-1120753526-cardhu_blackberry.rim.net-7512-@engine08-cell01> :-). We have all been well and truly spanked :-) Everything Tom says is (obviously) correct, and should apply to the moby CVS commits also. Thanks for giving Eddie access! M . -----Original Message----- From: Tom Oinn Date: Thu, 07 Jul 2005 16:50:31 To:markw@illuminae.com, Core developer announcements Cc:Martin Senger , mobydev Subject: Re: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 12:25:13 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu Jul 7 12:16:48 2005 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1289629624-1120753526-cardhu_blackberry.rim.net-7512-@engine08-cell01> :-). We have all been well and truly spanked :-) Everything Tom says is (obviously) correct, and should apply to the moby CVS commits also. Thanks for giving Eddie access! M . -----Original Message----- From: Tom Oinn Date: Thu, 07 Jul 2005 16:50:31 To:markw@illuminae.com, Core developer announcements Cc:Martin Senger , mobydev Subject: Re: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 16:07:48 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Thu Jul 7 15:58:50 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> Message-ID: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Thu Jul 7 16:07:48 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Thu Jul 7 15:58:52 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> Message-ID: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From schoof at mpiz-koeln.mpg.de Fri Jul 8 02:45:28 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Fri Jul 8 02:36:34 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> Message-ID: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Well, if I understand you correctly, Mark, than I was right all along in that they way I see collections is the way they should be seen. Then the problem is that that is not clear enough to everybody else, and that possibly the examples in the API are misleading, because it's been used in such a different way. It's definitely something that needs to be fixed before release 1.0. If I understand Tom correctly, the problem mainly appeared because the BioMoby support was rapidly and effectively incorporated into Taverna without proper integration, and what needs to happen is to define e.g. the iteration strategy for BioMoby: Taverna so far doesn't use the possibility to bundle multiple queries in a single BioMoby message, which is not a problem, but the Collection issue could point to a need for us BioMoby guys to look in more detail at Taverna data types and do a sound mapping. Heiko On 7. Jul 2005, at 22:07 Uhr, Mark Wilkinson wrote: On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From schoof at mpiz-koeln.mpg.de Fri Jul 8 02:45:28 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Fri Jul 8 02:36:36 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> Message-ID: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Well, if I understand you correctly, Mark, than I was right all along in that they way I see collections is the way they should be seen. Then the problem is that that is not clear enough to everybody else, and that possibly the examples in the API are misleading, because it's been used in such a different way. It's definitely something that needs to be fixed before release 1.0. If I understand Tom correctly, the problem mainly appeared because the BioMoby support was rapidly and effectively incorporated into Taverna without proper integration, and what needs to happen is to define e.g. the iteration strategy for BioMoby: Taverna so far doesn't use the possibility to bundle multiple queries in a single BioMoby message, which is not a problem, but the Collection issue could point to a need for us BioMoby guys to look in more detail at Taverna data types and do a sound mapping. Heiko On 7. Jul 2005, at 22:07 Uhr, Mark Wilkinson wrote: On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From rebecca.ernst at gsf.de Fri Jul 8 02:50:17 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Fri Jul 8 02:41:15 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> Message-ID: <42CE2229.1030504@gsf.de> Hi Mark! I will have to spank you a lot for either the last email or the one you sent me in February where I asked you how to use collections (see mail below): Well... the answer depends on what the generic case of your output will look like. the rule is that you have to register every output object that your service produces; i.e. every immediate child tag of a mobyData block. Therefore, if you are going to consistently output exactly 3 Genbank/gi's, then you would register your service as outputting 3 X Object(NCBI_gi). If you are outputting an indeterminate number of objects, then you register it as outputting 1 X Collection. Collections map nicely onto the rdf:Bag structure, if that helps you interpret them. M On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: >> Hi Mark! >> >> We have problems interpreting the BioMoby API for the output objects. >> >> The thing is that we have a service that gets ONE input object and >> executes a freetext search over several databases. This service will >> eventually return more that one result. >> We don't want to put the result into a collection object because the >> single objects don't build any entity. >> We also can't give back more that one mobyData block because we only >> have one query ID and therefore only one queryID to give back. >> >> The only solution we can think of is an object like this: >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> would this be a legal object or do we have to use collection even if the >> objects don't build an entity ? >> >> >> Cheers, >> Rebecca > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical Genetics, UBC, Canada ----------------------------------------------------------------------------------------------------------- Mark Wilkinson wrote: >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > >>I myself am confused about the use of collections. Originally I had in >>my mind that Collections were a construct to allow objects that >>inherently belong together to be "bagged". >> >> > >You are 100% correct here. > > > > >>The confusion starts with the output of services. My understanding was >>that ONLY a service that is guaranteed to output exactly one object for >>each query (e.g. an averaging service that outputs the average of a >>list of inputs) is registered as outputting a Simple, all others have >>to output collections >> >> > >No. > > > >> (as there must be exactly one mobyData matching >>the queryID of the input in the response, >> >> > >Yes > > > >> and a mobyData may contain >>multiple Simple elements only if wrapped by a Collection). >> >> > >No. A mobyData element can wrap any number of outputs, and/or >combinations of simples, collections, and secondaries. > > > > > >>In practice, the Collection tag has been used to indicate when more >>than one Simple occurs, with no "semantic" meaning. This imho is not >>necessary; when more than one Simple occurs, why not put more than one >>Simple? It's easy enough for everyone to figure that out. Then, >>Collection could be used to actually transfer meaning ;-) >> >> > >I think a lot of people are using Collections incorrectly, yes. > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back >>many Simple articles in response to a query very intuitively indicates >>to continue on with each one individually, whereas getting back a >>Collection indicates to put the whole thing as input into the next >>service, which is what they implemented. Makes perfect sense, as there >>can and will be services that consume Collections. >> >> > >I don't think that Taverna handled collections correctly in the past, >and it would be a shame if that identical functionality was added back >in :-) The functionality does need to go back into Taverna, but >correctly this time. > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue >>has led to quite some discussions between Rebecca, Dirk and myself, and >>we are all not happy with the way they are currently handled. >> >> > >I don't know if you are happier with them now that I have pointed out >where you are misinterpreting them...?? > >I agree, however, that their usage by many service providers is not >correct... but there's no way for MOBY Central to know that it isn't >correct, so we can't throw an error on registration of these "wonky" >services... > >M > > > > > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de From tmo at ebi.ac.uk Fri Jul 8 04:07:44 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Fri Jul 8 03:57:29 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <42CE3450.1000209@ebi.ac.uk> Heiko Schoof wrote: > If I understand Tom correctly, the problem mainly appeared because the > BioMoby support was rapidly and effectively incorporated into Taverna > without proper integration, and what needs to happen is to define e.g. > the iteration strategy for BioMoby: Taverna so far doesn't use the > possibility to bundle multiple queries in a single BioMoby message, > which is not a problem, but the Collection issue could point to a need > for us BioMoby guys to look in more detail at Taverna data types and do > a sound mapping. > Heiko That's kind of true, meaning actually not. There are three cases involving collections (Taverna 1.1 behaviour) : 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. The intent is that as with the other plugin types Taverna guarantees that the service sees the inputs it has asked for. Our experience with other service types suggests that this is an extremely powerful mechanism as it allows interoperability between services that would otherwise mismatch - it's worth noting that our users expect these services to match, while a CS perspective regards ProteinSequence and ProteinSequence[] as completely different types most of our biologists don't! Taverna behaves the way _they_ expect it to, remember who your user are. Taverna data types are pretty much trivial, they're opaque data blobs with the exception of collection structure which is exposed. We only expose the collection structure to ensure the above properties, other than that the framework is data agnostic (as it should be). Tom From tmo at ebi.ac.uk Fri Jul 8 04:07:44 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Fri Jul 8 03:57:32 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <42CE3450.1000209@ebi.ac.uk> Heiko Schoof wrote: > If I understand Tom correctly, the problem mainly appeared because the > BioMoby support was rapidly and effectively incorporated into Taverna > without proper integration, and what needs to happen is to define e.g. > the iteration strategy for BioMoby: Taverna so far doesn't use the > possibility to bundle multiple queries in a single BioMoby message, > which is not a problem, but the Collection issue could point to a need > for us BioMoby guys to look in more detail at Taverna data types and do > a sound mapping. > Heiko That's kind of true, meaning actually not. There are three cases involving collections (Taverna 1.1 behaviour) : 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. The intent is that as with the other plugin types Taverna guarantees that the service sees the inputs it has asked for. Our experience with other service types suggests that this is an extremely powerful mechanism as it allows interoperability between services that would otherwise mismatch - it's worth noting that our users expect these services to match, while a CS perspective regards ProteinSequence and ProteinSequence[] as completely different types most of our biologists don't! Taverna behaves the way _they_ expect it to, remember who your user are. Taverna data types are pretty much trivial, they're opaque data blobs with the exception of collection structure which is exposed. We only expose the collection structure to ensure the above properties, other than that the framework is data agnostic (as it should be). Tom From d.haase at gsf.de Fri Jul 8 04:49:58 2005 From: d.haase at gsf.de (Dirk Haase) Date: Fri Jul 8 04:41:06 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <200507081049.58799.d.haase@gsf.de> On Friday 08 July 2005 08:45, Heiko Schoof wrote: > Well, if I understand you correctly, Mark, than I was right all along > in that they way I see collections is the way they should be seen. Then > the problem is that that is not clear enough to everybody else, and > that possibly the examples in the API are misleading, because it's been > used in such a different way. It's definitely something that needs to > be fixed before release 1.0. Not only the examples are misleading, actually at least the PERL API is lacking a suitable method to construct an output consisting of several 'Simple's. One can not simply concat the output of multibple 'simpleResponse' calls, as this would create several mobyData blocks. In order to fill this gap, I created a new method 'multiSimpleResponse' (see below) within CommonSubs.pm which is more or less identical to 'collectionResponse' except that it obviously leaves out the moby:Collection tag. The provided articleName is put into each 'Simple' tag which should be appropriate. Any objections? Regards, dirk =head2 multiSimpleResponse name : multiSimpleResponse function : wraps a set of simple articles in the appropriate mobyData structure usage : return responseHeader . &multiSimpleResponse(\@objects, 'MyArticleName', $queryID) . responseFooter; args : (in order) \@objects - (optional) a listref of MOBY Objects as raw XML $article - (optional) an articeName for this article $queryID - (optional, but strongly recommended) the mobyData ID to which you are responding notes : as required by the API you must return a response for every input. If one of the inputs was invalid, you return a valid (empty) MOBY response by calling &multiSimpleResponse(undef, undef, $queryID). =cut sub multiSimpleResponse { my ( $data, $articleName, $qID ) = @_; # articleName optional my $content = ""; $data ||= []; $qID ||= ''; $articleName ||=""; unless ( ( ref( $data ) =~ /array/i ) && $data->[0] ) { # we're expecting an arrayref as input data,and it must not be empty return ""; } foreach ( @{$data} ) { if ( $_ ) { $content .= " $_ "; } else { $content .= " "; } } return " $content "; } From d.haase at gsf.de Fri Jul 8 04:49:58 2005 From: d.haase at gsf.de (Dirk Haase) Date: Fri Jul 8 04:41:09 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <200507081049.58799.d.haase@gsf.de> On Friday 08 July 2005 08:45, Heiko Schoof wrote: > Well, if I understand you correctly, Mark, than I was right all along > in that they way I see collections is the way they should be seen. Then > the problem is that that is not clear enough to everybody else, and > that possibly the examples in the API are misleading, because it's been > used in such a different way. It's definitely something that needs to > be fixed before release 1.0. Not only the examples are misleading, actually at least the PERL API is lacking a suitable method to construct an output consisting of several 'Simple's. One can not simply concat the output of multibple 'simpleResponse' calls, as this would create several mobyData blocks. In order to fill this gap, I created a new method 'multiSimpleResponse' (see below) within CommonSubs.pm which is more or less identical to 'collectionResponse' except that it obviously leaves out the moby:Collection tag. The provided articleName is put into each 'Simple' tag which should be appropriate. Any objections? Regards, dirk =head2 multiSimpleResponse name : multiSimpleResponse function : wraps a set of simple articles in the appropriate mobyData structure usage : return responseHeader . &multiSimpleResponse(\@objects, 'MyArticleName', $queryID) . responseFooter; args : (in order) \@objects - (optional) a listref of MOBY Objects as raw XML $article - (optional) an articeName for this article $queryID - (optional, but strongly recommended) the mobyData ID to which you are responding notes : as required by the API you must return a response for every input. If one of the inputs was invalid, you return a valid (empty) MOBY response by calling &multiSimpleResponse(undef, undef, $queryID). =cut sub multiSimpleResponse { my ( $data, $articleName, $qID ) = @_; # articleName optional my $content = ""; $data ||= []; $qID ||= ''; $articleName ||=""; unless ( ( ref( $data ) =~ /array/i ) && $data->[0] ) { # we're expecting an arrayref as input data,and it must not be empty return ""; } foreach ( @{$data} ) { if ( $_ ) { $content .= " $_ "; } else { $content .= " "; } } return " $content "; } From schoof at mpiz-koeln.mpg.de Fri Jul 8 04:54:03 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Fri Jul 8 04:45:01 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <24407d5355834f57f8ac3e2245daa77d@mpiz-koeln.mpg.de> Point taken to prove that indeed, I have to look into Taverna in more depth ;-) Anyway, Tom, what you state below is exactly what I would expect as behaviour, but I think on the BioMoby side we must ensure that our "Collections" behave in the right way, and I'm not yet sure they are precisely the same thing as what you call collection. Best, Heiko On 8. Jul 2005, at 10:07 Uhr, Tom Oinn wrote: Heiko Schoof wrote: > If I understand Tom correctly, the problem mainly appeared because the > BioMoby support was rapidly and effectively incorporated into Taverna > without proper integration, and what needs to happen is to define e.g. > the iteration strategy for BioMoby: Taverna so far doesn't use the > possibility to bundle multiple queries in a single BioMoby message, > which is not a problem, but the Collection issue could point to a need > for us BioMoby guys to look in more detail at Taverna data types and > do a sound mapping. > Heiko That's kind of true, meaning actually not. There are three cases involving collections (Taverna 1.1 behaviour) : 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. The intent is that as with the other plugin types Taverna guarantees that the service sees the inputs it has asked for. Our experience with other service types suggests that this is an extremely powerful mechanism as it allows interoperability between services that would otherwise mismatch - it's worth noting that our users expect these services to match, while a CS perspective regards ProteinSequence and ProteinSequence[] as completely different types most of our biologists don't! Taverna behaves the way _they_ expect it to, remember who your user are. Taverna data types are pretty much trivial, they're opaque data blobs with the exception of collection structure which is exposed. We only expose the collection structure to ensure the above properties, other than that the framework is data agnostic (as it should be). Tom _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From rebecca.ernst at gsf.de Fri Jul 8 05:07:50 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Fri Jul 8 04:58:55 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <42CE4266.3030304@gsf.de> Hi Tom and others This Taverna behaviour described below makes perfect sense to me! Does Taverna then check in MOBY-Central how this service is registered (taking simples or collections)? I guess the way to go would be to: 1. change the Taverna behaviour back to how it is supposed to work (and did in 1.1) 2. - change the BioMoby Perl code to allow more than one simple in the MobyData AND - change almost all services that output collections to the way we discussed yesterday (only output collections if the objects build an entity. ) Either of the two changes solves the problems but I believe both are due to changes and should take place. Best, Rebecca Tom Oinn wrote: > That's kind of true, meaning actually not. There are three cases > involving collections (Taverna 1.1 behaviour) : > > 1) Consumer declares it consumes singles, Producer emits a collection. > In this context Taverna iteratively calls the Consumer with each item > from the collection. This is probably what you'd expect to happen, the > result is that the Consumer effectively emits a collection of whatever > it would emit normally. > 2) Consumer declares it consumes a collection, Producer emits a > collection. In this case Taverna will indeed split the output > collection (because we always do) but it will be magically reassembled > before being given to the Consumer. > > 3) Consumer declares it consumes a collection, Producer emits a single > item. Taverna wraps the single item in a single element collection and > gives it to the Consumer. > > The intent is that as with the other plugin types Taverna guarantees > that the service sees the inputs it has asked for. Our experience with > other service types suggests that this is an extremely powerful > mechanism as it allows interoperability between services that would > otherwise mismatch - it's worth noting that our users expect these > services to match, while a CS perspective regards ProteinSequence and > ProteinSequence[] as completely different types most of our biologists > don't! Taverna behaves the way _they_ expect it to, remember who your > user are. > > Taverna data types are pretty much trivial, they're opaque data blobs > with the exception of collection structure which is exposed. We only > expose the collection structure to ensure the above properties, other > than that the framework is data agnostic (as it should be). > > Tom > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de From rebecca.ernst at gsf.de Fri Jul 8 05:07:50 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Fri Jul 8 04:59:00 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <42CE4266.3030304@gsf.de> Hi Tom and others This Taverna behaviour described below makes perfect sense to me! Does Taverna then check in MOBY-Central how this service is registered (taking simples or collections)? I guess the way to go would be to: 1. change the Taverna behaviour back to how it is supposed to work (and did in 1.1) 2. - change the BioMoby Perl code to allow more than one simple in the MobyData AND - change almost all services that output collections to the way we discussed yesterday (only output collections if the objects build an entity. ) Either of the two changes solves the problems but I believe both are due to changes and should take place. Best, Rebecca Tom Oinn wrote: > That's kind of true, meaning actually not. There are three cases > involving collections (Taverna 1.1 behaviour) : > > 1) Consumer declares it consumes singles, Producer emits a collection. > In this context Taverna iteratively calls the Consumer with each item > from the collection. This is probably what you'd expect to happen, the > result is that the Consumer effectively emits a collection of whatever > it would emit normally. > 2) Consumer declares it consumes a collection, Producer emits a > collection. In this case Taverna will indeed split the output > collection (because we always do) but it will be magically reassembled > before being given to the Consumer. > > 3) Consumer declares it consumes a collection, Producer emits a single > item. Taverna wraps the single item in a single element collection and > gives it to the Consumer. > > The intent is that as with the other plugin types Taverna guarantees > that the service sees the inputs it has asked for. Our experience with > other service types suggests that this is an extremely powerful > mechanism as it allows interoperability between services that would > otherwise mismatch - it's worth noting that our users expect these > services to match, while a CS perspective regards ProteinSequence and > ProteinSequence[] as completely different types most of our biologists > don't! Taverna behaves the way _they_ expect it to, remember who your > user are. > > Taverna data types are pretty much trivial, they're opaque data blobs > with the exception of collection structure which is exposed. We only > expose the collection structure to ensure the above properties, other > than that the framework is data agnostic (as it should be). > > Tom > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de From jmfernandez at cnb.uam.es Fri Jul 8 07:12:37 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Fri Jul 8 07:04:13 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna1.2 In-Reply-To: <42CE4266.3030304@gsf.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> Message-ID: <42CE5FA5.50901@cnb.uam.es> Hi everyone, the Taverna v1.1 behaviour for MOBY should be the default one for services which only takes one input parameter. But, which collection semantics should have Taverna MOBY plugin for services with more than one parameter? All of this is related to iteration strategies. For instance, imagine this scenario: we have a service which needs two input Simple parameters, A and B. What should we do? * If both input parameters receive a Simple, no problem, default behaviour. * If one of those parameters receive a Collection with N Simples, and the other a Simple, the default iteration strategy should be either invoking the MOBY service N times with N separate SOAP submissions, or invoking the MOBY service N times with only a SOAP submission (or something intermediate). * If each one of the parameters receive a Collection, then the iteration strategy *should* be explicitly set by the workflow creator: either cartesian product (first of A with each one of B, second of A with each one of B) or dot product (first for A with first for B, second for A with second for B, and so on, either discarding the last elements from the longest Collection, or firing an exception/error/message/whatever). The other two scenarios are: a two-input service with both parameters defined as Collection; and a two-input service with one the parameters as Collection and the other as Simple. I leave them as an exercise for the reader 8-), so I'm avoiding to write now the most boring e-mail in BioMOBY story ;-) Best Regards, Jos? Mar?a Rebecca Ernst wrote: > Hi Tom and others > > This Taverna behaviour described below makes perfect sense to me! Does > Taverna then check in MOBY-Central how this service is registered > (taking simples or collections)? > > I guess the way to go would be to: > 1. change the Taverna behaviour back to how it is supposed to work (and > did in 1.1) > 2. - change the BioMoby Perl code to allow more than one simple in the > MobyData AND > - change almost all services that output collections to the way we > discussed yesterday (only output collections if the objects build an > entity. ) > > Either of the two changes solves the problems but I believe both are due > to changes and should take place. > > Best, > Rebecca > > > > > > Tom Oinn wrote: > >> That's kind of true, meaning actually not. There are three cases >> involving collections (Taverna 1.1 behaviour) : >> >> 1) Consumer declares it consumes singles, Producer emits a collection. >> In this context Taverna iteratively calls the Consumer with each item >> from the collection. This is probably what you'd expect to happen, the >> result is that the Consumer effectively emits a collection of whatever >> it would emit normally. > > >> 2) Consumer declares it consumes a collection, Producer emits a >> collection. In this case Taverna will indeed split the output >> collection (because we always do) but it will be magically reassembled >> before being given to the Consumer. >> >> 3) Consumer declares it consumes a collection, Producer emits a single >> item. Taverna wraps the single item in a single element collection and >> gives it to the Consumer. >> >> The intent is that as with the other plugin types Taverna guarantees >> that the service sees the inputs it has asked for. Our experience with >> other service types suggests that this is an extremely powerful >> mechanism as it allows interoperability between services that would >> otherwise mismatch - it's worth noting that our users expect these >> services to match, while a CS perspective regards ProteinSequence and >> ProteinSequence[] as completely different types most of our biologists >> don't! Taverna behaves the way _they_ expect it to, remember who your >> user are. >> >> Taverna data types are pretty much trivial, they're opaque data blobs >> with the exception of collection structure which is exposed. We only >> expose the collection structure to ensure the above properties, other >> than that the framework is data agnostic (as it should be). >> >> Tom >> > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez@cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From jmfernandez at cnb.uam.es Fri Jul 8 07:12:37 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Fri Jul 8 07:04:23 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna1.2 In-Reply-To: <42CE4266.3030304@gsf.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> Message-ID: <42CE5FA5.50901@cnb.uam.es> Hi everyone, the Taverna v1.1 behaviour for MOBY should be the default one for services which only takes one input parameter. But, which collection semantics should have Taverna MOBY plugin for services with more than one parameter? All of this is related to iteration strategies. For instance, imagine this scenario: we have a service which needs two input Simple parameters, A and B. What should we do? * If both input parameters receive a Simple, no problem, default behaviour. * If one of those parameters receive a Collection with N Simples, and the other a Simple, the default iteration strategy should be either invoking the MOBY service N times with N separate SOAP submissions, or invoking the MOBY service N times with only a SOAP submission (or something intermediate). * If each one of the parameters receive a Collection, then the iteration strategy *should* be explicitly set by the workflow creator: either cartesian product (first of A with each one of B, second of A with each one of B) or dot product (first for A with first for B, second for A with second for B, and so on, either discarding the last elements from the longest Collection, or firing an exception/error/message/whatever). The other two scenarios are: a two-input service with both parameters defined as Collection; and a two-input service with one the parameters as Collection and the other as Simple. I leave them as an exercise for the reader 8-), so I'm avoiding to write now the most boring e-mail in BioMOBY story ;-) Best Regards, Jos? Mar?a Rebecca Ernst wrote: > Hi Tom and others > > This Taverna behaviour described below makes perfect sense to me! Does > Taverna then check in MOBY-Central how this service is registered > (taking simples or collections)? > > I guess the way to go would be to: > 1. change the Taverna behaviour back to how it is supposed to work (and > did in 1.1) > 2. - change the BioMoby Perl code to allow more than one simple in the > MobyData AND > - change almost all services that output collections to the way we > discussed yesterday (only output collections if the objects build an > entity. ) > > Either of the two changes solves the problems but I believe both are due > to changes and should take place. > > Best, > Rebecca > > > > > > Tom Oinn wrote: > >> That's kind of true, meaning actually not. There are three cases >> involving collections (Taverna 1.1 behaviour) : >> >> 1) Consumer declares it consumes singles, Producer emits a collection. >> In this context Taverna iteratively calls the Consumer with each item >> from the collection. This is probably what you'd expect to happen, the >> result is that the Consumer effectively emits a collection of whatever >> it would emit normally. > > >> 2) Consumer declares it consumes a collection, Producer emits a >> collection. In this case Taverna will indeed split the output >> collection (because we always do) but it will be magically reassembled >> before being given to the Consumer. >> >> 3) Consumer declares it consumes a collection, Producer emits a single >> item. Taverna wraps the single item in a single element collection and >> gives it to the Consumer. >> >> The intent is that as with the other plugin types Taverna guarantees >> that the service sees the inputs it has asked for. Our experience with >> other service types suggests that this is an extremely powerful >> mechanism as it allows interoperability between services that would >> otherwise mismatch - it's worth noting that our users expect these >> services to match, while a CS perspective regards ProteinSequence and >> ProteinSequence[] as completely different types most of our biologists >> don't! Taverna behaves the way _they_ expect it to, remember who your >> user are. >> >> Taverna data types are pretty much trivial, they're opaque data blobs >> with the exception of collection structure which is exposed. We only >> expose the collection structure to ensure the above properties, other >> than that the framework is data agnostic (as it should be). >> >> Tom >> > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez@cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From markw at illuminae.com Fri Jul 8 13:22:20 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri Jul 8 13:13:56 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE2229.1030504@gsf.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <42CE2229.1030504@gsf.de> Message-ID: <1120843340.15014.40.camel@mobycentral.mrl.ubc.ca> I see no inconsistency between what you quote below and what I said in my mail from yesterday... ?? M On Fri, 2005-07-08 at 08:50 +0200, Rebecca Ernst wrote: > Hi Mark! > > I will have to spank you a lot for either the last email or the one you > sent me in February where I asked you how to use collections (see mail > below): > > > Well... the answer depends on what the generic case of your output will > look like. the rule is that you have to register every output object > that your service produces; i.e. every immediate child tag of a mobyData > block. Therefore, if you are going to consistently output exactly 3 > Genbank/gi's, then you would register your service as outputting 3 X > Object(NCBI_gi). If you are outputting an indeterminate number of > objects, then you register it as outputting 1 X Collection. > > Collections map nicely onto the rdf:Bag structure, if that helps you > interpret them. > > M > > > > On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: > > >> Hi Mark! > >> > >> We have problems interpreting the BioMoby API for the output objects. > >> > >> The thing is that we have a service that gets ONE input object and > >> executes a freetext search over several databases. This service will > >> eventually return more that one result. > >> We don't want to put the result into a collection object because the > >> single objects don't build any entity. > >> We also can't give back more that one mobyData block because we only > >> have one query ID and therefore only one queryID to give back. > >> > >> The only solution we can think of is an object like this: > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> would this be a legal object or do we have to use collection even if the > >> objects don't build an entity ? > >> > >> > >> Cheers, > >> Rebecca > > > > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical > Genetics, UBC, Canada > > > ----------------------------------------------------------------------------------------------------------- > > > > > > > Mark Wilkinson wrote: > > >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > > > > > >>I myself am confused about the use of collections. Originally I had in > >>my mind that Collections were a construct to allow objects that > >>inherently belong together to be "bagged". > >> > >> > > > >You are 100% correct here. > > > > > > > > > >>The confusion starts with the output of services. My understanding was > >>that ONLY a service that is guaranteed to output exactly one object for > >>each query (e.g. an averaging service that outputs the average of a > >>list of inputs) is registered as outputting a Simple, all others have > >>to output collections > >> > >> > > > >No. > > > > > > > >> (as there must be exactly one mobyData matching > >>the queryID of the input in the response, > >> > >> > > > >Yes > > > > > > > >> and a mobyData may contain > >>multiple Simple elements only if wrapped by a Collection). > >> > >> > > > >No. A mobyData element can wrap any number of outputs, and/or > >combinations of simples, collections, and secondaries. > > > > > > > > > > > >>In practice, the Collection tag has been used to indicate when more > >>than one Simple occurs, with no "semantic" meaning. This imho is not > >>necessary; when more than one Simple occurs, why not put more than one > >>Simple? It's easy enough for everyone to figure that out. Then, > >>Collection could be used to actually transfer meaning ;-) > >> > >> > > > >I think a lot of people are using Collections incorrectly, yes. > > > > > > > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back > >>many Simple articles in response to a query very intuitively indicates > >>to continue on with each one individually, whereas getting back a > >>Collection indicates to put the whole thing as input into the next > >>service, which is what they implemented. Makes perfect sense, as there > >>can and will be services that consume Collections. > >> > >> > > > >I don't think that Taverna handled collections correctly in the past, > >and it would be a shame if that identical functionality was added back > >in :-) The functionality does need to go back into Taverna, but > >correctly this time. > > > > > > > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue > >>has led to quite some discussions between Rebecca, Dirk and myself, and > >>we are all not happy with the way they are currently handled. > >> > >> > > > >I don't know if you are happier with them now that I have pointed out > >where you are misinterpreting them...?? > > > >I agree, however, that their usage by many service providers is not > >correct... but there's no way for MOBY Central to know that it isn't > >correct, so we can't throw an error on registration of these "wonky" > >services... > > > >M > > > > > > > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 13:25:33 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri Jul 8 13:18:26 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <1120843533.15014.43.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 09:07 +0100, Tom Oinn wrote: > 1) Consumer declares it consumes singles, Producer emits a collection. > 2) Consumer declares it consumes a collection, Producer emits a > 3) Consumer declares it consumes a collection, Producer emits a single So the only case that is missing is where the Consumer declares it consumes a collection, the producer emits a single, but you want to "bulk" those simples (or even simples arising from disparate upstream service outputs) into a single input collection. I believe you said you had a widget that would do that already, but I can't remember which one it was...?? M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 13:25:33 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri Jul 8 13:18:35 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <1120843533.15014.43.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 09:07 +0100, Tom Oinn wrote: > 1) Consumer declares it consumes singles, Producer emits a collection. > 2) Consumer declares it consumes a collection, Producer emits a > 3) Consumer declares it consumes a collection, Producer emits a single So the only case that is missing is where the Consumer declares it consumes a collection, the producer emits a single, but you want to "bulk" those simples (or even simples arising from disparate upstream service outputs) into a single input collection. I believe you said you had a widget that would do that already, but I can't remember which one it was...?? M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 13:37:09 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri Jul 8 13:28:02 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <200507081049.58799.d.haase@gsf.de> References: <42CD31FF.6040803@gsf.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <200507081049.58799.d.haase@gsf.de> Message-ID: <1120844229.15014.53.camel@mobycentral.mrl.ubc.ca> I agree that the CommonSubs code does not facilitate every possibility allowed by the MOBY API, but the API should be considered canonical :-) I think that CommonSubs needs to be **completely** re-written, and in fact, this was one of the issues that came up at the last meeting - that we need to have some common implementation-independent API that we can all code to such that all possible message types can be intuitively created and unpacked in any language. Though I don't object to you adding that subroutine to the CVS (I haven't checked it for sanity, only for intent) I think it would be useful for us to operate on a higher-level at some point soon and decide what functions we *need*, and then decide how to build them, rather than adding functions on an ad hoc basis as we have been so far. For example, I might be tempted to have a function that built service responses in the following way: $OUT = CommonSubs::ServiceOutput->new(some header information - service provision info goes here); $OUT->addOutput(invocation_id => $id, simple => $simple); $OUT->addOutput(invocation_id => $id, collection => \@simples); $OUT->addOutput(invocation_id => $id2, simple => $simple2); etc. etc. That would cover all possibilities and would be more intuitive than the existing mechanism... ??? He who codes, wins! M On Fri, 2005-07-08 at 10:49 +0200, Dirk Haase wrote: > On Friday 08 July 2005 08:45, Heiko Schoof wrote: > > Well, if I understand you correctly, Mark, than I was right all along > > in that they way I see collections is the way they should be seen. Then > > the problem is that that is not clear enough to everybody else, and > > that possibly the examples in the API are misleading, because it's been > > used in such a different way. It's definitely something that needs to > > be fixed before release 1.0. > > Not only the examples are misleading, actually at least the PERL API is > lacking a suitable method to construct an output consisting of several > 'Simple's. One can not simply concat the output of multibple 'simpleResponse' > calls, as this would create several mobyData blocks. > > In order to fill this gap, I created a new method 'multiSimpleResponse' (see > below) within CommonSubs.pm which is more or less identical to > 'collectionResponse' except that it obviously leaves out the moby:Collection > tag. The provided articleName is put into each 'Simple' tag which should be > appropriate. > > Any objections? > > Regards, > dirk > > =head2 multiSimpleResponse > > name : multiSimpleResponse > function : wraps a set of simple articles in the appropriate mobyData > structure > usage : return responseHeader . &multiSimpleResponse(\@objects, > 'MyArticleName', $queryID) . responseFooter; > args : (in order) > \@objects - (optional) a listref of MOBY Objects as raw XML > $article - (optional) an articeName for this article > $queryID - (optional, but strongly recommended) the mobyData ID > to which you are responding > notes : as required by the API you must return a response for every input. > If one of the inputs was invalid, you return a valid (empty) MOBY > response by calling &multiSimpleResponse(undef, undef, $queryID). > > =cut > > > sub multiSimpleResponse { > my ( $data, $articleName, $qID ) = @_; # articleName optional > my $content = ""; > $data ||= []; > $qID ||= ''; > $articleName ||=""; > unless ( ( ref( $data ) =~ /array/i ) && $data->[0] ) > { # we're expecting an arrayref as input data,and it must not be > empty > return ""; > } > foreach ( @{$data} ) { > if ( $_ ) { > $content .= " > $_ > "; > } else { > $content .= " > > "; > } > } > return " > > $content > > "; > } > > > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From gordonp at ucalgary.ca Fri Jul 8 14:26:25 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Fri Jul 8 14:19:32 2005 Subject: [MOBY-dev] API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42CE5FA5.50901@cnb.uam.es> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> Message-ID: <42CEC551.4090108@ucalgary.ca> Hi everyone, At long last after our discussions at the last MOBY meeting in Vancouver, I have written a little document explaining the reason for and consequences of not inheriting from "primitive" objects such as String and Integer. I think that this is a distruptive change for only a few services, but is of great benefit. Please see http://coe02.ucalgary.ca/gordonp/moby_primitives.html Mark has suggested that this not be a formal change to the API, but that this new way of doing things be encouraged in all of the documentation, and the old method explicitly deprecated but not disabled. Hopefully most of you agree, and we can just make temporary exceptions in our code for the current non-compliant classes. Or even better, if most people agree, we can force the affected services to change over. :-) P.S. Yan, your object browser came in quite handy while I was writing this, nice work! One suggestion though, arranging the objects alphabetically in the tree (when within the same branch and level) would help ease navigation. P.P.S. Our Web server will be down briefly this afternoon, so you may get a timeout/cannot connect on the link. From d.haase at gsf.de Fri Jul 8 14:45:39 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Fri Jul 8 14:36:39 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Hi Mark, I would not call it inconsistency, they totally contradict ;-) Consider a service which returns an unpredictable number of database entries. The result set does not form any meaningful construct, each member is just an alternative answer to the query posed. Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. If I summed up both replies correctly, the contradiction is obvious, isn't it? dirk -----Original Message----- From: moby-dev-bounces@portal.open-bio.org on behalf of Mark Wilkinson Sent: Fri 7/8/2005 7:22 PM To: Ernst, Rebecca Cc: Martin Senger; mobydev Subject: Re: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 I see no inconsistency between what you quote below and what I said in my mail from yesterday... ?? M On Fri, 2005-07-08 at 08:50 +0200, Rebecca Ernst wrote: > Hi Mark! > > I will have to spank you a lot for either the last email or the one you > sent me in February where I asked you how to use collections (see mail > below): > > > Well... the answer depends on what the generic case of your output will > look like. the rule is that you have to register every output object > that your service produces; i.e. every immediate child tag of a mobyData > block. Therefore, if you are going to consistently output exactly 3 > Genbank/gi's, then you would register your service as outputting 3 X > Object(NCBI_gi). If you are outputting an indeterminate number of > objects, then you register it as outputting 1 X Collection. > > Collections map nicely onto the rdf:Bag structure, if that helps you > interpret them. > > M > > > > On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: > > >> Hi Mark! > >> > >> We have problems interpreting the BioMoby API for the output objects. > >> > >> The thing is that we have a service that gets ONE input object and > >> executes a freetext search over several databases. This service will > >> eventually return more that one result. > >> We don't want to put the result into a collection object because the > >> single objects don't build any entity. > >> We also can't give back more that one mobyData block because we only > >> have one query ID and therefore only one queryID to give back. > >> > >> The only solution we can think of is an object like this: > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> would this be a legal object or do we have to use collection even if the > >> objects don't build an entity ? > >> > >> > >> Cheers, > >> Rebecca > > > > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical > Genetics, UBC, Canada > > > ----------------------------------------------------------------------------------------------------------- > > > > > > > Mark Wilkinson wrote: > > >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > > > > > >>I myself am confused about the use of collections. Originally I had in > >>my mind that Collections were a construct to allow objects that > >>inherently belong together to be "bagged". > >> > >> > > > >You are 100% correct here. > > > > > > > > > >>The confusion starts with the output of services. My understanding was > >>that ONLY a service that is guaranteed to output exactly one object for > >>each query (e.g. an averaging service that outputs the average of a > >>list of inputs) is registered as outputting a Simple, all others have > >>to output collections > >> > >> > > > >No. > > > > > > > >> (as there must be exactly one mobyData matching > >>the queryID of the input in the response, > >> > >> > > > >Yes > > > > > > > >> and a mobyData may contain > >>multiple Simple elements only if wrapped by a Collection). > >> > >> > > > >No. A mobyData element can wrap any number of outputs, and/or > >combinations of simples, collections, and secondaries. > > > > > > > > > > > >>In practice, the Collection tag has been used to indicate when more > >>than one Simple occurs, with no "semantic" meaning. This imho is not > >>necessary; when more than one Simple occurs, why not put more than one > >>Simple? It's easy enough for everyone to figure that out. Then, > >>Collection could be used to actually transfer meaning ;-) > >> > >> > > > >I think a lot of people are using Collections incorrectly, yes. > > > > > > > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back > >>many Simple articles in response to a query very intuitively indicates > >>to continue on with each one individually, whereas getting back a > >>Collection indicates to put the whole thing as input into the next > >>service, which is what they implemented. Makes perfect sense, as there > >>can and will be services that consume Collections. > >> > >> > > > >I don't think that Taverna handled collections correctly in the past, > >and it would be a shame if that identical functionality was added back > >in :-) The functionality does need to go back into Taverna, but > >correctly this time. > > > > > > > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue > >>has led to quite some discussions between Rebecca, Dirk and myself, and > >>we are all not happy with the way they are currently handled. > >> > >> > > > >I don't know if you are happier with them now that I have pointed out > >where you are misinterpreting them...?? > > > >I agree, however, that their usage by many service providers is not > >correct... but there's no way for MOBY Central to know that it isn't > >correct, so we can't throw an error on registration of these "wonky" > >services... > > > >M > > > > > > > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From d.haase at gsf.de Fri Jul 8 14:45:39 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Fri Jul 8 14:36:41 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Hi Mark, I would not call it inconsistency, they totally contradict ;-) Consider a service which returns an unpredictable number of database entries. The result set does not form any meaningful construct, each member is just an alternative answer to the query posed. Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. If I summed up both replies correctly, the contradiction is obvious, isn't it? dirk -----Original Message----- From: moby-dev-bounces@portal.open-bio.org on behalf of Mark Wilkinson Sent: Fri 7/8/2005 7:22 PM To: Ernst, Rebecca Cc: Martin Senger; mobydev Subject: Re: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 I see no inconsistency between what you quote below and what I said in my mail from yesterday... ?? M On Fri, 2005-07-08 at 08:50 +0200, Rebecca Ernst wrote: > Hi Mark! > > I will have to spank you a lot for either the last email or the one you > sent me in February where I asked you how to use collections (see mail > below): > > > Well... the answer depends on what the generic case of your output will > look like. the rule is that you have to register every output object > that your service produces; i.e. every immediate child tag of a mobyData > block. Therefore, if you are going to consistently output exactly 3 > Genbank/gi's, then you would register your service as outputting 3 X > Object(NCBI_gi). If you are outputting an indeterminate number of > objects, then you register it as outputting 1 X Collection. > > Collections map nicely onto the rdf:Bag structure, if that helps you > interpret them. > > M > > > > On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: > > >> Hi Mark! > >> > >> We have problems interpreting the BioMoby API for the output objects. > >> > >> The thing is that we have a service that gets ONE input object and > >> executes a freetext search over several databases. This service will > >> eventually return more that one result. > >> We don't want to put the result into a collection object because the > >> single objects don't build any entity. > >> We also can't give back more that one mobyData block because we only > >> have one query ID and therefore only one queryID to give back. > >> > >> The only solution we can think of is an object like this: > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> would this be a legal object or do we have to use collection even if the > >> objects don't build an entity ? > >> > >> > >> Cheers, > >> Rebecca > > > > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical > Genetics, UBC, Canada > > > ----------------------------------------------------------------------------------------------------------- > > > > > > > Mark Wilkinson wrote: > > >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > > > > > >>I myself am confused about the use of collections. Originally I had in > >>my mind that Collections were a construct to allow objects that > >>inherently belong together to be "bagged". > >> > >> > > > >You are 100% correct here. > > > > > > > > > >>The confusion starts with the output of services. My understanding was > >>that ONLY a service that is guaranteed to output exactly one object for > >>each query (e.g. an averaging service that outputs the average of a > >>list of inputs) is registered as outputting a Simple, all others have > >>to output collections > >> > >> > > > >No. > > > > > > > >> (as there must be exactly one mobyData matching > >>the queryID of the input in the response, > >> > >> > > > >Yes > > > > > > > >> and a mobyData may contain > >>multiple Simple elements only if wrapped by a Collection). > >> > >> > > > >No. A mobyData element can wrap any number of outputs, and/or > >combinations of simples, collections, and secondaries. > > > > > > > > > > > >>In practice, the Collection tag has been used to indicate when more > >>than one Simple occurs, with no "semantic" meaning. This imho is not > >>necessary; when more than one Simple occurs, why not put more than one > >>Simple? It's easy enough for everyone to figure that out. Then, > >>Collection could be used to actually transfer meaning ;-) > >> > >> > > > >I think a lot of people are using Collections incorrectly, yes. > > > > > > > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back > >>many Simple articles in response to a query very intuitively indicates > >>to continue on with each one individually, whereas getting back a > >>Collection indicates to put the whole thing as input into the next > >>service, which is what they implemented. Makes perfect sense, as there > >>can and will be services that consume Collections. > >> > >> > > > >I don't think that Taverna handled collections correctly in the past, > >and it would be a shame if that identical functionality was added back > >in :-) The functionality does need to go back into Taverna, but > >correctly this time. > > > > > > > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue > >>has led to quite some discussions between Rebecca, Dirk and myself, and > >>we are all not happy with the way they are currently handled. > >> > >> > > > >I don't know if you are happier with them now that I have pointed out > >where you are misinterpreting them...?? > > > >I agree, however, that their usage by many service providers is not > >correct... but there's no way for MOBY Central to know that it isn't > >correct, so we can't throw an error on registration of these "wonky" > >services... > > > >M > > > > > > > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 6191 bytes Desc: not available Url : http://biomoby.org/pipermail/moby-dev/attachments/20050708/e13cc75c/attachment.bin From markw at illuminae.com Fri Jul 8 14:58:30 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri Jul 8 14:49:23 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Message-ID: <1120849110.15014.101.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 20:45 +0200, Haase, Dirk wrote: > Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. What I said in response to Heiko (or at least, what I intended to say) was that there was not a limitation of the mobyData block to having just one child element (regardless of its type - simple, collection, or secondary). So the following is vaid: I didn't say that you SHOULD return all of your Simples individually, only that you COULD return individual Simples in a single output... if that was meaningful in the context of your service... individual outputs must, however, be individually registered in the registry (output => [A,B,Q]) > But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. That's right.... and obvious, since you cannot create a service signature any other way - you can't say output => [A....infinite] > If I summed up both replies correctly, the contradiction is obvious, isn't it? I don't see it. M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 14:58:30 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri Jul 8 14:49:24 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Message-ID: <1120849110.15014.101.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 20:45 +0200, Haase, Dirk wrote: > Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. What I said in response to Heiko (or at least, what I intended to say) was that there was not a limitation of the mobyData block to having just one child element (regardless of its type - simple, collection, or secondary). So the following is vaid: I didn't say that you SHOULD return all of your Simples individually, only that you COULD return individual Simples in a single output... if that was meaningful in the context of your service... individual outputs must, however, be individually registered in the registry (output => [A,B,Q]) > But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. That's right.... and obvious, since you cannot create a service signature any other way - you can't say output => [A....infinite] > If I summed up both replies correctly, the contradiction is obvious, isn't it? I don't see it. M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From d.haase at gsf.de Fri Jul 8 15:43:41 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Fri Jul 8 15:34:35 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> OK, things become clearer I think. You didn't really get what Heiko was suggesting about the usage of collections... Heiko, Rebecca and me think that a collection is ueseful _only_ if the collection itself is a meaningful entity (example: several sequences make up a cluster). In our view it does not make sense to pack otherwise unrelated things into a collection just because they are all results for a certain query. I admit that our understanding is not compatible with the current way for registering services. The easy(?) solution would be to abandon any cardinality confirmations from the registry. That is if a service is registered with an output A it may output any number of A-objects. Another way to tackle this would be to introduce a sort of 'has/has-a' for service outputs, analogous to the object definitions. dirk -----Original Message----- From: moby-dev-bounces@portal.open-bio.org on behalf of Mark Wilkinson Sent: Fri 7/8/2005 8:58 PM To: Core developer announcements Cc: Ernst, Rebecca; Martin Senger; mobydev Subject: RE: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 On Fri, 2005-07-08 at 20:45 +0200, Haase, Dirk wrote: > Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. What I said in response to Heiko (or at least, what I intended to say) was that there was not a limitation of the mobyData block to having just one child element (regardless of its type - simple, collection, or secondary). So the following is vaid: I didn't say that you SHOULD return all of your Simples individually, only that you COULD return individual Simples in a single output... if that was meaningful in the context of your service... individual outputs must, however, be individually registered in the registry (output => [A,B,Q]) > But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. That's right.... and obvious, since you cannot create a service signature any other way - you can't say output => [A....infinite] > If I summed up both replies correctly, the contradiction is obvious, isn't it? I don't see it. M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 4339 bytes Desc: not available Url : http://biomoby.org/pipermail/moby-dev/attachments/20050708/07d07189/attachment-0001.bin From markw at illuminae.com Fri Jul 8 16:07:25 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri Jul 8 15:58:16 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> Message-ID: <1120853245.15014.131.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 21:43 +0200, Haase, Dirk wrote: > Heiko, Rebecca and me think that a collection is ueseful _only_ if the collection > itself is a meaningful entity (example: several sequences make up a cluster). > In our view it does not make sense to pack otherwise unrelated things into a > collection just because they are all results for a certain query. I agree 100%... so again, I still don't see the difference between what you are saying and what I am saying...?? M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From jmfernandez at cnb.uam.es Fri Jul 8 17:55:44 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Fri Jul 8 17:46:38 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> Message-ID: <42CEF660.8010602@cnb.uam.es> Hi everybody, I had almost forgotten a MOBY service (like a PL/SQL or ADA procedure) can have more than one output, each one belonging to its own type. I forgot it for one of the services I designed for INB. > > I admit that our understanding is not compatible with the current way > for registering services. The easy(?) solution would be to abandon > any cardinality confirmations from the registry. That is if a service > is registered with an output A it may output any number of A-objects. > I have in mind two or three bioinformatics time-expensive algorithms which generate more than one output (with different types) each time they are run. Because they are related, the common developer wraps them into an object, whose type inherits from Object, and most of the time it is an artificial creation. Those coders must also develop micro-services which pull the needed piece of information from the wrap, or if you are a Taverna user something like a beanshell. So, I agree with Mark: services with more than one output have their place. But remember: we need clients which are able to deal with this feature. Until now we had no one, but now Taverna could be the first one... Best regards, Jos? Mar?a > > Another way to tackle this would be to introduce a sort of > 'has/has-a' for service outputs, analogous to the object definitions. > > > dirk > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez@cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From markw at illuminae.com Fri Jul 8 18:00:33 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri Jul 8 17:51:27 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <42CEF660.8010602@cnb.uam.es> References: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> <42CEF660.8010602@cnb.uam.es> Message-ID: <1120860033.3721.7.camel@bioinfo.icapture.ubc.ca> Yeah... it's taking a long time for the tooling to catch up with the API. Just a lack of resources :-( M On Fri, 2005-07-08 at 23:55 +0200, Jos? Mar?a Fern?ndez Gonz?lez wrote: > So, I agree with Mark: > services with more than one output have their place. But remember: we > need clients which are able to deal with this feature. Until now we had > no one, but now Taverna could be the first one... -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From d.haase at gsf.de Sat Jul 9 05:02:01 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Sat Jul 9 04:53:22 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B28B@sw-rz010.gsf.de> Hi Jose Maria, Mark and others, I was not talking about sophisticated services with unpredictable type(s) of output, only about plain stupid lookup services where the output type is fix, but the number of result objects unknown. Like 'getAGILocusCodesByKeyword' for example. I'll try to explain it in other words... Why not making it the default behaviour that services can output 1 or many objects of the type it is registered for and save the 'collection' construct for situations where several output objects (of same of differing types) constitute a semantically meaningful complex. Does anybody understand what I'm trying to say?? dirk -----Original Message----- From: moby-dev-bounces@portal.open-bio.org on behalf of Jos? Mar?a Fern?ndez Gonz?lez Sent: Fri 7/8/2005 11:55 PM To: Core developer announcements Cc: markw@illuminae.com Subject: Re: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Hi everybody, I had almost forgotten a MOBY service (like a PL/SQL or ADA procedure) can have more than one output, each one belonging to its own type. I forgot it for one of the services I designed for INB. > > I admit that our understanding is not compatible with the current way > for registering services. The easy(?) solution would be to abandon > any cardinality confirmations from the registry. That is if a service > is registered with an output A it may output any number of A-objects. > I have in mind two or three bioinformatics time-expensive algorithms which generate more than one output (with different types) each time they are run. Because they are related, the common developer wraps them into an object, whose type inherits from Object, and most of the time it is an artificial creation. Those coders must also develop micro-services which pull the needed piece of information from the wrap, or if you are a Taverna user something like a beanshell. So, I agree with Mark: services with more than one output have their place. But remember: we need clients which are able to deal with this feature. Until now we had no one, but now Taverna could be the first one... Best regards, Jos? Mar?a > > Another way to tackle this would be to introduce a sort of > 'has/has-a' for service outputs, analogous to the object definitions. > > > dirk > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez@cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 4479 bytes Desc: not available Url : http://biomoby.org/pipermail/moby-dev/attachments/20050709/d8bdfc80/attachment.bin From chris at bioteam.net Sat Jul 9 13:40:52 2005 From: chris at bioteam.net (Chris Dagdigian) Date: Sat Jul 9 13:33:13 2005 Subject: [MOBY-dev] DNS changes for biomoby.org beginning today - should have no noticeable effect Message-ID: Hi folks, All of open-bio DNS is outsourced to a third party except for biomoby.org for which we manage the DNS zone ourselves due to moby requirements for specialized SRV zone entries that our other provider can't handle. The existing primary and secondary nameservers for biomoby.org are: 7200 IN NS portal.open-bio.org. 7200 IN NS pub.open-bio.org. These also happen to be the same servers we are currently trying to "retire" in favor of newer servers running in a different datacenter colocation cage. Last night I copied the biomoby.org zone file to our two new servers and activated service for them. The new machines are: (1) newportal.open-bio.org (2) dev.open-bio.org This morning I edited the master nameserver list that NetworkSolutions holds for the domain to add those new servers at the top of the list. Biomoby now has 4 dns servers; listed in the following order: 7200 IN NS newportal.open-bio.org. 7200 IN NS dev.open-bio.org. 7200 IN NS portal.open-bio.org. 7200 IN NS pub.open-bio.org. This means that over the next 24-72 hours as various DNS caches expire, biomoby.org will start to be serviced from the new pair of nameservers with the "old" pair acting as backup. If this works smoothly I'll leave it alone for 1-week. After which I plan on editing the NetworkSolution records to remove the "old" servers from the list. That should complete the full transition from the old primary/secondary pair to the new pair. Because of the overlap (both new and old server pairs are serving up the same info for biomoby.org) I don't expect anyone or any MOBY service user to experience any problems. I just wanted to mention the DNS change in case a problem does occur. For the DNS savvy on this list, feel free to query the biomoby.org zone on the new machines to confirm that the info is being served correctly. Your zone currently looks like this: > $ORIGIN org. > biomoby 7200 IN SOA portal.open-bio.org. root-l.open- > bio.org. ( > 0705200503 114400 7200 950400 7200 ) > 7200 IN NS newportal.open-bio.org. > 7200 IN NS dev.open-bio.org. > 7200 IN NS portal.open-bio.org. > 7200 IN NS pub.open-bio.org. > 7200 IN A 65.246.187.176 > 7200 IN MX 10 biomoby.org. > $ORIGIN biomoby.org. > portal 7200 IN A 65.246.187.176 > www 7200 IN CNAME biomoby.org. > cvs 7200 IN A 65.246.187.182 > 7200 IN MX 10 biomoby.org. > _lsid._tcp IN SRV 1 0 80 mobycentral.icapture.ubc.ca. Regards, Chris From schoof at mpiz-koeln.mpg.de Wed Jul 13 08:27:42 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Wed Jul 13 08:19:23 2005 Subject: [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42CEC551.4090108@ucalgary.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> Message-ID: <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> Hi Paul, thanks for the nice writeup. I agree. I just want to stress that I find one of the consequences you mention a very important argument for doing this: Services that act on Strings will no longer automatically be discovered starting from e.g. NCBI_BLAST_OUTPUT or DNA_SEQUENCE, but only when using object decomposition. Best, Heiko On 8. Jul 2005, at 20:26 Uhr, Paul Gordon wrote: Hi everyone, At long last after our discussions at the last MOBY meeting in Vancouver, I have written a little document explaining the reason for and consequences of not inheriting from "primitive" objects such as String and Integer. I think that this is a distruptive change for only a few services, but is of great benefit. Please see http://coe02.ucalgary.ca/gordonp/moby_primitives.html Mark has suggested that this not be a formal change to the API, but that this new way of doing things be encouraged in all of the documentation, and the old method explicitly deprecated but not disabled. Hopefully most of you agree, and we can just make temporary exceptions in our code for the current non-compliant classes. Or even better, if most people agree, we can force the affected services to change over. :-) P.S. Yan, your object browser came in quite handy while I was writing this, nice work! One suggestion though, arranging the objects alphabetically in the tree (when within the same branch and level) would help ease navigation. P.P.S. Our Web server will be down briefly this afternoon, so you may get a timeout/cannot connect on the link. _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From markw at illuminae.com Wed Jul 13 11:21:13 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed Jul 13 11:12:00 2005 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> Message-ID: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> The "virtual decomposition" of objects that MOBY does during service discovery can be switched on/off. Switching it off will prevent this behaviour also if it is an immediate concern. M On Wed, 2005-07-13 at 14:27 +0200, Heiko Schoof wrote: > Hi Paul, > thanks for the nice writeup. I agree. > > I just want to stress that I find one of the consequences you mention a > very important argument for doing this: Services that act on Strings > will no longer automatically be discovered starting from e.g. > NCBI_BLAST_OUTPUT or DNA_SEQUENCE, but only when using object > decomposition. > > Best, Heiko > > On 8. Jul 2005, at 20:26 Uhr, Paul Gordon wrote: > > Hi everyone, > > At long last after our discussions at the last MOBY meeting in > Vancouver, I have written a little document explaining the reason for > and consequences of not inheriting from "primitive" objects such as > String and Integer. I think that this is a distruptive change for only > a few services, but is of great benefit. Please see > > http://coe02.ucalgary.ca/gordonp/moby_primitives.html > > Mark has suggested that this not be a formal change to the API, but > that this new way of doing things be encouraged in all of the > documentation, and the old method explicitly deprecated but not > disabled. Hopefully most of you agree, and we can just make temporary > exceptions in our code for the current non-compliant classes. Or even > better, if most people agree, we can force the affected services to > change over. :-) > > P.S. Yan, your object browser came in quite handy while I was writing > this, nice work! One suggestion though, arranging the objects > alphabetically in the tree (when within the same branch and level) > would help ease navigation. > > P.P.S. Our Web server will be down briefly this afternoon, so you may > get a timeout/cannot connect on the link. > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From markw at illuminae.com Wed Jul 13 11:21:13 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed Jul 13 11:12:03 2005 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> Message-ID: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> The "virtual decomposition" of objects that MOBY does during service discovery can be switched on/off. Switching it off will prevent this behaviour also if it is an immediate concern. M On Wed, 2005-07-13 at 14:27 +0200, Heiko Schoof wrote: > Hi Paul, > thanks for the nice writeup. I agree. > > I just want to stress that I find one of the consequences you mention a > very important argument for doing this: Services that act on Strings > will no longer automatically be discovered starting from e.g. > NCBI_BLAST_OUTPUT or DNA_SEQUENCE, but only when using object > decomposition. > > Best, Heiko > > On 8. Jul 2005, at 20:26 Uhr, Paul Gordon wrote: > > Hi everyone, > > At long last after our discussions at the last MOBY meeting in > Vancouver, I have written a little document explaining the reason for > and consequences of not inheriting from "primitive" objects such as > String and Integer. I think that this is a distruptive change for only > a few services, but is of great benefit. Please see > > http://coe02.ucalgary.ca/gordonp/moby_primitives.html > > Mark has suggested that this not be a formal change to the API, but > that this new way of doing things be encouraged in all of the > documentation, and the old method explicitly deprecated but not > disabled. Hopefully most of you agree, and we can just make temporary > exceptions in our code for the current non-compliant classes. Or even > better, if most people agree, we can force the affected services to > change over. :-) > > P.S. Yan, your object browser came in quite handy while I was writing > this, nice work! One suggestion though, arranging the objects > alphabetically in the tree (when within the same branch and level) > would help ease navigation. > > P.P.S. Our Web server will be down briefly this afternoon, so you may > get a timeout/cannot connect on the link. > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From gordonp at ucalgary.ca Wed Jul 13 11:50:33 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed Jul 13 11:41:24 2005 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53849.7010700@ucalgary.ca> Mark, We are talking about decomposition. What you are talking about is inheritence. You can switch off finding services that match parent objects, of course. But an annotated base64 encoded GIF under the new scheme would NEVER match something that takes String as input, no matter how you set the service finding flag. The CLIENT would need to decompose (probably at the user's request) the GIF object into its constituent String members, such as the "contents" or the "description", and search MOBY central using JUST that member. That's what we mean by decomposition. >The "virtual decomposition" of objects that MOBY does during service >discovery can be switched on/off. Switching it off will prevent this >behaviour also if it is an immediate concern. > > > > From gordonp at ucalgary.ca Wed Jul 13 11:50:33 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed Jul 13 11:41:26 2005 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53849.7010700@ucalgary.ca> Mark, We are talking about decomposition. What you are talking about is inheritence. You can switch off finding services that match parent objects, of course. But an annotated base64 encoded GIF under the new scheme would NEVER match something that takes String as input, no matter how you set the service finding flag. The CLIENT would need to decompose (probably at the user's request) the GIF object into its constituent String members, such as the "contents" or the "description", and search MOBY central using JUST that member. That's what we mean by decomposition. >The "virtual decomposition" of objects that MOBY does during service >discovery can be switched on/off. Switching it off will prevent this >behaviour also if it is an immediate concern. > > > > From Pieter.Neerincx at wur.nl Wed Jul 13 11:53:41 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Wed Jul 13 11:44:30 2005 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: Hi all, Is there anyone out there that was able to make BioMOBY (realated) Java tools like Yan's Moby Object Browser or Taverna work with MOBY Centrals and/or services over HTTPS? I'm having trouble with the server's certificate. I used keytool to import the server's certificate client-side, but the Java tools keep on complaining: "unknown certificate"... Cheers, Pieter From tmo at ebi.ac.uk Wed Jul 13 12:05:55 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Wed Jul 13 11:54:58 2005 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53BE3.7000909@ebi.ac.uk> Hi all, We've had Taverna working for HTTPS web services so it should be fine for the MOBY operations. You need to check that the certificate has a valid signature chain and that the name of the entity in the certificate matches the hostname - we had issues with a certificate from a server in china that the java security framework rejected because the (I think) CN part of the DN in the cert wasn't the same as the hostname of the endpoint. Tom Pieter Neerincx wrote: > Hi all, > > Is there anyone out there that was able to make BioMOBY (realated) Java > tools like Yan's Moby Object Browser or Taverna work with MOBY Centrals > and/or services over HTTPS? I'm having trouble with the server's > certificate. I used keytool to import the server's certificate > client-side, but the Java tools keep on complaining: "unknown > certificate"... > > Cheers, > > Pieter _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev From tmo at ebi.ac.uk Wed Jul 13 12:05:55 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Wed Jul 13 11:54:59 2005 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53BE3.7000909@ebi.ac.uk> Hi all, We've had Taverna working for HTTPS web services so it should be fine for the MOBY operations. You need to check that the certificate has a valid signature chain and that the name of the entity in the certificate matches the hostname - we had issues with a certificate from a server in china that the java security framework rejected because the (I think) CN part of the DN in the cert wasn't the same as the hostname of the endpoint. Tom Pieter Neerincx wrote: > Hi all, > > Is there anyone out there that was able to make BioMOBY (realated) Java > tools like Yan's Moby Object Browser or Taverna work with MOBY Centrals > and/or services over HTTPS? I'm having trouble with the server's > certificate. I used keytool to import the server's certificate > client-side, but the Java tools keep on complaining: "unknown > certificate"... > > Cheers, > > Pieter _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev From markw at illuminae.com Wed Jul 13 12:16:27 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed Jul 13 12:07:14 2005 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42D53849.7010700@ucalgary.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> <42D53849.7010700@ucalgary.ca> Message-ID: <1121271387.6327.45.camel@bioinfo.icapture.ubc.ca> On Wed, 2005-07-13 at 09:50 -0600, Paul Gordon wrote: > We are talking about decomposition. What you are talking about is > inheritence. Yes, but I'm just saying that the *current* behaviour that Heiko was objecting to is due to inheritence, and the behaviour can be switched off if it is *immediately* troublesome. Of course, it will also disappear if we start to use a composition model rather than an inheritence model to put e.g. strings into objects, but that doesn't help anyone in the short-term. M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From markw at illuminae.com Wed Jul 13 12:16:27 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed Jul 13 12:07:40 2005 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42D53849.7010700@ucalgary.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> <42D53849.7010700@ucalgary.ca> Message-ID: <1121271387.6327.45.camel@bioinfo.icapture.ubc.ca> On Wed, 2005-07-13 at 09:50 -0600, Paul Gordon wrote: > We are talking about decomposition. What you are talking about is > inheritence. Yes, but I'm just saying that the *current* behaviour that Heiko was objecting to is due to inheritence, and the behaviour can be switched off if it is *immediately* troublesome. Of course, it will also disappear if we start to use a composition model rather than an inheritence model to put e.g. strings into objects, but that doesn't help anyone in the short-term. M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From Pieter.Neerincx at wur.nl Thu Jul 14 10:02:34 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Thu Jul 14 09:53:29 2005 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: <42D53BE3.7000909@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> <42D53BE3.7000909@ebi.ac.uk> Message-ID: <75FEE7B6-A0DE-4444-B27C-7C7C4F478FB9@wur.nl> Hi Tom, Thanks for the reply. No I at least knew it should be possible :). After googeling for days I finally got it to work. Maybe you can add a small addition for HTTPS to the configuration part of the Taverna manual. If anyone else ever runs into the same problem here is how I got it to work: I did find a lot of posts mentioning compatibility problems when importing openssl generated certificates in a java keystore with keytool. So I was converting my certificates in probably every certificate format known to man, but that was not the solution. In order to make the Java client accept a self-signed certificate from you webserver this is what needs to be done. Generate a self-signed certificate. If you use openssl this step is pretty well documented. The default output will be a certificate in pem format. The default extension for such a certificate is *.crt. This file can contain both a human readable form of the certificate and an encoded one. So it might look something like this: Certificate: Data: Version: 3 (0x2) Serial Number: 0 (0x0) Signature Algorithm: md5WithRSAEncryption Issuer: C=NL, ST=Gelderland, L=Wageningen, O=WUR, OU=Bioinformatics, CN=dev.bioinformatics.nl/ emailAddress=jack.leunissen@wur.nl Validity Not Before: Jul 8 16:27:14 2005 GMT Not After : Aug 7 16:27:14 2005 GMT Subject: C=NL, ST=Gelderland, L=Wageningen, O=WUR, OU=Bioinformatics, CN=dev.bioinformatics.nl/ emailAddress=jack.leunissen@wur.nl Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (1024 bit) Modulus (1024 bit): 00:a1:e2:f2:20:3c:17:da:c1:2c:d9:89:4b:9d:16: ........ 80:a9:e4:02:72:3c:1f:b3:3d Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Key Identifier: 87:F7:8D:B6:63:C1:F9:4D: X509v3 Authority Key Identifier: keyid:87:F7:8D:B6:63:C1:F9: DirName:/C=NL/ST=Gelderland/L=Wageningen/O=WUR/ OU=Bioinformatics/CN=dev.bioinformatics.nl/ emailAddress=jack.leunissen@wur.nl serial:00 X509v3 Basic Constraints: CA:TRUE Signature Algorithm: md5WithRSAEncryption 6e:5c:27:f4:b4:bd:1e:32:c0:ee:03:ce:76:43:c3:e8:3a:50: ........ 0c:e8:f6:98:10:2d:ac:ff:99:3a:5c:f5:f8:66:27:a5:53:c6: 5a:0b -----BEGIN CERTIFICATE----- MIIDxzCCAzCgAwIBAgIBADANBgkqhkiG9w0BAQQFADCBpDELMAkGA1UEBhMCTkwx EzARBgNVBAgTCkdlbGRlcmxhbmQxEzARBgNVBAcTCldhZ2VuaW5nZW4xDDAKBgNV BAoTA1dVUjEXMBUGA1UECxMOQmlvaW5mb3JtYXRpY3MxHjAcBgNVBAMTFWRldi5i ........ bDEkMCIGCSqGSIb3DQEJARYVamFjay5sZXVuaXNzZW5Ad3VyLm5sggEAMAwGA1Ud EwQFMAMBAf8wDQYJKoZIhvcNAQEEBQADgYEAblwn9LS9HjLA7gPOdkPD6DpQUBHF L/4Ihx+J6Y1PVr5NSgWLz5emr0wSqV+adjfYm/+OMV0AkYVQptlm2N61xX7YDbkL mEBS8n22L43f2WU4SP7J24kmhllet56QaS3ImfECP40RwvC+I+26DOj2mBAtrP+Z Olz1+GYnpVPGWgs= -----END CERTIFICATE----- The human readable part is a problem for keytool which you use to import the certificate in your Java keystore. You can convert the certificate in another format that is keytool compatible, but the easiest solution is to strip the human readable part from the certificate. Hence, simply copy -----BEGIN CERTIFICATE-----[encoded certificate]-----END CERTIFICATE----- to a new file. Make sure there are no blank lines left before -----BEGIN CERTIFICATE----- or after -----END CERTIFICATE-----. Now import the certificate in your keystore: keytool -import -trustcacerts -v -alias [some name for your cert.] -file [the cert. without the human readable part] I figured that since the certificate is self-signed, I wouldn't need another certificate authority to make java apps to validate the certificate, but that was wrong. You need to import the same self-signed certificate into your cacerts keystore as well: as root: keytool -import -v -storetype jks -keystore $JAVA_HOME/ lib/security/cacerts -file [the cert. without the human readable part] Now it should work. Cheers, Pieter On 13-Jul-2005, at 6:05 PM, Tom Oinn wrote: > Hi all, > > We've had Taverna working for HTTPS web services so it should be > fine for the MOBY operations. You need to check that the > certificate has a valid signature chain and that the name of the > entity in the certificate matches the hostname - we had issues with > a certificate from a server in china that the java security > framework rejected because the (I think) CN part of the DN in the > cert wasn't the same as the hostname of the endpoint. > > Tom > > Pieter Neerincx wrote: > >> Hi all, >> Is there anyone out there that was able to make BioMOBY >> (realated) Java tools like Yan's Moby Object Browser or Taverna >> work with MOBY Centrals and/or services over HTTPS? I'm having >> trouble with the server's certificate. I used keytool to import >> the server's certificate client-side, but the Java tools keep on >> complaining: "unknown certificate"... >> Cheers, >> Pieter _______________________________________________ >> MOBY-dev mailing list >> MOBY-dev@biomoby.org >> http://www.biomoby.org/mailman/listinfo/moby-dev >> > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > Wageningen University and Research centre (WUR) Laboratory of Bioinformatics Transitorium (building 312) room 1038 Dreijenlaan 3 6703 HA Wageningen phone: 0317-484 706 fax: 0317-483 584 mobile: 06-143 66 783 pieter.neerincx@wur.nl From senger at ebi.ac.uk Mon Jul 18 08:39:27 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Mon Jul 18 08:30:17 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <1120860033.3721.7.camel@bioinfo.icapture.ubc.ca> Message-ID: Hi all, Catching up my email piles I wonder if someone can summarize if the discussion about collections in this thread brought any (planned) changes in the API (I am not talking now about how it is, or should be, implemented in Taverna, that's, imho, an another story). My understanding is that (talking about one mobyData object): a) Any Moby service can have more outputs. If so, all of them must be registered. The number of such outputs must be fixed. b) Any of these outputs can be of type either Simple or Collection. c) If it is a Collection, this output can have one or more Simples in that Collection. Such Simples (and their number) are not individually registered. Has this vision been changed? Thanks, Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Mon Jul 18 11:09:47 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon Jul 18 11:00:24 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: References: Message-ID: <1121699387.15443.68.camel@bioinfo.icapture.ubc.ca> Hi Martin, That is a 100% correct description of the status quo (I would just re- word point c to say that the TYPE(s) of Simple(s) in the Collection are registered, but not their number) I have not been able to discern from any of the discussions what it is that others are objecting to, so I am a bit lost...?? I don't know if an API change is being requested, or if there is simply a widespread mis-understanding of how Collections are to be used. Your description, however, is completely correct. M On Mon, 2005-07-18 at 13:39 +0100, Martin Senger wrote: > Hi all, > Catching up my email piles I wonder if someone can summarize if the > discussion about collections in this thread brought any (planned) changes > in the API (I am not talking now about how it is, or should be, > implemented in Taverna, that's, imho, an another story). > My understanding is that (talking about one mobyData object): > a) Any Moby service can have more outputs. If so, all of them must be > registered. The number of such outputs must be fixed. > b) Any of these outputs can be of type either Simple or Collection. > c) If it is a Collection, this output can have one or more Simples in > that Collection. Such Simples (and their number) are not individually > registered. > > Has this vision been changed? > > Thanks, > Martin > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From rebecca.ernst at gsf.de Mon Jul 18 11:14:04 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Mon Jul 18 11:04:54 2005 Subject: [MOBY-dev] buggy new biomoby release Message-ID: <42DBC73C.5030302@gsf.de> Hi Mark (and rest)! As suggested Dirk and I sat down today to do a cvs update to get the last stable release. Now it's 5 hours later and we gave up and switched back to the last working code we had (and which we luckily saved before doing the update! :-)) There are several problems keeping us from updating our code: First of all the links on the release site (biomoby.org/releases) are crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't notice this and therefore received the older version. This version is VERY outdated and not functional (e.g.it uses DOM instead of LibXML) Then we modified the link to get 0.8.2. This version is also not functional as it uses lsid (although this was meant to be the version BEFORE the lsid change). Even when we added a column lsid to the database it didn't work (I guess one would also need the lsid resolver and so on.) Another try was to do an cvs update which crashed everything as completely buggy and syntactical wrong code was checked in (especially in Adaptor/.../mysql.pm) So luckily we could rescue everything because of our save but I guess this is not the way to make people happy installing BioMoby :-( Could we please make sure that releases are functional and the code in the cvs is not complete crap.... otherwise I'll have to do some more spanking ;-) Cheers, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de From markw at illuminae.com Mon Jul 18 11:23:51 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon Jul 18 11:15:07 2005 Subject: [MOBY-dev] buggy new biomoby release In-Reply-To: <42DBC73C.5030302@gsf.de> References: <42DBC73C.5030302@gsf.de> Message-ID: <42DBC987.2000401@illuminae.com> Hi Rebecca, The CVS version certainly wont work (as I noted on the list last week). The link to 0.8.2 was wrong, as you say - sorry about that! The 0.8.2 version should work, however, as I set it up on a completely virgin machine (the current MOBY Central public server) immediately before making the release and it worked perfectly...? Can you be more specific with your error report? v.v. the LSID libraries - it is possible that the requirement for the LSID libraries is in a "use" statement somewhere, but it definitely isn't a real requirement for the 0.8.2 code. If you see a "use" in there, or in the makefile, that is an error for sure. Can you tell me exactly what errors you were seeing? Cheers! M Rebecca Ernst wrote: > Hi Mark (and rest)! > > As suggested Dirk and I sat down today to do a cvs update to get the > last stable release. > > Now it's 5 hours later and we gave up and switched back to the last > working code we had (and which we luckily saved before doing the > update! :-)) > > There are several problems keeping us from updating our code: > > First of all the links on the release site (biomoby.org/releases) are > crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't > notice this and therefore received the older version. This version is > VERY outdated and not functional (e.g.it uses DOM instead of LibXML) > > Then we modified the link to get 0.8.2. This version is also not > functional as it uses lsid (although this was meant to be the version > BEFORE the lsid change). Even when we added a column lsid to the > database it didn't work (I guess one would also need the lsid resolver > and so on.) > > Another try was to do an cvs update which crashed everything as > completely buggy and syntactical wrong code was checked in (especially > in Adaptor/.../mysql.pm) > > > So luckily we could rescue everything because of our save but I guess > this is not the way to make people happy installing BioMoby :-( > > Could we please make sure that releases are functional and the code in > the cvs is not complete crap.... otherwise I'll have to do some more > spanking ;-) > > > Cheers, > Rebecca > > > > > > From markw at illuminae.com Mon Jul 18 11:39:32 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon Jul 18 11:30:08 2005 Subject: [MOBY-dev] Re: [moby] buggy new biomoby release In-Reply-To: <42DBC73C.5030302@gsf.de> References: <42DBC73C.5030302@gsf.de> Message-ID: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Doh! I missed one. There is an (unused) call to LS::ID->new in the code. Unfortunately, I do have the LSID libraries installed so I didn't pick up on that. I have commented-out the line in the release codebase and will make another release, but before I do I want to make sure that this was the only problem you saw. Let me know, M On Mon, 2005-07-18 at 17:14 +0200, Rebecca Ernst wrote: > Hi Mark (and rest)! > > As suggested Dirk and I sat down today to do a cvs update to get the > last stable release. > > Now it's 5 hours later and we gave up and switched back to the last > working code we had (and which we luckily saved before doing the update! > :-)) > > There are several problems keeping us from updating our code: > > First of all the links on the release site (biomoby.org/releases) are > crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't > notice this and therefore received the older version. This version is > VERY outdated and not functional (e.g.it uses DOM instead of LibXML) > > Then we modified the link to get 0.8.2. This version is also not > functional as it uses lsid (although this was meant to be the version > BEFORE the lsid change). Even when we added a column lsid to the > database it didn't work (I guess one would also need the lsid resolver > and so on.) > > Another try was to do an cvs update which crashed everything as > completely buggy and syntactical wrong code was checked in (especially > in Adaptor/.../mysql.pm) > > > So luckily we could rescue everything because of our save but I guess > this is not the way to make people happy installing BioMoby :-( > > Could we please make sure that releases are functional and the code in > the cvs is not complete crap.... otherwise I'll have to do some more > spanking ;-) > > > Cheers, > Rebecca > > > > > > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From edward.kawas at gmail.com Mon Jul 18 20:38:10 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Mon Jul 18 20:29:51 2005 Subject: [MOBY-dev] request for community feedback on certain issues In-Reply-To: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Message-ID: <42dc4b78.5905bfd9.1943.793b@mx.gmail.com> Hi 'BioMoby'ers, Below I am placing some questions, comments, etc that we would like your feedback on. Hopefully some time soon it will be placed on a twiki! I also attached the file in html format in case this email is too hard to read. Eddie ---Start--- Request for Feedback from the BioMoby Community! Before I begin, I would just like to say that the BioMoby Project is a open source, community driven project. As such, it is key that if the community doesn't like how something is done, then the community should fix it. If something is broken, doesn't work as intended or is vague, speak up and make your voice heard! BioMoby is not a one man project, but a community driven one. Things will only get better if we know what is wrong and if people take initiative. So I (Eddie) just got back from the INBs Technical Meeting in beautiful Malaga, Spain and I have some items that the community needs to talk about below. Also, be sure to keep an eye out for proposals about handling asynchronous services as well as error handling in Moby. When they come, please speak up with your likes, dislikes and ways to make things better. 1. API addition of a method called getResourcesURL: Returns an xml document that describes the location of RESOURCES script that we hear so much about. It is the resources script that contains the ontologies expressed in RDF/XML example: Services http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Services Objects http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Objects ... 2. How do I know what predicates are available in the RDF documents that describe the Moby ontologies? a. should we provide a list of predicates and their intended meanings (on a wiki, etc)? b. should we provide an RDF schema defining the predicates (* probably the way to go) c. Should we allow the registration of new predicates, if so, should we make this an api call, etc. 3. LSID resolution services a. is the LSID attribute available and documented? b. when will this occur, i have heard so much about this, yet i have seen nothing. 4. LSIDs versus simple names a. why is the name of an object, service, or service type is sometimes a simple name and sometimes an LSID? b. how can i even retrieve the LSID of an object, service, or service type? c. should api calls be added to getName and getLSID for objects, services and service types? 5. LSIDs and Service Instances a. If a service instance is registered by 2 different authorities, should they have the same LSID? b. if the service instance is registered in 2 registries, should the LSIDs be the same? c. will modified service instances have versioning information? 6. LSID resolution for objects in the BioMoby ontologies a. what should getData return for i.namespaces (the name of the namespace?) ii. service types (the name of the service type?) iii. objects (xml schema? - note that schemas for datatypes should be ready and available very soon :-) iv. service instances (the name of the instance?) b. should we allow the namespace part of an LSID to be configurable? c. should we allow the authority part of an LSID to be configurable? 7. Would you like to have 3rd party annotation of metadata? a. use case: adds mirroring b. use case: allows people to provide additional metadata i. if you want only one source for metadata, then ignore this question. 9. Predicates a. should we add the following predicates to the metadata of service instances? i. sampleInputData - specifies valid input for the service instance ii. sampleOutputData - specifies the output for the service instance based on the test input. iii. approvedBy - specifies whether this service has been approved. Currently, this would only be useful for the INB, but many more may find it useful. b. how should the sample input/output be formatted? c. should we allow the annotation of datatypes? 10. API and Code versioning a. why are the api and code versions different? 11. Datatype naming a. do you think that the names of datatypes should be regulated? i. what would be the right way to specify a datatypes name? ii. what should we do with text_plain and text-plain? 12. RDF Agent a. do you think that there should be an api call, getUpdate that tells the agent to check for modifications to the service instance described by the metadata in the RDF document at the signatureURL? b. what is the time line to get the agent running? c. besides curation, how is the agent useful to me? 13. Service a. should article names for service instance input and output datatypes be mandatory? b. why does a service that advertise an output datatype called 'foo' return 'bar' instead? ---END--- -------------- next part -------------- Request for Feedback from the BioMoby Community! Before I begin, I would just like to say that the BioMoby Project is a open source, community driven project. As such, it is key that if the should vague, project, we know what So I (Eddie) just got back from the INBs Technical Meeting in beautiful Malaga, Spain and I have some items that the community needs Moby. 1. API addition of a method called getResourcesURL: Returns an xml document that describes the location of RESOURCES script that we hear so much about. It is the resources script that contains the ontologies expressed in RDF/XML example: Services http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY-S/Services Objects http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY-S/Objects ... 2. How do I know what predicates are available in the RDF documents describe the Moby ontologies? a. should we provide a list of predicates and their intended meanings (on a wiki, etc)? b. should we provide an RDF schema defining the predicates (* probably the way to go) c. Should we allow the registration of new predicates, if so, should we make this an api call, etc. 3. LSID resolution services a. is the LSID attribute available and documented? b. when will this occur, i have heard so much about this, yet i have seen nothing. 4. LSIDs versus simple names a. why is the name of an object, service, or service type is sometimes a simple name and sometimes an LSID? b. how can i even retrieve the LSID of an object, service, or service type? c. should api calls be added to getName and getLSID for objects, services and service types? 5. LSIDs and Service Instances a..If a service instance is registered by 2 different authoritys, should they have the same LSID? b. if the service instance is registered in 2 registries, should the LSIDs be the same? c. will modified service instances have versioning information? 6. LSID resolution for objects in the BioMoby ontologies a. what should getData return for i.namespaces (the name of the namespace?) ii. service types (the name of the service type?) iii. objects (xml schema? - note that schemas for datatypes should be ready and available very soon :-) iv. service instances (the name of the instance?) b. should we allow the namespace part of an LSID to be configurable? c. should we allow the authority part of an LSID to be configurable? 7. Would you like to have 3rd party annotation of metadata? a. use case: adds mirroring b. use case: allows people to provide additional metadata i. if you want only one source for metadata, then ignore this question. 9. Predicates a. should we add the following predicates to the metadata of service instances? i. sampleInputData - specifies valid input for the service instance ii. sampleOutputData - specifies the output for the service instance based on the test input. iii. approvedBy - specifies whether this service has been approved. Currently, this would only be useful for the INB, but many more may find it useful. b. how should the sample input/output be formatted? c. should we allow the annotation of datatypes? 10. API and Code versioning a. why are the api and code versions different? 11. Datatype naming a. do you think that the names of datatypes should be regulated? i. what would be the right way to specify a datatypes name? ii. what should we do with text_plain and text-plain? 12. RDF Agent a. do you think that there should be an api call, getUpdate that tells the agent to check for modifications to the service instance signatureURL? b. what is the time line to get the agent running? c. besides curation, how is the agent useful to me? 13. Service a. should article names for service instance input and output datatypes be mandatory? b. why does a service that advertise an output datatype called 'foo' return 'bar' instead? From senger at ebi.ac.uk Tue Jul 19 05:18:58 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue Jul 19 05:09:48 2005 Subject: [MOBY-dev] request for community feedback on certain issues In-Reply-To: <42dc4b78.5905bfd9.1943.793b@mx.gmail.com> Message-ID: Hi Eddie, > BioMoby is not a one man project, but a community driven one. > Well, let's hope it is :-) Services are trully distributed, and some clients accessing Moby Central as well. The registry itself still seems to be a one-man area (imho). > 1. API addition of a method called getResourcesURL: > I understand that this is not a new requirement (we all asked for it many times) but this is a concrete suggestion how to make it. I agree that getting these URLs from an API call is cleaner than just document how such URLs should look like. Only I would suggest slightly different XML response for that (but no strong opinion on my side about it). What we need are actually name-value pairs, each of them representing a name of a resource (such as 'services; and a URL for the RDF describing such resource. So why not just use two attributes for that? Something like this: ... Cheers, Martin PS. Eddie, you should perhaps consider to send everything again, but splitted into few separate topics - that would be easier to find in our mailing lists archives... M. -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From rebecca.ernst at gsf.de Tue Jul 19 07:22:15 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue Jul 19 07:13:05 2005 Subject: [MOBY-dev] Re: [moby] buggy new biomoby release In-Reply-To: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> References: <42DBC73C.5030302@gsf.de> <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Message-ID: <42DCE267.8020609@gsf.de> Hi Mark! We tried once more and installed the new release (0.8.2). The problems we can see are that our perl services are not working anymore ("service returned no response") the bad thing is that it doesn't give any error so we have no clue where the problem is. Gbrowse works - the services can be listed and non-MIPS services and MIPS Java services can be executed. So Moby Central obviously works okay since (since we added the column lsid to the db) We then tried to remove the method "_getServiceInstanceRDF" in the MOBY/Central.pm because in there it uses lsid but this didn't solve anything. Just in case we find the problem and you build a new release I have some more suggestions leading to even more user-friendlyness ;-) : - include all libraries not only the perl libraries because all former releases contained everything and now we have to get the other libs from older releases and that means copying it. I guess that most people would expect that they get a full release otherwise consider renaming it (not biomoby release but perl_lib_release) - don't rename the moby-live path to moby-live_API_releasewhatever it now requires that people move it (or adjust their pathes) - next to the link (for downloading the release) add the information that for database provider they will have to change their database ("alter table service_instance add lsid VARCHAR (255) NOT NULL default ''; ") Cheers! Rebecca Mark Wilkinson wrote: >Doh! I missed one. > >There is an (unused) call to LS::ID->new in the code. Unfortunately, I >do have the LSID libraries installed so I didn't pick up on that. > >I have commented-out the line in the release codebase and will make >another release, but before I do I want to make sure that this was the >only problem you saw. > >Let me know, > >M > > > > >On Mon, 2005-07-18 at 17:14 +0200, Rebecca Ernst wrote: > > >>Hi Mark (and rest)! >> >>As suggested Dirk and I sat down today to do a cvs update to get the >>last stable release. >> >>Now it's 5 hours later and we gave up and switched back to the last >>working code we had (and which we luckily saved before doing the update! >>:-)) >> >>There are several problems keeping us from updating our code: >> >>First of all the links on the release site (biomoby.org/releases) are >>crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't >>notice this and therefore received the older version. This version is >>VERY outdated and not functional (e.g.it uses DOM instead of LibXML) >> >>Then we modified the link to get 0.8.2. This version is also not >>functional as it uses lsid (although this was meant to be the version >>BEFORE the lsid change). Even when we added a column lsid to the >>database it didn't work (I guess one would also need the lsid resolver >>and so on.) >> >>Another try was to do an cvs update which crashed everything as >>completely buggy and syntactical wrong code was checked in (especially >>in Adaptor/.../mysql.pm) >> >> >>So luckily we could rescue everything because of our save but I guess >>this is not the way to make people happy installing BioMoby :-( >> >>Could we please make sure that releases are functional and the code in >>the cvs is not complete crap.... otherwise I'll have to do some more >>spanking ;-) >> >> >>Cheers, >>Rebecca >> >> >> >> >> >> >> >> -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de From rebecca.ernst at gsf.de Tue Jul 19 09:15:44 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue Jul 19 09:06:46 2005 Subject: [MOBY-dev] NO buggy new biomoby release In-Reply-To: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> References: <42DBC73C.5030302@gsf.de> <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Message-ID: <42DCFD00.5020306@gsf.de> Hey Mark! Congratulations - you won a round of spanking ... You can spank Dirk and me. Shame on us - the release you provided was as perfect as one would expect ;-) The problem why the services didn't work anymore was because of one module (LoggingSubs.pm) which we use within Planet for logging of the services. This was (how ugly!) deployed in the moby-live directory. As the LoggingSubs.pm was not a part of the release this was missing and caused the services to fail. The call to LS::ID didn't matter at all you can leave everything (although you might take the suggestions into account which I mentioned in the last mail) Sorry again for suspecting you ;-) Cheers! Rebecca Mark Wilkinson wrote: >Doh! I missed one. > >There is an (unused) call to LS::ID->new in the code. Unfortunately, I >do have the LSID libraries installed so I didn't pick up on that. > >I have commented-out the line in the release codebase and will make >another release, but before I do I want to make sure that this was the >only problem you saw. > >Let me know, > >M > > > > >On Mon, 2005-07-18 at 17:14 +0200, Rebecca Ernst wrote: > > >>Hi Mark (and rest)! >> >>As suggested Dirk and I sat down today to do a cvs update to get the >>last stable release. >> >>Now it's 5 hours later and we gave up and switched back to the last >>working code we had (and which we luckily saved before doing the update! >>:-)) >> >>There are several problems keeping us from updating our code: >> >>First of all the links on the release site (biomoby.org/releases) are >>crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't >>notice this and therefore received the older version. This version is >>VERY outdated and not functional (e.g.it uses DOM instead of LibXML) >> >>Then we modified the link to get 0.8.2. This version is also not >>functional as it uses lsid (although this was meant to be the version >>BEFORE the lsid change). Even when we added a column lsid to the >>database it didn't work (I guess one would also need the lsid resolver >>and so on.) >> >>Another try was to do an cvs update which crashed everything as >>completely buggy and syntactical wrong code was checked in (especially >>in Adaptor/.../mysql.pm) >> >> >>So luckily we could rescue everything because of our save but I guess >>this is not the way to make people happy installing BioMoby :-( >> >>Could we please make sure that releases are functional and the code in >>the cvs is not complete crap.... otherwise I'll have to do some more >>spanking ;-) >> >> >>Cheers, >>Rebecca >> >> >> >> >> >> >> >> -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de From senger at ebi.ac.uk Tue Jul 19 10:00:20 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue Jul 19 09:50:53 2005 Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <42D39162.1090004@ac.upc.es> Message-ID: Hi, Several small changes have been done to jMoby (all of them should be backward compatible): * The default location and default namespace (URI) of Moby Central was updated (to ...icapture...). Actually, this change has been around already for some time but Eddie never announced it nor logged it in ChangeLog :-) * Roman found a design flaw in the getServiceNames() method returning Map where the keys are service names. This was wrong because there may be (and actually are, e.g. service genbankToGene) more services with the same name but served by different authorities. So I added a new method getServiceNamesByAuthorities() to the interface Central.java (and to its implementation CentralImpl.java), and I deprecated the old method (but it is still there so your code does not break). In future, you should use the new method. This also allowed to add a new command-line option '-la' to the MobyCmdLineClient. Try: build/run/run-cmdline-client -la and you get a list of all services grouped by authorities now. * The location and maintainance of the repository with the third-party libraries for jMoby is now fully "de-sengerized" (as requested in Vancouver). It is described in http://www.biomoby.org/moby-live/Java/docs/3rdPartyLibraries.html. What you should do is the following: cd moby-live/Java ./build-dev.sh cleanall ./build.sh This will populate your 'lib' directory with the same libraries as you have now, but from a different repository (it's just one time job). With regards, Martin PS. In the hopefully near future I plan to make additional changes/fixes: * fix bug (my misunderstanding) with the tag expandobject * added ways how to get RDF resources (Mark, I am desperately waiting for this one - in Malaga last week we created with Eddie a suggestion how to get URLs from the Central API...) * fix bug in the MobyGraphs (Ben, as you see, I am getting close to it...) M. -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From edward.kawas at gmail.com Tue Jul 19 10:08:56 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:00:12 2005 Subject: [MOBY-dev] request for community feedback on certain issues In-Reply-To: Message-ID: <42dd097b.4d90ae57.63bd.2f19@mx.gmail.com> Hi Once again, I have added the page to a twiki, at http://www.biomoby.org/twiki/bin//view/General/FeedBackDoc I have also added Martins comments to it. If this isn't good enough, and you do prefer to have the items split up, then I will do it. Eddie > -----Original Message----- > From: moby-dev-bounces@portal.open-bio.org [mailto:moby- > dev-bounces@portal.open-bio.org] On Behalf Of Martin > Senger > Sent: Tuesday, July 19, 2005 2:19 AM > To: Core developer announcements > Subject: Re: [MOBY-dev] request for community feedback on > certain issues > > Hi Eddie, > > > BioMoby is not a one man project, but a community driven > one. > > > Well, let's hope it is :-) > Services are trully distributed, and some clients > accessing Moby > Central as well. The registry itself still seems to be a > one-man area > (imho). > > > 1. API addition of a method called getResourcesURL: > > > I understand that this is not a new requirement (we > all asked for it > many times) but this is a concrete suggestion how to make > it. I agree that > getting these URLs from an API call is cleaner than just > document how such > URLs should look like. > Only I would suggest slightly different XML response > for that (but no > strong opinion on my side about it). What we need are > actually name-value > pairs, each of them representing a name of a resource > (such as 'services; > and a URL for the RDF describing such resource. So why not > just use two > attributes for that? Something like this: > > > > url="http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOB > Y-S/Services"/> > url="http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOB > Y-S/Objects"/> > ... > url="http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOB > Y-S/All"/> > > > > Cheers, > Martin > > PS. Eddie, you should perhaps consider to send everything > again, but > splitted into few separate topics - that would be easier > to find in our > mailing lists archives... M. > > -- > Martin Senger > > EMBL Outstation - Hinxton Senger@EBI.ac.uk > European Bioinformatics Institute Phone: (+44) 1223 > 494636 > Wellcome Trust Genome Campus (Switchboard: > 494444) > Hinxton Fax : (+44) 1223 > 494468 > Cambridge CB10 1SD > United Kingdom > http://industry.ebi.ac.uk/~senger > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev From gordonp at ucalgary.ca Tue Jul 19 10:14:13 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Tue Jul 19 10:09:31 2005 Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: References: Message-ID: <42DD0AB5.9020502@ucalgary.ca> Anyone else having trouble with MOBY Central? org.biomoby.shared.MobyException: ===ERROR=== Fault details: [stackTrace: null] Fault string: java.lang.NullPointerException Fault code: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException Fault actor: null When calling: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl =========== at org.biomoby.client.CentralImpl.doCall(CentralImpl.java:191) at org.biomoby.client.CentralImpl.getNamespaces(CentralImpl.java:779)... From markw at illuminae.com Tue Jul 19 10:22:36 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue Jul 19 10:13:28 2005 Subject: [moby] [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: References: Message-ID: <1121782956.21968.9.camel@bioinfo.icapture.ubc.ca> On Tue, 2005-07-19 at 15:00 +0100, Martin Senger wrote: > * added ways how to get RDF resources (Mark, I am desperately waiting > for this one - in Malaga last week we created with Eddie a suggestion how > to get URLs from the Central API...) HI Martin, Eddie gave us the update on his Spanish activities yesterday. He told me what you discussed and I agree that this is a good idea. I will add that function to the API right away, and I can code it into the currently deployed version of MOBY, so that it will work from the main registry, however it will take a couple of weeks to get it into the CVS since the CVS codebase is currently going through a major refactoring effort in anticipation of our trip to Manchester in two weeks... You will notice that I have added LSID's to the 0.85 API that is on the website (they are part of most of the messages coming back from MOBY Central). These can be resolved to metadata to give you the same RDF graphs. HOWEVER, the code is not yet up-to-date relative to the API (the API is at 0.85 but the codebase is only at 0.82). Again, I'm trying to get the 0.85 codebase out as quickly as possible, but it is taking some time as there are only my hands (5% of the time) and my (highly competent!) co-op student Dennis assigned to this task... we simply suffer from lack of resourcing at this end :-( Cheers! M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From akerhornou at imim.es Tue Jul 19 10:30:57 2005 From: akerhornou at imim.es (Arnaud Kerhornou) Date: Tue Jul 19 10:15:25 2005 Subject: [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries Message-ID: <42DD0EA1.7030106@imim.es> Hi, I am working for the INB in Spain setting up Moby Web services, and I am trying to switch my code to use the new release, BioMoby 0.8.2 (was using BioMoby 0.8.1 before). I have a client script that is interrogating a registry server and ask for a given service. If the registry server is the following one, http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if I want to interrogate another one, I am getting an error. I am instanciating a Central object with the following parameters, => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl => URI: http://chirimoyo.ac.uma.es/MOBY/Central => Proxy: No Proxy Server The instanciation of Central seems to go okay, but then when I want to execute 'findService' method, I get the following error: -------------------------> Can't call method "getValue" on an undefined value at /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 <------------------------- Central.pm line 1728 is the following one (in _parseServices method) my $lsid = $Service->getAttributeNode('lsid')->getValue; Any clue of what is going wrong or do I need to update my client code ? Thanks Arnaud ___________________________________________________________________________ Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger T?l?chargez cette version sur http://fr.messenger.yahoo.com From senger at ebi.ac.uk Tue Jul 19 10:35:26 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue Jul 19 10:26:00 2005 Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <42DD0AB5.9020502@ucalgary.ca> Message-ID: > Anyone else having trouble with MOBY Central? > No, I don't (using: build/run/run-testing-central). Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 10:35:26 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue Jul 19 10:26:36 2005 Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <42DD0AB5.9020502@ucalgary.ca> Message-ID: > Anyone else having trouble with MOBY Central? > No, I don't (using: build/run/run-testing-central). Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 10:37:22 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue Jul 19 10:28:19 2005 Subject: [moby] [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <1121782956.21968.9.camel@bioinfo.icapture.ubc.ca> Message-ID: Mark, > I will add that function to the API right away, and I can code it into > the currently deployed version of MOBY, so that it will work from the > main registry > That's great! Just let me/us know what is the exact XML structure you are going to return by this new method (the best would be to put it into the API doc I guess), and I will add it into jMoby at once. Which means that this new method will work from your moby central but not yet from the others centrals, correct? Until their providers implement it as well - but their role is here difficult because they cannoy use CVS now. Do I understand it correctly? > You will notice that I have added LSID's to the 0.85 API that is on the > website (they are part of most of the messages coming back from MOBY > Central). These can be resolved to metadata to give you the same RDF > graphs. HOWEVER, the code is not yet up-to-date relative to the API > (the API is at 0.85 but the codebase is only at 0.82). > You mean the code for returning attribute 'lsid' in XML, or code for the LSID resilution service (to use such attribute), or both? Cheers, Martin PS. The clock is ticking - in 12 days I will be sitting in Philippines and working on Biomoby :-) M. -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From rebecca.ernst at gsf.de Tue Jul 19 10:41:30 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue Jul 19 10:32:46 2005 Subject: [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <42DD0EA1.7030106@imim.es> References: <42DD0EA1.7030106@imim.es> Message-ID: <42DD111A.8060900@gsf.de> Hi Arnaud! I am not sure if you are facing the same problem that we did... possibly the registry you want to contact doesn't contain the colunm 'lsid' this needs to be updated along with the new code so you have to modify your database like this: "alter table service_instance add lsid VARCHAR (255) NOT NULL default ''; " Cheers, Rebecca Arnaud Kerhornou wrote: > Hi, > > I am working for the INB in Spain setting up Moby Web services, and I > am trying to switch my code to use the new release, BioMoby 0.8.2 (was > using BioMoby 0.8.1 before). > > I have a client script that is interrogating a registry server and ask > for a given service. If the registry server is the following one, > http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if > I want to interrogate another one, I am getting an error. > > I am instanciating a Central object with the following parameters, > > => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl > => URI: http://chirimoyo.ac.uma.es/MOBY/Central > => Proxy: No Proxy Server > > The instanciation of Central seems to go okay, but then when I want to > execute 'findService' method, I get the following error: > -------------------------> > Can't call method "getValue" on an undefined value at > /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 > <------------------------- > > Central.pm line 1728 is the following one (in _parseServices method) > > my $lsid = $Service->getAttributeNode('lsid')->getValue; > > Any clue of what is going wrong or do I need to update my client code ? > > Thanks > Arnaud > > > > > > ___________________________________________________________________________ > Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! > Messenger T?l?chargez cette version sur http://fr.messenger.yahoo.com > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de From rebecca.ernst at gsf.de Tue Jul 19 10:41:30 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue Jul 19 10:32:55 2005 Subject: [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <42DD0EA1.7030106@imim.es> References: <42DD0EA1.7030106@imim.es> Message-ID: <42DD111A.8060900@gsf.de> Hi Arnaud! I am not sure if you are facing the same problem that we did... possibly the registry you want to contact doesn't contain the colunm 'lsid' this needs to be updated along with the new code so you have to modify your database like this: "alter table service_instance add lsid VARCHAR (255) NOT NULL default ''; " Cheers, Rebecca Arnaud Kerhornou wrote: > Hi, > > I am working for the INB in Spain setting up Moby Web services, and I > am trying to switch my code to use the new release, BioMoby 0.8.2 (was > using BioMoby 0.8.1 before). > > I have a client script that is interrogating a registry server and ask > for a given service. If the registry server is the following one, > http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if > I want to interrogate another one, I am getting an error. > > I am instanciating a Central object with the following parameters, > > => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl > => URI: http://chirimoyo.ac.uma.es/MOBY/Central > => Proxy: No Proxy Server > > The instanciation of Central seems to go okay, but then when I want to > execute 'findService' method, I get the following error: > -------------------------> > Can't call method "getValue" on an undefined value at > /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 > <------------------------- > > Central.pm line 1728 is the following one (in _parseServices method) > > my $lsid = $Service->getAttributeNode('lsid')->getValue; > > Any clue of what is going wrong or do I need to update my client code ? > > Thanks > Arnaud > > > > > > ___________________________________________________________________________ > Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! > Messenger T?l?chargez cette version sur http://fr.messenger.yahoo.com > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de From edward.kawas at gmail.com Tue Jul 19 10:45:52 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:36:49 2005 Subject: [MOBY-dev] rfcf 1. API addition of a method called getResourcesURL: In-Reply-To: Message-ID: <42dd1223.513aad7a.369d.5415@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 1. API addition of a method called getResourcesURL: Returns an xml document that describes the location of RESOURCES script that we hear so much about. It is the resources script that contains the ontologies expressed in RDF/XML example: Services http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Services Objects http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Objects ... Comments: Martin: I understand that this is not a new requirement (we all asked for it many times) but this is a concrete suggestion how to make it. I agree that getting these URLs from an API call is cleaner than just document how such URLs should look like. Only I would suggest slightly different XML response for that (but no strong opinion on my side about it). What we need are actually name-value pairs, each of them representing a name of a resource (such as 'services; and a URL for the RDF describing such resource. So why not just use two attributes for that? Something like this: ... Mark: I will add that function to the API right away, and I can code it into the currently deployed version of MOBY, so that it will work from the main registry, however it will take a couple of weeks to get it into the CVS since the CVS codebase is currently going through a major refactoring effort in anticipation of our trip to Manchester in two weeks... From edward.kawas at gmail.com Tue Jul 19 10:46:03 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:37:04 2005 Subject: [MOBY-dev] rfcf 2. How do I know what predicates are available in the RDF In-Reply-To: Message-ID: <42dd122e.54b1a5ca.63bd.4a5c@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 2. How do I know what predicates are available in the RDF documents that describe the Moby ontologies? a. should we provide a list of predicates and their intended meanings (on a wiki, etc)? b. should we provide an RDF schema defining the predicates (* probably the way to go) c. Should we allow the registration of new predicates, if so, should we make this an api call, etc. From edward.kawas at gmail.com Tue Jul 19 10:46:15 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:37:22 2005 Subject: [MOBY-dev] rfcf 3. LSID resolution services In-Reply-To: Message-ID: <42dd123b.670fc02b.2dd4.ffff929f@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 3. LSID resolution services a. is the LSID attribute available and documented? comments: Mark: You will notice that I have added LSID's to the 0.85 API that is on the website (they are part of most of the messages coming back from MOBY Central). These can be resolved to metadata to give you the same RDF graphs. HOWEVER, the code is not yet up-to-date relative to the API (the API is at 0.85 but the codebase is only at 0.82). Again, I'm trying to get the 0.85 codebase out as quickly as possible, but it is taking some time as there are only my hands (5% of the time) and my (highly competent!) co-op student Dennis assigned to this task. b. when will this occur, i have heard so much about this, yet i have seen nothing. From edward.kawas at gmail.com Tue Jul 19 10:46:23 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:37:26 2005 Subject: [MOBY-dev] rfcf 4. LSIDs versus simple names In-Reply-To: Message-ID: <42dd1240.1e9853c8.2dd4.ffff92af@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 4. LSIDs versus simple names a. why is the name of an object, service, or service type is sometimes a simple name and sometimes an LSID? b. how can i even retrieve the LSID of an object, service, or service type? c. should api calls be added to getName and getLSID for objects, services and service types? From edward.kawas at gmail.com Tue Jul 19 10:46:33 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:37:36 2005 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: Message-ID: <42dd124c.7fdce064.76fe.ffff9b01@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 5. LSIDs and Service Instances a..If a service instance is registered by 2 different authoritys, should they have the same LSID? b. if the service instance is registered in 2 registries, should the LSIDs be the same? c. will modified service instances have versioning information? From edward.kawas at gmail.com Tue Jul 19 10:46:40 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:37:45 2005 Subject: [MOBY-dev] rfcf 6. LSID resolution for objects in the BioMoby ontologies In-Reply-To: Message-ID: <42dd1252.3e967c5f.76fe.ffff9b10@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 6. LSID resolution for objects in the BioMoby ontologies a. what should getData return for i.namespaces (the name of the namespace?) ii. service types (the name of the service type?) iii. objects (xml schema? - note that schemas for datatypes should be ready and available very soon :-) iv. service instances (the name of the instance?) b. should we allow the namespace part of an LSID to be configurable? c. should we allow the authority part of an LSID to be configurable? From markw at illuminae.com Tue Jul 19 10:47:18 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue Jul 19 10:37:50 2005 Subject: [moby] [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <42DD0EA1.7030106@imim.es> References: <42DD0EA1.7030106@imim.es> Message-ID: <1121784438.21968.16.camel@bioinfo.icapture.ubc.ca> I've just packaged up another release (0.8.2a) that includes fixes for the 0.8.2 release bugs that have been reported so far. Please try this one and tell me if it still causes problems. M On Tue, 2005-07-19 at 16:30 +0200, Arnaud Kerhornou wrote: > Hi, > > I am working for the INB in Spain setting up Moby Web services, and I am > trying to switch my code to use the new release, BioMoby 0.8.2 (was > using BioMoby 0.8.1 before). > > I have a client script that is interrogating a registry server and ask > for a given service. If the registry server is the following one, > http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if I > want to interrogate another one, I am getting an error. > > I am instanciating a Central object with the following parameters, > > => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl > => URI: http://chirimoyo.ac.uma.es/MOBY/Central > => Proxy: No Proxy Server > > The instanciation of Central seems to go okay, but then when I want to > execute 'findService' method, I get the following error: > -------------------------> > Can't call method "getValue" on an undefined value at > /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 > <------------------------- > > Central.pm line 1728 is the following one (in _parseServices method) > > my $lsid = $Service->getAttributeNode('lsid')->getValue; > > Any clue of what is going wrong or do I need to update my client code ? > > Thanks > Arnaud > > > > > > ___________________________________________________________________________ > Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger > T?l?chargez cette version sur http://fr.messenger.yahoo.com > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From edward.kawas at gmail.com Tue Jul 19 10:46:48 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:37:54 2005 Subject: [MOBY-dev] rfcf 7. Would you like to have 3rd party annotation of metadata? In-Reply-To: Message-ID: <42dd125a.5ec5bd58.43d9.3f96@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 7. Would you like to have 3rd party annotation of metadata? a. use case: adds mirroring b. use case: allows people to provide additional metadata i. if you want only one source for metadata, then ignore this question. From edward.kawas at gmail.com Tue Jul 19 10:46:59 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:37:57 2005 Subject: [MOBY-dev] rfcf 8. Predicates In-Reply-To: Message-ID: <42dd1264.5c49d513.43d9.3fa7@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 8. Predicates a. should we add the following predicates to the metadata of service instances? i. sampleInputData - specifies valid input for the service instance ii. sampleOutputData - specifies the output for the service instance based on the test input. iii. approvedBy - specifies whether this service has been approved. Currently, this would only be useful for the INB, but many more may find it useful. b. how should the sample input/output be formatted? c. should we allow the annotation of datatypes? From edward.kawas at gmail.com Tue Jul 19 10:47:01 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:38:01 2005 Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: Message-ID: <42dd1266.6b7805fa.43d9.3fae@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 12. Services a. should article names for service instance input and output datatypes be mandatory? b. why does a service that advertise an output datatype called 'foo' return 'bar' instead? From edward.kawas at gmail.com Tue Jul 19 10:47:06 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:38:08 2005 Subject: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: Message-ID: <42dd126b.46e6258d.43d9.3fb7@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 11. RDF Agent a. do you think that there should be an api call, getUpdate that tells the agent to check for modifications to the service instance described by the metadata in the RDF document at the signatureURL? b. what is the time line to get the agent running? c. besides curation, how is the agent useful to me? From edward.kawas at gmail.com Tue Jul 19 10:47:11 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:38:09 2005 Subject: [MOBY-dev] rfcf 10. Datatype naming In-Reply-To: Message-ID: <42dd1271.3247625d.71ce.2bb7@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 10. Datatype naming a. do you think that the names of datatypes should be regulated? i. what would be the right way to specify a datatypes name? ii. what should we do with text_plain and text-plain? From edward.kawas at gmail.com Tue Jul 19 10:47:16 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue Jul 19 10:38:26 2005 Subject: [MOBY-dev] rfcf 9. API and Code versioning In-Reply-To: Message-ID: <42dd1275.5d41a999.71ce.2bc3@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 9. API and Code versioning a. why are the api and code versions different? Mark: the code is not yet up-to-date relative to the API (the API is at 0.85 but the codebase is only at 0.82). Again, I'm trying to get the 0.85 codebase out as quickly as possible, but it is taking some time as there are only my hands (5% of the time) and my (highly competent!) co-op student Dennis assigned to this task. From axk at sanger.ac.uk Tue Jul 19 11:01:08 2005 From: axk at sanger.ac.uk (Arnaud Kerhornou) Date: Tue Jul 19 10:47:47 2005 Subject: [moby] [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <1121784438.21968.16.camel@bioinfo.icapture.ubc.ca> References: <42DD0EA1.7030106@imim.es> <1121784438.21968.16.camel@bioinfo.icapture.ubc.ca> Message-ID: <42DD15B4.3000202@sanger.ac.uk> Hi Mark, I have downloaded the new release, and the script works fine now, Thanks Arnaud Mark Wilkinson wrote: >I've just packaged up another release (0.8.2a) that includes fixes for >the 0.8.2 release bugs that have been reported so far. > >Please try this one and tell me if it still causes problems. > >M > > >On Tue, 2005-07-19 at 16:30 +0200, Arnaud Kerhornou wrote: > > >>Hi, >> >>I am working for the INB in Spain setting up Moby Web services, and I am >>trying to switch my code to use the new release, BioMoby 0.8.2 (was >>using BioMoby 0.8.1 before). >> >>I have a client script that is interrogating a registry server and ask >>for a given service. If the registry server is the following one, >>http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if I >>want to interrogate another one, I am getting an error. >> >>I am instanciating a Central object with the following parameters, >> >> => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl >> => URI: http://chirimoyo.ac.uma.es/MOBY/Central >> => Proxy: No Proxy Server >> >>The instanciation of Central seems to go okay, but then when I want to >>execute 'findService' method, I get the following error: >>-------------------------> >>Can't call method "getValue" on an undefined value at >>/home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 >><------------------------- >> >>Central.pm line 1728 is the following one (in _parseServices method) >> >>my $lsid = $Service->getAttributeNode('lsid')->getValue; >> >>Any clue of what is going wrong or do I need to update my client code ? >> >>Thanks >>Arnaud >> >> >> >> >> >>___________________________________________________________________________ >>Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger >>Téléchargez cette version sur http://fr.messenger.yahoo.com >> >>_______________________________________________ >>MOBY-dev mailing list >>MOBY-dev@biomoby.org >>http://www.biomoby.org/mailman/listinfo/moby-dev >> >> ___________________________________________________________________________ Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger Téléchargez cette version sur http://fr.messenger.yahoo.com From senger at ebi.ac.uk Tue Jul 19 11:26:11 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue Jul 19 11:16:53 2005 Subject: [MOBY-dev] rfcf 8. Predicates In-Reply-To: <42dd1264.5c49d513.43d9.3fa7@mx.gmail.com> Message-ID: > a. should we add the following predicates to the metadata of > service instances? > Well, we should. But predicates should be unique, and I thought that they will be labeled as LSIDs... M. -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 16:22:56 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue Jul 19 16:13:46 2005 Subject: [MOBY-dev] jMoby does not compile!!!!! In-Reply-To: <42dd126b.46e6258d.43d9.3fb7@mx.gmail.com> Message-ID: Eddie, Four hours ago, you committed changes that do not compile! (e.g. in client/rdf/builder). You are the biggest sinner one can be. Please, please, correct it fast... Thanks, Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 16:28:50 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue Jul 19 16:24:07 2005 Subject: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: <42dd126b.46e6258d.43d9.3fb7@mx.gmail.com> Message-ID: > a. do you think that there should be an api call, getUpdate > that tells the agent to check for modifications to the > service instance described by the metadata in the RDF > document at the signatureURL? > Well, it depends how LSID metadata are collected and used, and where they are stored. The bottom line what I wish is this: When a service provider updates her/his metadata about his/her services, I wish that this update is visible asap to the others. Therefore, if it is the RDF agent you collect metadata and stores in a registry, then I wish to have an option to tell him "come and get it now". If, however, this is not the task of the RDF agent, but if there is yet another way how to collect data from my site (presumably done by the LSID resolution service) then I do not need such method in the API (because the LSID resolver will come to me when somebody asks for my metadata). Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Tue Jul 19 18:41:40 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue Jul 19 18:32:32 2005 Subject: [moby] RE: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: References: Message-ID: <1121812900.22947.57.camel@bioinfo.icapture.ubc.ca> I think the intent is as follows: The agent will collect the "core" data for the registry on a regular basis. You can chose to explicitly call the agent to you if you like, but it will only record the information that MOBY Central currently records. During service discovery, you will receive the LSID representing a particular service as part of the response message from MOBY Central (this is already in the API 0.85, but has not been coded yet). MOBY Central will act as the Authority for that LSID, and will give you a WSDL document that (at a minimum) directs you to GET the SignatureURL of that service as the getMetaData resolution function - the SignatureURL should hold an RDF document describing that service FULLY; i.e. not only what MOBY Central needs to know, but also whatever else the service provider wants to tell you about the service. Some of the RDF predicates in that document will be part of the official MOBY API, while others may be defined specifically by that service provider. Of course, we will try to make as many of the useful ones part of the MOBY API :-) Does that clarify how I am thinking about the process? Are there better ways to accomplish this that I haven't thought of? This is the perfect time to speak-up as we are in the process of re-coding the API right now... Cheers! M On Tue, 2005-07-19 at 21:28 +0100, Martin Senger wrote: > > a. do you think that there should be an api call, getUpdate > > that tells the agent to check for modifications to the > > service instance described by the metadata in the RDF > > document at the signatureURL? > > > Well, it depends how LSID metadata are collected and used, and where > they are stored. The bottom line what I wish is this: > When a service provider updates her/his metadata about his/her > services, I wish that this update is visible asap to the others. > Therefore, if it is the RDF agent you collect metadata and stores in a > registry, then I wish to have an option to tell him "come and get it now". > If, however, this is not the task of the RDF agent, but if there is yet > another way how to collect data from my site (presumably done by the LSID > resolution service) then I do not need such method in the API (because the > LSID resolver will come to me when somebody asks for my metadata). > > Martin > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From senger at ebi.ac.uk Tue Jul 19 19:50:57 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue Jul 19 19:41:32 2005 Subject: [moby] RE: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: <1121812900.22947.57.camel@bioinfo.icapture.ubc.ca> Message-ID: > The agent will collect the "core" data for the registry on a regular > basis. You can chose to explicitly call the agent to you if you like, > but it will only record the information that MOBY Central currently > records. > Okay. So far, so clear. How can I explicitly call the agent to me? > During service discovery, you will receive the LSID representing a > particular service as part of the response message from MOBY Central > (this is already in the API 0.85, but has not been coded yet). > This is done by the XML attribute 'lsid', correct? > MOBY Central will act as the Authority for that LSID, and will give > you a WSDL document that (at a minimum) directs you to GET the > SignatureURL of that service as the getMetaData resolution function > While I understand this I doubt that all service providers are aware that actually their SignatureURL is supposed to play a role of a LSID metadata resolution service. Concretely, a service that is defined by the LSIDMetadataHTTPBindingDirect (as defined in the LSID spec). Which means, for example, that the returned HTTP response should have Content-Type "x-application/rdf+xml" and Expires HTTP headers. Also, strictly speaking, an LSID should resolve to the metadata describing this LSID, but in the case of using SignatureURL there is no guarantee that a service provider does not put into the same document metadata about more services. So, we need to have this very clearly documented and explained, so the clients cannot be surprised when they get more than they asked for. I think that it would be also good if you can show an example of the WSDL with that binding, and instructions how clients should use such WSDL (which part they should take - my understanding is that the SignatureURL will be in the tag tag). > time to speak-up as we are in the process of re-coding the API right > now... > One more thing (not about API but about implementation) - and I think that Eddie mentioned it somewhere else, as well): In CSHL, we spoke about a special moby service that can be used for registering (and for retrieving) endpoints of service metadata created by the third-parties. Such service does not exist yet (nor its API) but if it exists (and if the registry is configured to use it), the registry should use it to put more metadata locations into returned WSDL. Please keep it in mind when you are implementing this part, so it can be easier updated when such service exists. Cheers, Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Tue Jul 19 20:07:15 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue Jul 19 20:01:19 2005 Subject: [unclassified] Re: [moby] RE: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: References: Message-ID: <1121818035.22947.85.camel@bioinfo.icapture.ubc.ca> On Wed, 2005-07-20 at 00:50 +0100, Martin Senger wrote: > Okay. So far, so clear. How can I explicitly call the agent to me? Yet to be decided... we'll make it an API call of some sort. We're still trying to get the agent to work so I need to focus on that first :-) > This is done by the XML attribute 'lsid', correct? correct. > Which means, for example, that the returned HTTP response should have > Content-Type "x-application/rdf+xml" and Expires HTTP headers. > Also, strictly speaking, an LSID should resolve to the metadata > describing this LSID, but in the case of using SignatureURL there is no > guarantee that a service provider does not put into the same document > metadata about more services. This is a very good point, and I hadn't fully considered the consequences of this v.v. having more than one service definition in the metadata document. Let's tease out what behaviour we want from this part of the API before we make any further decisions. It seems to me that the objective is for the service provider to be able to say things about their service that are not recorded in the Registry, but are still available to the consumer through a predictable API. We accomplish this through both avenues (having only one SignatureRDF at the URL or by having multiple concatenated SignatureRDF's at a single URL), but you are 100% correct that it might be a more confusing document to parse if the client is not expecting it. Are there other reasons to chose one over the other? I guess having a proliferation of SignatureRDF's on the server-side is an unpleasant thought also (though from a service providers perspective, it might be easier to manage, since RDF isn't particularly human-readable anyway...) ?? other thoughts ?? > I think that it would be also good if you can show an example of the > WSDL with that binding, and instructions how clients should use such WSDL > (which part they should take - my understanding is that the SignatureURL > will be in the tag tag). That was what I had in mind, yes. We'll certainly document everything well once we have it set up the way we want it. > One more thing (not about API but about implementation) - and I think > that Eddie mentioned it somewhere else, as well): In CSHL, we spoke about > a special moby service that can be used for registering (and for > retrieving) endpoints of service metadata created by the third-parties. > Such service does not exist yet (nor its API) but if it exists (and if the > registry is configured to use it), the registry should use it to put more > metadata locations into returned WSDL. Please keep it in mind when you are > implementing this part, so it can be easier updated when such service > exists. OK Cheers! M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From rebecca.ernst at gsf.de Wed Jul 20 03:24:38 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Wed Jul 20 03:15:34 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: References: Message-ID: <42DDFC36.4090409@gsf.de> Hi Martin! this vision has not been changed yet, mainly because noone seems to understand Heikos, Dirks and my problems with the existing solution. If you re-read the mails you can try to understand if you get the point I can't make myself any clearer than that. Basically I think it doesn't make sense to register an output as being a collection if the results don't form an entity! In our case e.g. the service 'getAGILocusCodes'. You give it a keyword and it returns all AGI Codes somehow related to this keyword. These AGI codes don't form an entity! Just because the keyword appears in their annotation they are given back - but with no relation to each other. For us it would be only logical if we would give many simple responses. This is not possible as currently if the number of results is not known we have to give back a collection. cheers, Rebecca Martin Senger wrote: >Hi all, > Catching up my email piles I wonder if someone can summarize if the >discussion about collections in this thread brought any (planned) changes >in the API (I am not talking now about how it is, or should be, >implemented in Taverna, that's, imho, an another story). > My understanding is that (talking about one mobyData object): > a) Any Moby service can have more outputs. If so, all of them must be >registered. The number of such outputs must be fixed. > b) Any of these outputs can be of type either Simple or Collection. > c) If it is a Collection, this output can have one or more Simples in >that Collection. Such Simples (and their number) are not individually >registered. > > Has this vision been changed? > > Thanks, > Martin > > > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst@gsf.de From senger at ebi.ac.uk Wed Jul 20 05:24:48 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed Jul 20 05:18:11 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <42DDFC36.4090409@gsf.de> Message-ID: Rebecca, Many thanks for the explanation. I think this is the critical part: > Basically I think it doesn't make sense to register an output as being a > collection if the results don't form an entity! > I assume that by entity you mean 'set of pieces that belong semantically together'. I think that the current concept of Simples and Collections gives you full freedom to model your output in any way you consider proper. For example, if you do not feel comfortable to put all AGI Codes in one collection because they represent semantically different things you can split them by types (or whatever semantics you choose) into several collections, or you can actually deploy several services, each of them returning a different set (collection) of AGI Codes given the same input keyword. But you know all that. I believe that we need to keep the concept of Collections open as are the general data types in other languages (e.g. hashtables). I am sorry to bother you with these obvious comments. Cheers, Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From d.haase at gsf.de Wed Jul 20 08:24:08 2005 From: d.haase at gsf.de (Dirk Haase) Date: Wed Jul 20 08:15:46 2005 Subject: [MOBY-dev] Moby Collections (was: Problems with Biomoby servicesin Taverna 1.2) In-Reply-To: References: Message-ID: <200507201424.08853.d.haase@gsf.de> On Wednesday 20 July 2005 11:24, Martin Senger wrote: > Rebecca, > Many thanks for the explanation. > > I think this is the critical part: > > Basically I think it doesn't make sense to register an output as being a > > collection if the results don't form an entity! > > I assume that by entity you mean 'set of pieces that belong > semantically together'. Exactly. > I think that the current concept of Simples and Collections gives you > full freedom to model your output in any way you consider proper. This is actually _not_ the case and that is why we brought up the issue. There is no proper way to output things which really make up a 'set of pieces that belong semantically together'. For example there is no way to output the results of a clustering service (a cluster is a set of sequences, the service result is a set of clusters), because a collection of collections is not allowed afaik. Of course one can create a 'cluster object' but this creates rather artificial hurdles in workflow creation. Suppose that after the clustering, each cluster should be fed into a multiple alignment service. This would presumably take a collection of sequences as input. So one would have to decompose the cluster object first before it can be further processed. I think we all agree that on the input side a collection only makes sense if it is meant as one semantical entity (example from the API docs: several sequences make up the BLAST database that is to be queried). But not so on the output side. So we have an inherent unequality which complicates workflow building. > For > example, if you do not feel comfortable to put all AGI Codes in one > collection because they represent semantically different things you can > split them by types (or whatever semantics you choose) into several > collections, The claim is they are not related... The only relation is that they pop up for the same input but that is obvious for this kind of services and does not need to be emphasized by stuffing them into a collection. > I believe that we need to keep the concept of Collections open as are > the general data types in other languages (e.g. hashtables). That is a good point, but I'm not sure if it is appropriate to compare the concept of Simples/Collections to general data types in programming languages. > I am sorry to > bother you with these obvious comments. No, they are very welcome indeed! Servus, dirk From senger at ebi.ac.uk Wed Jul 20 08:44:10 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed Jul 20 08:35:33 2005 Subject: [MOBY-dev] Moby Collections (was: Problems with Biomoby servicesin Taverna 1.2) In-Reply-To: <200507201424.08853.d.haase@gsf.de> Message-ID: > because a collection of collections is not allowed afaik. > You are right - I have forgotten that we cannot have collections of collections. So we cannot model all possible scenarios. But: BioMoby (afaik) tries to be simple. To allow completely complex data types (build on collections of collectios) would mean to assume that all clients will be able to understand such complex data types. And it would be hard. We would end up anyway with many clients that would simply ignore such complex types (or they break on them). So why not to go with the current API (I may even argue if it is really neccessary to allow a collection having different types of Simples, but that's another story) and to try to solve the problem of semantically relevant clusters on the service level - simply to create more specific service instances. BioMoby was/is about a simple transformation. If the transformation is not simple let's try to divide it into more transformations that can be simple. Cheers, Martin PS. > but I'm not sure if it is appropriate to compare the concept of > Simples/Collections to general data types in programming languages. > Why not? Except that biomoby collections are less general (by not allowing coles of coles). M. -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From gordonp at ucalgary.ca Wed Jul 20 10:22:15 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed Jul 20 10:15:13 2005 Subject: [MOBY-dev] Moby Collections (was: Problems with Biomoby servicesin Taverna 1.2) In-Reply-To: References: Message-ID: <42DE5E17.8080302@ucalgary.ca> I think Martin has hit the nail on the head here. One of the primary advantages of MOBY is that you can't (easily) build arbitrarily syntactically complex output. This ensures that you can chain together services more easily. You should be building new objects in the ontology to represent the cluster concept, not squeezing that cluster logic into the syntax of the collection. Otherwise clients will do very wierd things with your implicitly encoded concepts. You always have to look at how your data could be used by someone else who wasn't expecting it... >>because a collection of collections is not allowed afaik. >> >> >> > You are right - I have forgotten that we cannot have collections of >collections. So we cannot model all possible scenarios. > > From jmfernandez at cnb.uam.es Wed Jul 20 10:53:44 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Wed Jul 20 10:45:27 2005 Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: <42dd1266.6b7805fa.43d9.3fae@mx.gmail.com> References: <42dd1266.6b7805fa.43d9.3fae@mx.gmail.com> Message-ID: <42DE6578.3080008@cnb.uam.es> > > 12. Services > > a. should article names for service instance input and > output datatypes be mandatory? > Well, in my opinion, I think they should be mandatory when there is more than one input (or output). And when there is only one and the article name it is not declared, then it should be a default value for the article name. > b. why does a service that advertise an output datatype > called 'foo' return 'bar' instead? > If datatype 'bar' inherits from 'foo', then it could be allowed, but I don't have a good example for that. But we must take into account that most times it happens it is a bug. > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez@cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From jmfernandez at cnb.uam.es Wed Jul 20 10:46:11 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Wed Jul 20 10:50:33 2005 Subject: [MOBY-dev] rfcf 2. How do I know what predicates are available inthe RDF In-Reply-To: <42dd122e.54b1a5ca.63bd.4a5c@mx.gmail.com> References: <42dd122e.54b1a5ca.63bd.4a5c@mx.gmail.com> Message-ID: <42DE63B3.503@cnb.uam.es> Hi everybody, I think a mixure of b) and c) could be feasible. The RDF Schema would be generated from the registered new predicates. Edward Kawas wrote: > Since there was a request to keep these messages in email > and away from twiki, I have added each heading to an email. > > 2. How do I know what predicates are available in the RDF > documents that describe the Moby ontologies? > > a. should we provide a list of predicates and their intended > meanings (on a wiki, etc)? > > b. should we provide an RDF schema defining the predicates > (* probably the way to go) > > c. Should we allow the registration of new predicates, if > so, should we make this an api call, etc. > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev@biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez@cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From jmfernandez at cnb.uam.es Wed Jul 20 11:03:56 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Wed Jul 20 10:54:31 2005 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42dd124c.7fdce064.76fe.ffff9b01@mx.gmail.com> References: <42dd124c.7fdce064.76fe.ffff9b01@mx.gmail.com> Message-ID: <42DE67DC.5060007@cnb.uam.es> > 5. LSIDs and Service Instances > > a..If a service instance is registered by 2 different > authoritys, should they have the same LSID? > Only if we can assure they are the SAME service (replicated services), which would require external help. > b. if the service instance is registered in 2 registries, > should the LSIDs be the same? > No, they shouldn't, unless they are registered by the same authority. > c. will modified service instances have versioning > information? > I think it should, because versioning information could help in the use of services from different registries, or the integration of those registries. -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez@cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From d.haase at gsf.de Wed Jul 20 11:12:06 2005 From: d.haase at gsf.de (Dirk Haase) Date: Wed Jul 20 11:03:31 2005 Subject: [MOBY-dev] Moby Collections In-Reply-To: <42DE5E17.8080302@ucalgary.ca> References: <42DE5E17.8080302@ucalgary.ca> Message-ID: <200507201712.06848.d.haase@gsf.de> On Wednesday 20 July 2005 16:22, Paul Gordon wrote: > I think Martin has hit the nail on the head here. One of the primary > advantages of MOBY is that you can't (easily) build arbitrarily > syntactically complex output. Correct. That's why we did not ask for nested collections. We suggest to allow for multiple Simple output as default - which I think is very appropriate because especially query services will usually return more than one result. This way we would save the collection concept for semantically related entities. > This ensures that you can chain together > services more easily. That is exactly our intention... > You should be building new objects in the > ontology to represent the cluster concept, not squeezing that cluster > logic into the syntax of the collection. Otherwise clients will do very > wierd things with your implicitly encoded concepts. What bears more implicitness: 1) passing totally unlrelated things together just because of a coincidental preceding step in the workflow or 2) putting things together because they build a semantic entity which is intended to be processed as a whole? > You always have to > look at how your data could be used by someone else who wasn't expecting > it... Well, expectations are ruled by the API - and that is what we are working on, aren't we? ;-) From senger at ebi.ac.uk Wed Jul 20 11:32:37 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed Jul 20 11:24:33 2005 Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: <42DE6578.3080008@cnb.uam.es> Message-ID: > > a. should article names for service instance input and > > output datatypes be mandatory? > > Well, in my opinion, I think they should be mandatory when there is more > than one input (or output). > This question hides actually two situations: a) article names for inputs (or outputs), and b) article names for elements attached by HAS[A] relationships. For b) situation I would say hat the article name should be mandatory, because one never knows who is going to extend his object and add new attributes. Having it mandatory will make life (and verification) easier. Also, it will help to understand what individual HASA attributes mean if they have an article name. For a) situation it is not that crucial, I would say. But I know that INB people will come with a suggestion how to handle errors in a new way, and for their solution the article names are important (if I remember it correctly). And a similar situation may appear again in the future. And because having article names mandatory is not hard to do (and if your current services do not do it, it will not break them, only some verifiers may complain) I am inclined to say make them mandatory. Regards, Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From gordonp at ucalgary.ca Wed Jul 20 12:25:42 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed Jul 20 12:16:44 2005 Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: References: Message-ID: <42DE7B06.8050102@ucalgary.ca> I absolutely agree on (b). Anonymous data members is crazy. For (a) I would say that articleNnames should be manditory if there is more than one parameter. You could make a lot fancier rules, but you start dealing with a lot of parameter ambiguity issues. It's easier to just require them if there is any question. This would be a very minor change, since I don't think that there are many services that take more than one parameter, are there? All of the simple one-parameter services would still be valid. >>>a. should article names for service instance input and >>>output datatypes be mandatory? >>> >>> >>Well, in my opinion, I think they should be mandatory when there is more >>than one input (or output). >> >> >> > This question hides actually two situations: a) article names for >inputs (or outputs), and b) article names for elements attached by HAS[A] >relationships. > For b) situation I would say hat the article name should be mandatory, >because one never knows who is going to extend his object and add new >attributes. Having it mandatory will make life (and verification) easier. >Also, it will help to understand what individual HASA attributes mean if >they have an article name. > For a) situation it is not that crucial, I would say. But I know that >INB people will come with a suggestion how to handle errors in a new way, >and for their solution the article names are important (if I remember it >correctly). And a similar situation may appear again in the future. And >because having article names mandatory is not hard to do (and if your >current services do not do it, it will not break them, only some verifiers >may complain) I am inclined to say make them mandatory. > > Regards, > Martin > > > From schoof at mpiz-koeln.mpg.de Thu Jul 21 09:42:32 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Thu Jul 21 09:33:09 2005 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: References: Message-ID: <776ef61c9a8901624450044125ab9b81@mpiz-koeln.mpg.de> Hi all again, I'll respond to Martin's query here: My understanding is that (talking about one mobyData object): a) Any Moby service can have more outputs. If so, all of them must be registered. The number of such outputs must be fixed. b) Any of these outputs can be of type either Simple or Collection. c) If it is a Collection, this output can have one or more Simples in that Collection. Such Simples (and their number) are not individually registered. I think the issue is point a, the number of such outputs must be fixed. Or, as the API says "each of these articles will appear EXACTLY ONCE in the output from the service". I request to change this to "each of these articles will appear AT LEAST ONCE in the output from the service". Why is this necessary? Currently, services that return potentially many results are registered as outputting a Collection. But within workflows, the iteration strategy or how often the next service is called should distinguish between an output made up of many independent items versus one or many groups of connected items. In the first case, the following service should be called once for each item, in the second case, only once for each group. See below for full example. What I am suggesting is to separate the cardinality from the Simple/Collection issue; meaning, that a service that performs e.g. a database lookup and returns 1 or many outputs (or none, but in the Moby world this means it returns 1 empty output) will be registered as returning Simple, not Collection, if the outputs are otherwise semantically unrelated (aside from the fact that they arose from the same query). And reserve the Collection article for grouping of outputs that need to be seen as a single entity. E.g. a service that outputs ortholog pairs given as input a pair of organisms: Each ortholog pair could be represented as a pair of GenericSequence objects in a Collection, with the service outputing 1 or many of these Collections. The same service, given as input three organisms, could still output many Collections, then containing three sequences each. This prevents ugly explosion of specialized BioMoby objects like "MultipleSequences", "HAS GenericSequence(s)"... that would otherwise be needed to wrap this. For service discovery, this should not make a difference. Services would still be required to return every Class of object that they register, as you state in a: all output object *Classes* must be registered, and the number of *Classes* fixed. I.e., a service registered as returning GenericSequence and AnnotatedJpeg objects must always return at least one GenericSequence plus at least one AnnotatedJpeg. I can't recall a service that actually does that... I can only think of this being meaningful if the AnnotatedJpeg is semantically connected to a specific GenericSequence, and in that case both should be connected through putting them in a Collection imho. For inputs, I'm not so worried; if multiple inputs are intended to go into a single service call, they will probably be connected and could go into a Collection. Example above: Input is a Collection of Organisms. Basically, I see that as the only way to register services that require AT LEAST two equal inputs. No problem with b, but with c: To my understanding, when registering a Collection, also the classes of objects in Simples that it contains must be registered. Otherwise, no discovery. However, see API, " A collection may contain zero or more Objects of each of the Classes", not all these classes must actually be included. So far, I do not see the need to distinguish between services that return EXACTLY ONE output and those that return one or more. Taverna seems to make that distinction, and bases iteration strategies on that, but I would want to do that dynamically, and it may be that that's what Taverna does. I'd by default assume that there will be multiple outputs and iterate over them, but if the workflow designer so wishes make it possible to (using e.g. a local processor) combine all the outputs into a Collection that can be used as the single input to a following service. This distinction is necessary: Use case examples would be a service returning a number of sequences, that in one scenario (iterate) should each be run through a BLAST service individually and in a different scenario (bag or Collection) should be all together input into a single call of a multiple alignment service. The current problem arises because Taverna now, in what is for me the semantically correct interpretation, if it receives a Collection as output from a service, it inputs that into a single execution of the following service if that service consumes Collection. In version 1.1 and before, Collections were decomposed by Taverna and iterated over. For the workflows being used, that was the wanted behavior, as e.g. keyword queries returning a set of sequences should be linked to services that act on each individual sequence. The mistake is imho on the BioMoby side, where we use Collection to wrap multiple outputs even if these are individual results that should be processed individually, and then, in order to be able to pipeline, register services that actually act on single inputs as consuming Collections. Consider Tom Oinn's comment 080705: 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. This is the same logic that we'd need to implement into BioMoby to allow meaningful links between Collection producing services and Simple consuming services! And NOT register services as consuming or producing Collections if all they do is mimic this behaviour internally by iterating over the Collection items. Many words and I'm not sure this is making anything any clearer. But I try ;-) Best, Heiko On 18. Jul 2005, at 14:39 Uhr, Martin Senger wrote: Hi all, Catching up my email piles I wonder if someone can summarize if the discussion about collections in this thread brought any (planned) changes in the API (I am not talking now about how it is, or should be, implemented in Taverna, that's, imho, an another story). My understanding is that (talking about one mobyData object): a) Any Moby service can have more outputs. If so, all of them must be registered. The number of such outputs must be fixed. b) Any of these outputs can be of type either Simple or Collection. c) If it is a Collection, this output can have one or more Simples in that Collection. Such Simples (and their number) are not individually registered. Has this vision been changed? Thanks, Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger _______________________________________________ MOBY-dev mailing list MOBY-dev@biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From edward.kawas at gmail.com Fri Jul 22 11:08:09 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Fri Jul 22 12:16:53 2005 Subject: [MOBY-dev] FW: theseu - upload the first beta and unstable version Message-ID: <42e10bdc.61a5c3e4.09f0.062d@mx.gmail.com> -----Original Message----- From: Roman Roset [mailto:roset@ac.upc.edu] Sent: Friday, July 22, 2005 9:02 AM To: inb-tecnico@everest.cnb.uam.es; Eddie Kawas; Martin Senger Subject: theseu - upload the first beta and unstable version Hi all, The INB Theseu project is a framework for biomoby developers. The requirements of Theseu project are driven to develop BioMoby Services and to administrate BioMoby Repositories (locally or remotally). Future extensions of this project could include features such as RSS systems to get the biomoby news, local agents to perform some tasks directly with the RDF,etc... in short, the needs of biomoby and INB developers comunity. DOWNLOAD ======== 0. Documentation At the moment there is no user documentation. Sorry. Any voluntaries to help me writing the online documentation :) ? 1. Requirements These are the prerequisites to install the Theseu Project. They must be downloaded (EMF via eclipse site or manually) and installed before Theseu: Eclipse build eclipse-SDK-3.0.2: http://download.eclipse.org/eclipse/downloads/index.php EMF runtime 2.0.2 (emf-sdo-runtime-2.0.2): http://download.eclipse.org/tools/emf/scripts/downloads-view er.php?s=2.0.2/R200503151315 2. Theseu Project Runtime I have uploaded the first beta and unstable version of theseu at this site: Remote Site name: Theseu Project URL: http://inb.lsi.upc.edu/theseu/updates You can download easily via help -> software updates -> find & install -> "search for new features..." --> "new remote site". This beta distribution has 3 features grouped in two diferents topics: * Theseu Project |-> Theseu Project plugins |-> other (this is jmoby and others) Select all that you can checked :) and follow the afirmative buttons. 3. Mini example to start playing Once Theseu is installed, follow this steps: 1. click the Navigator window and press ctrl+N to open a new project wizard. 1.a. Select BioMoby Services Wizard --> new Moby Project 2. click on the label of the new project into de navigator window and press ctrl+N again to create a new repository: 2.a Select BioMoby Services Wizard --> new Moby Repository: 2.b. Select the server or push the preferences button to add new biomoby services. 2.c Select the authority or push the preferences button to add new authors. 3.d push finish button. 3. Now you can view in the editor window your empty local repository. The next step is to populate the ouline view with the element of the repository. So, click the label of a moby element into the editor (for instance "services") and click via the GeneralEditor menu or a popup menu (clicking over the "services" with the mouse right button) the "load..." option. 3.1 wait and don't despair... this is only necessary the first time. 4. Now you could open some views before to start with the edition: 4.1 Window --> show view --> Problems 4.2 Window --> show view --> Properties Remember that when you validate an element, the information to navigate through the messages is saved into the Problems view. 5. Register elements: it's only possible in this "demo" to register services. ABOUT THIS VERSION ================== I have uploaded this version with the aim of show the actual state of this project. I repeat that is only a demo version and it's very unstable, however you can have an idea of what Theseu can do and what will be able to do. Please send me your opinions about this. All your help is welcome. Thanks. Roman -- Roman Roset Mayals Technical Coordinator Instituto Nacional de Bioinformatica (www.inab.org) Barcelona Supercomputing Center Node (www.bsc.org.es) c/. Jordi Girona 1-3 Modul C6-E201 Tel. : 934 011 650 E-08034 Barcelona Fax : 934 017 014 Catalunya (Spain) e-mail: rroset@lsi.upc.edu Note: I'm on vacation until August 21th From roset at ac.upc.edu Fri Jul 22 00:23:49 2005 From: roset at ac.upc.edu (Roman Roset) Date: Fri Jul 22 12:44:32 2005 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances Message-ID: <42E074D5.2010205@ac.upc.es> >> 5. LSIDs and Service Instances >> >> a..If a service instance is registered by 2 different >> authoritys, should they have the same LSID? >> > Only if we can assure they are the SAME service (replicated services), > which would require external help. If I am not wrong, among other things, the lsid of a service would have to serve to obtain the RDF and I think that it should be interesting to have one-to-one relationship between RDF and service (or instance of a replicate service). For instance, if the RDF of a replicated service needs to save information about a concrete instance (such as the service is temporally unavailable, cpu's, os, etc...) These values could be different for two "same services". (but maybe I don't have clear what are same services - My understanding is that, two services are the same if given the same input then the output of both is exactly the same). However, I have some doubts about the way to specify the LSID of the services in biomoby. There are an example in the biomoby metadata-resolver: urn:lsid:biomoby.org:serviceinstance:www.illuminae.com,getSHound3DNeighboursFromGi (The LSID syntax is urn:lsid:authority:namespace:identifier:revision) Why is the authority "biomoby.org" and this one is not www.illuminae.com? Why is the service name separated of the lsid? If we use biomoby as authority but we wrote the services as identifiers then an approach to denote more of one instance (or replicated service) into the same lsid could be using the revision field. For instance: urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighboursFromGi:1 urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighboursFromGi:2 But sincerely, my opinion is that it's better to use an lsid like: urn:lsid:www.illuminae.com:biomoby.org:getSHound3DNeighboursFromGi urn:lsid:www.other.com:biomoby.org:getSHound3DNeighboursFromGi With this kind of LSID the MetaData-resolver could know that if the namespace is biomoby then the element to resolve is a service, and we should have one LSID per service. >> b. if the service instance is registered in 2 registries, >> should the LSIDs be the same? >> > No, they shouldn't, unless they are registered by the same authority. Is the RDF of this sevice shared by both registries? Is the response is yes then I feel that the LSID should be the same. (and the opposite, if the response is not the LSIDs should be different.) >> c. will modified service instances have versioning >> information? >> > I think it should, because versioning information could help in the use > of services from different registries, or the integration of those > registries. From my point of view, for one lsid of a service (independently of what thing is identifying exactly) there are two ways to understand the versioning information. I think that the version has to "explain" (useful) changes in the meta-information. But the service metainformation (the RDF) can change at two levels: the values of the predicates and the schema. I think that versions should be useful (at least for the agent) in the second case. -- Roman Roset Mayals Technical Coordinator Instituto Nacional de Bioinformatica (www.inab.org) Barcelona Supercomputing Center Node (www.bsc.org.es) c/. Jordi Girona 1-3 Modul C6-E201 Tel. : 934 011 650 E-08034 Barcelona Fax : 934 017 014 Catalunya (Spain) e-mail: rroset@lsi.upc.edu Note: I'm on vacation until August 21th From edward.kawas at gmail.com Fri Jul 22 13:12:54 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Fri Jul 22 13:03:46 2005 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42E074D5.2010205@ac.upc.es> Message-ID: <42e1291a.51939a07.08bd.ffff8d38@mx.gmail.com> > However, I have some doubts about the way to specify > the LSID of the > services in biomoby. There are an example in the biomoby > metadata-resolver: > > urn:lsid:biomoby.org:serviceinstance:www.illuminae.com,get > SHound3DNeighboursFromGi > > (The LSID syntax is > urn:lsid:authority:namespace:identifier:revision) > > Why is the authority "biomoby.org" and this one is not > www.illuminae.com? The authority is BioMoby.org because it's the issuing authority. In other words, it is the place where we can go and ask for information regarding a particular service and retrieve metadata about it. www.illuminae.com is the authority that registered the service getSHound3DNeighboursFromGi into the registry hosted by BioMoby.org. Does this make sense? > Why is the service name separated of the lsid? The name was separated so that in the event that there were 2 services registered with the same service name we wouldn't run into conflicts. Also, a service should be uniquely identified by the authority that registered it and its name. > If we use biomoby as authority but we wrote the services > as identifiers > then an approach to denote more of one instance (or > replicated service) > into the same lsid could be using the revision field. > For instance: > > urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours > FromGi:1 > urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours > FromGi:2 > The revision id is meant to convey to someone that the underlying object 'pointed to' by the lsid has changed. So for instance, when the agent is running, and it notices that the service instance has changed a little (new description added or a new input or output, etc), then the revision id would have to be used. > urn:lsid:www.illuminae.com:biomoby.org:getSHound3DNeighbou > rsFromGi > urn:lsid:www.other.com:biomoby.org:getSHound3DNeighboursFr > omGi > > With this kind of LSID the MetaData-resolver could know > that if the namespace is biomoby then the element to > resolve is a > service, and we should have one LSID per service. One of the problems with this approach is that how do you go about resolving other Moby objects like service types, datatypes, namespaces,etc? I believe that your proposal also doesn't follow the correct use of the LSID protocol. From markw at illuminae.com Fri Jul 22 13:26:06 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri Jul 22 13:16:57 2005 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42e1291a.51939a07.08bd.ffff8d38@mx.gmail.com> References: <42e1291a.51939a07.08bd.ffff8d38@mx.gmail.com> Message-ID: <42E12C2E.8010406@illuminae.com> Edward Kawas wrote: >> However, I have some doubts about the way to specify >>the LSID of the >>services in biomoby. There are an example in the biomoby >>metadata-resolver: >> >>urn:lsid:biomoby.org:serviceinstance:www.illuminae.com,get >>SHound3DNeighboursFromGi >> >>(The LSID syntax is >>urn:lsid:authority:namespace:identifier:revision) >> >>Why is the authority "biomoby.org" and this one is not >>www.illuminae.com? >> >> >The authority is BioMoby.org because it's the issuing >authority. In other words, it is the place where we can go >and ask for information regarding a particular service and >retrieve metadata about it. www.illuminae.com is the >authority that registered the service >getSHound3DNeighboursFromGi into the registry hosted by >BioMoby.org. Does this make sense? > > Just to expand this a little bit more - biomoby.org will NOT be the resolver for that LSID, but it will be the authority server for that LSID. >>Why is the service name separated of the lsid? >> >> >The name was separated so that in the event that there were >2 services registered with the same service name we wouldn't >run into conflicts. Also, a service should be uniquely >identified by the authority that registered it and its name. > > In the early days when we were first considering LSID's in MOBY, there wasn't a widely agreed-upon standard for "wrapping" someone elses identifier. It seems now that there is a "standard" (I don't know if this is a recommendation from teh LSID community or not, but it seems that the core LSID developers do it themselves, so it must be supported). We might consider changing the LSID structure for MOBY Service Instances to e.g.: urn:lsid:biomoby.org.www.illuminae.com:serviceinstance:getGenBankff It makes no difference at all at the end of the day, since LSID's are opaque identifiers :-) >>If we use biomoby as authority but we wrote the services >>as identifiers >>then an approach to denote more of one instance (or >>replicated service) >> into the same lsid could be using the revision field. >>For instance: >> >>urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours >>FromGi:1 >>urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours >>FromGi:2 >> >> >> >The revision id is meant to convey to someone that the >underlying object 'pointed to' by the lsid has changed. So >for instance, when the agent is running, and it notices that >the service instance has changed a little (new description >added or a new input or output, etc), then the revision id >would have to be used. > > Yup... that suggestion (to use the version position to denote replication) would be very dangerous! However, it is an interesting problem, since I don't think the LSID communuty would be very comfortable with a single LSID referring to multiple resources... even if they are "identical". Martin, do you have an opinion about this? M From roset at ac.upc.edu Sat Jul 23 15:56:08 2005 From: roset at ac.upc.edu (Roman Roset) Date: Sat Jul 23 14:42:46 2005 Subject: [MOBY-dev] FW: theseu - upload the first beta and unstable version Message-ID: <42E2A0D8.4070109@ac.upc.es> Hello, thank you very much eddie, to forward my email in this list :). I've uploaded a ppt preview of INB-Theseu: http://inb.lsi.upc.es/theseu/docs/ppt/TheseuPreview.ppt There are bugs on this demo, but my intention is to know the general opinion of moby-developers in order to driven the design of future functionalities. So, all your comments, questions or suggestions are most welcome. INB Theseu people: Author: Roman Roset (roset@ac.upc.edu) Coordinator: David G. Pisano (dgpisano@cnb.uam.es) Supervised by: Phd. Xavier Messeguer and Phd. Oswaldo Trelles Theseu is property of INB - Instituto Nacional de Bioinform?tica (www.inab.org) Thanks. Note: I'm on vacation until August 21th -- Roman Roset Mayals Technical Coordinator Instituto Nacional de Bioinformatica (www.inab.org) Barcelona Supercomputing Center Node (www.bsc.org.es) c/. Jordi Girona 1-3 Modul C6-E201 Tel. : 934 011 650 E-08034 Barcelona Fax : 934 017 014 Catalunya (Spain) e-mail: rroset@lsi.upc.edu From markw at illuminae.com Mon Jul 25 11:37:11 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon Jul 25 11:29:53 2005 Subject: [moby] [MOBY-dev] FW: theseu - upload the first beta and unstable version In-Reply-To: <42E2A0D8.4070109@ac.upc.es> References: <42E2A0D8.4070109@ac.upc.es> Message-ID: <1122305831.29736.58.camel@bioinfo.icapture.ubc.ca> Hokey Dinah! That's AMAZING!!! M On Sat, 2005-07-23 at 20:56 +0100, Roman Roset wrote: > Hello, > > thank you very much eddie, to forward my email in this list :). > > I've uploaded a ppt preview of INB-Theseu: > > http://inb.lsi.upc.es/theseu/docs/ppt/TheseuPreview.ppt > > > There are bugs on this demo, but my intention is to know the general > opinion of moby-developers in order to driven the design of future > functionalities. So, all your comments, questions or suggestions are > most welcome. > > INB Theseu people: > > Author: Roman Roset (roset@ac.upc.edu) > Coordinator: David G. Pisano (dgpisano@cnb.uam.es) > Supervised by: Phd. Xavier Messeguer and Phd. Oswaldo Trelles > > Theseu is property of INB - Instituto Nacional de Bioinform?tica > (www.inab.org) > > Thanks. > > Note: I'm on vacation until August 21th > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From markw at illuminae.com Mon Jul 25 11:37:11 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon Jul 25 11:30:20 2005 Subject: [moby] [MOBY-dev] FW: theseu - upload the first beta and unstable version In-Reply-To: <42E2A0D8.4070109@ac.upc.es> References: <42E2A0D8.4070109@ac.upc.es> Message-ID: <1122305831.29736.58.camel@bioinfo.icapture.ubc.ca> Hokey Dinah! That's AMAZING!!! M On Sat, 2005-07-23 at 20:56 +0100, Roman Roset wrote: > Hello, > > thank you very much eddie, to forward my email in this list :). > > I've uploaded a ppt preview of INB-Theseu: > > http://inb.lsi.upc.es/theseu/docs/ppt/TheseuPreview.ppt > > > There are bugs on this demo, but my intention is to know the general > opinion of moby-developers in order to driven the design of future > functionalities. So, all your comments, questions or suggestions are > most welcome. > > INB Theseu people: > > Author: Roman Roset (roset@ac.upc.edu) > Coordinator: David G. Pisano (dgpisano@cnb.uam.es) > Supervised by: Phd. Xavier Messeguer and Phd. Oswaldo Trelles > > Theseu is property of INB - Instituto Nacional de Bioinform?tica > (www.inab.org) > > Thanks. > > Note: I'm on vacation until August 21th > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From senger at ebi.ac.uk Tue Jul 26 08:14:43 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue Jul 26 08:05:23 2005 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42E12C2E.8010406@illuminae.com> Message-ID: > Yup... that suggestion (to use the version position to denote > replication) would be very dangerous! However, it is an interesting > problem, since I don't think the LSID communuty would be very > comfortable with a single LSID referring to multiple resources... even > if they are "identical". Martin, do you have an opinion about this? > This is about replicated services that were (afaik) first mentioned (at least with some depth) in Vancouver. Before we decide about how to create LSID for such kind of services, we need to define what are replicated services. So far, each service was uniquely defined by three elements: - its endpoint, - its authority (in the Biomoby sense, not the LSID sense), and - its (simple) name (unique within its authority). So what are the replicated services? My take (after what I heard in Vancouver) is that the replicated services will have the same two of the attributes above: authority and name, but they will not share the same endpoint. (Additionally, just in documenttaion, we claim that such replicated services do the same data transformation - but this is not check-able.) Is this how we want to define replicated services? If the answer is yes, here is how I would feel about the LSID: 1) Resolving Data. Eddie mentioned that he (or he and Mark, I was not sure) were/are thinking about using LSID resolution method 'getData' for returning a WSDL of this service. I am not sure that I identify myself with this idea. Mainly because we already have a way how to get the same WSDL from the registry (there is an API method for that) so why to duplicate it. And also because doing this we will immediately need different LSID for replicated services (because the endpoint is - at least, usully - part of such WSDL). So I would suggest to return nothing by this method. If this is accepted, I ca elaborate further what to do with metadata. 2) Resolving metadata. Mark is not sure that the replicated services should have the same LSID. I am not sure either - but I am more inclined to say 'Why not?'. They do the same think - except they some of them can do it better (faster, more reliably etc.) But this can be reflected in metadata - it includes the endpoint, so the user will see which metadata refers to which replicated service. We would need to make some consensus on the metadata predicates to assure that the metadata can be identified: which are meant for all such replicated services, and which are meand only for individual such services. If we do not want to have same LSID for replicated services we will need to solve the following with more efforts: 3) Problems with *not* having the same LSID for replicated services: a) The Registry API is going to change because of replicated services (as we talked about it in V.) - but the change would have me more significant than just to chnage the 'endpoint URL' to a set of 'endpoint URLs' - because when the registry returns LSID, it must return more/many LSIDs about the same service. b) The registry would have more problems to collect all available metadata (or to send SignatureURLs of all of them) if their are more LSIDs for the same service. These problems are surely solveable but I wonder why the LSID cannot be same for the replicated services (I also feel that it is a bit strange, but I cannot find the real arguments why not to do it.). Regards, Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 26 08:21:40 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue Jul 26 08:12:45 2005 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42E12C2E.8010406@illuminae.com> Message-ID: > Just to expand this a little bit more - biomoby.org will NOT be the > resolver for that LSID, but it will be the authority server for that LSID. > Roman, this remark from Mark is true, but it may be obscure for you unless you know details of the LSID spec. The LSID resolution service specification actually specifies that the data/metadata resolution is done in several steps: - The first is called 'authority server resolution' (or similarly) - and this will be done at biomoby.org. This returns back a list of URLs (and protocols, but the protocols in our case will be only HTTP-GET). - The next step is to use these URLs and fetch what they represent - and this means that you are going to fetch RDF documents from the service provider site (so actually the service providers themselves suddenly become LSID resolution providers). Does it make sense for you? Cheers, Martin -- Martin Senger EMBL Outstation - Hinxton Senger@EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Fri Jul 29 17:35:17 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri Jul 29 17:58:33 2005 Subject: [MOBY-dev] 0.85 codebase running on test server Message-ID: <1122672917.19643.31.camel@bioinfo.icapture.ubc.ca> Hi dev-ers Anyone watching the commit logs will have seen a lot of activity in the past week! The 0.85 codebase is running now on a test server so that you can start to code to it. Endpoint information is as follows: URL => 'http://bioinfo.icapture.ubc.ca/cgi-bin/mobycentral/MOBY- Central.pl', URI => 'http://bioinfo.icapture.ubc.ca/MOBY/Central The main (visible) difference is that many of the messages now contain LSID information - see the 0.85 API on the biomoby.org website for full details. In addition the use of LSID's in MOBY Central messages should be MUCH more consistent now. The LSID resolver is (should be!) working, so if you try to resolve those LSID's to metadata you will get useful information. Under the hood, Dennis Wang has factored out all of the data-access calls into a single adaptor module (good job Dennis!). Hopefully this will simplify our migration toward the myGrid registry, since we will only need to modify one module. heads-up that Eddie is working on this same test server with the RDF agent, so things you register there may not last for long as they are obliterated by agent activity (and bugs ;-) ), but the MOBY Central code (what is currently in the CVS) seems to be running correctly on that machine so you can at least use it to observe the new message structure and update your code if necessary in anticipation of the production server switching to this codebase. If you have time/ambition/interest please do test this code and let me know if you notice anything incorrect. One of the biggest improvements, in my opinion, is that there is now a very comprehensive test suite for the Perl code - 122 tests! (up from just 12 before). All tests pass on two of my servers, so I think the code is now ~stable. From here on, any bugs that are reported will be fixed together with an addition to the test suite to make sure they don't reappear. As such, the codebase should never degrade again :-) Yes, we're growing up and becoming a mature project! Nota Bene that the test suite DOES NOT PASS ALL TESTS against the current production registry (@mobycentral.icapture.ubc.ca). This is because there are some latent bugs in the current production code that are now revealed by the test suite!! That's all the news. In the pipeline: 1) I will soon commit code changes that prevent inheritance from Primitives 2) I'll create and register replacement objects for all objects that currently inherit from primitives such that they follow the paradigm for object construction that has been extensively discussed since the last meeting. 2a) I will attempt to contact all service providers who will be affected by this change based on their registered contactEmail address 3) Eddie has a test suite of ~50 tests for the RDF agent, however it is not yet passing all tests. Once we believe this is stable/safe we will switch it on and from that point onwards deregistration of services will occur only through agent activity. 4) We are assembling a thesaurus of RDF predicates for service providers to use in their service instance metadata. This will be published shortly. Eddie and I leave for 10 days in Manchester on Sunday, so hopefully there will be more news and progress by the time we get home. Cheers everyone! Mark -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From edward.kawas at gmail.com Mon Jul 25 12:36:58 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Mon Aug 15 10:52:04 2005 Subject: [MOBY-dev] FW: Error handling proposal Message-ID: <42e5173f.2fdbe4fa.4c5a.ffffd788@mx.gmail.com> -----Original Message----- From: Oswaldo Trelles [mailto:ots@ac.uma.es] Sent: Monday, July 25, 2005 9:43 AM To: Eddie Kawas Subject: Error handling proposal Eddie, could you please circulate this message and make the document available following your protocol? thanks in advance Oswaldo Please let me introduce myself. My name is Oswaldo Trelles and I work at the Computer architecture Department of the University of Malaga, Spain (where all of you are invited and welcomed). Our group is in charge of the "Integrated Bioinformatics" aspects at the INB (National Institute of Bioinformatics). As David Gonz?les Pisano informed you at the Vancouver meeting we are developing an integrated platform to deploy general and specific services using BioMOBY as the underlying protocol. Thus we are quite concerned with BM specifications and we would like to contribute in the development, extension and improvements in the protocol. At this moment Roman has made a preliminary version and description of Theseu project available. Now we would like to put over the table the error handling issue. We have prepared a document, which represent an extension of our current implementation. This proposal was discussed during the INB-meeting here in Malaga (July 2005) and now is distributed as an open document for further discussion. Please feel free to comment. best regards, Oswaldo + GNV5 team + INB team -------------- next part -------------- A non-text attachment was scrubbed... Name: 0509GNV5-ErrorHandling-v034distributed.pdf Type: application/pdf Size: 338293 bytes Desc: not available Url : http://biomoby.org/pipermail/moby-dev/attachments/20050725/8461ee06/0509GNV5-ErrorHandling-v034distributed-0001.pdf From edward.kawas at gmail.com Wed Jul 27 11:20:26 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Mon Aug 15 10:52:09 2005 Subject: [MOBY-dev] FW: Error handling proposal Message-ID: <42e7a864.0e06ba61.207b.ffffdb6b@mx.gmail.com> Please let me introduce myself. My name is Oswaldo Trelles and I work at the Computer architecture Department of the University of Malaga, Spain (where all of you are invited and welcomed). Our group is in charge of the "Integrated Bioinformatics" aspects at the INB (National Institute of Bioinformatics). As David Gonz?les Pisano informed you at the Vancouver meeting we are developing an integrated platform to deploy general and specific services using BioMOBY as the underlying protocol. Thus we are quite concerned with BM specifications and we would like to contribute in the development, extension and improvements in the protocol. At this moment Roman has made a preliminary version and description of Theseu project available. Now we would like to put over the table the error handling issue. We have prepared a document, which represent an extension of our current implementation. This proposal was discussed during the INB-meeting here in Malaga (July 2005) and now is distributed as an open document for further discussion. Please feel free to comment. best regards, Oswaldo + GNV5 team + INB team -------------- next part -------------- A non-text attachment was scrubbed... Name: 0509GNV5-ErrorHandling-v034distributed.pdf Type: application/pdf Size: 190789 bytes Desc: not available Url : http://biomoby.org/pipermail/moby-dev/attachments/20050727/0549ca76/0509GNV5-ErrorHandling-v034distributed-0001.pdf From Pieter.Neerincx at wur.nl Wed Jul 6 10:50:42 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Wed, 6 Jul 2005 16:50:42 +0200 Subject: [MOBY-dev] Perl API: MOBY::Central::DUMP_MySQL() In-Reply-To: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> References: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> Message-ID: <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> Hi all, This message is for those who use Perl and have local MOBY Centrals. I had some problems with the way MOBY::Central::DUMP_MySQL() currently works. Getting a DUMP fails for example with the new central at: URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl URI: http://mobycentral.icapture.ubc.ca/MOBY/Central Currently DUMP_MySQL() has the mysql username, the path to mysqldump and the lack of a password, host and portnumber hard coded. Hence it ignores whatever you have in your mobycentral.config file. Therefore I propose the code below which uses MOBY::Config->new() to get the connection details from your mobycentral.config. The path to mysqldump is no longer hard coded. Hence if it's not in the path of webserver user, I think it's better to modify the path env var as compared to hard coding it in the perl modules:) Unless anyone has objections against the porposed code I'll merge this with the code in the CVS in a few days... Cheers, Pieter -------------------------------------------------- sub DUMP_MySQL { my ( $pkg ) = @_; my $config = MOBY::Config->new(); my @dbsections = ('mobycentral', 'mobyobject', 'mobyservice', 'mobynamespace', 'mobyrelationship'); my @response; foreach my $dbsection (@dbsections) { my $dbname = ${${$config}{$dbsection}}{'dbname'}; my $username = ${${$config}{$dbsection}}{'username'}; my $password = ${${$config}{$dbsection}}{'password'}; my $host = ${${$config}{$dbsection}}{'url'}; my $port = ${${$config}{$dbsection}}{'port'}; open( IN, "mysqldump -h $host -P $port -u $username --password=$password $dbname|" ) || die "can't open $dbname for dumping"; my @dbdump; while ( ) { push @dbdump, $_; } my $dbdump = ( join "", @dbdump ); push @response, $dbdump; } return [@response]; } ------------------------------------------------ Wageningen University and Research centre (WUR) Laboratory of Bioinformatics Transitorium (building 312) room 1038 Dreijenlaan 3 6703 HA Wageningen phone: 0317-484 706 fax: 0317-483 584 mobile: 06-143 66 783 pieter.neerincx at wur.nl From markw at illuminae.com Wed Jul 6 11:09:44 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 06 Jul 2005 08:09:44 -0700 Subject: [moby] [MOBY-dev] Perl API: MOBY::Central::DUMP_MySQL() In-Reply-To: <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> References: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> Message-ID: <1120662584.8869.38.camel@mobycentral.mrl.ubc.ca> Hi Pieter, I didn't realize that the calls were hard-coded! Ugh! Thanks for catching that error - please do go ahead and fix the code. We will be committing a major change to the organization of SQL in the codebase in the next few days, so once you have committed I am going to tag this version and make a release before the chaos ensues :-) M On Wed, 2005-07-06 at 16:50 +0200, Pieter Neerincx wrote: > Hi all, > > This message is for those who use Perl and have local MOBY Centrals. > I had some problems with the way MOBY::Central::DUMP_MySQL() > currently works. Getting a DUMP fails for example with the new > central at: > > URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl > URI: http://mobycentral.icapture.ubc.ca/MOBY/Central > > Currently DUMP_MySQL() has the mysql username, the path to mysqldump > and the lack of a password, host and portnumber hard coded. Hence it > ignores whatever you have in your mobycentral.config file. Therefore > I propose the code below which uses MOBY::Config->new() to get the > connection details from your mobycentral.config. The path to > mysqldump is no longer hard coded. Hence if it's not in the path of > webserver user, I think it's better to modify the path env var as > compared to hard coding it in the perl modules:) Unless anyone has > objections against the porposed code I'll merge this with the code in > the CVS in a few days... > > Cheers, > > Pieter > > -------------------------------------------------- > sub DUMP_MySQL { > my ( $pkg ) = @_; > my $config = MOBY::Config->new(); > my @dbsections = ('mobycentral', 'mobyobject', > 'mobyservice', 'mobynamespace', 'mobyrelationship'); > my @response; > > foreach my $dbsection (@dbsections) { > my $dbname = ${${$config}{$dbsection}}{'dbname'}; > my $username = ${${$config}{$dbsection}}{'username'}; > my $password = ${${$config}{$dbsection}}{'password'}; > my $host = ${${$config}{$dbsection}}{'url'}; > my $port = ${${$config}{$dbsection}}{'port'}; > open( IN, "mysqldump -h $host -P $port -u $username > --password=$password $dbname|" ) > || die "can't open $dbname for dumping"; > my @dbdump; > while ( ) { > push @dbdump, $_; > } > my $dbdump = ( join "", @dbdump ); > push @response, $dbdump; > } > return [@response]; > } > ------------------------------------------------ > > > > Wageningen University and Research centre (WUR) > Laboratory of Bioinformatics > Transitorium (building 312) room 1038 > Dreijenlaan 3 > 6703 HA Wageningen > phone: 0317-484 706 > fax: 0317-483 584 > mobile: 06-143 66 783 > pieter.neerincx at wur.nl > > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From Pieter.Neerincx at wur.nl Wed Jul 6 11:41:59 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Wed, 6 Jul 2005 17:41:59 +0200 Subject: [moby] [MOBY-dev] Perl API: MOBY::Central::DUMP_MySQL() In-Reply-To: <1120662584.8869.38.camel@mobycentral.mrl.ubc.ca> References: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> <1120662584.8869.38.camel@mobycentral.mrl.ubc.ca> Message-ID: Hi Mark, On 6-Jul-2005, at 5:09 PM, Mark Wilkinson wrote: > Hi Pieter, > > I didn't realize that the calls were hard-coded! Ugh! > > Thanks for catching that error - please do go ahead and fix the code. Ok, done. > > We will be committing a major change to the organization of SQL in the > codebase in the next few days, so once you have committed I am > going to > tag this version and make a release before the chaos ensues :-) Whoops, good luck! Cheers, Pieter > > M > > > On Wed, 2005-07-06 at 16:50 +0200, Pieter Neerincx wrote: > >> Hi all, >> >> This message is for those who use Perl and have local MOBY Centrals. >> I had some problems with the way MOBY::Central::DUMP_MySQL() >> currently works. Getting a DUMP fails for example with the new >> central at: >> >> URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl >> URI: http://mobycentral.icapture.ubc.ca/MOBY/Central >> >> Currently DUMP_MySQL() has the mysql username, the path to mysqldump >> and the lack of a password, host and portnumber hard coded. Hence it >> ignores whatever you have in your mobycentral.config file. Therefore >> I propose the code below which uses MOBY::Config->new() to get the >> connection details from your mobycentral.config. The path to >> mysqldump is no longer hard coded. Hence if it's not in the path of >> webserver user, I think it's better to modify the path env var as >> compared to hard coding it in the perl modules:) Unless anyone has >> objections against the porposed code I'll merge this with the code in >> the CVS in a few days... >> >> Cheers, >> >> Pieter >> >> -------------------------------------------------- >> sub DUMP_MySQL { >> my ( $pkg ) = @_; >> my $config = MOBY::Config->new(); >> my @dbsections = ('mobycentral', 'mobyobject', >> 'mobyservice', 'mobynamespace', 'mobyrelationship'); >> my @response; >> >> foreach my $dbsection (@dbsections) { >> my $dbname = ${${$config}{$dbsection}}{'dbname'}; >> my $username = ${${$config}{$dbsection}} >> {'username'}; >> my $password = ${${$config}{$dbsection}} >> {'password'}; >> my $host = ${${$config}{$dbsection}}{'url'}; >> my $port = ${${$config}{$dbsection}}{'port'}; >> open( IN, "mysqldump -h $host -P $port -u $username >> --password=$password $dbname|" ) >> || die "can't open $dbname for dumping"; >> my @dbdump; >> while ( ) { >> push @dbdump, $_; >> } >> my $dbdump = ( join "", @dbdump ); >> push @response, $dbdump; >> } >> return [@response]; >> } >> ------------------------------------------------ >> >> >> >> Wageningen University and Research centre (WUR) >> Laboratory of Bioinformatics >> Transitorium (building 312) room 1038 >> Dreijenlaan 3 >> 6703 HA Wageningen >> phone: 0317-484 706 >> fax: 0317-483 584 >> mobile: 06-143 66 783 >> pieter.neerincx at wur.nl >> >> >> _______________________________________________ >> MOBY-dev mailing list >> MOBY-dev at biomoby.org >> http://www.biomoby.org/mailman/listinfo/moby-dev >> > -- > Mark Wilkinson > Asst. Professor > Dept. of Medical Genetics > University of British Columbia > PI in Bioinformatics > iCAPTURE Centre > St. Paul's Hospital > Vancouver, BC > > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > Wageningen University and Research centre (WUR) Laboratory of Bioinformatics Transitorium (building 312) room 1038 Dreijenlaan 3 6703 HA Wageningen phone: 0317-484 706 fax: 0317-483 584 mobile: 06-143 66 783 pieter.neerincx at wur.nl From markw at illuminae.com Wed Jul 6 12:35:19 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 06 Jul 2005 09:35:19 -0700 Subject: [MOBY-dev] Bad documetation for Secondary parameters in MOBY Central Message-ID: <1120667719.8869.75.camel@mobycentral.mrl.ubc.ca> Hi all, I just got a heads-up from Eddie that the documentation for MOBY::Central was incorrect. It had enumerated the valid secondary data-types as STRING|INT|FLOAT, etc. They are, in fact, String, Integer, Float, DateTime (capitalization differences). This is tested at the DB level, so it causes some mis-registration if you aren't aware of the problem. We're fixing the code to throw a warning when it detects a mistaken parameter, but I wanted to let you know that this is an issue in case you have been wondering why you can't register secondaries :-/ M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From rebecca.ernst at gsf.de Thu Jul 7 09:45:35 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Thu, 07 Jul 2005 15:45:35 +0200 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <42CD31FF.6040803@gsf.de> Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From markw at illuminae.com Thu Jul 7 09:56:16 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 13:56:16 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1562815689-1120744590-cardhu_blackberry.rim.net-11062-@engine12-cell01> I guess it doesn't make you feel any better if I say "yes, we know..." :-) M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas at gmail.com, markw at illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 09:56:16 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 13:56:16 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1562815689-1120744590-cardhu_blackberry.rim.net-11062-@engine12-cell01> I guess it doesn't make you feel any better if I say "yes, we know..." :-) M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas at gmail.com, markw at illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 09:58:53 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 13:58:53 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> We really need to get write-access to the Taverna CVS so that we can fix these kinds of things in a timely way... Hint hint :-). Anyone from myGrid listening? M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas at gmail.com, markw at illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 09:58:53 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 13:58:53 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> We really need to get write-access to the Taverna CVS so that we can fix these kinds of things in a timely way... Hint hint :-). Anyone from myGrid listening? M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas at gmail.com, markw at illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From schoof at mpiz-koeln.mpg.de Thu Jul 7 11:34:39 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Thu, 7 Jul 2005 17:34:39 +0200 Subject: [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CD31FF.6040803@gsf.de> References: <42CD31FF.6040803@gsf.de> Message-ID: <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> Please let's take a minute and review the use of collections in BioMoby, before we go off to create "workarounds"... I myself am confused about the use of collections. Originally I had in my mind that Collections were a construct to allow objects that inherently belong together to be "bagged". Example: A multiple alignment program that takes any number of sequences as input. Example2: A keyword search that takes any number of keywords and then does a query combining all of them. As opposed to: Inputing a list of keywords and executing the keyword search for each of them, which would require a separate moby:mobyData and queryID for each of the keywords. The confusion starts with the output of services. My understanding was that ONLY a service that is guaranteed to output exactly one object for each query (e.g. an averaging service that outputs the average of a list of inputs) is registered as outputting a Simple, all others have to output collections (as there must be exactly one mobyData matching the queryID of the input in the response, and a mobyData may contain multiple Simple elements only if wrapped by a Collection). This makes it impossible to distinguish between the situation where one query produces multiple unrelated results, versus one or more bags of related results. Imagine a "getnonoverlappingsubstrings" service: with input abc, it should output [a, bc], [ab, c] and [a, b, c]. Outputting [a, bc, ab, c, a, b, c] would not be useful. More biological example: Dirk has a service that returns sets of orthologous genes when given a set of species. For this, he requires collections both for input and for output. The ideal situation in my view would be: Input: Collection of Species Output: Many Collections, each Collection contains orthologous genes (one from each species). The Collection here defines a set of orthologs, and using a collection would be more elegant than having to define a Moby object "OrhologSet" has (2...n) Objects In practice, the Collection tag has been used to indicate when more than one Simple occurs, with no "semantic" meaning. This imho is not necessary; when more than one Simple occurs, why not put more than one Simple? It's easy enough for everyone to figure that out. Then, Collection could be used to actually transfer meaning ;-) This is a drastic change in the way the API is being interpreted, and will break code. So it needs calm thinking. But with the "big" API change coming up for the Primitives, it could be done. And it would make things clearer, also to e.g. the Taverna developers: Getting back many Simple articles in response to a query very intuitively indicates to continue on with each one individually, whereas getting back a Collection indicates to put the whole thing as input into the next service, which is what they implemented. Makes perfect sense, as there can and will be services that consume Collections. E.g., the ortholog set case above: Pipe it into a Multiple Alignment service, and you get all the alignments for each of the set of orthologs. Getting one massive alignment of everything wouldn't make sense. Maybe I've made myself clear, maybe not. Anyway, the Collection issue has led to quite some discussions between Rebecca, Dirk and myself, and we are all not happy with the way they are currently handled. Best, Heiko On 7. Jul 2005, at 15:45 Uhr, Rebecca Ernst wrote: Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From tmo at ebi.ac.uk Thu Jul 7 11:50:31 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Thu, 07 Jul 2005 16:50:31 +0100 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> References: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> Message-ID: <42CD4F47.4030901@ebi.ac.uk> mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom From tmo at ebi.ac.uk Thu Jul 7 11:50:31 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Thu, 07 Jul 2005 16:50:31 +0100 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> References: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> Message-ID: <42CD4F47.4030901@ebi.ac.uk> mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom From markw at illuminae.com Thu Jul 7 12:25:13 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 16:25:13 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1289629624-1120753526-cardhu_blackberry.rim.net-7512-@engine08-cell01> :-). We have all been well and truly spanked :-) Everything Tom says is (obviously) correct, and should apply to the moby CVS commits also. Thanks for giving Eddie access! M . -----Original Message----- From: Tom Oinn Date: Thu, 07 Jul 2005 16:50:31 To:markw at illuminae.com, Core developer announcements Cc:Martin Senger , mobydev Subject: Re: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 12:25:13 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 16:25:13 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1289629624-1120753526-cardhu_blackberry.rim.net-7512-@engine08-cell01> :-). We have all been well and truly spanked :-) Everything Tom says is (obviously) correct, and should apply to the moby CVS commits also. Thanks for giving Eddie access! M . -----Original Message----- From: Tom Oinn Date: Thu, 07 Jul 2005 16:50:31 To:markw at illuminae.com, Core developer announcements Cc:Martin Senger , mobydev Subject: Re: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 16:07:48 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Thu, 07 Jul 2005 13:07:48 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> Message-ID: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Thu Jul 7 16:07:48 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Thu, 07 Jul 2005 13:07:48 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> Message-ID: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From schoof at mpiz-koeln.mpg.de Fri Jul 8 02:45:28 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Fri, 8 Jul 2005 08:45:28 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> Message-ID: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Well, if I understand you correctly, Mark, than I was right all along in that they way I see collections is the way they should be seen. Then the problem is that that is not clear enough to everybody else, and that possibly the examples in the API are misleading, because it's been used in such a different way. It's definitely something that needs to be fixed before release 1.0. If I understand Tom correctly, the problem mainly appeared because the BioMoby support was rapidly and effectively incorporated into Taverna without proper integration, and what needs to happen is to define e.g. the iteration strategy for BioMoby: Taverna so far doesn't use the possibility to bundle multiple queries in a single BioMoby message, which is not a problem, but the Collection issue could point to a need for us BioMoby guys to look in more detail at Taverna data types and do a sound mapping. Heiko On 7. Jul 2005, at 22:07 Uhr, Mark Wilkinson wrote: On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From schoof at mpiz-koeln.mpg.de Fri Jul 8 02:45:28 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Fri, 8 Jul 2005 08:45:28 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> Message-ID: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Well, if I understand you correctly, Mark, than I was right all along in that they way I see collections is the way they should be seen. Then the problem is that that is not clear enough to everybody else, and that possibly the examples in the API are misleading, because it's been used in such a different way. It's definitely something that needs to be fixed before release 1.0. If I understand Tom correctly, the problem mainly appeared because the BioMoby support was rapidly and effectively incorporated into Taverna without proper integration, and what needs to happen is to define e.g. the iteration strategy for BioMoby: Taverna so far doesn't use the possibility to bundle multiple queries in a single BioMoby message, which is not a problem, but the Collection issue could point to a need for us BioMoby guys to look in more detail at Taverna data types and do a sound mapping. Heiko On 7. Jul 2005, at 22:07 Uhr, Mark Wilkinson wrote: On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From rebecca.ernst at gsf.de Fri Jul 8 02:50:17 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Fri, 08 Jul 2005 08:50:17 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> Message-ID: <42CE2229.1030504@gsf.de> Hi Mark! I will have to spank you a lot for either the last email or the one you sent me in February where I asked you how to use collections (see mail below): Well... the answer depends on what the generic case of your output will look like. the rule is that you have to register every output object that your service produces; i.e. every immediate child tag of a mobyData block. Therefore, if you are going to consistently output exactly 3 Genbank/gi's, then you would register your service as outputting 3 X Object(NCBI_gi). If you are outputting an indeterminate number of objects, then you register it as outputting 1 X Collection. Collections map nicely onto the rdf:Bag structure, if that helps you interpret them. M On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: >> Hi Mark! >> >> We have problems interpreting the BioMoby API for the output objects. >> >> The thing is that we have a service that gets ONE input object and >> executes a freetext search over several databases. This service will >> eventually return more that one result. >> We don't want to put the result into a collection object because the >> single objects don't build any entity. >> We also can't give back more that one mobyData block because we only >> have one query ID and therefore only one queryID to give back. >> >> The only solution we can think of is an object like this: >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> would this be a legal object or do we have to use collection even if the >> objects don't build an entity ? >> >> >> Cheers, >> Rebecca > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical Genetics, UBC, Canada ----------------------------------------------------------------------------------------------------------- Mark Wilkinson wrote: >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > >>I myself am confused about the use of collections. Originally I had in >>my mind that Collections were a construct to allow objects that >>inherently belong together to be "bagged". >> >> > >You are 100% correct here. > > > > >>The confusion starts with the output of services. My understanding was >>that ONLY a service that is guaranteed to output exactly one object for >>each query (e.g. an averaging service that outputs the average of a >>list of inputs) is registered as outputting a Simple, all others have >>to output collections >> >> > >No. > > > >> (as there must be exactly one mobyData matching >>the queryID of the input in the response, >> >> > >Yes > > > >> and a mobyData may contain >>multiple Simple elements only if wrapped by a Collection). >> >> > >No. A mobyData element can wrap any number of outputs, and/or >combinations of simples, collections, and secondaries. > > > > > >>In practice, the Collection tag has been used to indicate when more >>than one Simple occurs, with no "semantic" meaning. This imho is not >>necessary; when more than one Simple occurs, why not put more than one >>Simple? It's easy enough for everyone to figure that out. Then, >>Collection could be used to actually transfer meaning ;-) >> >> > >I think a lot of people are using Collections incorrectly, yes. > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back >>many Simple articles in response to a query very intuitively indicates >>to continue on with each one individually, whereas getting back a >>Collection indicates to put the whole thing as input into the next >>service, which is what they implemented. Makes perfect sense, as there >>can and will be services that consume Collections. >> >> > >I don't think that Taverna handled collections correctly in the past, >and it would be a shame if that identical functionality was added back >in :-) The functionality does need to go back into Taverna, but >correctly this time. > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue >>has led to quite some discussions between Rebecca, Dirk and myself, and >>we are all not happy with the way they are currently handled. >> >> > >I don't know if you are happier with them now that I have pointed out >where you are misinterpreting them...?? > >I agree, however, that their usage by many service providers is not >correct... but there's no way for MOBY Central to know that it isn't >correct, so we can't throw an error on registration of these "wonky" >services... > >M > > > > > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From tmo at ebi.ac.uk Fri Jul 8 04:07:44 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Fri, 08 Jul 2005 09:07:44 +0100 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <42CE3450.1000209@ebi.ac.uk> Heiko Schoof wrote: > If I understand Tom correctly, the problem mainly appeared because the > BioMoby support was rapidly and effectively incorporated into Taverna > without proper integration, and what needs to happen is to define e.g. > the iteration strategy for BioMoby: Taverna so far doesn't use the > possibility to bundle multiple queries in a single BioMoby message, > which is not a problem, but the Collection issue could point to a need > for us BioMoby guys to look in more detail at Taverna data types and do > a sound mapping. > Heiko That's kind of true, meaning actually not. There are three cases involving collections (Taverna 1.1 behaviour) : 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. The intent is that as with the other plugin types Taverna guarantees that the service sees the inputs it has asked for. Our experience with other service types suggests that this is an extremely powerful mechanism as it allows interoperability between services that would otherwise mismatch - it's worth noting that our users expect these services to match, while a CS perspective regards ProteinSequence and ProteinSequence[] as completely different types most of our biologists don't! Taverna behaves the way _they_ expect it to, remember who your user are. Taverna data types are pretty much trivial, they're opaque data blobs with the exception of collection structure which is exposed. We only expose the collection structure to ensure the above properties, other than that the framework is data agnostic (as it should be). Tom From tmo at ebi.ac.uk Fri Jul 8 04:07:44 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Fri, 08 Jul 2005 09:07:44 +0100 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <42CE3450.1000209@ebi.ac.uk> Heiko Schoof wrote: > If I understand Tom correctly, the problem mainly appeared because the > BioMoby support was rapidly and effectively incorporated into Taverna > without proper integration, and what needs to happen is to define e.g. > the iteration strategy for BioMoby: Taverna so far doesn't use the > possibility to bundle multiple queries in a single BioMoby message, > which is not a problem, but the Collection issue could point to a need > for us BioMoby guys to look in more detail at Taverna data types and do > a sound mapping. > Heiko That's kind of true, meaning actually not. There are three cases involving collections (Taverna 1.1 behaviour) : 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. The intent is that as with the other plugin types Taverna guarantees that the service sees the inputs it has asked for. Our experience with other service types suggests that this is an extremely powerful mechanism as it allows interoperability between services that would otherwise mismatch - it's worth noting that our users expect these services to match, while a CS perspective regards ProteinSequence and ProteinSequence[] as completely different types most of our biologists don't! Taverna behaves the way _they_ expect it to, remember who your user are. Taverna data types are pretty much trivial, they're opaque data blobs with the exception of collection structure which is exposed. We only expose the collection structure to ensure the above properties, other than that the framework is data agnostic (as it should be). Tom From d.haase at gsf.de Fri Jul 8 04:49:58 2005 From: d.haase at gsf.de (Dirk Haase) Date: Fri, 8 Jul 2005 10:49:58 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <200507081049.58799.d.haase@gsf.de> On Friday 08 July 2005 08:45, Heiko Schoof wrote: > Well, if I understand you correctly, Mark, than I was right all along > in that they way I see collections is the way they should be seen. Then > the problem is that that is not clear enough to everybody else, and > that possibly the examples in the API are misleading, because it's been > used in such a different way. It's definitely something that needs to > be fixed before release 1.0. Not only the examples are misleading, actually at least the PERL API is lacking a suitable method to construct an output consisting of several 'Simple's. One can not simply concat the output of multibple 'simpleResponse' calls, as this would create several mobyData blocks. In order to fill this gap, I created a new method 'multiSimpleResponse' (see below) within CommonSubs.pm which is more or less identical to 'collectionResponse' except that it obviously leaves out the moby:Collection tag. The provided articleName is put into each 'Simple' tag which should be appropriate. Any objections? Regards, dirk =head2 multiSimpleResponse name : multiSimpleResponse function : wraps a set of simple articles in the appropriate mobyData structure usage : return responseHeader . &multiSimpleResponse(\@objects, 'MyArticleName', $queryID) . responseFooter; args : (in order) \@objects - (optional) a listref of MOBY Objects as raw XML $article - (optional) an articeName for this article $queryID - (optional, but strongly recommended) the mobyData ID to which you are responding notes : as required by the API you must return a response for every input. If one of the inputs was invalid, you return a valid (empty) MOBY response by calling &multiSimpleResponse(undef, undef, $queryID). =cut sub multiSimpleResponse { my ( $data, $articleName, $qID ) = @_; # articleName optional my $content = ""; $data ||= []; $qID ||= ''; $articleName ||=""; unless ( ( ref( $data ) =~ /array/i ) && $data->[0] ) { # we're expecting an arrayref as input data,and it must not be empty return ""; } foreach ( @{$data} ) { if ( $_ ) { $content .= " $_ "; } else { $content .= " "; } } return " $content "; } From d.haase at gsf.de Fri Jul 8 04:49:58 2005 From: d.haase at gsf.de (Dirk Haase) Date: Fri, 8 Jul 2005 10:49:58 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <200507081049.58799.d.haase@gsf.de> On Friday 08 July 2005 08:45, Heiko Schoof wrote: > Well, if I understand you correctly, Mark, than I was right all along > in that they way I see collections is the way they should be seen. Then > the problem is that that is not clear enough to everybody else, and > that possibly the examples in the API are misleading, because it's been > used in such a different way. It's definitely something that needs to > be fixed before release 1.0. Not only the examples are misleading, actually at least the PERL API is lacking a suitable method to construct an output consisting of several 'Simple's. One can not simply concat the output of multibple 'simpleResponse' calls, as this would create several mobyData blocks. In order to fill this gap, I created a new method 'multiSimpleResponse' (see below) within CommonSubs.pm which is more or less identical to 'collectionResponse' except that it obviously leaves out the moby:Collection tag. The provided articleName is put into each 'Simple' tag which should be appropriate. Any objections? Regards, dirk =head2 multiSimpleResponse name : multiSimpleResponse function : wraps a set of simple articles in the appropriate mobyData structure usage : return responseHeader . &multiSimpleResponse(\@objects, 'MyArticleName', $queryID) . responseFooter; args : (in order) \@objects - (optional) a listref of MOBY Objects as raw XML $article - (optional) an articeName for this article $queryID - (optional, but strongly recommended) the mobyData ID to which you are responding notes : as required by the API you must return a response for every input. If one of the inputs was invalid, you return a valid (empty) MOBY response by calling &multiSimpleResponse(undef, undef, $queryID). =cut sub multiSimpleResponse { my ( $data, $articleName, $qID ) = @_; # articleName optional my $content = ""; $data ||= []; $qID ||= ''; $articleName ||=""; unless ( ( ref( $data ) =~ /array/i ) && $data->[0] ) { # we're expecting an arrayref as input data,and it must not be empty return ""; } foreach ( @{$data} ) { if ( $_ ) { $content .= " $_ "; } else { $content .= " "; } } return " $content "; } From schoof at mpiz-koeln.mpg.de Fri Jul 8 04:54:03 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Fri, 8 Jul 2005 10:54:03 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <24407d5355834f57f8ac3e2245daa77d@mpiz-koeln.mpg.de> Point taken to prove that indeed, I have to look into Taverna in more depth ;-) Anyway, Tom, what you state below is exactly what I would expect as behaviour, but I think on the BioMoby side we must ensure that our "Collections" behave in the right way, and I'm not yet sure they are precisely the same thing as what you call collection. Best, Heiko On 8. Jul 2005, at 10:07 Uhr, Tom Oinn wrote: Heiko Schoof wrote: > If I understand Tom correctly, the problem mainly appeared because the > BioMoby support was rapidly and effectively incorporated into Taverna > without proper integration, and what needs to happen is to define e.g. > the iteration strategy for BioMoby: Taverna so far doesn't use the > possibility to bundle multiple queries in a single BioMoby message, > which is not a problem, but the Collection issue could point to a need > for us BioMoby guys to look in more detail at Taverna data types and > do a sound mapping. > Heiko That's kind of true, meaning actually not. There are three cases involving collections (Taverna 1.1 behaviour) : 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. The intent is that as with the other plugin types Taverna guarantees that the service sees the inputs it has asked for. Our experience with other service types suggests that this is an extremely powerful mechanism as it allows interoperability between services that would otherwise mismatch - it's worth noting that our users expect these services to match, while a CS perspective regards ProteinSequence and ProteinSequence[] as completely different types most of our biologists don't! Taverna behaves the way _they_ expect it to, remember who your user are. Taverna data types are pretty much trivial, they're opaque data blobs with the exception of collection structure which is exposed. We only expose the collection structure to ensure the above properties, other than that the framework is data agnostic (as it should be). Tom _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From rebecca.ernst at gsf.de Fri Jul 8 05:07:50 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Fri, 08 Jul 2005 11:07:50 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <42CE4266.3030304@gsf.de> Hi Tom and others This Taverna behaviour described below makes perfect sense to me! Does Taverna then check in MOBY-Central how this service is registered (taking simples or collections)? I guess the way to go would be to: 1. change the Taverna behaviour back to how it is supposed to work (and did in 1.1) 2. - change the BioMoby Perl code to allow more than one simple in the MobyData AND - change almost all services that output collections to the way we discussed yesterday (only output collections if the objects build an entity. ) Either of the two changes solves the problems but I believe both are due to changes and should take place. Best, Rebecca Tom Oinn wrote: > That's kind of true, meaning actually not. There are three cases > involving collections (Taverna 1.1 behaviour) : > > 1) Consumer declares it consumes singles, Producer emits a collection. > In this context Taverna iteratively calls the Consumer with each item > from the collection. This is probably what you'd expect to happen, the > result is that the Consumer effectively emits a collection of whatever > it would emit normally. > 2) Consumer declares it consumes a collection, Producer emits a > collection. In this case Taverna will indeed split the output > collection (because we always do) but it will be magically reassembled > before being given to the Consumer. > > 3) Consumer declares it consumes a collection, Producer emits a single > item. Taverna wraps the single item in a single element collection and > gives it to the Consumer. > > The intent is that as with the other plugin types Taverna guarantees > that the service sees the inputs it has asked for. Our experience with > other service types suggests that this is an extremely powerful > mechanism as it allows interoperability between services that would > otherwise mismatch - it's worth noting that our users expect these > services to match, while a CS perspective regards ProteinSequence and > ProteinSequence[] as completely different types most of our biologists > don't! Taverna behaves the way _they_ expect it to, remember who your > user are. > > Taverna data types are pretty much trivial, they're opaque data blobs > with the exception of collection structure which is exposed. We only > expose the collection structure to ensure the above properties, other > than that the framework is data agnostic (as it should be). > > Tom > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From rebecca.ernst at gsf.de Fri Jul 8 05:07:50 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Fri, 08 Jul 2005 11:07:50 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <42CE4266.3030304@gsf.de> Hi Tom and others This Taverna behaviour described below makes perfect sense to me! Does Taverna then check in MOBY-Central how this service is registered (taking simples or collections)? I guess the way to go would be to: 1. change the Taverna behaviour back to how it is supposed to work (and did in 1.1) 2. - change the BioMoby Perl code to allow more than one simple in the MobyData AND - change almost all services that output collections to the way we discussed yesterday (only output collections if the objects build an entity. ) Either of the two changes solves the problems but I believe both are due to changes and should take place. Best, Rebecca Tom Oinn wrote: > That's kind of true, meaning actually not. There are three cases > involving collections (Taverna 1.1 behaviour) : > > 1) Consumer declares it consumes singles, Producer emits a collection. > In this context Taverna iteratively calls the Consumer with each item > from the collection. This is probably what you'd expect to happen, the > result is that the Consumer effectively emits a collection of whatever > it would emit normally. > 2) Consumer declares it consumes a collection, Producer emits a > collection. In this case Taverna will indeed split the output > collection (because we always do) but it will be magically reassembled > before being given to the Consumer. > > 3) Consumer declares it consumes a collection, Producer emits a single > item. Taverna wraps the single item in a single element collection and > gives it to the Consumer. > > The intent is that as with the other plugin types Taverna guarantees > that the service sees the inputs it has asked for. Our experience with > other service types suggests that this is an extremely powerful > mechanism as it allows interoperability between services that would > otherwise mismatch - it's worth noting that our users expect these > services to match, while a CS perspective regards ProteinSequence and > ProteinSequence[] as completely different types most of our biologists > don't! Taverna behaves the way _they_ expect it to, remember who your > user are. > > Taverna data types are pretty much trivial, they're opaque data blobs > with the exception of collection structure which is exposed. We only > expose the collection structure to ensure the above properties, other > than that the framework is data agnostic (as it should be). > > Tom > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From jmfernandez at cnb.uam.es Fri Jul 8 07:12:37 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Fri, 08 Jul 2005 13:12:37 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna1.2 In-Reply-To: <42CE4266.3030304@gsf.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> Message-ID: <42CE5FA5.50901@cnb.uam.es> Hi everyone, the Taverna v1.1 behaviour for MOBY should be the default one for services which only takes one input parameter. But, which collection semantics should have Taverna MOBY plugin for services with more than one parameter? All of this is related to iteration strategies. For instance, imagine this scenario: we have a service which needs two input Simple parameters, A and B. What should we do? * If both input parameters receive a Simple, no problem, default behaviour. * If one of those parameters receive a Collection with N Simples, and the other a Simple, the default iteration strategy should be either invoking the MOBY service N times with N separate SOAP submissions, or invoking the MOBY service N times with only a SOAP submission (or something intermediate). * If each one of the parameters receive a Collection, then the iteration strategy *should* be explicitly set by the workflow creator: either cartesian product (first of A with each one of B, second of A with each one of B) or dot product (first for A with first for B, second for A with second for B, and so on, either discarding the last elements from the longest Collection, or firing an exception/error/message/whatever). The other two scenarios are: a two-input service with both parameters defined as Collection; and a two-input service with one the parameters as Collection and the other as Simple. I leave them as an exercise for the reader 8-), so I'm avoiding to write now the most boring e-mail in BioMOBY story ;-) Best Regards, Jos? Mar?a Rebecca Ernst wrote: > Hi Tom and others > > This Taverna behaviour described below makes perfect sense to me! Does > Taverna then check in MOBY-Central how this service is registered > (taking simples or collections)? > > I guess the way to go would be to: > 1. change the Taverna behaviour back to how it is supposed to work (and > did in 1.1) > 2. - change the BioMoby Perl code to allow more than one simple in the > MobyData AND > - change almost all services that output collections to the way we > discussed yesterday (only output collections if the objects build an > entity. ) > > Either of the two changes solves the problems but I believe both are due > to changes and should take place. > > Best, > Rebecca > > > > > > Tom Oinn wrote: > >> That's kind of true, meaning actually not. There are three cases >> involving collections (Taverna 1.1 behaviour) : >> >> 1) Consumer declares it consumes singles, Producer emits a collection. >> In this context Taverna iteratively calls the Consumer with each item >> from the collection. This is probably what you'd expect to happen, the >> result is that the Consumer effectively emits a collection of whatever >> it would emit normally. > > >> 2) Consumer declares it consumes a collection, Producer emits a >> collection. In this case Taverna will indeed split the output >> collection (because we always do) but it will be magically reassembled >> before being given to the Consumer. >> >> 3) Consumer declares it consumes a collection, Producer emits a single >> item. Taverna wraps the single item in a single element collection and >> gives it to the Consumer. >> >> The intent is that as with the other plugin types Taverna guarantees >> that the service sees the inputs it has asked for. Our experience with >> other service types suggests that this is an extremely powerful >> mechanism as it allows interoperability between services that would >> otherwise mismatch - it's worth noting that our users expect these >> services to match, while a CS perspective regards ProteinSequence and >> ProteinSequence[] as completely different types most of our biologists >> don't! Taverna behaves the way _they_ expect it to, remember who your >> user are. >> >> Taverna data types are pretty much trivial, they're opaque data blobs >> with the exception of collection structure which is exposed. We only >> expose the collection structure to ensure the above properties, other >> than that the framework is data agnostic (as it should be). >> >> Tom >> > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From jmfernandez at cnb.uam.es Fri Jul 8 07:12:37 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Fri, 08 Jul 2005 13:12:37 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna1.2 In-Reply-To: <42CE4266.3030304@gsf.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> Message-ID: <42CE5FA5.50901@cnb.uam.es> Hi everyone, the Taverna v1.1 behaviour for MOBY should be the default one for services which only takes one input parameter. But, which collection semantics should have Taverna MOBY plugin for services with more than one parameter? All of this is related to iteration strategies. For instance, imagine this scenario: we have a service which needs two input Simple parameters, A and B. What should we do? * If both input parameters receive a Simple, no problem, default behaviour. * If one of those parameters receive a Collection with N Simples, and the other a Simple, the default iteration strategy should be either invoking the MOBY service N times with N separate SOAP submissions, or invoking the MOBY service N times with only a SOAP submission (or something intermediate). * If each one of the parameters receive a Collection, then the iteration strategy *should* be explicitly set by the workflow creator: either cartesian product (first of A with each one of B, second of A with each one of B) or dot product (first for A with first for B, second for A with second for B, and so on, either discarding the last elements from the longest Collection, or firing an exception/error/message/whatever). The other two scenarios are: a two-input service with both parameters defined as Collection; and a two-input service with one the parameters as Collection and the other as Simple. I leave them as an exercise for the reader 8-), so I'm avoiding to write now the most boring e-mail in BioMOBY story ;-) Best Regards, Jos? Mar?a Rebecca Ernst wrote: > Hi Tom and others > > This Taverna behaviour described below makes perfect sense to me! Does > Taverna then check in MOBY-Central how this service is registered > (taking simples or collections)? > > I guess the way to go would be to: > 1. change the Taverna behaviour back to how it is supposed to work (and > did in 1.1) > 2. - change the BioMoby Perl code to allow more than one simple in the > MobyData AND > - change almost all services that output collections to the way we > discussed yesterday (only output collections if the objects build an > entity. ) > > Either of the two changes solves the problems but I believe both are due > to changes and should take place. > > Best, > Rebecca > > > > > > Tom Oinn wrote: > >> That's kind of true, meaning actually not. There are three cases >> involving collections (Taverna 1.1 behaviour) : >> >> 1) Consumer declares it consumes singles, Producer emits a collection. >> In this context Taverna iteratively calls the Consumer with each item >> from the collection. This is probably what you'd expect to happen, the >> result is that the Consumer effectively emits a collection of whatever >> it would emit normally. > > >> 2) Consumer declares it consumes a collection, Producer emits a >> collection. In this case Taverna will indeed split the output >> collection (because we always do) but it will be magically reassembled >> before being given to the Consumer. >> >> 3) Consumer declares it consumes a collection, Producer emits a single >> item. Taverna wraps the single item in a single element collection and >> gives it to the Consumer. >> >> The intent is that as with the other plugin types Taverna guarantees >> that the service sees the inputs it has asked for. Our experience with >> other service types suggests that this is an extremely powerful >> mechanism as it allows interoperability between services that would >> otherwise mismatch - it's worth noting that our users expect these >> services to match, while a CS perspective regards ProteinSequence and >> ProteinSequence[] as completely different types most of our biologists >> don't! Taverna behaves the way _they_ expect it to, remember who your >> user are. >> >> Taverna data types are pretty much trivial, they're opaque data blobs >> with the exception of collection structure which is exposed. We only >> expose the collection structure to ensure the above properties, other >> than that the framework is data agnostic (as it should be). >> >> Tom >> > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From markw at illuminae.com Fri Jul 8 13:22:20 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 10:22:20 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE2229.1030504@gsf.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <42CE2229.1030504@gsf.de> Message-ID: <1120843340.15014.40.camel@mobycentral.mrl.ubc.ca> I see no inconsistency between what you quote below and what I said in my mail from yesterday... ?? M On Fri, 2005-07-08 at 08:50 +0200, Rebecca Ernst wrote: > Hi Mark! > > I will have to spank you a lot for either the last email or the one you > sent me in February where I asked you how to use collections (see mail > below): > > > Well... the answer depends on what the generic case of your output will > look like. the rule is that you have to register every output object > that your service produces; i.e. every immediate child tag of a mobyData > block. Therefore, if you are going to consistently output exactly 3 > Genbank/gi's, then you would register your service as outputting 3 X > Object(NCBI_gi). If you are outputting an indeterminate number of > objects, then you register it as outputting 1 X Collection. > > Collections map nicely onto the rdf:Bag structure, if that helps you > interpret them. > > M > > > > On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: > > >> Hi Mark! > >> > >> We have problems interpreting the BioMoby API for the output objects. > >> > >> The thing is that we have a service that gets ONE input object and > >> executes a freetext search over several databases. This service will > >> eventually return more that one result. > >> We don't want to put the result into a collection object because the > >> single objects don't build any entity. > >> We also can't give back more that one mobyData block because we only > >> have one query ID and therefore only one queryID to give back. > >> > >> The only solution we can think of is an object like this: > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> would this be a legal object or do we have to use collection even if the > >> objects don't build an entity ? > >> > >> > >> Cheers, > >> Rebecca > > > > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical > Genetics, UBC, Canada > > > ----------------------------------------------------------------------------------------------------------- > > > > > > > Mark Wilkinson wrote: > > >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > > > > > >>I myself am confused about the use of collections. Originally I had in > >>my mind that Collections were a construct to allow objects that > >>inherently belong together to be "bagged". > >> > >> > > > >You are 100% correct here. > > > > > > > > > >>The confusion starts with the output of services. My understanding was > >>that ONLY a service that is guaranteed to output exactly one object for > >>each query (e.g. an averaging service that outputs the average of a > >>list of inputs) is registered as outputting a Simple, all others have > >>to output collections > >> > >> > > > >No. > > > > > > > >> (as there must be exactly one mobyData matching > >>the queryID of the input in the response, > >> > >> > > > >Yes > > > > > > > >> and a mobyData may contain > >>multiple Simple elements only if wrapped by a Collection). > >> > >> > > > >No. A mobyData element can wrap any number of outputs, and/or > >combinations of simples, collections, and secondaries. > > > > > > > > > > > >>In practice, the Collection tag has been used to indicate when more > >>than one Simple occurs, with no "semantic" meaning. This imho is not > >>necessary; when more than one Simple occurs, why not put more than one > >>Simple? It's easy enough for everyone to figure that out. Then, > >>Collection could be used to actually transfer meaning ;-) > >> > >> > > > >I think a lot of people are using Collections incorrectly, yes. > > > > > > > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back > >>many Simple articles in response to a query very intuitively indicates > >>to continue on with each one individually, whereas getting back a > >>Collection indicates to put the whole thing as input into the next > >>service, which is what they implemented. Makes perfect sense, as there > >>can and will be services that consume Collections. > >> > >> > > > >I don't think that Taverna handled collections correctly in the past, > >and it would be a shame if that identical functionality was added back > >in :-) The functionality does need to go back into Taverna, but > >correctly this time. > > > > > > > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue > >>has led to quite some discussions between Rebecca, Dirk and myself, and > >>we are all not happy with the way they are currently handled. > >> > >> > > > >I don't know if you are happier with them now that I have pointed out > >where you are misinterpreting them...?? > > > >I agree, however, that their usage by many service providers is not > >correct... but there's no way for MOBY Central to know that it isn't > >correct, so we can't throw an error on registration of these "wonky" > >services... > > > >M > > > > > > > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 13:25:33 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 10:25:33 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <1120843533.15014.43.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 09:07 +0100, Tom Oinn wrote: > 1) Consumer declares it consumes singles, Producer emits a collection. > 2) Consumer declares it consumes a collection, Producer emits a > 3) Consumer declares it consumes a collection, Producer emits a single So the only case that is missing is where the Consumer declares it consumes a collection, the producer emits a single, but you want to "bulk" those simples (or even simples arising from disparate upstream service outputs) into a single input collection. I believe you said you had a widget that would do that already, but I can't remember which one it was...?? M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 13:25:33 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 10:25:33 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <1120843533.15014.43.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 09:07 +0100, Tom Oinn wrote: > 1) Consumer declares it consumes singles, Producer emits a collection. > 2) Consumer declares it consumes a collection, Producer emits a > 3) Consumer declares it consumes a collection, Producer emits a single So the only case that is missing is where the Consumer declares it consumes a collection, the producer emits a single, but you want to "bulk" those simples (or even simples arising from disparate upstream service outputs) into a single input collection. I believe you said you had a widget that would do that already, but I can't remember which one it was...?? M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 13:37:09 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 10:37:09 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <200507081049.58799.d.haase@gsf.de> References: <42CD31FF.6040803@gsf.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <200507081049.58799.d.haase@gsf.de> Message-ID: <1120844229.15014.53.camel@mobycentral.mrl.ubc.ca> I agree that the CommonSubs code does not facilitate every possibility allowed by the MOBY API, but the API should be considered canonical :-) I think that CommonSubs needs to be **completely** re-written, and in fact, this was one of the issues that came up at the last meeting - that we need to have some common implementation-independent API that we can all code to such that all possible message types can be intuitively created and unpacked in any language. Though I don't object to you adding that subroutine to the CVS (I haven't checked it for sanity, only for intent) I think it would be useful for us to operate on a higher-level at some point soon and decide what functions we *need*, and then decide how to build them, rather than adding functions on an ad hoc basis as we have been so far. For example, I might be tempted to have a function that built service responses in the following way: $OUT = CommonSubs::ServiceOutput->new(some header information - service provision info goes here); $OUT->addOutput(invocation_id => $id, simple => $simple); $OUT->addOutput(invocation_id => $id, collection => \@simples); $OUT->addOutput(invocation_id => $id2, simple => $simple2); etc. etc. That would cover all possibilities and would be more intuitive than the existing mechanism... ??? He who codes, wins! M On Fri, 2005-07-08 at 10:49 +0200, Dirk Haase wrote: > On Friday 08 July 2005 08:45, Heiko Schoof wrote: > > Well, if I understand you correctly, Mark, than I was right all along > > in that they way I see collections is the way they should be seen. Then > > the problem is that that is not clear enough to everybody else, and > > that possibly the examples in the API are misleading, because it's been > > used in such a different way. It's definitely something that needs to > > be fixed before release 1.0. > > Not only the examples are misleading, actually at least the PERL API is > lacking a suitable method to construct an output consisting of several > 'Simple's. One can not simply concat the output of multibple 'simpleResponse' > calls, as this would create several mobyData blocks. > > In order to fill this gap, I created a new method 'multiSimpleResponse' (see > below) within CommonSubs.pm which is more or less identical to > 'collectionResponse' except that it obviously leaves out the moby:Collection > tag. The provided articleName is put into each 'Simple' tag which should be > appropriate. > > Any objections? > > Regards, > dirk > > =head2 multiSimpleResponse > > name : multiSimpleResponse > function : wraps a set of simple articles in the appropriate mobyData > structure > usage : return responseHeader . &multiSimpleResponse(\@objects, > 'MyArticleName', $queryID) . responseFooter; > args : (in order) > \@objects - (optional) a listref of MOBY Objects as raw XML > $article - (optional) an articeName for this article > $queryID - (optional, but strongly recommended) the mobyData ID > to which you are responding > notes : as required by the API you must return a response for every input. > If one of the inputs was invalid, you return a valid (empty) MOBY > response by calling &multiSimpleResponse(undef, undef, $queryID). > > =cut > > > sub multiSimpleResponse { > my ( $data, $articleName, $qID ) = @_; # articleName optional > my $content = ""; > $data ||= []; > $qID ||= ''; > $articleName ||=""; > unless ( ( ref( $data ) =~ /array/i ) && $data->[0] ) > { # we're expecting an arrayref as input data,and it must not be > empty > return ""; > } > foreach ( @{$data} ) { > if ( $_ ) { > $content .= " > $_ > "; > } else { > $content .= " > > "; > } > } > return " > > $content > > "; > } > > > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From gordonp at ucalgary.ca Fri Jul 8 14:26:25 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Fri, 08 Jul 2005 12:26:25 -0600 Subject: [MOBY-dev] API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42CE5FA5.50901@cnb.uam.es> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> Message-ID: <42CEC551.4090108@ucalgary.ca> Hi everyone, At long last after our discussions at the last MOBY meeting in Vancouver, I have written a little document explaining the reason for and consequences of not inheriting from "primitive" objects such as String and Integer. I think that this is a distruptive change for only a few services, but is of great benefit. Please see http://coe02.ucalgary.ca/gordonp/moby_primitives.html Mark has suggested that this not be a formal change to the API, but that this new way of doing things be encouraged in all of the documentation, and the old method explicitly deprecated but not disabled. Hopefully most of you agree, and we can just make temporary exceptions in our code for the current non-compliant classes. Or even better, if most people agree, we can force the affected services to change over. :-) P.S. Yan, your object browser came in quite handy while I was writing this, nice work! One suggestion though, arranging the objects alphabetically in the tree (when within the same branch and level) would help ease navigation. P.P.S. Our Web server will be down briefly this afternoon, so you may get a timeout/cannot connect on the link. From d.haase at gsf.de Fri Jul 8 14:45:39 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Fri, 8 Jul 2005 20:45:39 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Hi Mark, I would not call it inconsistency, they totally contradict ;-) Consider a service which returns an unpredictable number of database entries. The result set does not form any meaningful construct, each member is just an alternative answer to the query posed. Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. If I summed up both replies correctly, the contradiction is obvious, isn't it? dirk -----Original Message----- From: moby-dev-bounces at portal.open-bio.org on behalf of Mark Wilkinson Sent: Fri 7/8/2005 7:22 PM To: Ernst, Rebecca Cc: Martin Senger; mobydev Subject: Re: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 I see no inconsistency between what you quote below and what I said in my mail from yesterday... ?? M On Fri, 2005-07-08 at 08:50 +0200, Rebecca Ernst wrote: > Hi Mark! > > I will have to spank you a lot for either the last email or the one you > sent me in February where I asked you how to use collections (see mail > below): > > > Well... the answer depends on what the generic case of your output will > look like. the rule is that you have to register every output object > that your service produces; i.e. every immediate child tag of a mobyData > block. Therefore, if you are going to consistently output exactly 3 > Genbank/gi's, then you would register your service as outputting 3 X > Object(NCBI_gi). If you are outputting an indeterminate number of > objects, then you register it as outputting 1 X Collection. > > Collections map nicely onto the rdf:Bag structure, if that helps you > interpret them. > > M > > > > On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: > > >> Hi Mark! > >> > >> We have problems interpreting the BioMoby API for the output objects. > >> > >> The thing is that we have a service that gets ONE input object and > >> executes a freetext search over several databases. This service will > >> eventually return more that one result. > >> We don't want to put the result into a collection object because the > >> single objects don't build any entity. > >> We also can't give back more that one mobyData block because we only > >> have one query ID and therefore only one queryID to give back. > >> > >> The only solution we can think of is an object like this: > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> would this be a legal object or do we have to use collection even if the > >> objects don't build an entity ? > >> > >> > >> Cheers, > >> Rebecca > > > > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical > Genetics, UBC, Canada > > > ----------------------------------------------------------------------------------------------------------- > > > > > > > Mark Wilkinson wrote: > > >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > > > > > >>I myself am confused about the use of collections. Originally I had in > >>my mind that Collections were a construct to allow objects that > >>inherently belong together to be "bagged". > >> > >> > > > >You are 100% correct here. > > > > > > > > > >>The confusion starts with the output of services. My understanding was > >>that ONLY a service that is guaranteed to output exactly one object for > >>each query (e.g. an averaging service that outputs the average of a > >>list of inputs) is registered as outputting a Simple, all others have > >>to output collections > >> > >> > > > >No. > > > > > > > >> (as there must be exactly one mobyData matching > >>the queryID of the input in the response, > >> > >> > > > >Yes > > > > > > > >> and a mobyData may contain > >>multiple Simple elements only if wrapped by a Collection). > >> > >> > > > >No. A mobyData element can wrap any number of outputs, and/or > >combinations of simples, collections, and secondaries. > > > > > > > > > > > >>In practice, the Collection tag has been used to indicate when more > >>than one Simple occurs, with no "semantic" meaning. This imho is not > >>necessary; when more than one Simple occurs, why not put more than one > >>Simple? It's easy enough for everyone to figure that out. Then, > >>Collection could be used to actually transfer meaning ;-) > >> > >> > > > >I think a lot of people are using Collections incorrectly, yes. > > > > > > > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back > >>many Simple articles in response to a query very intuitively indicates > >>to continue on with each one individually, whereas getting back a > >>Collection indicates to put the whole thing as input into the next > >>service, which is what they implemented. Makes perfect sense, as there > >>can and will be services that consume Collections. > >> > >> > > > >I don't think that Taverna handled collections correctly in the past, > >and it would be a shame if that identical functionality was added back > >in :-) The functionality does need to go back into Taverna, but > >correctly this time. > > > > > > > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue > >>has led to quite some discussions between Rebecca, Dirk and myself, and > >>we are all not happy with the way they are currently handled. > >> > >> > > > >I don't know if you are happier with them now that I have pointed out > >where you are misinterpreting them...?? > > > >I agree, however, that their usage by many service providers is not > >correct... but there's no way for MOBY Central to know that it isn't > >correct, so we can't throw an error on registration of these "wonky" > >services... > > > >M > > > > > > > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From d.haase at gsf.de Fri Jul 8 14:45:39 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Fri, 8 Jul 2005 20:45:39 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Hi Mark, I would not call it inconsistency, they totally contradict ;-) Consider a service which returns an unpredictable number of database entries. The result set does not form any meaningful construct, each member is just an alternative answer to the query posed. Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. If I summed up both replies correctly, the contradiction is obvious, isn't it? dirk -----Original Message----- From: moby-dev-bounces at portal.open-bio.org on behalf of Mark Wilkinson Sent: Fri 7/8/2005 7:22 PM To: Ernst, Rebecca Cc: Martin Senger; mobydev Subject: Re: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 I see no inconsistency between what you quote below and what I said in my mail from yesterday... ?? M On Fri, 2005-07-08 at 08:50 +0200, Rebecca Ernst wrote: > Hi Mark! > > I will have to spank you a lot for either the last email or the one you > sent me in February where I asked you how to use collections (see mail > below): > > > Well... the answer depends on what the generic case of your output will > look like. the rule is that you have to register every output object > that your service produces; i.e. every immediate child tag of a mobyData > block. Therefore, if you are going to consistently output exactly 3 > Genbank/gi's, then you would register your service as outputting 3 X > Object(NCBI_gi). If you are outputting an indeterminate number of > objects, then you register it as outputting 1 X Collection. > > Collections map nicely onto the rdf:Bag structure, if that helps you > interpret them. > > M > > > > On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: > > >> Hi Mark! > >> > >> We have problems interpreting the BioMoby API for the output objects. > >> > >> The thing is that we have a service that gets ONE input object and > >> executes a freetext search over several databases. This service will > >> eventually return more that one result. > >> We don't want to put the result into a collection object because the > >> single objects don't build any entity. > >> We also can't give back more that one mobyData block because we only > >> have one query ID and therefore only one queryID to give back. > >> > >> The only solution we can think of is an object like this: > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> would this be a legal object or do we have to use collection even if the > >> objects don't build an entity ? > >> > >> > >> Cheers, > >> Rebecca > > > > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical > Genetics, UBC, Canada > > > ----------------------------------------------------------------------------------------------------------- > > > > > > > Mark Wilkinson wrote: > > >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > > > > > >>I myself am confused about the use of collections. Originally I had in > >>my mind that Collections were a construct to allow objects that > >>inherently belong together to be "bagged". > >> > >> > > > >You are 100% correct here. > > > > > > > > > >>The confusion starts with the output of services. My understanding was > >>that ONLY a service that is guaranteed to output exactly one object for > >>each query (e.g. an averaging service that outputs the average of a > >>list of inputs) is registered as outputting a Simple, all others have > >>to output collections > >> > >> > > > >No. > > > > > > > >> (as there must be exactly one mobyData matching > >>the queryID of the input in the response, > >> > >> > > > >Yes > > > > > > > >> and a mobyData may contain > >>multiple Simple elements only if wrapped by a Collection). > >> > >> > > > >No. A mobyData element can wrap any number of outputs, and/or > >combinations of simples, collections, and secondaries. > > > > > > > > > > > >>In practice, the Collection tag has been used to indicate when more > >>than one Simple occurs, with no "semantic" meaning. This imho is not > >>necessary; when more than one Simple occurs, why not put more than one > >>Simple? It's easy enough for everyone to figure that out. Then, > >>Collection could be used to actually transfer meaning ;-) > >> > >> > > > >I think a lot of people are using Collections incorrectly, yes. > > > > > > > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back > >>many Simple articles in response to a query very intuitively indicates > >>to continue on with each one individually, whereas getting back a > >>Collection indicates to put the whole thing as input into the next > >>service, which is what they implemented. Makes perfect sense, as there > >>can and will be services that consume Collections. > >> > >> > > > >I don't think that Taverna handled collections correctly in the past, > >and it would be a shame if that identical functionality was added back > >in :-) The functionality does need to go back into Taverna, but > >correctly this time. > > > > > > > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue > >>has led to quite some discussions between Rebecca, Dirk and myself, and > >>we are all not happy with the way they are currently handled. > >> > >> > > > >I don't know if you are happier with them now that I have pointed out > >where you are misinterpreting them...?? > > > >I agree, however, that their usage by many service providers is not > >correct... but there's no way for MOBY Central to know that it isn't > >correct, so we can't throw an error on registration of these "wonky" > >services... > > > >M > > > > > > > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 6191 bytes Desc: not available Url : http://www.biomoby.org/pipermail/moby-dev/attachments/20050708/e13cc75c/attachment-0002.bin From markw at illuminae.com Fri Jul 8 14:58:30 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 11:58:30 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Message-ID: <1120849110.15014.101.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 20:45 +0200, Haase, Dirk wrote: > Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. What I said in response to Heiko (or at least, what I intended to say) was that there was not a limitation of the mobyData block to having just one child element (regardless of its type - simple, collection, or secondary). So the following is vaid: I didn't say that you SHOULD return all of your Simples individually, only that you COULD return individual Simples in a single output... if that was meaningful in the context of your service... individual outputs must, however, be individually registered in the registry (output => [A,B,Q]) > But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. That's right.... and obvious, since you cannot create a service signature any other way - you can't say output => [A....infinite] > If I summed up both replies correctly, the contradiction is obvious, isn't it? I don't see it. M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 14:58:30 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 11:58:30 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Message-ID: <1120849110.15014.101.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 20:45 +0200, Haase, Dirk wrote: > Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. What I said in response to Heiko (or at least, what I intended to say) was that there was not a limitation of the mobyData block to having just one child element (regardless of its type - simple, collection, or secondary). So the following is vaid: I didn't say that you SHOULD return all of your Simples individually, only that you COULD return individual Simples in a single output... if that was meaningful in the context of your service... individual outputs must, however, be individually registered in the registry (output => [A,B,Q]) > But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. That's right.... and obvious, since you cannot create a service signature any other way - you can't say output => [A....infinite] > If I summed up both replies correctly, the contradiction is obvious, isn't it? I don't see it. M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From d.haase at gsf.de Fri Jul 8 15:43:41 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Fri, 8 Jul 2005 21:43:41 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> OK, things become clearer I think. You didn't really get what Heiko was suggesting about the usage of collections... Heiko, Rebecca and me think that a collection is ueseful _only_ if the collection itself is a meaningful entity (example: several sequences make up a cluster). In our view it does not make sense to pack otherwise unrelated things into a collection just because they are all results for a certain query. I admit that our understanding is not compatible with the current way for registering services. The easy(?) solution would be to abandon any cardinality confirmations from the registry. That is if a service is registered with an output A it may output any number of A-objects. Another way to tackle this would be to introduce a sort of 'has/has-a' for service outputs, analogous to the object definitions. dirk -----Original Message----- From: moby-dev-bounces at portal.open-bio.org on behalf of Mark Wilkinson Sent: Fri 7/8/2005 8:58 PM To: Core developer announcements Cc: Ernst, Rebecca; Martin Senger; mobydev Subject: RE: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 On Fri, 2005-07-08 at 20:45 +0200, Haase, Dirk wrote: > Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. What I said in response to Heiko (or at least, what I intended to say) was that there was not a limitation of the mobyData block to having just one child element (regardless of its type - simple, collection, or secondary). So the following is vaid: I didn't say that you SHOULD return all of your Simples individually, only that you COULD return individual Simples in a single output... if that was meaningful in the context of your service... individual outputs must, however, be individually registered in the registry (output => [A,B,Q]) > But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. That's right.... and obvious, since you cannot create a service signature any other way - you can't say output => [A....infinite] > If I summed up both replies correctly, the contradiction is obvious, isn't it? I don't see it. M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 4339 bytes Desc: not available Url : http://www.biomoby.org/pipermail/moby-dev/attachments/20050708/07d07189/attachment-0002.bin From markw at illuminae.com Fri Jul 8 16:07:25 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 13:07:25 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> Message-ID: <1120853245.15014.131.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 21:43 +0200, Haase, Dirk wrote: > Heiko, Rebecca and me think that a collection is ueseful _only_ if the collection > itself is a meaningful entity (example: several sequences make up a cluster). > In our view it does not make sense to pack otherwise unrelated things into a > collection just because they are all results for a certain query. I agree 100%... so again, I still don't see the difference between what you are saying and what I am saying...?? M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From jmfernandez at cnb.uam.es Fri Jul 8 17:55:44 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Fri, 08 Jul 2005 23:55:44 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> Message-ID: <42CEF660.8010602@cnb.uam.es> Hi everybody, I had almost forgotten a MOBY service (like a PL/SQL or ADA procedure) can have more than one output, each one belonging to its own type. I forgot it for one of the services I designed for INB. > > I admit that our understanding is not compatible with the current way > for registering services. The easy(?) solution would be to abandon > any cardinality confirmations from the registry. That is if a service > is registered with an output A it may output any number of A-objects. > I have in mind two or three bioinformatics time-expensive algorithms which generate more than one output (with different types) each time they are run. Because they are related, the common developer wraps them into an object, whose type inherits from Object, and most of the time it is an artificial creation. Those coders must also develop micro-services which pull the needed piece of information from the wrap, or if you are a Taverna user something like a beanshell. So, I agree with Mark: services with more than one output have their place. But remember: we need clients which are able to deal with this feature. Until now we had no one, but now Taverna could be the first one... Best regards, Jos? Mar?a > > Another way to tackle this would be to introduce a sort of > 'has/has-a' for service outputs, analogous to the object definitions. > > > dirk > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From markw at illuminae.com Fri Jul 8 18:00:33 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 15:00:33 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <42CEF660.8010602@cnb.uam.es> References: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> <42CEF660.8010602@cnb.uam.es> Message-ID: <1120860033.3721.7.camel@bioinfo.icapture.ubc.ca> Yeah... it's taking a long time for the tooling to catch up with the API. Just a lack of resources :-( M On Fri, 2005-07-08 at 23:55 +0200, Jos? Mar?a Fern?ndez Gonz?lez wrote: > So, I agree with Mark: > services with more than one output have their place. But remember: we > need clients which are able to deal with this feature. Until now we had > no one, but now Taverna could be the first one... -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From d.haase at gsf.de Sat Jul 9 05:02:01 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Sat, 9 Jul 2005 11:02:01 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B28B@sw-rz010.gsf.de> Hi Jose Maria, Mark and others, I was not talking about sophisticated services with unpredictable type(s) of output, only about plain stupid lookup services where the output type is fix, but the number of result objects unknown. Like 'getAGILocusCodesByKeyword' for example. I'll try to explain it in other words... Why not making it the default behaviour that services can output 1 or many objects of the type it is registered for and save the 'collection' construct for situations where several output objects (of same of differing types) constitute a semantically meaningful complex. Does anybody understand what I'm trying to say?? dirk -----Original Message----- From: moby-dev-bounces at portal.open-bio.org on behalf of Jos? Mar?a Fern?ndez Gonz?lez Sent: Fri 7/8/2005 11:55 PM To: Core developer announcements Cc: markw at illuminae.com Subject: Re: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Hi everybody, I had almost forgotten a MOBY service (like a PL/SQL or ADA procedure) can have more than one output, each one belonging to its own type. I forgot it for one of the services I designed for INB. > > I admit that our understanding is not compatible with the current way > for registering services. The easy(?) solution would be to abandon > any cardinality confirmations from the registry. That is if a service > is registered with an output A it may output any number of A-objects. > I have in mind two or three bioinformatics time-expensive algorithms which generate more than one output (with different types) each time they are run. Because they are related, the common developer wraps them into an object, whose type inherits from Object, and most of the time it is an artificial creation. Those coders must also develop micro-services which pull the needed piece of information from the wrap, or if you are a Taverna user something like a beanshell. So, I agree with Mark: services with more than one output have their place. But remember: we need clients which are able to deal with this feature. Until now we had no one, but now Taverna could be the first one... Best regards, Jos? Mar?a > > Another way to tackle this would be to introduce a sort of > 'has/has-a' for service outputs, analogous to the object definitions. > > > dirk > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 4479 bytes Desc: not available Url : http://www.biomoby.org/pipermail/moby-dev/attachments/20050709/d8bdfc80/attachment-0002.bin From chris at bioteam.net Sat Jul 9 13:40:52 2005 From: chris at bioteam.net (Chris Dagdigian) Date: Sat, 9 Jul 2005 13:40:52 -0400 Subject: [MOBY-dev] DNS changes for biomoby.org beginning today - should have no noticeable effect Message-ID: Hi folks, All of open-bio DNS is outsourced to a third party except for biomoby.org for which we manage the DNS zone ourselves due to moby requirements for specialized SRV zone entries that our other provider can't handle. The existing primary and secondary nameservers for biomoby.org are: 7200 IN NS portal.open-bio.org. 7200 IN NS pub.open-bio.org. These also happen to be the same servers we are currently trying to "retire" in favor of newer servers running in a different datacenter colocation cage. Last night I copied the biomoby.org zone file to our two new servers and activated service for them. The new machines are: (1) newportal.open-bio.org (2) dev.open-bio.org This morning I edited the master nameserver list that NetworkSolutions holds for the domain to add those new servers at the top of the list. Biomoby now has 4 dns servers; listed in the following order: 7200 IN NS newportal.open-bio.org. 7200 IN NS dev.open-bio.org. 7200 IN NS portal.open-bio.org. 7200 IN NS pub.open-bio.org. This means that over the next 24-72 hours as various DNS caches expire, biomoby.org will start to be serviced from the new pair of nameservers with the "old" pair acting as backup. If this works smoothly I'll leave it alone for 1-week. After which I plan on editing the NetworkSolution records to remove the "old" servers from the list. That should complete the full transition from the old primary/secondary pair to the new pair. Because of the overlap (both new and old server pairs are serving up the same info for biomoby.org) I don't expect anyone or any MOBY service user to experience any problems. I just wanted to mention the DNS change in case a problem does occur. For the DNS savvy on this list, feel free to query the biomoby.org zone on the new machines to confirm that the info is being served correctly. Your zone currently looks like this: > $ORIGIN org. > biomoby 7200 IN SOA portal.open-bio.org. root-l.open- > bio.org. ( > 0705200503 114400 7200 950400 7200 ) > 7200 IN NS newportal.open-bio.org. > 7200 IN NS dev.open-bio.org. > 7200 IN NS portal.open-bio.org. > 7200 IN NS pub.open-bio.org. > 7200 IN A 65.246.187.176 > 7200 IN MX 10 biomoby.org. > $ORIGIN biomoby.org. > portal 7200 IN A 65.246.187.176 > www 7200 IN CNAME biomoby.org. > cvs 7200 IN A 65.246.187.182 > 7200 IN MX 10 biomoby.org. > _lsid._tcp IN SRV 1 0 80 mobycentral.icapture.ubc.ca. Regards, Chris From schoof at mpiz-koeln.mpg.de Wed Jul 13 08:27:42 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Wed, 13 Jul 2005 14:27:42 +0200 Subject: [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42CEC551.4090108@ucalgary.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> Message-ID: <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> Hi Paul, thanks for the nice writeup. I agree. I just want to stress that I find one of the consequences you mention a very important argument for doing this: Services that act on Strings will no longer automatically be discovered starting from e.g. NCBI_BLAST_OUTPUT or DNA_SEQUENCE, but only when using object decomposition. Best, Heiko On 8. Jul 2005, at 20:26 Uhr, Paul Gordon wrote: Hi everyone, At long last after our discussions at the last MOBY meeting in Vancouver, I have written a little document explaining the reason for and consequences of not inheriting from "primitive" objects such as String and Integer. I think that this is a distruptive change for only a few services, but is of great benefit. Please see http://coe02.ucalgary.ca/gordonp/moby_primitives.html Mark has suggested that this not be a formal change to the API, but that this new way of doing things be encouraged in all of the documentation, and the old method explicitly deprecated but not disabled. Hopefully most of you agree, and we can just make temporary exceptions in our code for the current non-compliant classes. Or even better, if most people agree, we can force the affected services to change over. :-) P.S. Yan, your object browser came in quite handy while I was writing this, nice work! One suggestion though, arranging the objects alphabetically in the tree (when within the same branch and level) would help ease navigation. P.P.S. Our Web server will be down briefly this afternoon, so you may get a timeout/cannot connect on the link. _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From markw at illuminae.com Wed Jul 13 11:21:13 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 13 Jul 2005 08:21:13 -0700 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> Message-ID: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> The "virtual decomposition" of objects that MOBY does during service discovery can be switched on/off. Switching it off will prevent this behaviour also if it is an immediate concern. M On Wed, 2005-07-13 at 14:27 +0200, Heiko Schoof wrote: > Hi Paul, > thanks for the nice writeup. I agree. > > I just want to stress that I find one of the consequences you mention a > very important argument for doing this: Services that act on Strings > will no longer automatically be discovered starting from e.g. > NCBI_BLAST_OUTPUT or DNA_SEQUENCE, but only when using object > decomposition. > > Best, Heiko > > On 8. Jul 2005, at 20:26 Uhr, Paul Gordon wrote: > > Hi everyone, > > At long last after our discussions at the last MOBY meeting in > Vancouver, I have written a little document explaining the reason for > and consequences of not inheriting from "primitive" objects such as > String and Integer. I think that this is a distruptive change for only > a few services, but is of great benefit. Please see > > http://coe02.ucalgary.ca/gordonp/moby_primitives.html > > Mark has suggested that this not be a formal change to the API, but > that this new way of doing things be encouraged in all of the > documentation, and the old method explicitly deprecated but not > disabled. Hopefully most of you agree, and we can just make temporary > exceptions in our code for the current non-compliant classes. Or even > better, if most people agree, we can force the affected services to > change over. :-) > > P.S. Yan, your object browser came in quite handy while I was writing > this, nice work! One suggestion though, arranging the objects > alphabetically in the tree (when within the same branch and level) > would help ease navigation. > > P.P.S. Our Web server will be down briefly this afternoon, so you may > get a timeout/cannot connect on the link. > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From markw at illuminae.com Wed Jul 13 11:21:13 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 13 Jul 2005 08:21:13 -0700 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> Message-ID: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> The "virtual decomposition" of objects that MOBY does during service discovery can be switched on/off. Switching it off will prevent this behaviour also if it is an immediate concern. M On Wed, 2005-07-13 at 14:27 +0200, Heiko Schoof wrote: > Hi Paul, > thanks for the nice writeup. I agree. > > I just want to stress that I find one of the consequences you mention a > very important argument for doing this: Services that act on Strings > will no longer automatically be discovered starting from e.g. > NCBI_BLAST_OUTPUT or DNA_SEQUENCE, but only when using object > decomposition. > > Best, Heiko > > On 8. Jul 2005, at 20:26 Uhr, Paul Gordon wrote: > > Hi everyone, > > At long last after our discussions at the last MOBY meeting in > Vancouver, I have written a little document explaining the reason for > and consequences of not inheriting from "primitive" objects such as > String and Integer. I think that this is a distruptive change for only > a few services, but is of great benefit. Please see > > http://coe02.ucalgary.ca/gordonp/moby_primitives.html > > Mark has suggested that this not be a formal change to the API, but > that this new way of doing things be encouraged in all of the > documentation, and the old method explicitly deprecated but not > disabled. Hopefully most of you agree, and we can just make temporary > exceptions in our code for the current non-compliant classes. Or even > better, if most people agree, we can force the affected services to > change over. :-) > > P.S. Yan, your object browser came in quite handy while I was writing > this, nice work! One suggestion though, arranging the objects > alphabetically in the tree (when within the same branch and level) > would help ease navigation. > > P.P.S. Our Web server will be down briefly this afternoon, so you may > get a timeout/cannot connect on the link. > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From gordonp at ucalgary.ca Wed Jul 13 11:50:33 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed, 13 Jul 2005 09:50:33 -0600 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53849.7010700@ucalgary.ca> Mark, We are talking about decomposition. What you are talking about is inheritence. You can switch off finding services that match parent objects, of course. But an annotated base64 encoded GIF under the new scheme would NEVER match something that takes String as input, no matter how you set the service finding flag. The CLIENT would need to decompose (probably at the user's request) the GIF object into its constituent String members, such as the "contents" or the "description", and search MOBY central using JUST that member. That's what we mean by decomposition. >The "virtual decomposition" of objects that MOBY does during service >discovery can be switched on/off. Switching it off will prevent this >behaviour also if it is an immediate concern. > > > > From gordonp at ucalgary.ca Wed Jul 13 11:50:33 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed, 13 Jul 2005 09:50:33 -0600 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53849.7010700@ucalgary.ca> Mark, We are talking about decomposition. What you are talking about is inheritence. You can switch off finding services that match parent objects, of course. But an annotated base64 encoded GIF under the new scheme would NEVER match something that takes String as input, no matter how you set the service finding flag. The CLIENT would need to decompose (probably at the user's request) the GIF object into its constituent String members, such as the "contents" or the "description", and search MOBY central using JUST that member. That's what we mean by decomposition. >The "virtual decomposition" of objects that MOBY does during service >discovery can be switched on/off. Switching it off will prevent this >behaviour also if it is an immediate concern. > > > > From Pieter.Neerincx at wur.nl Wed Jul 13 11:53:41 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Wed, 13 Jul 2005 17:53:41 +0200 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: Hi all, Is there anyone out there that was able to make BioMOBY (realated) Java tools like Yan's Moby Object Browser or Taverna work with MOBY Centrals and/or services over HTTPS? I'm having trouble with the server's certificate. I used keytool to import the server's certificate client-side, but the Java tools keep on complaining: "unknown certificate"... Cheers, Pieter From tmo at ebi.ac.uk Wed Jul 13 12:05:55 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Wed, 13 Jul 2005 17:05:55 +0100 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53BE3.7000909@ebi.ac.uk> Hi all, We've had Taverna working for HTTPS web services so it should be fine for the MOBY operations. You need to check that the certificate has a valid signature chain and that the name of the entity in the certificate matches the hostname - we had issues with a certificate from a server in china that the java security framework rejected because the (I think) CN part of the DN in the cert wasn't the same as the hostname of the endpoint. Tom Pieter Neerincx wrote: > Hi all, > > Is there anyone out there that was able to make BioMOBY (realated) Java > tools like Yan's Moby Object Browser or Taverna work with MOBY Centrals > and/or services over HTTPS? I'm having trouble with the server's > certificate. I used keytool to import the server's certificate > client-side, but the Java tools keep on complaining: "unknown > certificate"... > > Cheers, > > Pieter _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev From tmo at ebi.ac.uk Wed Jul 13 12:05:55 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Wed, 13 Jul 2005 17:05:55 +0100 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53BE3.7000909@ebi.ac.uk> Hi all, We've had Taverna working for HTTPS web services so it should be fine for the MOBY operations. You need to check that the certificate has a valid signature chain and that the name of the entity in the certificate matches the hostname - we had issues with a certificate from a server in china that the java security framework rejected because the (I think) CN part of the DN in the cert wasn't the same as the hostname of the endpoint. Tom Pieter Neerincx wrote: > Hi all, > > Is there anyone out there that was able to make BioMOBY (realated) Java > tools like Yan's Moby Object Browser or Taverna work with MOBY Centrals > and/or services over HTTPS? I'm having trouble with the server's > certificate. I used keytool to import the server's certificate > client-side, but the Java tools keep on complaining: "unknown > certificate"... > > Cheers, > > Pieter _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev From markw at illuminae.com Wed Jul 13 12:16:27 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 13 Jul 2005 09:16:27 -0700 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42D53849.7010700@ucalgary.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> <42D53849.7010700@ucalgary.ca> Message-ID: <1121271387.6327.45.camel@bioinfo.icapture.ubc.ca> On Wed, 2005-07-13 at 09:50 -0600, Paul Gordon wrote: > We are talking about decomposition. What you are talking about is > inheritence. Yes, but I'm just saying that the *current* behaviour that Heiko was objecting to is due to inheritence, and the behaviour can be switched off if it is *immediately* troublesome. Of course, it will also disappear if we start to use a composition model rather than an inheritence model to put e.g. strings into objects, but that doesn't help anyone in the short-term. M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From markw at illuminae.com Wed Jul 13 12:16:27 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 13 Jul 2005 09:16:27 -0700 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42D53849.7010700@ucalgary.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> <42D53849.7010700@ucalgary.ca> Message-ID: <1121271387.6327.45.camel@bioinfo.icapture.ubc.ca> On Wed, 2005-07-13 at 09:50 -0600, Paul Gordon wrote: > We are talking about decomposition. What you are talking about is > inheritence. Yes, but I'm just saying that the *current* behaviour that Heiko was objecting to is due to inheritence, and the behaviour can be switched off if it is *immediately* troublesome. Of course, it will also disappear if we start to use a composition model rather than an inheritence model to put e.g. strings into objects, but that doesn't help anyone in the short-term. M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From Pieter.Neerincx at wur.nl Thu Jul 14 10:02:34 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Thu, 14 Jul 2005 16:02:34 +0200 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: <42D53BE3.7000909@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> <42D53BE3.7000909@ebi.ac.uk> Message-ID: <75FEE7B6-A0DE-4444-B27C-7C7C4F478FB9@wur.nl> Hi Tom, Thanks for the reply. No I at least knew it should be possible :). After googeling for days I finally got it to work. Maybe you can add a small addition for HTTPS to the configuration part of the Taverna manual. If anyone else ever runs into the same problem here is how I got it to work: I did find a lot of posts mentioning compatibility problems when importing openssl generated certificates in a java keystore with keytool. So I was converting my certificates in probably every certificate format known to man, but that was not the solution. In order to make the Java client accept a self-signed certificate from you webserver this is what needs to be done. Generate a self-signed certificate. If you use openssl this step is pretty well documented. The default output will be a certificate in pem format. The default extension for such a certificate is *.crt. This file can contain both a human readable form of the certificate and an encoded one. So it might look something like this: Certificate: Data: Version: 3 (0x2) Serial Number: 0 (0x0) Signature Algorithm: md5WithRSAEncryption Issuer: C=NL, ST=Gelderland, L=Wageningen, O=WUR, OU=Bioinformatics, CN=dev.bioinformatics.nl/ emailAddress=jack.leunissen at wur.nl Validity Not Before: Jul 8 16:27:14 2005 GMT Not After : Aug 7 16:27:14 2005 GMT Subject: C=NL, ST=Gelderland, L=Wageningen, O=WUR, OU=Bioinformatics, CN=dev.bioinformatics.nl/ emailAddress=jack.leunissen at wur.nl Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (1024 bit) Modulus (1024 bit): 00:a1:e2:f2:20:3c:17:da:c1:2c:d9:89:4b:9d:16: ........ 80:a9:e4:02:72:3c:1f:b3:3d Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Key Identifier: 87:F7:8D:B6:63:C1:F9:4D: X509v3 Authority Key Identifier: keyid:87:F7:8D:B6:63:C1:F9: DirName:/C=NL/ST=Gelderland/L=Wageningen/O=WUR/ OU=Bioinformatics/CN=dev.bioinformatics.nl/ emailAddress=jack.leunissen at wur.nl serial:00 X509v3 Basic Constraints: CA:TRUE Signature Algorithm: md5WithRSAEncryption 6e:5c:27:f4:b4:bd:1e:32:c0:ee:03:ce:76:43:c3:e8:3a:50: ........ 0c:e8:f6:98:10:2d:ac:ff:99:3a:5c:f5:f8:66:27:a5:53:c6: 5a:0b -----BEGIN CERTIFICATE----- MIIDxzCCAzCgAwIBAgIBADANBgkqhkiG9w0BAQQFADCBpDELMAkGA1UEBhMCTkwx EzARBgNVBAgTCkdlbGRlcmxhbmQxEzARBgNVBAcTCldhZ2VuaW5nZW4xDDAKBgNV BAoTA1dVUjEXMBUGA1UECxMOQmlvaW5mb3JtYXRpY3MxHjAcBgNVBAMTFWRldi5i ........ bDEkMCIGCSqGSIb3DQEJARYVamFjay5sZXVuaXNzZW5Ad3VyLm5sggEAMAwGA1Ud EwQFMAMBAf8wDQYJKoZIhvcNAQEEBQADgYEAblwn9LS9HjLA7gPOdkPD6DpQUBHF L/4Ihx+J6Y1PVr5NSgWLz5emr0wSqV+adjfYm/+OMV0AkYVQptlm2N61xX7YDbkL mEBS8n22L43f2WU4SP7J24kmhllet56QaS3ImfECP40RwvC+I+26DOj2mBAtrP+Z Olz1+GYnpVPGWgs= -----END CERTIFICATE----- The human readable part is a problem for keytool which you use to import the certificate in your Java keystore. You can convert the certificate in another format that is keytool compatible, but the easiest solution is to strip the human readable part from the certificate. Hence, simply copy -----BEGIN CERTIFICATE-----[encoded certificate]-----END CERTIFICATE----- to a new file. Make sure there are no blank lines left before -----BEGIN CERTIFICATE----- or after -----END CERTIFICATE-----. Now import the certificate in your keystore: keytool -import -trustcacerts -v -alias [some name for your cert.] -file [the cert. without the human readable part] I figured that since the certificate is self-signed, I wouldn't need another certificate authority to make java apps to validate the certificate, but that was wrong. You need to import the same self-signed certificate into your cacerts keystore as well: as root: keytool -import -v -storetype jks -keystore $JAVA_HOME/ lib/security/cacerts -file [the cert. without the human readable part] Now it should work. Cheers, Pieter On 13-Jul-2005, at 6:05 PM, Tom Oinn wrote: > Hi all, > > We've had Taverna working for HTTPS web services so it should be > fine for the MOBY operations. You need to check that the > certificate has a valid signature chain and that the name of the > entity in the certificate matches the hostname - we had issues with > a certificate from a server in china that the java security > framework rejected because the (I think) CN part of the DN in the > cert wasn't the same as the hostname of the endpoint. > > Tom > > Pieter Neerincx wrote: > >> Hi all, >> Is there anyone out there that was able to make BioMOBY >> (realated) Java tools like Yan's Moby Object Browser or Taverna >> work with MOBY Centrals and/or services over HTTPS? I'm having >> trouble with the server's certificate. I used keytool to import >> the server's certificate client-side, but the Java tools keep on >> complaining: "unknown certificate"... >> Cheers, >> Pieter _______________________________________________ >> MOBY-dev mailing list >> MOBY-dev at biomoby.org >> http://www.biomoby.org/mailman/listinfo/moby-dev >> > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > Wageningen University and Research centre (WUR) Laboratory of Bioinformatics Transitorium (building 312) room 1038 Dreijenlaan 3 6703 HA Wageningen phone: 0317-484 706 fax: 0317-483 584 mobile: 06-143 66 783 pieter.neerincx at wur.nl From senger at ebi.ac.uk Mon Jul 18 08:39:27 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Mon, 18 Jul 2005 13:39:27 +0100 (BST) Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <1120860033.3721.7.camel@bioinfo.icapture.ubc.ca> Message-ID: Hi all, Catching up my email piles I wonder if someone can summarize if the discussion about collections in this thread brought any (planned) changes in the API (I am not talking now about how it is, or should be, implemented in Taverna, that's, imho, an another story). My understanding is that (talking about one mobyData object): a) Any Moby service can have more outputs. If so, all of them must be registered. The number of such outputs must be fixed. b) Any of these outputs can be of type either Simple or Collection. c) If it is a Collection, this output can have one or more Simples in that Collection. Such Simples (and their number) are not individually registered. Has this vision been changed? Thanks, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Mon Jul 18 11:09:47 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 18 Jul 2005 08:09:47 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: References: Message-ID: <1121699387.15443.68.camel@bioinfo.icapture.ubc.ca> Hi Martin, That is a 100% correct description of the status quo (I would just re- word point c to say that the TYPE(s) of Simple(s) in the Collection are registered, but not their number) I have not been able to discern from any of the discussions what it is that others are objecting to, so I am a bit lost...?? I don't know if an API change is being requested, or if there is simply a widespread mis-understanding of how Collections are to be used. Your description, however, is completely correct. M On Mon, 2005-07-18 at 13:39 +0100, Martin Senger wrote: > Hi all, > Catching up my email piles I wonder if someone can summarize if the > discussion about collections in this thread brought any (planned) changes > in the API (I am not talking now about how it is, or should be, > implemented in Taverna, that's, imho, an another story). > My understanding is that (talking about one mobyData object): > a) Any Moby service can have more outputs. If so, all of them must be > registered. The number of such outputs must be fixed. > b) Any of these outputs can be of type either Simple or Collection. > c) If it is a Collection, this output can have one or more Simples in > that Collection. Such Simples (and their number) are not individually > registered. > > Has this vision been changed? > > Thanks, > Martin > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From rebecca.ernst at gsf.de Mon Jul 18 11:14:04 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Mon, 18 Jul 2005 17:14:04 +0200 Subject: [MOBY-dev] buggy new biomoby release Message-ID: <42DBC73C.5030302@gsf.de> Hi Mark (and rest)! As suggested Dirk and I sat down today to do a cvs update to get the last stable release. Now it's 5 hours later and we gave up and switched back to the last working code we had (and which we luckily saved before doing the update! :-)) There are several problems keeping us from updating our code: First of all the links on the release site (biomoby.org/releases) are crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't notice this and therefore received the older version. This version is VERY outdated and not functional (e.g.it uses DOM instead of LibXML) Then we modified the link to get 0.8.2. This version is also not functional as it uses lsid (although this was meant to be the version BEFORE the lsid change). Even when we added a column lsid to the database it didn't work (I guess one would also need the lsid resolver and so on.) Another try was to do an cvs update which crashed everything as completely buggy and syntactical wrong code was checked in (especially in Adaptor/.../mysql.pm) So luckily we could rescue everything because of our save but I guess this is not the way to make people happy installing BioMoby :-( Could we please make sure that releases are functional and the code in the cvs is not complete crap.... otherwise I'll have to do some more spanking ;-) Cheers, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From markw at illuminae.com Mon Jul 18 11:23:51 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 18 Jul 2005 08:23:51 -0700 Subject: [MOBY-dev] buggy new biomoby release In-Reply-To: <42DBC73C.5030302@gsf.de> References: <42DBC73C.5030302@gsf.de> Message-ID: <42DBC987.2000401@illuminae.com> Hi Rebecca, The CVS version certainly wont work (as I noted on the list last week). The link to 0.8.2 was wrong, as you say - sorry about that! The 0.8.2 version should work, however, as I set it up on a completely virgin machine (the current MOBY Central public server) immediately before making the release and it worked perfectly...? Can you be more specific with your error report? v.v. the LSID libraries - it is possible that the requirement for the LSID libraries is in a "use" statement somewhere, but it definitely isn't a real requirement for the 0.8.2 code. If you see a "use" in there, or in the makefile, that is an error for sure. Can you tell me exactly what errors you were seeing? Cheers! M Rebecca Ernst wrote: > Hi Mark (and rest)! > > As suggested Dirk and I sat down today to do a cvs update to get the > last stable release. > > Now it's 5 hours later and we gave up and switched back to the last > working code we had (and which we luckily saved before doing the > update! :-)) > > There are several problems keeping us from updating our code: > > First of all the links on the release site (biomoby.org/releases) are > crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't > notice this and therefore received the older version. This version is > VERY outdated and not functional (e.g.it uses DOM instead of LibXML) > > Then we modified the link to get 0.8.2. This version is also not > functional as it uses lsid (although this was meant to be the version > BEFORE the lsid change). Even when we added a column lsid to the > database it didn't work (I guess one would also need the lsid resolver > and so on.) > > Another try was to do an cvs update which crashed everything as > completely buggy and syntactical wrong code was checked in (especially > in Adaptor/.../mysql.pm) > > > So luckily we could rescue everything because of our save but I guess > this is not the way to make people happy installing BioMoby :-( > > Could we please make sure that releases are functional and the code in > the cvs is not complete crap.... otherwise I'll have to do some more > spanking ;-) > > > Cheers, > Rebecca > > > > > > From markw at illuminae.com Mon Jul 18 11:39:32 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 18 Jul 2005 08:39:32 -0700 Subject: [MOBY-dev] Re: [moby] buggy new biomoby release In-Reply-To: <42DBC73C.5030302@gsf.de> References: <42DBC73C.5030302@gsf.de> Message-ID: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Doh! I missed one. There is an (unused) call to LS::ID->new in the code. Unfortunately, I do have the LSID libraries installed so I didn't pick up on that. I have commented-out the line in the release codebase and will make another release, but before I do I want to make sure that this was the only problem you saw. Let me know, M On Mon, 2005-07-18 at 17:14 +0200, Rebecca Ernst wrote: > Hi Mark (and rest)! > > As suggested Dirk and I sat down today to do a cvs update to get the > last stable release. > > Now it's 5 hours later and we gave up and switched back to the last > working code we had (and which we luckily saved before doing the update! > :-)) > > There are several problems keeping us from updating our code: > > First of all the links on the release site (biomoby.org/releases) are > crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't > notice this and therefore received the older version. This version is > VERY outdated and not functional (e.g.it uses DOM instead of LibXML) > > Then we modified the link to get 0.8.2. This version is also not > functional as it uses lsid (although this was meant to be the version > BEFORE the lsid change). Even when we added a column lsid to the > database it didn't work (I guess one would also need the lsid resolver > and so on.) > > Another try was to do an cvs update which crashed everything as > completely buggy and syntactical wrong code was checked in (especially > in Adaptor/.../mysql.pm) > > > So luckily we could rescue everything because of our save but I guess > this is not the way to make people happy installing BioMoby :-( > > Could we please make sure that releases are functional and the code in > the cvs is not complete crap.... otherwise I'll have to do some more > spanking ;-) > > > Cheers, > Rebecca > > > > > > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From edward.kawas at gmail.com Mon Jul 18 20:38:10 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Mon, 18 Jul 2005 17:38:10 -0700 Subject: [MOBY-dev] request for community feedback on certain issues In-Reply-To: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Message-ID: <42dc4b78.5905bfd9.1943.793b@mx.gmail.com> Hi 'BioMoby'ers, Below I am placing some questions, comments, etc that we would like your feedback on. Hopefully some time soon it will be placed on a twiki! I also attached the file in html format in case this email is too hard to read. Eddie ---Start--- Request for Feedback from the BioMoby Community! Before I begin, I would just like to say that the BioMoby Project is a open source, community driven project. As such, it is key that if the community doesn't like how something is done, then the community should fix it. If something is broken, doesn't work as intended or is vague, speak up and make your voice heard! BioMoby is not a one man project, but a community driven one. Things will only get better if we know what is wrong and if people take initiative. So I (Eddie) just got back from the INBs Technical Meeting in beautiful Malaga, Spain and I have some items that the community needs to talk about below. Also, be sure to keep an eye out for proposals about handling asynchronous services as well as error handling in Moby. When they come, please speak up with your likes, dislikes and ways to make things better. 1. API addition of a method called getResourcesURL: Returns an xml document that describes the location of RESOURCES script that we hear so much about. It is the resources script that contains the ontologies expressed in RDF/XML example: Services http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Services Objects http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Objects ... 2. How do I know what predicates are available in the RDF documents that describe the Moby ontologies? a. should we provide a list of predicates and their intended meanings (on a wiki, etc)? b. should we provide an RDF schema defining the predicates (* probably the way to go) c. Should we allow the registration of new predicates, if so, should we make this an api call, etc. 3. LSID resolution services a. is the LSID attribute available and documented? b. when will this occur, i have heard so much about this, yet i have seen nothing. 4. LSIDs versus simple names a. why is the name of an object, service, or service type is sometimes a simple name and sometimes an LSID? b. how can i even retrieve the LSID of an object, service, or service type? c. should api calls be added to getName and getLSID for objects, services and service types? 5. LSIDs and Service Instances a. If a service instance is registered by 2 different authorities, should they have the same LSID? b. if the service instance is registered in 2 registries, should the LSIDs be the same? c. will modified service instances have versioning information? 6. LSID resolution for objects in the BioMoby ontologies a. what should getData return for i.namespaces (the name of the namespace?) ii. service types (the name of the service type?) iii. objects (xml schema? - note that schemas for datatypes should be ready and available very soon :-) iv. service instances (the name of the instance?) b. should we allow the namespace part of an LSID to be configurable? c. should we allow the authority part of an LSID to be configurable? 7. Would you like to have 3rd party annotation of metadata? a. use case: adds mirroring b. use case: allows people to provide additional metadata i. if you want only one source for metadata, then ignore this question. 9. Predicates a. should we add the following predicates to the metadata of service instances? i. sampleInputData - specifies valid input for the service instance ii. sampleOutputData - specifies the output for the service instance based on the test input. iii. approvedBy - specifies whether this service has been approved. Currently, this would only be useful for the INB, but many more may find it useful. b. how should the sample input/output be formatted? c. should we allow the annotation of datatypes? 10. API and Code versioning a. why are the api and code versions different? 11. Datatype naming a. do you think that the names of datatypes should be regulated? i. what would be the right way to specify a datatypes name? ii. what should we do with text_plain and text-plain? 12. RDF Agent a. do you think that there should be an api call, getUpdate that tells the agent to check for modifications to the service instance described by the metadata in the RDF document at the signatureURL? b. what is the time line to get the agent running? c. besides curation, how is the agent useful to me? 13. Service a. should article names for service instance input and output datatypes be mandatory? b. why does a service that advertise an output datatype called 'foo' return 'bar' instead? ---END--- -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: request_for_feedback.html Url: http://www.biomoby.org/pipermail/moby-dev/attachments/20050718/27d352a6/request_for_feedback.pl From senger at ebi.ac.uk Tue Jul 19 05:18:58 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 10:18:58 +0100 (BST) Subject: [MOBY-dev] request for community feedback on certain issues In-Reply-To: <42dc4b78.5905bfd9.1943.793b@mx.gmail.com> Message-ID: Hi Eddie, > BioMoby is not a one man project, but a community driven one. > Well, let's hope it is :-) Services are trully distributed, and some clients accessing Moby Central as well. The registry itself still seems to be a one-man area (imho). > 1. API addition of a method called getResourcesURL: > I understand that this is not a new requirement (we all asked for it many times) but this is a concrete suggestion how to make it. I agree that getting these URLs from an API call is cleaner than just document how such URLs should look like. Only I would suggest slightly different XML response for that (but no strong opinion on my side about it). What we need are actually name-value pairs, each of them representing a name of a resource (such as 'services; and a URL for the RDF describing such resource. So why not just use two attributes for that? Something like this: ... Cheers, Martin PS. Eddie, you should perhaps consider to send everything again, but splitted into few separate topics - that would be easier to find in our mailing lists archives... M. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From rebecca.ernst at gsf.de Tue Jul 19 07:22:15 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue, 19 Jul 2005 13:22:15 +0200 Subject: [MOBY-dev] Re: [moby] buggy new biomoby release In-Reply-To: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> References: <42DBC73C.5030302@gsf.de> <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Message-ID: <42DCE267.8020609@gsf.de> Hi Mark! We tried once more and installed the new release (0.8.2). The problems we can see are that our perl services are not working anymore ("service returned no response") the bad thing is that it doesn't give any error so we have no clue where the problem is. Gbrowse works - the services can be listed and non-MIPS services and MIPS Java services can be executed. So Moby Central obviously works okay since (since we added the column lsid to the db) We then tried to remove the method "_getServiceInstanceRDF" in the MOBY/Central.pm because in there it uses lsid but this didn't solve anything. Just in case we find the problem and you build a new release I have some more suggestions leading to even more user-friendlyness ;-) : - include all libraries not only the perl libraries because all former releases contained everything and now we have to get the other libs from older releases and that means copying it. I guess that most people would expect that they get a full release otherwise consider renaming it (not biomoby release but perl_lib_release) - don't rename the moby-live path to moby-live_API_releasewhatever it now requires that people move it (or adjust their pathes) - next to the link (for downloading the release) add the information that for database provider they will have to change their database ("alter table service_instance add lsid VARCHAR (255) NOT NULL default ''; ") Cheers! Rebecca Mark Wilkinson wrote: >Doh! I missed one. > >There is an (unused) call to LS::ID->new in the code. Unfortunately, I >do have the LSID libraries installed so I didn't pick up on that. > >I have commented-out the line in the release codebase and will make >another release, but before I do I want to make sure that this was the >only problem you saw. > >Let me know, > >M > > > > >On Mon, 2005-07-18 at 17:14 +0200, Rebecca Ernst wrote: > > >>Hi Mark (and rest)! >> >>As suggested Dirk and I sat down today to do a cvs update to get the >>last stable release. >> >>Now it's 5 hours later and we gave up and switched back to the last >>working code we had (and which we luckily saved before doing the update! >>:-)) >> >>There are several problems keeping us from updating our code: >> >>First of all the links on the release site (biomoby.org/releases) are >>crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't >>notice this and therefore received the older version. This version is >>VERY outdated and not functional (e.g.it uses DOM instead of LibXML) >> >>Then we modified the link to get 0.8.2. This version is also not >>functional as it uses lsid (although this was meant to be the version >>BEFORE the lsid change). Even when we added a column lsid to the >>database it didn't work (I guess one would also need the lsid resolver >>and so on.) >> >>Another try was to do an cvs update which crashed everything as >>completely buggy and syntactical wrong code was checked in (especially >>in Adaptor/.../mysql.pm) >> >> >>So luckily we could rescue everything because of our save but I guess >>this is not the way to make people happy installing BioMoby :-( >> >>Could we please make sure that releases are functional and the code in >>the cvs is not complete crap.... otherwise I'll have to do some more >>spanking ;-) >> >> >>Cheers, >>Rebecca >> >> >> >> >> >> >> >> -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From rebecca.ernst at gsf.de Tue Jul 19 09:15:44 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue, 19 Jul 2005 15:15:44 +0200 Subject: [MOBY-dev] NO buggy new biomoby release In-Reply-To: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> References: <42DBC73C.5030302@gsf.de> <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Message-ID: <42DCFD00.5020306@gsf.de> Hey Mark! Congratulations - you won a round of spanking ... You can spank Dirk and me. Shame on us - the release you provided was as perfect as one would expect ;-) The problem why the services didn't work anymore was because of one module (LoggingSubs.pm) which we use within Planet for logging of the services. This was (how ugly!) deployed in the moby-live directory. As the LoggingSubs.pm was not a part of the release this was missing and caused the services to fail. The call to LS::ID didn't matter at all you can leave everything (although you might take the suggestions into account which I mentioned in the last mail) Sorry again for suspecting you ;-) Cheers! Rebecca Mark Wilkinson wrote: >Doh! I missed one. > >There is an (unused) call to LS::ID->new in the code. Unfortunately, I >do have the LSID libraries installed so I didn't pick up on that. > >I have commented-out the line in the release codebase and will make >another release, but before I do I want to make sure that this was the >only problem you saw. > >Let me know, > >M > > > > >On Mon, 2005-07-18 at 17:14 +0200, Rebecca Ernst wrote: > > >>Hi Mark (and rest)! >> >>As suggested Dirk and I sat down today to do a cvs update to get the >>last stable release. >> >>Now it's 5 hours later and we gave up and switched back to the last >>working code we had (and which we luckily saved before doing the update! >>:-)) >> >>There are several problems keeping us from updating our code: >> >>First of all the links on the release site (biomoby.org/releases) are >>crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't >>notice this and therefore received the older version. This version is >>VERY outdated and not functional (e.g.it uses DOM instead of LibXML) >> >>Then we modified the link to get 0.8.2. This version is also not >>functional as it uses lsid (although this was meant to be the version >>BEFORE the lsid change). Even when we added a column lsid to the >>database it didn't work (I guess one would also need the lsid resolver >>and so on.) >> >>Another try was to do an cvs update which crashed everything as >>completely buggy and syntactical wrong code was checked in (especially >>in Adaptor/.../mysql.pm) >> >> >>So luckily we could rescue everything because of our save but I guess >>this is not the way to make people happy installing BioMoby :-( >> >>Could we please make sure that releases are functional and the code in >>the cvs is not complete crap.... otherwise I'll have to do some more >>spanking ;-) >> >> >>Cheers, >>Rebecca >> >> >> >> >> >> >> >> -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From senger at ebi.ac.uk Tue Jul 19 10:00:20 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 15:00:20 +0100 (BST) Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <42D39162.1090004@ac.upc.es> Message-ID: Hi, Several small changes have been done to jMoby (all of them should be backward compatible): * The default location and default namespace (URI) of Moby Central was updated (to ...icapture...). Actually, this change has been around already for some time but Eddie never announced it nor logged it in ChangeLog :-) * Roman found a design flaw in the getServiceNames() method returning Map where the keys are service names. This was wrong because there may be (and actually are, e.g. service genbankToGene) more services with the same name but served by different authorities. So I added a new method getServiceNamesByAuthorities() to the interface Central.java (and to its implementation CentralImpl.java), and I deprecated the old method (but it is still there so your code does not break). In future, you should use the new method. This also allowed to add a new command-line option '-la' to the MobyCmdLineClient. Try: build/run/run-cmdline-client -la and you get a list of all services grouped by authorities now. * The location and maintainance of the repository with the third-party libraries for jMoby is now fully "de-sengerized" (as requested in Vancouver). It is described in http://www.biomoby.org/moby-live/Java/docs/3rdPartyLibraries.html. What you should do is the following: cd moby-live/Java ./build-dev.sh cleanall ./build.sh This will populate your 'lib' directory with the same libraries as you have now, but from a different repository (it's just one time job). With regards, Martin PS. In the hopefully near future I plan to make additional changes/fixes: * fix bug (my misunderstanding) with the tag expandobject * added ways how to get RDF resources (Mark, I am desperately waiting for this one - in Malaga last week we created with Eddie a suggestion how to get URLs from the Central API...) * fix bug in the MobyGraphs (Ben, as you see, I am getting close to it...) M. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From edward.kawas at gmail.com Tue Jul 19 10:08:56 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:08:56 -0700 Subject: [MOBY-dev] request for community feedback on certain issues In-Reply-To: Message-ID: <42dd097b.4d90ae57.63bd.2f19@mx.gmail.com> Hi Once again, I have added the page to a twiki, at http://www.biomoby.org/twiki/bin//view/General/FeedBackDoc I have also added Martins comments to it. If this isn't good enough, and you do prefer to have the items split up, then I will do it. Eddie > -----Original Message----- > From: moby-dev-bounces at portal.open-bio.org [mailto:moby- > dev-bounces at portal.open-bio.org] On Behalf Of Martin > Senger > Sent: Tuesday, July 19, 2005 2:19 AM > To: Core developer announcements > Subject: Re: [MOBY-dev] request for community feedback on > certain issues > > Hi Eddie, > > > BioMoby is not a one man project, but a community driven > one. > > > Well, let's hope it is :-) > Services are trully distributed, and some clients > accessing Moby > Central as well. The registry itself still seems to be a > one-man area > (imho). > > > 1. API addition of a method called getResourcesURL: > > > I understand that this is not a new requirement (we > all asked for it > many times) but this is a concrete suggestion how to make > it. I agree that > getting these URLs from an API call is cleaner than just > document how such > URLs should look like. > Only I would suggest slightly different XML response > for that (but no > strong opinion on my side about it). What we need are > actually name-value > pairs, each of them representing a name of a resource > (such as 'services; > and a URL for the RDF describing such resource. So why not > just use two > attributes for that? Something like this: > > > > url="http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOB > Y-S/Services"/> > url="http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOB > Y-S/Objects"/> > ... > url="http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOB > Y-S/All"/> > > > > Cheers, > Martin > > PS. Eddie, you should perhaps consider to send everything > again, but > splitted into few separate topics - that would be easier > to find in our > mailing lists archives... M. > > -- > Martin Senger > > EMBL Outstation - Hinxton Senger at EBI.ac.uk > European Bioinformatics Institute Phone: (+44) 1223 > 494636 > Wellcome Trust Genome Campus (Switchboard: > 494444) > Hinxton Fax : (+44) 1223 > 494468 > Cambridge CB10 1SD > United Kingdom > http://industry.ebi.ac.uk/~senger > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev From gordonp at ucalgary.ca Tue Jul 19 10:14:13 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Tue, 19 Jul 2005 08:14:13 -0600 Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: References: Message-ID: <42DD0AB5.9020502@ucalgary.ca> Anyone else having trouble with MOBY Central? org.biomoby.shared.MobyException: ===ERROR=== Fault details: [stackTrace: null] Fault string: java.lang.NullPointerException Fault code: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException Fault actor: null When calling: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl =========== at org.biomoby.client.CentralImpl.doCall(CentralImpl.java:191) at org.biomoby.client.CentralImpl.getNamespaces(CentralImpl.java:779)... From markw at illuminae.com Tue Jul 19 10:22:36 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue, 19 Jul 2005 07:22:36 -0700 Subject: [moby] [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: References: Message-ID: <1121782956.21968.9.camel@bioinfo.icapture.ubc.ca> On Tue, 2005-07-19 at 15:00 +0100, Martin Senger wrote: > * added ways how to get RDF resources (Mark, I am desperately waiting > for this one - in Malaga last week we created with Eddie a suggestion how > to get URLs from the Central API...) HI Martin, Eddie gave us the update on his Spanish activities yesterday. He told me what you discussed and I agree that this is a good idea. I will add that function to the API right away, and I can code it into the currently deployed version of MOBY, so that it will work from the main registry, however it will take a couple of weeks to get it into the CVS since the CVS codebase is currently going through a major refactoring effort in anticipation of our trip to Manchester in two weeks... You will notice that I have added LSID's to the 0.85 API that is on the website (they are part of most of the messages coming back from MOBY Central). These can be resolved to metadata to give you the same RDF graphs. HOWEVER, the code is not yet up-to-date relative to the API (the API is at 0.85 but the codebase is only at 0.82). Again, I'm trying to get the 0.85 codebase out as quickly as possible, but it is taking some time as there are only my hands (5% of the time) and my (highly competent!) co-op student Dennis assigned to this task... we simply suffer from lack of resourcing at this end :-( Cheers! M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From akerhornou at imim.es Tue Jul 19 10:30:57 2005 From: akerhornou at imim.es (Arnaud Kerhornou) Date: Tue, 19 Jul 2005 16:30:57 +0200 Subject: [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries Message-ID: <42DD0EA1.7030106@imim.es> Hi, I am working for the INB in Spain setting up Moby Web services, and I am trying to switch my code to use the new release, BioMoby 0.8.2 (was using BioMoby 0.8.1 before). I have a client script that is interrogating a registry server and ask for a given service. If the registry server is the following one, http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if I want to interrogate another one, I am getting an error. I am instanciating a Central object with the following parameters, => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl => URI: http://chirimoyo.ac.uma.es/MOBY/Central => Proxy: No Proxy Server The instanciation of Central seems to go okay, but then when I want to execute 'findService' method, I get the following error: -------------------------> Can't call method "getValue" on an undefined value at /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 <------------------------- Central.pm line 1728 is the following one (in _parseServices method) my $lsid = $Service->getAttributeNode('lsid')->getValue; Any clue of what is going wrong or do I need to update my client code ? Thanks Arnaud ___________________________________________________________________________ Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger T?l?chargez cette version sur http://fr.messenger.yahoo.com From senger at ebi.ac.uk Tue Jul 19 10:35:26 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 15:35:26 +0100 (BST) Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <42DD0AB5.9020502@ucalgary.ca> Message-ID: > Anyone else having trouble with MOBY Central? > No, I don't (using: build/run/run-testing-central). Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 10:35:26 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 15:35:26 +0100 (BST) Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <42DD0AB5.9020502@ucalgary.ca> Message-ID: > Anyone else having trouble with MOBY Central? > No, I don't (using: build/run/run-testing-central). Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 10:37:22 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 15:37:22 +0100 (BST) Subject: [moby] [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <1121782956.21968.9.camel@bioinfo.icapture.ubc.ca> Message-ID: Mark, > I will add that function to the API right away, and I can code it into > the currently deployed version of MOBY, so that it will work from the > main registry > That's great! Just let me/us know what is the exact XML structure you are going to return by this new method (the best would be to put it into the API doc I guess), and I will add it into jMoby at once. Which means that this new method will work from your moby central but not yet from the others centrals, correct? Until their providers implement it as well - but their role is here difficult because they cannoy use CVS now. Do I understand it correctly? > You will notice that I have added LSID's to the 0.85 API that is on the > website (they are part of most of the messages coming back from MOBY > Central). These can be resolved to metadata to give you the same RDF > graphs. HOWEVER, the code is not yet up-to-date relative to the API > (the API is at 0.85 but the codebase is only at 0.82). > You mean the code for returning attribute 'lsid' in XML, or code for the LSID resilution service (to use such attribute), or both? Cheers, Martin PS. The clock is ticking - in 12 days I will be sitting in Philippines and working on Biomoby :-) M. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From rebecca.ernst at gsf.de Tue Jul 19 10:41:30 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue, 19 Jul 2005 16:41:30 +0200 Subject: [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <42DD0EA1.7030106@imim.es> References: <42DD0EA1.7030106@imim.es> Message-ID: <42DD111A.8060900@gsf.de> Hi Arnaud! I am not sure if you are facing the same problem that we did... possibly the registry you want to contact doesn't contain the colunm 'lsid' this needs to be updated along with the new code so you have to modify your database like this: "alter table service_instance add lsid VARCHAR (255) NOT NULL default ''; " Cheers, Rebecca Arnaud Kerhornou wrote: > Hi, > > I am working for the INB in Spain setting up Moby Web services, and I > am trying to switch my code to use the new release, BioMoby 0.8.2 (was > using BioMoby 0.8.1 before). > > I have a client script that is interrogating a registry server and ask > for a given service. If the registry server is the following one, > http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if > I want to interrogate another one, I am getting an error. > > I am instanciating a Central object with the following parameters, > > => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl > => URI: http://chirimoyo.ac.uma.es/MOBY/Central > => Proxy: No Proxy Server > > The instanciation of Central seems to go okay, but then when I want to > execute 'findService' method, I get the following error: > -------------------------> > Can't call method "getValue" on an undefined value at > /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 > <------------------------- > > Central.pm line 1728 is the following one (in _parseServices method) > > my $lsid = $Service->getAttributeNode('lsid')->getValue; > > Any clue of what is going wrong or do I need to update my client code ? > > Thanks > Arnaud > > > > > > ___________________________________________________________________________ > Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! > Messenger T?l?chargez cette version sur http://fr.messenger.yahoo.com > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From rebecca.ernst at gsf.de Tue Jul 19 10:41:30 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue, 19 Jul 2005 16:41:30 +0200 Subject: [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <42DD0EA1.7030106@imim.es> References: <42DD0EA1.7030106@imim.es> Message-ID: <42DD111A.8060900@gsf.de> Hi Arnaud! I am not sure if you are facing the same problem that we did... possibly the registry you want to contact doesn't contain the colunm 'lsid' this needs to be updated along with the new code so you have to modify your database like this: "alter table service_instance add lsid VARCHAR (255) NOT NULL default ''; " Cheers, Rebecca Arnaud Kerhornou wrote: > Hi, > > I am working for the INB in Spain setting up Moby Web services, and I > am trying to switch my code to use the new release, BioMoby 0.8.2 (was > using BioMoby 0.8.1 before). > > I have a client script that is interrogating a registry server and ask > for a given service. If the registry server is the following one, > http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if > I want to interrogate another one, I am getting an error. > > I am instanciating a Central object with the following parameters, > > => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl > => URI: http://chirimoyo.ac.uma.es/MOBY/Central > => Proxy: No Proxy Server > > The instanciation of Central seems to go okay, but then when I want to > execute 'findService' method, I get the following error: > -------------------------> > Can't call method "getValue" on an undefined value at > /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 > <------------------------- > > Central.pm line 1728 is the following one (in _parseServices method) > > my $lsid = $Service->getAttributeNode('lsid')->getValue; > > Any clue of what is going wrong or do I need to update my client code ? > > Thanks > Arnaud > > > > > > ___________________________________________________________________________ > Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! > Messenger T?l?chargez cette version sur http://fr.messenger.yahoo.com > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From edward.kawas at gmail.com Tue Jul 19 10:45:52 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:45:52 -0700 Subject: [MOBY-dev] rfcf 1. API addition of a method called getResourcesURL: In-Reply-To: Message-ID: <42dd1223.513aad7a.369d.5415@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 1. API addition of a method called getResourcesURL: Returns an xml document that describes the location of RESOURCES script that we hear so much about. It is the resources script that contains the ontologies expressed in RDF/XML example: Services http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Services Objects http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Objects ... Comments: Martin: I understand that this is not a new requirement (we all asked for it many times) but this is a concrete suggestion how to make it. I agree that getting these URLs from an API call is cleaner than just document how such URLs should look like. Only I would suggest slightly different XML response for that (but no strong opinion on my side about it). What we need are actually name-value pairs, each of them representing a name of a resource (such as 'services; and a URL for the RDF describing such resource. So why not just use two attributes for that? Something like this: ... Mark: I will add that function to the API right away, and I can code it into the currently deployed version of MOBY, so that it will work from the main registry, however it will take a couple of weeks to get it into the CVS since the CVS codebase is currently going through a major refactoring effort in anticipation of our trip to Manchester in two weeks... From edward.kawas at gmail.com Tue Jul 19 10:46:03 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:03 -0700 Subject: [MOBY-dev] rfcf 2. How do I know what predicates are available in the RDF In-Reply-To: Message-ID: <42dd122e.54b1a5ca.63bd.4a5c@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 2. How do I know what predicates are available in the RDF documents that describe the Moby ontologies? a. should we provide a list of predicates and their intended meanings (on a wiki, etc)? b. should we provide an RDF schema defining the predicates (* probably the way to go) c. Should we allow the registration of new predicates, if so, should we make this an api call, etc. From edward.kawas at gmail.com Tue Jul 19 10:46:15 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:15 -0700 Subject: [MOBY-dev] rfcf 3. LSID resolution services In-Reply-To: Message-ID: <42dd123b.670fc02b.2dd4.ffff929f@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 3. LSID resolution services a. is the LSID attribute available and documented? comments: Mark: You will notice that I have added LSID's to the 0.85 API that is on the website (they are part of most of the messages coming back from MOBY Central). These can be resolved to metadata to give you the same RDF graphs. HOWEVER, the code is not yet up-to-date relative to the API (the API is at 0.85 but the codebase is only at 0.82). Again, I'm trying to get the 0.85 codebase out as quickly as possible, but it is taking some time as there are only my hands (5% of the time) and my (highly competent!) co-op student Dennis assigned to this task. b. when will this occur, i have heard so much about this, yet i have seen nothing. From edward.kawas at gmail.com Tue Jul 19 10:46:23 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:23 -0700 Subject: [MOBY-dev] rfcf 4. LSIDs versus simple names In-Reply-To: Message-ID: <42dd1240.1e9853c8.2dd4.ffff92af@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 4. LSIDs versus simple names a. why is the name of an object, service, or service type is sometimes a simple name and sometimes an LSID? b. how can i even retrieve the LSID of an object, service, or service type? c. should api calls be added to getName and getLSID for objects, services and service types? From edward.kawas at gmail.com Tue Jul 19 10:46:33 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:33 -0700 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: Message-ID: <42dd124c.7fdce064.76fe.ffff9b01@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 5. LSIDs and Service Instances a..If a service instance is registered by 2 different authoritys, should they have the same LSID? b. if the service instance is registered in 2 registries, should the LSIDs be the same? c. will modified service instances have versioning information? From edward.kawas at gmail.com Tue Jul 19 10:46:40 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:40 -0700 Subject: [MOBY-dev] rfcf 6. LSID resolution for objects in the BioMoby ontologies In-Reply-To: Message-ID: <42dd1252.3e967c5f.76fe.ffff9b10@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 6. LSID resolution for objects in the BioMoby ontologies a. what should getData return for i.namespaces (the name of the namespace?) ii. service types (the name of the service type?) iii. objects (xml schema? - note that schemas for datatypes should be ready and available very soon :-) iv. service instances (the name of the instance?) b. should we allow the namespace part of an LSID to be configurable? c. should we allow the authority part of an LSID to be configurable? From markw at illuminae.com Tue Jul 19 10:47:18 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue, 19 Jul 2005 07:47:18 -0700 Subject: [moby] [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <42DD0EA1.7030106@imim.es> References: <42DD0EA1.7030106@imim.es> Message-ID: <1121784438.21968.16.camel@bioinfo.icapture.ubc.ca> I've just packaged up another release (0.8.2a) that includes fixes for the 0.8.2 release bugs that have been reported so far. Please try this one and tell me if it still causes problems. M On Tue, 2005-07-19 at 16:30 +0200, Arnaud Kerhornou wrote: > Hi, > > I am working for the INB in Spain setting up Moby Web services, and I am > trying to switch my code to use the new release, BioMoby 0.8.2 (was > using BioMoby 0.8.1 before). > > I have a client script that is interrogating a registry server and ask > for a given service. If the registry server is the following one, > http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if I > want to interrogate another one, I am getting an error. > > I am instanciating a Central object with the following parameters, > > => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl > => URI: http://chirimoyo.ac.uma.es/MOBY/Central > => Proxy: No Proxy Server > > The instanciation of Central seems to go okay, but then when I want to > execute 'findService' method, I get the following error: > -------------------------> > Can't call method "getValue" on an undefined value at > /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 > <------------------------- > > Central.pm line 1728 is the following one (in _parseServices method) > > my $lsid = $Service->getAttributeNode('lsid')->getValue; > > Any clue of what is going wrong or do I need to update my client code ? > > Thanks > Arnaud > > > > > > ___________________________________________________________________________ > Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger > T?l?chargez cette version sur http://fr.messenger.yahoo.com > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From edward.kawas at gmail.com Tue Jul 19 10:46:48 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:48 -0700 Subject: [MOBY-dev] rfcf 7. Would you like to have 3rd party annotation of metadata? In-Reply-To: Message-ID: <42dd125a.5ec5bd58.43d9.3f96@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 7. Would you like to have 3rd party annotation of metadata? a. use case: adds mirroring b. use case: allows people to provide additional metadata i. if you want only one source for metadata, then ignore this question. From edward.kawas at gmail.com Tue Jul 19 10:46:59 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:59 -0700 Subject: [MOBY-dev] rfcf 8. Predicates In-Reply-To: Message-ID: <42dd1264.5c49d513.43d9.3fa7@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 8. Predicates a. should we add the following predicates to the metadata of service instances? i. sampleInputData - specifies valid input for the service instance ii. sampleOutputData - specifies the output for the service instance based on the test input. iii. approvedBy - specifies whether this service has been approved. Currently, this would only be useful for the INB, but many more may find it useful. b. how should the sample input/output be formatted? c. should we allow the annotation of datatypes? From edward.kawas at gmail.com Tue Jul 19 10:47:01 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:47:01 -0700 Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: Message-ID: <42dd1266.6b7805fa.43d9.3fae@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 12. Services a. should article names for service instance input and output datatypes be mandatory? b. why does a service that advertise an output datatype called 'foo' return 'bar' instead? From edward.kawas at gmail.com Tue Jul 19 10:47:06 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:47:06 -0700 Subject: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: Message-ID: <42dd126b.46e6258d.43d9.3fb7@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 11. RDF Agent a. do you think that there should be an api call, getUpdate that tells the agent to check for modifications to the service instance described by the metadata in the RDF document at the signatureURL? b. what is the time line to get the agent running? c. besides curation, how is the agent useful to me? From edward.kawas at gmail.com Tue Jul 19 10:47:11 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:47:11 -0700 Subject: [MOBY-dev] rfcf 10. Datatype naming In-Reply-To: Message-ID: <42dd1271.3247625d.71ce.2bb7@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 10. Datatype naming a. do you think that the names of datatypes should be regulated? i. what would be the right way to specify a datatypes name? ii. what should we do with text_plain and text-plain? From edward.kawas at gmail.com Tue Jul 19 10:47:16 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:47:16 -0700 Subject: [MOBY-dev] rfcf 9. API and Code versioning In-Reply-To: Message-ID: <42dd1275.5d41a999.71ce.2bc3@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 9. API and Code versioning a. why are the api and code versions different? Mark: the code is not yet up-to-date relative to the API (the API is at 0.85 but the codebase is only at 0.82). Again, I'm trying to get the 0.85 codebase out as quickly as possible, but it is taking some time as there are only my hands (5% of the time) and my (highly competent!) co-op student Dennis assigned to this task. From axk at sanger.ac.uk Tue Jul 19 11:01:08 2005 From: axk at sanger.ac.uk (Arnaud Kerhornou) Date: Tue, 19 Jul 2005 17:01:08 +0200 Subject: [moby] [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <1121784438.21968.16.camel@bioinfo.icapture.ubc.ca> References: <42DD0EA1.7030106@imim.es> <1121784438.21968.16.camel@bioinfo.icapture.ubc.ca> Message-ID: <42DD15B4.3000202@sanger.ac.uk> Hi Mark, I have downloaded the new release, and the script works fine now, Thanks Arnaud Mark Wilkinson wrote: >I've just packaged up another release (0.8.2a) that includes fixes for >the 0.8.2 release bugs that have been reported so far. > >Please try this one and tell me if it still causes problems. > >M > > >On Tue, 2005-07-19 at 16:30 +0200, Arnaud Kerhornou wrote: > > >>Hi, >> >>I am working for the INB in Spain setting up Moby Web services, and I am >>trying to switch my code to use the new release, BioMoby 0.8.2 (was >>using BioMoby 0.8.1 before). >> >>I have a client script that is interrogating a registry server and ask >>for a given service. If the registry server is the following one, >>http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if I >>want to interrogate another one, I am getting an error. >> >>I am instanciating a Central object with the following parameters, >> >> => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl >> => URI: http://chirimoyo.ac.uma.es/MOBY/Central >> => Proxy: No Proxy Server >> >>The instanciation of Central seems to go okay, but then when I want to >>execute 'findService' method, I get the following error: >>-------------------------> >>Can't call method "getValue" on an undefined value at >>/home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 >><------------------------- >> >>Central.pm line 1728 is the following one (in _parseServices method) >> >>my $lsid = $Service->getAttributeNode('lsid')->getValue; >> >>Any clue of what is going wrong or do I need to update my client code ? >> >>Thanks >>Arnaud >> >> >> >> >> >>___________________________________________________________________________ >>Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger >>Téléchargez cette version sur http://fr.messenger.yahoo.com >> >>_______________________________________________ >>MOBY-dev mailing list >>MOBY-dev at biomoby.org >>http://www.biomoby.org/mailman/listinfo/moby-dev >> >> ___________________________________________________________________________ Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger Téléchargez cette version sur http://fr.messenger.yahoo.com From senger at ebi.ac.uk Tue Jul 19 11:26:11 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 16:26:11 +0100 (BST) Subject: [MOBY-dev] rfcf 8. Predicates In-Reply-To: <42dd1264.5c49d513.43d9.3fa7@mx.gmail.com> Message-ID: > a. should we add the following predicates to the metadata of > service instances? > Well, we should. But predicates should be unique, and I thought that they will be labeled as LSIDs... M. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 16:22:56 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 21:22:56 +0100 (BST) Subject: [MOBY-dev] jMoby does not compile!!!!! In-Reply-To: <42dd126b.46e6258d.43d9.3fb7@mx.gmail.com> Message-ID: Eddie, Four hours ago, you committed changes that do not compile! (e.g. in client/rdf/builder). You are the biggest sinner one can be. Please, please, correct it fast... Thanks, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 16:28:50 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 21:28:50 +0100 (BST) Subject: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: <42dd126b.46e6258d.43d9.3fb7@mx.gmail.com> Message-ID: > a. do you think that there should be an api call, getUpdate > that tells the agent to check for modifications to the > service instance described by the metadata in the RDF > document at the signatureURL? > Well, it depends how LSID metadata are collected and used, and where they are stored. The bottom line what I wish is this: When a service provider updates her/his metadata about his/her services, I wish that this update is visible asap to the others. Therefore, if it is the RDF agent you collect metadata and stores in a registry, then I wish to have an option to tell him "come and get it now". If, however, this is not the task of the RDF agent, but if there is yet another way how to collect data from my site (presumably done by the LSID resolution service) then I do not need such method in the API (because the LSID resolver will come to me when somebody asks for my metadata). Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Tue Jul 19 18:41:40 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue, 19 Jul 2005 15:41:40 -0700 Subject: [moby] RE: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: References: Message-ID: <1121812900.22947.57.camel@bioinfo.icapture.ubc.ca> I think the intent is as follows: The agent will collect the "core" data for the registry on a regular basis. You can chose to explicitly call the agent to you if you like, but it will only record the information that MOBY Central currently records. During service discovery, you will receive the LSID representing a particular service as part of the response message from MOBY Central (this is already in the API 0.85, but has not been coded yet). MOBY Central will act as the Authority for that LSID, and will give you a WSDL document that (at a minimum) directs you to GET the SignatureURL of that service as the getMetaData resolution function - the SignatureURL should hold an RDF document describing that service FULLY; i.e. not only what MOBY Central needs to know, but also whatever else the service provider wants to tell you about the service. Some of the RDF predicates in that document will be part of the official MOBY API, while others may be defined specifically by that service provider. Of course, we will try to make as many of the useful ones part of the MOBY API :-) Does that clarify how I am thinking about the process? Are there better ways to accomplish this that I haven't thought of? This is the perfect time to speak-up as we are in the process of re-coding the API right now... Cheers! M On Tue, 2005-07-19 at 21:28 +0100, Martin Senger wrote: > > a. do you think that there should be an api call, getUpdate > > that tells the agent to check for modifications to the > > service instance described by the metadata in the RDF > > document at the signatureURL? > > > Well, it depends how LSID metadata are collected and used, and where > they are stored. The bottom line what I wish is this: > When a service provider updates her/his metadata about his/her > services, I wish that this update is visible asap to the others. > Therefore, if it is the RDF agent you collect metadata and stores in a > registry, then I wish to have an option to tell him "come and get it now". > If, however, this is not the task of the RDF agent, but if there is yet > another way how to collect data from my site (presumably done by the LSID > resolution service) then I do not need such method in the API (because the > LSID resolver will come to me when somebody asks for my metadata). > > Martin > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From senger at ebi.ac.uk Tue Jul 19 19:50:57 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 20 Jul 2005 00:50:57 +0100 (BST) Subject: [moby] RE: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: <1121812900.22947.57.camel@bioinfo.icapture.ubc.ca> Message-ID: > The agent will collect the "core" data for the registry on a regular > basis. You can chose to explicitly call the agent to you if you like, > but it will only record the information that MOBY Central currently > records. > Okay. So far, so clear. How can I explicitly call the agent to me? > During service discovery, you will receive the LSID representing a > particular service as part of the response message from MOBY Central > (this is already in the API 0.85, but has not been coded yet). > This is done by the XML attribute 'lsid', correct? > MOBY Central will act as the Authority for that LSID, and will give > you a WSDL document that (at a minimum) directs you to GET the > SignatureURL of that service as the getMetaData resolution function > While I understand this I doubt that all service providers are aware that actually their SignatureURL is supposed to play a role of a LSID metadata resolution service. Concretely, a service that is defined by the LSIDMetadataHTTPBindingDirect (as defined in the LSID spec). Which means, for example, that the returned HTTP response should have Content-Type "x-application/rdf+xml" and Expires HTTP headers. Also, strictly speaking, an LSID should resolve to the metadata describing this LSID, but in the case of using SignatureURL there is no guarantee that a service provider does not put into the same document metadata about more services. So, we need to have this very clearly documented and explained, so the clients cannot be surprised when they get more than they asked for. I think that it would be also good if you can show an example of the WSDL with that binding, and instructions how clients should use such WSDL (which part they should take - my understanding is that the SignatureURL will be in the tag tag). > time to speak-up as we are in the process of re-coding the API right > now... > One more thing (not about API but about implementation) - and I think that Eddie mentioned it somewhere else, as well): In CSHL, we spoke about a special moby service that can be used for registering (and for retrieving) endpoints of service metadata created by the third-parties. Such service does not exist yet (nor its API) but if it exists (and if the registry is configured to use it), the registry should use it to put more metadata locations into returned WSDL. Please keep it in mind when you are implementing this part, so it can be easier updated when such service exists. Cheers, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Tue Jul 19 20:07:15 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue, 19 Jul 2005 17:07:15 -0700 Subject: [unclassified] Re: [moby] RE: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: References: Message-ID: <1121818035.22947.85.camel@bioinfo.icapture.ubc.ca> On Wed, 2005-07-20 at 00:50 +0100, Martin Senger wrote: > Okay. So far, so clear. How can I explicitly call the agent to me? Yet to be decided... we'll make it an API call of some sort. We're still trying to get the agent to work so I need to focus on that first :-) > This is done by the XML attribute 'lsid', correct? correct. > Which means, for example, that the returned HTTP response should have > Content-Type "x-application/rdf+xml" and Expires HTTP headers. > Also, strictly speaking, an LSID should resolve to the metadata > describing this LSID, but in the case of using SignatureURL there is no > guarantee that a service provider does not put into the same document > metadata about more services. This is a very good point, and I hadn't fully considered the consequences of this v.v. having more than one service definition in the metadata document. Let's tease out what behaviour we want from this part of the API before we make any further decisions. It seems to me that the objective is for the service provider to be able to say things about their service that are not recorded in the Registry, but are still available to the consumer through a predictable API. We accomplish this through both avenues (having only one SignatureRDF at the URL or by having multiple concatenated SignatureRDF's at a single URL), but you are 100% correct that it might be a more confusing document to parse if the client is not expecting it. Are there other reasons to chose one over the other? I guess having a proliferation of SignatureRDF's on the server-side is an unpleasant thought also (though from a service providers perspective, it might be easier to manage, since RDF isn't particularly human-readable anyway...) ?? other thoughts ?? > I think that it would be also good if you can show an example of the > WSDL with that binding, and instructions how clients should use such WSDL > (which part they should take - my understanding is that the SignatureURL > will be in the tag tag). That was what I had in mind, yes. We'll certainly document everything well once we have it set up the way we want it. > One more thing (not about API but about implementation) - and I think > that Eddie mentioned it somewhere else, as well): In CSHL, we spoke about > a special moby service that can be used for registering (and for > retrieving) endpoints of service metadata created by the third-parties. > Such service does not exist yet (nor its API) but if it exists (and if the > registry is configured to use it), the registry should use it to put more > metadata locations into returned WSDL. Please keep it in mind when you are > implementing this part, so it can be easier updated when such service > exists. OK Cheers! M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From rebecca.ernst at gsf.de Wed Jul 20 03:24:38 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Wed, 20 Jul 2005 09:24:38 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: References: Message-ID: <42DDFC36.4090409@gsf.de> Hi Martin! this vision has not been changed yet, mainly because noone seems to understand Heikos, Dirks and my problems with the existing solution. If you re-read the mails you can try to understand if you get the point I can't make myself any clearer than that. Basically I think it doesn't make sense to register an output as being a collection if the results don't form an entity! In our case e.g. the service 'getAGILocusCodes'. You give it a keyword and it returns all AGI Codes somehow related to this keyword. These AGI codes don't form an entity! Just because the keyword appears in their annotation they are given back - but with no relation to each other. For us it would be only logical if we would give many simple responses. This is not possible as currently if the number of results is not known we have to give back a collection. cheers, Rebecca Martin Senger wrote: >Hi all, > Catching up my email piles I wonder if someone can summarize if the >discussion about collections in this thread brought any (planned) changes >in the API (I am not talking now about how it is, or should be, >implemented in Taverna, that's, imho, an another story). > My understanding is that (talking about one mobyData object): > a) Any Moby service can have more outputs. If so, all of them must be >registered. The number of such outputs must be fixed. > b) Any of these outputs can be of type either Simple or Collection. > c) If it is a Collection, this output can have one or more Simples in >that Collection. Such Simples (and their number) are not individually >registered. > > Has this vision been changed? > > Thanks, > Martin > > > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From senger at ebi.ac.uk Wed Jul 20 05:24:48 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 20 Jul 2005 10:24:48 +0100 (BST) Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <42DDFC36.4090409@gsf.de> Message-ID: Rebecca, Many thanks for the explanation. I think this is the critical part: > Basically I think it doesn't make sense to register an output as being a > collection if the results don't form an entity! > I assume that by entity you mean 'set of pieces that belong semantically together'. I think that the current concept of Simples and Collections gives you full freedom to model your output in any way you consider proper. For example, if you do not feel comfortable to put all AGI Codes in one collection because they represent semantically different things you can split them by types (or whatever semantics you choose) into several collections, or you can actually deploy several services, each of them returning a different set (collection) of AGI Codes given the same input keyword. But you know all that. I believe that we need to keep the concept of Collections open as are the general data types in other languages (e.g. hashtables). I am sorry to bother you with these obvious comments. Cheers, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From d.haase at gsf.de Wed Jul 20 08:24:08 2005 From: d.haase at gsf.de (Dirk Haase) Date: Wed, 20 Jul 2005 14:24:08 +0200 Subject: [MOBY-dev] Moby Collections (was: Problems with Biomoby servicesin Taverna 1.2) In-Reply-To: References: Message-ID: <200507201424.08853.d.haase@gsf.de> On Wednesday 20 July 2005 11:24, Martin Senger wrote: > Rebecca, > Many thanks for the explanation. > > I think this is the critical part: > > Basically I think it doesn't make sense to register an output as being a > > collection if the results don't form an entity! > > I assume that by entity you mean 'set of pieces that belong > semantically together'. Exactly. > I think that the current concept of Simples and Collections gives you > full freedom to model your output in any way you consider proper. This is actually _not_ the case and that is why we brought up the issue. There is no proper way to output things which really make up a 'set of pieces that belong semantically together'. For example there is no way to output the results of a clustering service (a cluster is a set of sequences, the service result is a set of clusters), because a collection of collections is not allowed afaik. Of course one can create a 'cluster object' but this creates rather artificial hurdles in workflow creation. Suppose that after the clustering, each cluster should be fed into a multiple alignment service. This would presumably take a collection of sequences as input. So one would have to decompose the cluster object first before it can be further processed. I think we all agree that on the input side a collection only makes sense if it is meant as one semantical entity (example from the API docs: several sequences make up the BLAST database that is to be queried). But not so on the output side. So we have an inherent unequality which complicates workflow building. > For > example, if you do not feel comfortable to put all AGI Codes in one > collection because they represent semantically different things you can > split them by types (or whatever semantics you choose) into several > collections, The claim is they are not related... The only relation is that they pop up for the same input but that is obvious for this kind of services and does not need to be emphasized by stuffing them into a collection. > I believe that we need to keep the concept of Collections open as are > the general data types in other languages (e.g. hashtables). That is a good point, but I'm not sure if it is appropriate to compare the concept of Simples/Collections to general data types in programming languages. > I am sorry to > bother you with these obvious comments. No, they are very welcome indeed! Servus, dirk From senger at ebi.ac.uk Wed Jul 20 08:44:10 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 20 Jul 2005 13:44:10 +0100 (BST) Subject: [MOBY-dev] Moby Collections (was: Problems with Biomoby servicesin Taverna 1.2) In-Reply-To: <200507201424.08853.d.haase@gsf.de> Message-ID: > because a collection of collections is not allowed afaik. > You are right - I have forgotten that we cannot have collections of collections. So we cannot model all possible scenarios. But: BioMoby (afaik) tries to be simple. To allow completely complex data types (build on collections of collectios) would mean to assume that all clients will be able to understand such complex data types. And it would be hard. We would end up anyway with many clients that would simply ignore such complex types (or they break on them). So why not to go with the current API (I may even argue if it is really neccessary to allow a collection having different types of Simples, but that's another story) and to try to solve the problem of semantically relevant clusters on the service level - simply to create more specific service instances. BioMoby was/is about a simple transformation. If the transformation is not simple let's try to divide it into more transformations that can be simple. Cheers, Martin PS. > but I'm not sure if it is appropriate to compare the concept of > Simples/Collections to general data types in programming languages. > Why not? Except that biomoby collections are less general (by not allowing coles of coles). M. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From gordonp at ucalgary.ca Wed Jul 20 10:22:15 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed, 20 Jul 2005 08:22:15 -0600 Subject: [MOBY-dev] Moby Collections (was: Problems with Biomoby servicesin Taverna 1.2) In-Reply-To: References: Message-ID: <42DE5E17.8080302@ucalgary.ca> I think Martin has hit the nail on the head here. One of the primary advantages of MOBY is that you can't (easily) build arbitrarily syntactically complex output. This ensures that you can chain together services more easily. You should be building new objects in the ontology to represent the cluster concept, not squeezing that cluster logic into the syntax of the collection. Otherwise clients will do very wierd things with your implicitly encoded concepts. You always have to look at how your data could be used by someone else who wasn't expecting it... >>because a collection of collections is not allowed afaik. >> >> >> > You are right - I have forgotten that we cannot have collections of >collections. So we cannot model all possible scenarios. > > From jmfernandez at cnb.uam.es Wed Jul 20 10:53:44 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Wed, 20 Jul 2005 16:53:44 +0200 Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: <42dd1266.6b7805fa.43d9.3fae@mx.gmail.com> References: <42dd1266.6b7805fa.43d9.3fae@mx.gmail.com> Message-ID: <42DE6578.3080008@cnb.uam.es> > > 12. Services > > a. should article names for service instance input and > output datatypes be mandatory? > Well, in my opinion, I think they should be mandatory when there is more than one input (or output). And when there is only one and the article name it is not declared, then it should be a default value for the article name. > b. why does a service that advertise an output datatype > called 'foo' return 'bar' instead? > If datatype 'bar' inherits from 'foo', then it could be allowed, but I don't have a good example for that. But we must take into account that most times it happens it is a bug. > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From jmfernandez at cnb.uam.es Wed Jul 20 10:46:11 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Wed, 20 Jul 2005 16:46:11 +0200 Subject: [MOBY-dev] rfcf 2. How do I know what predicates are available inthe RDF In-Reply-To: <42dd122e.54b1a5ca.63bd.4a5c@mx.gmail.com> References: <42dd122e.54b1a5ca.63bd.4a5c@mx.gmail.com> Message-ID: <42DE63B3.503@cnb.uam.es> Hi everybody, I think a mixure of b) and c) could be feasible. The RDF Schema would be generated from the registered new predicates. Edward Kawas wrote: > Since there was a request to keep these messages in email > and away from twiki, I have added each heading to an email. > > 2. How do I know what predicates are available in the RDF > documents that describe the Moby ontologies? > > a. should we provide a list of predicates and their intended > meanings (on a wiki, etc)? > > b. should we provide an RDF schema defining the predicates > (* probably the way to go) > > c. Should we allow the registration of new predicates, if > so, should we make this an api call, etc. > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From jmfernandez at cnb.uam.es Wed Jul 20 11:03:56 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Wed, 20 Jul 2005 17:03:56 +0200 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42dd124c.7fdce064.76fe.ffff9b01@mx.gmail.com> References: <42dd124c.7fdce064.76fe.ffff9b01@mx.gmail.com> Message-ID: <42DE67DC.5060007@cnb.uam.es> > 5. LSIDs and Service Instances > > a..If a service instance is registered by 2 different > authoritys, should they have the same LSID? > Only if we can assure they are the SAME service (replicated services), which would require external help. > b. if the service instance is registered in 2 registries, > should the LSIDs be the same? > No, they shouldn't, unless they are registered by the same authority. > c. will modified service instances have versioning > information? > I think it should, because versioning information could help in the use of services from different registries, or the integration of those registries. -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From d.haase at gsf.de Wed Jul 20 11:12:06 2005 From: d.haase at gsf.de (Dirk Haase) Date: Wed, 20 Jul 2005 17:12:06 +0200 Subject: [MOBY-dev] Moby Collections In-Reply-To: <42DE5E17.8080302@ucalgary.ca> References: <42DE5E17.8080302@ucalgary.ca> Message-ID: <200507201712.06848.d.haase@gsf.de> On Wednesday 20 July 2005 16:22, Paul Gordon wrote: > I think Martin has hit the nail on the head here. One of the primary > advantages of MOBY is that you can't (easily) build arbitrarily > syntactically complex output. Correct. That's why we did not ask for nested collections. We suggest to allow for multiple Simple output as default - which I think is very appropriate because especially query services will usually return more than one result. This way we would save the collection concept for semantically related entities. > This ensures that you can chain together > services more easily. That is exactly our intention... > You should be building new objects in the > ontology to represent the cluster concept, not squeezing that cluster > logic into the syntax of the collection. Otherwise clients will do very > wierd things with your implicitly encoded concepts. What bears more implicitness: 1) passing totally unlrelated things together just because of a coincidental preceding step in the workflow or 2) putting things together because they build a semantic entity which is intended to be processed as a whole? > You always have to > look at how your data could be used by someone else who wasn't expecting > it... Well, expectations are ruled by the API - and that is what we are working on, aren't we? ;-) From senger at ebi.ac.uk Wed Jul 20 11:32:37 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 20 Jul 2005 16:32:37 +0100 (BST) Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: <42DE6578.3080008@cnb.uam.es> Message-ID: > > a. should article names for service instance input and > > output datatypes be mandatory? > > Well, in my opinion, I think they should be mandatory when there is more > than one input (or output). > This question hides actually two situations: a) article names for inputs (or outputs), and b) article names for elements attached by HAS[A] relationships. For b) situation I would say hat the article name should be mandatory, because one never knows who is going to extend his object and add new attributes. Having it mandatory will make life (and verification) easier. Also, it will help to understand what individual HASA attributes mean if they have an article name. For a) situation it is not that crucial, I would say. But I know that INB people will come with a suggestion how to handle errors in a new way, and for their solution the article names are important (if I remember it correctly). And a similar situation may appear again in the future. And because having article names mandatory is not hard to do (and if your current services do not do it, it will not break them, only some verifiers may complain) I am inclined to say make them mandatory. Regards, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From gordonp at ucalgary.ca Wed Jul 20 12:25:42 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed, 20 Jul 2005 10:25:42 -0600 Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: References: Message-ID: <42DE7B06.8050102@ucalgary.ca> I absolutely agree on (b). Anonymous data members is crazy. For (a) I would say that articleNnames should be manditory if there is more than one parameter. You could make a lot fancier rules, but you start dealing with a lot of parameter ambiguity issues. It's easier to just require them if there is any question. This would be a very minor change, since I don't think that there are many services that take more than one parameter, are there? All of the simple one-parameter services would still be valid. >>>a. should article names for service instance input and >>>output datatypes be mandatory? >>> >>> >>Well, in my opinion, I think they should be mandatory when there is more >>than one input (or output). >> >> >> > This question hides actually two situations: a) article names for >inputs (or outputs), and b) article names for elements attached by HAS[A] >relationships. > For b) situation I would say hat the article name should be mandatory, >because one never knows who is going to extend his object and add new >attributes. Having it mandatory will make life (and verification) easier. >Also, it will help to understand what individual HASA attributes mean if >they have an article name. > For a) situation it is not that crucial, I would say. But I know that >INB people will come with a suggestion how to handle errors in a new way, >and for their solution the article names are important (if I remember it >correctly). And a similar situation may appear again in the future. And >because having article names mandatory is not hard to do (and if your >current services do not do it, it will not break them, only some verifiers >may complain) I am inclined to say make them mandatory. > > Regards, > Martin > > > From schoof at mpiz-koeln.mpg.de Thu Jul 21 09:42:32 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Thu, 21 Jul 2005 15:42:32 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: References: Message-ID: <776ef61c9a8901624450044125ab9b81@mpiz-koeln.mpg.de> Hi all again, I'll respond to Martin's query here: My understanding is that (talking about one mobyData object): a) Any Moby service can have more outputs. If so, all of them must be registered. The number of such outputs must be fixed. b) Any of these outputs can be of type either Simple or Collection. c) If it is a Collection, this output can have one or more Simples in that Collection. Such Simples (and their number) are not individually registered. I think the issue is point a, the number of such outputs must be fixed. Or, as the API says "each of these articles will appear EXACTLY ONCE in the output from the service". I request to change this to "each of these articles will appear AT LEAST ONCE in the output from the service". Why is this necessary? Currently, services that return potentially many results are registered as outputting a Collection. But within workflows, the iteration strategy or how often the next service is called should distinguish between an output made up of many independent items versus one or many groups of connected items. In the first case, the following service should be called once for each item, in the second case, only once for each group. See below for full example. What I am suggesting is to separate the cardinality from the Simple/Collection issue; meaning, that a service that performs e.g. a database lookup and returns 1 or many outputs (or none, but in the Moby world this means it returns 1 empty output) will be registered as returning Simple, not Collection, if the outputs are otherwise semantically unrelated (aside from the fact that they arose from the same query). And reserve the Collection article for grouping of outputs that need to be seen as a single entity. E.g. a service that outputs ortholog pairs given as input a pair of organisms: Each ortholog pair could be represented as a pair of GenericSequence objects in a Collection, with the service outputing 1 or many of these Collections. The same service, given as input three organisms, could still output many Collections, then containing three sequences each. This prevents ugly explosion of specialized BioMoby objects like "MultipleSequences", "HAS GenericSequence(s)"... that would otherwise be needed to wrap this. For service discovery, this should not make a difference. Services would still be required to return every Class of object that they register, as you state in a: all output object *Classes* must be registered, and the number of *Classes* fixed. I.e., a service registered as returning GenericSequence and AnnotatedJpeg objects must always return at least one GenericSequence plus at least one AnnotatedJpeg. I can't recall a service that actually does that... I can only think of this being meaningful if the AnnotatedJpeg is semantically connected to a specific GenericSequence, and in that case both should be connected through putting them in a Collection imho. For inputs, I'm not so worried; if multiple inputs are intended to go into a single service call, they will probably be connected and could go into a Collection. Example above: Input is a Collection of Organisms. Basically, I see that as the only way to register services that require AT LEAST two equal inputs. No problem with b, but with c: To my understanding, when registering a Collection, also the classes of objects in Simples that it contains must be registered. Otherwise, no discovery. However, see API, " A collection may contain zero or more Objects of each of the Classes", not all these classes must actually be included. So far, I do not see the need to distinguish between services that return EXACTLY ONE output and those that return one or more. Taverna seems to make that distinction, and bases iteration strategies on that, but I would want to do that dynamically, and it may be that that's what Taverna does. I'd by default assume that there will be multiple outputs and iterate over them, but if the workflow designer so wishes make it possible to (using e.g. a local processor) combine all the outputs into a Collection that can be used as the single input to a following service. This distinction is necessary: Use case examples would be a service returning a number of sequences, that in one scenario (iterate) should each be run through a BLAST service individually and in a different scenario (bag or Collection) should be all together input into a single call of a multiple alignment service. The current problem arises because Taverna now, in what is for me the semantically correct interpretation, if it receives a Collection as output from a service, it inputs that into a single execution of the following service if that service consumes Collection. In version 1.1 and before, Collections were decomposed by Taverna and iterated over. For the workflows being used, that was the wanted behavior, as e.g. keyword queries returning a set of sequences should be linked to services that act on each individual sequence. The mistake is imho on the BioMoby side, where we use Collection to wrap multiple outputs even if these are individual results that should be processed individually, and then, in order to be able to pipeline, register services that actually act on single inputs as consuming Collections. Consider Tom Oinn's comment 080705: 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. This is the same logic that we'd need to implement into BioMoby to allow meaningful links between Collection producing services and Simple consuming services! And NOT register services as consuming or producing Collections if all they do is mimic this behaviour internally by iterating over the Collection items. Many words and I'm not sure this is making anything any clearer. But I try ;-) Best, Heiko On 18. Jul 2005, at 14:39 Uhr, Martin Senger wrote: Hi all, Catching up my email piles I wonder if someone can summarize if the discussion about collections in this thread brought any (planned) changes in the API (I am not talking now about how it is, or should be, implemented in Taverna, that's, imho, an another story). My understanding is that (talking about one mobyData object): a) Any Moby service can have more outputs. If so, all of them must be registered. The number of such outputs must be fixed. b) Any of these outputs can be of type either Simple or Collection. c) If it is a Collection, this output can have one or more Simples in that Collection. Such Simples (and their number) are not individually registered. Has this vision been changed? Thanks, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From edward.kawas at gmail.com Fri Jul 22 11:08:09 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Fri, 22 Jul 2005 08:08:09 -0700 Subject: [MOBY-dev] FW: theseu - upload the first beta and unstable version Message-ID: <42e10bdc.61a5c3e4.09f0.062d@mx.gmail.com> -----Original Message----- From: Roman Roset [mailto:roset at ac.upc.edu] Sent: Friday, July 22, 2005 9:02 AM To: inb-tecnico at everest.cnb.uam.es; Eddie Kawas; Martin Senger Subject: theseu - upload the first beta and unstable version Hi all, The INB Theseu project is a framework for biomoby developers. The requirements of Theseu project are driven to develop BioMoby Services and to administrate BioMoby Repositories (locally or remotally). Future extensions of this project could include features such as RSS systems to get the biomoby news, local agents to perform some tasks directly with the RDF,etc... in short, the needs of biomoby and INB developers comunity. DOWNLOAD ======== 0. Documentation At the moment there is no user documentation. Sorry. Any voluntaries to help me writing the online documentation :) ? 1. Requirements These are the prerequisites to install the Theseu Project. They must be downloaded (EMF via eclipse site or manually) and installed before Theseu: Eclipse build eclipse-SDK-3.0.2: http://download.eclipse.org/eclipse/downloads/index.php EMF runtime 2.0.2 (emf-sdo-runtime-2.0.2): http://download.eclipse.org/tools/emf/scripts/downloads-view er.php?s=2.0.2/R200503151315 2. Theseu Project Runtime I have uploaded the first beta and unstable version of theseu at this site: Remote Site name: Theseu Project URL: http://inb.lsi.upc.edu/theseu/updates You can download easily via help -> software updates -> find & install -> "search for new features..." --> "new remote site". This beta distribution has 3 features grouped in two diferents topics: * Theseu Project |-> Theseu Project plugins |-> other (this is jmoby and others) Select all that you can checked :) and follow the afirmative buttons. 3. Mini example to start playing Once Theseu is installed, follow this steps: 1. click the Navigator window and press ctrl+N to open a new project wizard. 1.a. Select BioMoby Services Wizard --> new Moby Project 2. click on the label of the new project into de navigator window and press ctrl+N again to create a new repository: 2.a Select BioMoby Services Wizard --> new Moby Repository: 2.b. Select the server or push the preferences button to add new biomoby services. 2.c Select the authority or push the preferences button to add new authors. 3.d push finish button. 3. Now you can view in the editor window your empty local repository. The next step is to populate the ouline view with the element of the repository. So, click the label of a moby element into the editor (for instance "services") and click via the GeneralEditor menu or a popup menu (clicking over the "services" with the mouse right button) the "load..." option. 3.1 wait and don't despair... this is only necessary the first time. 4. Now you could open some views before to start with the edition: 4.1 Window --> show view --> Problems 4.2 Window --> show view --> Properties Remember that when you validate an element, the information to navigate through the messages is saved into the Problems view. 5. Register elements: it's only possible in this "demo" to register services. ABOUT THIS VERSION ================== I have uploaded this version with the aim of show the actual state of this project. I repeat that is only a demo version and it's very unstable, however you can have an idea of what Theseu can do and what will be able to do. Please send me your opinions about this. All your help is welcome. Thanks. Roman -- Roman Roset Mayals Technical Coordinator Instituto Nacional de Bioinformatica (www.inab.org) Barcelona Supercomputing Center Node (www.bsc.org.es) c/. Jordi Girona 1-3 Modul C6-E201 Tel. : 934 011 650 E-08034 Barcelona Fax : 934 017 014 Catalunya (Spain) e-mail: rroset at lsi.upc.edu Note: I'm on vacation until August 21th From roset at ac.upc.edu Fri Jul 22 00:23:49 2005 From: roset at ac.upc.edu (Roman Roset) Date: Fri, 22 Jul 2005 05:23:49 +0100 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances Message-ID: <42E074D5.2010205@ac.upc.es> >> 5. LSIDs and Service Instances >> >> a..If a service instance is registered by 2 different >> authoritys, should they have the same LSID? >> > Only if we can assure they are the SAME service (replicated services), > which would require external help. If I am not wrong, among other things, the lsid of a service would have to serve to obtain the RDF and I think that it should be interesting to have one-to-one relationship between RDF and service (or instance of a replicate service). For instance, if the RDF of a replicated service needs to save information about a concrete instance (such as the service is temporally unavailable, cpu's, os, etc...) These values could be different for two "same services". (but maybe I don't have clear what are same services - My understanding is that, two services are the same if given the same input then the output of both is exactly the same). However, I have some doubts about the way to specify the LSID of the services in biomoby. There are an example in the biomoby metadata-resolver: urn:lsid:biomoby.org:serviceinstance:www.illuminae.com,getSHound3DNeighboursFromGi (The LSID syntax is urn:lsid:authority:namespace:identifier:revision) Why is the authority "biomoby.org" and this one is not www.illuminae.com? Why is the service name separated of the lsid? If we use biomoby as authority but we wrote the services as identifiers then an approach to denote more of one instance (or replicated service) into the same lsid could be using the revision field. For instance: urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighboursFromGi:1 urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighboursFromGi:2 But sincerely, my opinion is that it's better to use an lsid like: urn:lsid:www.illuminae.com:biomoby.org:getSHound3DNeighboursFromGi urn:lsid:www.other.com:biomoby.org:getSHound3DNeighboursFromGi With this kind of LSID the MetaData-resolver could know that if the namespace is biomoby then the element to resolve is a service, and we should have one LSID per service. >> b. if the service instance is registered in 2 registries, >> should the LSIDs be the same? >> > No, they shouldn't, unless they are registered by the same authority. Is the RDF of this sevice shared by both registries? Is the response is yes then I feel that the LSID should be the same. (and the opposite, if the response is not the LSIDs should be different.) >> c. will modified service instances have versioning >> information? >> > I think it should, because versioning information could help in the use > of services from different registries, or the integration of those > registries. From my point of view, for one lsid of a service (independently of what thing is identifying exactly) there are two ways to understand the versioning information. I think that the version has to "explain" (useful) changes in the meta-information. But the service metainformation (the RDF) can change at two levels: the values of the predicates and the schema. I think that versions should be useful (at least for the agent) in the second case. -- Roman Roset Mayals Technical Coordinator Instituto Nacional de Bioinformatica (www.inab.org) Barcelona Supercomputing Center Node (www.bsc.org.es) c/. Jordi Girona 1-3 Modul C6-E201 Tel. : 934 011 650 E-08034 Barcelona Fax : 934 017 014 Catalunya (Spain) e-mail: rroset at lsi.upc.edu Note: I'm on vacation until August 21th From edward.kawas at gmail.com Fri Jul 22 13:12:54 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Fri, 22 Jul 2005 10:12:54 -0700 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42E074D5.2010205@ac.upc.es> Message-ID: <42e1291a.51939a07.08bd.ffff8d38@mx.gmail.com> > However, I have some doubts about the way to specify > the LSID of the > services in biomoby. There are an example in the biomoby > metadata-resolver: > > urn:lsid:biomoby.org:serviceinstance:www.illuminae.com,get > SHound3DNeighboursFromGi > > (The LSID syntax is > urn:lsid:authority:namespace:identifier:revision) > > Why is the authority "biomoby.org" and this one is not > www.illuminae.com? The authority is BioMoby.org because it's the issuing authority. In other words, it is the place where we can go and ask for information regarding a particular service and retrieve metadata about it. www.illuminae.com is the authority that registered the service getSHound3DNeighboursFromGi into the registry hosted by BioMoby.org. Does this make sense? > Why is the service name separated of the lsid? The name was separated so that in the event that there were 2 services registered with the same service name we wouldn't run into conflicts. Also, a service should be uniquely identified by the authority that registered it and its name. > If we use biomoby as authority but we wrote the services > as identifiers > then an approach to denote more of one instance (or > replicated service) > into the same lsid could be using the revision field. > For instance: > > urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours > FromGi:1 > urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours > FromGi:2 > The revision id is meant to convey to someone that the underlying object 'pointed to' by the lsid has changed. So for instance, when the agent is running, and it notices that the service instance has changed a little (new description added or a new input or output, etc), then the revision id would have to be used. > urn:lsid:www.illuminae.com:biomoby.org:getSHound3DNeighbou > rsFromGi > urn:lsid:www.other.com:biomoby.org:getSHound3DNeighboursFr > omGi > > With this kind of LSID the MetaData-resolver could know > that if the namespace is biomoby then the element to > resolve is a > service, and we should have one LSID per service. One of the problems with this approach is that how do you go about resolving other Moby objects like service types, datatypes, namespaces,etc? I believe that your proposal also doesn't follow the correct use of the LSID protocol. From markw at illuminae.com Fri Jul 22 13:26:06 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 22 Jul 2005 10:26:06 -0700 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42e1291a.51939a07.08bd.ffff8d38@mx.gmail.com> References: <42e1291a.51939a07.08bd.ffff8d38@mx.gmail.com> Message-ID: <42E12C2E.8010406@illuminae.com> Edward Kawas wrote: >> However, I have some doubts about the way to specify >>the LSID of the >>services in biomoby. There are an example in the biomoby >>metadata-resolver: >> >>urn:lsid:biomoby.org:serviceinstance:www.illuminae.com,get >>SHound3DNeighboursFromGi >> >>(The LSID syntax is >>urn:lsid:authority:namespace:identifier:revision) >> >>Why is the authority "biomoby.org" and this one is not >>www.illuminae.com? >> >> >The authority is BioMoby.org because it's the issuing >authority. In other words, it is the place where we can go >and ask for information regarding a particular service and >retrieve metadata about it. www.illuminae.com is the >authority that registered the service >getSHound3DNeighboursFromGi into the registry hosted by >BioMoby.org. Does this make sense? > > Just to expand this a little bit more - biomoby.org will NOT be the resolver for that LSID, but it will be the authority server for that LSID. >>Why is the service name separated of the lsid? >> >> >The name was separated so that in the event that there were >2 services registered with the same service name we wouldn't >run into conflicts. Also, a service should be uniquely >identified by the authority that registered it and its name. > > In the early days when we were first considering LSID's in MOBY, there wasn't a widely agreed-upon standard for "wrapping" someone elses identifier. It seems now that there is a "standard" (I don't know if this is a recommendation from teh LSID community or not, but it seems that the core LSID developers do it themselves, so it must be supported). We might consider changing the LSID structure for MOBY Service Instances to e.g.: urn:lsid:biomoby.org.www.illuminae.com:serviceinstance:getGenBankff It makes no difference at all at the end of the day, since LSID's are opaque identifiers :-) >>If we use biomoby as authority but we wrote the services >>as identifiers >>then an approach to denote more of one instance (or >>replicated service) >> into the same lsid could be using the revision field. >>For instance: >> >>urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours >>FromGi:1 >>urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours >>FromGi:2 >> >> >> >The revision id is meant to convey to someone that the >underlying object 'pointed to' by the lsid has changed. So >for instance, when the agent is running, and it notices that >the service instance has changed a little (new description >added or a new input or output, etc), then the revision id >would have to be used. > > Yup... that suggestion (to use the version position to denote replication) would be very dangerous! However, it is an interesting problem, since I don't think the LSID communuty would be very comfortable with a single LSID referring to multiple resources... even if they are "identical". Martin, do you have an opinion about this? M From roset at ac.upc.edu Sat Jul 23 15:56:08 2005 From: roset at ac.upc.edu (Roman Roset) Date: Sat, 23 Jul 2005 20:56:08 +0100 Subject: [MOBY-dev] FW: theseu - upload the first beta and unstable version Message-ID: <42E2A0D8.4070109@ac.upc.es> Hello, thank you very much eddie, to forward my email in this list :). I've uploaded a ppt preview of INB-Theseu: http://inb.lsi.upc.es/theseu/docs/ppt/TheseuPreview.ppt There are bugs on this demo, but my intention is to know the general opinion of moby-developers in order to driven the design of future functionalities. So, all your comments, questions or suggestions are most welcome. INB Theseu people: Author: Roman Roset (roset at ac.upc.edu) Coordinator: David G. Pisano (dgpisano at cnb.uam.es) Supervised by: Phd. Xavier Messeguer and Phd. Oswaldo Trelles Theseu is property of INB - Instituto Nacional de Bioinform?tica (www.inab.org) Thanks. Note: I'm on vacation until August 21th -- Roman Roset Mayals Technical Coordinator Instituto Nacional de Bioinformatica (www.inab.org) Barcelona Supercomputing Center Node (www.bsc.org.es) c/. Jordi Girona 1-3 Modul C6-E201 Tel. : 934 011 650 E-08034 Barcelona Fax : 934 017 014 Catalunya (Spain) e-mail: rroset at lsi.upc.edu From markw at illuminae.com Mon Jul 25 11:37:11 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 25 Jul 2005 08:37:11 -0700 Subject: [moby] [MOBY-dev] FW: theseu - upload the first beta and unstable version In-Reply-To: <42E2A0D8.4070109@ac.upc.es> References: <42E2A0D8.4070109@ac.upc.es> Message-ID: <1122305831.29736.58.camel@bioinfo.icapture.ubc.ca> Hokey Dinah! That's AMAZING!!! M On Sat, 2005-07-23 at 20:56 +0100, Roman Roset wrote: > Hello, > > thank you very much eddie, to forward my email in this list :). > > I've uploaded a ppt preview of INB-Theseu: > > http://inb.lsi.upc.es/theseu/docs/ppt/TheseuPreview.ppt > > > There are bugs on this demo, but my intention is to know the general > opinion of moby-developers in order to driven the design of future > functionalities. So, all your comments, questions or suggestions are > most welcome. > > INB Theseu people: > > Author: Roman Roset (roset at ac.upc.edu) > Coordinator: David G. Pisano (dgpisano at cnb.uam.es) > Supervised by: Phd. Xavier Messeguer and Phd. Oswaldo Trelles > > Theseu is property of INB - Instituto Nacional de Bioinform?tica > (www.inab.org) > > Thanks. > > Note: I'm on vacation until August 21th > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From markw at illuminae.com Mon Jul 25 11:37:11 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 25 Jul 2005 08:37:11 -0700 Subject: [moby] [MOBY-dev] FW: theseu - upload the first beta and unstable version In-Reply-To: <42E2A0D8.4070109@ac.upc.es> References: <42E2A0D8.4070109@ac.upc.es> Message-ID: <1122305831.29736.58.camel@bioinfo.icapture.ubc.ca> Hokey Dinah! That's AMAZING!!! M On Sat, 2005-07-23 at 20:56 +0100, Roman Roset wrote: > Hello, > > thank you very much eddie, to forward my email in this list :). > > I've uploaded a ppt preview of INB-Theseu: > > http://inb.lsi.upc.es/theseu/docs/ppt/TheseuPreview.ppt > > > There are bugs on this demo, but my intention is to know the general > opinion of moby-developers in order to driven the design of future > functionalities. So, all your comments, questions or suggestions are > most welcome. > > INB Theseu people: > > Author: Roman Roset (roset at ac.upc.edu) > Coordinator: David G. Pisano (dgpisano at cnb.uam.es) > Supervised by: Phd. Xavier Messeguer and Phd. Oswaldo Trelles > > Theseu is property of INB - Instituto Nacional de Bioinform?tica > (www.inab.org) > > Thanks. > > Note: I'm on vacation until August 21th > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From senger at ebi.ac.uk Tue Jul 26 08:14:43 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 26 Jul 2005 13:14:43 +0100 (BST) Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42E12C2E.8010406@illuminae.com> Message-ID: > Yup... that suggestion (to use the version position to denote > replication) would be very dangerous! However, it is an interesting > problem, since I don't think the LSID communuty would be very > comfortable with a single LSID referring to multiple resources... even > if they are "identical". Martin, do you have an opinion about this? > This is about replicated services that were (afaik) first mentioned (at least with some depth) in Vancouver. Before we decide about how to create LSID for such kind of services, we need to define what are replicated services. So far, each service was uniquely defined by three elements: - its endpoint, - its authority (in the Biomoby sense, not the LSID sense), and - its (simple) name (unique within its authority). So what are the replicated services? My take (after what I heard in Vancouver) is that the replicated services will have the same two of the attributes above: authority and name, but they will not share the same endpoint. (Additionally, just in documenttaion, we claim that such replicated services do the same data transformation - but this is not check-able.) Is this how we want to define replicated services? If the answer is yes, here is how I would feel about the LSID: 1) Resolving Data. Eddie mentioned that he (or he and Mark, I was not sure) were/are thinking about using LSID resolution method 'getData' for returning a WSDL of this service. I am not sure that I identify myself with this idea. Mainly because we already have a way how to get the same WSDL from the registry (there is an API method for that) so why to duplicate it. And also because doing this we will immediately need different LSID for replicated services (because the endpoint is - at least, usully - part of such WSDL). So I would suggest to return nothing by this method. If this is accepted, I ca elaborate further what to do with metadata. 2) Resolving metadata. Mark is not sure that the replicated services should have the same LSID. I am not sure either - but I am more inclined to say 'Why not?'. They do the same think - except they some of them can do it better (faster, more reliably etc.) But this can be reflected in metadata - it includes the endpoint, so the user will see which metadata refers to which replicated service. We would need to make some consensus on the metadata predicates to assure that the metadata can be identified: which are meant for all such replicated services, and which are meand only for individual such services. If we do not want to have same LSID for replicated services we will need to solve the following with more efforts: 3) Problems with *not* having the same LSID for replicated services: a) The Registry API is going to change because of replicated services (as we talked about it in V.) - but the change would have me more significant than just to chnage the 'endpoint URL' to a set of 'endpoint URLs' - because when the registry returns LSID, it must return more/many LSIDs about the same service. b) The registry would have more problems to collect all available metadata (or to send SignatureURLs of all of them) if their are more LSIDs for the same service. These problems are surely solveable but I wonder why the LSID cannot be same for the replicated services (I also feel that it is a bit strange, but I cannot find the real arguments why not to do it.). Regards, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 26 08:21:40 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 26 Jul 2005 13:21:40 +0100 (BST) Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42E12C2E.8010406@illuminae.com> Message-ID: > Just to expand this a little bit more - biomoby.org will NOT be the > resolver for that LSID, but it will be the authority server for that LSID. > Roman, this remark from Mark is true, but it may be obscure for you unless you know details of the LSID spec. The LSID resolution service specification actually specifies that the data/metadata resolution is done in several steps: - The first is called 'authority server resolution' (or similarly) - and this will be done at biomoby.org. This returns back a list of URLs (and protocols, but the protocols in our case will be only HTTP-GET). - The next step is to use these URLs and fetch what they represent - and this means that you are going to fetch RDF documents from the service provider site (so actually the service providers themselves suddenly become LSID resolution providers). Does it make sense for you? Cheers, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Fri Jul 29 17:35:17 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 29 Jul 2005 14:35:17 -0700 Subject: [MOBY-dev] 0.85 codebase running on test server Message-ID: <1122672917.19643.31.camel@bioinfo.icapture.ubc.ca> Hi dev-ers Anyone watching the commit logs will have seen a lot of activity in the past week! The 0.85 codebase is running now on a test server so that you can start to code to it. Endpoint information is as follows: URL => 'http://bioinfo.icapture.ubc.ca/cgi-bin/mobycentral/MOBY- Central.pl', URI => 'http://bioinfo.icapture.ubc.ca/MOBY/Central The main (visible) difference is that many of the messages now contain LSID information - see the 0.85 API on the biomoby.org website for full details. In addition the use of LSID's in MOBY Central messages should be MUCH more consistent now. The LSID resolver is (should be!) working, so if you try to resolve those LSID's to metadata you will get useful information. Under the hood, Dennis Wang has factored out all of the data-access calls into a single adaptor module (good job Dennis!). Hopefully this will simplify our migration toward the myGrid registry, since we will only need to modify one module. heads-up that Eddie is working on this same test server with the RDF agent, so things you register there may not last for long as they are obliterated by agent activity (and bugs ;-) ), but the MOBY Central code (what is currently in the CVS) seems to be running correctly on that machine so you can at least use it to observe the new message structure and update your code if necessary in anticipation of the production server switching to this codebase. If you have time/ambition/interest please do test this code and let me know if you notice anything incorrect. One of the biggest improvements, in my opinion, is that there is now a very comprehensive test suite for the Perl code - 122 tests! (up from just 12 before). All tests pass on two of my servers, so I think the code is now ~stable. From here on, any bugs that are reported will be fixed together with an addition to the test suite to make sure they don't reappear. As such, the codebase should never degrade again :-) Yes, we're growing up and becoming a mature project! Nota Bene that the test suite DOES NOT PASS ALL TESTS against the current production registry (@mobycentral.icapture.ubc.ca). This is because there are some latent bugs in the current production code that are now revealed by the test suite!! That's all the news. In the pipeline: 1) I will soon commit code changes that prevent inheritance from Primitives 2) I'll create and register replacement objects for all objects that currently inherit from primitives such that they follow the paradigm for object construction that has been extensively discussed since the last meeting. 2a) I will attempt to contact all service providers who will be affected by this change based on their registered contactEmail address 3) Eddie has a test suite of ~50 tests for the RDF agent, however it is not yet passing all tests. Once we believe this is stable/safe we will switch it on and from that point onwards deregistration of services will occur only through agent activity. 4) We are assembling a thesaurus of RDF predicates for service providers to use in their service instance metadata. This will be published shortly. Eddie and I leave for 10 days in Manchester on Sunday, so hopefully there will be more news and progress by the time we get home. Cheers everyone! Mark -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From edward.kawas at gmail.com Mon Jul 25 12:36:58 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Mon, 25 Jul 2005 16:36:58 -0000 Subject: [MOBY-dev] FW: Error handling proposal Message-ID: <42e5173f.2fdbe4fa.4c5a.ffffd788@mx.gmail.com> -----Original Message----- From: Oswaldo Trelles [mailto:ots at ac.uma.es] Sent: Monday, July 25, 2005 9:43 AM To: Eddie Kawas Subject: Error handling proposal Eddie, could you please circulate this message and make the document available following your protocol? thanks in advance Oswaldo Please let me introduce myself. My name is Oswaldo Trelles and I work at the Computer architecture Department of the University of Malaga, Spain (where all of you are invited and welcomed). Our group is in charge of the "Integrated Bioinformatics" aspects at the INB (National Institute of Bioinformatics). As David Gonz?les Pisano informed you at the Vancouver meeting we are developing an integrated platform to deploy general and specific services using BioMOBY as the underlying protocol. Thus we are quite concerned with BM specifications and we would like to contribute in the development, extension and improvements in the protocol. At this moment Roman has made a preliminary version and description of Theseu project available. Now we would like to put over the table the error handling issue. We have prepared a document, which represent an extension of our current implementation. This proposal was discussed during the INB-meeting here in Malaga (July 2005) and now is distributed as an open document for further discussion. Please feel free to comment. best regards, Oswaldo + GNV5 team + INB team -------------- next part -------------- A non-text attachment was scrubbed... Name: 0509GNV5-ErrorHandling-v034distributed.pdf Type: application/pdf Size: 338293 bytes Desc: not available Url : http://www.biomoby.org/pipermail/moby-dev/attachments/20050725/8461ee06/0509GNV5-ErrorHandling-v034distributed-0002.pdf From edward.kawas at gmail.com Wed Jul 27 11:20:26 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Wed, 27 Jul 2005 15:20:26 -0000 Subject: [MOBY-dev] FW: Error handling proposal Message-ID: <42e7a864.0e06ba61.207b.ffffdb6b@mx.gmail.com> Please let me introduce myself. My name is Oswaldo Trelles and I work at the Computer architecture Department of the University of Malaga, Spain (where all of you are invited and welcomed). Our group is in charge of the "Integrated Bioinformatics" aspects at the INB (National Institute of Bioinformatics). As David Gonz?les Pisano informed you at the Vancouver meeting we are developing an integrated platform to deploy general and specific services using BioMOBY as the underlying protocol. Thus we are quite concerned with BM specifications and we would like to contribute in the development, extension and improvements in the protocol. At this moment Roman has made a preliminary version and description of Theseu project available. Now we would like to put over the table the error handling issue. We have prepared a document, which represent an extension of our current implementation. This proposal was discussed during the INB-meeting here in Malaga (July 2005) and now is distributed as an open document for further discussion. Please feel free to comment. best regards, Oswaldo + GNV5 team + INB team -------------- next part -------------- A non-text attachment was scrubbed... Name: 0509GNV5-ErrorHandling-v034distributed.pdf Type: application/pdf Size: 190789 bytes Desc: not available Url : http://www.biomoby.org/pipermail/moby-dev/attachments/20050727/0549ca76/0509GNV5-ErrorHandling-v034distributed-0002.pdf From Pieter.Neerincx at wur.nl Wed Jul 6 14:50:42 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Wed, 6 Jul 2005 16:50:42 +0200 Subject: [MOBY-dev] Perl API: MOBY::Central::DUMP_MySQL() In-Reply-To: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> References: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> Message-ID: <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> Hi all, This message is for those who use Perl and have local MOBY Centrals. I had some problems with the way MOBY::Central::DUMP_MySQL() currently works. Getting a DUMP fails for example with the new central at: URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl URI: http://mobycentral.icapture.ubc.ca/MOBY/Central Currently DUMP_MySQL() has the mysql username, the path to mysqldump and the lack of a password, host and portnumber hard coded. Hence it ignores whatever you have in your mobycentral.config file. Therefore I propose the code below which uses MOBY::Config->new() to get the connection details from your mobycentral.config. The path to mysqldump is no longer hard coded. Hence if it's not in the path of webserver user, I think it's better to modify the path env var as compared to hard coding it in the perl modules:) Unless anyone has objections against the porposed code I'll merge this with the code in the CVS in a few days... Cheers, Pieter -------------------------------------------------- sub DUMP_MySQL { my ( $pkg ) = @_; my $config = MOBY::Config->new(); my @dbsections = ('mobycentral', 'mobyobject', 'mobyservice', 'mobynamespace', 'mobyrelationship'); my @response; foreach my $dbsection (@dbsections) { my $dbname = ${${$config}{$dbsection}}{'dbname'}; my $username = ${${$config}{$dbsection}}{'username'}; my $password = ${${$config}{$dbsection}}{'password'}; my $host = ${${$config}{$dbsection}}{'url'}; my $port = ${${$config}{$dbsection}}{'port'}; open( IN, "mysqldump -h $host -P $port -u $username --password=$password $dbname|" ) || die "can't open $dbname for dumping"; my @dbdump; while ( ) { push @dbdump, $_; } my $dbdump = ( join "", @dbdump ); push @response, $dbdump; } return [@response]; } ------------------------------------------------ Wageningen University and Research centre (WUR) Laboratory of Bioinformatics Transitorium (building 312) room 1038 Dreijenlaan 3 6703 HA Wageningen phone: 0317-484 706 fax: 0317-483 584 mobile: 06-143 66 783 pieter.neerincx at wur.nl From markw at illuminae.com Wed Jul 6 15:09:44 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 06 Jul 2005 08:09:44 -0700 Subject: [moby] [MOBY-dev] Perl API: MOBY::Central::DUMP_MySQL() In-Reply-To: <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> References: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> Message-ID: <1120662584.8869.38.camel@mobycentral.mrl.ubc.ca> Hi Pieter, I didn't realize that the calls were hard-coded! Ugh! Thanks for catching that error - please do go ahead and fix the code. We will be committing a major change to the organization of SQL in the codebase in the next few days, so once you have committed I am going to tag this version and make a release before the chaos ensues :-) M On Wed, 2005-07-06 at 16:50 +0200, Pieter Neerincx wrote: > Hi all, > > This message is for those who use Perl and have local MOBY Centrals. > I had some problems with the way MOBY::Central::DUMP_MySQL() > currently works. Getting a DUMP fails for example with the new > central at: > > URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl > URI: http://mobycentral.icapture.ubc.ca/MOBY/Central > > Currently DUMP_MySQL() has the mysql username, the path to mysqldump > and the lack of a password, host and portnumber hard coded. Hence it > ignores whatever you have in your mobycentral.config file. Therefore > I propose the code below which uses MOBY::Config->new() to get the > connection details from your mobycentral.config. The path to > mysqldump is no longer hard coded. Hence if it's not in the path of > webserver user, I think it's better to modify the path env var as > compared to hard coding it in the perl modules:) Unless anyone has > objections against the porposed code I'll merge this with the code in > the CVS in a few days... > > Cheers, > > Pieter > > -------------------------------------------------- > sub DUMP_MySQL { > my ( $pkg ) = @_; > my $config = MOBY::Config->new(); > my @dbsections = ('mobycentral', 'mobyobject', > 'mobyservice', 'mobynamespace', 'mobyrelationship'); > my @response; > > foreach my $dbsection (@dbsections) { > my $dbname = ${${$config}{$dbsection}}{'dbname'}; > my $username = ${${$config}{$dbsection}}{'username'}; > my $password = ${${$config}{$dbsection}}{'password'}; > my $host = ${${$config}{$dbsection}}{'url'}; > my $port = ${${$config}{$dbsection}}{'port'}; > open( IN, "mysqldump -h $host -P $port -u $username > --password=$password $dbname|" ) > || die "can't open $dbname for dumping"; > my @dbdump; > while ( ) { > push @dbdump, $_; > } > my $dbdump = ( join "", @dbdump ); > push @response, $dbdump; > } > return [@response]; > } > ------------------------------------------------ > > > > Wageningen University and Research centre (WUR) > Laboratory of Bioinformatics > Transitorium (building 312) room 1038 > Dreijenlaan 3 > 6703 HA Wageningen > phone: 0317-484 706 > fax: 0317-483 584 > mobile: 06-143 66 783 > pieter.neerincx at wur.nl > > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From Pieter.Neerincx at wur.nl Wed Jul 6 15:41:59 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Wed, 6 Jul 2005 17:41:59 +0200 Subject: [moby] [MOBY-dev] Perl API: MOBY::Central::DUMP_MySQL() In-Reply-To: <1120662584.8869.38.camel@mobycentral.mrl.ubc.ca> References: <1120174725.31990.6.camel@mobycentral.mrl.ubc.ca> <7F3A55F3-89D5-45B8-869D-8C322D3436A9@wur.nl> <1120662584.8869.38.camel@mobycentral.mrl.ubc.ca> Message-ID: Hi Mark, On 6-Jul-2005, at 5:09 PM, Mark Wilkinson wrote: > Hi Pieter, > > I didn't realize that the calls were hard-coded! Ugh! > > Thanks for catching that error - please do go ahead and fix the code. Ok, done. > > We will be committing a major change to the organization of SQL in the > codebase in the next few days, so once you have committed I am > going to > tag this version and make a release before the chaos ensues :-) Whoops, good luck! Cheers, Pieter > > M > > > On Wed, 2005-07-06 at 16:50 +0200, Pieter Neerincx wrote: > >> Hi all, >> >> This message is for those who use Perl and have local MOBY Centrals. >> I had some problems with the way MOBY::Central::DUMP_MySQL() >> currently works. Getting a DUMP fails for example with the new >> central at: >> >> URL: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl >> URI: http://mobycentral.icapture.ubc.ca/MOBY/Central >> >> Currently DUMP_MySQL() has the mysql username, the path to mysqldump >> and the lack of a password, host and portnumber hard coded. Hence it >> ignores whatever you have in your mobycentral.config file. Therefore >> I propose the code below which uses MOBY::Config->new() to get the >> connection details from your mobycentral.config. The path to >> mysqldump is no longer hard coded. Hence if it's not in the path of >> webserver user, I think it's better to modify the path env var as >> compared to hard coding it in the perl modules:) Unless anyone has >> objections against the porposed code I'll merge this with the code in >> the CVS in a few days... >> >> Cheers, >> >> Pieter >> >> -------------------------------------------------- >> sub DUMP_MySQL { >> my ( $pkg ) = @_; >> my $config = MOBY::Config->new(); >> my @dbsections = ('mobycentral', 'mobyobject', >> 'mobyservice', 'mobynamespace', 'mobyrelationship'); >> my @response; >> >> foreach my $dbsection (@dbsections) { >> my $dbname = ${${$config}{$dbsection}}{'dbname'}; >> my $username = ${${$config}{$dbsection}} >> {'username'}; >> my $password = ${${$config}{$dbsection}} >> {'password'}; >> my $host = ${${$config}{$dbsection}}{'url'}; >> my $port = ${${$config}{$dbsection}}{'port'}; >> open( IN, "mysqldump -h $host -P $port -u $username >> --password=$password $dbname|" ) >> || die "can't open $dbname for dumping"; >> my @dbdump; >> while ( ) { >> push @dbdump, $_; >> } >> my $dbdump = ( join "", @dbdump ); >> push @response, $dbdump; >> } >> return [@response]; >> } >> ------------------------------------------------ >> >> >> >> Wageningen University and Research centre (WUR) >> Laboratory of Bioinformatics >> Transitorium (building 312) room 1038 >> Dreijenlaan 3 >> 6703 HA Wageningen >> phone: 0317-484 706 >> fax: 0317-483 584 >> mobile: 06-143 66 783 >> pieter.neerincx at wur.nl >> >> >> _______________________________________________ >> MOBY-dev mailing list >> MOBY-dev at biomoby.org >> http://www.biomoby.org/mailman/listinfo/moby-dev >> > -- > Mark Wilkinson > Asst. Professor > Dept. of Medical Genetics > University of British Columbia > PI in Bioinformatics > iCAPTURE Centre > St. Paul's Hospital > Vancouver, BC > > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > Wageningen University and Research centre (WUR) Laboratory of Bioinformatics Transitorium (building 312) room 1038 Dreijenlaan 3 6703 HA Wageningen phone: 0317-484 706 fax: 0317-483 584 mobile: 06-143 66 783 pieter.neerincx at wur.nl From markw at illuminae.com Wed Jul 6 16:35:19 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 06 Jul 2005 09:35:19 -0700 Subject: [MOBY-dev] Bad documetation for Secondary parameters in MOBY Central Message-ID: <1120667719.8869.75.camel@mobycentral.mrl.ubc.ca> Hi all, I just got a heads-up from Eddie that the documentation for MOBY::Central was incorrect. It had enumerated the valid secondary data-types as STRING|INT|FLOAT, etc. They are, in fact, String, Integer, Float, DateTime (capitalization differences). This is tested at the DB level, so it causes some mis-registration if you aren't aware of the problem. We're fixing the code to throw a warning when it detects a mistaken parameter, but I wanted to let you know that this is an issue in case you have been wondering why you can't register secondaries :-/ M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From rebecca.ernst at gsf.de Thu Jul 7 13:45:35 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Thu, 07 Jul 2005 15:45:35 +0200 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <42CD31FF.6040803@gsf.de> Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From markw at illuminae.com Thu Jul 7 13:56:16 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 13:56:16 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1562815689-1120744590-cardhu_blackberry.rim.net-11062-@engine12-cell01> I guess it doesn't make you feel any better if I say "yes, we know..." :-) M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas at gmail.com, markw at illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 13:56:16 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 13:56:16 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1562815689-1120744590-cardhu_blackberry.rim.net-11062-@engine12-cell01> I guess it doesn't make you feel any better if I say "yes, we know..." :-) M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas at gmail.com, markw at illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 13:58:53 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 13:58:53 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> We really need to get write-access to the Taverna CVS so that we can fix these kinds of things in a timely way... Hint hint :-). Anyone from myGrid listening? M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas at gmail.com, markw at illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 13:58:53 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 13:58:53 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> We really need to get write-access to the Taverna CVS so that we can fix these kinds of things in a timely way... Hint hint :-). Anyone from myGrid listening? M -----Original Message----- From: Rebecca Ernst Date: Thu, 07 Jul 2005 15:45:35 To:edward.kawas at gmail.com, markw at illuminae.com, Martin Senger , Dirk Haase , Heiko Schoof Cc:mobydev Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From schoof at mpiz-koeln.mpg.de Thu Jul 7 15:34:39 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Thu, 7 Jul 2005 17:34:39 +0200 Subject: [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CD31FF.6040803@gsf.de> References: <42CD31FF.6040803@gsf.de> Message-ID: <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> Please let's take a minute and review the use of collections in BioMoby, before we go off to create "workarounds"... I myself am confused about the use of collections. Originally I had in my mind that Collections were a construct to allow objects that inherently belong together to be "bagged". Example: A multiple alignment program that takes any number of sequences as input. Example2: A keyword search that takes any number of keywords and then does a query combining all of them. As opposed to: Inputing a list of keywords and executing the keyword search for each of them, which would require a separate moby:mobyData and queryID for each of the keywords. The confusion starts with the output of services. My understanding was that ONLY a service that is guaranteed to output exactly one object for each query (e.g. an averaging service that outputs the average of a list of inputs) is registered as outputting a Simple, all others have to output collections (as there must be exactly one mobyData matching the queryID of the input in the response, and a mobyData may contain multiple Simple elements only if wrapped by a Collection). This makes it impossible to distinguish between the situation where one query produces multiple unrelated results, versus one or more bags of related results. Imagine a "getnonoverlappingsubstrings" service: with input abc, it should output [a, bc], [ab, c] and [a, b, c]. Outputting [a, bc, ab, c, a, b, c] would not be useful. More biological example: Dirk has a service that returns sets of orthologous genes when given a set of species. For this, he requires collections both for input and for output. The ideal situation in my view would be: Input: Collection of Species Output: Many Collections, each Collection contains orthologous genes (one from each species). The Collection here defines a set of orthologs, and using a collection would be more elegant than having to define a Moby object "OrhologSet" has (2...n) Objects In practice, the Collection tag has been used to indicate when more than one Simple occurs, with no "semantic" meaning. This imho is not necessary; when more than one Simple occurs, why not put more than one Simple? It's easy enough for everyone to figure that out. Then, Collection could be used to actually transfer meaning ;-) This is a drastic change in the way the API is being interpreted, and will break code. So it needs calm thinking. But with the "big" API change coming up for the Primitives, it could be done. And it would make things clearer, also to e.g. the Taverna developers: Getting back many Simple articles in response to a query very intuitively indicates to continue on with each one individually, whereas getting back a Collection indicates to put the whole thing as input into the next service, which is what they implemented. Makes perfect sense, as there can and will be services that consume Collections. E.g., the ortholog set case above: Pipe it into a Multiple Alignment service, and you get all the alignments for each of the set of orthologs. Getting one massive alignment of everything wouldn't make sense. Maybe I've made myself clear, maybe not. Anyway, the Collection issue has led to quite some discussions between Rebecca, Dirk and myself, and we are all not happy with the way they are currently handled. Best, Heiko On 7. Jul 2005, at 15:45 Uhr, Rebecca Ernst wrote: Hi Eddie, Mark, Martin, Heiko, etc.! we recently downloaded Taverna 1.2 and found that basically no workflow is functional anymore. After a look into it we found that there was one major change from Taverna 1.1 to Taverna 1.2. which is that obviously Taverna 1.2 passes collections on to the next services whereas Taverna 1.1 took a collection, splitted it into singles and passed the singles to the next service. There are very many services around that give back collections and very few that operate on those. For the moment this means that most of the Biomoby services are not compatible to each other even if they give back the same class of object! (e.g. using the service getAGILocusCodes you'll receive a collection of AGI Codes and if you pass this on to the service getArabidopsisProteinSequences it'll fail as this service takes simple inputs only ) As far as we see there would only be one solution for that - we would need a new moby widget 'splitCollectionsIntoSingles' which still would be 'ugly' as you would also need to know what the first service gives back and what the next service needs... Any other ideas / suggestions? Best, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From tmo at ebi.ac.uk Thu Jul 7 15:50:31 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Thu, 07 Jul 2005 16:50:31 +0100 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> References: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> Message-ID: <42CD4F47.4030901@ebi.ac.uk> mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom From tmo at ebi.ac.uk Thu Jul 7 15:50:31 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Thu, 07 Jul 2005 16:50:31 +0100 Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> References: <1417829463-1120744749-cardhu_blackberry.rim.net-1377-@engine12-cell01> Message-ID: <42CD4F47.4030901@ebi.ac.uk> mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom From markw at illuminae.com Thu Jul 7 16:25:13 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 16:25:13 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1289629624-1120753526-cardhu_blackberry.rim.net-7512-@engine08-cell01> :-). We have all been well and truly spanked :-) Everything Tom says is (obviously) correct, and should apply to the moby CVS commits also. Thanks for giving Eddie access! M . -----Original Message----- From: Tom Oinn Date: Thu, 07 Jul 2005 16:50:31 To:markw at illuminae.com, Core developer announcements Cc:Martin Senger , mobydev Subject: Re: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 16:25:13 2005 From: markw at illuminae.com (mark wilkinson) Date: Thu, 7 Jul 2005 16:25:13 +0000 GMT Subject: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 Message-ID: <1289629624-1120753526-cardhu_blackberry.rim.net-7512-@engine08-cell01> :-). We have all been well and truly spanked :-) Everything Tom says is (obviously) correct, and should apply to the moby CVS commits also. Thanks for giving Eddie access! M . -----Original Message----- From: Tom Oinn Date: Thu, 07 Jul 2005 16:50:31 To:markw at illuminae.com, Core developer announcements Cc:Martin Senger , mobydev Subject: Re: [MOBY-dev] Problems with Biomoby services in Taverna 1.2 mark wilkinson wrote: > We really need to get write-access to the Taverna CVS so that we can > fix these kinds of things in a timely way... Hint hint :-). Anyone > from myGrid listening? What you need to do is TEST YOUR CODE!!! I'll give you access to our CVS repository but I'm highly unimpressed that a) noone thought to check that the plugin behaviour was equivalent and b) I still have no documentation from anyone on the moby team for this code. I know Taverna is hardly perfect from a docs standpoint but you can understand my reluctance to import code from someone I've never met without any evidence of either testing, architecture design, specification or documentation. For the future (i.e. Taverna2) we will reject ALL code (including mine) that does not have at least 90% line and branch coverage from unit tests, I wish I'd done this first time around but hey, we live and learn :) Similarly we will not accept code that is undocumented at both API and user level - there is ABSOLUTELY NO POINT in writing some neat functionality (such as the object creator) and not producing good documentation. Good documentation in this context consists of well written English suitable for a user of the workbench along with examples and screenshots where appropriate. In this particular case the behaviour that Rebecca refers to needs to be reintroduced, it is imperative that workflows from 1.1 work in 1.2 and subsequent versions with no alterations to the workflow - at least no manual ones. This might involve adding logic to the parser to transform older workflows into the newer form. The main requirement is that moby collections be exposed to Taverna's type system, the previous behaviour was to split the collections into taverna collections of moby simples thus allowing our existing iteration framework to take care of any type mismatches. This feature must be reintroduced with some urgency, removing it changes the fundamental behaviour of the workflow engine and we must avoid doing this. I am of course largely to blame for not insisting on this prior to the release, mea culpa. Eddie - you have sf.net CVS access to taverna but please tell us via the developers mailing list when you change things, don't rely on the cvs logs (but do put properly informative messages in them!) Tom _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson ...on the road! From markw at illuminae.com Thu Jul 7 20:07:48 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Thu, 07 Jul 2005 13:07:48 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> Message-ID: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Thu Jul 7 20:07:48 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Thu, 07 Jul 2005 13:07:48 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> Message-ID: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From schoof at mpiz-koeln.mpg.de Fri Jul 8 06:45:28 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Fri, 8 Jul 2005 08:45:28 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> Message-ID: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Well, if I understand you correctly, Mark, than I was right all along in that they way I see collections is the way they should be seen. Then the problem is that that is not clear enough to everybody else, and that possibly the examples in the API are misleading, because it's been used in such a different way. It's definitely something that needs to be fixed before release 1.0. If I understand Tom correctly, the problem mainly appeared because the BioMoby support was rapidly and effectively incorporated into Taverna without proper integration, and what needs to happen is to define e.g. the iteration strategy for BioMoby: Taverna so far doesn't use the possibility to bundle multiple queries in a single BioMoby message, which is not a problem, but the Collection issue could point to a need for us BioMoby guys to look in more detail at Taverna data types and do a sound mapping. Heiko On 7. Jul 2005, at 22:07 Uhr, Mark Wilkinson wrote: On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From schoof at mpiz-koeln.mpg.de Fri Jul 8 06:45:28 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Fri, 8 Jul 2005 08:45:28 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> Message-ID: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Well, if I understand you correctly, Mark, than I was right all along in that they way I see collections is the way they should be seen. Then the problem is that that is not clear enough to everybody else, and that possibly the examples in the API are misleading, because it's been used in such a different way. It's definitely something that needs to be fixed before release 1.0. If I understand Tom correctly, the problem mainly appeared because the BioMoby support was rapidly and effectively incorporated into Taverna without proper integration, and what needs to happen is to define e.g. the iteration strategy for BioMoby: Taverna so far doesn't use the possibility to bundle multiple queries in a single BioMoby message, which is not a problem, but the Collection issue could point to a need for us BioMoby guys to look in more detail at Taverna data types and do a sound mapping. Heiko On 7. Jul 2005, at 22:07 Uhr, Mark Wilkinson wrote: On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > I myself am confused about the use of collections. Originally I had in > my mind that Collections were a construct to allow objects that > inherently belong together to be "bagged". You are 100% correct here. > The confusion starts with the output of services. My understanding was > that ONLY a service that is guaranteed to output exactly one object for > each query (e.g. an averaging service that outputs the average of a > list of inputs) is registered as outputting a Simple, all others have > to output collections No. > (as there must be exactly one mobyData matching > the queryID of the input in the response, Yes > and a mobyData may contain > multiple Simple elements only if wrapped by a Collection). No. A mobyData element can wrap any number of outputs, and/or combinations of simples, collections, and secondaries. > In practice, the Collection tag has been used to indicate when more > than one Simple occurs, with no "semantic" meaning. This imho is not > necessary; when more than one Simple occurs, why not put more than one > Simple? It's easy enough for everyone to figure that out. Then, > Collection could be used to actually transfer meaning ;-) I think a lot of people are using Collections incorrectly, yes. > make things clearer, also to e.g. the Taverna developers: Getting back > many Simple articles in response to a query very intuitively indicates > to continue on with each one individually, whereas getting back a > Collection indicates to put the whole thing as input into the next > service, which is what they implemented. Makes perfect sense, as there > can and will be services that consume Collections. I don't think that Taverna handled collections correctly in the past, and it would be a shame if that identical functionality was added back in :-) The functionality does need to go back into Taverna, but correctly this time. > Maybe I've made myself clear, maybe not. Anyway, the Collection issue > has led to quite some discussions between Rebecca, Dirk and myself, and > we are all not happy with the way they are currently handled. I don't know if you are happier with them now that I have pointed out where you are misinterpreting them...?? I agree, however, that their usage by many service providers is not correct... but there's no way for MOBY Central to know that it isn't correct, so we can't throw an error on registration of these "wonky" services... M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From rebecca.ernst at gsf.de Fri Jul 8 06:50:17 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Fri, 08 Jul 2005 08:50:17 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> Message-ID: <42CE2229.1030504@gsf.de> Hi Mark! I will have to spank you a lot for either the last email or the one you sent me in February where I asked you how to use collections (see mail below): Well... the answer depends on what the generic case of your output will look like. the rule is that you have to register every output object that your service produces; i.e. every immediate child tag of a mobyData block. Therefore, if you are going to consistently output exactly 3 Genbank/gi's, then you would register your service as outputting 3 X Object(NCBI_gi). If you are outputting an indeterminate number of objects, then you register it as outputting 1 X Collection. Collections map nicely onto the rdf:Bag structure, if that helps you interpret them. M On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: >> Hi Mark! >> >> We have problems interpreting the BioMoby API for the output objects. >> >> The thing is that we have a service that gets ONE input object and >> executes a freetext search over several databases. This service will >> eventually return more that one result. >> We don't want to put the result into a collection object because the >> single objects don't build any entity. >> We also can't give back more that one mobyData block because we only >> have one query ID and therefore only one queryID to give back. >> >> The only solution we can think of is an object like this: >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> would this be a legal object or do we have to use collection even if the >> objects don't build an entity ? >> >> >> Cheers, >> Rebecca > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical Genetics, UBC, Canada ----------------------------------------------------------------------------------------------------------- Mark Wilkinson wrote: >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > >>I myself am confused about the use of collections. Originally I had in >>my mind that Collections were a construct to allow objects that >>inherently belong together to be "bagged". >> >> > >You are 100% correct here. > > > > >>The confusion starts with the output of services. My understanding was >>that ONLY a service that is guaranteed to output exactly one object for >>each query (e.g. an averaging service that outputs the average of a >>list of inputs) is registered as outputting a Simple, all others have >>to output collections >> >> > >No. > > > >> (as there must be exactly one mobyData matching >>the queryID of the input in the response, >> >> > >Yes > > > >> and a mobyData may contain >>multiple Simple elements only if wrapped by a Collection). >> >> > >No. A mobyData element can wrap any number of outputs, and/or >combinations of simples, collections, and secondaries. > > > > > >>In practice, the Collection tag has been used to indicate when more >>than one Simple occurs, with no "semantic" meaning. This imho is not >>necessary; when more than one Simple occurs, why not put more than one >>Simple? It's easy enough for everyone to figure that out. Then, >>Collection could be used to actually transfer meaning ;-) >> >> > >I think a lot of people are using Collections incorrectly, yes. > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back >>many Simple articles in response to a query very intuitively indicates >>to continue on with each one individually, whereas getting back a >>Collection indicates to put the whole thing as input into the next >>service, which is what they implemented. Makes perfect sense, as there >>can and will be services that consume Collections. >> >> > >I don't think that Taverna handled collections correctly in the past, >and it would be a shame if that identical functionality was added back >in :-) The functionality does need to go back into Taverna, but >correctly this time. > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue >>has led to quite some discussions between Rebecca, Dirk and myself, and >>we are all not happy with the way they are currently handled. >> >> > >I don't know if you are happier with them now that I have pointed out >where you are misinterpreting them...?? > >I agree, however, that their usage by many service providers is not >correct... but there's no way for MOBY Central to know that it isn't >correct, so we can't throw an error on registration of these "wonky" >services... > >M > > > > > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From tmo at ebi.ac.uk Fri Jul 8 08:07:44 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Fri, 08 Jul 2005 09:07:44 +0100 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <42CE3450.1000209@ebi.ac.uk> Heiko Schoof wrote: > If I understand Tom correctly, the problem mainly appeared because the > BioMoby support was rapidly and effectively incorporated into Taverna > without proper integration, and what needs to happen is to define e.g. > the iteration strategy for BioMoby: Taverna so far doesn't use the > possibility to bundle multiple queries in a single BioMoby message, > which is not a problem, but the Collection issue could point to a need > for us BioMoby guys to look in more detail at Taverna data types and do > a sound mapping. > Heiko That's kind of true, meaning actually not. There are three cases involving collections (Taverna 1.1 behaviour) : 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. The intent is that as with the other plugin types Taverna guarantees that the service sees the inputs it has asked for. Our experience with other service types suggests that this is an extremely powerful mechanism as it allows interoperability between services that would otherwise mismatch - it's worth noting that our users expect these services to match, while a CS perspective regards ProteinSequence and ProteinSequence[] as completely different types most of our biologists don't! Taverna behaves the way _they_ expect it to, remember who your user are. Taverna data types are pretty much trivial, they're opaque data blobs with the exception of collection structure which is exposed. We only expose the collection structure to ensure the above properties, other than that the framework is data agnostic (as it should be). Tom From tmo at ebi.ac.uk Fri Jul 8 08:07:44 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Fri, 08 Jul 2005 09:07:44 +0100 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <42CE3450.1000209@ebi.ac.uk> Heiko Schoof wrote: > If I understand Tom correctly, the problem mainly appeared because the > BioMoby support was rapidly and effectively incorporated into Taverna > without proper integration, and what needs to happen is to define e.g. > the iteration strategy for BioMoby: Taverna so far doesn't use the > possibility to bundle multiple queries in a single BioMoby message, > which is not a problem, but the Collection issue could point to a need > for us BioMoby guys to look in more detail at Taverna data types and do > a sound mapping. > Heiko That's kind of true, meaning actually not. There are three cases involving collections (Taverna 1.1 behaviour) : 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. The intent is that as with the other plugin types Taverna guarantees that the service sees the inputs it has asked for. Our experience with other service types suggests that this is an extremely powerful mechanism as it allows interoperability between services that would otherwise mismatch - it's worth noting that our users expect these services to match, while a CS perspective regards ProteinSequence and ProteinSequence[] as completely different types most of our biologists don't! Taverna behaves the way _they_ expect it to, remember who your user are. Taverna data types are pretty much trivial, they're opaque data blobs with the exception of collection structure which is exposed. We only expose the collection structure to ensure the above properties, other than that the framework is data agnostic (as it should be). Tom From d.haase at gsf.de Fri Jul 8 08:49:58 2005 From: d.haase at gsf.de (Dirk Haase) Date: Fri, 8 Jul 2005 10:49:58 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <200507081049.58799.d.haase@gsf.de> On Friday 08 July 2005 08:45, Heiko Schoof wrote: > Well, if I understand you correctly, Mark, than I was right all along > in that they way I see collections is the way they should be seen. Then > the problem is that that is not clear enough to everybody else, and > that possibly the examples in the API are misleading, because it's been > used in such a different way. It's definitely something that needs to > be fixed before release 1.0. Not only the examples are misleading, actually at least the PERL API is lacking a suitable method to construct an output consisting of several 'Simple's. One can not simply concat the output of multibple 'simpleResponse' calls, as this would create several mobyData blocks. In order to fill this gap, I created a new method 'multiSimpleResponse' (see below) within CommonSubs.pm which is more or less identical to 'collectionResponse' except that it obviously leaves out the moby:Collection tag. The provided articleName is put into each 'Simple' tag which should be appropriate. Any objections? Regards, dirk =head2 multiSimpleResponse name : multiSimpleResponse function : wraps a set of simple articles in the appropriate mobyData structure usage : return responseHeader . &multiSimpleResponse(\@objects, 'MyArticleName', $queryID) . responseFooter; args : (in order) \@objects - (optional) a listref of MOBY Objects as raw XML $article - (optional) an articeName for this article $queryID - (optional, but strongly recommended) the mobyData ID to which you are responding notes : as required by the API you must return a response for every input. If one of the inputs was invalid, you return a valid (empty) MOBY response by calling &multiSimpleResponse(undef, undef, $queryID). =cut sub multiSimpleResponse { my ( $data, $articleName, $qID ) = @_; # articleName optional my $content = ""; $data ||= []; $qID ||= ''; $articleName ||=""; unless ( ( ref( $data ) =~ /array/i ) && $data->[0] ) { # we're expecting an arrayref as input data,and it must not be empty return ""; } foreach ( @{$data} ) { if ( $_ ) { $content .= " $_ "; } else { $content .= " "; } } return " $content "; } From d.haase at gsf.de Fri Jul 8 08:49:58 2005 From: d.haase at gsf.de (Dirk Haase) Date: Fri, 8 Jul 2005 10:49:58 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> Message-ID: <200507081049.58799.d.haase@gsf.de> On Friday 08 July 2005 08:45, Heiko Schoof wrote: > Well, if I understand you correctly, Mark, than I was right all along > in that they way I see collections is the way they should be seen. Then > the problem is that that is not clear enough to everybody else, and > that possibly the examples in the API are misleading, because it's been > used in such a different way. It's definitely something that needs to > be fixed before release 1.0. Not only the examples are misleading, actually at least the PERL API is lacking a suitable method to construct an output consisting of several 'Simple's. One can not simply concat the output of multibple 'simpleResponse' calls, as this would create several mobyData blocks. In order to fill this gap, I created a new method 'multiSimpleResponse' (see below) within CommonSubs.pm which is more or less identical to 'collectionResponse' except that it obviously leaves out the moby:Collection tag. The provided articleName is put into each 'Simple' tag which should be appropriate. Any objections? Regards, dirk =head2 multiSimpleResponse name : multiSimpleResponse function : wraps a set of simple articles in the appropriate mobyData structure usage : return responseHeader . &multiSimpleResponse(\@objects, 'MyArticleName', $queryID) . responseFooter; args : (in order) \@objects - (optional) a listref of MOBY Objects as raw XML $article - (optional) an articeName for this article $queryID - (optional, but strongly recommended) the mobyData ID to which you are responding notes : as required by the API you must return a response for every input. If one of the inputs was invalid, you return a valid (empty) MOBY response by calling &multiSimpleResponse(undef, undef, $queryID). =cut sub multiSimpleResponse { my ( $data, $articleName, $qID ) = @_; # articleName optional my $content = ""; $data ||= []; $qID ||= ''; $articleName ||=""; unless ( ( ref( $data ) =~ /array/i ) && $data->[0] ) { # we're expecting an arrayref as input data,and it must not be empty return ""; } foreach ( @{$data} ) { if ( $_ ) { $content .= " $_ "; } else { $content .= " "; } } return " $content "; } From schoof at mpiz-koeln.mpg.de Fri Jul 8 08:54:03 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Fri, 8 Jul 2005 10:54:03 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <24407d5355834f57f8ac3e2245daa77d@mpiz-koeln.mpg.de> Point taken to prove that indeed, I have to look into Taverna in more depth ;-) Anyway, Tom, what you state below is exactly what I would expect as behaviour, but I think on the BioMoby side we must ensure that our "Collections" behave in the right way, and I'm not yet sure they are precisely the same thing as what you call collection. Best, Heiko On 8. Jul 2005, at 10:07 Uhr, Tom Oinn wrote: Heiko Schoof wrote: > If I understand Tom correctly, the problem mainly appeared because the > BioMoby support was rapidly and effectively incorporated into Taverna > without proper integration, and what needs to happen is to define e.g. > the iteration strategy for BioMoby: Taverna so far doesn't use the > possibility to bundle multiple queries in a single BioMoby message, > which is not a problem, but the Collection issue could point to a need > for us BioMoby guys to look in more detail at Taverna data types and > do a sound mapping. > Heiko That's kind of true, meaning actually not. There are three cases involving collections (Taverna 1.1 behaviour) : 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. The intent is that as with the other plugin types Taverna guarantees that the service sees the inputs it has asked for. Our experience with other service types suggests that this is an extremely powerful mechanism as it allows interoperability between services that would otherwise mismatch - it's worth noting that our users expect these services to match, while a CS perspective regards ProteinSequence and ProteinSequence[] as completely different types most of our biologists don't! Taverna behaves the way _they_ expect it to, remember who your user are. Taverna data types are pretty much trivial, they're opaque data blobs with the exception of collection structure which is exposed. We only expose the collection structure to ensure the above properties, other than that the framework is data agnostic (as it should be). Tom _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From rebecca.ernst at gsf.de Fri Jul 8 09:07:50 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Fri, 08 Jul 2005 11:07:50 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <42CE4266.3030304@gsf.de> Hi Tom and others This Taverna behaviour described below makes perfect sense to me! Does Taverna then check in MOBY-Central how this service is registered (taking simples or collections)? I guess the way to go would be to: 1. change the Taverna behaviour back to how it is supposed to work (and did in 1.1) 2. - change the BioMoby Perl code to allow more than one simple in the MobyData AND - change almost all services that output collections to the way we discussed yesterday (only output collections if the objects build an entity. ) Either of the two changes solves the problems but I believe both are due to changes and should take place. Best, Rebecca Tom Oinn wrote: > That's kind of true, meaning actually not. There are three cases > involving collections (Taverna 1.1 behaviour) : > > 1) Consumer declares it consumes singles, Producer emits a collection. > In this context Taverna iteratively calls the Consumer with each item > from the collection. This is probably what you'd expect to happen, the > result is that the Consumer effectively emits a collection of whatever > it would emit normally. > 2) Consumer declares it consumes a collection, Producer emits a > collection. In this case Taverna will indeed split the output > collection (because we always do) but it will be magically reassembled > before being given to the Consumer. > > 3) Consumer declares it consumes a collection, Producer emits a single > item. Taverna wraps the single item in a single element collection and > gives it to the Consumer. > > The intent is that as with the other plugin types Taverna guarantees > that the service sees the inputs it has asked for. Our experience with > other service types suggests that this is an extremely powerful > mechanism as it allows interoperability between services that would > otherwise mismatch - it's worth noting that our users expect these > services to match, while a CS perspective regards ProteinSequence and > ProteinSequence[] as completely different types most of our biologists > don't! Taverna behaves the way _they_ expect it to, remember who your > user are. > > Taverna data types are pretty much trivial, they're opaque data blobs > with the exception of collection structure which is exposed. We only > expose the collection structure to ensure the above properties, other > than that the framework is data agnostic (as it should be). > > Tom > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From rebecca.ernst at gsf.de Fri Jul 8 09:07:50 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Fri, 08 Jul 2005 11:07:50 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <42CE4266.3030304@gsf.de> Hi Tom and others This Taverna behaviour described below makes perfect sense to me! Does Taverna then check in MOBY-Central how this service is registered (taking simples or collections)? I guess the way to go would be to: 1. change the Taverna behaviour back to how it is supposed to work (and did in 1.1) 2. - change the BioMoby Perl code to allow more than one simple in the MobyData AND - change almost all services that output collections to the way we discussed yesterday (only output collections if the objects build an entity. ) Either of the two changes solves the problems but I believe both are due to changes and should take place. Best, Rebecca Tom Oinn wrote: > That's kind of true, meaning actually not. There are three cases > involving collections (Taverna 1.1 behaviour) : > > 1) Consumer declares it consumes singles, Producer emits a collection. > In this context Taverna iteratively calls the Consumer with each item > from the collection. This is probably what you'd expect to happen, the > result is that the Consumer effectively emits a collection of whatever > it would emit normally. > 2) Consumer declares it consumes a collection, Producer emits a > collection. In this case Taverna will indeed split the output > collection (because we always do) but it will be magically reassembled > before being given to the Consumer. > > 3) Consumer declares it consumes a collection, Producer emits a single > item. Taverna wraps the single item in a single element collection and > gives it to the Consumer. > > The intent is that as with the other plugin types Taverna guarantees > that the service sees the inputs it has asked for. Our experience with > other service types suggests that this is an extremely powerful > mechanism as it allows interoperability between services that would > otherwise mismatch - it's worth noting that our users expect these > services to match, while a CS perspective regards ProteinSequence and > ProteinSequence[] as completely different types most of our biologists > don't! Taverna behaves the way _they_ expect it to, remember who your > user are. > > Taverna data types are pretty much trivial, they're opaque data blobs > with the exception of collection structure which is exposed. We only > expose the collection structure to ensure the above properties, other > than that the framework is data agnostic (as it should be). > > Tom > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From jmfernandez at cnb.uam.es Fri Jul 8 11:12:37 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Fri, 08 Jul 2005 13:12:37 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna1.2 In-Reply-To: <42CE4266.3030304@gsf.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> Message-ID: <42CE5FA5.50901@cnb.uam.es> Hi everyone, the Taverna v1.1 behaviour for MOBY should be the default one for services which only takes one input parameter. But, which collection semantics should have Taverna MOBY plugin for services with more than one parameter? All of this is related to iteration strategies. For instance, imagine this scenario: we have a service which needs two input Simple parameters, A and B. What should we do? * If both input parameters receive a Simple, no problem, default behaviour. * If one of those parameters receive a Collection with N Simples, and the other a Simple, the default iteration strategy should be either invoking the MOBY service N times with N separate SOAP submissions, or invoking the MOBY service N times with only a SOAP submission (or something intermediate). * If each one of the parameters receive a Collection, then the iteration strategy *should* be explicitly set by the workflow creator: either cartesian product (first of A with each one of B, second of A with each one of B) or dot product (first for A with first for B, second for A with second for B, and so on, either discarding the last elements from the longest Collection, or firing an exception/error/message/whatever). The other two scenarios are: a two-input service with both parameters defined as Collection; and a two-input service with one the parameters as Collection and the other as Simple. I leave them as an exercise for the reader 8-), so I'm avoiding to write now the most boring e-mail in BioMOBY story ;-) Best Regards, Jos? Mar?a Rebecca Ernst wrote: > Hi Tom and others > > This Taverna behaviour described below makes perfect sense to me! Does > Taverna then check in MOBY-Central how this service is registered > (taking simples or collections)? > > I guess the way to go would be to: > 1. change the Taverna behaviour back to how it is supposed to work (and > did in 1.1) > 2. - change the BioMoby Perl code to allow more than one simple in the > MobyData AND > - change almost all services that output collections to the way we > discussed yesterday (only output collections if the objects build an > entity. ) > > Either of the two changes solves the problems but I believe both are due > to changes and should take place. > > Best, > Rebecca > > > > > > Tom Oinn wrote: > >> That's kind of true, meaning actually not. There are three cases >> involving collections (Taverna 1.1 behaviour) : >> >> 1) Consumer declares it consumes singles, Producer emits a collection. >> In this context Taverna iteratively calls the Consumer with each item >> from the collection. This is probably what you'd expect to happen, the >> result is that the Consumer effectively emits a collection of whatever >> it would emit normally. > > >> 2) Consumer declares it consumes a collection, Producer emits a >> collection. In this case Taverna will indeed split the output >> collection (because we always do) but it will be magically reassembled >> before being given to the Consumer. >> >> 3) Consumer declares it consumes a collection, Producer emits a single >> item. Taverna wraps the single item in a single element collection and >> gives it to the Consumer. >> >> The intent is that as with the other plugin types Taverna guarantees >> that the service sees the inputs it has asked for. Our experience with >> other service types suggests that this is an extremely powerful >> mechanism as it allows interoperability between services that would >> otherwise mismatch - it's worth noting that our users expect these >> services to match, while a CS perspective regards ProteinSequence and >> ProteinSequence[] as completely different types most of our biologists >> don't! Taverna behaves the way _they_ expect it to, remember who your >> user are. >> >> Taverna data types are pretty much trivial, they're opaque data blobs >> with the exception of collection structure which is exposed. We only >> expose the collection structure to ensure the above properties, other >> than that the framework is data agnostic (as it should be). >> >> Tom >> > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From jmfernandez at cnb.uam.es Fri Jul 8 11:12:37 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Fri, 08 Jul 2005 13:12:37 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna1.2 In-Reply-To: <42CE4266.3030304@gsf.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> Message-ID: <42CE5FA5.50901@cnb.uam.es> Hi everyone, the Taverna v1.1 behaviour for MOBY should be the default one for services which only takes one input parameter. But, which collection semantics should have Taverna MOBY plugin for services with more than one parameter? All of this is related to iteration strategies. For instance, imagine this scenario: we have a service which needs two input Simple parameters, A and B. What should we do? * If both input parameters receive a Simple, no problem, default behaviour. * If one of those parameters receive a Collection with N Simples, and the other a Simple, the default iteration strategy should be either invoking the MOBY service N times with N separate SOAP submissions, or invoking the MOBY service N times with only a SOAP submission (or something intermediate). * If each one of the parameters receive a Collection, then the iteration strategy *should* be explicitly set by the workflow creator: either cartesian product (first of A with each one of B, second of A with each one of B) or dot product (first for A with first for B, second for A with second for B, and so on, either discarding the last elements from the longest Collection, or firing an exception/error/message/whatever). The other two scenarios are: a two-input service with both parameters defined as Collection; and a two-input service with one the parameters as Collection and the other as Simple. I leave them as an exercise for the reader 8-), so I'm avoiding to write now the most boring e-mail in BioMOBY story ;-) Best Regards, Jos? Mar?a Rebecca Ernst wrote: > Hi Tom and others > > This Taverna behaviour described below makes perfect sense to me! Does > Taverna then check in MOBY-Central how this service is registered > (taking simples or collections)? > > I guess the way to go would be to: > 1. change the Taverna behaviour back to how it is supposed to work (and > did in 1.1) > 2. - change the BioMoby Perl code to allow more than one simple in the > MobyData AND > - change almost all services that output collections to the way we > discussed yesterday (only output collections if the objects build an > entity. ) > > Either of the two changes solves the problems but I believe both are due > to changes and should take place. > > Best, > Rebecca > > > > > > Tom Oinn wrote: > >> That's kind of true, meaning actually not. There are three cases >> involving collections (Taverna 1.1 behaviour) : >> >> 1) Consumer declares it consumes singles, Producer emits a collection. >> In this context Taverna iteratively calls the Consumer with each item >> from the collection. This is probably what you'd expect to happen, the >> result is that the Consumer effectively emits a collection of whatever >> it would emit normally. > > >> 2) Consumer declares it consumes a collection, Producer emits a >> collection. In this case Taverna will indeed split the output >> collection (because we always do) but it will be magically reassembled >> before being given to the Consumer. >> >> 3) Consumer declares it consumes a collection, Producer emits a single >> item. Taverna wraps the single item in a single element collection and >> gives it to the Consumer. >> >> The intent is that as with the other plugin types Taverna guarantees >> that the service sees the inputs it has asked for. Our experience with >> other service types suggests that this is an extremely powerful >> mechanism as it allows interoperability between services that would >> otherwise mismatch - it's worth noting that our users expect these >> services to match, while a CS perspective regards ProteinSequence and >> ProteinSequence[] as completely different types most of our biologists >> don't! Taverna behaves the way _they_ expect it to, remember who your >> user are. >> >> Taverna data types are pretty much trivial, they're opaque data blobs >> with the exception of collection structure which is exposed. We only >> expose the collection structure to ensure the above properties, other >> than that the framework is data agnostic (as it should be). >> >> Tom >> > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From markw at illuminae.com Fri Jul 8 17:22:20 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 10:22:20 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE2229.1030504@gsf.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <42CE2229.1030504@gsf.de> Message-ID: <1120843340.15014.40.camel@mobycentral.mrl.ubc.ca> I see no inconsistency between what you quote below and what I said in my mail from yesterday... ?? M On Fri, 2005-07-08 at 08:50 +0200, Rebecca Ernst wrote: > Hi Mark! > > I will have to spank you a lot for either the last email or the one you > sent me in February where I asked you how to use collections (see mail > below): > > > Well... the answer depends on what the generic case of your output will > look like. the rule is that you have to register every output object > that your service produces; i.e. every immediate child tag of a mobyData > block. Therefore, if you are going to consistently output exactly 3 > Genbank/gi's, then you would register your service as outputting 3 X > Object(NCBI_gi). If you are outputting an indeterminate number of > objects, then you register it as outputting 1 X Collection. > > Collections map nicely onto the rdf:Bag structure, if that helps you > interpret them. > > M > > > > On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: > > >> Hi Mark! > >> > >> We have problems interpreting the BioMoby API for the output objects. > >> > >> The thing is that we have a service that gets ONE input object and > >> executes a freetext search over several databases. This service will > >> eventually return more that one result. > >> We don't want to put the result into a collection object because the > >> single objects don't build any entity. > >> We also can't give back more that one mobyData block because we only > >> have one query ID and therefore only one queryID to give back. > >> > >> The only solution we can think of is an object like this: > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> would this be a legal object or do we have to use collection even if the > >> objects don't build an entity ? > >> > >> > >> Cheers, > >> Rebecca > > > > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical > Genetics, UBC, Canada > > > ----------------------------------------------------------------------------------------------------------- > > > > > > > Mark Wilkinson wrote: > > >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > > > > > >>I myself am confused about the use of collections. Originally I had in > >>my mind that Collections were a construct to allow objects that > >>inherently belong together to be "bagged". > >> > >> > > > >You are 100% correct here. > > > > > > > > > >>The confusion starts with the output of services. My understanding was > >>that ONLY a service that is guaranteed to output exactly one object for > >>each query (e.g. an averaging service that outputs the average of a > >>list of inputs) is registered as outputting a Simple, all others have > >>to output collections > >> > >> > > > >No. > > > > > > > >> (as there must be exactly one mobyData matching > >>the queryID of the input in the response, > >> > >> > > > >Yes > > > > > > > >> and a mobyData may contain > >>multiple Simple elements only if wrapped by a Collection). > >> > >> > > > >No. A mobyData element can wrap any number of outputs, and/or > >combinations of simples, collections, and secondaries. > > > > > > > > > > > >>In practice, the Collection tag has been used to indicate when more > >>than one Simple occurs, with no "semantic" meaning. This imho is not > >>necessary; when more than one Simple occurs, why not put more than one > >>Simple? It's easy enough for everyone to figure that out. Then, > >>Collection could be used to actually transfer meaning ;-) > >> > >> > > > >I think a lot of people are using Collections incorrectly, yes. > > > > > > > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back > >>many Simple articles in response to a query very intuitively indicates > >>to continue on with each one individually, whereas getting back a > >>Collection indicates to put the whole thing as input into the next > >>service, which is what they implemented. Makes perfect sense, as there > >>can and will be services that consume Collections. > >> > >> > > > >I don't think that Taverna handled collections correctly in the past, > >and it would be a shame if that identical functionality was added back > >in :-) The functionality does need to go back into Taverna, but > >correctly this time. > > > > > > > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue > >>has led to quite some discussions between Rebecca, Dirk and myself, and > >>we are all not happy with the way they are currently handled. > >> > >> > > > >I don't know if you are happier with them now that I have pointed out > >where you are misinterpreting them...?? > > > >I agree, however, that their usage by many service providers is not > >correct... but there's no way for MOBY Central to know that it isn't > >correct, so we can't throw an error on registration of these "wonky" > >services... > > > >M > > > > > > > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 17:25:33 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 10:25:33 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <1120843533.15014.43.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 09:07 +0100, Tom Oinn wrote: > 1) Consumer declares it consumes singles, Producer emits a collection. > 2) Consumer declares it consumes a collection, Producer emits a > 3) Consumer declares it consumes a collection, Producer emits a single So the only case that is missing is where the Consumer declares it consumes a collection, the producer emits a single, but you want to "bulk" those simples (or even simples arising from disparate upstream service outputs) into a single input collection. I believe you said you had a widget that would do that already, but I can't remember which one it was...?? M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 17:25:33 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 10:25:33 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <42CE3450.1000209@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz-koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> Message-ID: <1120843533.15014.43.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 09:07 +0100, Tom Oinn wrote: > 1) Consumer declares it consumes singles, Producer emits a collection. > 2) Consumer declares it consumes a collection, Producer emits a > 3) Consumer declares it consumes a collection, Producer emits a single So the only case that is missing is where the Consumer declares it consumes a collection, the producer emits a single, but you want to "bulk" those simples (or even simples arising from disparate upstream service outputs) into a single input collection. I believe you said you had a widget that would do that already, but I can't remember which one it was...?? M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 17:37:09 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 10:37:09 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby services in Taverna 1.2 In-Reply-To: <200507081049.58799.d.haase@gsf.de> References: <42CD31FF.6040803@gsf.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e55afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <200507081049.58799.d.haase@gsf.de> Message-ID: <1120844229.15014.53.camel@mobycentral.mrl.ubc.ca> I agree that the CommonSubs code does not facilitate every possibility allowed by the MOBY API, but the API should be considered canonical :-) I think that CommonSubs needs to be **completely** re-written, and in fact, this was one of the issues that came up at the last meeting - that we need to have some common implementation-independent API that we can all code to such that all possible message types can be intuitively created and unpacked in any language. Though I don't object to you adding that subroutine to the CVS (I haven't checked it for sanity, only for intent) I think it would be useful for us to operate on a higher-level at some point soon and decide what functions we *need*, and then decide how to build them, rather than adding functions on an ad hoc basis as we have been so far. For example, I might be tempted to have a function that built service responses in the following way: $OUT = CommonSubs::ServiceOutput->new(some header information - service provision info goes here); $OUT->addOutput(invocation_id => $id, simple => $simple); $OUT->addOutput(invocation_id => $id, collection => \@simples); $OUT->addOutput(invocation_id => $id2, simple => $simple2); etc. etc. That would cover all possibilities and would be more intuitive than the existing mechanism... ??? He who codes, wins! M On Fri, 2005-07-08 at 10:49 +0200, Dirk Haase wrote: > On Friday 08 July 2005 08:45, Heiko Schoof wrote: > > Well, if I understand you correctly, Mark, than I was right all along > > in that they way I see collections is the way they should be seen. Then > > the problem is that that is not clear enough to everybody else, and > > that possibly the examples in the API are misleading, because it's been > > used in such a different way. It's definitely something that needs to > > be fixed before release 1.0. > > Not only the examples are misleading, actually at least the PERL API is > lacking a suitable method to construct an output consisting of several > 'Simple's. One can not simply concat the output of multibple 'simpleResponse' > calls, as this would create several mobyData blocks. > > In order to fill this gap, I created a new method 'multiSimpleResponse' (see > below) within CommonSubs.pm which is more or less identical to > 'collectionResponse' except that it obviously leaves out the moby:Collection > tag. The provided articleName is put into each 'Simple' tag which should be > appropriate. > > Any objections? > > Regards, > dirk > > =head2 multiSimpleResponse > > name : multiSimpleResponse > function : wraps a set of simple articles in the appropriate mobyData > structure > usage : return responseHeader . &multiSimpleResponse(\@objects, > 'MyArticleName', $queryID) . responseFooter; > args : (in order) > \@objects - (optional) a listref of MOBY Objects as raw XML > $article - (optional) an articeName for this article > $queryID - (optional, but strongly recommended) the mobyData ID > to which you are responding > notes : as required by the API you must return a response for every input. > If one of the inputs was invalid, you return a valid (empty) MOBY > response by calling &multiSimpleResponse(undef, undef, $queryID). > > =cut > > > sub multiSimpleResponse { > my ( $data, $articleName, $qID ) = @_; # articleName optional > my $content = ""; > $data ||= []; > $qID ||= ''; > $articleName ||=""; > unless ( ( ref( $data ) =~ /array/i ) && $data->[0] ) > { # we're expecting an arrayref as input data,and it must not be > empty > return ""; > } > foreach ( @{$data} ) { > if ( $_ ) { > $content .= " > $_ > "; > } else { > $content .= " > > "; > } > } > return " > > $content > > "; > } > > > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From gordonp at ucalgary.ca Fri Jul 8 18:26:25 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Fri, 08 Jul 2005 12:26:25 -0600 Subject: [MOBY-dev] API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42CE5FA5.50901@cnb.uam.es> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> Message-ID: <42CEC551.4090108@ucalgary.ca> Hi everyone, At long last after our discussions at the last MOBY meeting in Vancouver, I have written a little document explaining the reason for and consequences of not inheriting from "primitive" objects such as String and Integer. I think that this is a distruptive change for only a few services, but is of great benefit. Please see http://coe02.ucalgary.ca/gordonp/moby_primitives.html Mark has suggested that this not be a formal change to the API, but that this new way of doing things be encouraged in all of the documentation, and the old method explicitly deprecated but not disabled. Hopefully most of you agree, and we can just make temporary exceptions in our code for the current non-compliant classes. Or even better, if most people agree, we can force the affected services to change over. :-) P.S. Yan, your object browser came in quite handy while I was writing this, nice work! One suggestion though, arranging the objects alphabetically in the tree (when within the same branch and level) would help ease navigation. P.P.S. Our Web server will be down briefly this afternoon, so you may get a timeout/cannot connect on the link. From d.haase at gsf.de Fri Jul 8 18:45:39 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Fri, 8 Jul 2005 20:45:39 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Hi Mark, I would not call it inconsistency, they totally contradict ;-) Consider a service which returns an unpredictable number of database entries. The result set does not form any meaningful construct, each member is just an alternative answer to the query posed. Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. If I summed up both replies correctly, the contradiction is obvious, isn't it? dirk -----Original Message----- From: moby-dev-bounces at portal.open-bio.org on behalf of Mark Wilkinson Sent: Fri 7/8/2005 7:22 PM To: Ernst, Rebecca Cc: Martin Senger; mobydev Subject: Re: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 I see no inconsistency between what you quote below and what I said in my mail from yesterday... ?? M On Fri, 2005-07-08 at 08:50 +0200, Rebecca Ernst wrote: > Hi Mark! > > I will have to spank you a lot for either the last email or the one you > sent me in February where I asked you how to use collections (see mail > below): > > > Well... the answer depends on what the generic case of your output will > look like. the rule is that you have to register every output object > that your service produces; i.e. every immediate child tag of a mobyData > block. Therefore, if you are going to consistently output exactly 3 > Genbank/gi's, then you would register your service as outputting 3 X > Object(NCBI_gi). If you are outputting an indeterminate number of > objects, then you register it as outputting 1 X Collection. > > Collections map nicely onto the rdf:Bag structure, if that helps you > interpret them. > > M > > > > On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: > > >> Hi Mark! > >> > >> We have problems interpreting the BioMoby API for the output objects. > >> > >> The thing is that we have a service that gets ONE input object and > >> executes a freetext search over several databases. This service will > >> eventually return more that one result. > >> We don't want to put the result into a collection object because the > >> single objects don't build any entity. > >> We also can't give back more that one mobyData block because we only > >> have one query ID and therefore only one queryID to give back. > >> > >> The only solution we can think of is an object like this: > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> would this be a legal object or do we have to use collection even if the > >> objects don't build an entity ? > >> > >> > >> Cheers, > >> Rebecca > > > > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical > Genetics, UBC, Canada > > > ----------------------------------------------------------------------------------------------------------- > > > > > > > Mark Wilkinson wrote: > > >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > > > > > >>I myself am confused about the use of collections. Originally I had in > >>my mind that Collections were a construct to allow objects that > >>inherently belong together to be "bagged". > >> > >> > > > >You are 100% correct here. > > > > > > > > > >>The confusion starts with the output of services. My understanding was > >>that ONLY a service that is guaranteed to output exactly one object for > >>each query (e.g. an averaging service that outputs the average of a > >>list of inputs) is registered as outputting a Simple, all others have > >>to output collections > >> > >> > > > >No. > > > > > > > >> (as there must be exactly one mobyData matching > >>the queryID of the input in the response, > >> > >> > > > >Yes > > > > > > > >> and a mobyData may contain > >>multiple Simple elements only if wrapped by a Collection). > >> > >> > > > >No. A mobyData element can wrap any number of outputs, and/or > >combinations of simples, collections, and secondaries. > > > > > > > > > > > >>In practice, the Collection tag has been used to indicate when more > >>than one Simple occurs, with no "semantic" meaning. This imho is not > >>necessary; when more than one Simple occurs, why not put more than one > >>Simple? It's easy enough for everyone to figure that out. Then, > >>Collection could be used to actually transfer meaning ;-) > >> > >> > > > >I think a lot of people are using Collections incorrectly, yes. > > > > > > > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back > >>many Simple articles in response to a query very intuitively indicates > >>to continue on with each one individually, whereas getting back a > >>Collection indicates to put the whole thing as input into the next > >>service, which is what they implemented. Makes perfect sense, as there > >>can and will be services that consume Collections. > >> > >> > > > >I don't think that Taverna handled collections correctly in the past, > >and it would be a shame if that identical functionality was added back > >in :-) The functionality does need to go back into Taverna, but > >correctly this time. > > > > > > > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue > >>has led to quite some discussions between Rebecca, Dirk and myself, and > >>we are all not happy with the way they are currently handled. > >> > >> > > > >I don't know if you are happier with them now that I have pointed out > >where you are misinterpreting them...?? > > > >I agree, however, that their usage by many service providers is not > >correct... but there's no way for MOBY Central to know that it isn't > >correct, so we can't throw an error on registration of these "wonky" > >services... > > > >M > > > > > > > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From d.haase at gsf.de Fri Jul 8 18:45:39 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Fri, 8 Jul 2005 20:45:39 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Hi Mark, I would not call it inconsistency, they totally contradict ;-) Consider a service which returns an unpredictable number of database entries. The result set does not form any meaningful construct, each member is just an alternative answer to the query posed. Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. If I summed up both replies correctly, the contradiction is obvious, isn't it? dirk -----Original Message----- From: moby-dev-bounces at portal.open-bio.org on behalf of Mark Wilkinson Sent: Fri 7/8/2005 7:22 PM To: Ernst, Rebecca Cc: Martin Senger; mobydev Subject: Re: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 I see no inconsistency between what you quote below and what I said in my mail from yesterday... ?? M On Fri, 2005-07-08 at 08:50 +0200, Rebecca Ernst wrote: > Hi Mark! > > I will have to spank you a lot for either the last email or the one you > sent me in February where I asked you how to use collections (see mail > below): > > > Well... the answer depends on what the generic case of your output will > look like. the rule is that you have to register every output object > that your service produces; i.e. every immediate child tag of a mobyData > block. Therefore, if you are going to consistently output exactly 3 > Genbank/gi's, then you would register your service as outputting 3 X > Object(NCBI_gi). If you are outputting an indeterminate number of > objects, then you register it as outputting 1 X Collection. > > Collections map nicely onto the rdf:Bag structure, if that helps you > interpret them. > > M > > > > On Wed, 2005-02-02 at 02:36, Rebecca Ernst wrote: > > >> Hi Mark! > >> > >> We have problems interpreting the BioMoby API for the output objects. > >> > >> The thing is that we have a service that gets ONE input object and > >> executes a freetext search over several databases. This service will > >> eventually return more that one result. > >> We don't want to put the result into a collection object because the > >> single objects don't build any entity. > >> We also can't give back more that one mobyData block because we only > >> have one query ID and therefore only one queryID to give back. > >> > >> The only solution we can think of is an object like this: > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> would this be a legal object or do we have to use collection even if the > >> objects don't build an entity ? > >> > >> > >> Cheers, > >> Rebecca > > > > > -- Mark Wilkinson Assistant Professor (Bioinformatics) Dept. Medical > Genetics, UBC, Canada > > > ----------------------------------------------------------------------------------------------------------- > > > > > > > Mark Wilkinson wrote: > > >On Thu, 2005-07-07 at 17:34 +0200, Heiko Schoof wrote: > > > > > > > >>I myself am confused about the use of collections. Originally I had in > >>my mind that Collections were a construct to allow objects that > >>inherently belong together to be "bagged". > >> > >> > > > >You are 100% correct here. > > > > > > > > > >>The confusion starts with the output of services. My understanding was > >>that ONLY a service that is guaranteed to output exactly one object for > >>each query (e.g. an averaging service that outputs the average of a > >>list of inputs) is registered as outputting a Simple, all others have > >>to output collections > >> > >> > > > >No. > > > > > > > >> (as there must be exactly one mobyData matching > >>the queryID of the input in the response, > >> > >> > > > >Yes > > > > > > > >> and a mobyData may contain > >>multiple Simple elements only if wrapped by a Collection). > >> > >> > > > >No. A mobyData element can wrap any number of outputs, and/or > >combinations of simples, collections, and secondaries. > > > > > > > > > > > >>In practice, the Collection tag has been used to indicate when more > >>than one Simple occurs, with no "semantic" meaning. This imho is not > >>necessary; when more than one Simple occurs, why not put more than one > >>Simple? It's easy enough for everyone to figure that out. Then, > >>Collection could be used to actually transfer meaning ;-) > >> > >> > > > >I think a lot of people are using Collections incorrectly, yes. > > > > > > > > > > > >>make things clearer, also to e.g. the Taverna developers: Getting back > >>many Simple articles in response to a query very intuitively indicates > >>to continue on with each one individually, whereas getting back a > >>Collection indicates to put the whole thing as input into the next > >>service, which is what they implemented. Makes perfect sense, as there > >>can and will be services that consume Collections. > >> > >> > > > >I don't think that Taverna handled collections correctly in the past, > >and it would be a shame if that identical functionality was added back > >in :-) The functionality does need to go back into Taverna, but > >correctly this time. > > > > > > > > > > > >>Maybe I've made myself clear, maybe not. Anyway, the Collection issue > >>has led to quite some discussions between Rebecca, Dirk and myself, and > >>we are all not happy with the way they are currently handled. > >> > >> > > > >I don't know if you are happier with them now that I have pointed out > >where you are misinterpreting them...?? > > > >I agree, however, that their usage by many service providers is not > >correct... but there's no way for MOBY Central to know that it isn't > >correct, so we can't throw an error on registration of these "wonky" > >services... > > > >M > > > > > > > > > > > -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 6191 bytes Desc: not available URL: From markw at illuminae.com Fri Jul 8 18:58:30 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 11:58:30 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Message-ID: <1120849110.15014.101.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 20:45 +0200, Haase, Dirk wrote: > Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. What I said in response to Heiko (or at least, what I intended to say) was that there was not a limitation of the mobyData block to having just one child element (regardless of its type - simple, collection, or secondary). So the following is vaid: I didn't say that you SHOULD return all of your Simples individually, only that you COULD return individual Simples in a single output... if that was meaningful in the context of your service... individual outputs must, however, be individually registered in the registry (output => [A,B,Q]) > But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. That's right.... and obvious, since you cannot create a service signature any other way - you can't say output => [A....infinite] > If I summed up both replies correctly, the contradiction is obvious, isn't it? I don't see it. M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From markw at illuminae.com Fri Jul 8 18:58:30 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 11:58:30 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B289@sw-rz010.gsf.de> Message-ID: <1120849110.15014.101.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 20:45 +0200, Haase, Dirk wrote: > Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. What I said in response to Heiko (or at least, what I intended to say) was that there was not a limitation of the mobyData block to having just one child element (regardless of its type - simple, collection, or secondary). So the following is vaid: I didn't say that you SHOULD return all of your Simples individually, only that you COULD return individual Simples in a single output... if that was meaningful in the context of your service... individual outputs must, however, be individually registered in the registry (output => [A,B,Q]) > But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. That's right.... and obvious, since you cannot create a service signature any other way - you can't say output => [A....infinite] > If I summed up both replies correctly, the contradiction is obvious, isn't it? I don't see it. M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From d.haase at gsf.de Fri Jul 8 19:43:41 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Fri, 8 Jul 2005 21:43:41 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> OK, things become clearer I think. You didn't really get what Heiko was suggesting about the usage of collections... Heiko, Rebecca and me think that a collection is ueseful _only_ if the collection itself is a meaningful entity (example: several sequences make up a cluster). In our view it does not make sense to pack otherwise unrelated things into a collection just because they are all results for a certain query. I admit that our understanding is not compatible with the current way for registering services. The easy(?) solution would be to abandon any cardinality confirmations from the registry. That is if a service is registered with an output A it may output any number of A-objects. Another way to tackle this would be to introduce a sort of 'has/has-a' for service outputs, analogous to the object definitions. dirk -----Original Message----- From: moby-dev-bounces at portal.open-bio.org on behalf of Mark Wilkinson Sent: Fri 7/8/2005 8:58 PM To: Core developer announcements Cc: Ernst, Rebecca; Martin Senger; mobydev Subject: RE: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 On Fri, 2005-07-08 at 20:45 +0200, Haase, Dirk wrote: > Once you say (in the reply to Heiko's mail): you can safely return each entry as a Simple element, there can be any number of them in one mobyData block. Moreover, this would actually be a misuse of a collection. What I said in response to Heiko (or at least, what I intended to say) was that there was not a limitation of the mobyData block to having just one child element (regardless of its type - simple, collection, or secondary). So the following is vaid: I didn't say that you SHOULD return all of your Simples individually, only that you COULD return individual Simples in a single output... if that was meaningful in the context of your service... individual outputs must, however, be individually registered in the registry (output => [A,B,Q]) > But in the reply to Rebecca's question from February you clearly stated: if the number of results is unpredictable they have to go into a collection. That's right.... and obvious, since you cannot create a service signature any other way - you can't say output => [A....infinite] > If I summed up both replies correctly, the contradiction is obvious, isn't it? I don't see it. M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 4339 bytes Desc: not available URL: From markw at illuminae.com Fri Jul 8 20:07:25 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 13:07:25 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> Message-ID: <1120853245.15014.131.camel@mobycentral.mrl.ubc.ca> On Fri, 2005-07-08 at 21:43 +0200, Haase, Dirk wrote: > Heiko, Rebecca and me think that a collection is ueseful _only_ if the collection > itself is a meaningful entity (example: several sequences make up a cluster). > In our view it does not make sense to pack otherwise unrelated things into a > collection just because they are all results for a certain query. I agree 100%... so again, I still don't see the difference between what you are saying and what I am saying...?? M -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From jmfernandez at cnb.uam.es Fri Jul 8 21:55:44 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Fri, 08 Jul 2005 23:55:44 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> References: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> Message-ID: <42CEF660.8010602@cnb.uam.es> Hi everybody, I had almost forgotten a MOBY service (like a PL/SQL or ADA procedure) can have more than one output, each one belonging to its own type. I forgot it for one of the services I designed for INB. > > I admit that our understanding is not compatible with the current way > for registering services. The easy(?) solution would be to abandon > any cardinality confirmations from the registry. That is if a service > is registered with an output A it may output any number of A-objects. > I have in mind two or three bioinformatics time-expensive algorithms which generate more than one output (with different types) each time they are run. Because they are related, the common developer wraps them into an object, whose type inherits from Object, and most of the time it is an artificial creation. Those coders must also develop micro-services which pull the needed piece of information from the wrap, or if you are a Taverna user something like a beanshell. So, I agree with Mark: services with more than one output have their place. But remember: we need clients which are able to deal with this feature. Until now we had no one, but now Taverna could be the first one... Best regards, Jos? Mar?a > > Another way to tackle this would be to introduce a sort of > 'has/has-a' for service outputs, analogous to the object definitions. > > > dirk > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From markw at illuminae.com Fri Jul 8 22:00:33 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 08 Jul 2005 15:00:33 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <42CEF660.8010602@cnb.uam.es> References: <8306780C060A7D4790F8BA02CBA1B69104B28A@sw-rz010.gsf.de> <42CEF660.8010602@cnb.uam.es> Message-ID: <1120860033.3721.7.camel@bioinfo.icapture.ubc.ca> Yeah... it's taking a long time for the tooling to catch up with the API. Just a lack of resources :-( M On Fri, 2005-07-08 at 23:55 +0200, Jos? Mar?a Fern?ndez Gonz?lez wrote: > So, I agree with Mark: > services with more than one output have their place. But remember: we > need clients which are able to deal with this feature. Until now we had > no one, but now Taverna could be the first one... -- Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Vancouver, BC From d.haase at gsf.de Sat Jul 9 09:02:01 2005 From: d.haase at gsf.de (Haase, Dirk) Date: Sat, 9 Jul 2005 11:02:01 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Message-ID: <8306780C060A7D4790F8BA02CBA1B69104B28B@sw-rz010.gsf.de> Hi Jose Maria, Mark and others, I was not talking about sophisticated services with unpredictable type(s) of output, only about plain stupid lookup services where the output type is fix, but the number of result objects unknown. Like 'getAGILocusCodesByKeyword' for example. I'll try to explain it in other words... Why not making it the default behaviour that services can output 1 or many objects of the type it is registered for and save the 'collection' construct for situations where several output objects (of same of differing types) constitute a semantically meaningful complex. Does anybody understand what I'm trying to say?? dirk -----Original Message----- From: moby-dev-bounces at portal.open-bio.org on behalf of Jos? Mar?a Fern?ndez Gonz?lez Sent: Fri 7/8/2005 11:55 PM To: Core developer announcements Cc: markw at illuminae.com Subject: Re: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 Hi everybody, I had almost forgotten a MOBY service (like a PL/SQL or ADA procedure) can have more than one output, each one belonging to its own type. I forgot it for one of the services I designed for INB. > > I admit that our understanding is not compatible with the current way > for registering services. The easy(?) solution would be to abandon > any cardinality confirmations from the registry. That is if a service > is registered with an output A it may output any number of A-objects. > I have in mind two or three bioinformatics time-expensive algorithms which generate more than one output (with different types) each time they are run. Because they are related, the common developer wraps them into an object, whose type inherits from Object, and most of the time it is an artificial creation. Those coders must also develop micro-services which pull the needed piece of information from the wrap, or if you are a Taverna user something like a beanshell. So, I agree with Mark: services with more than one output have their place. But remember: we need clients which are able to deal with this feature. Until now we had no one, but now Taverna could be the first one... Best regards, Jos? Mar?a > > Another way to tackle this would be to introduce a sort of > 'has/has-a' for service outputs, analogous to the object definitions. > > > dirk > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 4479 bytes Desc: not available URL: From chris at bioteam.net Sat Jul 9 17:40:52 2005 From: chris at bioteam.net (Chris Dagdigian) Date: Sat, 9 Jul 2005 13:40:52 -0400 Subject: [MOBY-dev] DNS changes for biomoby.org beginning today - should have no noticeable effect Message-ID: Hi folks, All of open-bio DNS is outsourced to a third party except for biomoby.org for which we manage the DNS zone ourselves due to moby requirements for specialized SRV zone entries that our other provider can't handle. The existing primary and secondary nameservers for biomoby.org are: 7200 IN NS portal.open-bio.org. 7200 IN NS pub.open-bio.org. These also happen to be the same servers we are currently trying to "retire" in favor of newer servers running in a different datacenter colocation cage. Last night I copied the biomoby.org zone file to our two new servers and activated service for them. The new machines are: (1) newportal.open-bio.org (2) dev.open-bio.org This morning I edited the master nameserver list that NetworkSolutions holds for the domain to add those new servers at the top of the list. Biomoby now has 4 dns servers; listed in the following order: 7200 IN NS newportal.open-bio.org. 7200 IN NS dev.open-bio.org. 7200 IN NS portal.open-bio.org. 7200 IN NS pub.open-bio.org. This means that over the next 24-72 hours as various DNS caches expire, biomoby.org will start to be serviced from the new pair of nameservers with the "old" pair acting as backup. If this works smoothly I'll leave it alone for 1-week. After which I plan on editing the NetworkSolution records to remove the "old" servers from the list. That should complete the full transition from the old primary/secondary pair to the new pair. Because of the overlap (both new and old server pairs are serving up the same info for biomoby.org) I don't expect anyone or any MOBY service user to experience any problems. I just wanted to mention the DNS change in case a problem does occur. For the DNS savvy on this list, feel free to query the biomoby.org zone on the new machines to confirm that the info is being served correctly. Your zone currently looks like this: > $ORIGIN org. > biomoby 7200 IN SOA portal.open-bio.org. root-l.open- > bio.org. ( > 0705200503 114400 7200 950400 7200 ) > 7200 IN NS newportal.open-bio.org. > 7200 IN NS dev.open-bio.org. > 7200 IN NS portal.open-bio.org. > 7200 IN NS pub.open-bio.org. > 7200 IN A 65.246.187.176 > 7200 IN MX 10 biomoby.org. > $ORIGIN biomoby.org. > portal 7200 IN A 65.246.187.176 > www 7200 IN CNAME biomoby.org. > cvs 7200 IN A 65.246.187.182 > 7200 IN MX 10 biomoby.org. > _lsid._tcp IN SRV 1 0 80 mobycentral.icapture.ubc.ca. Regards, Chris From schoof at mpiz-koeln.mpg.de Wed Jul 13 12:27:42 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Wed, 13 Jul 2005 14:27:42 +0200 Subject: [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42CEC551.4090108@ucalgary.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca><46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de><42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> Message-ID: <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> Hi Paul, thanks for the nice writeup. I agree. I just want to stress that I find one of the consequences you mention a very important argument for doing this: Services that act on Strings will no longer automatically be discovered starting from e.g. NCBI_BLAST_OUTPUT or DNA_SEQUENCE, but only when using object decomposition. Best, Heiko On 8. Jul 2005, at 20:26 Uhr, Paul Gordon wrote: Hi everyone, At long last after our discussions at the last MOBY meeting in Vancouver, I have written a little document explaining the reason for and consequences of not inheriting from "primitive" objects such as String and Integer. I think that this is a distruptive change for only a few services, but is of great benefit. Please see http://coe02.ucalgary.ca/gordonp/moby_primitives.html Mark has suggested that this not be a formal change to the API, but that this new way of doing things be encouraged in all of the documentation, and the old method explicitly deprecated but not disabled. Hopefully most of you agree, and we can just make temporary exceptions in our code for the current non-compliant classes. Or even better, if most people agree, we can force the affected services to change over. :-) P.S. Yan, your object browser came in quite handy while I was writing this, nice work! One suggestion though, arranging the objects alphabetically in the tree (when within the same branch and level) would help ease navigation. P.P.S. Our Web server will be down briefly this afternoon, so you may get a timeout/cannot connect on the link. _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From markw at illuminae.com Wed Jul 13 15:21:13 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 13 Jul 2005 08:21:13 -0700 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> Message-ID: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> The "virtual decomposition" of objects that MOBY does during service discovery can be switched on/off. Switching it off will prevent this behaviour also if it is an immediate concern. M On Wed, 2005-07-13 at 14:27 +0200, Heiko Schoof wrote: > Hi Paul, > thanks for the nice writeup. I agree. > > I just want to stress that I find one of the consequences you mention a > very important argument for doing this: Services that act on Strings > will no longer automatically be discovered starting from e.g. > NCBI_BLAST_OUTPUT or DNA_SEQUENCE, but only when using object > decomposition. > > Best, Heiko > > On 8. Jul 2005, at 20:26 Uhr, Paul Gordon wrote: > > Hi everyone, > > At long last after our discussions at the last MOBY meeting in > Vancouver, I have written a little document explaining the reason for > and consequences of not inheriting from "primitive" objects such as > String and Integer. I think that this is a distruptive change for only > a few services, but is of great benefit. Please see > > http://coe02.ucalgary.ca/gordonp/moby_primitives.html > > Mark has suggested that this not be a formal change to the API, but > that this new way of doing things be encouraged in all of the > documentation, and the old method explicitly deprecated but not > disabled. Hopefully most of you agree, and we can just make temporary > exceptions in our code for the current non-compliant classes. Or even > better, if most people agree, we can force the affected services to > change over. :-) > > P.S. Yan, your object browser came in quite handy while I was writing > this, nice work! One suggestion though, arranging the objects > alphabetically in the tree (when within the same branch and level) > would help ease navigation. > > P.P.S. Our Web server will be down briefly this afternoon, so you may > get a timeout/cannot connect on the link. > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From markw at illuminae.com Wed Jul 13 15:21:13 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 13 Jul 2005 08:21:13 -0700 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> Message-ID: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> The "virtual decomposition" of objects that MOBY does during service discovery can be switched on/off. Switching it off will prevent this behaviour also if it is an immediate concern. M On Wed, 2005-07-13 at 14:27 +0200, Heiko Schoof wrote: > Hi Paul, > thanks for the nice writeup. I agree. > > I just want to stress that I find one of the consequences you mention a > very important argument for doing this: Services that act on Strings > will no longer automatically be discovered starting from e.g. > NCBI_BLAST_OUTPUT or DNA_SEQUENCE, but only when using object > decomposition. > > Best, Heiko > > On 8. Jul 2005, at 20:26 Uhr, Paul Gordon wrote: > > Hi everyone, > > At long last after our discussions at the last MOBY meeting in > Vancouver, I have written a little document explaining the reason for > and consequences of not inheriting from "primitive" objects such as > String and Integer. I think that this is a distruptive change for only > a few services, but is of great benefit. Please see > > http://coe02.ucalgary.ca/gordonp/moby_primitives.html > > Mark has suggested that this not be a formal change to the API, but > that this new way of doing things be encouraged in all of the > documentation, and the old method explicitly deprecated but not > disabled. Hopefully most of you agree, and we can just make temporary > exceptions in our code for the current non-compliant classes. Or even > better, if most people agree, we can force the affected services to > change over. :-) > > P.S. Yan, your object browser came in quite handy while I was writing > this, nice work! One suggestion though, arranging the objects > alphabetically in the tree (when within the same branch and level) > would help ease navigation. > > P.P.S. Our Web server will be down briefly this afternoon, so you may > get a timeout/cannot connect on the link. > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From gordonp at ucalgary.ca Wed Jul 13 15:50:33 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed, 13 Jul 2005 09:50:33 -0600 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53849.7010700@ucalgary.ca> Mark, We are talking about decomposition. What you are talking about is inheritence. You can switch off finding services that match parent objects, of course. But an annotated base64 encoded GIF under the new scheme would NEVER match something that takes String as input, no matter how you set the service finding flag. The CLIENT would need to decompose (probably at the user's request) the GIF object into its constituent String members, such as the "contents" or the "description", and search MOBY central using JUST that member. That's what we mean by decomposition. >The "virtual decomposition" of objects that MOBY does during service >discovery can be switched on/off. Switching it off will prevent this >behaviour also if it is an immediate concern. > > > > From gordonp at ucalgary.ca Wed Jul 13 15:50:33 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed, 13 Jul 2005 09:50:33 -0600 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53849.7010700@ucalgary.ca> Mark, We are talking about decomposition. What you are talking about is inheritence. You can switch off finding services that match parent objects, of course. But an annotated base64 encoded GIF under the new scheme would NEVER match something that takes String as input, no matter how you set the service finding flag. The CLIENT would need to decompose (probably at the user's request) the GIF object into its constituent String members, such as the "contents" or the "description", and search MOBY central using JUST that member. That's what we mean by decomposition. >The "virtual decomposition" of objects that MOBY does during service >discovery can be switched on/off. Switching it off will prevent this >behaviour also if it is an immediate concern. > > > > From Pieter.Neerincx at wur.nl Wed Jul 13 15:53:41 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Wed, 13 Jul 2005 17:53:41 +0200 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: Hi all, Is there anyone out there that was able to make BioMOBY (realated) Java tools like Yan's Moby Object Browser or Taverna work with MOBY Centrals and/or services over HTTPS? I'm having trouble with the server's certificate. I used keytool to import the server's certificate client-side, but the Java tools keep on complaining: "unknown certificate"... Cheers, Pieter From tmo at ebi.ac.uk Wed Jul 13 16:05:55 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Wed, 13 Jul 2005 17:05:55 +0100 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53BE3.7000909@ebi.ac.uk> Hi all, We've had Taverna working for HTTPS web services so it should be fine for the MOBY operations. You need to check that the certificate has a valid signature chain and that the name of the entity in the certificate matches the hostname - we had issues with a certificate from a server in china that the java security framework rejected because the (I think) CN part of the DN in the cert wasn't the same as the hostname of the endpoint. Tom Pieter Neerincx wrote: > Hi all, > > Is there anyone out there that was able to make BioMOBY (realated) Java > tools like Yan's Moby Object Browser or Taverna work with MOBY Centrals > and/or services over HTTPS? I'm having trouble with the server's > certificate. I used keytool to import the server's certificate > client-side, but the Java tools keep on complaining: "unknown > certificate"... > > Cheers, > > Pieter _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev From tmo at ebi.ac.uk Wed Jul 13 16:05:55 2005 From: tmo at ebi.ac.uk (Tom Oinn) Date: Wed, 13 Jul 2005 17:05:55 +0100 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> Message-ID: <42D53BE3.7000909@ebi.ac.uk> Hi all, We've had Taverna working for HTTPS web services so it should be fine for the MOBY operations. You need to check that the certificate has a valid signature chain and that the name of the entity in the certificate matches the hostname - we had issues with a certificate from a server in china that the java security framework rejected because the (I think) CN part of the DN in the cert wasn't the same as the hostname of the endpoint. Tom Pieter Neerincx wrote: > Hi all, > > Is there anyone out there that was able to make BioMOBY (realated) Java > tools like Yan's Moby Object Browser or Taverna work with MOBY Centrals > and/or services over HTTPS? I'm having trouble with the server's > certificate. I used keytool to import the server's certificate > client-side, but the Java tools keep on complaining: "unknown > certificate"... > > Cheers, > > Pieter _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev From markw at illuminae.com Wed Jul 13 16:16:27 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 13 Jul 2005 09:16:27 -0700 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42D53849.7010700@ucalgary.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> <42D53849.7010700@ucalgary.ca> Message-ID: <1121271387.6327.45.camel@bioinfo.icapture.ubc.ca> On Wed, 2005-07-13 at 09:50 -0600, Paul Gordon wrote: > We are talking about decomposition. What you are talking about is > inheritence. Yes, but I'm just saying that the *current* behaviour that Heiko was objecting to is due to inheritence, and the behaviour can be switched off if it is *immediately* troublesome. Of course, it will also disappear if we start to use a composition model rather than an inheritence model to put e.g. strings into objects, but that doesn't help anyone in the short-term. M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From markw at illuminae.com Wed Jul 13 16:16:27 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Wed, 13 Jul 2005 09:16:27 -0700 Subject: [moby] [MOBY-dev] Re: API Change/Recommendation/Best-Practice/Meme Proposal In-Reply-To: <42D53849.7010700@ucalgary.ca> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> <42D53849.7010700@ucalgary.ca> Message-ID: <1121271387.6327.45.camel@bioinfo.icapture.ubc.ca> On Wed, 2005-07-13 at 09:50 -0600, Paul Gordon wrote: > We are talking about decomposition. What you are talking about is > inheritence. Yes, but I'm just saying that the *current* behaviour that Heiko was objecting to is due to inheritence, and the behaviour can be switched off if it is *immediately* troublesome. Of course, it will also disappear if we start to use a composition model rather than an inheritence model to put e.g. strings into objects, but that doesn't help anyone in the short-term. M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From Pieter.Neerincx at wur.nl Thu Jul 14 14:02:34 2005 From: Pieter.Neerincx at wur.nl (Pieter Neerincx) Date: Thu, 14 Jul 2005 16:02:34 +0200 Subject: [MOBY-dev] Java tools and BioMOBY over HTTPS? In-Reply-To: <42D53BE3.7000909@ebi.ac.uk> References: <42CD31FF.6040803@gsf.de> <4dbc46af77487aef3b43ee2d682600fa@mpiz -koeln.mpg.de> <1120766868.12900.40.camel@mobycentral.mrl.ubc.ca> <46b5ae1e5 5afaed2de2fd80b0ad8219c@mpiz-koeln.mpg.de> <42CE3450.1000209@ebi.ac.uk> <42CE4266.3030304@gsf.de> <42CE5FA5.50901@cnb.uam.es> <42CEC551.4090108@ucalgary.ca> <5138d2e1605a0c8e8f1ae47add89dd01@mpiz-koeln.mpg.de> <1121268073.6327.17.camel@bioinfo.icapture.ubc.ca> <42D53BE3.7000909@ebi.ac.uk> Message-ID: <75FEE7B6-A0DE-4444-B27C-7C7C4F478FB9@wur.nl> Hi Tom, Thanks for the reply. No I at least knew it should be possible :). After googeling for days I finally got it to work. Maybe you can add a small addition for HTTPS to the configuration part of the Taverna manual. If anyone else ever runs into the same problem here is how I got it to work: I did find a lot of posts mentioning compatibility problems when importing openssl generated certificates in a java keystore with keytool. So I was converting my certificates in probably every certificate format known to man, but that was not the solution. In order to make the Java client accept a self-signed certificate from you webserver this is what needs to be done. Generate a self-signed certificate. If you use openssl this step is pretty well documented. The default output will be a certificate in pem format. The default extension for such a certificate is *.crt. This file can contain both a human readable form of the certificate and an encoded one. So it might look something like this: Certificate: Data: Version: 3 (0x2) Serial Number: 0 (0x0) Signature Algorithm: md5WithRSAEncryption Issuer: C=NL, ST=Gelderland, L=Wageningen, O=WUR, OU=Bioinformatics, CN=dev.bioinformatics.nl/ emailAddress=jack.leunissen at wur.nl Validity Not Before: Jul 8 16:27:14 2005 GMT Not After : Aug 7 16:27:14 2005 GMT Subject: C=NL, ST=Gelderland, L=Wageningen, O=WUR, OU=Bioinformatics, CN=dev.bioinformatics.nl/ emailAddress=jack.leunissen at wur.nl Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (1024 bit) Modulus (1024 bit): 00:a1:e2:f2:20:3c:17:da:c1:2c:d9:89:4b:9d:16: ........ 80:a9:e4:02:72:3c:1f:b3:3d Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Key Identifier: 87:F7:8D:B6:63:C1:F9:4D: X509v3 Authority Key Identifier: keyid:87:F7:8D:B6:63:C1:F9: DirName:/C=NL/ST=Gelderland/L=Wageningen/O=WUR/ OU=Bioinformatics/CN=dev.bioinformatics.nl/ emailAddress=jack.leunissen at wur.nl serial:00 X509v3 Basic Constraints: CA:TRUE Signature Algorithm: md5WithRSAEncryption 6e:5c:27:f4:b4:bd:1e:32:c0:ee:03:ce:76:43:c3:e8:3a:50: ........ 0c:e8:f6:98:10:2d:ac:ff:99:3a:5c:f5:f8:66:27:a5:53:c6: 5a:0b -----BEGIN CERTIFICATE----- MIIDxzCCAzCgAwIBAgIBADANBgkqhkiG9w0BAQQFADCBpDELMAkGA1UEBhMCTkwx EzARBgNVBAgTCkdlbGRlcmxhbmQxEzARBgNVBAcTCldhZ2VuaW5nZW4xDDAKBgNV BAoTA1dVUjEXMBUGA1UECxMOQmlvaW5mb3JtYXRpY3MxHjAcBgNVBAMTFWRldi5i ........ bDEkMCIGCSqGSIb3DQEJARYVamFjay5sZXVuaXNzZW5Ad3VyLm5sggEAMAwGA1Ud EwQFMAMBAf8wDQYJKoZIhvcNAQEEBQADgYEAblwn9LS9HjLA7gPOdkPD6DpQUBHF L/4Ihx+J6Y1PVr5NSgWLz5emr0wSqV+adjfYm/+OMV0AkYVQptlm2N61xX7YDbkL mEBS8n22L43f2WU4SP7J24kmhllet56QaS3ImfECP40RwvC+I+26DOj2mBAtrP+Z Olz1+GYnpVPGWgs= -----END CERTIFICATE----- The human readable part is a problem for keytool which you use to import the certificate in your Java keystore. You can convert the certificate in another format that is keytool compatible, but the easiest solution is to strip the human readable part from the certificate. Hence, simply copy -----BEGIN CERTIFICATE-----[encoded certificate]-----END CERTIFICATE----- to a new file. Make sure there are no blank lines left before -----BEGIN CERTIFICATE----- or after -----END CERTIFICATE-----. Now import the certificate in your keystore: keytool -import -trustcacerts -v -alias [some name for your cert.] -file [the cert. without the human readable part] I figured that since the certificate is self-signed, I wouldn't need another certificate authority to make java apps to validate the certificate, but that was wrong. You need to import the same self-signed certificate into your cacerts keystore as well: as root: keytool -import -v -storetype jks -keystore $JAVA_HOME/ lib/security/cacerts -file [the cert. without the human readable part] Now it should work. Cheers, Pieter On 13-Jul-2005, at 6:05 PM, Tom Oinn wrote: > Hi all, > > We've had Taverna working for HTTPS web services so it should be > fine for the MOBY operations. You need to check that the > certificate has a valid signature chain and that the name of the > entity in the certificate matches the hostname - we had issues with > a certificate from a server in china that the java security > framework rejected because the (I think) CN part of the DN in the > cert wasn't the same as the hostname of the endpoint. > > Tom > > Pieter Neerincx wrote: > >> Hi all, >> Is there anyone out there that was able to make BioMOBY >> (realated) Java tools like Yan's Moby Object Browser or Taverna >> work with MOBY Centrals and/or services over HTTPS? I'm having >> trouble with the server's certificate. I used keytool to import >> the server's certificate client-side, but the Java tools keep on >> complaining: "unknown certificate"... >> Cheers, >> Pieter _______________________________________________ >> MOBY-dev mailing list >> MOBY-dev at biomoby.org >> http://www.biomoby.org/mailman/listinfo/moby-dev >> > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > Wageningen University and Research centre (WUR) Laboratory of Bioinformatics Transitorium (building 312) room 1038 Dreijenlaan 3 6703 HA Wageningen phone: 0317-484 706 fax: 0317-483 584 mobile: 06-143 66 783 pieter.neerincx at wur.nl From senger at ebi.ac.uk Mon Jul 18 12:39:27 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Mon, 18 Jul 2005 13:39:27 +0100 (BST) Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <1120860033.3721.7.camel@bioinfo.icapture.ubc.ca> Message-ID: Hi all, Catching up my email piles I wonder if someone can summarize if the discussion about collections in this thread brought any (planned) changes in the API (I am not talking now about how it is, or should be, implemented in Taverna, that's, imho, an another story). My understanding is that (talking about one mobyData object): a) Any Moby service can have more outputs. If so, all of them must be registered. The number of such outputs must be fixed. b) Any of these outputs can be of type either Simple or Collection. c) If it is a Collection, this output can have one or more Simples in that Collection. Such Simples (and their number) are not individually registered. Has this vision been changed? Thanks, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Mon Jul 18 15:09:47 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 18 Jul 2005 08:09:47 -0700 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: References: Message-ID: <1121699387.15443.68.camel@bioinfo.icapture.ubc.ca> Hi Martin, That is a 100% correct description of the status quo (I would just re- word point c to say that the TYPE(s) of Simple(s) in the Collection are registered, but not their number) I have not been able to discern from any of the discussions what it is that others are objecting to, so I am a bit lost...?? I don't know if an API change is being requested, or if there is simply a widespread mis-understanding of how Collections are to be used. Your description, however, is completely correct. M On Mon, 2005-07-18 at 13:39 +0100, Martin Senger wrote: > Hi all, > Catching up my email piles I wonder if someone can summarize if the > discussion about collections in this thread brought any (planned) changes > in the API (I am not talking now about how it is, or should be, > implemented in Taverna, that's, imho, an another story). > My understanding is that (talking about one mobyData object): > a) Any Moby service can have more outputs. If so, all of them must be > registered. The number of such outputs must be fixed. > b) Any of these outputs can be of type either Simple or Collection. > c) If it is a Collection, this output can have one or more Simples in > that Collection. Such Simples (and their number) are not individually > registered. > > Has this vision been changed? > > Thanks, > Martin > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From rebecca.ernst at gsf.de Mon Jul 18 15:14:04 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Mon, 18 Jul 2005 17:14:04 +0200 Subject: [MOBY-dev] buggy new biomoby release Message-ID: <42DBC73C.5030302@gsf.de> Hi Mark (and rest)! As suggested Dirk and I sat down today to do a cvs update to get the last stable release. Now it's 5 hours later and we gave up and switched back to the last working code we had (and which we luckily saved before doing the update! :-)) There are several problems keeping us from updating our code: First of all the links on the release site (biomoby.org/releases) are crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't notice this and therefore received the older version. This version is VERY outdated and not functional (e.g.it uses DOM instead of LibXML) Then we modified the link to get 0.8.2. This version is also not functional as it uses lsid (although this was meant to be the version BEFORE the lsid change). Even when we added a column lsid to the database it didn't work (I guess one would also need the lsid resolver and so on.) Another try was to do an cvs update which crashed everything as completely buggy and syntactical wrong code was checked in (especially in Adaptor/.../mysql.pm) So luckily we could rescue everything because of our save but I guess this is not the way to make people happy installing BioMoby :-( Could we please make sure that releases are functional and the code in the cvs is not complete crap.... otherwise I'll have to do some more spanking ;-) Cheers, Rebecca -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From markw at illuminae.com Mon Jul 18 15:23:51 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 18 Jul 2005 08:23:51 -0700 Subject: [MOBY-dev] buggy new biomoby release In-Reply-To: <42DBC73C.5030302@gsf.de> References: <42DBC73C.5030302@gsf.de> Message-ID: <42DBC987.2000401@illuminae.com> Hi Rebecca, The CVS version certainly wont work (as I noted on the list last week). The link to 0.8.2 was wrong, as you say - sorry about that! The 0.8.2 version should work, however, as I set it up on a completely virgin machine (the current MOBY Central public server) immediately before making the release and it worked perfectly...? Can you be more specific with your error report? v.v. the LSID libraries - it is possible that the requirement for the LSID libraries is in a "use" statement somewhere, but it definitely isn't a real requirement for the 0.8.2 code. If you see a "use" in there, or in the makefile, that is an error for sure. Can you tell me exactly what errors you were seeing? Cheers! M Rebecca Ernst wrote: > Hi Mark (and rest)! > > As suggested Dirk and I sat down today to do a cvs update to get the > last stable release. > > Now it's 5 hours later and we gave up and switched back to the last > working code we had (and which we luckily saved before doing the > update! :-)) > > There are several problems keeping us from updating our code: > > First of all the links on the release site (biomoby.org/releases) are > crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't > notice this and therefore received the older version. This version is > VERY outdated and not functional (e.g.it uses DOM instead of LibXML) > > Then we modified the link to get 0.8.2. This version is also not > functional as it uses lsid (although this was meant to be the version > BEFORE the lsid change). Even when we added a column lsid to the > database it didn't work (I guess one would also need the lsid resolver > and so on.) > > Another try was to do an cvs update which crashed everything as > completely buggy and syntactical wrong code was checked in (especially > in Adaptor/.../mysql.pm) > > > So luckily we could rescue everything because of our save but I guess > this is not the way to make people happy installing BioMoby :-( > > Could we please make sure that releases are functional and the code in > the cvs is not complete crap.... otherwise I'll have to do some more > spanking ;-) > > > Cheers, > Rebecca > > > > > > From markw at illuminae.com Mon Jul 18 15:39:32 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 18 Jul 2005 08:39:32 -0700 Subject: [MOBY-dev] Re: [moby] buggy new biomoby release In-Reply-To: <42DBC73C.5030302@gsf.de> References: <42DBC73C.5030302@gsf.de> Message-ID: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Doh! I missed one. There is an (unused) call to LS::ID->new in the code. Unfortunately, I do have the LSID libraries installed so I didn't pick up on that. I have commented-out the line in the release codebase and will make another release, but before I do I want to make sure that this was the only problem you saw. Let me know, M On Mon, 2005-07-18 at 17:14 +0200, Rebecca Ernst wrote: > Hi Mark (and rest)! > > As suggested Dirk and I sat down today to do a cvs update to get the > last stable release. > > Now it's 5 hours later and we gave up and switched back to the last > working code we had (and which we luckily saved before doing the update! > :-)) > > There are several problems keeping us from updating our code: > > First of all the links on the release site (biomoby.org/releases) are > crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't > notice this and therefore received the older version. This version is > VERY outdated and not functional (e.g.it uses DOM instead of LibXML) > > Then we modified the link to get 0.8.2. This version is also not > functional as it uses lsid (although this was meant to be the version > BEFORE the lsid change). Even when we added a column lsid to the > database it didn't work (I guess one would also need the lsid resolver > and so on.) > > Another try was to do an cvs update which crashed everything as > completely buggy and syntactical wrong code was checked in (especially > in Adaptor/.../mysql.pm) > > > So luckily we could rescue everything because of our save but I guess > this is not the way to make people happy installing BioMoby :-( > > Could we please make sure that releases are functional and the code in > the cvs is not complete crap.... otherwise I'll have to do some more > spanking ;-) > > > Cheers, > Rebecca > > > > > > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From edward.kawas at gmail.com Tue Jul 19 00:38:10 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Mon, 18 Jul 2005 17:38:10 -0700 Subject: [MOBY-dev] request for community feedback on certain issues In-Reply-To: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Message-ID: <42dc4b78.5905bfd9.1943.793b@mx.gmail.com> Hi 'BioMoby'ers, Below I am placing some questions, comments, etc that we would like your feedback on. Hopefully some time soon it will be placed on a twiki! I also attached the file in html format in case this email is too hard to read. Eddie ---Start--- Request for Feedback from the BioMoby Community! Before I begin, I would just like to say that the BioMoby Project is a open source, community driven project. As such, it is key that if the community doesn't like how something is done, then the community should fix it. If something is broken, doesn't work as intended or is vague, speak up and make your voice heard! BioMoby is not a one man project, but a community driven one. Things will only get better if we know what is wrong and if people take initiative. So I (Eddie) just got back from the INBs Technical Meeting in beautiful Malaga, Spain and I have some items that the community needs to talk about below. Also, be sure to keep an eye out for proposals about handling asynchronous services as well as error handling in Moby. When they come, please speak up with your likes, dislikes and ways to make things better. 1. API addition of a method called getResourcesURL: Returns an xml document that describes the location of RESOURCES script that we hear so much about. It is the resources script that contains the ontologies expressed in RDF/XML example: Services http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Services Objects http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Objects ... 2. How do I know what predicates are available in the RDF documents that describe the Moby ontologies? a. should we provide a list of predicates and their intended meanings (on a wiki, etc)? b. should we provide an RDF schema defining the predicates (* probably the way to go) c. Should we allow the registration of new predicates, if so, should we make this an api call, etc. 3. LSID resolution services a. is the LSID attribute available and documented? b. when will this occur, i have heard so much about this, yet i have seen nothing. 4. LSIDs versus simple names a. why is the name of an object, service, or service type is sometimes a simple name and sometimes an LSID? b. how can i even retrieve the LSID of an object, service, or service type? c. should api calls be added to getName and getLSID for objects, services and service types? 5. LSIDs and Service Instances a. If a service instance is registered by 2 different authorities, should they have the same LSID? b. if the service instance is registered in 2 registries, should the LSIDs be the same? c. will modified service instances have versioning information? 6. LSID resolution for objects in the BioMoby ontologies a. what should getData return for i.namespaces (the name of the namespace?) ii. service types (the name of the service type?) iii. objects (xml schema? - note that schemas for datatypes should be ready and available very soon :-) iv. service instances (the name of the instance?) b. should we allow the namespace part of an LSID to be configurable? c. should we allow the authority part of an LSID to be configurable? 7. Would you like to have 3rd party annotation of metadata? a. use case: adds mirroring b. use case: allows people to provide additional metadata i. if you want only one source for metadata, then ignore this question. 9. Predicates a. should we add the following predicates to the metadata of service instances? i. sampleInputData - specifies valid input for the service instance ii. sampleOutputData - specifies the output for the service instance based on the test input. iii. approvedBy - specifies whether this service has been approved. Currently, this would only be useful for the INB, but many more may find it useful. b. how should the sample input/output be formatted? c. should we allow the annotation of datatypes? 10. API and Code versioning a. why are the api and code versions different? 11. Datatype naming a. do you think that the names of datatypes should be regulated? i. what would be the right way to specify a datatypes name? ii. what should we do with text_plain and text-plain? 12. RDF Agent a. do you think that there should be an api call, getUpdate that tells the agent to check for modifications to the service instance described by the metadata in the RDF document at the signatureURL? b. what is the time line to get the agent running? c. besides curation, how is the agent useful to me? 13. Service a. should article names for service instance input and output datatypes be mandatory? b. why does a service that advertise an output datatype called 'foo' return 'bar' instead? ---END--- -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: request_for_feedback.html URL: From senger at ebi.ac.uk Tue Jul 19 09:18:58 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 10:18:58 +0100 (BST) Subject: [MOBY-dev] request for community feedback on certain issues In-Reply-To: <42dc4b78.5905bfd9.1943.793b@mx.gmail.com> Message-ID: Hi Eddie, > BioMoby is not a one man project, but a community driven one. > Well, let's hope it is :-) Services are trully distributed, and some clients accessing Moby Central as well. The registry itself still seems to be a one-man area (imho). > 1. API addition of a method called getResourcesURL: > I understand that this is not a new requirement (we all asked for it many times) but this is a concrete suggestion how to make it. I agree that getting these URLs from an API call is cleaner than just document how such URLs should look like. Only I would suggest slightly different XML response for that (but no strong opinion on my side about it). What we need are actually name-value pairs, each of them representing a name of a resource (such as 'services; and a URL for the RDF describing such resource. So why not just use two attributes for that? Something like this: ... Cheers, Martin PS. Eddie, you should perhaps consider to send everything again, but splitted into few separate topics - that would be easier to find in our mailing lists archives... M. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From rebecca.ernst at gsf.de Tue Jul 19 11:22:15 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue, 19 Jul 2005 13:22:15 +0200 Subject: [MOBY-dev] Re: [moby] buggy new biomoby release In-Reply-To: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> References: <42DBC73C.5030302@gsf.de> <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Message-ID: <42DCE267.8020609@gsf.de> Hi Mark! We tried once more and installed the new release (0.8.2). The problems we can see are that our perl services are not working anymore ("service returned no response") the bad thing is that it doesn't give any error so we have no clue where the problem is. Gbrowse works - the services can be listed and non-MIPS services and MIPS Java services can be executed. So Moby Central obviously works okay since (since we added the column lsid to the db) We then tried to remove the method "_getServiceInstanceRDF" in the MOBY/Central.pm because in there it uses lsid but this didn't solve anything. Just in case we find the problem and you build a new release I have some more suggestions leading to even more user-friendlyness ;-) : - include all libraries not only the perl libraries because all former releases contained everything and now we have to get the other libs from older releases and that means copying it. I guess that most people would expect that they get a full release otherwise consider renaming it (not biomoby release but perl_lib_release) - don't rename the moby-live path to moby-live_API_releasewhatever it now requires that people move it (or adjust their pathes) - next to the link (for downloading the release) add the information that for database provider they will have to change their database ("alter table service_instance add lsid VARCHAR (255) NOT NULL default ''; ") Cheers! Rebecca Mark Wilkinson wrote: >Doh! I missed one. > >There is an (unused) call to LS::ID->new in the code. Unfortunately, I >do have the LSID libraries installed so I didn't pick up on that. > >I have commented-out the line in the release codebase and will make >another release, but before I do I want to make sure that this was the >only problem you saw. > >Let me know, > >M > > > > >On Mon, 2005-07-18 at 17:14 +0200, Rebecca Ernst wrote: > > >>Hi Mark (and rest)! >> >>As suggested Dirk and I sat down today to do a cvs update to get the >>last stable release. >> >>Now it's 5 hours later and we gave up and switched back to the last >>working code we had (and which we luckily saved before doing the update! >>:-)) >> >>There are several problems keeping us from updating our code: >> >>First of all the links on the release site (biomoby.org/releases) are >>crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't >>notice this and therefore received the older version. This version is >>VERY outdated and not functional (e.g.it uses DOM instead of LibXML) >> >>Then we modified the link to get 0.8.2. This version is also not >>functional as it uses lsid (although this was meant to be the version >>BEFORE the lsid change). Even when we added a column lsid to the >>database it didn't work (I guess one would also need the lsid resolver >>and so on.) >> >>Another try was to do an cvs update which crashed everything as >>completely buggy and syntactical wrong code was checked in (especially >>in Adaptor/.../mysql.pm) >> >> >>So luckily we could rescue everything because of our save but I guess >>this is not the way to make people happy installing BioMoby :-( >> >>Could we please make sure that releases are functional and the code in >>the cvs is not complete crap.... otherwise I'll have to do some more >>spanking ;-) >> >> >>Cheers, >>Rebecca >> >> >> >> >> >> >> >> -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From rebecca.ernst at gsf.de Tue Jul 19 13:15:44 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue, 19 Jul 2005 15:15:44 +0200 Subject: [MOBY-dev] NO buggy new biomoby release In-Reply-To: <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> References: <42DBC73C.5030302@gsf.de> <1121701172.15443.76.camel@bioinfo.icapture.ubc.ca> Message-ID: <42DCFD00.5020306@gsf.de> Hey Mark! Congratulations - you won a round of spanking ... You can spank Dirk and me. Shame on us - the release you provided was as perfect as one would expect ;-) The problem why the services didn't work anymore was because of one module (LoggingSubs.pm) which we use within Planet for logging of the services. This was (how ugly!) deployed in the moby-live directory. As the LoggingSubs.pm was not a part of the release this was missing and caused the services to fail. The call to LS::ID didn't matter at all you can leave everything (although you might take the suggestions into account which I mentioned in the last mail) Sorry again for suspecting you ;-) Cheers! Rebecca Mark Wilkinson wrote: >Doh! I missed one. > >There is an (unused) call to LS::ID->new in the code. Unfortunately, I >do have the LSID libraries installed so I didn't pick up on that. > >I have commented-out the line in the release codebase and will make >another release, but before I do I want to make sure that this was the >only problem you saw. > >Let me know, > >M > > > > >On Mon, 2005-07-18 at 17:14 +0200, Rebecca Ernst wrote: > > >>Hi Mark (and rest)! >> >>As suggested Dirk and I sat down today to do a cvs update to get the >>last stable release. >> >>Now it's 5 hours later and we gave up and switched back to the last >>working code we had (and which we luckily saved before doing the update! >>:-)) >> >>There are several problems keeping us from updating our code: >> >>First of all the links on the release site (biomoby.org/releases) are >>crappy. The link to biomoby.0.8.2 links to biomoby.0.8.1 we didn't >>notice this and therefore received the older version. This version is >>VERY outdated and not functional (e.g.it uses DOM instead of LibXML) >> >>Then we modified the link to get 0.8.2. This version is also not >>functional as it uses lsid (although this was meant to be the version >>BEFORE the lsid change). Even when we added a column lsid to the >>database it didn't work (I guess one would also need the lsid resolver >>and so on.) >> >>Another try was to do an cvs update which crashed everything as >>completely buggy and syntactical wrong code was checked in (especially >>in Adaptor/.../mysql.pm) >> >> >>So luckily we could rescue everything because of our save but I guess >>this is not the way to make people happy installing BioMoby :-( >> >>Could we please make sure that releases are functional and the code in >>the cvs is not complete crap.... otherwise I'll have to do some more >>spanking ;-) >> >> >>Cheers, >>Rebecca >> >> >> >> >> >> >> >> -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From senger at ebi.ac.uk Tue Jul 19 14:00:20 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 15:00:20 +0100 (BST) Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <42D39162.1090004@ac.upc.es> Message-ID: Hi, Several small changes have been done to jMoby (all of them should be backward compatible): * The default location and default namespace (URI) of Moby Central was updated (to ...icapture...). Actually, this change has been around already for some time but Eddie never announced it nor logged it in ChangeLog :-) * Roman found a design flaw in the getServiceNames() method returning Map where the keys are service names. This was wrong because there may be (and actually are, e.g. service genbankToGene) more services with the same name but served by different authorities. So I added a new method getServiceNamesByAuthorities() to the interface Central.java (and to its implementation CentralImpl.java), and I deprecated the old method (but it is still there so your code does not break). In future, you should use the new method. This also allowed to add a new command-line option '-la' to the MobyCmdLineClient. Try: build/run/run-cmdline-client -la and you get a list of all services grouped by authorities now. * The location and maintainance of the repository with the third-party libraries for jMoby is now fully "de-sengerized" (as requested in Vancouver). It is described in http://www.biomoby.org/moby-live/Java/docs/3rdPartyLibraries.html. What you should do is the following: cd moby-live/Java ./build-dev.sh cleanall ./build.sh This will populate your 'lib' directory with the same libraries as you have now, but from a different repository (it's just one time job). With regards, Martin PS. In the hopefully near future I plan to make additional changes/fixes: * fix bug (my misunderstanding) with the tag expandobject * added ways how to get RDF resources (Mark, I am desperately waiting for this one - in Malaga last week we created with Eddie a suggestion how to get URLs from the Central API...) * fix bug in the MobyGraphs (Ben, as you see, I am getting close to it...) M. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From edward.kawas at gmail.com Tue Jul 19 14:08:56 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:08:56 -0700 Subject: [MOBY-dev] request for community feedback on certain issues In-Reply-To: Message-ID: <42dd097b.4d90ae57.63bd.2f19@mx.gmail.com> Hi Once again, I have added the page to a twiki, at http://www.biomoby.org/twiki/bin//view/General/FeedBackDoc I have also added Martins comments to it. If this isn't good enough, and you do prefer to have the items split up, then I will do it. Eddie > -----Original Message----- > From: moby-dev-bounces at portal.open-bio.org [mailto:moby- > dev-bounces at portal.open-bio.org] On Behalf Of Martin > Senger > Sent: Tuesday, July 19, 2005 2:19 AM > To: Core developer announcements > Subject: Re: [MOBY-dev] request for community feedback on > certain issues > > Hi Eddie, > > > BioMoby is not a one man project, but a community driven > one. > > > Well, let's hope it is :-) > Services are trully distributed, and some clients > accessing Moby > Central as well. The registry itself still seems to be a > one-man area > (imho). > > > 1. API addition of a method called getResourcesURL: > > > I understand that this is not a new requirement (we > all asked for it > many times) but this is a concrete suggestion how to make > it. I agree that > getting these URLs from an API call is cleaner than just > document how such > URLs should look like. > Only I would suggest slightly different XML response > for that (but no > strong opinion on my side about it). What we need are > actually name-value > pairs, each of them representing a name of a resource > (such as 'services; > and a URL for the RDF describing such resource. So why not > just use two > attributes for that? Something like this: > > > > url="http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOB > Y-S/Services"/> > url="http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOB > Y-S/Objects"/> > ... > url="http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOB > Y-S/All"/> > > > > Cheers, > Martin > > PS. Eddie, you should perhaps consider to send everything > again, but > splitted into few separate topics - that would be easier > to find in our > mailing lists archives... M. > > -- > Martin Senger > > EMBL Outstation - Hinxton Senger at EBI.ac.uk > European Bioinformatics Institute Phone: (+44) 1223 > 494636 > Wellcome Trust Genome Campus (Switchboard: > 494444) > Hinxton Fax : (+44) 1223 > 494468 > Cambridge CB10 1SD > United Kingdom > http://industry.ebi.ac.uk/~senger > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev From gordonp at ucalgary.ca Tue Jul 19 14:14:13 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Tue, 19 Jul 2005 08:14:13 -0600 Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: References: Message-ID: <42DD0AB5.9020502@ucalgary.ca> Anyone else having trouble with MOBY Central? org.biomoby.shared.MobyException: ===ERROR=== Fault details: [stackTrace: null] Fault string: java.lang.NullPointerException Fault code: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException Fault actor: null When calling: http://mobycentral.icapture.ubc.ca/cgi-bin/MOBY05/mobycentral.pl =========== at org.biomoby.client.CentralImpl.doCall(CentralImpl.java:191) at org.biomoby.client.CentralImpl.getNamespaces(CentralImpl.java:779)... From markw at illuminae.com Tue Jul 19 14:22:36 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue, 19 Jul 2005 07:22:36 -0700 Subject: [moby] [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: References: Message-ID: <1121782956.21968.9.camel@bioinfo.icapture.ubc.ca> On Tue, 2005-07-19 at 15:00 +0100, Martin Senger wrote: > * added ways how to get RDF resources (Mark, I am desperately waiting > for this one - in Malaga last week we created with Eddie a suggestion how > to get URLs from the Central API...) HI Martin, Eddie gave us the update on his Spanish activities yesterday. He told me what you discussed and I agree that this is a good idea. I will add that function to the API right away, and I can code it into the currently deployed version of MOBY, so that it will work from the main registry, however it will take a couple of weeks to get it into the CVS since the CVS codebase is currently going through a major refactoring effort in anticipation of our trip to Manchester in two weeks... You will notice that I have added LSID's to the 0.85 API that is on the website (they are part of most of the messages coming back from MOBY Central). These can be resolved to metadata to give you the same RDF graphs. HOWEVER, the code is not yet up-to-date relative to the API (the API is at 0.85 but the codebase is only at 0.82). Again, I'm trying to get the 0.85 codebase out as quickly as possible, but it is taking some time as there are only my hands (5% of the time) and my (highly competent!) co-op student Dennis assigned to this task... we simply suffer from lack of resourcing at this end :-( Cheers! M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From akerhornou at imim.es Tue Jul 19 14:30:57 2005 From: akerhornou at imim.es (Arnaud Kerhornou) Date: Tue, 19 Jul 2005 16:30:57 +0200 Subject: [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries Message-ID: <42DD0EA1.7030106@imim.es> Hi, I am working for the INB in Spain setting up Moby Web services, and I am trying to switch my code to use the new release, BioMoby 0.8.2 (was using BioMoby 0.8.1 before). I have a client script that is interrogating a registry server and ask for a given service. If the registry server is the following one, http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if I want to interrogate another one, I am getting an error. I am instanciating a Central object with the following parameters, => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl => URI: http://chirimoyo.ac.uma.es/MOBY/Central => Proxy: No Proxy Server The instanciation of Central seems to go okay, but then when I want to execute 'findService' method, I get the following error: -------------------------> Can't call method "getValue" on an undefined value at /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 <------------------------- Central.pm line 1728 is the following one (in _parseServices method) my $lsid = $Service->getAttributeNode('lsid')->getValue; Any clue of what is going wrong or do I need to update my client code ? Thanks Arnaud ___________________________________________________________________________ Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger T?l?chargez cette version sur http://fr.messenger.yahoo.com From senger at ebi.ac.uk Tue Jul 19 14:35:26 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 15:35:26 +0100 (BST) Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <42DD0AB5.9020502@ucalgary.ca> Message-ID: > Anyone else having trouble with MOBY Central? > No, I don't (using: build/run/run-testing-central). Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 14:35:26 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 15:35:26 +0100 (BST) Subject: [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <42DD0AB5.9020502@ucalgary.ca> Message-ID: > Anyone else having trouble with MOBY Central? > No, I don't (using: build/run/run-testing-central). Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 14:37:22 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 15:37:22 +0100 (BST) Subject: [moby] [MOBY-dev] [jMoby] few recent changes in jMoby In-Reply-To: <1121782956.21968.9.camel@bioinfo.icapture.ubc.ca> Message-ID: Mark, > I will add that function to the API right away, and I can code it into > the currently deployed version of MOBY, so that it will work from the > main registry > That's great! Just let me/us know what is the exact XML structure you are going to return by this new method (the best would be to put it into the API doc I guess), and I will add it into jMoby at once. Which means that this new method will work from your moby central but not yet from the others centrals, correct? Until their providers implement it as well - but their role is here difficult because they cannoy use CVS now. Do I understand it correctly? > You will notice that I have added LSID's to the 0.85 API that is on the > website (they are part of most of the messages coming back from MOBY > Central). These can be resolved to metadata to give you the same RDF > graphs. HOWEVER, the code is not yet up-to-date relative to the API > (the API is at 0.85 but the codebase is only at 0.82). > You mean the code for returning attribute 'lsid' in XML, or code for the LSID resilution service (to use such attribute), or both? Cheers, Martin PS. The clock is ticking - in 12 days I will be sitting in Philippines and working on Biomoby :-) M. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From rebecca.ernst at gsf.de Tue Jul 19 14:41:30 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue, 19 Jul 2005 16:41:30 +0200 Subject: [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <42DD0EA1.7030106@imim.es> References: <42DD0EA1.7030106@imim.es> Message-ID: <42DD111A.8060900@gsf.de> Hi Arnaud! I am not sure if you are facing the same problem that we did... possibly the registry you want to contact doesn't contain the colunm 'lsid' this needs to be updated along with the new code so you have to modify your database like this: "alter table service_instance add lsid VARCHAR (255) NOT NULL default ''; " Cheers, Rebecca Arnaud Kerhornou wrote: > Hi, > > I am working for the INB in Spain setting up Moby Web services, and I > am trying to switch my code to use the new release, BioMoby 0.8.2 (was > using BioMoby 0.8.1 before). > > I have a client script that is interrogating a registry server and ask > for a given service. If the registry server is the following one, > http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if > I want to interrogate another one, I am getting an error. > > I am instanciating a Central object with the following parameters, > > => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl > => URI: http://chirimoyo.ac.uma.es/MOBY/Central > => Proxy: No Proxy Server > > The instanciation of Central seems to go okay, but then when I want to > execute 'findService' method, I get the following error: > -------------------------> > Can't call method "getValue" on an undefined value at > /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 > <------------------------- > > Central.pm line 1728 is the following one (in _parseServices method) > > my $lsid = $Service->getAttributeNode('lsid')->getValue; > > Any clue of what is going wrong or do I need to update my client code ? > > Thanks > Arnaud > > > > > > ___________________________________________________________________________ > Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! > Messenger T?l?chargez cette version sur http://fr.messenger.yahoo.com > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From rebecca.ernst at gsf.de Tue Jul 19 14:41:30 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Tue, 19 Jul 2005 16:41:30 +0200 Subject: [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <42DD0EA1.7030106@imim.es> References: <42DD0EA1.7030106@imim.es> Message-ID: <42DD111A.8060900@gsf.de> Hi Arnaud! I am not sure if you are facing the same problem that we did... possibly the registry you want to contact doesn't contain the colunm 'lsid' this needs to be updated along with the new code so you have to modify your database like this: "alter table service_instance add lsid VARCHAR (255) NOT NULL default ''; " Cheers, Rebecca Arnaud Kerhornou wrote: > Hi, > > I am working for the INB in Spain setting up Moby Web services, and I > am trying to switch my code to use the new release, BioMoby 0.8.2 (was > using BioMoby 0.8.1 before). > > I have a client script that is interrogating a registry server and ask > for a given service. If the registry server is the following one, > http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if > I want to interrogate another one, I am getting an error. > > I am instanciating a Central object with the following parameters, > > => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl > => URI: http://chirimoyo.ac.uma.es/MOBY/Central > => Proxy: No Proxy Server > > The instanciation of Central seems to go okay, but then when I want to > execute 'findService' method, I get the following error: > -------------------------> > Can't call method "getValue" on an undefined value at > /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 > <------------------------- > > Central.pm line 1728 is the following one (in _parseServices method) > > my $lsid = $Service->getAttributeNode('lsid')->getValue; > > Any clue of what is going wrong or do I need to update my client code ? > > Thanks > Arnaud > > > > > > ___________________________________________________________________________ > Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! > Messenger T?l?chargez cette version sur http://fr.messenger.yahoo.com > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From edward.kawas at gmail.com Tue Jul 19 14:45:52 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:45:52 -0700 Subject: [MOBY-dev] rfcf 1. API addition of a method called getResourcesURL: In-Reply-To: Message-ID: <42dd1223.513aad7a.369d.5415@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 1. API addition of a method called getResourcesURL: Returns an xml document that describes the location of RESOURCES script that we hear so much about. It is the resources script that contains the ontologies expressed in RDF/XML example: Services http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Services Objects http://mobycentral.icapture.ubc.ca:8090/RESOURCES/MOBY- S/Objects ... Comments: Martin: I understand that this is not a new requirement (we all asked for it many times) but this is a concrete suggestion how to make it. I agree that getting these URLs from an API call is cleaner than just document how such URLs should look like. Only I would suggest slightly different XML response for that (but no strong opinion on my side about it). What we need are actually name-value pairs, each of them representing a name of a resource (such as 'services; and a URL for the RDF describing such resource. So why not just use two attributes for that? Something like this: ... Mark: I will add that function to the API right away, and I can code it into the currently deployed version of MOBY, so that it will work from the main registry, however it will take a couple of weeks to get it into the CVS since the CVS codebase is currently going through a major refactoring effort in anticipation of our trip to Manchester in two weeks... From edward.kawas at gmail.com Tue Jul 19 14:46:03 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:03 -0700 Subject: [MOBY-dev] rfcf 2. How do I know what predicates are available in the RDF In-Reply-To: Message-ID: <42dd122e.54b1a5ca.63bd.4a5c@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 2. How do I know what predicates are available in the RDF documents that describe the Moby ontologies? a. should we provide a list of predicates and their intended meanings (on a wiki, etc)? b. should we provide an RDF schema defining the predicates (* probably the way to go) c. Should we allow the registration of new predicates, if so, should we make this an api call, etc. From edward.kawas at gmail.com Tue Jul 19 14:46:15 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:15 -0700 Subject: [MOBY-dev] rfcf 3. LSID resolution services In-Reply-To: Message-ID: <42dd123b.670fc02b.2dd4.ffff929f@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 3. LSID resolution services a. is the LSID attribute available and documented? comments: Mark: You will notice that I have added LSID's to the 0.85 API that is on the website (they are part of most of the messages coming back from MOBY Central). These can be resolved to metadata to give you the same RDF graphs. HOWEVER, the code is not yet up-to-date relative to the API (the API is at 0.85 but the codebase is only at 0.82). Again, I'm trying to get the 0.85 codebase out as quickly as possible, but it is taking some time as there are only my hands (5% of the time) and my (highly competent!) co-op student Dennis assigned to this task. b. when will this occur, i have heard so much about this, yet i have seen nothing. From edward.kawas at gmail.com Tue Jul 19 14:46:23 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:23 -0700 Subject: [MOBY-dev] rfcf 4. LSIDs versus simple names In-Reply-To: Message-ID: <42dd1240.1e9853c8.2dd4.ffff92af@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 4. LSIDs versus simple names a. why is the name of an object, service, or service type is sometimes a simple name and sometimes an LSID? b. how can i even retrieve the LSID of an object, service, or service type? c. should api calls be added to getName and getLSID for objects, services and service types? From edward.kawas at gmail.com Tue Jul 19 14:46:33 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:33 -0700 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: Message-ID: <42dd124c.7fdce064.76fe.ffff9b01@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 5. LSIDs and Service Instances a..If a service instance is registered by 2 different authoritys, should they have the same LSID? b. if the service instance is registered in 2 registries, should the LSIDs be the same? c. will modified service instances have versioning information? From edward.kawas at gmail.com Tue Jul 19 14:46:40 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:40 -0700 Subject: [MOBY-dev] rfcf 6. LSID resolution for objects in the BioMoby ontologies In-Reply-To: Message-ID: <42dd1252.3e967c5f.76fe.ffff9b10@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 6. LSID resolution for objects in the BioMoby ontologies a. what should getData return for i.namespaces (the name of the namespace?) ii. service types (the name of the service type?) iii. objects (xml schema? - note that schemas for datatypes should be ready and available very soon :-) iv. service instances (the name of the instance?) b. should we allow the namespace part of an LSID to be configurable? c. should we allow the authority part of an LSID to be configurable? From markw at illuminae.com Tue Jul 19 14:47:18 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue, 19 Jul 2005 07:47:18 -0700 Subject: [moby] [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <42DD0EA1.7030106@imim.es> References: <42DD0EA1.7030106@imim.es> Message-ID: <1121784438.21968.16.camel@bioinfo.icapture.ubc.ca> I've just packaged up another release (0.8.2a) that includes fixes for the 0.8.2 release bugs that have been reported so far. Please try this one and tell me if it still causes problems. M On Tue, 2005-07-19 at 16:30 +0200, Arnaud Kerhornou wrote: > Hi, > > I am working for the INB in Spain setting up Moby Web services, and I am > trying to switch my code to use the new release, BioMoby 0.8.2 (was > using BioMoby 0.8.1 before). > > I have a client script that is interrogating a registry server and ask > for a given service. If the registry server is the following one, > http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if I > want to interrogate another one, I am getting an error. > > I am instanciating a Central object with the following parameters, > > => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl > => URI: http://chirimoyo.ac.uma.es/MOBY/Central > => Proxy: No Proxy Server > > The instanciation of Central seems to go okay, but then when I want to > execute 'findService' method, I get the following error: > -------------------------> > Can't call method "getValue" on an undefined value at > /home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 > <------------------------- > > Central.pm line 1728 is the following one (in _parseServices method) > > my $lsid = $Service->getAttributeNode('lsid')->getValue; > > Any clue of what is going wrong or do I need to update my client code ? > > Thanks > Arnaud > > > > > > ___________________________________________________________________________ > Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger > T?l?chargez cette version sur http://fr.messenger.yahoo.com > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From edward.kawas at gmail.com Tue Jul 19 14:46:48 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:48 -0700 Subject: [MOBY-dev] rfcf 7. Would you like to have 3rd party annotation of metadata? In-Reply-To: Message-ID: <42dd125a.5ec5bd58.43d9.3f96@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 7. Would you like to have 3rd party annotation of metadata? a. use case: adds mirroring b. use case: allows people to provide additional metadata i. if you want only one source for metadata, then ignore this question. From edward.kawas at gmail.com Tue Jul 19 14:46:59 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:46:59 -0700 Subject: [MOBY-dev] rfcf 8. Predicates In-Reply-To: Message-ID: <42dd1264.5c49d513.43d9.3fa7@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 8. Predicates a. should we add the following predicates to the metadata of service instances? i. sampleInputData - specifies valid input for the service instance ii. sampleOutputData - specifies the output for the service instance based on the test input. iii. approvedBy - specifies whether this service has been approved. Currently, this would only be useful for the INB, but many more may find it useful. b. how should the sample input/output be formatted? c. should we allow the annotation of datatypes? From edward.kawas at gmail.com Tue Jul 19 14:47:01 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:47:01 -0700 Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: Message-ID: <42dd1266.6b7805fa.43d9.3fae@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 12. Services a. should article names for service instance input and output datatypes be mandatory? b. why does a service that advertise an output datatype called 'foo' return 'bar' instead? From edward.kawas at gmail.com Tue Jul 19 14:47:06 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:47:06 -0700 Subject: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: Message-ID: <42dd126b.46e6258d.43d9.3fb7@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 11. RDF Agent a. do you think that there should be an api call, getUpdate that tells the agent to check for modifications to the service instance described by the metadata in the RDF document at the signatureURL? b. what is the time line to get the agent running? c. besides curation, how is the agent useful to me? From edward.kawas at gmail.com Tue Jul 19 14:47:11 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:47:11 -0700 Subject: [MOBY-dev] rfcf 10. Datatype naming In-Reply-To: Message-ID: <42dd1271.3247625d.71ce.2bb7@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 10. Datatype naming a. do you think that the names of datatypes should be regulated? i. what would be the right way to specify a datatypes name? ii. what should we do with text_plain and text-plain? From edward.kawas at gmail.com Tue Jul 19 14:47:16 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Tue, 19 Jul 2005 07:47:16 -0700 Subject: [MOBY-dev] rfcf 9. API and Code versioning In-Reply-To: Message-ID: <42dd1275.5d41a999.71ce.2bc3@mx.gmail.com> Since there was a request to keep these messages in email and away from twiki, I have added each heading to an email. 9. API and Code versioning a. why are the api and code versions different? Mark: the code is not yet up-to-date relative to the API (the API is at 0.85 but the codebase is only at 0.82). Again, I'm trying to get the 0.85 codebase out as quickly as possible, but it is taking some time as there are only my hands (5% of the time) and my (highly competent!) co-op student Dennis assigned to this task. From axk at sanger.ac.uk Tue Jul 19 15:01:08 2005 From: axk at sanger.ac.uk (Arnaud Kerhornou) Date: Tue, 19 Jul 2005 17:01:08 +0200 Subject: [moby] [MOBY-dev] Problem with Biomoby 0.8.2 Perl libraries In-Reply-To: <1121784438.21968.16.camel@bioinfo.icapture.ubc.ca> References: <42DD0EA1.7030106@imim.es> <1121784438.21968.16.camel@bioinfo.icapture.ubc.ca> Message-ID: <42DD15B4.3000202@sanger.ac.uk> Hi Mark, I have downloaded the new release, and the script works fine now, Thanks Arnaud Mark Wilkinson wrote: >I've just packaged up another release (0.8.2a) that includes fixes for >the 0.8.2 release bugs that have been reported so far. > >Please try this one and tell me if it still causes problems. > >M > > >On Tue, 2005-07-19 at 16:30 +0200, Arnaud Kerhornou wrote: > > >>Hi, >> >>I am working for the INB in Spain setting up Moby Web services, and I am >>trying to switch my code to use the new release, BioMoby 0.8.2 (was >>using BioMoby 0.8.1 before). >> >>I have a client script that is interrogating a registry server and ask >>for a given service. If the registry server is the following one, >>http://mobycentral.icapture.ubc.ca/MOBY/Central, it works fine but if I >>want to interrogate another one, I am getting an error. >> >>I am instanciating a Central object with the following parameters, >> >> => URL: http://chirimoyo.ac.uma.es/cgi-bin/MOBY-Central.pl >> => URI: http://chirimoyo.ac.uma.es/MOBY/Central >> => Proxy: No Proxy Server >> >>The instanciation of Central seems to go okay, but then when I want to >>execute 'findService' method, I get the following error: >>-------------------------> >>Can't call method "getValue" on an undefined value at >>/home/ug/arnau/lib/biomoby.0.8.2/Perl/MOBY/Client/Central.pm line 1728 >><------------------------- >> >>Central.pm line 1728 is the following one (in _parseServices method) >> >>my $lsid = $Service->getAttributeNode('lsid')->getValue; >> >>Any clue of what is going wrong or do I need to update my client code ? >> >>Thanks >>Arnaud >> >> >> >> >> >>___________________________________________________________________________ >>Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger >>T??l??chargez cette version sur http://fr.messenger.yahoo.com >> >>_______________________________________________ >>MOBY-dev mailing list >>MOBY-dev at biomoby.org >>http://www.biomoby.org/mailman/listinfo/moby-dev >> >> ___________________________________________________________________________ Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger T?l?chargez cette version sur http://fr.messenger.yahoo.com From senger at ebi.ac.uk Tue Jul 19 15:26:11 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 16:26:11 +0100 (BST) Subject: [MOBY-dev] rfcf 8. Predicates In-Reply-To: <42dd1264.5c49d513.43d9.3fa7@mx.gmail.com> Message-ID: > a. should we add the following predicates to the metadata of > service instances? > Well, we should. But predicates should be unique, and I thought that they will be labeled as LSIDs... M. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 20:22:56 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 21:22:56 +0100 (BST) Subject: [MOBY-dev] jMoby does not compile!!!!! In-Reply-To: <42dd126b.46e6258d.43d9.3fb7@mx.gmail.com> Message-ID: Eddie, Four hours ago, you committed changes that do not compile! (e.g. in client/rdf/builder). You are the biggest sinner one can be. Please, please, correct it fast... Thanks, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 19 20:28:50 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 19 Jul 2005 21:28:50 +0100 (BST) Subject: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: <42dd126b.46e6258d.43d9.3fb7@mx.gmail.com> Message-ID: > a. do you think that there should be an api call, getUpdate > that tells the agent to check for modifications to the > service instance described by the metadata in the RDF > document at the signatureURL? > Well, it depends how LSID metadata are collected and used, and where they are stored. The bottom line what I wish is this: When a service provider updates her/his metadata about his/her services, I wish that this update is visible asap to the others. Therefore, if it is the RDF agent you collect metadata and stores in a registry, then I wish to have an option to tell him "come and get it now". If, however, this is not the task of the RDF agent, but if there is yet another way how to collect data from my site (presumably done by the LSID resolution service) then I do not need such method in the API (because the LSID resolver will come to me when somebody asks for my metadata). Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Tue Jul 19 22:41:40 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue, 19 Jul 2005 15:41:40 -0700 Subject: [moby] RE: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: References: Message-ID: <1121812900.22947.57.camel@bioinfo.icapture.ubc.ca> I think the intent is as follows: The agent will collect the "core" data for the registry on a regular basis. You can chose to explicitly call the agent to you if you like, but it will only record the information that MOBY Central currently records. During service discovery, you will receive the LSID representing a particular service as part of the response message from MOBY Central (this is already in the API 0.85, but has not been coded yet). MOBY Central will act as the Authority for that LSID, and will give you a WSDL document that (at a minimum) directs you to GET the SignatureURL of that service as the getMetaData resolution function - the SignatureURL should hold an RDF document describing that service FULLY; i.e. not only what MOBY Central needs to know, but also whatever else the service provider wants to tell you about the service. Some of the RDF predicates in that document will be part of the official MOBY API, while others may be defined specifically by that service provider. Of course, we will try to make as many of the useful ones part of the MOBY API :-) Does that clarify how I am thinking about the process? Are there better ways to accomplish this that I haven't thought of? This is the perfect time to speak-up as we are in the process of re-coding the API right now... Cheers! M On Tue, 2005-07-19 at 21:28 +0100, Martin Senger wrote: > > a. do you think that there should be an api call, getUpdate > > that tells the agent to check for modifications to the > > service instance described by the metadata in the RDF > > document at the signatureURL? > > > Well, it depends how LSID metadata are collected and used, and where > they are stored. The bottom line what I wish is this: > When a service provider updates her/his metadata about his/her > services, I wish that this update is visible asap to the others. > Therefore, if it is the RDF agent you collect metadata and stores in a > registry, then I wish to have an option to tell him "come and get it now". > If, however, this is not the task of the RDF agent, but if there is yet > another way how to collect data from my site (presumably done by the LSID > resolution service) then I do not need such method in the API (because the > LSID resolver will come to me when somebody asks for my metadata). > > Martin > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From senger at ebi.ac.uk Tue Jul 19 23:50:57 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 20 Jul 2005 00:50:57 +0100 (BST) Subject: [moby] RE: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: <1121812900.22947.57.camel@bioinfo.icapture.ubc.ca> Message-ID: > The agent will collect the "core" data for the registry on a regular > basis. You can chose to explicitly call the agent to you if you like, > but it will only record the information that MOBY Central currently > records. > Okay. So far, so clear. How can I explicitly call the agent to me? > During service discovery, you will receive the LSID representing a > particular service as part of the response message from MOBY Central > (this is already in the API 0.85, but has not been coded yet). > This is done by the XML attribute 'lsid', correct? > MOBY Central will act as the Authority for that LSID, and will give > you a WSDL document that (at a minimum) directs you to GET the > SignatureURL of that service as the getMetaData resolution function > While I understand this I doubt that all service providers are aware that actually their SignatureURL is supposed to play a role of a LSID metadata resolution service. Concretely, a service that is defined by the LSIDMetadataHTTPBindingDirect (as defined in the LSID spec). Which means, for example, that the returned HTTP response should have Content-Type "x-application/rdf+xml" and Expires HTTP headers. Also, strictly speaking, an LSID should resolve to the metadata describing this LSID, but in the case of using SignatureURL there is no guarantee that a service provider does not put into the same document metadata about more services. So, we need to have this very clearly documented and explained, so the clients cannot be surprised when they get more than they asked for. I think that it would be also good if you can show an example of the WSDL with that binding, and instructions how clients should use such WSDL (which part they should take - my understanding is that the SignatureURL will be in the tag tag). > time to speak-up as we are in the process of re-coding the API right > now... > One more thing (not about API but about implementation) - and I think that Eddie mentioned it somewhere else, as well): In CSHL, we spoke about a special moby service that can be used for registering (and for retrieving) endpoints of service metadata created by the third-parties. Such service does not exist yet (nor its API) but if it exists (and if the registry is configured to use it), the registry should use it to put more metadata locations into returned WSDL. Please keep it in mind when you are implementing this part, so it can be easier updated when such service exists. Cheers, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Wed Jul 20 00:07:15 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Tue, 19 Jul 2005 17:07:15 -0700 Subject: [unclassified] Re: [moby] RE: [MOBY-dev] rfcf 11. RDF Agent In-Reply-To: References: Message-ID: <1121818035.22947.85.camel@bioinfo.icapture.ubc.ca> On Wed, 2005-07-20 at 00:50 +0100, Martin Senger wrote: > Okay. So far, so clear. How can I explicitly call the agent to me? Yet to be decided... we'll make it an API call of some sort. We're still trying to get the agent to work so I need to focus on that first :-) > This is done by the XML attribute 'lsid', correct? correct. > Which means, for example, that the returned HTTP response should have > Content-Type "x-application/rdf+xml" and Expires HTTP headers. > Also, strictly speaking, an LSID should resolve to the metadata > describing this LSID, but in the case of using SignatureURL there is no > guarantee that a service provider does not put into the same document > metadata about more services. This is a very good point, and I hadn't fully considered the consequences of this v.v. having more than one service definition in the metadata document. Let's tease out what behaviour we want from this part of the API before we make any further decisions. It seems to me that the objective is for the service provider to be able to say things about their service that are not recorded in the Registry, but are still available to the consumer through a predictable API. We accomplish this through both avenues (having only one SignatureRDF at the URL or by having multiple concatenated SignatureRDF's at a single URL), but you are 100% correct that it might be a more confusing document to parse if the client is not expecting it. Are there other reasons to chose one over the other? I guess having a proliferation of SignatureRDF's on the server-side is an unpleasant thought also (though from a service providers perspective, it might be easier to manage, since RDF isn't particularly human-readable anyway...) ?? other thoughts ?? > I think that it would be also good if you can show an example of the > WSDL with that binding, and instructions how clients should use such WSDL > (which part they should take - my understanding is that the SignatureURL > will be in the tag tag). That was what I had in mind, yes. We'll certainly document everything well once we have it set up the way we want it. > One more thing (not about API but about implementation) - and I think > that Eddie mentioned it somewhere else, as well): In CSHL, we spoke about > a special moby service that can be used for registering (and for > retrieving) endpoints of service metadata created by the third-parties. > Such service does not exist yet (nor its API) but if it exists (and if the > registry is configured to use it), the registry should use it to put more > metadata locations into returned WSDL. Please keep it in mind when you are > implementing this part, so it can be easier updated when such service > exists. OK Cheers! M -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From rebecca.ernst at gsf.de Wed Jul 20 07:24:38 2005 From: rebecca.ernst at gsf.de (Rebecca Ernst) Date: Wed, 20 Jul 2005 09:24:38 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: References: Message-ID: <42DDFC36.4090409@gsf.de> Hi Martin! this vision has not been changed yet, mainly because noone seems to understand Heikos, Dirks and my problems with the existing solution. If you re-read the mails you can try to understand if you get the point I can't make myself any clearer than that. Basically I think it doesn't make sense to register an output as being a collection if the results don't form an entity! In our case e.g. the service 'getAGILocusCodes'. You give it a keyword and it returns all AGI Codes somehow related to this keyword. These AGI codes don't form an entity! Just because the keyword appears in their annotation they are given back - but with no relation to each other. For us it would be only logical if we would give many simple responses. This is not possible as currently if the number of results is not known we have to give back a collection. cheers, Rebecca Martin Senger wrote: >Hi all, > Catching up my email piles I wonder if someone can summarize if the >discussion about collections in this thread brought any (planned) changes >in the API (I am not talking now about how it is, or should be, >implemented in Taverna, that's, imho, an another story). > My understanding is that (talking about one mobyData object): > a) Any Moby service can have more outputs. If so, all of them must be >registered. The number of such outputs must be fixed. > b) Any of these outputs can be of type either Simple or Collection. > c) If it is a Collection, this output can have one or more Simples in >that Collection. Such Simples (and their number) are not individually >registered. > > Has this vision been changed? > > Thanks, > Martin > > > -- Rebecca Ernst MIPS, Inst. for Bioinformatics GSF Research Center for Environment and Health Ingolstaedter Landstr. 1 85764 Neuherberg fon: +49 89 3187 3583 email: Rebecca.Ernst at gsf.de From senger at ebi.ac.uk Wed Jul 20 09:24:48 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 20 Jul 2005 10:24:48 +0100 (BST) Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: <42DDFC36.4090409@gsf.de> Message-ID: Rebecca, Many thanks for the explanation. I think this is the critical part: > Basically I think it doesn't make sense to register an output as being a > collection if the results don't form an entity! > I assume that by entity you mean 'set of pieces that belong semantically together'. I think that the current concept of Simples and Collections gives you full freedom to model your output in any way you consider proper. For example, if you do not feel comfortable to put all AGI Codes in one collection because they represent semantically different things you can split them by types (or whatever semantics you choose) into several collections, or you can actually deploy several services, each of them returning a different set (collection) of AGI Codes given the same input keyword. But you know all that. I believe that we need to keep the concept of Collections open as are the general data types in other languages (e.g. hashtables). I am sorry to bother you with these obvious comments. Cheers, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From d.haase at gsf.de Wed Jul 20 12:24:08 2005 From: d.haase at gsf.de (Dirk Haase) Date: Wed, 20 Jul 2005 14:24:08 +0200 Subject: [MOBY-dev] Moby Collections (was: Problems with Biomoby servicesin Taverna 1.2) In-Reply-To: References: Message-ID: <200507201424.08853.d.haase@gsf.de> On Wednesday 20 July 2005 11:24, Martin Senger wrote: > Rebecca, > Many thanks for the explanation. > > I think this is the critical part: > > Basically I think it doesn't make sense to register an output as being a > > collection if the results don't form an entity! > > I assume that by entity you mean 'set of pieces that belong > semantically together'. Exactly. > I think that the current concept of Simples and Collections gives you > full freedom to model your output in any way you consider proper. This is actually _not_ the case and that is why we brought up the issue. There is no proper way to output things which really make up a 'set of pieces that belong semantically together'. For example there is no way to output the results of a clustering service (a cluster is a set of sequences, the service result is a set of clusters), because a collection of collections is not allowed afaik. Of course one can create a 'cluster object' but this creates rather artificial hurdles in workflow creation. Suppose that after the clustering, each cluster should be fed into a multiple alignment service. This would presumably take a collection of sequences as input. So one would have to decompose the cluster object first before it can be further processed. I think we all agree that on the input side a collection only makes sense if it is meant as one semantical entity (example from the API docs: several sequences make up the BLAST database that is to be queried). But not so on the output side. So we have an inherent unequality which complicates workflow building. > For > example, if you do not feel comfortable to put all AGI Codes in one > collection because they represent semantically different things you can > split them by types (or whatever semantics you choose) into several > collections, The claim is they are not related... The only relation is that they pop up for the same input but that is obvious for this kind of services and does not need to be emphasized by stuffing them into a collection. > I believe that we need to keep the concept of Collections open as are > the general data types in other languages (e.g. hashtables). That is a good point, but I'm not sure if it is appropriate to compare the concept of Simples/Collections to general data types in programming languages. > I am sorry to > bother you with these obvious comments. No, they are very welcome indeed! Servus, dirk From senger at ebi.ac.uk Wed Jul 20 12:44:10 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 20 Jul 2005 13:44:10 +0100 (BST) Subject: [MOBY-dev] Moby Collections (was: Problems with Biomoby servicesin Taverna 1.2) In-Reply-To: <200507201424.08853.d.haase@gsf.de> Message-ID: > because a collection of collections is not allowed afaik. > You are right - I have forgotten that we cannot have collections of collections. So we cannot model all possible scenarios. But: BioMoby (afaik) tries to be simple. To allow completely complex data types (build on collections of collectios) would mean to assume that all clients will be able to understand such complex data types. And it would be hard. We would end up anyway with many clients that would simply ignore such complex types (or they break on them). So why not to go with the current API (I may even argue if it is really neccessary to allow a collection having different types of Simples, but that's another story) and to try to solve the problem of semantically relevant clusters on the service level - simply to create more specific service instances. BioMoby was/is about a simple transformation. If the transformation is not simple let's try to divide it into more transformations that can be simple. Cheers, Martin PS. > but I'm not sure if it is appropriate to compare the concept of > Simples/Collections to general data types in programming languages. > Why not? Except that biomoby collections are less general (by not allowing coles of coles). M. -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From gordonp at ucalgary.ca Wed Jul 20 14:22:15 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed, 20 Jul 2005 08:22:15 -0600 Subject: [MOBY-dev] Moby Collections (was: Problems with Biomoby servicesin Taverna 1.2) In-Reply-To: References: Message-ID: <42DE5E17.8080302@ucalgary.ca> I think Martin has hit the nail on the head here. One of the primary advantages of MOBY is that you can't (easily) build arbitrarily syntactically complex output. This ensures that you can chain together services more easily. You should be building new objects in the ontology to represent the cluster concept, not squeezing that cluster logic into the syntax of the collection. Otherwise clients will do very wierd things with your implicitly encoded concepts. You always have to look at how your data could be used by someone else who wasn't expecting it... >>because a collection of collections is not allowed afaik. >> >> >> > You are right - I have forgotten that we cannot have collections of >collections. So we cannot model all possible scenarios. > > From jmfernandez at cnb.uam.es Wed Jul 20 14:53:44 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Wed, 20 Jul 2005 16:53:44 +0200 Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: <42dd1266.6b7805fa.43d9.3fae@mx.gmail.com> References: <42dd1266.6b7805fa.43d9.3fae@mx.gmail.com> Message-ID: <42DE6578.3080008@cnb.uam.es> > > 12. Services > > a. should article names for service instance input and > output datatypes be mandatory? > Well, in my opinion, I think they should be mandatory when there is more than one input (or output). And when there is only one and the article name it is not declared, then it should be a default value for the article name. > b. why does a service that advertise an output datatype > called 'foo' return 'bar' instead? > If datatype 'bar' inherits from 'foo', then it could be allowed, but I don't have a good example for that. But we must take into account that most times it happens it is a bug. > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From jmfernandez at cnb.uam.es Wed Jul 20 14:46:11 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Wed, 20 Jul 2005 16:46:11 +0200 Subject: [MOBY-dev] rfcf 2. How do I know what predicates are available inthe RDF In-Reply-To: <42dd122e.54b1a5ca.63bd.4a5c@mx.gmail.com> References: <42dd122e.54b1a5ca.63bd.4a5c@mx.gmail.com> Message-ID: <42DE63B3.503@cnb.uam.es> Hi everybody, I think a mixure of b) and c) could be feasible. The RDF Schema would be generated from the registered new predicates. Edward Kawas wrote: > Since there was a request to keep these messages in email > and away from twiki, I have added each heading to an email. > > 2. How do I know what predicates are available in the RDF > documents that describe the Moby ontologies? > > a. should we provide a list of predicates and their intended > meanings (on a wiki, etc)? > > b. should we provide an RDF schema defining the predicates > (* probably the way to go) > > c. Should we allow the registration of new predicates, if > so, should we make this an api call, etc. > > _______________________________________________ > MOBY-dev mailing list > MOBY-dev at biomoby.org > http://www.biomoby.org/mailman/listinfo/moby-dev > > -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From jmfernandez at cnb.uam.es Wed Jul 20 15:03:56 2005 From: jmfernandez at cnb.uam.es (=?ISO-8859-1?Q?Jos=E9_Mar=EDa_Fern=E1ndez_Gonz=E1lez?=) Date: Wed, 20 Jul 2005 17:03:56 +0200 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42dd124c.7fdce064.76fe.ffff9b01@mx.gmail.com> References: <42dd124c.7fdce064.76fe.ffff9b01@mx.gmail.com> Message-ID: <42DE67DC.5060007@cnb.uam.es> > 5. LSIDs and Service Instances > > a..If a service instance is registered by 2 different > authoritys, should they have the same LSID? > Only if we can assure they are the SAME service (replicated services), which would require external help. > b. if the service instance is registered in 2 registries, > should the LSIDs be the same? > No, they shouldn't, unless they are registered by the same authority. > c. will modified service instances have versioning > information? > I think it should, because versioning information could help in the use of services from different registries, or the integration of those registries. -- Jos? Mar?a Fern?ndez Gonz?lez e-mail: jmfernandez at cnb.uam.es Tlfn: (+34) 91 585 54 50 Fax: (+34) 91 585 45 06 Grupo de Dise?o de Proteinas Protein Design Group Centro Nacional de Biotecnolog?a National Center of Biotechnology C.P.: 28049 Zip Code: 28049 C/. Darwin n? 3 (Campus Cantoblanco, U. Aut?noma), Madrid (Spain) From d.haase at gsf.de Wed Jul 20 15:12:06 2005 From: d.haase at gsf.de (Dirk Haase) Date: Wed, 20 Jul 2005 17:12:06 +0200 Subject: [MOBY-dev] Moby Collections In-Reply-To: <42DE5E17.8080302@ucalgary.ca> References: <42DE5E17.8080302@ucalgary.ca> Message-ID: <200507201712.06848.d.haase@gsf.de> On Wednesday 20 July 2005 16:22, Paul Gordon wrote: > I think Martin has hit the nail on the head here. One of the primary > advantages of MOBY is that you can't (easily) build arbitrarily > syntactically complex output. Correct. That's why we did not ask for nested collections. We suggest to allow for multiple Simple output as default - which I think is very appropriate because especially query services will usually return more than one result. This way we would save the collection concept for semantically related entities. > This ensures that you can chain together > services more easily. That is exactly our intention... > You should be building new objects in the > ontology to represent the cluster concept, not squeezing that cluster > logic into the syntax of the collection. Otherwise clients will do very > wierd things with your implicitly encoded concepts. What bears more implicitness: 1) passing totally unlrelated things together just because of a coincidental preceding step in the workflow or 2) putting things together because they build a semantic entity which is intended to be processed as a whole? > You always have to > look at how your data could be used by someone else who wasn't expecting > it... Well, expectations are ruled by the API - and that is what we are working on, aren't we? ;-) From senger at ebi.ac.uk Wed Jul 20 15:32:37 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Wed, 20 Jul 2005 16:32:37 +0100 (BST) Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: <42DE6578.3080008@cnb.uam.es> Message-ID: > > a. should article names for service instance input and > > output datatypes be mandatory? > > Well, in my opinion, I think they should be mandatory when there is more > than one input (or output). > This question hides actually two situations: a) article names for inputs (or outputs), and b) article names for elements attached by HAS[A] relationships. For b) situation I would say hat the article name should be mandatory, because one never knows who is going to extend his object and add new attributes. Having it mandatory will make life (and verification) easier. Also, it will help to understand what individual HASA attributes mean if they have an article name. For a) situation it is not that crucial, I would say. But I know that INB people will come with a suggestion how to handle errors in a new way, and for their solution the article names are important (if I remember it correctly). And a similar situation may appear again in the future. And because having article names mandatory is not hard to do (and if your current services do not do it, it will not break them, only some verifiers may complain) I am inclined to say make them mandatory. Regards, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From gordonp at ucalgary.ca Wed Jul 20 16:25:42 2005 From: gordonp at ucalgary.ca (Paul Gordon) Date: Wed, 20 Jul 2005 10:25:42 -0600 Subject: [MOBY-dev] rfcf 12. Services In-Reply-To: References: Message-ID: <42DE7B06.8050102@ucalgary.ca> I absolutely agree on (b). Anonymous data members is crazy. For (a) I would say that articleNnames should be manditory if there is more than one parameter. You could make a lot fancier rules, but you start dealing with a lot of parameter ambiguity issues. It's easier to just require them if there is any question. This would be a very minor change, since I don't think that there are many services that take more than one parameter, are there? All of the simple one-parameter services would still be valid. >>>a. should article names for service instance input and >>>output datatypes be mandatory? >>> >>> >>Well, in my opinion, I think they should be mandatory when there is more >>than one input (or output). >> >> >> > This question hides actually two situations: a) article names for >inputs (or outputs), and b) article names for elements attached by HAS[A] >relationships. > For b) situation I would say hat the article name should be mandatory, >because one never knows who is going to extend his object and add new >attributes. Having it mandatory will make life (and verification) easier. >Also, it will help to understand what individual HASA attributes mean if >they have an article name. > For a) situation it is not that crucial, I would say. But I know that >INB people will come with a suggestion how to handle errors in a new way, >and for their solution the article names are important (if I remember it >correctly). And a similar situation may appear again in the future. And >because having article names mandatory is not hard to do (and if your >current services do not do it, it will not break them, only some verifiers >may complain) I am inclined to say make them mandatory. > > Regards, > Martin > > > From schoof at mpiz-koeln.mpg.de Thu Jul 21 13:42:32 2005 From: schoof at mpiz-koeln.mpg.de (Heiko Schoof) Date: Thu, 21 Jul 2005 15:42:32 +0200 Subject: [moby] [MOBY-dev] Re: Problems with Biomoby servicesin Taverna 1.2 In-Reply-To: References: Message-ID: <776ef61c9a8901624450044125ab9b81@mpiz-koeln.mpg.de> Hi all again, I'll respond to Martin's query here: My understanding is that (talking about one mobyData object): a) Any Moby service can have more outputs. If so, all of them must be registered. The number of such outputs must be fixed. b) Any of these outputs can be of type either Simple or Collection. c) If it is a Collection, this output can have one or more Simples in that Collection. Such Simples (and their number) are not individually registered. I think the issue is point a, the number of such outputs must be fixed. Or, as the API says "each of these articles will appear EXACTLY ONCE in the output from the service". I request to change this to "each of these articles will appear AT LEAST ONCE in the output from the service". Why is this necessary? Currently, services that return potentially many results are registered as outputting a Collection. But within workflows, the iteration strategy or how often the next service is called should distinguish between an output made up of many independent items versus one or many groups of connected items. In the first case, the following service should be called once for each item, in the second case, only once for each group. See below for full example. What I am suggesting is to separate the cardinality from the Simple/Collection issue; meaning, that a service that performs e.g. a database lookup and returns 1 or many outputs (or none, but in the Moby world this means it returns 1 empty output) will be registered as returning Simple, not Collection, if the outputs are otherwise semantically unrelated (aside from the fact that they arose from the same query). And reserve the Collection article for grouping of outputs that need to be seen as a single entity. E.g. a service that outputs ortholog pairs given as input a pair of organisms: Each ortholog pair could be represented as a pair of GenericSequence objects in a Collection, with the service outputing 1 or many of these Collections. The same service, given as input three organisms, could still output many Collections, then containing three sequences each. This prevents ugly explosion of specialized BioMoby objects like "MultipleSequences", "HAS GenericSequence(s)"... that would otherwise be needed to wrap this. For service discovery, this should not make a difference. Services would still be required to return every Class of object that they register, as you state in a: all output object *Classes* must be registered, and the number of *Classes* fixed. I.e., a service registered as returning GenericSequence and AnnotatedJpeg objects must always return at least one GenericSequence plus at least one AnnotatedJpeg. I can't recall a service that actually does that... I can only think of this being meaningful if the AnnotatedJpeg is semantically connected to a specific GenericSequence, and in that case both should be connected through putting them in a Collection imho. For inputs, I'm not so worried; if multiple inputs are intended to go into a single service call, they will probably be connected and could go into a Collection. Example above: Input is a Collection of Organisms. Basically, I see that as the only way to register services that require AT LEAST two equal inputs. No problem with b, but with c: To my understanding, when registering a Collection, also the classes of objects in Simples that it contains must be registered. Otherwise, no discovery. However, see API, " A collection may contain zero or more Objects of each of the Classes", not all these classes must actually be included. So far, I do not see the need to distinguish between services that return EXACTLY ONE output and those that return one or more. Taverna seems to make that distinction, and bases iteration strategies on that, but I would want to do that dynamically, and it may be that that's what Taverna does. I'd by default assume that there will be multiple outputs and iterate over them, but if the workflow designer so wishes make it possible to (using e.g. a local processor) combine all the outputs into a Collection that can be used as the single input to a following service. This distinction is necessary: Use case examples would be a service returning a number of sequences, that in one scenario (iterate) should each be run through a BLAST service individually and in a different scenario (bag or Collection) should be all together input into a single call of a multiple alignment service. The current problem arises because Taverna now, in what is for me the semantically correct interpretation, if it receives a Collection as output from a service, it inputs that into a single execution of the following service if that service consumes Collection. In version 1.1 and before, Collections were decomposed by Taverna and iterated over. For the workflows being used, that was the wanted behavior, as e.g. keyword queries returning a set of sequences should be linked to services that act on each individual sequence. The mistake is imho on the BioMoby side, where we use Collection to wrap multiple outputs even if these are individual results that should be processed individually, and then, in order to be able to pipeline, register services that actually act on single inputs as consuming Collections. Consider Tom Oinn's comment 080705: 1) Consumer declares it consumes singles, Producer emits a collection. In this context Taverna iteratively calls the Consumer with each item from the collection. This is probably what you'd expect to happen, the result is that the Consumer effectively emits a collection of whatever it would emit normally. 2) Consumer declares it consumes a collection, Producer emits a collection. In this case Taverna will indeed split the output collection (because we always do) but it will be magically reassembled before being given to the Consumer. 3) Consumer declares it consumes a collection, Producer emits a single item. Taverna wraps the single item in a single element collection and gives it to the Consumer. This is the same logic that we'd need to implement into BioMoby to allow meaningful links between Collection producing services and Simple consuming services! And NOT register services as consuming or producing Collections if all they do is mimic this behaviour internally by iterating over the Collection items. Many words and I'm not sure this is making anything any clearer. But I try ;-) Best, Heiko On 18. Jul 2005, at 14:39 Uhr, Martin Senger wrote: Hi all, Catching up my email piles I wonder if someone can summarize if the discussion about collections in this thread brought any (planned) changes in the API (I am not talking now about how it is, or should be, implemented in Taverna, that's, imho, an another story). My understanding is that (talking about one mobyData object): a) Any Moby service can have more outputs. If so, all of them must be registered. The number of such outputs must be fixed. b) Any of these outputs can be of type either Simple or Collection. c) If it is a Collection, this output can have one or more Simples in that Collection. Such Simples (and their number) are not individually registered. Has this vision been changed? Thanks, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger _______________________________________________ MOBY-dev mailing list MOBY-dev at biomoby.org http://www.biomoby.org/mailman/listinfo/moby-dev From edward.kawas at gmail.com Fri Jul 22 15:08:09 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Fri, 22 Jul 2005 08:08:09 -0700 Subject: [MOBY-dev] FW: theseu - upload the first beta and unstable version Message-ID: <42e10bdc.61a5c3e4.09f0.062d@mx.gmail.com> -----Original Message----- From: Roman Roset [mailto:roset at ac.upc.edu] Sent: Friday, July 22, 2005 9:02 AM To: inb-tecnico at everest.cnb.uam.es; Eddie Kawas; Martin Senger Subject: theseu - upload the first beta and unstable version Hi all, The INB Theseu project is a framework for biomoby developers. The requirements of Theseu project are driven to develop BioMoby Services and to administrate BioMoby Repositories (locally or remotally). Future extensions of this project could include features such as RSS systems to get the biomoby news, local agents to perform some tasks directly with the RDF,etc... in short, the needs of biomoby and INB developers comunity. DOWNLOAD ======== 0. Documentation At the moment there is no user documentation. Sorry. Any voluntaries to help me writing the online documentation :) ? 1. Requirements These are the prerequisites to install the Theseu Project. They must be downloaded (EMF via eclipse site or manually) and installed before Theseu: Eclipse build eclipse-SDK-3.0.2: http://download.eclipse.org/eclipse/downloads/index.php EMF runtime 2.0.2 (emf-sdo-runtime-2.0.2): http://download.eclipse.org/tools/emf/scripts/downloads-view er.php?s=2.0.2/R200503151315 2. Theseu Project Runtime I have uploaded the first beta and unstable version of theseu at this site: Remote Site name: Theseu Project URL: http://inb.lsi.upc.edu/theseu/updates You can download easily via help -> software updates -> find & install -> "search for new features..." --> "new remote site". This beta distribution has 3 features grouped in two diferents topics: * Theseu Project |-> Theseu Project plugins |-> other (this is jmoby and others) Select all that you can checked :) and follow the afirmative buttons. 3. Mini example to start playing Once Theseu is installed, follow this steps: 1. click the Navigator window and press ctrl+N to open a new project wizard. 1.a. Select BioMoby Services Wizard --> new Moby Project 2. click on the label of the new project into de navigator window and press ctrl+N again to create a new repository: 2.a Select BioMoby Services Wizard --> new Moby Repository: 2.b. Select the server or push the preferences button to add new biomoby services. 2.c Select the authority or push the preferences button to add new authors. 3.d push finish button. 3. Now you can view in the editor window your empty local repository. The next step is to populate the ouline view with the element of the repository. So, click the label of a moby element into the editor (for instance "services") and click via the GeneralEditor menu or a popup menu (clicking over the "services" with the mouse right button) the "load..." option. 3.1 wait and don't despair... this is only necessary the first time. 4. Now you could open some views before to start with the edition: 4.1 Window --> show view --> Problems 4.2 Window --> show view --> Properties Remember that when you validate an element, the information to navigate through the messages is saved into the Problems view. 5. Register elements: it's only possible in this "demo" to register services. ABOUT THIS VERSION ================== I have uploaded this version with the aim of show the actual state of this project. I repeat that is only a demo version and it's very unstable, however you can have an idea of what Theseu can do and what will be able to do. Please send me your opinions about this. All your help is welcome. Thanks. Roman -- Roman Roset Mayals Technical Coordinator Instituto Nacional de Bioinformatica (www.inab.org) Barcelona Supercomputing Center Node (www.bsc.org.es) c/. Jordi Girona 1-3 Modul C6-E201 Tel. : 934 011 650 E-08034 Barcelona Fax : 934 017 014 Catalunya (Spain) e-mail: rroset at lsi.upc.edu Note: I'm on vacation until August 21th From roset at ac.upc.edu Fri Jul 22 04:23:49 2005 From: roset at ac.upc.edu (Roman Roset) Date: Fri, 22 Jul 2005 05:23:49 +0100 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances Message-ID: <42E074D5.2010205@ac.upc.es> >> 5. LSIDs and Service Instances >> >> a..If a service instance is registered by 2 different >> authoritys, should they have the same LSID? >> > Only if we can assure they are the SAME service (replicated services), > which would require external help. If I am not wrong, among other things, the lsid of a service would have to serve to obtain the RDF and I think that it should be interesting to have one-to-one relationship between RDF and service (or instance of a replicate service). For instance, if the RDF of a replicated service needs to save information about a concrete instance (such as the service is temporally unavailable, cpu's, os, etc...) These values could be different for two "same services". (but maybe I don't have clear what are same services - My understanding is that, two services are the same if given the same input then the output of both is exactly the same). However, I have some doubts about the way to specify the LSID of the services in biomoby. There are an example in the biomoby metadata-resolver: urn:lsid:biomoby.org:serviceinstance:www.illuminae.com,getSHound3DNeighboursFromGi (The LSID syntax is urn:lsid:authority:namespace:identifier:revision) Why is the authority "biomoby.org" and this one is not www.illuminae.com? Why is the service name separated of the lsid? If we use biomoby as authority but we wrote the services as identifiers then an approach to denote more of one instance (or replicated service) into the same lsid could be using the revision field. For instance: urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighboursFromGi:1 urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighboursFromGi:2 But sincerely, my opinion is that it's better to use an lsid like: urn:lsid:www.illuminae.com:biomoby.org:getSHound3DNeighboursFromGi urn:lsid:www.other.com:biomoby.org:getSHound3DNeighboursFromGi With this kind of LSID the MetaData-resolver could know that if the namespace is biomoby then the element to resolve is a service, and we should have one LSID per service. >> b. if the service instance is registered in 2 registries, >> should the LSIDs be the same? >> > No, they shouldn't, unless they are registered by the same authority. Is the RDF of this sevice shared by both registries? Is the response is yes then I feel that the LSID should be the same. (and the opposite, if the response is not the LSIDs should be different.) >> c. will modified service instances have versioning >> information? >> > I think it should, because versioning information could help in the use > of services from different registries, or the integration of those > registries. From my point of view, for one lsid of a service (independently of what thing is identifying exactly) there are two ways to understand the versioning information. I think that the version has to "explain" (useful) changes in the meta-information. But the service metainformation (the RDF) can change at two levels: the values of the predicates and the schema. I think that versions should be useful (at least for the agent) in the second case. -- Roman Roset Mayals Technical Coordinator Instituto Nacional de Bioinformatica (www.inab.org) Barcelona Supercomputing Center Node (www.bsc.org.es) c/. Jordi Girona 1-3 Modul C6-E201 Tel. : 934 011 650 E-08034 Barcelona Fax : 934 017 014 Catalunya (Spain) e-mail: rroset at lsi.upc.edu Note: I'm on vacation until August 21th From edward.kawas at gmail.com Fri Jul 22 17:12:54 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Fri, 22 Jul 2005 10:12:54 -0700 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42E074D5.2010205@ac.upc.es> Message-ID: <42e1291a.51939a07.08bd.ffff8d38@mx.gmail.com> > However, I have some doubts about the way to specify > the LSID of the > services in biomoby. There are an example in the biomoby > metadata-resolver: > > urn:lsid:biomoby.org:serviceinstance:www.illuminae.com,get > SHound3DNeighboursFromGi > > (The LSID syntax is > urn:lsid:authority:namespace:identifier:revision) > > Why is the authority "biomoby.org" and this one is not > www.illuminae.com? The authority is BioMoby.org because it's the issuing authority. In other words, it is the place where we can go and ask for information regarding a particular service and retrieve metadata about it. www.illuminae.com is the authority that registered the service getSHound3DNeighboursFromGi into the registry hosted by BioMoby.org. Does this make sense? > Why is the service name separated of the lsid? The name was separated so that in the event that there were 2 services registered with the same service name we wouldn't run into conflicts. Also, a service should be uniquely identified by the authority that registered it and its name. > If we use biomoby as authority but we wrote the services > as identifiers > then an approach to denote more of one instance (or > replicated service) > into the same lsid could be using the revision field. > For instance: > > urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours > FromGi:1 > urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours > FromGi:2 > The revision id is meant to convey to someone that the underlying object 'pointed to' by the lsid has changed. So for instance, when the agent is running, and it notices that the service instance has changed a little (new description added or a new input or output, etc), then the revision id would have to be used. > urn:lsid:www.illuminae.com:biomoby.org:getSHound3DNeighbou > rsFromGi > urn:lsid:www.other.com:biomoby.org:getSHound3DNeighboursFr > omGi > > With this kind of LSID the MetaData-resolver could know > that if the namespace is biomoby then the element to > resolve is a > service, and we should have one LSID per service. One of the problems with this approach is that how do you go about resolving other Moby objects like service types, datatypes, namespaces,etc? I believe that your proposal also doesn't follow the correct use of the LSID protocol. From markw at illuminae.com Fri Jul 22 17:26:06 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 22 Jul 2005 10:26:06 -0700 Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42e1291a.51939a07.08bd.ffff8d38@mx.gmail.com> References: <42e1291a.51939a07.08bd.ffff8d38@mx.gmail.com> Message-ID: <42E12C2E.8010406@illuminae.com> Edward Kawas wrote: >> However, I have some doubts about the way to specify >>the LSID of the >>services in biomoby. There are an example in the biomoby >>metadata-resolver: >> >>urn:lsid:biomoby.org:serviceinstance:www.illuminae.com,get >>SHound3DNeighboursFromGi >> >>(The LSID syntax is >>urn:lsid:authority:namespace:identifier:revision) >> >>Why is the authority "biomoby.org" and this one is not >>www.illuminae.com? >> >> >The authority is BioMoby.org because it's the issuing >authority. In other words, it is the place where we can go >and ask for information regarding a particular service and >retrieve metadata about it. www.illuminae.com is the >authority that registered the service >getSHound3DNeighboursFromGi into the registry hosted by >BioMoby.org. Does this make sense? > > Just to expand this a little bit more - biomoby.org will NOT be the resolver for that LSID, but it will be the authority server for that LSID. >>Why is the service name separated of the lsid? >> >> >The name was separated so that in the event that there were >2 services registered with the same service name we wouldn't >run into conflicts. Also, a service should be uniquely >identified by the authority that registered it and its name. > > In the early days when we were first considering LSID's in MOBY, there wasn't a widely agreed-upon standard for "wrapping" someone elses identifier. It seems now that there is a "standard" (I don't know if this is a recommendation from teh LSID community or not, but it seems that the core LSID developers do it themselves, so it must be supported). We might consider changing the LSID structure for MOBY Service Instances to e.g.: urn:lsid:biomoby.org.www.illuminae.com:serviceinstance:getGenBankff It makes no difference at all at the end of the day, since LSID's are opaque identifiers :-) >>If we use biomoby as authority but we wrote the services >>as identifiers >>then an approach to denote more of one instance (or >>replicated service) >> into the same lsid could be using the revision field. >>For instance: >> >>urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours >>FromGi:1 >>urn:lsid:biomoby.org:serviceinstance:getSHound3DNeighbours >>FromGi:2 >> >> >> >The revision id is meant to convey to someone that the >underlying object 'pointed to' by the lsid has changed. So >for instance, when the agent is running, and it notices that >the service instance has changed a little (new description >added or a new input or output, etc), then the revision id >would have to be used. > > Yup... that suggestion (to use the version position to denote replication) would be very dangerous! However, it is an interesting problem, since I don't think the LSID communuty would be very comfortable with a single LSID referring to multiple resources... even if they are "identical". Martin, do you have an opinion about this? M From roset at ac.upc.edu Sat Jul 23 19:56:08 2005 From: roset at ac.upc.edu (Roman Roset) Date: Sat, 23 Jul 2005 20:56:08 +0100 Subject: [MOBY-dev] FW: theseu - upload the first beta and unstable version Message-ID: <42E2A0D8.4070109@ac.upc.es> Hello, thank you very much eddie, to forward my email in this list :). I've uploaded a ppt preview of INB-Theseu: http://inb.lsi.upc.es/theseu/docs/ppt/TheseuPreview.ppt There are bugs on this demo, but my intention is to know the general opinion of moby-developers in order to driven the design of future functionalities. So, all your comments, questions or suggestions are most welcome. INB Theseu people: Author: Roman Roset (roset at ac.upc.edu) Coordinator: David G. Pisano (dgpisano at cnb.uam.es) Supervised by: Phd. Xavier Messeguer and Phd. Oswaldo Trelles Theseu is property of INB - Instituto Nacional de Bioinform?tica (www.inab.org) Thanks. Note: I'm on vacation until August 21th -- Roman Roset Mayals Technical Coordinator Instituto Nacional de Bioinformatica (www.inab.org) Barcelona Supercomputing Center Node (www.bsc.org.es) c/. Jordi Girona 1-3 Modul C6-E201 Tel. : 934 011 650 E-08034 Barcelona Fax : 934 017 014 Catalunya (Spain) e-mail: rroset at lsi.upc.edu From markw at illuminae.com Mon Jul 25 15:37:11 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 25 Jul 2005 08:37:11 -0700 Subject: [moby] [MOBY-dev] FW: theseu - upload the first beta and unstable version In-Reply-To: <42E2A0D8.4070109@ac.upc.es> References: <42E2A0D8.4070109@ac.upc.es> Message-ID: <1122305831.29736.58.camel@bioinfo.icapture.ubc.ca> Hokey Dinah! That's AMAZING!!! M On Sat, 2005-07-23 at 20:56 +0100, Roman Roset wrote: > Hello, > > thank you very much eddie, to forward my email in this list :). > > I've uploaded a ppt preview of INB-Theseu: > > http://inb.lsi.upc.es/theseu/docs/ppt/TheseuPreview.ppt > > > There are bugs on this demo, but my intention is to know the general > opinion of moby-developers in order to driven the design of future > functionalities. So, all your comments, questions or suggestions are > most welcome. > > INB Theseu people: > > Author: Roman Roset (roset at ac.upc.edu) > Coordinator: David G. Pisano (dgpisano at cnb.uam.es) > Supervised by: Phd. Xavier Messeguer and Phd. Oswaldo Trelles > > Theseu is property of INB - Instituto Nacional de Bioinform?tica > (www.inab.org) > > Thanks. > > Note: I'm on vacation until August 21th > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From markw at illuminae.com Mon Jul 25 15:37:11 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Mon, 25 Jul 2005 08:37:11 -0700 Subject: [moby] [MOBY-dev] FW: theseu - upload the first beta and unstable version In-Reply-To: <42E2A0D8.4070109@ac.upc.es> References: <42E2A0D8.4070109@ac.upc.es> Message-ID: <1122305831.29736.58.camel@bioinfo.icapture.ubc.ca> Hokey Dinah! That's AMAZING!!! M On Sat, 2005-07-23 at 20:56 +0100, Roman Roset wrote: > Hello, > > thank you very much eddie, to forward my email in this list :). > > I've uploaded a ppt preview of INB-Theseu: > > http://inb.lsi.upc.es/theseu/docs/ppt/TheseuPreview.ppt > > > There are bugs on this demo, but my intention is to know the general > opinion of moby-developers in order to driven the design of future > functionalities. So, all your comments, questions or suggestions are > most welcome. > > INB Theseu people: > > Author: Roman Roset (roset at ac.upc.edu) > Coordinator: David G. Pisano (dgpisano at cnb.uam.es) > Supervised by: Phd. Xavier Messeguer and Phd. Oswaldo Trelles > > Theseu is property of INB - Instituto Nacional de Bioinform?tica > (www.inab.org) > > Thanks. > > Note: I'm on vacation until August 21th > -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From senger at ebi.ac.uk Tue Jul 26 12:14:43 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 26 Jul 2005 13:14:43 +0100 (BST) Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42E12C2E.8010406@illuminae.com> Message-ID: > Yup... that suggestion (to use the version position to denote > replication) would be very dangerous! However, it is an interesting > problem, since I don't think the LSID communuty would be very > comfortable with a single LSID referring to multiple resources... even > if they are "identical". Martin, do you have an opinion about this? > This is about replicated services that were (afaik) first mentioned (at least with some depth) in Vancouver. Before we decide about how to create LSID for such kind of services, we need to define what are replicated services. So far, each service was uniquely defined by three elements: - its endpoint, - its authority (in the Biomoby sense, not the LSID sense), and - its (simple) name (unique within its authority). So what are the replicated services? My take (after what I heard in Vancouver) is that the replicated services will have the same two of the attributes above: authority and name, but they will not share the same endpoint. (Additionally, just in documenttaion, we claim that such replicated services do the same data transformation - but this is not check-able.) Is this how we want to define replicated services? If the answer is yes, here is how I would feel about the LSID: 1) Resolving Data. Eddie mentioned that he (or he and Mark, I was not sure) were/are thinking about using LSID resolution method 'getData' for returning a WSDL of this service. I am not sure that I identify myself with this idea. Mainly because we already have a way how to get the same WSDL from the registry (there is an API method for that) so why to duplicate it. And also because doing this we will immediately need different LSID for replicated services (because the endpoint is - at least, usully - part of such WSDL). So I would suggest to return nothing by this method. If this is accepted, I ca elaborate further what to do with metadata. 2) Resolving metadata. Mark is not sure that the replicated services should have the same LSID. I am not sure either - but I am more inclined to say 'Why not?'. They do the same think - except they some of them can do it better (faster, more reliably etc.) But this can be reflected in metadata - it includes the endpoint, so the user will see which metadata refers to which replicated service. We would need to make some consensus on the metadata predicates to assure that the metadata can be identified: which are meant for all such replicated services, and which are meand only for individual such services. If we do not want to have same LSID for replicated services we will need to solve the following with more efforts: 3) Problems with *not* having the same LSID for replicated services: a) The Registry API is going to change because of replicated services (as we talked about it in V.) - but the change would have me more significant than just to chnage the 'endpoint URL' to a set of 'endpoint URLs' - because when the registry returns LSID, it must return more/many LSIDs about the same service. b) The registry would have more problems to collect all available metadata (or to send SignatureURLs of all of them) if their are more LSIDs for the same service. These problems are surely solveable but I wonder why the LSID cannot be same for the replicated services (I also feel that it is a bit strange, but I cannot find the real arguments why not to do it.). Regards, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From senger at ebi.ac.uk Tue Jul 26 12:21:40 2005 From: senger at ebi.ac.uk (Martin Senger) Date: Tue, 26 Jul 2005 13:21:40 +0100 (BST) Subject: [MOBY-dev] rfcf 5. LSIDs and Service Instances In-Reply-To: <42E12C2E.8010406@illuminae.com> Message-ID: > Just to expand this a little bit more - biomoby.org will NOT be the > resolver for that LSID, but it will be the authority server for that LSID. > Roman, this remark from Mark is true, but it may be obscure for you unless you know details of the LSID spec. The LSID resolution service specification actually specifies that the data/metadata resolution is done in several steps: - The first is called 'authority server resolution' (or similarly) - and this will be done at biomoby.org. This returns back a list of URLs (and protocols, but the protocols in our case will be only HTTP-GET). - The next step is to use these URLs and fetch what they represent - and this means that you are going to fetch RDF documents from the service provider site (so actually the service providers themselves suddenly become LSID resolution providers). Does it make sense for you? Cheers, Martin -- Martin Senger EMBL Outstation - Hinxton Senger at EBI.ac.uk European Bioinformatics Institute Phone: (+44) 1223 494636 Wellcome Trust Genome Campus (Switchboard: 494444) Hinxton Fax : (+44) 1223 494468 Cambridge CB10 1SD United Kingdom http://industry.ebi.ac.uk/~senger From markw at illuminae.com Fri Jul 29 21:35:17 2005 From: markw at illuminae.com (Mark Wilkinson) Date: Fri, 29 Jul 2005 14:35:17 -0700 Subject: [MOBY-dev] 0.85 codebase running on test server Message-ID: <1122672917.19643.31.camel@bioinfo.icapture.ubc.ca> Hi dev-ers Anyone watching the commit logs will have seen a lot of activity in the past week! The 0.85 codebase is running now on a test server so that you can start to code to it. Endpoint information is as follows: URL => 'http://bioinfo.icapture.ubc.ca/cgi-bin/mobycentral/MOBY- Central.pl', URI => 'http://bioinfo.icapture.ubc.ca/MOBY/Central The main (visible) difference is that many of the messages now contain LSID information - see the 0.85 API on the biomoby.org website for full details. In addition the use of LSID's in MOBY Central messages should be MUCH more consistent now. The LSID resolver is (should be!) working, so if you try to resolve those LSID's to metadata you will get useful information. Under the hood, Dennis Wang has factored out all of the data-access calls into a single adaptor module (good job Dennis!). Hopefully this will simplify our migration toward the myGrid registry, since we will only need to modify one module. heads-up that Eddie is working on this same test server with the RDF agent, so things you register there may not last for long as they are obliterated by agent activity (and bugs ;-) ), but the MOBY Central code (what is currently in the CVS) seems to be running correctly on that machine so you can at least use it to observe the new message structure and update your code if necessary in anticipation of the production server switching to this codebase. If you have time/ambition/interest please do test this code and let me know if you notice anything incorrect. One of the biggest improvements, in my opinion, is that there is now a very comprehensive test suite for the Perl code - 122 tests! (up from just 12 before). All tests pass on two of my servers, so I think the code is now ~stable. From here on, any bugs that are reported will be fixed together with an addition to the test suite to make sure they don't reappear. As such, the codebase should never degrade again :-) Yes, we're growing up and becoming a mature project! Nota Bene that the test suite DOES NOT PASS ALL TESTS against the current production registry (@mobycentral.icapture.ubc.ca). This is because there are some latent bugs in the current production code that are now revealed by the test suite!! That's all the news. In the pipeline: 1) I will soon commit code changes that prevent inheritance from Primitives 2) I'll create and register replacement objects for all objects that currently inherit from primitives such that they follow the paradigm for object construction that has been extensively discussed since the last meeting. 2a) I will attempt to contact all service providers who will be affected by this change based on their registered contactEmail address 3) Eddie has a test suite of ~50 tests for the RDF agent, however it is not yet passing all tests. Once we believe this is stable/safe we will switch it on and from that point onwards deregistration of services will occur only through agent activity. 4) We are assembling a thesaurus of RDF predicates for service providers to use in their service instance metadata. This will be published shortly. Eddie and I leave for 10 days in Manchester on Sunday, so hopefully there will be more news and progress by the time we get home. Cheers everyone! Mark -- "Ontologists do it with the edges!" Mark Wilkinson Asst. Professor Dept. of Medical Genetics University of British Columbia PI in Bioinformatics iCAPTURE Centre St. Paul's Hospital Rm. 166, 1081 Burrard St. Vancouver, BC, V6Z 1Y6 tel: 604 682 2344 x62129 fax: 604 806 9274 From edward.kawas at gmail.com Mon Jul 25 16:36:58 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Mon, 25 Jul 2005 16:36:58 -0000 Subject: [MOBY-dev] FW: Error handling proposal Message-ID: <42e5173f.2fdbe4fa.4c5a.ffffd788@mx.gmail.com> -----Original Message----- From: Oswaldo Trelles [mailto:ots at ac.uma.es] Sent: Monday, July 25, 2005 9:43 AM To: Eddie Kawas Subject: Error handling proposal Eddie, could you please circulate this message and make the document available following your protocol? thanks in advance Oswaldo Please let me introduce myself. My name is Oswaldo Trelles and I work at the Computer architecture Department of the University of Malaga, Spain (where all of you are invited and welcomed). Our group is in charge of the "Integrated Bioinformatics" aspects at the INB (National Institute of Bioinformatics). As David Gonz?les Pisano informed you at the Vancouver meeting we are developing an integrated platform to deploy general and specific services using BioMOBY as the underlying protocol. Thus we are quite concerned with BM specifications and we would like to contribute in the development, extension and improvements in the protocol. At this moment Roman has made a preliminary version and description of Theseu project available. Now we would like to put over the table the error handling issue. We have prepared a document, which represent an extension of our current implementation. This proposal was discussed during the INB-meeting here in Malaga (July 2005) and now is distributed as an open document for further discussion. Please feel free to comment. best regards, Oswaldo + GNV5 team + INB team -------------- next part -------------- A non-text attachment was scrubbed... Name: 0509GNV5-ErrorHandling-v034distributed.pdf Type: application/pdf Size: 338293 bytes Desc: not available URL: From edward.kawas at gmail.com Wed Jul 27 15:20:26 2005 From: edward.kawas at gmail.com (Edward Kawas) Date: Wed, 27 Jul 2005 15:20:26 -0000 Subject: [MOBY-dev] FW: Error handling proposal Message-ID: <42e7a864.0e06ba61.207b.ffffdb6b@mx.gmail.com> Please let me introduce myself. My name is Oswaldo Trelles and I work at the Computer architecture Department of the University of Malaga, Spain (where all of you are invited and welcomed). Our group is in charge of the "Integrated Bioinformatics" aspects at the INB (National Institute of Bioinformatics). As David Gonz?les Pisano informed you at the Vancouver meeting we are developing an integrated platform to deploy general and specific services using BioMOBY as the underlying protocol. Thus we are quite concerned with BM specifications and we would like to contribute in the development, extension and improvements in the protocol. At this moment Roman has made a preliminary version and description of Theseu project available. Now we would like to put over the table the error handling issue. We have prepared a document, which represent an extension of our current implementation. This proposal was discussed during the INB-meeting here in Malaga (July 2005) and now is distributed as an open document for further discussion. Please feel free to comment. best regards, Oswaldo + GNV5 team + INB team -------------- next part -------------- A non-text attachment was scrubbed... Name: 0509GNV5-ErrorHandling-v034distributed.pdf Type: application/pdf Size: 190789 bytes Desc: not available URL: