From dalke at dalkescientific.com Wed Apr 6 03:53:00 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Wed, 6 Apr 2005 01:53:00 -0600 Subject: [DAS2] feature filter examples Message-ID: <4ccf0d170ccaf8cbd0ce8dc398c7fe8f@dalkescientific.com> I'm having some difficulty understanding the feature query language. There's an example in the spec which is overlaps=Chr3/1000:2000;type=exon,intron,splice_site (Features of type exon, intron, or splice_site that overlap region 1000-2000 on Chr3.) Based on that, the following appears to be legal overlaps=Contig1/1:1000;att=hand:fizzbin;overlaps=Contig1/5:200 That is, it is legal for the same feature filter ("overlaps") to exist multiple times in the query. If so there should be a note pointing that out. Are the following allowed? 1) an OR of the values for a region overlaps=Contig1/1:1000,Contig8/5:200 2) an OR of an att query att=hand:fizzbin,slark 3) and OR and an AND of the same attribute in an att filter att=hand:fizzbin,slark;att=hand:royal This is the same as (hand contains "fizzbin" OR hand contains "slark") AND (hand contains "royal") so would match "royal fizzbin" hands. I think they are all legal. I think I can improve the wording a bit for clarification. Andrew dalke at dalkescientific.com From dalke at dalkescientific.com Wed Apr 6 04:28:24 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Wed, 6 Apr 2005 02:28:24 -0600 Subject: [DAS2] Re: feature filter examples In-Reply-To: <4ccf0d170ccaf8cbd0ce8dc398c7fe8f@dalkescientific.com> References: <4ccf0d170ccaf8cbd0ce8dc398c7fe8f@dalkescientific.com> Message-ID: <4852f837562e1f84abcf986f4400a037@dalkescientific.com> Another confusing situation. Is the following allowed att=name:Andrew,lang:en to mean attribute name contains "Andrew" OR attribute lang contains "en" This appears to be legal under the current spec because Within the URL query string, multiple criterion=value pairs are ANDED together: ... To describe OR relationships, you may separate filter values by commas. In my above example the "criterion" is "att" (because it's to the left of the "=") and the "value" is "name:Andrew,lang:en" which is made of two value fields "name:Andrew" "lang:en" Note though that that means att=name:Andrew,lang:en does not mean to find attribute "name" contains "Andrew" OR attribute "name" contains "lang:en" Nor is it attribute "name" contains "Andrew" OR attribute "name" contains "lang" SYNTAX ERROR - ':' not allowed at this point It also means that att=name:Andrew,Gregg IS NOT legal, because "Gregg" is not a valid value for the "att" criterion. Andrew dalke at dalkescientific.com From allenday at ucla.edu Wed Apr 6 12:02:40 2005 From: allenday at ucla.edu (Allen Day) Date: Wed, 6 Apr 2005 09:02:40 -0700 (PDT) Subject: [DAS2] feature filter examples In-Reply-To: <4ccf0d170ccaf8cbd0ce8dc398c7fe8f@dalkescientific.com> References: <4ccf0d170ccaf8cbd0ce8dc398c7fe8f@dalkescientific.com> Message-ID: You're right -- this is unclear. I was talking to Lincoln about this Monday. The following are true by the current document for key=value strings: [1] comma-delimited values for the same key are ORed [2] different keys are ANDed But is unclear how to handle the separate values for keys of the same name. I would interpret separate values as a case of [2], as you do here: > att=hand:fizzbin,slark;att=hand:royal > > This is the same as > (hand contains "fizzbin" OR hand contains "slark") AND > (hand contains "royal") matching features with "hand" attribute subset values of {royal,fizzbin} and {royal,slark}. We might consider revising this though. It could create problems with other technologies that use GET, for example HTML forms with GET method. You might want to have a multi-select list that allows you to select attribute values for "hand", and if you chose "fizzbin" and "slark", you'd get a URI param string looking like: .../feature?att=hand:fizzbin;att=hand:slark As far as I'm aware there's no way to change this short of having another script handle this request, munging the parameters to what the user really meant, and returning the browser a redirect header. -Allen On Wed, 6 Apr 2005, Andrew Dalke wrote: > I'm having some difficulty understanding the feature query > language. There's an example in the spec which is > > overlaps=Chr3/1000:2000;type=exon,intron,splice_site > > (Features of type exon, intron, or splice_site that > overlap region 1000-2000 on Chr3.) > > Based on that, the following appears to be legal > > overlaps=Contig1/1:1000;att=hand:fizzbin;overlaps=Contig1/5:200 > > That is, it is legal for the same feature filter ("overlaps") > to exist multiple times in the query. If so there should > be a note pointing that out. > > > Are the following allowed? > > 1) an OR of the values for a region > > overlaps=Contig1/1:1000,Contig8/5:200 > > 2) an OR of an att query > > att=hand:fizzbin,slark > > 3) and OR and an AND of the same attribute in an att filter > > att=hand:fizzbin,slark;att=hand:royal > > This is the same as > (hand contains "fizzbin" OR hand contains "slark") AND > (hand contains "royal") > > so would match "royal fizzbin" hands. > > I think they are all legal. I think I can improve > the wording a bit for clarification. > > > Andrew > dalke at dalkescientific.com > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 > From ed_erwin at affymetrix.com Wed Apr 6 18:29:57 2005 From: ed_erwin at affymetrix.com (Ed Erwin) Date: Wed, 06 Apr 2005 15:29:57 -0700 Subject: [DAS2] feature filter examples In-Reply-To: References: Message-ID: <425462E5.5080000@affymetrix.com> I'm glad someone is paying attention to these details. Please also keep in mind making the parsing of the parameters straitforward. There should be some simple deterministic algorithm like, for example, this: 1. Split the string at ';' 2. apply URL-decoding (to convert "%20" and "+" to " ", etc.) 3. split at the first "=" into left-hand and right-hand 4. split right-hand at the first ":" into right-hand-one and right-hand-two 5. split right-hand-two at each "," I'm certain this is not the exactly right formula, but I'm saying we need to make sure that there is some known formula that works and is unambiguous. Consider the tough cases like spaces and punctuation characters like "hand:fizz bin 3.0,genscan+" Allen Day wrote: > You're right -- this is unclear. I was talking to Lincoln about this > Monday. The following are true by the current document for key=value > strings: > > [1] comma-delimited values for the same key are ORed > [2] different keys are ANDed > > But is unclear how to handle the separate values for keys of the same > name. I would interpret separate values as a case of [2], as you do here: > > > att=hand:fizzbin,slark;att=hand:royal > > > > This is the same as > > (hand contains "fizzbin" OR hand contains "slark") AND > > (hand contains "royal") > > matching features with "hand" attribute subset values of {royal,fizzbin} > and {royal,slark}. > > We might consider revising this though. It could create problems with > other technologies that use GET, for example HTML forms with GET method. > You might want to have a multi-select list that allows you to select > attribute values for "hand", and if you chose "fizzbin" and "slark", you'd > get a URI param string looking like: > > .../feature?att=hand:fizzbin;att=hand:slark > > As far as I'm aware there's no way to change this short of having another > script handle this request, munging the parameters to what the user really > meant, and returning the browser a redirect header. > > -Allen > > > On Wed, 6 Apr 2005, Andrew Dalke wrote: > > > I'm having some difficulty understanding the feature query > > language. There's an example in the spec which is > > > > overlaps=Chr3/1000:2000;type=exon,intron,splice_site > > > > (Features of type exon, intron, or splice_site that > > overlap region 1000-2000 on Chr3.) > > > > Based on that, the following appears to be legal > > > > overlaps=Contig1/1:1000;att=hand:fizzbin;overlaps=Contig1/5:200 > > > > That is, it is legal for the same feature filter ("overlaps") > > to exist multiple times in the query. If so there should > > be a note pointing that out. > > > > > > Are the following allowed? > > > > 1) an OR of the values for a region > > > > overlaps=Contig1/1:1000,Contig8/5:200 > > > > 2) an OR of an att query > > > > att=hand:fizzbin,slark > > > > 3) and OR and an AND of the same attribute in an att filter > > > > att=hand:fizzbin,slark;att=hand:royal > > > > This is the same as > > (hand contains "fizzbin" OR hand contains "slark") AND > > (hand contains "royal") > > > > so would match "royal fizzbin" hands. > > > > I think they are all legal. I think I can improve > > the wording a bit for clarification. > > > > > > Andrew > > dalke at dalkescientific.com > > > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 > From dalke at dalkescientific.com Thu Apr 7 01:53:37 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Wed, 6 Apr 2005 23:53:37 -0600 Subject: [DAS2] feature filter examples In-Reply-To: <425462E5.5080000@affymetrix.com> References: <425462E5.5080000@affymetrix.com> Message-ID: Ed: > I'm glad someone is paying attention to these details. That's what I'm paid for. :) > Please also keep in mind making the parsing of the parameters > straitforward. There should be some simple deterministic > algorithm like, for example, this: One concern I have is the mix of two different levels of URL escaping. Suppose I searched for the "name" "Dalke,Andrew" To make it fit in with at least Python's standard CGI GET query parser it would need to look like this: name=Dalke%252C+Andrew >>> import cgi >>> cgi.parse_qs("name=Dalke%252CAndrew") {'name': ['Dalke%2CAndrew']} >>> There is a double escape of the "," because the normal cgi processing rules allow anything to be escaped, even the special characters of ';' and ','. If I only had one level of escaping then it would be processed as >>> cgi.parse_qs("name=Dalke%2CAndrew") {'name': ['Dalke,Andrew']} >>> Note that my search for the exact string "Dalke,Andrew" has now been turned into a search for "Dalke" OR "Andrew". I think the algorithm for parsing this string is as follows. 1. URL-decode the query string 2. Split the string at ';'. These sub terms will be ANDed together. 3. For each subterm, find the category (the part before the '=' sign) and the value (the part after the '='). 4. Split the value on ','. These sub terms will be ORed together. 5a. If the category is 'att' then look for the attribute name before the ':' and the attribute value after the ':'. This is a URL-encoded glob query. 5b. Otherwise this is a URL-encoded query as appropriate for the given category. Steps 1, 2 and 3 are done as part of any standard CGI library. Another option which came up at the pre-DAS/2 meeting at CSHL 1.5 years ago was use a Google (or Entrez ;) style query, which would look like type:exon contains:Chr1/50:99 name:"Dalke, Andrew" or pfam:"protein kinase" overlaps:Chr3/1000:2000:-1 Google allows a simple OR in its syntax. pfam:"protein kinase" OR pfam:connexin inside:Chr3 The OR only applies to the two terms on either side so this is the same as (pfam:"protein kinase" OR pfam:connexin) AND inside:Chr3 I assume that name:Andrew OR name:Gregg OR name:Ed AND inside:Chr3 would be the same as ((name:Andrew OR name:Gregg) OR name:Ed) AND inside:Chr3 And a recollection from logic class - all boolean queries can be written in conjunctive normal form; as the AND of a set of OR statements. It just might be very verbose. ;) Note in my example that I've eliminated the "att" prefix so there nothing visually to distinguish between generic/database-defined attributes and DAS/2 required properties. Do we need that? An advantage is that it keeps the DAS/2 search names distinct from properties in the database. If that's important then I'll suggest we allow the prefix of "att:" if people want to be sure that there won't be a conflict, but that it's optional. That is: name:Andrew att:name:Andrew selects features which have the DAS/2 defined feature name of "Andrew" AND the arbitrary database "name" attribute containing "Andrew" The other possibility is to reserve "das:" for fields defined in the spec, making this query das:name:Andrew name:Andrew or to make things prettier (I don't like the double ':') das-name:Andrew name:Andrew A few other things I've noticed in the query spec. The definition for the "att" field says Glob-style wildcards are allowed in the values. There's an example which looks like att=est_evidence:1 Match features with an "est_evidence" property of "1" Does this match "1" identically or does it also match "100" and "91"? If the latter aren't allowed then suppose I do att=est_evidence:*1* Are 100 and 91 now possible matches? Why don't we weaken the requirements for how a server must respond to a given query? Right now all attribute fields must be searchable as strings by glob. Some fields may only support searching by term, perhaps with limited stemming. Others may be numeric with no glob support at all. Some implementers may even want to support range searches for a given numeric field, perhaps like: weight:1000..2500 BTW, why are we using "att" for this prefix instead of "prop"? After all, aren't these the property names? Speaking of which, given a property attribute name like "weight" how does a client get to a description of that property? It looks like it's magic - it's tacked on to the base property URL so base property URL: http://www/das/genome/volvox/1/property/ property name: weight means the description can be found at: http://www/das/genome/volvox/1/property/weight Going the other way around, to get a list of potential property fields, does a client start with the PROPERTIES request at http://www/das/genome/volvox/1/property/ then get the fully resolved 'id' for each PROPERTY (which may be joined with the base or may be a full URI itself) and extract the last term in the id? If so, that won't work with LSIDs or other opaque identifier. Better would be to add a "name" attribute to the PROPERTY element so the whole thing looks like and the 'name' field is what's used in the queries. This could be expanded some more to flag which fields are searchable and a documentation for how to do the search. Eg, the "can_query" attribute if it exists and is anything other the empty string means that searches are allowed on the given property and the "query_href" attribute points to documentation about the format of that query. Andrew dalke at dalkescientific.com From ed_erwin at affymetrix.com Thu Apr 7 19:54:47 2005 From: ed_erwin at affymetrix.com (Ed Erwin) Date: Thu, 07 Apr 2005 16:54:47 -0700 Subject: [DAS2] feature filter examples In-Reply-To: References: Message-ID: <4255C847.3090104@affymetrix.com> What happened to the "+" symbol in "name=Dalke%252C+Andrew" ? Was that supposed to be a URL-encoded space? If so, is that your recommended way of encoding it (single-encoded vs. double-encoded) ? Andrew Dalke wrote: > One concern I have is the mix of two different levels of URL > escaping. Suppose I searched for the "name" "Dalke,Andrew" > To make it fit in with at least Python's standard CGI GET > query parser it would need to look like this: > > name=Dalke%252C+Andrew > > >>> import cgi > >>> cgi.parse_qs("name=Dalke%252CAndrew") > {'name': ['Dalke%2CAndrew']} > >>> > From dalke at dalkescientific.com Thu Apr 7 20:14:41 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Thu, 7 Apr 2005 18:14:41 -0600 Subject: [DAS2] feature filter examples In-Reply-To: <4255C847.3090104@affymetrix.com> References: <4255C847.3090104@affymetrix.com> Message-ID: <74a05dc72b4afc8bd6de623c96e6e11b@dalkescientific.com> Ed: > What happened to the "+" symbol in "name=Dalke%252C+Andrew" ? > > Was that supposed to be a URL-encoded space? If so, is that your > recommended way of encoding it (single-encoded vs. double-encoded) ? Whoops! I started with a space but then figured it would distract from the discussion (having two escaped fields instead of one) so I hand-deleted it. Looks like I didn't catch everything. Andrew dalke at dalkescientific.com From dalke at dalkescientific.com Fri Apr 8 02:10:57 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Fri, 8 Apr 2005 00:10:57 -0600 Subject: [DAS2] feature filter examples In-Reply-To: <4255C847.3090104@affymetrix.com> References: <4255C847.3090104@affymetrix.com> Message-ID: <45f24ce628766b59916de4e4a5ec05dc@dalkescientific.com> I just realized I forgot today's conference call! Ed: > Was that supposed to be a URL-encoded space? If so, is that your > recommended way of encoding it (single-encoded vs. double-encoded) ? I didn't answer that one. It doesn't make a difference. The unescaping for both '+' and '%xx' are done by the same piece of code. I've been thinking more about simple query languages. The one I proposed is Google like. I also remembered that Lucene has a Google like query language. http://lucene.apache.org/java/docs/queryparsersyntax.html except it's rather more complicated because the query engine is more powerful. And unlike Google multiple terms with implicit conjunction are treated as an OR search instead of an AND search. (So "this that" is the same as "this OR that".) Then there's MySQL's boolean search language at http://dev.mysql.com/doc/mysql/en/fulltext-boolean.html which has a different set of operations but doesn't allow qualified searches like name:Romana . All are different, but at least have a common core of qualifier:search_text Compare to Entrez which looks like search_text[qualifier] In taking a closer look at the corner cases I noticed that it's awfully hard to handle non-alpha characters. Consider a search for 5'. In Google that gets turned into a search for 5. "5' end" (without double quotes in the search) has a few non-relevant hits in the top 10, like for NetScreen-5 End of Life (EOL) Announcement How well do we want to support special characters? DAS/1 includes examples like "5'UTR" Is case sensitivity important? How much should be required by the spec and how much can be left up to the implementor? Realistically speaking I don't think we can make that many restrictions on what the server can do. As I mentioned, I'm not even sure if we can always support the glob-style searches specified by the current spec. (Eg, with numeric fields. Also, how can it be implemented in MySQL?) I'm fine with a simple query language like this: query ::= term* term ::= "OR" word quoted_phrase qualifier ":" word qualifier ":" quoted_phrase word ::= /[A-Za-z0-9_-]+/ quoted_phrase ::= /"[^"]*"/ qualifier ::= /[a-zA-z][a-zA-Z0-9_-]*/ If more selectivity is needed then query ::= modified_term* modified_term ::= prefix term suffix prefix ::= /[+-]?/ # +gene means "gene" must be found # -gene means "gene" must not be present suffix ::= /[*]?/ # gene* allow "gene" and "genetic" This appears to be supportable under both MySQL and Lucene. Perhaps we can have another parameter to the search to specify the query language ("ql") used. If not specified the default is the DAS/2 query language. This would let server implementers tweak the interface as need be. BTW, in looking over the DAS/1 spec the reason this is an issue now is that we allow searches over arbitrary fields and searches for something other than an identifier. Andrew dalke at dalkescientific.com From allenday at ucla.edu Sun Apr 10 00:27:50 2005 From: allenday at ucla.edu (Allen Day) Date: Sat, 9 Apr 2005 21:27:50 -0700 (PDT) Subject: [DAS2] modification to /type request & response Message-ID: Hi, I'm proposing a change to the way type requests are handled. Background: According to the spec, a genomic feature may be typed by a term from the Sequence Ontology Feature Annotation (sofa), or may be typed by a term from another ontology so long as the term inherits from a sofa term. I have two problems with this. First, there may be other records we want to type -- as a concrete example, let's say we want to type a protein feature as part of protein-DAS, and the type is "alpha helix", which is not in sofa and may be difficult to have inherit from an existing sofa type. I have other real-world examples from the array-DAS document I'm working on, if anyone cares to see them -- related to biological sample annotation. I'd like to remove the constraint that all types inherit from a type. It makes sense to keep a constraint like this in place for genome sequence features, but not any other record type. Second, making a /type request is, according to the spec, supposed to return all types known to the server. If we allow non-sofa-inheriting types into the datasource, the result can be a *very* large document. As a concrete example, serving up just sofa produces a das2xml 64KB document containing 243 types. By linear extrapolation, mixing in the Gene Ontology, which contains 18651 types, will produce a document in excess of 4.8MB. My DAS server actually contains ~15 ontologies that I could conceivably want to annotate with. I'd like to be able to restrict a /type response to a subset of types known by the server, perhaps using the following: * /das/genome/type returns all types known to the server * /das/genome/type?namespace=sofa.ontology returns all types known to the server under the "sofa.ontology" namespace * /das/genome/type?namespaces returns all type namespaces known to the server -Allen From ed_erwin at affymetrix.com Mon Apr 11 13:59:52 2005 From: ed_erwin at affymetrix.com (Ed Erwin) Date: Mon, 11 Apr 2005 10:59:52 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: References: Message-ID: <425ABB18.8060008@affymetrix.com> Adding the "namespaces" query seems reasonable to me. But additionally, how about /das/genome/type returning only those types for which the server actually has some existing annotations, not all the annotations it "knows" about. Allen Day wrote: > > Second, making a /type request is, according to the spec, supposed to > return all types known to the server. If we allow non-sofa-inheriting > types into the datasource, the result can be a *very* large document. > .......... > > I'd like to be able to restrict a /type response to a subset of types > known by the server, perhaps using the following: > > * /das/genome/type > > returns all types known to the server > > * /das/genome/type?namespace=sofa.ontology > > returns all types known to the server under the "sofa.ontology" > namespace > > * /das/genome/type?namespaces > > returns all type namespaces known to the server > From Steve_Chervitz at affymetrix.com Mon Apr 11 15:35:03 2005 From: Steve_Chervitz at affymetrix.com (Chervitz, Steve) Date: Mon, 11 Apr 2005 12:35:03 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: Message-ID: Allen Day writes: > I'd like to remove the constraint that all types inherit from a type. It > makes sense to keep a constraint like this in place for genome sequence > features, but not any other record type. One aim of DAS/2 is to disallow the "anything goes" policy for feature typing, and to constrain them to be part of some ontology or related to a term in an existing ontology so that DAS/2 clients have a chance of making sense of them. The DAS/2 grant contemplates how to extend an existing ontology, but doesn't discuss plans for incorporating types that don't fit into any established ontology. Here's the relevant snippet from the grant: "Using a standard sequence ontology does not preclude extending the ontology. One of the advantages of using RDF and OWL to specify ontologies is that there is a formal mechanism for anyone to extend an existing ontology, which gives rise to distributed ontologies. This ability to extend ontologies is built on the same URI linking concepts previously discussed regarding feature ids. Thus for example someone with detailed knowledge of some the biology of repetitive sequences, for instance, may be annotating repeat data and want a much finer classification of repeat types than is specified in the reference ontology. They can define an extension of the existing ontology via RDF that refers to the existing ontology and specifies the extension (such as LINE-variantX isa type of LINE), then serve up DAS sequence features that refer to this extended ontology rather than the reference ontology. But because the referenced ontology formally extends the standard ontology, a DAS client could still be aware that LINE-variantX is a type of repeat, if for example it wants to display all repeats using the same color. Many software tools are now available to make such ontology extensions easy." Regarding protein feature types, I think the SO team plans to put them in SO/SOFA since they summarize it as, "a set of terms used to describe features on a nucleotide or protein sequence." But there is no support for proteins in SO/SOFA yet. Suzi? It does seem like there should be an avenue for people to say, "Here's a new type that doesn't belong to any established ontology (or if it does, I'm not sure where it goes)." Could such an avenue be opened without having it be abused? Interesting google result: Searching for "protein sequence feature ontology" gives about 35,000 hits, the second of which is the DAS/2 spec introduction! Steve From Steve_Chervitz at affymetrix.com Mon Apr 11 15:38:21 2005 From: Steve_Chervitz at affymetrix.com (Chervitz, Steve) Date: Mon, 11 Apr 2005 12:38:21 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: <425ABB18.8060008@affymetrix.com> Message-ID: "Erwin, Ed" writes: > Adding the "namespaces" query seems reasonable to me. > > But additionally, how about /das/genome/type returning only those types > for which the server actually has some existing annotations, not all the > annotations it "knows" about. I like Ed's restriction that it returns only the types for which it has annotations. But I can see a need for asking a server, "What are all the valid feature types?" This would be useful in the context of a user wanting to add a new annotation. Perhaps it would be reasonable to have the server's default behavior to return only types for which it has annotations. If a client wants all types, it could be specified via a flag. So we would have * /das/genome/type?namespace=sofa.ontology returns only sofa types for which it has annotations * /das/genome/type?namespace=sofa.ontology;all=1 returns all known sofa types Steve > Allen Day wrote: >> >> Second, making a /type request is, according to the spec, supposed to >> return all types known to the server. If we allow non-sofa-inheriting >> types into the datasource, the result can be a *very* large document. >> .......... >> >> I'd like to be able to restrict a /type response to a subset of types >> known by the server, perhaps using the following: >> >> * /das/genome/type >> >> returns all types known to the server >> >> * /das/genome/type?namespace=sofa.ontology >> >> returns all types known to the server under the "sofa.ontology" >> namespace >> >> * /das/genome/type?namespaces >> >> returns all type namespaces known to the server >> From suzi at fruitfly.org Mon Apr 11 15:52:13 2005 From: suzi at fruitfly.org (Suzanna Lewis) Date: Mon, 11 Apr 2005 12:52:13 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: References: Message-ID: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> On Apr 11, 2005, at 12:35 PM, Chervitz, Steve wrote: > > Allen Day writes: >> I'd like to remove the constraint that all types inherit from a type. >> It >> makes sense to keep a constraint like this in place for genome >> sequence >> features, but not any other record type. > > One aim of DAS/2 is to disallow the "anything goes" policy for feature > typing, and to constrain them to be part of some ontology or related > to a > term in an existing ontology so that DAS/2 clients have a chance of > making > sense of them. > > The DAS/2 grant contemplates how to extend an existing ontology, but > doesn't > discuss plans for incorporating types that don't fit into any > established > ontology. Here's the relevant snippet from the grant: > > "Using a standard sequence ontology does not preclude extending the > ontology. One of the advantages of using RDF and OWL to specify > ontologies > is that there is a formal mechanism for anyone to extend an existing > ontology, which gives rise to distributed ontologies. This ability to > extend ontologies is built on the same URI linking concepts previously > discussed regarding feature ids. Thus for example someone with > detailed > knowledge of some the biology of repetitive sequences, for instance, > may be > annotating repeat data and want a much finer classification of repeat > types > than is specified in the reference ontology. They can define an > extension of > the existing ontology via RDF that refers to the existing ontology and > specifies the extension (such as LINE-variantX isa type of LINE), then > serve > up DAS sequence features that refer to this extended ontology rather > than > the reference ontology. But because the referenced ontology formally > extends the standard ontology, a DAS client could still be aware that > LINE-variantX is a type of repeat, if for example it wants to display > all > repeats using the same color. Many software tools are now available > to make > such ontology extensions easy." > > Regarding protein feature types, I think the SO team plans to put them > in > SO/SOFA since they summarize it as, "a set of terms used to describe > features on a nucleotide or protein sequence." But there is no support > for > proteins in SO/SOFA yet. Suzi? Yes, that is pretty much where it stands. There is the intention of doing this, but we have not yet gotten to that point. Including the basic structural descriptions of a protein (alpha helices, beta sheets...) seems an obvious new addition and easy enough to do. I strongly agree with Steve here that we absolutely must stay away from ad hoc types. They are the sirens on the rocks: alluring, but lethal. -S > > It does seem like there should be an avenue for people to say, "Here's > a new > type that doesn't belong to any established ontology (or if it does, > I'm not > sure where it goes)." Could such an avenue be opened without having it > be > abused? > > Interesting google result: Searching for "protein sequence feature > ontology" > gives about 35,000 hits, the second of which is the DAS/2 spec > introduction! > > Steve > > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 From allenday at ucla.edu Mon Apr 11 16:33:40 2005 From: allenday at ucla.edu (Allen Day) Date: Mon, 11 Apr 2005 13:33:40 -0700 (PDT) Subject: [DAS2] modification to /type request & response In-Reply-To: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> References: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> Message-ID: >> I'd like to remove the constraint that all types inherit from a type. >> It >> makes sense to keep a constraint like this in place for genome >> sequence >> features, but not any other record type. [...] > > Regarding protein feature types, I think the SO team plans to put them > > in > > SO/SOFA since they summarize it as, "a set of terms used to describe > > features on a nucleotide or protein sequence." But there is no support > > for > > proteins in SO/SOFA yet. Suzi? > > Yes, that is pretty much where it stands. There is the intention of > doing this, but we have not yet gotten to that point. Including the > basic structural descriptions of a protein (alpha helices, beta > sheets...) > seems an obvious new addition and easy enough to do. > > I strongly agree with Steve here that we absolutely must stay > away from ad hoc types. They are the sirens on the rocks: > alluring, but lethal. Regarding ad hoc types, I agree as well. What I meant to say is that we ought to allow non-sequence feature types be served up by DAS/2, and to allow extensions to those types. Basically I'm proposing to have DAS/2 handle non-sequence feature ontologies in the same way that it handles the sequence feature ontology. -Allen From suzi at fruitfly.org Mon Apr 11 17:27:27 2005 From: suzi at fruitfly.org (Suzanna Lewis) Date: Mon, 11 Apr 2005 14:27:27 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: References: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> Message-ID: <285afb885549cab731fa7a6eb5cd1e03@fruitfly.org> On Apr 11, 2005, at 1:33 PM, Allen Day wrote: >>> I'd like to remove the constraint that all types inherit from a type. >>> It >>> makes sense to keep a constraint like this in place for genome >>> sequence >>> features, but not any other record type. > [...] >>> Regarding protein feature types, I think the SO team plans to put >>> them >>> in >>> SO/SOFA since they summarize it as, "a set of terms used to describe >>> features on a nucleotide or protein sequence." But there is no >>> support >>> for >>> proteins in SO/SOFA yet. Suzi? >> >> Yes, that is pretty much where it stands. There is the intention of >> doing this, but we have not yet gotten to that point. Including the >> basic structural descriptions of a protein (alpha helices, beta >> sheets...) >> seems an obvious new addition and easy enough to do. >> >> I strongly agree with Steve here that we absolutely must stay >> away from ad hoc types. They are the sirens on the rocks: >> alluring, but lethal. > > Regarding ad hoc types, I agree as well. What I meant to say is that > we > ought to allow non-sequence feature types be served up by DAS/2, and to > allow extensions to those types. Basically I'm proposing to have DAS/2 > handle non-sequence feature ontologies in the same way that it handles > the > sequence feature ontology. Hi Allen, I don't understand, but maybe I'm muddling GFF3 with DAS/2. In GFF3 clearly everything is sequence-based in one way or another. Perhaps you could provide some scenarios/examples where this would apply in DAS/2? -S From allenday at ucla.edu Mon Apr 11 17:49:57 2005 From: allenday at ucla.edu (Allen Day) Date: Mon, 11 Apr 2005 14:49:57 -0700 (PDT) Subject: [DAS2] modification to /type request & response In-Reply-To: <285afb885549cab731fa7a6eb5cd1e03@fruitfly.org> References: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> <285afb885549cab731fa7a6eb5cd1e03@fruitfly.org> Message-ID: On Mon, 11 Apr 2005, Suzanna Lewis wrote: > > On Apr 11, 2005, at 1:33 PM, Allen Day wrote: > > >>> I'd like to remove the constraint that all types inherit from a type. > >>> It > >>> makes sense to keep a constraint like this in place for genome > >>> sequence > >>> features, but not any other record type. > > [...] > >>> Regarding protein feature types, I think the SO team plans to put > >>> them > >>> in > >>> SO/SOFA since they summarize it as, "a set of terms used to describe > >>> features on a nucleotide or protein sequence." But there is no > >>> support > >>> for > >>> proteins in SO/SOFA yet. Suzi? > >> > >> Yes, that is pretty much where it stands. There is the intention of > >> doing this, but we have not yet gotten to that point. Including the > >> basic structural descriptions of a protein (alpha helices, beta > >> sheets...) > >> seems an obvious new addition and easy enough to do. > >> > >> I strongly agree with Steve here that we absolutely must stay > >> away from ad hoc types. They are the sirens on the rocks: > >> alluring, but lethal. > > > > Regarding ad hoc types, I agree as well. What I meant to say is that > > we > > ought to allow non-sequence feature types be served up by DAS/2, and to > > allow extensions to those types. Basically I'm proposing to have DAS/2 > > handle non-sequence feature ontologies in the same way that it handles > > the > > sequence feature ontology. > > Hi Allen, > > I don't understand, but maybe I'm muddling GFF3 with DAS/2. > In GFF3 clearly everything is sequence-based in one way or > another. > Perhaps you could provide some scenarios/examples where > this would apply in DAS/2? I can't think of anything in genome-DAS that should not have its primary type be a sequence feature type, or some derivative of an existing sequence feature type. An example of where I need to do this is outside the genome service -- in the array-DAS service. I need to be able to attach types to biological source materials that participate in experiments. For example, I might want to annotate a skeletal muscle sample from a morbidly obese human from the mouse anatomical ontology as "skeletal muscle". I'd also attach the type "morbidly obese", which is an extension to the mammalian phenotype ontology type "obese". -Allen From suzi at fruitfly.org Mon Apr 11 17:57:17 2005 From: suzi at fruitfly.org (Suzanna Lewis) Date: Mon, 11 Apr 2005 14:57:17 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: References: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> <285afb885549cab731fa7a6eb5cd1e03@fruitfly.org> Message-ID: <0cc393bb37f9d7537fdd3169daf32f81@fruitfly.org> You also might want to check the FMA for human anatomical terms: http://sig.biostr.washington.edu/projects/fm/ I think we will be using it for phenotypes one day (some day...) It would be, like SO, structured and something that could be referenced as the ontology. -S On Apr 11, 2005, at 2:49 PM, Allen Day wrote: > On Mon, 11 Apr 2005, Suzanna Lewis wrote: > >> >> On Apr 11, 2005, at 1:33 PM, Allen Day wrote: >> >>>>> I'd like to remove the constraint that all types inherit from a >>>>> type. >>>>> It >>>>> makes sense to keep a constraint like this in place for genome >>>>> sequence >>>>> features, but not any other record type. >>> [...] >>>>> Regarding protein feature types, I think the SO team plans to put >>>>> them >>>>> in >>>>> SO/SOFA since they summarize it as, "a set of terms used to >>>>> describe >>>>> features on a nucleotide or protein sequence." But there is no >>>>> support >>>>> for >>>>> proteins in SO/SOFA yet. Suzi? >>>> >>>> Yes, that is pretty much where it stands. There is the intention of >>>> doing this, but we have not yet gotten to that point. Including the >>>> basic structural descriptions of a protein (alpha helices, beta >>>> sheets...) >>>> seems an obvious new addition and easy enough to do. >>>> >>>> I strongly agree with Steve here that we absolutely must stay >>>> away from ad hoc types. They are the sirens on the rocks: >>>> alluring, but lethal. >>> >>> Regarding ad hoc types, I agree as well. What I meant to say is that >>> we >>> ought to allow non-sequence feature types be served up by DAS/2, and >>> to >>> allow extensions to those types. Basically I'm proposing to have >>> DAS/2 >>> handle non-sequence feature ontologies in the same way that it >>> handles >>> the >>> sequence feature ontology. >> >> Hi Allen, >> >> I don't understand, but maybe I'm muddling GFF3 with DAS/2. >> In GFF3 clearly everything is sequence-based in one way or >> another. >> Perhaps you could provide some scenarios/examples where >> this would apply in DAS/2? > > I can't think of anything in genome-DAS that should not have its > primary > type be a sequence feature type, or some derivative of an existing > sequence feature type. > > An example of where I need to do this is outside the genome service -- > in > the array-DAS service. I need to be able to attach types to biological > source materials that participate in experiments. For example, I might > want to annotate a skeletal muscle sample from a morbidly obese human > from > the mouse anatomical ontology as "skeletal muscle". I'd also attach > the > type "morbidly obese", which is an extension to the mammalian phenotype > ontology type "obese". > > -Allen From lstein at cshl.edu Tue Apr 12 12:36:26 2005 From: lstein at cshl.edu (Lincoln Stein) Date: Tue, 12 Apr 2005 12:36:26 -0400 Subject: [DAS2] modification to /type request & response In-Reply-To: References: Message-ID: <200504121236.34290.lstein@cshl.edu> Hello, The Types request returns the ontology you are using in the "ontology" attribute of the tag. I think it is fair to insist that if you add a new type to the system you must prepare an ontology that describes it if there isn't one ready-made to inherit from. The /type request is only supposed to return those types which are used. I never expected someone to load the entire sofa database into the thing -- just that subportion of sofa which is actively used to annotate the database objects. Lincoln On Sunday 10 April 2005 12:27 am, Allen Day wrote: > Hi, > > I'm proposing a change to the way type requests are handled. > > Background: According to the spec, a genomic feature may be typed by a > term from the Sequence Ontology Feature Annotation (sofa), or may be > typed by a term from another ontology so long as the term inherits from a > sofa term. > > I have two problems with this. > > First, there may be other records we want to type -- as a concrete > example, let's say we want to type a protein feature as part of > protein-DAS, and the type is "alpha helix", which is not in sofa and may > be difficult to have inherit from an existing sofa type. I have other > real-world examples from the array-DAS document I'm working on, if anyone > cares to see them -- related to biological sample annotation. > > I'd like to remove the constraint that all types inherit from a type. It > makes sense to keep a constraint like this in place for genome sequence > features, but not any other record type. > > Second, making a /type request is, according to the spec, supposed to > return all types known to the server. If we allow non-sofa-inheriting > types into the datasource, the result can be a *very* large document. As > a concrete example, serving up just sofa produces a das2xml 64KB document > containing 243 types. By linear extrapolation, mixing in the Gene > Ontology, which contains 18651 types, will produce a document in excess > of 4.8MB. My DAS server actually contains ~15 ontologies that I could > conceivably want to annotate with. > > I'd like to be able to restrict a /type response to a subset of types > known by the server, perhaps using the following: > > * /das/genome/type > > returns all types known to the server > > * /das/genome/type?namespace=sofa.ontology > > returns all types known to the server under the "sofa.ontology" > namespace > > * /das/genome/type?namespaces > > returns all type namespaces known to the server > > -Allen > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 -- Lincoln D. Stein Cold Spring Harbor Laboratory 1 Bungtown Road Cold Spring Harbor, NY 11724 NOTE: Please copy Sandra Michelsen on all emails regarding scheduling and other time-critical topics. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From lstein at cshl.edu Tue Apr 12 12:38:41 2005 From: lstein at cshl.edu (Lincoln Stein) Date: Tue, 12 Apr 2005 12:38:41 -0400 Subject: [DAS2] modification to /type request & response In-Reply-To: References: <285afb885549cab731fa7a6eb5cd1e03@fruitfly.org> Message-ID: <200504121238.48866.lstein@cshl.edu> The /types?ontology=XXXXXX is a good idea. I'll make that change. Maybe there should be a "brief" form of the /types request which returns the ontology names. Lincoln > > >>> features, but not any other record type. > > > > > > [...] > > > > > >>> Regarding protein feature types, I think the SO team plans to put > > >>> them > > >>> in > > >>> SO/SOFA since they summarize it as, "a set of terms used to > > >>> describe features on a nucleotide or protein sequence." But there > > >>> is no support > > >>> for > > >>> proteins in SO/SOFA yet. Suzi? > > >> > > >> Yes, that is pretty much where it stands. There is the intention of > > >> doing this, but we have not yet gotten to that point. Including the > > >> basic structural descriptions of a protein (alpha helices, beta > > >> sheets...) > > >> seems an obvious new addition and easy enough to do. > > >> > > >> I strongly agree with Steve here that we absolutely must stay > > >> away from ad hoc types. They are the sirens on the rocks: > > >> alluring, but lethal. > > > > > > Regarding ad hoc types, I agree as well. What I meant to say is that > > > we > > > ought to allow non-sequence feature types be served up by DAS/2, and > > > to allow extensions to those types. Basically I'm proposing to have > > > DAS/2 handle non-sequence feature ontologies in the same way that it > > > handles the > > > sequence feature ontology. > > > > Hi Allen, > > > > I don't understand, but maybe I'm muddling GFF3 with DAS/2. > > In GFF3 clearly everything is sequence-based in one way or > > another. > > Perhaps you could provide some scenarios/examples where > > this would apply in DAS/2? > > I can't think of anything in genome-DAS that should not have its primary > type be a sequence feature type, or some derivative of an existing > sequence feature type. > > An example of where I need to do this is outside the genome service -- in > the array-DAS service. I need to be able to attach types to biological > source materials that participate in experiments. For example, I might > want to annotate a skeletal muscle sample from a morbidly obese human > from the mouse anatomical ontology as "skeletal muscle". I'd also attach > the type "morbidly obese", which is an extension to the mammalian > phenotype ontology type "obese". > > -Allen > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 -- Lincoln D. Stein Cold Spring Harbor Laboratory 1 Bungtown Road Cold Spring Harbor, NY 11724 NOTE: Please copy Sandra Michelsen on all emails regarding scheduling and other time-critical topics. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From allenday at ucla.edu Tue Apr 12 13:47:03 2005 From: allenday at ucla.edu (Allen Day) Date: Tue, 12 Apr 2005 10:47:03 -0700 (PDT) Subject: [DAS2] modification to /type request & response In-Reply-To: <200504121236.34290.lstein@cshl.edu> References: <200504121236.34290.lstein@cshl.edu> Message-ID: On Tue, 12 Apr 2005, Lincoln Stein wrote: > Hello, > > The Types request returns the ontology you are using in the "ontology" > attribute of the tag. I think it is fair to insist that if you add > a new type to the system you must prepare an ontology that describes it if > there isn't one ready-made to inherit from. > > The /type request is only supposed to return those types which are used. I > never expected someone to load the entire sofa database into the thing -- > just that subportion of sofa which is actively used to annotate the > database objects. Okay, that makes things easier for the client. Just a small qualifier though -- we should return the types used as well as the union of type graphs *above* those terms. That way the client is aware it can query with a generic type (for instance "region") and get back everything. -Allen > > Lincoln > > On Sunday 10 April 2005 12:27 am, Allen Day wrote: > > Hi, > > > > I'm proposing a change to the way type requests are handled. > > > > Background: According to the spec, a genomic feature may be typed by a > > term from the Sequence Ontology Feature Annotation (sofa), or may be > > typed by a term from another ontology so long as the term inherits from a > > sofa term. > > > > I have two problems with this. > > > > First, there may be other records we want to type -- as a concrete > > example, let's say we want to type a protein feature as part of > > protein-DAS, and the type is "alpha helix", which is not in sofa and may > > be difficult to have inherit from an existing sofa type. I have other > > real-world examples from the array-DAS document I'm working on, if anyone > > cares to see them -- related to biological sample annotation. > > > > I'd like to remove the constraint that all types inherit from a type. It > > makes sense to keep a constraint like this in place for genome sequence > > features, but not any other record type. > > > > Second, making a /type request is, according to the spec, supposed to > > return all types known to the server. If we allow non-sofa-inheriting > > types into the datasource, the result can be a *very* large document. As > > a concrete example, serving up just sofa produces a das2xml 64KB document > > containing 243 types. By linear extrapolation, mixing in the Gene > > Ontology, which contains 18651 types, will produce a document in excess > > of 4.8MB. My DAS server actually contains ~15 ontologies that I could > > conceivably want to annotate with. > > > > I'd like to be able to restrict a /type response to a subset of types > > known by the server, perhaps using the following: > > > > * /das/genome/type > > > > returns all types known to the server > > > > * /das/genome/type?namespace=sofa.ontology > > > > returns all types known to the server under the "sofa.ontology" > > namespace > > > > * /das/genome/type?namespaces > > > > returns all type namespaces known to the server > > > > -Allen > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > > From allenday at ucla.edu Thu Apr 21 01:42:21 2005 From: allenday at ucla.edu (Allen Day) Date: Wed, 20 Apr 2005 22:42:21 -0700 (PDT) Subject: [DAS2] Human prototype server up Message-ID: Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" sequences aligned to the genome w/ > 90% identity. http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region Here's a URL to get you going. This is running on 2 hosts: a dual opteron/8GB webserver frontend and a dual opteron/16GB database backend. There are no other applications on these hosts right now, so hammer away. You might want to use HTTP 1.1 keepalive though, the responses can be *really* slow for /feature requests on large regions. The reason for this is the feature/part subpart requests are done feature by feature due to the 1-many relationship... I'm still trying to figure out how to improve this. Components that aren't working in even a basic way: * alignments * assemblies * individual /feature/ requests * versioned source response w/ supported formats * most feature filters (overlaps, type, and exacttype work) -Allen From Gregg_Helt at affymetrix.com Thu Apr 21 06:28:07 2005 From: Gregg_Helt at affymetrix.com (Helt,Gregg) Date: Thu, 21 Apr 2005 03:28:07 -0700 Subject: [DAS2] Human prototype server up Message-ID: Cool, great to see a public server up and running! However, I'm having trouble with feature queries. For example: http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap s=chr21/21500000:22500000;type=0000110 Which is an example of the most typical query IGB will be making -- one overlap filter combined with one type filter. But this query returns with a single line, no . I've tried it for every type returned by the type query and get the same results. Is there something wrong with my query syntax? If this is not a good region to test, can you send me an example feature query URL where you know what the response should be? thanks! gregg > -----Original Message----- > From: das2-bounces at portal.open-bio.org [mailto:das2-bounces at portal.open- > bio.org] On Behalf Of Allen Day > Sent: Wednesday, April 20, 2005 10:42 PM > To: das2 at portal.open-bio.org > Subject: [DAS2] Human prototype server up > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" sequences > aligned to the genome w/ > 90% identity. > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > Here's a URL to get you going. This is running on 2 hosts: a dual > opteron/8GB webserver frontend and a dual opteron/16GB database backend. > There are no other applications on these hosts right now, so hammer away. > > You might want to use HTTP 1.1 keepalive though, the responses can be > *really* slow for /feature requests on large regions. The reason for this > is the feature/part subpart requests are done feature by feature due to > the 1-many relationship... I'm still trying to figure out how to improve > this. > > Components that aren't working in even a basic way: > > * alignments > * assemblies > * individual /feature/ requests > * versioned source response w/ supported formats > * most feature filters (overlaps, type, and exacttype work) > > -Allen > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 From allenday at ucla.edu Thu Apr 21 13:17:25 2005 From: allenday at ucla.edu (Allen Day) Date: Thu, 21 Apr 2005 10:17:25 -0700 (PDT) Subject: [DAS2] Human prototype server up In-Reply-To: References: Message-ID: This is a good region. I just realized the type ids coming back in the /type response don't match the type ids given in the /feature response. I'm in the process of switching over from using table PKs for types to using type names, so rather than using a filter "type=9770" you can instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string is the term's namespace. SO has already changed it to "SOFA" in their most recent SOFA file in CVS, but I haven't updated yet). So, sorry about that. I just change thed /type response back to give primary keys until I get names working. All the types should now work with the type= filter. Here's a quick document you can grab, a subregion of what you asked for in your request that has something interesting inside: http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlaps=chr21/21292691:21324284;type=9688 -Allen On Thu, 21 Apr 2005, Helt,Gregg wrote: > Cool, great to see a public server up and running! > > However, I'm having trouble with feature queries. For example: > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > s=chr21/21500000:22500000;type=0000110 > > Which is an example of the most typical query IGB will be making -- one > overlap filter combined with one type filter. But this query returns > with a single line, no . I've tried it for > every type returned by the type query and get the same results. > > Is there something wrong with my query syntax? > > If this is not a good region to test, can you send me an example feature > query URL where you know what the response should be? > > thanks! > gregg > > > -----Original Message----- > > From: das2-bounces at portal.open-bio.org > [mailto:das2-bounces at portal.open- > > bio.org] On Behalf Of Allen Day > > Sent: Wednesday, April 20, 2005 10:42 PM > > To: das2 at portal.open-bio.org > > Subject: [DAS2] Human prototype server up > > > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > sequences > > aligned to the genome w/ > 90% identity. > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > > > Here's a URL to get you going. This is running on 2 hosts: a dual > > opteron/8GB webserver frontend and a dual opteron/16GB database > backend. > > There are no other applications on these hosts right now, so hammer > away. > > > > You might want to use HTTP 1.1 keepalive though, the responses can be > > *really* slow for /feature requests on large regions. The reason for > this > > is the feature/part subpart requests are done feature by feature due > to > > the 1-many relationship... I'm still trying to figure out how to > improve > > this. > > > > Components that aren't working in even a basic way: > > > > * alignments > > * assemblies > > * individual /feature/ requests > > * versioned source response w/ supported formats > > * most feature filters (overlaps, type, and exacttype work) > > > > -Allen > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > From allenday at ucla.edu Thu Apr 21 17:30:33 2005 From: allenday at ucla.edu (Allen Day) Date: Thu, 21 Apr 2005 14:30:33 -0700 (PDT) Subject: [DAS2] Human prototype server up In-Reply-To: <4267EBE7.6030603@affymetrix.com> References: <4267E339.6060301@affymetrix.com> <4267E958.30600@affymetrix.com> <4267EBE7.6030603@affymetrix.com> Message-ID: Gregg, Please post your instructions for how to use your server test application. Thanks. -Allen On Thu, 21 Apr 2005, Helt,Gregg wrote: > Cool, great to see a public server up and running! > > However, I'm having trouble with feature queries. For example: > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > s=chr21/21500000:22500000;type=0000110 > > Which is an example of the most typical query IGB will be making -- one > overlap filter combined with one type filter. But this query returns > with a single line, no . I've tried it for > every type returned by the type query and get the same results. > > Is there something wrong with my query syntax? > > If this is not a good region to test, can you send me an example feature > query URL where you know what the response should be? > > thanks! > gregg > > > -----Original Message----- > > From: das2-bounces at portal.open-bio.org > [mailto:das2-bounces at portal.open- > > bio.org] On Behalf Of Allen Day > > Sent: Wednesday, April 20, 2005 10:42 PM > > To: das2 at portal.open-bio.org > > Subject: [DAS2] Human prototype server up > > > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > sequences > > aligned to the genome w/ > 90% identity. > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > > > Here's a URL to get you going. This is running on 2 hosts: a dual > > opteron/8GB webserver frontend and a dual opteron/16GB database > backend. > > There are no other applications on these hosts right now, so hammer > away. > > > > You might want to use HTTP 1.1 keepalive though, the responses can be > > *really* slow for /feature requests on large regions. The reason for > this > > is the feature/part subpart requests are done feature by feature due > to > > the 1-many relationship... I'm still trying to figure out how to > improve > > this. > > > > Components that aren't working in even a basic way: > > > > * alignments > > * assemblies > > * individual /feature/ requests > > * versioned source response w/ supported formats > > * most feature filters (overlaps, type, and exacttype work) > > > > -Allen > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > On Thu, 21 Apr 2005, Allen Day wrote: > This is a good region. I just realized the type ids coming back in the > /type response don't match the type ids given in the /feature response. > I'm in the process of switching over from using table PKs for types to > using type names, so rather than using a filter "type=9770" you can > instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string > is the term's namespace. SO has already changed it to "SOFA" in their > most recent SOFA file in CVS, but I haven't updated yet). > > So, sorry about that. I just change thed /type response back to give > primary keys until I get names working. All the types should now work > with the type= filter. Here's a quick document you can grab, a subregion > of what you asked for in your request that has something interesting > inside: > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlaps=chr21/21292691:21324284;type=9688 > > -Allen > > > > On Thu, 21 Apr 2005, Helt,Gregg wrote: > > > Cool, great to see a public server up and running! > > > > However, I'm having trouble with feature queries. For example: > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > > s=chr21/21500000:22500000;type=0000110 > > > > Which is an example of the most typical query IGB will be making -- one > > overlap filter combined with one type filter. But this query returns > > with a single line, no . I've tried it for > > every type returned by the type query and get the same results. > > > > Is there something wrong with my query syntax? > > > > If this is not a good region to test, can you send me an example feature > > query URL where you know what the response should be? > > > > thanks! > > gregg > > > > > -----Original Message----- > > > From: das2-bounces at portal.open-bio.org > > [mailto:das2-bounces at portal.open- > > > bio.org] On Behalf Of Allen Day > > > Sent: Wednesday, April 20, 2005 10:42 PM > > > To: das2 at portal.open-bio.org > > > Subject: [DAS2] Human prototype server up > > > > > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > sequences > > > aligned to the genome w/ > 90% identity. > > > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > > > > > Here's a URL to get you going. This is running on 2 hosts: a dual > > > opteron/8GB webserver frontend and a dual opteron/16GB database > > backend. > > > There are no other applications on these hosts right now, so hammer > > away. > > > > > > You might want to use HTTP 1.1 keepalive though, the responses can be > > > *really* slow for /feature requests on large regions. The reason for > > this > > > is the feature/part subpart requests are done feature by feature due > > to > > > the 1-many relationship... I'm still trying to figure out how to > > improve > > > this. > > > > > > Components that aren't working in even a basic way: > > > > > > * alignments > > > * assemblies > > > * individual /feature/ requests > > > * versioned source response w/ supported formats > > > * most feature filters (overlaps, type, and exacttype work) > > > > > > -Allen > > > _______________________________________________ > > > DAS2 mailing list > > > DAS2 at portal.open-bio.org > > > http://portal.open-bio.org/mailman/listinfo/das2 > > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 > On Thu, 21 Apr 2005, Ed Erwin wrote: > That response comes back with some "%20" things in it. That seems like > a bug to me, but is it? > > > > > Allen Day wrote: > > This is a good region. I just realized the type ids coming back in the > > /type response don't match the type ids given in the /feature response. > > I'm in the process of switching over from using table PKs for types to > > using type names, so rather than using a filter "type=9770" you can > > instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string > > is the term's namespace. SO has already changed it to "SOFA" in their > > most recent SOFA file in CVS, but I haven't updated yet). > > > > So, sorry about that. I just change thed /type response back to give > > primary keys until I get names working. All the types should now work > > with the type= filter. Here's a quick document you can grab, a subregion > > of what you asked for in your request that has something interesting > > inside: > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlaps=chr21/21292691:21324284;type=9688 > > > > -Allen > > > > > > > > On Thu, 21 Apr 2005, Helt,Gregg wrote: > > > > > >>Cool, great to see a public server up and running! > >> > >>However, I'm having trouble with feature queries. For example: > >>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > >>s=chr21/21500000:22500000;type=0000110 > >> > >>Which is an example of the most typical query IGB will be making -- one > >>overlap filter combined with one type filter. But this query returns > >>with a single line, no . I've tried it for > >>every type returned by the type query and get the same results. > >> > >>Is there something wrong with my query syntax? > >> > >>If this is not a good region to test, can you send me an example feature > >>query URL where you know what the response should be? > >> > >> thanks! > >> gregg > >> > >> > >>>-----Original Message----- > >>>From: das2-bounces at portal.open-bio.org > >> > >>[mailto:das2-bounces at portal.open- > >> > >>>bio.org] On Behalf Of Allen Day > >>>Sent: Wednesday, April 20, 2005 10:42 PM > >>>To: das2 at portal.open-bio.org > >>>Subject: [DAS2] Human prototype server up > >>> > >>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > >> > >>sequences > >> > >>>aligned to the genome w/ > 90% identity. > >>> > >>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > >>> > >>>Here's a URL to get you going. This is running on 2 hosts: a dual > >>>opteron/8GB webserver frontend and a dual opteron/16GB database > >> > >>backend. > >> > >>>There are no other applications on these hosts right now, so hammer > >> > >>away. > >> > >>>You might want to use HTTP 1.1 keepalive though, the responses can be > >>>*really* slow for /feature requests on large regions. The reason for > >> > >>this > >> > >>>is the feature/part subpart requests are done feature by feature due > >> > >>to > >> > >>>the 1-many relationship... I'm still trying to figure out how to > >> > >>improve > >> > >>>this. > >>> > >>>Components that aren't working in even a basic way: > >>> > >>> * alignments > >>> * assemblies > >>> * individual /feature/ requests > >>> * versioned source response w/ supported formats > >>> * most feature filters (overlaps, type, and exacttype work) > >>> > >>>-Allen > >>>_______________________________________________ > >>>DAS2 mailing list > >>>DAS2 at portal.open-bio.org > >>>http://portal.open-bio.org/mailman/listinfo/das2 > >> > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > On Thu, 21 Apr 2005, Ed Erwin wrote: > > This is a question for Andrew Dalke. > > > Allen Day wrote: > > I don't know, is it? I have found the description of properties and the > > /property response document examples to be extremely unclear. I opted to > > URI escape here, but maybe it's not the right thing to do. > > > > -Allen > > > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > > > >>That response comes back with some "%20" things in it. That seems like > >>a bug to me, but is it? > >> > >> > >> > >> > >>Allen Day wrote: > >> > >>>This is a good region. I just realized the type ids coming back in the > >>>/type response don't match the type ids given in the /feature response. > >>>I'm in the process of switching over from using table PKs for types to > >>>using type names, so rather than using a filter "type=9770" you can > >>>instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string > >>>is the term's namespace. SO has already changed it to "SOFA" in their > >>>most recent SOFA file in CVS, but I haven't updated yet). > >>> > >>>So, sorry about that. I just change thed /type response back to give > >>>primary keys until I get names working. All the types should now work > >>>with the type= filter. Here's a quick document you can grab, a subregion > >>>of what you asked for in your request that has something interesting > >>>inside: > >>> > >>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlaps=chr21/21292691:21324284;type=9688 > >>> > >>>-Allen > >>> > >>> > >>> > >>>On Thu, 21 Apr 2005, Helt,Gregg wrote: > >>> > >>> > >>> > >>>>Cool, great to see a public server up and running! > >>>> > >>>>However, I'm having trouble with feature queries. For example: > >>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > >>>>s=chr21/21500000:22500000;type=0000110 > >>>> > >>>>Which is an example of the most typical query IGB will be making -- one > >>>>overlap filter combined with one type filter. But this query returns > >>>>with a single line, no . I've tried it for > >>>>every type returned by the type query and get the same results. > >>>> > >>>>Is there something wrong with my query syntax? > >>>> > >>>>If this is not a good region to test, can you send me an example feature > >>>>query URL where you know what the response should be? > >>>> > >>>> thanks! > >>>> gregg > >>>> > >>>> > >>>> > >>>>>-----Original Message----- > >>>>>From: das2-bounces at portal.open-bio.org > >>>> > >>>>[mailto:das2-bounces at portal.open- > >>>> > >>>> > >>>>>bio.org] On Behalf Of Allen Day > >>>>>Sent: Wednesday, April 20, 2005 10:42 PM > >>>>>To: das2 at portal.open-bio.org > >>>>>Subject: [DAS2] Human prototype server up > >>>>> > >>>>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > >>>> > >>>>sequences > >>>> > >>>> > >>>>>aligned to the genome w/ > 90% identity. > >>>>> > >>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > >>>>> > >>>>>Here's a URL to get you going. This is running on 2 hosts: a dual > >>>>>opteron/8GB webserver frontend and a dual opteron/16GB database > >>>> > >>>>backend. > >>>> > >>>> > >>>>>There are no other applications on these hosts right now, so hammer > >>>> > >>>>away. > >>>> > >>>> > >>>>>You might want to use HTTP 1.1 keepalive though, the responses can be > >>>>>*really* slow for /feature requests on large regions. The reason for > >>>> > >>>>this > >>>> > >>>> > >>>>>is the feature/part subpart requests are done feature by feature due > >>>> > >>>>to > >>>> > >>>> > >>>>>the 1-many relationship... I'm still trying to figure out how to > >>>> > >>>>improve > >>>> > >>>> > >>>>>this. > >>>>> > >>>>>Components that aren't working in even a basic way: > >>>>> > >>>>> * alignments > >>>>> * assemblies > >>>>> * individual /feature/ requests > >>>>> * versioned source response w/ supported formats > >>>>> * most feature filters (overlaps, type, and exacttype work) > >>>>> > >>>>>-Allen > >>>>>_______________________________________________ > >>>>>DAS2 mailing list > >>>>>DAS2 at portal.open-bio.org > >>>>>http://portal.open-bio.org/mailman/listinfo/das2 > >>>> > >>>_______________________________________________ > >>>DAS2 mailing list > >>>DAS2 at portal.open-bio.org > >>>http://portal.open-bio.org/mailman/listinfo/das2 > >> > On Thu, 21 Apr 2005, Ed Erwin wrote: > My reasoning is that it ISN'T a URL yet, so you don't escape it until > you turn it into a URL. But Andrew can figure out which is right and > document it. > > Allen Day wrote: > > exactly. my reasoning was that it was a relative url and needed to be > > escaped. > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > > > >>This is a question for Andrew Dalke. > >> > >> > >>Allen Day wrote: > >> > >>>I don't know, is it? I have found the description of properties and the > >>>/property response document examples to be extremely unclear. I opted to > >>>URI escape here, but maybe it's not the right thing to do. > >>> > >>>-Allen > >>> > >>> > >>>On Thu, 21 Apr 2005, Ed Erwin wrote: > >>> > >>> > >>> > >>>>That response comes back with some "%20" things in it. That seems like > >>>>a bug to me, but is it? > >>>> > >>>> > >>>> > >>>> > >>>>Allen Day wrote: > >>>> > >>>> > >>>>>This is a good region. I just realized the type ids coming back in the > >>>>>/type response don't match the type ids given in the /feature response. > >>>>>I'm in the process of switching over from using table PKs for types to > >>>>>using type names, so rather than using a filter "type=9770" you can > >>>>>instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string > >>>>>is the term's namespace. SO has already changed it to "SOFA" in their > >>>>>most recent SOFA file in CVS, but I haven't updated yet). > >>>>> > >>>>>So, sorry about that. I just change thed /type response back to give > >>>>>primary keys until I get names working. All the types should now work > >>>>>with the type= filter. Here's a quick document you can grab, a subregion > >>>>>of what you asked for in your request that has something interesting > >>>>>inside: > >>>>> > >>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlaps=chr21/21292691:21324284;type=9688 > >>>>> > >>>>>-Allen > >>>>> > >>>>> > >>>>> > >>>>>On Thu, 21 Apr 2005, Helt,Gregg wrote: > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>>Cool, great to see a public server up and running! > >>>>>> > >>>>>>However, I'm having trouble with feature queries. For example: > >>>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > >>>>>>s=chr21/21500000:22500000;type=0000110 > >>>>>> > >>>>>>Which is an example of the most typical query IGB will be making -- one > >>>>>>overlap filter combined with one type filter. But this query returns > >>>>>>with a single line, no . I've tried it for > >>>>>>every type returned by the type query and get the same results. > >>>>>> > >>>>>>Is there something wrong with my query syntax? > >>>>>> > >>>>>>If this is not a good region to test, can you send me an example feature > >>>>>>query URL where you know what the response should be? > >>>>>> > >>>>>> thanks! > >>>>>> gregg > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>>>-----Original Message----- > >>>>>>>From: das2-bounces at portal.open-bio.org > >>>>>> > >>>>>>[mailto:das2-bounces at portal.open- > >>>>>> > >>>>>> > >>>>>> > >>>>>>>bio.org] On Behalf Of Allen Day > >>>>>>>Sent: Wednesday, April 20, 2005 10:42 PM > >>>>>>>To: das2 at portal.open-bio.org > >>>>>>>Subject: [DAS2] Human prototype server up > >>>>>>> > >>>>>>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > >>>>>> > >>>>>>sequences > >>>>>> > >>>>>> > >>>>>> > >>>>>>>aligned to the genome w/ > 90% identity. > >>>>>>> > >>>>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > >>>>>>> > >>>>>>>Here's a URL to get you going. This is running on 2 hosts: a dual > >>>>>>>opteron/8GB webserver frontend and a dual opteron/16GB database > >>>>>> > >>>>>>backend. > >>>>>> > >>>>>> > >>>>>> > >>>>>>>There are no other applications on these hosts right now, so hammer > >>>>>> > >>>>>>away. > >>>>>> > >>>>>> > >>>>>> > >>>>>>>You might want to use HTTP 1.1 keepalive though, the responses can be > >>>>>>>*really* slow for /feature requests on large regions. The reason for > >>>>>> > >>>>>>this > >>>>>> > >>>>>> > >>>>>> > >>>>>>>is the feature/part subpart requests are done feature by feature due > >>>>>> > >>>>>>to > >>>>>> > >>>>>> > >>>>>> > >>>>>>>the 1-many relationship... I'm still trying to figure out how to > >>>>>> > >>>>>>improve > >>>>>> > >>>>>> > >>>>>> > >>>>>>>this. > >>>>>>> > >>>>>>>Components that aren't working in even a basic way: > >>>>>>> > >>>>>>>* alignments > >>>>>>>* assemblies > >>>>>>>* individual /feature/ requests > >>>>>>>* versioned source response w/ supported formats > >>>>>>>* most feature filters (overlaps, type, and exacttype work) > >>>>>>> > >>>>>>>-Allen > >>>>>>>_______________________________________________ > >>>>>>>DAS2 mailing list > >>>>>>>DAS2 at portal.open-bio.org > >>>>>>>http://portal.open-bio.org/mailman/listinfo/das2 > >>>>>> > >>>>>_______________________________________________ > >>>>>DAS2 mailing list > >>>>>DAS2 at portal.open-bio.org > >>>>>http://portal.open-bio.org/mailman/listinfo/das2 > >>>> > From Gregg_Helt at affymetrix.com Thu Apr 21 19:04:10 2005 From: Gregg_Helt at affymetrix.com (Helt,Gregg) Date: Thu, 21 Apr 2005 16:04:10 -0700 Subject: [DAS2] Human prototype server up Message-ID: You need to do an anonymous check-out the genoviz CVS module at http://sourceforge.net/projects/genoviz/ and build the class files. This assumes you have a Java SDK installed, version 1.4 or later. You'll also need the xerces XML parser jar. The test UI I'm working on for the DAS2 client is in the class com.affymetrix.igb.view.Das2LoadView. Once you've compiled the source, you should be able to run it from the command line like this: [java executable] -classpath "[path for genoviz];[path for xerces]" com.affymetrix.igb.view.Das2LoadView On my machine this translates to: java -classpath "c:/projects/genoviz/classes;c:/projects/genoviz/lib/xerces.jar" com.affymetrix.igb.view.Das2LoadView You're presented with a cascading series of choices: DAS2 servers --> DAS2 sources --> DAS2 versions --> DAS2 regions, and your choice triggers the appropriate DAS2 query to populate the next choice in the cascade. Once a version is selected, a DAS2 types query populates a checkbox list of DAS2 types. You can also test feature queries with overlap and type filters by entering the min and max for the overlap filter, selecting which types to filter by, and clicking on "Load Features". This will bring up a rudimentary visualization of the sequence and the feature hierarchies returned by the query. By default it shows the whole length of the sequence, so you'll probably want to zoom in using the slider on the top and the scrollbar on the bottom. This viewer shows feature hierarchies as nested outlined boxes. For client-side data model, there are two additional levels of hierarchy added above the root features -- the parent of the root features is a request feature encompassing all the features returned by a single feature request, and the grandparent is a container for all feature requests of a given type on a single sequence (the model underneath this breaks requests for multiple types into separate request for each type). I just checked interaction of Das2LoadView with DAS2 test server at das.biopackages.net, and it looks like there are still some issues with correct handling of feature child/parent relationships. I'll try and figure out if it's a problem on the client end. thanks, gregg > -----Original Message----- > From: Allen Day [mailto:allenday at ucla.edu] > Sent: Thursday, April 21, 2005 2:31 PM > To: Helt,Gregg; Erwin, Ed > Cc: das2 at portal.open-bio.org; Andrew Dalke > Subject: RE: [DAS2] Human prototype server up > > Gregg, > > Please post your instructions for how to use your server test application. > Thanks. > > -Allen > > On Thu, 21 Apr 2005, Helt,Gregg wrote: > > > Cool, great to see a public server up and running! > > > > However, I'm having trouble with feature queries. For example: > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > > s=chr21/21500000:22500000;type=0000110 > > > > Which is an example of the most typical query IGB will be making -- one > > overlap filter combined with one type filter. But this query returns > > with a single line, no . I've tried it for > > every type returned by the type query and get the same results. > > > > Is there something wrong with my query syntax? > > > > If this is not a good region to test, can you send me an example feature > > query URL where you know what the response should be? > > > > thanks! > > gregg > > > > > -----Original Message----- > > > From: das2-bounces at portal.open-bio.org > > [mailto:das2-bounces at portal.open- > > > bio.org] On Behalf Of Allen Day > > > Sent: Wednesday, April 20, 2005 10:42 PM > > > To: das2 at portal.open-bio.org > > > Subject: [DAS2] Human prototype server up > > > > > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > sequences > > > aligned to the genome w/ > 90% identity. > > > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > > > > > Here's a URL to get you going. This is running on 2 hosts: a dual > > > opteron/8GB webserver frontend and a dual opteron/16GB database > > backend. > > > There are no other applications on these hosts right now, so hammer > > away. > > > > > > You might want to use HTTP 1.1 keepalive though, the responses can be > > > *really* slow for /feature requests on large regions. The reason for > > this > > > is the feature/part subpart requests are done feature by feature due > > to > > > the 1-many relationship... I'm still trying to figure out how to > > improve > > > this. > > > > > > Components that aren't working in even a basic way: > > > > > > * alignments > > > * assemblies > > > * individual /feature/ requests > > > * versioned source response w/ supported formats > > > * most feature filters (overlaps, type, and exacttype work) > > > > > > -Allen > > > _______________________________________________ > > > DAS2 mailing list > > > DAS2 at portal.open-bio.org > > > http://portal.open-bio.org/mailman/listinfo/das2 > > > > On Thu, 21 Apr 2005, Allen Day wrote: > > > This is a good region. I just realized the type ids coming back in the > > /type response don't match the type ids given in the /feature response. > > I'm in the process of switching over from using table PKs for types to > > using type names, so rather than using a filter "type=9770" you can > > instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string > > is the term's namespace. SO has already changed it to "SOFA" in their > > most recent SOFA file in CVS, but I haven't updated yet). > > > > So, sorry about that. I just change thed /type response back to give > > primary keys until I get names working. All the types should now work > > with the type= filter. Here's a quick document you can grab, a > subregion > > of what you asked for in your request that has something interesting > > inside: > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlaps=chr21/21292691:21324284;type=9688 > > > > -Allen > > > > > > > > On Thu, 21 Apr 2005, Helt,Gregg wrote: > > > > > Cool, great to see a public server up and running! > > > > > > However, I'm having trouble with feature queries. For example: > > > http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlap > > > s=chr21/21500000:22500000;type=0000110 > > > > > > Which is an example of the most typical query IGB will be making -- > one > > > overlap filter combined with one type filter. But this query returns > > > with a single line, no . I've tried it for > > > every type returned by the type query and get the same results. > > > > > > Is there something wrong with my query syntax? > > > > > > If this is not a good region to test, can you send me an example > feature > > > query URL where you know what the response should be? > > > > > > thanks! > > > gregg > > > > > > > -----Original Message----- > > > > From: das2-bounces at portal.open-bio.org > > > [mailto:das2-bounces at portal.open- > > > > bio.org] On Behalf Of Allen Day > > > > Sent: Wednesday, April 20, 2005 10:42 PM > > > > To: das2 at portal.open-bio.org > > > > Subject: [DAS2] Human prototype server up > > > > > > > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > > sequences > > > > aligned to the genome w/ > 90% identity. > > > > > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > > > > > > > Here's a URL to get you going. This is running on 2 hosts: a dual > > > > opteron/8GB webserver frontend and a dual opteron/16GB database > > > backend. > > > > There are no other applications on these hosts right now, so hammer > > > away. > > > > > > > > You might want to use HTTP 1.1 keepalive though, the responses can > be > > > > *really* slow for /feature requests on large regions. The reason > for > > > this > > > > is the feature/part subpart requests are done feature by feature due > > > to > > > > the 1-many relationship... I'm still trying to figure out how to > > > improve > > > > this. > > > > > > > > Components that aren't working in even a basic way: > > > > > > > > * alignments > > > > * assemblies > > > > * individual /feature/ requests > > > > * versioned source response w/ supported formats > > > > * most feature filters (overlaps, type, and exacttype work) > > > > > > > > -Allen > > > > _______________________________________________ > > > > DAS2 mailing list > > > > DAS2 at portal.open-bio.org > > > > http://portal.open-bio.org/mailman/listinfo/das2 > > > > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > That response comes back with some "%20" things in it. That seems like > > a bug to me, but is it? > > > > > > > > > > Allen Day wrote: > > > This is a good region. I just realized the type ids coming back in > the > > > /type response don't match the type ids given in the /feature response. > > > I'm in the process of switching over from using table PKs for types to > > > using type names, so rather than using a filter "type=9770" you can > > > instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" > string > > > is the term's namespace. SO has already changed it to "SOFA" in their > > > most recent SOFA file in CVS, but I haven't updated yet). > > > > > > So, sorry about that. I just change thed /type response back to give > > > primary keys until I get names working. All the types should now work > > > with the type= filter. Here's a quick document you can grab, a > subregion > > > of what you asked for in your request that has something interesting > > > inside: > > > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlaps=chr21/21292691:21324284;type=9688 > > > > > > -Allen > > > > > > > > > > > > On Thu, 21 Apr 2005, Helt,Gregg wrote: > > > > > > > > >>Cool, great to see a public server up and running! > > >> > > >>However, I'm having trouble with feature queries. For example: > > >>http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlap > > >>s=chr21/21500000:22500000;type=0000110 > > >> > > >>Which is an example of the most typical query IGB will be making -- > one > > >>overlap filter combined with one type filter. But this query returns > > >>with a single line, no . I've tried it for > > >>every type returned by the type query and get the same results. > > >> > > >>Is there something wrong with my query syntax? > > >> > > >>If this is not a good region to test, can you send me an example > feature > > >>query URL where you know what the response should be? > > >> > > >> thanks! > > >> gregg > > >> > > >> > > >>>-----Original Message----- > > >>>From: das2-bounces at portal.open-bio.org > > >> > > >>[mailto:das2-bounces at portal.open- > > >> > > >>>bio.org] On Behalf Of Allen Day > > >>>Sent: Wednesday, April 20, 2005 10:42 PM > > >>>To: das2 at portal.open-bio.org > > >>>Subject: [DAS2] Human prototype server up > > >>> > > >>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > >> > > >>sequences > > >> > > >>>aligned to the genome w/ > 90% identity. > > >>> > > >>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > >>> > > >>>Here's a URL to get you going. This is running on 2 hosts: a dual > > >>>opteron/8GB webserver frontend and a dual opteron/16GB database > > >> > > >>backend. > > >> > > >>>There are no other applications on these hosts right now, so hammer > > >> > > >>away. > > >> > > >>>You might want to use HTTP 1.1 keepalive though, the responses can be > > >>>*really* slow for /feature requests on large regions. The reason for > > >> > > >>this > > >> > > >>>is the feature/part subpart requests are done feature by feature due > > >> > > >>to > > >> > > >>>the 1-many relationship... I'm still trying to figure out how to > > >> > > >>improve > > >> > > >>>this. > > >>> > > >>>Components that aren't working in even a basic way: > > >>> > > >>> * alignments > > >>> * assemblies > > >>> * individual /feature/ requests > > >>> * versioned source response w/ supported formats > > >>> * most feature filters (overlaps, type, and exacttype work) > > >>> > > >>>-Allen > > >>>_______________________________________________ > > >>>DAS2 mailing list > > >>>DAS2 at portal.open-bio.org > > >>>http://portal.open-bio.org/mailman/listinfo/das2 > > >> > > > _______________________________________________ > > > DAS2 mailing list > > > DAS2 at portal.open-bio.org > > > http://portal.open-bio.org/mailman/listinfo/das2 > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > > > This is a question for Andrew Dalke. > > > > > > Allen Day wrote: > > > I don't know, is it? I have found the description of properties and > the > > > /property response document examples to be extremely unclear. I opted > to > > > URI escape here, but maybe it's not the right thing to do. > > > > > > -Allen > > > > > > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > > > > > > >>That response comes back with some "%20" things in it. That seems > like > > >>a bug to me, but is it? > > >> > > >> > > >> > > >> > > >>Allen Day wrote: > > >> > > >>>This is a good region. I just realized the type ids coming back in > the > > >>>/type response don't match the type ids given in the /feature > response. > > >>>I'm in the process of switching over from using table PKs for types > to > > >>>using type names, so rather than using a filter "type=9770" you can > > >>>instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" > string > > >>>is the term's namespace. SO has already changed it to "SOFA" in > their > > >>>most recent SOFA file in CVS, but I haven't updated yet). > > >>> > > >>>So, sorry about that. I just change thed /type response back to give > > >>>primary keys until I get names working. All the types should now > work > > >>>with the type= filter. Here's a quick document you can grab, a > subregion > > >>>of what you asked for in your request that has something interesting > > >>>inside: > > >>> > > >>>http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlaps=chr21/21292691:21324284;type=9688 > > >>> > > >>>-Allen > > >>> > > >>> > > >>> > > >>>On Thu, 21 Apr 2005, Helt,Gregg wrote: > > >>> > > >>> > > >>> > > >>>>Cool, great to see a public server up and running! > > >>>> > > >>>>However, I'm having trouble with feature queries. For example: > > >>>>http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlap > > >>>>s=chr21/21500000:22500000;type=0000110 > > >>>> > > >>>>Which is an example of the most typical query IGB will be making -- > one > > >>>>overlap filter combined with one type filter. But this query > returns > > >>>>with a single line, no . I've tried it for > > >>>>every type returned by the type query and get the same results. > > >>>> > > >>>>Is there something wrong with my query syntax? > > >>>> > > >>>>If this is not a good region to test, can you send me an example > feature > > >>>>query URL where you know what the response should be? > > >>>> > > >>>> thanks! > > >>>> gregg > > >>>> > > >>>> > > >>>> > > >>>>>-----Original Message----- > > >>>>>From: das2-bounces at portal.open-bio.org > > >>>> > > >>>>[mailto:das2-bounces at portal.open- > > >>>> > > >>>> > > >>>>>bio.org] On Behalf Of Allen Day > > >>>>>Sent: Wednesday, April 20, 2005 10:42 PM > > >>>>>To: das2 at portal.open-bio.org > > >>>>>Subject: [DAS2] Human prototype server up > > >>>>> > > >>>>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > >>>> > > >>>>sequences > > >>>> > > >>>> > > >>>>>aligned to the genome w/ > 90% identity. > > >>>>> > > >>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > >>>>> > > >>>>>Here's a URL to get you going. This is running on 2 hosts: a dual > > >>>>>opteron/8GB webserver frontend and a dual opteron/16GB database > > >>>> > > >>>>backend. > > >>>> > > >>>> > > >>>>>There are no other applications on these hosts right now, so hammer > > >>>> > > >>>>away. > > >>>> > > >>>> > > >>>>>You might want to use HTTP 1.1 keepalive though, the responses can > be > > >>>>>*really* slow for /feature requests on large regions. The reason > for > > >>>> > > >>>>this > > >>>> > > >>>> > > >>>>>is the feature/part subpart requests are done feature by feature > due > > >>>> > > >>>>to > > >>>> > > >>>> > > >>>>>the 1-many relationship... I'm still trying to figure out how to > > >>>> > > >>>>improve > > >>>> > > >>>> > > >>>>>this. > > >>>>> > > >>>>>Components that aren't working in even a basic way: > > >>>>> > > >>>>> * alignments > > >>>>> * assemblies > > >>>>> * individual /feature/ requests > > >>>>> * versioned source response w/ supported formats > > >>>>> * most feature filters (overlaps, type, and exacttype work) > > >>>>> > > >>>>>-Allen > > >>>>>_______________________________________________ > > >>>>>DAS2 mailing list > > >>>>>DAS2 at portal.open-bio.org > > >>>>>http://portal.open-bio.org/mailman/listinfo/das2 > > >>>> > > >>>_______________________________________________ > > >>>DAS2 mailing list > > >>>DAS2 at portal.open-bio.org > > >>>http://portal.open-bio.org/mailman/listinfo/das2 > > >> > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > My reasoning is that it ISN'T a URL yet, so you don't escape it until > > you turn it into a URL. But Andrew can figure out which is right and > > document it. > > > > Allen Day wrote: > > > exactly. my reasoning was that it was a relative url and needed to be > > > escaped. > > > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > > > > > > >>This is a question for Andrew Dalke. > > >> > > >> > > >>Allen Day wrote: > > >> > > >>>I don't know, is it? I have found the description of properties and > the > > >>>/property response document examples to be extremely unclear. I > opted to > > >>>URI escape here, but maybe it's not the right thing to do. > > >>> > > >>>-Allen > > >>> > > >>> > > >>>On Thu, 21 Apr 2005, Ed Erwin wrote: > > >>> > > >>> > > >>> > > >>>>That response comes back with some "%20" things in it. That seems > like > > >>>>a bug to me, but is it? > > >>>> > > >>>> > > >>>> > > >>>> > > >>>>Allen Day wrote: > > >>>> > > >>>> > > >>>>>This is a good region. I just realized the type ids coming back in > the > > >>>>>/type response don't match the type ids given in the /feature > response. > > >>>>>I'm in the process of switching over from using table PKs for types > to > > >>>>>using type names, so rather than using a filter "type=9770" you can > > >>>>>instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" > string > > >>>>>is the term's namespace. SO has already changed it to "SOFA" in > their > > >>>>>most recent SOFA file in CVS, but I haven't updated yet). > > >>>>> > > >>>>>So, sorry about that. I just change thed /type response back to > give > > >>>>>primary keys until I get names working. All the types should now > work > > >>>>>with the type= filter. Here's a quick document you can grab, a > subregion > > >>>>>of what you asked for in your request that has something > interesting > > >>>>>inside: > > >>>>> > > >>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlaps=chr21/21292691:21324284;type=9688 > > >>>>> > > >>>>>-Allen > > >>>>> > > >>>>> > > >>>>> > > >>>>>On Thu, 21 Apr 2005, Helt,Gregg wrote: > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>>>Cool, great to see a public server up and running! > > >>>>>> > > >>>>>>However, I'm having trouble with feature queries. For example: > > >>>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlap > > >>>>>>s=chr21/21500000:22500000;type=0000110 > > >>>>>> > > >>>>>>Which is an example of the most typical query IGB will be making - > - one > > >>>>>>overlap filter combined with one type filter. But this query > returns > > >>>>>>with a single line, no . I've tried it > for > > >>>>>>every type returned by the type query and get the same results. > > >>>>>> > > >>>>>>Is there something wrong with my query syntax? > > >>>>>> > > >>>>>>If this is not a good region to test, can you send me an example > feature > > >>>>>>query URL where you know what the response should be? > > >>>>>> > > >>>>>> thanks! > > >>>>>> gregg > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>-----Original Message----- > > >>>>>>>From: das2-bounces at portal.open-bio.org > > >>>>>> > > >>>>>>[mailto:das2-bounces at portal.open- > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>bio.org] On Behalf Of Allen Day > > >>>>>>>Sent: Wednesday, April 20, 2005 10:42 PM > > >>>>>>>To: das2 at portal.open-bio.org > > >>>>>>>Subject: [DAS2] Human prototype server up > > >>>>>>> > > >>>>>>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > >>>>>> > > >>>>>>sequences > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>aligned to the genome w/ > 90% identity. > > >>>>>>> > > >>>>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > >>>>>>> > > >>>>>>>Here's a URL to get you going. This is running on 2 hosts: a > dual > > >>>>>>>opteron/8GB webserver frontend and a dual opteron/16GB database > > >>>>>> > > >>>>>>backend. > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>There are no other applications on these hosts right now, so > hammer > > >>>>>> > > >>>>>>away. > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>You might want to use HTTP 1.1 keepalive though, the responses > can be > > >>>>>>>*really* slow for /feature requests on large regions. The reason > for > > >>>>>> > > >>>>>>this > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>is the feature/part subpart requests are done feature by feature > due > > >>>>>> > > >>>>>>to > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>the 1-many relationship... I'm still trying to figure out how to > > >>>>>> > > >>>>>>improve > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>this. > > >>>>>>> > > >>>>>>>Components that aren't working in even a basic way: > > >>>>>>> > > >>>>>>>* alignments > > >>>>>>>* assemblies > > >>>>>>>* individual /feature/ requests > > >>>>>>>* versioned source response w/ supported formats > > >>>>>>>* most feature filters (overlaps, type, and exacttype work) > > >>>>>>> > > >>>>>>>-Allen > > >>>>>>>_______________________________________________ > > >>>>>>>DAS2 mailing list > > >>>>>>>DAS2 at portal.open-bio.org > > >>>>>>>http://portal.open-bio.org/mailman/listinfo/das2 > > >>>>>> > > >>>>>_______________________________________________ > > >>>>>DAS2 mailing list > > >>>>>DAS2 at portal.open-bio.org > > >>>>>http://portal.open-bio.org/mailman/listinfo/das2 > > >>>> > > From dalke at dalkescientific.com Thu Apr 21 23:11:54 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Thu, 21 Apr 2005 21:11:54 -0600 Subject: [DAS2] updated DAS/2 spec files Message-ID: <75d8f270590d2a0603e3926340743c15@dalkescientific.com> I've updated the spec in CVS. Here are the changes: - folded in the various changes I brought up last month ("PROPERTY-LIST" -> "PROPERTIES"; RFC dates -> ISO dates; etc.) Lincoln said yes to all the changes. - results of the template file are now saved to "das2_get.html" Given what Steve said today this means the web site should be updated to this version next time the cron is run - added a README describing how the spec is made from the template. Note: You can edit the spec directly as a normal HTML file. The templating language I use is ZopePageTemplates (also called "TAL") which extends HTML only through attributes. This are ignored by browsers so if Gregg wants to modify things, go right ahead. - checked ZopePageTemplates and ElementTree into CVS under the das/das2 directory. These are Python libraries used by the template processor program "convert_template.py" I checked them in so anyone can assemble the template from that directory doing either make or python convert_template.py das2_get_template.py das2_get.py I have verified that you can build the final "das2_get.py" from the template file on the open-bio machine "pub.open-bio.org" using the current setup. NOTE: If you change the RELAX-NG RNC files you'll need to rebuild the RNG and DTD files, and you'll need to configure the TRANG variable in the Makefile to point to your local copy of trang. I did not attempt to make this easy for someone who isn't on a unix machine. On unix the dependencies are rebuilt as part of the normal "make" NOTE: If you want to run the schema validation regression tests you must set the Makefile's JING variable to your local copy of java and jing. Then you can do "make verify" Andrew dalke at dalkescientific.com From Gregg_Helt at affymetrix.com Sat Apr 23 08:11:53 2005 From: Gregg_Helt at affymetrix.com (Helt,Gregg) Date: Sat, 23 Apr 2005 05:11:53 -0700 Subject: [DAS2] updated DAS/2 spec files Message-ID: I really like the links you've added out to RNC, RNG, and DTD schemas after each XML example. Very nice. One problem -- the DTDs aren't parsing as correct XML. The parser in my web browser is demanding that the " -----Original Message----- > From: das2-bounces at portal.open-bio.org [mailto:das2-bounces at portal.open- > bio.org] On Behalf Of Andrew Dalke > Sent: Thursday, April 21, 2005 8:12 PM > To: das2 at portal.open-bio.org > Subject: [DAS2] updated DAS/2 spec files > > I've updated the spec in CVS. Here are the changes: > > - folded in the various changes I brought up last month > ("PROPERTY-LIST" -> "PROPERTIES"; RFC dates -> ISO dates; etc.) > Lincoln said yes to all the changes. > > - results of the template file are now saved to "das2_get.html" > Given what Steve said today this means the web site should be > updated to this version next time the cron is run > > - added a README describing how the spec is made from the > template. > > Note: You can edit the spec directly as a normal HTML file. > The templating language I use is ZopePageTemplates (also > called "TAL") which extends HTML only through attributes. > This are ignored by browsers so if Gregg wants to modify > things, go right ahead. > > - checked ZopePageTemplates and ElementTree into CVS > under the das/das2 directory. These are Python libraries > used by the template processor program "convert_template.py" > I checked them in so anyone can assemble the template from > that directory doing either > > make > > or > > python convert_template.py das2_get_template.py das2_get.py > > I have verified that you can build the final "das2_get.py" > from the template file on the open-bio machine "pub.open-bio.org" > using the current setup. > > > NOTE: If you change the RELAX-NG RNC files you'll need > to rebuild the RNG and DTD files, and you'll need to > configure the TRANG variable in the Makefile to point to > your local copy of trang. I did not attempt to make this > easy for someone who isn't on a unix machine. On unix > the dependencies are rebuilt as part of the normal "make" > > NOTE: If you want to run the schema validation regression > tests you must set the Makefile's JING variable to your > local copy of java and jing. Then you can do "make verify" > > > Andrew > dalke at dalkescientific.com > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 From Gregg_Helt at affymetrix.com Mon Apr 25 12:52:10 2005 From: Gregg_Helt at affymetrix.com (Helt,Gregg) Date: Mon, 25 Apr 2005 09:52:10 -0700 Subject: [DAS2] DAS/2 abstract for BOSC 2005 Message-ID: Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. If you have time, please take a look and let me know if there needs to be any changes. If it's accepted I'll be giving a 20 minute presentation at BOSC. thanks! gregg -------------- next part -------------- A non-text attachment was scrubbed... Name: BOSC_2005_abstract.doc Type: application/msword Size: 32768 bytes Desc: BOSC_2005_abstract.doc URL: From edgrif at sanger.ac.uk Mon Apr 25 13:15:44 2005 From: edgrif at sanger.ac.uk (Ed Griffiths) Date: Mon, 25 Apr 2005 18:15:44 +0100 (BST) Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: References: Message-ID: gregg, > Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. > If you have time, please take a look and let me know if there needs to > be any changes. If it's accepted I'll be giving a 20 minute > presentation at BOSC. A couple of minor things: My name needs an "s": Ed Griffiths "Sanger Centre" should now read "Sanger Institute" otherwise it seems good to me, thanks for doing it. Ed -- ------------------------------------------------------------------------ | Ed Griffiths, Acedb development, Informatics Group, | | Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus, | | Hinxton, Cambridge CB10 1SA, UK | | | | email: edgrif at sanger.ac.uk Tel: +44-1223-494780 Fax: +44-1223-494919 | ------------------------------------------------------------------------ From lstein at cshl.edu Mon Apr 25 18:07:01 2005 From: lstein at cshl.edu (Lincoln Stein) Date: Mon, 25 Apr 2005 18:07:01 -0400 Subject: [DAS2] updated DAS/2 spec files In-Reply-To: <75d8f270590d2a0603e3926340743c15@dalkescientific.com> References: <75d8f270590d2a0603e3926340743c15@dalkescientific.com> Message-ID: <200504251807.04241.lstein@cshl.edu> Hi, I've done a chunk of work on the das2_put.html update document. I would be happy to migrate this over to Andrew's templating system, but I'd like to know where I can get the TRANG and JING jar files so that I can run "make." Is there anything else I need for the Python part of the conversion? Lincoln On Thursday 21 April 2005 11:11 pm, Andrew Dalke wrote: > I've updated the spec in CVS. Here are the changes: > > - folded in the various changes I brought up last month > ("PROPERTY-LIST" -> "PROPERTIES"; RFC dates -> ISO dates; etc.) > Lincoln said yes to all the changes. > > - results of the template file are now saved to "das2_get.html" > Given what Steve said today this means the web site should be > updated to this version next time the cron is run > > - added a README describing how the spec is made from the > template. > > Note: You can edit the spec directly as a normal HTML file. > The templating language I use is ZopePageTemplates (also > called "TAL") which extends HTML only through attributes. > This are ignored by browsers so if Gregg wants to modify > things, go right ahead. > > - checked ZopePageTemplates and ElementTree into CVS > under the das/das2 directory. These are Python libraries > used by the template processor program "convert_template.py" > I checked them in so anyone can assemble the template from > that directory doing either > > make > > or > > python convert_template.py das2_get_template.py das2_get.py > > I have verified that you can build the final "das2_get.py" > from the template file on the open-bio machine "pub.open-bio.org" > using the current setup. > > > NOTE: If you change the RELAX-NG RNC files you'll need > to rebuild the RNG and DTD files, and you'll need to > configure the TRANG variable in the Makefile to point to > your local copy of trang. I did not attempt to make this > easy for someone who isn't on a unix machine. On unix > the dependencies are rebuilt as part of the normal "make" > > NOTE: If you want to run the schema validation regression > tests you must set the Makefile's JING variable to your > local copy of java and jing. Then you can do "make verify" > > > Andrew > dalke at dalkescientific.com > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 -- Lincoln D. Stein Cold Spring Harbor Laboratory 1 Bungtown Road Cold Spring Harbor, NY 11724 NOTE: Please copy Sandra Michelsen on all emails regarding scheduling and other time-critical topics. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From Steve_Chervitz at affymetrix.com Mon Apr 25 20:43:39 2005 From: Steve_Chervitz at affymetrix.com (Chervitz, Steve) Date: Mon, 25 Apr 2005 17:43:39 -0700 Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: Message-ID: Looks good to me. Here's one minor change for the second to last sentence: "Client, server, and validation suite are all available as open source." to "Client, server, and validation suite are all available under open source licenses." (and all different ones at that!) (the parenthetical part is optional ;-). Steve > From: "Helt,Gregg" > Date: Mon, 25 Apr 2005 09:52:10 -0700 > To: > Subject: [DAS2] DAS/2 abstract for BOSC 2005 > > Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. > If you have time, please take a look and let me know if there needs to > be any changes. If it's accepted I'll be giving a 20 minute > presentation at BOSC. > > thanks! > gregg > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 From allenday at ucla.edu Mon Apr 25 23:26:45 2005 From: allenday at ucla.edu (Allen Day) Date: Mon, 25 Apr 2005 20:26:45 -0700 (PDT) Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: References: Message-ID: I'm willing to change my license to match others if it makes life easier for someone. -Allen On Mon, 25 Apr 2005, Chervitz, Steve wrote: > Looks good to me. Here's one minor change for the second to last sentence: > > "Client, server, and validation suite are all available as open source." > > to > > "Client, server, and validation suite are all available under open source > licenses." (and all different ones at that!) > > (the parenthetical part is optional ;-). > > Steve > > > > From: "Helt,Gregg" > > Date: Mon, 25 Apr 2005 09:52:10 -0700 > > To: > > Subject: [DAS2] DAS/2 abstract for BOSC 2005 > > > > Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. > > If you have time, please take a look and let me know if there needs to > > be any changes. If it's accepted I'll be giving a 20 minute > > presentation at BOSC. > > > > thanks! > > gregg > > > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 > From Steve_Chervitz at affymetrix.com Tue Apr 26 15:47:46 2005 From: Steve_Chervitz at affymetrix.com (Chervitz, Steve) Date: Tue, 26 Apr 2005 12:47:46 -0700 Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: Message-ID: I think it is something worth discussing. Having multiple licenses runs the risk of incompatibilities and could raise the barrier for involvement from the corporate world, where it would be easier for the legal department to evaluate a single license rather than multiple licenses before giving the OK to their developers. Steve > From: Allen Day > Date: Mon, 25 Apr 2005 20:26:45 -0700 (PDT) > To: "Chervitz, Steve" > Cc: Gregg Helt , > Subject: Re: [DAS2] DAS/2 abstract for BOSC 2005 > > I'm willing to change my license to match others if it makes life easier > for someone. > > -Allen > > On Mon, 25 Apr 2005, Chervitz, Steve wrote: > >> Looks good to me. Here's one minor change for the second to last sentence: >> >> "Client, server, and validation suite are all available as open source." >> >> to >> >> "Client, server, and validation suite are all available under open source >> licenses." (and all different ones at that!) >> >> (the parenthetical part is optional ;-). >> >> Steve >> >> >>> From: "Helt,Gregg" >>> Date: Mon, 25 Apr 2005 09:52:10 -0700 >>> To: >>> Subject: [DAS2] DAS/2 abstract for BOSC 2005 >>> >>> Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. >>> If you have time, please take a look and let me know if there needs to >>> be any changes. If it's accepted I'll be giving a 20 minute >>> presentation at BOSC. >>> >>> thanks! >>> gregg >>> >>> _______________________________________________ >>> DAS2 mailing list >>> DAS2 at portal.open-bio.org >>> http://portal.open-bio.org/mailman/listinfo/das2 >> >> _______________________________________________ >> DAS2 mailing list >> DAS2 at portal.open-bio.org >> http://portal.open-bio.org/mailman/listinfo/das2 >> From edgrif at sanger.ac.uk Tue Apr 26 15:50:39 2005 From: edgrif at sanger.ac.uk (Ed Griffiths) Date: Tue, 26 Apr 2005 20:50:39 +0100 (BST) Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: References: Message-ID: > I think it is something worth discussing. Having multiple licenses runs the > risk of incompatibilities and could raise the barrier for involvement from > the corporate world, where it would be easier for the legal department to > evaluate a single license rather than multiple licenses before giving the OK > to their developers. I'd just like to add that as someone who worked for IBM for 14 years I can tell you that this is undoubtedly the case ;-) Ed > > Steve > > > > From: Allen Day > > Date: Mon, 25 Apr 2005 20:26:45 -0700 (PDT) > > To: "Chervitz, Steve" > > Cc: Gregg Helt , > > Subject: Re: [DAS2] DAS/2 abstract for BOSC 2005 > > > > I'm willing to change my license to match others if it makes life easier > > for someone. > > > > -Allen > > > > On Mon, 25 Apr 2005, Chervitz, Steve wrote: > > > >> Looks good to me. Here's one minor change for the second to last sentence: > >> > >> "Client, server, and validation suite are all available as open source." > >> > >> to > >> > >> "Client, server, and validation suite are all available under open source > >> licenses." (and all different ones at that!) > >> > >> (the parenthetical part is optional ;-). > >> > >> Steve > >> > >> > >>> From: "Helt,Gregg" > >>> Date: Mon, 25 Apr 2005 09:52:10 -0700 > >>> To: > >>> Subject: [DAS2] DAS/2 abstract for BOSC 2005 > >>> > >>> Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. > >>> If you have time, please take a look and let me know if there needs to > >>> be any changes. If it's accepted I'll be giving a 20 minute > >>> presentation at BOSC. > >>> > >>> thanks! > >>> gregg > >>> > >>> _______________________________________________ > >>> DAS2 mailing list > >>> DAS2 at portal.open-bio.org > >>> http://portal.open-bio.org/mailman/listinfo/das2 > >> > >> _______________________________________________ > >> DAS2 mailing list > >> DAS2 at portal.open-bio.org > >> http://portal.open-bio.org/mailman/listinfo/das2 > >> > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 > > > -- ------------------------------------------------------------------------ | Ed Griffiths, Acedb development, Informatics Group, | | Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus, | | Hinxton, Cambridge CB10 1SA, UK | | | | email: edgrif at sanger.ac.uk Tel: +44-1223-494780 Fax: +44-1223-494919 | ------------------------------------------------------------------------ From lstein at cshl.edu Tue Apr 26 17:15:59 2005 From: lstein at cshl.edu (Lincoln Stein) Date: Tue, 26 Apr 2005 17:15:59 -0400 Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: References: Message-ID: <200504261716.00047.lstein@cshl.edu> Hi Gregg, It looks great, aside from the little things that others noticed earlier. Lincoln On Monday 25 April 2005 12:52 pm, Helt,Gregg wrote: > Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. > If you have time, please take a look and let me know if there needs to > be any changes. If it's accepted I'll be giving a 20 minute > presentation at BOSC. > > thanks! > gregg -- Lincoln D. Stein Cold Spring Harbor Laboratory 1 Bungtown Road Cold Spring Harbor, NY 11724 NOTE: Please copy Sandra Michelsen on all emails regarding scheduling and other time-critical topics. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From edgrif at sanger.ac.uk Wed Apr 27 11:06:25 2005 From: edgrif at sanger.ac.uk (Ed Griffiths) Date: Wed, 27 Apr 2005 16:06:25 +0100 (BST) Subject: [DAS2] jing, trang, python...the world... In-Reply-To: <45f24ce628766b59916de4e4a5ec05dc@dalkescientific.com> References: <4255C847.3090104@affymetrix.com> <45f24ce628766b59916de4e4a5ec05dc@dalkescientific.com> Message-ID: Andrew, I've got jing and trang and installed them and as far as I can tell they seem to be doing something....python is another matter though: cd /Users/edgrif/BIOINFORMATICS/das/das2/ make python ./convert_template.py das2_get_template.html das2_get.html Traceback (most recent call last): File "./convert_template.py", line 123, in ? main(sys.argv) File "./convert_template.py", line 120, in main open(output_filename, "w").write(pt(context=context)) File "./convert_template.py", line 34, in __call__ return self.pt_render(extra_context=context) File "/Users/edgrif/BIOINFORMATICS/das/das2/ZopePageTemplates/__init__.py", line 86, in pt_render raise PTRuntimeError, 'Page Template %s has errors.' % self.id ZopePageTemplates.PTRuntimeError: Page Template (unknown) has errors. make: *** [das2_get.html] Error 1 Compilation exited abnormally with code 2 at Wed Apr 27 16:02:01 Is this something that you came across, can you give any clues ? Is it the input to python that is mangled or is it a problem with the python code ? Ed -- ------------------------------------------------------------------------ | Ed Griffiths, Acedb development, Informatics Group, | | Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus, | | Hinxton, Cambridge CB10 1SA, UK | | | | email: edgrif at sanger.ac.uk Tel: +44-1223-494780 Fax: +44-1223-494919 | ------------------------------------------------------------------------ From Gregg_Helt at affymetrix.com Thu Apr 28 14:16:35 2005 From: Gregg_Helt at affymetrix.com (Helt,Gregg) Date: Thu, 28 Apr 2005 11:16:35 -0700 Subject: [DAS2] DAS/2 UML Class Diagram Message-ID: I've attached a jpeg of my latest attempt at making UML class diagrams based on the DAS/2 spec. I sent an earlier attempt a few weeks ago, but I think somehow the mailing list filters ate it. The zuml file (zipped up xmi) this is based on is checked into the genoviz sourceforge repository as documentation/GenometryWithDas.zuml. There's a lot of my "genometry" data modeling in the xmi, but the abstract DAS/2 model doesn't refer to the genometry stuff - I wanted to have an abstract model that doesn't depend on other stuff. My actual DAS/2 client implemention is a tighter integration between DAS and genometry models. Anyway, I've found having this diagram very useful when thinking about the spec. For example two of the spec changes I've been thinking about recently, removing the element from the versioned source info and adding a tag for types, I've highlighted in the model in red and green, respectively. If you have a chance please take a look at the diagram and let me know if anything looks wrong relative to the spec. gregg -------------- next part -------------- A non-text attachment was scrubbed... Name: Das2AbstractAPI.JPG Type: image/jpeg Size: 103971 bytes Desc: Das2AbstractAPI.JPG URL: From edgrif at sanger.ac.uk Thu Apr 28 15:15:23 2005 From: edgrif at sanger.ac.uk (Ed Griffiths) Date: Thu, 28 Apr 2005 20:15:23 +0100 (BST) Subject: [DAS2] das docs updates... In-Reply-To: References: Message-ID: All, while I've been over here with Lincoln working on the das2 write back spec I've followed Andrews good example and have put some new template style files into the repository. There are now 3 das2 spec files: das2_client_template.html das2_get_template.html das2_put_template.html These files get munged by a process set up by Andrew via a makefile into the actual spec docs: das2_client.html das2_get.html das2_put.html There are a couple of points to note here if you decide to alter the files: 1) You will need to get hold trang and jing (easy to find/download on the web) 2) You will need to edit the Makefiles first couple of lines to point at where you installed trang/jing on your system 3) You need to edit the files that are inputs, not the output files (!), e.g. you should edit das2_put_template.html, _not_ das2_put.html !! 4) You should also be aware that trang/jing/python scripts used to make the final spec docs are sensitive to bad html, you may find the html checking program "tiny" helpful in finding problems (again easily found/downloaded on the web). 5) If you want to read the specs then read the final versions: das2_client.html das2_get.html das2_put.html 6) The output files are currently stored in cvs, this is probably a good idea in fact otherwise we will all have to set up the environment to make them. Please do make comments/corrections etc., I expect I will be adding more over today and tomorrow but then its back off to "sunny" hinxton for acedb/zmap the world... Ed -- ------------------------------------------------------------------------ | Ed Griffiths, Acedb development, Informatics Group, | | Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus, | | Hinxton, Cambridge CB10 1SA, UK | | | | email: edgrif at sanger.ac.uk Tel: +44-1223-494780 Fax: +44-1223-494919 | ------------------------------------------------------------------------ From allenday at ucla.edu Sat Apr 30 03:58:28 2005 From: allenday at ucla.edu (Allen Day) Date: Sat, 30 Apr 2005 00:58:28 -0700 (PDT) Subject: [DAS2] alteration to /type response In-Reply-To: References: Message-ID: Hi, I'd like to have the element contain 0..N subelements. Consider the following subgraph of SOFA, described as das2xml: Showing the types/supertypes available in this way allows the client to infer that a request for "feature?type=SO:mRNA" is going to return both "SO:mRNA" and "SO:CDS" records -- even if the "SO:CDS" are not directly attached to a "SO:mRNA" parent (review "type" vs. "exacttype" feature filters to understand why). This information would also be useful for rendering a tree widget in a client for selecting types of interest. Also, for the "type" feature filter, we may only want to traverse "OBO_REL:is_a" edges and not "OBO_REL:part_of" edges, or other edge types such as "OBO_REL:adjacent_to"... this would prevent the server from returning "SO:CDS" features when "SO:mRNA" is requested, but not prevent the return of "SO:polycistronic_mRNA" features, because "SO:polycistronic_mRNA" "OBO_REL:is_a" "SO:mRNA". -Allen From allenday at ucla.edu Sat Apr 30 04:35:26 2005 From: allenday at ucla.edu (Allen Day) Date: Sat, 30 Apr 2005 01:35:26 -0700 (PDT) Subject: [DAS2] Human prototype server up In-Reply-To: References: Message-ID: More feature filters are now available. * contains * identical * inside * name Retrieve all tropomyosins and receptor tyrosine kinases by gene symbol: http://radius.genomics.ctrl.ucla.edu/das/genome/chado/chado-Hsa-17/feature?name=TPM*,*RTK* I'll probably have the "att" filter working tomorrow -- will post here again if so. -Allen From dalke at dalkescientific.com Wed Apr 6 07:53:00 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Wed, 6 Apr 2005 01:53:00 -0600 Subject: [DAS2] feature filter examples Message-ID: <4ccf0d170ccaf8cbd0ce8dc398c7fe8f@dalkescientific.com> I'm having some difficulty understanding the feature query language. There's an example in the spec which is overlaps=Chr3/1000:2000;type=exon,intron,splice_site (Features of type exon, intron, or splice_site that overlap region 1000-2000 on Chr3.) Based on that, the following appears to be legal overlaps=Contig1/1:1000;att=hand:fizzbin;overlaps=Contig1/5:200 That is, it is legal for the same feature filter ("overlaps") to exist multiple times in the query. If so there should be a note pointing that out. Are the following allowed? 1) an OR of the values for a region overlaps=Contig1/1:1000,Contig8/5:200 2) an OR of an att query att=hand:fizzbin,slark 3) and OR and an AND of the same attribute in an att filter att=hand:fizzbin,slark;att=hand:royal This is the same as (hand contains "fizzbin" OR hand contains "slark") AND (hand contains "royal") so would match "royal fizzbin" hands. I think they are all legal. I think I can improve the wording a bit for clarification. Andrew dalke at dalkescientific.com From dalke at dalkescientific.com Wed Apr 6 08:28:24 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Wed, 6 Apr 2005 02:28:24 -0600 Subject: [DAS2] Re: feature filter examples In-Reply-To: <4ccf0d170ccaf8cbd0ce8dc398c7fe8f@dalkescientific.com> References: <4ccf0d170ccaf8cbd0ce8dc398c7fe8f@dalkescientific.com> Message-ID: <4852f837562e1f84abcf986f4400a037@dalkescientific.com> Another confusing situation. Is the following allowed att=name:Andrew,lang:en to mean attribute name contains "Andrew" OR attribute lang contains "en" This appears to be legal under the current spec because Within the URL query string, multiple criterion=value pairs are ANDED together: ... To describe OR relationships, you may separate filter values by commas. In my above example the "criterion" is "att" (because it's to the left of the "=") and the "value" is "name:Andrew,lang:en" which is made of two value fields "name:Andrew" "lang:en" Note though that that means att=name:Andrew,lang:en does not mean to find attribute "name" contains "Andrew" OR attribute "name" contains "lang:en" Nor is it attribute "name" contains "Andrew" OR attribute "name" contains "lang" SYNTAX ERROR - ':' not allowed at this point It also means that att=name:Andrew,Gregg IS NOT legal, because "Gregg" is not a valid value for the "att" criterion. Andrew dalke at dalkescientific.com From allenday at ucla.edu Wed Apr 6 16:02:40 2005 From: allenday at ucla.edu (Allen Day) Date: Wed, 6 Apr 2005 09:02:40 -0700 (PDT) Subject: [DAS2] feature filter examples In-Reply-To: <4ccf0d170ccaf8cbd0ce8dc398c7fe8f@dalkescientific.com> References: <4ccf0d170ccaf8cbd0ce8dc398c7fe8f@dalkescientific.com> Message-ID: You're right -- this is unclear. I was talking to Lincoln about this Monday. The following are true by the current document for key=value strings: [1] comma-delimited values for the same key are ORed [2] different keys are ANDed But is unclear how to handle the separate values for keys of the same name. I would interpret separate values as a case of [2], as you do here: > att=hand:fizzbin,slark;att=hand:royal > > This is the same as > (hand contains "fizzbin" OR hand contains "slark") AND > (hand contains "royal") matching features with "hand" attribute subset values of {royal,fizzbin} and {royal,slark}. We might consider revising this though. It could create problems with other technologies that use GET, for example HTML forms with GET method. You might want to have a multi-select list that allows you to select attribute values for "hand", and if you chose "fizzbin" and "slark", you'd get a URI param string looking like: .../feature?att=hand:fizzbin;att=hand:slark As far as I'm aware there's no way to change this short of having another script handle this request, munging the parameters to what the user really meant, and returning the browser a redirect header. -Allen On Wed, 6 Apr 2005, Andrew Dalke wrote: > I'm having some difficulty understanding the feature query > language. There's an example in the spec which is > > overlaps=Chr3/1000:2000;type=exon,intron,splice_site > > (Features of type exon, intron, or splice_site that > overlap region 1000-2000 on Chr3.) > > Based on that, the following appears to be legal > > overlaps=Contig1/1:1000;att=hand:fizzbin;overlaps=Contig1/5:200 > > That is, it is legal for the same feature filter ("overlaps") > to exist multiple times in the query. If so there should > be a note pointing that out. > > > Are the following allowed? > > 1) an OR of the values for a region > > overlaps=Contig1/1:1000,Contig8/5:200 > > 2) an OR of an att query > > att=hand:fizzbin,slark > > 3) and OR and an AND of the same attribute in an att filter > > att=hand:fizzbin,slark;att=hand:royal > > This is the same as > (hand contains "fizzbin" OR hand contains "slark") AND > (hand contains "royal") > > so would match "royal fizzbin" hands. > > I think they are all legal. I think I can improve > the wording a bit for clarification. > > > Andrew > dalke at dalkescientific.com > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 > From ed_erwin at affymetrix.com Wed Apr 6 22:29:57 2005 From: ed_erwin at affymetrix.com (Ed Erwin) Date: Wed, 06 Apr 2005 15:29:57 -0700 Subject: [DAS2] feature filter examples In-Reply-To: References: Message-ID: <425462E5.5080000@affymetrix.com> I'm glad someone is paying attention to these details. Please also keep in mind making the parsing of the parameters straitforward. There should be some simple deterministic algorithm like, for example, this: 1. Split the string at ';' 2. apply URL-decoding (to convert "%20" and "+" to " ", etc.) 3. split at the first "=" into left-hand and right-hand 4. split right-hand at the first ":" into right-hand-one and right-hand-two 5. split right-hand-two at each "," I'm certain this is not the exactly right formula, but I'm saying we need to make sure that there is some known formula that works and is unambiguous. Consider the tough cases like spaces and punctuation characters like "hand:fizz bin 3.0,genscan+" Allen Day wrote: > You're right -- this is unclear. I was talking to Lincoln about this > Monday. The following are true by the current document for key=value > strings: > > [1] comma-delimited values for the same key are ORed > [2] different keys are ANDed > > But is unclear how to handle the separate values for keys of the same > name. I would interpret separate values as a case of [2], as you do here: > > > att=hand:fizzbin,slark;att=hand:royal > > > > This is the same as > > (hand contains "fizzbin" OR hand contains "slark") AND > > (hand contains "royal") > > matching features with "hand" attribute subset values of {royal,fizzbin} > and {royal,slark}. > > We might consider revising this though. It could create problems with > other technologies that use GET, for example HTML forms with GET method. > You might want to have a multi-select list that allows you to select > attribute values for "hand", and if you chose "fizzbin" and "slark", you'd > get a URI param string looking like: > > .../feature?att=hand:fizzbin;att=hand:slark > > As far as I'm aware there's no way to change this short of having another > script handle this request, munging the parameters to what the user really > meant, and returning the browser a redirect header. > > -Allen > > > On Wed, 6 Apr 2005, Andrew Dalke wrote: > > > I'm having some difficulty understanding the feature query > > language. There's an example in the spec which is > > > > overlaps=Chr3/1000:2000;type=exon,intron,splice_site > > > > (Features of type exon, intron, or splice_site that > > overlap region 1000-2000 on Chr3.) > > > > Based on that, the following appears to be legal > > > > overlaps=Contig1/1:1000;att=hand:fizzbin;overlaps=Contig1/5:200 > > > > That is, it is legal for the same feature filter ("overlaps") > > to exist multiple times in the query. If so there should > > be a note pointing that out. > > > > > > Are the following allowed? > > > > 1) an OR of the values for a region > > > > overlaps=Contig1/1:1000,Contig8/5:200 > > > > 2) an OR of an att query > > > > att=hand:fizzbin,slark > > > > 3) and OR and an AND of the same attribute in an att filter > > > > att=hand:fizzbin,slark;att=hand:royal > > > > This is the same as > > (hand contains "fizzbin" OR hand contains "slark") AND > > (hand contains "royal") > > > > so would match "royal fizzbin" hands. > > > > I think they are all legal. I think I can improve > > the wording a bit for clarification. > > > > > > Andrew > > dalke at dalkescientific.com > > > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 > From dalke at dalkescientific.com Thu Apr 7 05:53:37 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Wed, 6 Apr 2005 23:53:37 -0600 Subject: [DAS2] feature filter examples In-Reply-To: <425462E5.5080000@affymetrix.com> References: <425462E5.5080000@affymetrix.com> Message-ID: Ed: > I'm glad someone is paying attention to these details. That's what I'm paid for. :) > Please also keep in mind making the parsing of the parameters > straitforward. There should be some simple deterministic > algorithm like, for example, this: One concern I have is the mix of two different levels of URL escaping. Suppose I searched for the "name" "Dalke,Andrew" To make it fit in with at least Python's standard CGI GET query parser it would need to look like this: name=Dalke%252C+Andrew >>> import cgi >>> cgi.parse_qs("name=Dalke%252CAndrew") {'name': ['Dalke%2CAndrew']} >>> There is a double escape of the "," because the normal cgi processing rules allow anything to be escaped, even the special characters of ';' and ','. If I only had one level of escaping then it would be processed as >>> cgi.parse_qs("name=Dalke%2CAndrew") {'name': ['Dalke,Andrew']} >>> Note that my search for the exact string "Dalke,Andrew" has now been turned into a search for "Dalke" OR "Andrew". I think the algorithm for parsing this string is as follows. 1. URL-decode the query string 2. Split the string at ';'. These sub terms will be ANDed together. 3. For each subterm, find the category (the part before the '=' sign) and the value (the part after the '='). 4. Split the value on ','. These sub terms will be ORed together. 5a. If the category is 'att' then look for the attribute name before the ':' and the attribute value after the ':'. This is a URL-encoded glob query. 5b. Otherwise this is a URL-encoded query as appropriate for the given category. Steps 1, 2 and 3 are done as part of any standard CGI library. Another option which came up at the pre-DAS/2 meeting at CSHL 1.5 years ago was use a Google (or Entrez ;) style query, which would look like type:exon contains:Chr1/50:99 name:"Dalke, Andrew" or pfam:"protein kinase" overlaps:Chr3/1000:2000:-1 Google allows a simple OR in its syntax. pfam:"protein kinase" OR pfam:connexin inside:Chr3 The OR only applies to the two terms on either side so this is the same as (pfam:"protein kinase" OR pfam:connexin) AND inside:Chr3 I assume that name:Andrew OR name:Gregg OR name:Ed AND inside:Chr3 would be the same as ((name:Andrew OR name:Gregg) OR name:Ed) AND inside:Chr3 And a recollection from logic class - all boolean queries can be written in conjunctive normal form; as the AND of a set of OR statements. It just might be very verbose. ;) Note in my example that I've eliminated the "att" prefix so there nothing visually to distinguish between generic/database-defined attributes and DAS/2 required properties. Do we need that? An advantage is that it keeps the DAS/2 search names distinct from properties in the database. If that's important then I'll suggest we allow the prefix of "att:" if people want to be sure that there won't be a conflict, but that it's optional. That is: name:Andrew att:name:Andrew selects features which have the DAS/2 defined feature name of "Andrew" AND the arbitrary database "name" attribute containing "Andrew" The other possibility is to reserve "das:" for fields defined in the spec, making this query das:name:Andrew name:Andrew or to make things prettier (I don't like the double ':') das-name:Andrew name:Andrew A few other things I've noticed in the query spec. The definition for the "att" field says Glob-style wildcards are allowed in the values. There's an example which looks like att=est_evidence:1 Match features with an "est_evidence" property of "1" Does this match "1" identically or does it also match "100" and "91"? If the latter aren't allowed then suppose I do att=est_evidence:*1* Are 100 and 91 now possible matches? Why don't we weaken the requirements for how a server must respond to a given query? Right now all attribute fields must be searchable as strings by glob. Some fields may only support searching by term, perhaps with limited stemming. Others may be numeric with no glob support at all. Some implementers may even want to support range searches for a given numeric field, perhaps like: weight:1000..2500 BTW, why are we using "att" for this prefix instead of "prop"? After all, aren't these the property names? Speaking of which, given a property attribute name like "weight" how does a client get to a description of that property? It looks like it's magic - it's tacked on to the base property URL so base property URL: http://www/das/genome/volvox/1/property/ property name: weight means the description can be found at: http://www/das/genome/volvox/1/property/weight Going the other way around, to get a list of potential property fields, does a client start with the PROPERTIES request at http://www/das/genome/volvox/1/property/ then get the fully resolved 'id' for each PROPERTY (which may be joined with the base or may be a full URI itself) and extract the last term in the id? If so, that won't work with LSIDs or other opaque identifier. Better would be to add a "name" attribute to the PROPERTY element so the whole thing looks like and the 'name' field is what's used in the queries. This could be expanded some more to flag which fields are searchable and a documentation for how to do the search. Eg, the "can_query" attribute if it exists and is anything other the empty string means that searches are allowed on the given property and the "query_href" attribute points to documentation about the format of that query. Andrew dalke at dalkescientific.com From ed_erwin at affymetrix.com Thu Apr 7 23:54:47 2005 From: ed_erwin at affymetrix.com (Ed Erwin) Date: Thu, 07 Apr 2005 16:54:47 -0700 Subject: [DAS2] feature filter examples In-Reply-To: References: Message-ID: <4255C847.3090104@affymetrix.com> What happened to the "+" symbol in "name=Dalke%252C+Andrew" ? Was that supposed to be a URL-encoded space? If so, is that your recommended way of encoding it (single-encoded vs. double-encoded) ? Andrew Dalke wrote: > One concern I have is the mix of two different levels of URL > escaping. Suppose I searched for the "name" "Dalke,Andrew" > To make it fit in with at least Python's standard CGI GET > query parser it would need to look like this: > > name=Dalke%252C+Andrew > > >>> import cgi > >>> cgi.parse_qs("name=Dalke%252CAndrew") > {'name': ['Dalke%2CAndrew']} > >>> > From dalke at dalkescientific.com Fri Apr 8 00:14:41 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Thu, 7 Apr 2005 18:14:41 -0600 Subject: [DAS2] feature filter examples In-Reply-To: <4255C847.3090104@affymetrix.com> References: <4255C847.3090104@affymetrix.com> Message-ID: <74a05dc72b4afc8bd6de623c96e6e11b@dalkescientific.com> Ed: > What happened to the "+" symbol in "name=Dalke%252C+Andrew" ? > > Was that supposed to be a URL-encoded space? If so, is that your > recommended way of encoding it (single-encoded vs. double-encoded) ? Whoops! I started with a space but then figured it would distract from the discussion (having two escaped fields instead of one) so I hand-deleted it. Looks like I didn't catch everything. Andrew dalke at dalkescientific.com From dalke at dalkescientific.com Fri Apr 8 06:10:57 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Fri, 8 Apr 2005 00:10:57 -0600 Subject: [DAS2] feature filter examples In-Reply-To: <4255C847.3090104@affymetrix.com> References: <4255C847.3090104@affymetrix.com> Message-ID: <45f24ce628766b59916de4e4a5ec05dc@dalkescientific.com> I just realized I forgot today's conference call! Ed: > Was that supposed to be a URL-encoded space? If so, is that your > recommended way of encoding it (single-encoded vs. double-encoded) ? I didn't answer that one. It doesn't make a difference. The unescaping for both '+' and '%xx' are done by the same piece of code. I've been thinking more about simple query languages. The one I proposed is Google like. I also remembered that Lucene has a Google like query language. http://lucene.apache.org/java/docs/queryparsersyntax.html except it's rather more complicated because the query engine is more powerful. And unlike Google multiple terms with implicit conjunction are treated as an OR search instead of an AND search. (So "this that" is the same as "this OR that".) Then there's MySQL's boolean search language at http://dev.mysql.com/doc/mysql/en/fulltext-boolean.html which has a different set of operations but doesn't allow qualified searches like name:Romana . All are different, but at least have a common core of qualifier:search_text Compare to Entrez which looks like search_text[qualifier] In taking a closer look at the corner cases I noticed that it's awfully hard to handle non-alpha characters. Consider a search for 5'. In Google that gets turned into a search for 5. "5' end" (without double quotes in the search) has a few non-relevant hits in the top 10, like for NetScreen-5 End of Life (EOL) Announcement How well do we want to support special characters? DAS/1 includes examples like "5'UTR" Is case sensitivity important? How much should be required by the spec and how much can be left up to the implementor? Realistically speaking I don't think we can make that many restrictions on what the server can do. As I mentioned, I'm not even sure if we can always support the glob-style searches specified by the current spec. (Eg, with numeric fields. Also, how can it be implemented in MySQL?) I'm fine with a simple query language like this: query ::= term* term ::= "OR" word quoted_phrase qualifier ":" word qualifier ":" quoted_phrase word ::= /[A-Za-z0-9_-]+/ quoted_phrase ::= /"[^"]*"/ qualifier ::= /[a-zA-z][a-zA-Z0-9_-]*/ If more selectivity is needed then query ::= modified_term* modified_term ::= prefix term suffix prefix ::= /[+-]?/ # +gene means "gene" must be found # -gene means "gene" must not be present suffix ::= /[*]?/ # gene* allow "gene" and "genetic" This appears to be supportable under both MySQL and Lucene. Perhaps we can have another parameter to the search to specify the query language ("ql") used. If not specified the default is the DAS/2 query language. This would let server implementers tweak the interface as need be. BTW, in looking over the DAS/1 spec the reason this is an issue now is that we allow searches over arbitrary fields and searches for something other than an identifier. Andrew dalke at dalkescientific.com From allenday at ucla.edu Sun Apr 10 04:27:50 2005 From: allenday at ucla.edu (Allen Day) Date: Sat, 9 Apr 2005 21:27:50 -0700 (PDT) Subject: [DAS2] modification to /type request & response Message-ID: Hi, I'm proposing a change to the way type requests are handled. Background: According to the spec, a genomic feature may be typed by a term from the Sequence Ontology Feature Annotation (sofa), or may be typed by a term from another ontology so long as the term inherits from a sofa term. I have two problems with this. First, there may be other records we want to type -- as a concrete example, let's say we want to type a protein feature as part of protein-DAS, and the type is "alpha helix", which is not in sofa and may be difficult to have inherit from an existing sofa type. I have other real-world examples from the array-DAS document I'm working on, if anyone cares to see them -- related to biological sample annotation. I'd like to remove the constraint that all types inherit from a type. It makes sense to keep a constraint like this in place for genome sequence features, but not any other record type. Second, making a /type request is, according to the spec, supposed to return all types known to the server. If we allow non-sofa-inheriting types into the datasource, the result can be a *very* large document. As a concrete example, serving up just sofa produces a das2xml 64KB document containing 243 types. By linear extrapolation, mixing in the Gene Ontology, which contains 18651 types, will produce a document in excess of 4.8MB. My DAS server actually contains ~15 ontologies that I could conceivably want to annotate with. I'd like to be able to restrict a /type response to a subset of types known by the server, perhaps using the following: * /das/genome/type returns all types known to the server * /das/genome/type?namespace=sofa.ontology returns all types known to the server under the "sofa.ontology" namespace * /das/genome/type?namespaces returns all type namespaces known to the server -Allen From ed_erwin at affymetrix.com Mon Apr 11 17:59:52 2005 From: ed_erwin at affymetrix.com (Ed Erwin) Date: Mon, 11 Apr 2005 10:59:52 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: References: Message-ID: <425ABB18.8060008@affymetrix.com> Adding the "namespaces" query seems reasonable to me. But additionally, how about /das/genome/type returning only those types for which the server actually has some existing annotations, not all the annotations it "knows" about. Allen Day wrote: > > Second, making a /type request is, according to the spec, supposed to > return all types known to the server. If we allow non-sofa-inheriting > types into the datasource, the result can be a *very* large document. > .......... > > I'd like to be able to restrict a /type response to a subset of types > known by the server, perhaps using the following: > > * /das/genome/type > > returns all types known to the server > > * /das/genome/type?namespace=sofa.ontology > > returns all types known to the server under the "sofa.ontology" > namespace > > * /das/genome/type?namespaces > > returns all type namespaces known to the server > From Steve_Chervitz at affymetrix.com Mon Apr 11 19:35:03 2005 From: Steve_Chervitz at affymetrix.com (Chervitz, Steve) Date: Mon, 11 Apr 2005 12:35:03 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: Message-ID: Allen Day writes: > I'd like to remove the constraint that all types inherit from a type. It > makes sense to keep a constraint like this in place for genome sequence > features, but not any other record type. One aim of DAS/2 is to disallow the "anything goes" policy for feature typing, and to constrain them to be part of some ontology or related to a term in an existing ontology so that DAS/2 clients have a chance of making sense of them. The DAS/2 grant contemplates how to extend an existing ontology, but doesn't discuss plans for incorporating types that don't fit into any established ontology. Here's the relevant snippet from the grant: "Using a standard sequence ontology does not preclude extending the ontology. One of the advantages of using RDF and OWL to specify ontologies is that there is a formal mechanism for anyone to extend an existing ontology, which gives rise to distributed ontologies. This ability to extend ontologies is built on the same URI linking concepts previously discussed regarding feature ids. Thus for example someone with detailed knowledge of some the biology of repetitive sequences, for instance, may be annotating repeat data and want a much finer classification of repeat types than is specified in the reference ontology. They can define an extension of the existing ontology via RDF that refers to the existing ontology and specifies the extension (such as LINE-variantX isa type of LINE), then serve up DAS sequence features that refer to this extended ontology rather than the reference ontology. But because the referenced ontology formally extends the standard ontology, a DAS client could still be aware that LINE-variantX is a type of repeat, if for example it wants to display all repeats using the same color. Many software tools are now available to make such ontology extensions easy." Regarding protein feature types, I think the SO team plans to put them in SO/SOFA since they summarize it as, "a set of terms used to describe features on a nucleotide or protein sequence." But there is no support for proteins in SO/SOFA yet. Suzi? It does seem like there should be an avenue for people to say, "Here's a new type that doesn't belong to any established ontology (or if it does, I'm not sure where it goes)." Could such an avenue be opened without having it be abused? Interesting google result: Searching for "protein sequence feature ontology" gives about 35,000 hits, the second of which is the DAS/2 spec introduction! Steve From Steve_Chervitz at affymetrix.com Mon Apr 11 19:38:21 2005 From: Steve_Chervitz at affymetrix.com (Chervitz, Steve) Date: Mon, 11 Apr 2005 12:38:21 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: <425ABB18.8060008@affymetrix.com> Message-ID: "Erwin, Ed" writes: > Adding the "namespaces" query seems reasonable to me. > > But additionally, how about /das/genome/type returning only those types > for which the server actually has some existing annotations, not all the > annotations it "knows" about. I like Ed's restriction that it returns only the types for which it has annotations. But I can see a need for asking a server, "What are all the valid feature types?" This would be useful in the context of a user wanting to add a new annotation. Perhaps it would be reasonable to have the server's default behavior to return only types for which it has annotations. If a client wants all types, it could be specified via a flag. So we would have * /das/genome/type?namespace=sofa.ontology returns only sofa types for which it has annotations * /das/genome/type?namespace=sofa.ontology;all=1 returns all known sofa types Steve > Allen Day wrote: >> >> Second, making a /type request is, according to the spec, supposed to >> return all types known to the server. If we allow non-sofa-inheriting >> types into the datasource, the result can be a *very* large document. >> .......... >> >> I'd like to be able to restrict a /type response to a subset of types >> known by the server, perhaps using the following: >> >> * /das/genome/type >> >> returns all types known to the server >> >> * /das/genome/type?namespace=sofa.ontology >> >> returns all types known to the server under the "sofa.ontology" >> namespace >> >> * /das/genome/type?namespaces >> >> returns all type namespaces known to the server >> From suzi at fruitfly.org Mon Apr 11 19:52:13 2005 From: suzi at fruitfly.org (Suzanna Lewis) Date: Mon, 11 Apr 2005 12:52:13 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: References: Message-ID: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> On Apr 11, 2005, at 12:35 PM, Chervitz, Steve wrote: > > Allen Day writes: >> I'd like to remove the constraint that all types inherit from a type. >> It >> makes sense to keep a constraint like this in place for genome >> sequence >> features, but not any other record type. > > One aim of DAS/2 is to disallow the "anything goes" policy for feature > typing, and to constrain them to be part of some ontology or related > to a > term in an existing ontology so that DAS/2 clients have a chance of > making > sense of them. > > The DAS/2 grant contemplates how to extend an existing ontology, but > doesn't > discuss plans for incorporating types that don't fit into any > established > ontology. Here's the relevant snippet from the grant: > > "Using a standard sequence ontology does not preclude extending the > ontology. One of the advantages of using RDF and OWL to specify > ontologies > is that there is a formal mechanism for anyone to extend an existing > ontology, which gives rise to distributed ontologies. This ability to > extend ontologies is built on the same URI linking concepts previously > discussed regarding feature ids. Thus for example someone with > detailed > knowledge of some the biology of repetitive sequences, for instance, > may be > annotating repeat data and want a much finer classification of repeat > types > than is specified in the reference ontology. They can define an > extension of > the existing ontology via RDF that refers to the existing ontology and > specifies the extension (such as LINE-variantX isa type of LINE), then > serve > up DAS sequence features that refer to this extended ontology rather > than > the reference ontology. But because the referenced ontology formally > extends the standard ontology, a DAS client could still be aware that > LINE-variantX is a type of repeat, if for example it wants to display > all > repeats using the same color. Many software tools are now available > to make > such ontology extensions easy." > > Regarding protein feature types, I think the SO team plans to put them > in > SO/SOFA since they summarize it as, "a set of terms used to describe > features on a nucleotide or protein sequence." But there is no support > for > proteins in SO/SOFA yet. Suzi? Yes, that is pretty much where it stands. There is the intention of doing this, but we have not yet gotten to that point. Including the basic structural descriptions of a protein (alpha helices, beta sheets...) seems an obvious new addition and easy enough to do. I strongly agree with Steve here that we absolutely must stay away from ad hoc types. They are the sirens on the rocks: alluring, but lethal. -S > > It does seem like there should be an avenue for people to say, "Here's > a new > type that doesn't belong to any established ontology (or if it does, > I'm not > sure where it goes)." Could such an avenue be opened without having it > be > abused? > > Interesting google result: Searching for "protein sequence feature > ontology" > gives about 35,000 hits, the second of which is the DAS/2 spec > introduction! > > Steve > > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 From allenday at ucla.edu Mon Apr 11 20:33:40 2005 From: allenday at ucla.edu (Allen Day) Date: Mon, 11 Apr 2005 13:33:40 -0700 (PDT) Subject: [DAS2] modification to /type request & response In-Reply-To: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> References: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> Message-ID: >> I'd like to remove the constraint that all types inherit from a type. >> It >> makes sense to keep a constraint like this in place for genome >> sequence >> features, but not any other record type. [...] > > Regarding protein feature types, I think the SO team plans to put them > > in > > SO/SOFA since they summarize it as, "a set of terms used to describe > > features on a nucleotide or protein sequence." But there is no support > > for > > proteins in SO/SOFA yet. Suzi? > > Yes, that is pretty much where it stands. There is the intention of > doing this, but we have not yet gotten to that point. Including the > basic structural descriptions of a protein (alpha helices, beta > sheets...) > seems an obvious new addition and easy enough to do. > > I strongly agree with Steve here that we absolutely must stay > away from ad hoc types. They are the sirens on the rocks: > alluring, but lethal. Regarding ad hoc types, I agree as well. What I meant to say is that we ought to allow non-sequence feature types be served up by DAS/2, and to allow extensions to those types. Basically I'm proposing to have DAS/2 handle non-sequence feature ontologies in the same way that it handles the sequence feature ontology. -Allen From suzi at fruitfly.org Mon Apr 11 21:27:27 2005 From: suzi at fruitfly.org (Suzanna Lewis) Date: Mon, 11 Apr 2005 14:27:27 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: References: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> Message-ID: <285afb885549cab731fa7a6eb5cd1e03@fruitfly.org> On Apr 11, 2005, at 1:33 PM, Allen Day wrote: >>> I'd like to remove the constraint that all types inherit from a type. >>> It >>> makes sense to keep a constraint like this in place for genome >>> sequence >>> features, but not any other record type. > [...] >>> Regarding protein feature types, I think the SO team plans to put >>> them >>> in >>> SO/SOFA since they summarize it as, "a set of terms used to describe >>> features on a nucleotide or protein sequence." But there is no >>> support >>> for >>> proteins in SO/SOFA yet. Suzi? >> >> Yes, that is pretty much where it stands. There is the intention of >> doing this, but we have not yet gotten to that point. Including the >> basic structural descriptions of a protein (alpha helices, beta >> sheets...) >> seems an obvious new addition and easy enough to do. >> >> I strongly agree with Steve here that we absolutely must stay >> away from ad hoc types. They are the sirens on the rocks: >> alluring, but lethal. > > Regarding ad hoc types, I agree as well. What I meant to say is that > we > ought to allow non-sequence feature types be served up by DAS/2, and to > allow extensions to those types. Basically I'm proposing to have DAS/2 > handle non-sequence feature ontologies in the same way that it handles > the > sequence feature ontology. Hi Allen, I don't understand, but maybe I'm muddling GFF3 with DAS/2. In GFF3 clearly everything is sequence-based in one way or another. Perhaps you could provide some scenarios/examples where this would apply in DAS/2? -S From allenday at ucla.edu Mon Apr 11 21:49:57 2005 From: allenday at ucla.edu (Allen Day) Date: Mon, 11 Apr 2005 14:49:57 -0700 (PDT) Subject: [DAS2] modification to /type request & response In-Reply-To: <285afb885549cab731fa7a6eb5cd1e03@fruitfly.org> References: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> <285afb885549cab731fa7a6eb5cd1e03@fruitfly.org> Message-ID: On Mon, 11 Apr 2005, Suzanna Lewis wrote: > > On Apr 11, 2005, at 1:33 PM, Allen Day wrote: > > >>> I'd like to remove the constraint that all types inherit from a type. > >>> It > >>> makes sense to keep a constraint like this in place for genome > >>> sequence > >>> features, but not any other record type. > > [...] > >>> Regarding protein feature types, I think the SO team plans to put > >>> them > >>> in > >>> SO/SOFA since they summarize it as, "a set of terms used to describe > >>> features on a nucleotide or protein sequence." But there is no > >>> support > >>> for > >>> proteins in SO/SOFA yet. Suzi? > >> > >> Yes, that is pretty much where it stands. There is the intention of > >> doing this, but we have not yet gotten to that point. Including the > >> basic structural descriptions of a protein (alpha helices, beta > >> sheets...) > >> seems an obvious new addition and easy enough to do. > >> > >> I strongly agree with Steve here that we absolutely must stay > >> away from ad hoc types. They are the sirens on the rocks: > >> alluring, but lethal. > > > > Regarding ad hoc types, I agree as well. What I meant to say is that > > we > > ought to allow non-sequence feature types be served up by DAS/2, and to > > allow extensions to those types. Basically I'm proposing to have DAS/2 > > handle non-sequence feature ontologies in the same way that it handles > > the > > sequence feature ontology. > > Hi Allen, > > I don't understand, but maybe I'm muddling GFF3 with DAS/2. > In GFF3 clearly everything is sequence-based in one way or > another. > Perhaps you could provide some scenarios/examples where > this would apply in DAS/2? I can't think of anything in genome-DAS that should not have its primary type be a sequence feature type, or some derivative of an existing sequence feature type. An example of where I need to do this is outside the genome service -- in the array-DAS service. I need to be able to attach types to biological source materials that participate in experiments. For example, I might want to annotate a skeletal muscle sample from a morbidly obese human from the mouse anatomical ontology as "skeletal muscle". I'd also attach the type "morbidly obese", which is an extension to the mammalian phenotype ontology type "obese". -Allen From suzi at fruitfly.org Mon Apr 11 21:57:17 2005 From: suzi at fruitfly.org (Suzanna Lewis) Date: Mon, 11 Apr 2005 14:57:17 -0700 Subject: [DAS2] modification to /type request & response In-Reply-To: References: <05c373a5002dfeb0548f5bb11b3dd44e@fruitfly.org> <285afb885549cab731fa7a6eb5cd1e03@fruitfly.org> Message-ID: <0cc393bb37f9d7537fdd3169daf32f81@fruitfly.org> You also might want to check the FMA for human anatomical terms: http://sig.biostr.washington.edu/projects/fm/ I think we will be using it for phenotypes one day (some day...) It would be, like SO, structured and something that could be referenced as the ontology. -S On Apr 11, 2005, at 2:49 PM, Allen Day wrote: > On Mon, 11 Apr 2005, Suzanna Lewis wrote: > >> >> On Apr 11, 2005, at 1:33 PM, Allen Day wrote: >> >>>>> I'd like to remove the constraint that all types inherit from a >>>>> type. >>>>> It >>>>> makes sense to keep a constraint like this in place for genome >>>>> sequence >>>>> features, but not any other record type. >>> [...] >>>>> Regarding protein feature types, I think the SO team plans to put >>>>> them >>>>> in >>>>> SO/SOFA since they summarize it as, "a set of terms used to >>>>> describe >>>>> features on a nucleotide or protein sequence." But there is no >>>>> support >>>>> for >>>>> proteins in SO/SOFA yet. Suzi? >>>> >>>> Yes, that is pretty much where it stands. There is the intention of >>>> doing this, but we have not yet gotten to that point. Including the >>>> basic structural descriptions of a protein (alpha helices, beta >>>> sheets...) >>>> seems an obvious new addition and easy enough to do. >>>> >>>> I strongly agree with Steve here that we absolutely must stay >>>> away from ad hoc types. They are the sirens on the rocks: >>>> alluring, but lethal. >>> >>> Regarding ad hoc types, I agree as well. What I meant to say is that >>> we >>> ought to allow non-sequence feature types be served up by DAS/2, and >>> to >>> allow extensions to those types. Basically I'm proposing to have >>> DAS/2 >>> handle non-sequence feature ontologies in the same way that it >>> handles >>> the >>> sequence feature ontology. >> >> Hi Allen, >> >> I don't understand, but maybe I'm muddling GFF3 with DAS/2. >> In GFF3 clearly everything is sequence-based in one way or >> another. >> Perhaps you could provide some scenarios/examples where >> this would apply in DAS/2? > > I can't think of anything in genome-DAS that should not have its > primary > type be a sequence feature type, or some derivative of an existing > sequence feature type. > > An example of where I need to do this is outside the genome service -- > in > the array-DAS service. I need to be able to attach types to biological > source materials that participate in experiments. For example, I might > want to annotate a skeletal muscle sample from a morbidly obese human > from > the mouse anatomical ontology as "skeletal muscle". I'd also attach > the > type "morbidly obese", which is an extension to the mammalian phenotype > ontology type "obese". > > -Allen From lstein at cshl.edu Tue Apr 12 16:36:26 2005 From: lstein at cshl.edu (Lincoln Stein) Date: Tue, 12 Apr 2005 12:36:26 -0400 Subject: [DAS2] modification to /type request & response In-Reply-To: References: Message-ID: <200504121236.34290.lstein@cshl.edu> Hello, The Types request returns the ontology you are using in the "ontology" attribute of the tag. I think it is fair to insist that if you add a new type to the system you must prepare an ontology that describes it if there isn't one ready-made to inherit from. The /type request is only supposed to return those types which are used. I never expected someone to load the entire sofa database into the thing -- just that subportion of sofa which is actively used to annotate the database objects. Lincoln On Sunday 10 April 2005 12:27 am, Allen Day wrote: > Hi, > > I'm proposing a change to the way type requests are handled. > > Background: According to the spec, a genomic feature may be typed by a > term from the Sequence Ontology Feature Annotation (sofa), or may be > typed by a term from another ontology so long as the term inherits from a > sofa term. > > I have two problems with this. > > First, there may be other records we want to type -- as a concrete > example, let's say we want to type a protein feature as part of > protein-DAS, and the type is "alpha helix", which is not in sofa and may > be difficult to have inherit from an existing sofa type. I have other > real-world examples from the array-DAS document I'm working on, if anyone > cares to see them -- related to biological sample annotation. > > I'd like to remove the constraint that all types inherit from a type. It > makes sense to keep a constraint like this in place for genome sequence > features, but not any other record type. > > Second, making a /type request is, according to the spec, supposed to > return all types known to the server. If we allow non-sofa-inheriting > types into the datasource, the result can be a *very* large document. As > a concrete example, serving up just sofa produces a das2xml 64KB document > containing 243 types. By linear extrapolation, mixing in the Gene > Ontology, which contains 18651 types, will produce a document in excess > of 4.8MB. My DAS server actually contains ~15 ontologies that I could > conceivably want to annotate with. > > I'd like to be able to restrict a /type response to a subset of types > known by the server, perhaps using the following: > > * /das/genome/type > > returns all types known to the server > > * /das/genome/type?namespace=sofa.ontology > > returns all types known to the server under the "sofa.ontology" > namespace > > * /das/genome/type?namespaces > > returns all type namespaces known to the server > > -Allen > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 -- Lincoln D. Stein Cold Spring Harbor Laboratory 1 Bungtown Road Cold Spring Harbor, NY 11724 NOTE: Please copy Sandra Michelsen on all emails regarding scheduling and other time-critical topics. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From lstein at cshl.edu Tue Apr 12 16:38:41 2005 From: lstein at cshl.edu (Lincoln Stein) Date: Tue, 12 Apr 2005 12:38:41 -0400 Subject: [DAS2] modification to /type request & response In-Reply-To: References: <285afb885549cab731fa7a6eb5cd1e03@fruitfly.org> Message-ID: <200504121238.48866.lstein@cshl.edu> The /types?ontology=XXXXXX is a good idea. I'll make that change. Maybe there should be a "brief" form of the /types request which returns the ontology names. Lincoln > > >>> features, but not any other record type. > > > > > > [...] > > > > > >>> Regarding protein feature types, I think the SO team plans to put > > >>> them > > >>> in > > >>> SO/SOFA since they summarize it as, "a set of terms used to > > >>> describe features on a nucleotide or protein sequence." But there > > >>> is no support > > >>> for > > >>> proteins in SO/SOFA yet. Suzi? > > >> > > >> Yes, that is pretty much where it stands. There is the intention of > > >> doing this, but we have not yet gotten to that point. Including the > > >> basic structural descriptions of a protein (alpha helices, beta > > >> sheets...) > > >> seems an obvious new addition and easy enough to do. > > >> > > >> I strongly agree with Steve here that we absolutely must stay > > >> away from ad hoc types. They are the sirens on the rocks: > > >> alluring, but lethal. > > > > > > Regarding ad hoc types, I agree as well. What I meant to say is that > > > we > > > ought to allow non-sequence feature types be served up by DAS/2, and > > > to allow extensions to those types. Basically I'm proposing to have > > > DAS/2 handle non-sequence feature ontologies in the same way that it > > > handles the > > > sequence feature ontology. > > > > Hi Allen, > > > > I don't understand, but maybe I'm muddling GFF3 with DAS/2. > > In GFF3 clearly everything is sequence-based in one way or > > another. > > Perhaps you could provide some scenarios/examples where > > this would apply in DAS/2? > > I can't think of anything in genome-DAS that should not have its primary > type be a sequence feature type, or some derivative of an existing > sequence feature type. > > An example of where I need to do this is outside the genome service -- in > the array-DAS service. I need to be able to attach types to biological > source materials that participate in experiments. For example, I might > want to annotate a skeletal muscle sample from a morbidly obese human > from the mouse anatomical ontology as "skeletal muscle". I'd also attach > the type "morbidly obese", which is an extension to the mammalian > phenotype ontology type "obese". > > -Allen > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 -- Lincoln D. Stein Cold Spring Harbor Laboratory 1 Bungtown Road Cold Spring Harbor, NY 11724 NOTE: Please copy Sandra Michelsen on all emails regarding scheduling and other time-critical topics. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From allenday at ucla.edu Tue Apr 12 17:47:03 2005 From: allenday at ucla.edu (Allen Day) Date: Tue, 12 Apr 2005 10:47:03 -0700 (PDT) Subject: [DAS2] modification to /type request & response In-Reply-To: <200504121236.34290.lstein@cshl.edu> References: <200504121236.34290.lstein@cshl.edu> Message-ID: On Tue, 12 Apr 2005, Lincoln Stein wrote: > Hello, > > The Types request returns the ontology you are using in the "ontology" > attribute of the tag. I think it is fair to insist that if you add > a new type to the system you must prepare an ontology that describes it if > there isn't one ready-made to inherit from. > > The /type request is only supposed to return those types which are used. I > never expected someone to load the entire sofa database into the thing -- > just that subportion of sofa which is actively used to annotate the > database objects. Okay, that makes things easier for the client. Just a small qualifier though -- we should return the types used as well as the union of type graphs *above* those terms. That way the client is aware it can query with a generic type (for instance "region") and get back everything. -Allen > > Lincoln > > On Sunday 10 April 2005 12:27 am, Allen Day wrote: > > Hi, > > > > I'm proposing a change to the way type requests are handled. > > > > Background: According to the spec, a genomic feature may be typed by a > > term from the Sequence Ontology Feature Annotation (sofa), or may be > > typed by a term from another ontology so long as the term inherits from a > > sofa term. > > > > I have two problems with this. > > > > First, there may be other records we want to type -- as a concrete > > example, let's say we want to type a protein feature as part of > > protein-DAS, and the type is "alpha helix", which is not in sofa and may > > be difficult to have inherit from an existing sofa type. I have other > > real-world examples from the array-DAS document I'm working on, if anyone > > cares to see them -- related to biological sample annotation. > > > > I'd like to remove the constraint that all types inherit from a type. It > > makes sense to keep a constraint like this in place for genome sequence > > features, but not any other record type. > > > > Second, making a /type request is, according to the spec, supposed to > > return all types known to the server. If we allow non-sofa-inheriting > > types into the datasource, the result can be a *very* large document. As > > a concrete example, serving up just sofa produces a das2xml 64KB document > > containing 243 types. By linear extrapolation, mixing in the Gene > > Ontology, which contains 18651 types, will produce a document in excess > > of 4.8MB. My DAS server actually contains ~15 ontologies that I could > > conceivably want to annotate with. > > > > I'd like to be able to restrict a /type response to a subset of types > > known by the server, perhaps using the following: > > > > * /das/genome/type > > > > returns all types known to the server > > > > * /das/genome/type?namespace=sofa.ontology > > > > returns all types known to the server under the "sofa.ontology" > > namespace > > > > * /das/genome/type?namespaces > > > > returns all type namespaces known to the server > > > > -Allen > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > > From allenday at ucla.edu Thu Apr 21 05:42:21 2005 From: allenday at ucla.edu (Allen Day) Date: Wed, 20 Apr 2005 22:42:21 -0700 (PDT) Subject: [DAS2] Human prototype server up Message-ID: Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" sequences aligned to the genome w/ > 90% identity. http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region Here's a URL to get you going. This is running on 2 hosts: a dual opteron/8GB webserver frontend and a dual opteron/16GB database backend. There are no other applications on these hosts right now, so hammer away. You might want to use HTTP 1.1 keepalive though, the responses can be *really* slow for /feature requests on large regions. The reason for this is the feature/part subpart requests are done feature by feature due to the 1-many relationship... I'm still trying to figure out how to improve this. Components that aren't working in even a basic way: * alignments * assemblies * individual /feature/ requests * versioned source response w/ supported formats * most feature filters (overlaps, type, and exacttype work) -Allen From Gregg_Helt at affymetrix.com Thu Apr 21 10:28:07 2005 From: Gregg_Helt at affymetrix.com (Helt,Gregg) Date: Thu, 21 Apr 2005 03:28:07 -0700 Subject: [DAS2] Human prototype server up Message-ID: Cool, great to see a public server up and running! However, I'm having trouble with feature queries. For example: http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap s=chr21/21500000:22500000;type=0000110 Which is an example of the most typical query IGB will be making -- one overlap filter combined with one type filter. But this query returns with a single line, no . I've tried it for every type returned by the type query and get the same results. Is there something wrong with my query syntax? If this is not a good region to test, can you send me an example feature query URL where you know what the response should be? thanks! gregg > -----Original Message----- > From: das2-bounces at portal.open-bio.org [mailto:das2-bounces at portal.open- > bio.org] On Behalf Of Allen Day > Sent: Wednesday, April 20, 2005 10:42 PM > To: das2 at portal.open-bio.org > Subject: [DAS2] Human prototype server up > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" sequences > aligned to the genome w/ > 90% identity. > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > Here's a URL to get you going. This is running on 2 hosts: a dual > opteron/8GB webserver frontend and a dual opteron/16GB database backend. > There are no other applications on these hosts right now, so hammer away. > > You might want to use HTTP 1.1 keepalive though, the responses can be > *really* slow for /feature requests on large regions. The reason for this > is the feature/part subpart requests are done feature by feature due to > the 1-many relationship... I'm still trying to figure out how to improve > this. > > Components that aren't working in even a basic way: > > * alignments > * assemblies > * individual /feature/ requests > * versioned source response w/ supported formats > * most feature filters (overlaps, type, and exacttype work) > > -Allen > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 From allenday at ucla.edu Thu Apr 21 17:17:25 2005 From: allenday at ucla.edu (Allen Day) Date: Thu, 21 Apr 2005 10:17:25 -0700 (PDT) Subject: [DAS2] Human prototype server up In-Reply-To: References: Message-ID: This is a good region. I just realized the type ids coming back in the /type response don't match the type ids given in the /feature response. I'm in the process of switching over from using table PKs for types to using type names, so rather than using a filter "type=9770" you can instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string is the term's namespace. SO has already changed it to "SOFA" in their most recent SOFA file in CVS, but I haven't updated yet). So, sorry about that. I just change thed /type response back to give primary keys until I get names working. All the types should now work with the type= filter. Here's a quick document you can grab, a subregion of what you asked for in your request that has something interesting inside: http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlaps=chr21/21292691:21324284;type=9688 -Allen On Thu, 21 Apr 2005, Helt,Gregg wrote: > Cool, great to see a public server up and running! > > However, I'm having trouble with feature queries. For example: > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > s=chr21/21500000:22500000;type=0000110 > > Which is an example of the most typical query IGB will be making -- one > overlap filter combined with one type filter. But this query returns > with a single line, no . I've tried it for > every type returned by the type query and get the same results. > > Is there something wrong with my query syntax? > > If this is not a good region to test, can you send me an example feature > query URL where you know what the response should be? > > thanks! > gregg > > > -----Original Message----- > > From: das2-bounces at portal.open-bio.org > [mailto:das2-bounces at portal.open- > > bio.org] On Behalf Of Allen Day > > Sent: Wednesday, April 20, 2005 10:42 PM > > To: das2 at portal.open-bio.org > > Subject: [DAS2] Human prototype server up > > > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > sequences > > aligned to the genome w/ > 90% identity. > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > > > Here's a URL to get you going. This is running on 2 hosts: a dual > > opteron/8GB webserver frontend and a dual opteron/16GB database > backend. > > There are no other applications on these hosts right now, so hammer > away. > > > > You might want to use HTTP 1.1 keepalive though, the responses can be > > *really* slow for /feature requests on large regions. The reason for > this > > is the feature/part subpart requests are done feature by feature due > to > > the 1-many relationship... I'm still trying to figure out how to > improve > > this. > > > > Components that aren't working in even a basic way: > > > > * alignments > > * assemblies > > * individual /feature/ requests > > * versioned source response w/ supported formats > > * most feature filters (overlaps, type, and exacttype work) > > > > -Allen > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > From allenday at ucla.edu Thu Apr 21 21:30:33 2005 From: allenday at ucla.edu (Allen Day) Date: Thu, 21 Apr 2005 14:30:33 -0700 (PDT) Subject: [DAS2] Human prototype server up In-Reply-To: <4267EBE7.6030603@affymetrix.com> References: <4267E339.6060301@affymetrix.com> <4267E958.30600@affymetrix.com> <4267EBE7.6030603@affymetrix.com> Message-ID: Gregg, Please post your instructions for how to use your server test application. Thanks. -Allen On Thu, 21 Apr 2005, Helt,Gregg wrote: > Cool, great to see a public server up and running! > > However, I'm having trouble with feature queries. For example: > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > s=chr21/21500000:22500000;type=0000110 > > Which is an example of the most typical query IGB will be making -- one > overlap filter combined with one type filter. But this query returns > with a single line, no . I've tried it for > every type returned by the type query and get the same results. > > Is there something wrong with my query syntax? > > If this is not a good region to test, can you send me an example feature > query URL where you know what the response should be? > > thanks! > gregg > > > -----Original Message----- > > From: das2-bounces at portal.open-bio.org > [mailto:das2-bounces at portal.open- > > bio.org] On Behalf Of Allen Day > > Sent: Wednesday, April 20, 2005 10:42 PM > > To: das2 at portal.open-bio.org > > Subject: [DAS2] Human prototype server up > > > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > sequences > > aligned to the genome w/ > 90% identity. > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > > > Here's a URL to get you going. This is running on 2 hosts: a dual > > opteron/8GB webserver frontend and a dual opteron/16GB database > backend. > > There are no other applications on these hosts right now, so hammer > away. > > > > You might want to use HTTP 1.1 keepalive though, the responses can be > > *really* slow for /feature requests on large regions. The reason for > this > > is the feature/part subpart requests are done feature by feature due > to > > the 1-many relationship... I'm still trying to figure out how to > improve > > this. > > > > Components that aren't working in even a basic way: > > > > * alignments > > * assemblies > > * individual /feature/ requests > > * versioned source response w/ supported formats > > * most feature filters (overlaps, type, and exacttype work) > > > > -Allen > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > On Thu, 21 Apr 2005, Allen Day wrote: > This is a good region. I just realized the type ids coming back in the > /type response don't match the type ids given in the /feature response. > I'm in the process of switching over from using table PKs for types to > using type names, so rather than using a filter "type=9770" you can > instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string > is the term's namespace. SO has already changed it to "SOFA" in their > most recent SOFA file in CVS, but I haven't updated yet). > > So, sorry about that. I just change thed /type response back to give > primary keys until I get names working. All the types should now work > with the type= filter. Here's a quick document you can grab, a subregion > of what you asked for in your request that has something interesting > inside: > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlaps=chr21/21292691:21324284;type=9688 > > -Allen > > > > On Thu, 21 Apr 2005, Helt,Gregg wrote: > > > Cool, great to see a public server up and running! > > > > However, I'm having trouble with feature queries. For example: > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > > s=chr21/21500000:22500000;type=0000110 > > > > Which is an example of the most typical query IGB will be making -- one > > overlap filter combined with one type filter. But this query returns > > with a single line, no . I've tried it for > > every type returned by the type query and get the same results. > > > > Is there something wrong with my query syntax? > > > > If this is not a good region to test, can you send me an example feature > > query URL where you know what the response should be? > > > > thanks! > > gregg > > > > > -----Original Message----- > > > From: das2-bounces at portal.open-bio.org > > [mailto:das2-bounces at portal.open- > > > bio.org] On Behalf Of Allen Day > > > Sent: Wednesday, April 20, 2005 10:42 PM > > > To: das2 at portal.open-bio.org > > > Subject: [DAS2] Human prototype server up > > > > > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > sequences > > > aligned to the genome w/ > 90% identity. > > > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > > > > > Here's a URL to get you going. This is running on 2 hosts: a dual > > > opteron/8GB webserver frontend and a dual opteron/16GB database > > backend. > > > There are no other applications on these hosts right now, so hammer > > away. > > > > > > You might want to use HTTP 1.1 keepalive though, the responses can be > > > *really* slow for /feature requests on large regions. The reason for > > this > > > is the feature/part subpart requests are done feature by feature due > > to > > > the 1-many relationship... I'm still trying to figure out how to > > improve > > > this. > > > > > > Components that aren't working in even a basic way: > > > > > > * alignments > > > * assemblies > > > * individual /feature/ requests > > > * versioned source response w/ supported formats > > > * most feature filters (overlaps, type, and exacttype work) > > > > > > -Allen > > > _______________________________________________ > > > DAS2 mailing list > > > DAS2 at portal.open-bio.org > > > http://portal.open-bio.org/mailman/listinfo/das2 > > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 > On Thu, 21 Apr 2005, Ed Erwin wrote: > That response comes back with some "%20" things in it. That seems like > a bug to me, but is it? > > > > > Allen Day wrote: > > This is a good region. I just realized the type ids coming back in the > > /type response don't match the type ids given in the /feature response. > > I'm in the process of switching over from using table PKs for types to > > using type names, so rather than using a filter "type=9770" you can > > instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string > > is the term's namespace. SO has already changed it to "SOFA" in their > > most recent SOFA file in CVS, but I haven't updated yet). > > > > So, sorry about that. I just change thed /type response back to give > > primary keys until I get names working. All the types should now work > > with the type= filter. Here's a quick document you can grab, a subregion > > of what you asked for in your request that has something interesting > > inside: > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlaps=chr21/21292691:21324284;type=9688 > > > > -Allen > > > > > > > > On Thu, 21 Apr 2005, Helt,Gregg wrote: > > > > > >>Cool, great to see a public server up and running! > >> > >>However, I'm having trouble with feature queries. For example: > >>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > >>s=chr21/21500000:22500000;type=0000110 > >> > >>Which is an example of the most typical query IGB will be making -- one > >>overlap filter combined with one type filter. But this query returns > >>with a single line, no . I've tried it for > >>every type returned by the type query and get the same results. > >> > >>Is there something wrong with my query syntax? > >> > >>If this is not a good region to test, can you send me an example feature > >>query URL where you know what the response should be? > >> > >> thanks! > >> gregg > >> > >> > >>>-----Original Message----- > >>>From: das2-bounces at portal.open-bio.org > >> > >>[mailto:das2-bounces at portal.open- > >> > >>>bio.org] On Behalf Of Allen Day > >>>Sent: Wednesday, April 20, 2005 10:42 PM > >>>To: das2 at portal.open-bio.org > >>>Subject: [DAS2] Human prototype server up > >>> > >>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > >> > >>sequences > >> > >>>aligned to the genome w/ > 90% identity. > >>> > >>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > >>> > >>>Here's a URL to get you going. This is running on 2 hosts: a dual > >>>opteron/8GB webserver frontend and a dual opteron/16GB database > >> > >>backend. > >> > >>>There are no other applications on these hosts right now, so hammer > >> > >>away. > >> > >>>You might want to use HTTP 1.1 keepalive though, the responses can be > >>>*really* slow for /feature requests on large regions. The reason for > >> > >>this > >> > >>>is the feature/part subpart requests are done feature by feature due > >> > >>to > >> > >>>the 1-many relationship... I'm still trying to figure out how to > >> > >>improve > >> > >>>this. > >>> > >>>Components that aren't working in even a basic way: > >>> > >>> * alignments > >>> * assemblies > >>> * individual /feature/ requests > >>> * versioned source response w/ supported formats > >>> * most feature filters (overlaps, type, and exacttype work) > >>> > >>>-Allen > >>>_______________________________________________ > >>>DAS2 mailing list > >>>DAS2 at portal.open-bio.org > >>>http://portal.open-bio.org/mailman/listinfo/das2 > >> > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > On Thu, 21 Apr 2005, Ed Erwin wrote: > > This is a question for Andrew Dalke. > > > Allen Day wrote: > > I don't know, is it? I have found the description of properties and the > > /property response document examples to be extremely unclear. I opted to > > URI escape here, but maybe it's not the right thing to do. > > > > -Allen > > > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > > > >>That response comes back with some "%20" things in it. That seems like > >>a bug to me, but is it? > >> > >> > >> > >> > >>Allen Day wrote: > >> > >>>This is a good region. I just realized the type ids coming back in the > >>>/type response don't match the type ids given in the /feature response. > >>>I'm in the process of switching over from using table PKs for types to > >>>using type names, so rather than using a filter "type=9770" you can > >>>instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string > >>>is the term's namespace. SO has already changed it to "SOFA" in their > >>>most recent SOFA file in CVS, but I haven't updated yet). > >>> > >>>So, sorry about that. I just change thed /type response back to give > >>>primary keys until I get names working. All the types should now work > >>>with the type= filter. Here's a quick document you can grab, a subregion > >>>of what you asked for in your request that has something interesting > >>>inside: > >>> > >>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlaps=chr21/21292691:21324284;type=9688 > >>> > >>>-Allen > >>> > >>> > >>> > >>>On Thu, 21 Apr 2005, Helt,Gregg wrote: > >>> > >>> > >>> > >>>>Cool, great to see a public server up and running! > >>>> > >>>>However, I'm having trouble with feature queries. For example: > >>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > >>>>s=chr21/21500000:22500000;type=0000110 > >>>> > >>>>Which is an example of the most typical query IGB will be making -- one > >>>>overlap filter combined with one type filter. But this query returns > >>>>with a single line, no . I've tried it for > >>>>every type returned by the type query and get the same results. > >>>> > >>>>Is there something wrong with my query syntax? > >>>> > >>>>If this is not a good region to test, can you send me an example feature > >>>>query URL where you know what the response should be? > >>>> > >>>> thanks! > >>>> gregg > >>>> > >>>> > >>>> > >>>>>-----Original Message----- > >>>>>From: das2-bounces at portal.open-bio.org > >>>> > >>>>[mailto:das2-bounces at portal.open- > >>>> > >>>> > >>>>>bio.org] On Behalf Of Allen Day > >>>>>Sent: Wednesday, April 20, 2005 10:42 PM > >>>>>To: das2 at portal.open-bio.org > >>>>>Subject: [DAS2] Human prototype server up > >>>>> > >>>>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > >>>> > >>>>sequences > >>>> > >>>> > >>>>>aligned to the genome w/ > 90% identity. > >>>>> > >>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > >>>>> > >>>>>Here's a URL to get you going. This is running on 2 hosts: a dual > >>>>>opteron/8GB webserver frontend and a dual opteron/16GB database > >>>> > >>>>backend. > >>>> > >>>> > >>>>>There are no other applications on these hosts right now, so hammer > >>>> > >>>>away. > >>>> > >>>> > >>>>>You might want to use HTTP 1.1 keepalive though, the responses can be > >>>>>*really* slow for /feature requests on large regions. The reason for > >>>> > >>>>this > >>>> > >>>> > >>>>>is the feature/part subpart requests are done feature by feature due > >>>> > >>>>to > >>>> > >>>> > >>>>>the 1-many relationship... I'm still trying to figure out how to > >>>> > >>>>improve > >>>> > >>>> > >>>>>this. > >>>>> > >>>>>Components that aren't working in even a basic way: > >>>>> > >>>>> * alignments > >>>>> * assemblies > >>>>> * individual /feature/ requests > >>>>> * versioned source response w/ supported formats > >>>>> * most feature filters (overlaps, type, and exacttype work) > >>>>> > >>>>>-Allen > >>>>>_______________________________________________ > >>>>>DAS2 mailing list > >>>>>DAS2 at portal.open-bio.org > >>>>>http://portal.open-bio.org/mailman/listinfo/das2 > >>>> > >>>_______________________________________________ > >>>DAS2 mailing list > >>>DAS2 at portal.open-bio.org > >>>http://portal.open-bio.org/mailman/listinfo/das2 > >> > On Thu, 21 Apr 2005, Ed Erwin wrote: > My reasoning is that it ISN'T a URL yet, so you don't escape it until > you turn it into a URL. But Andrew can figure out which is right and > document it. > > Allen Day wrote: > > exactly. my reasoning was that it was a relative url and needed to be > > escaped. > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > > > >>This is a question for Andrew Dalke. > >> > >> > >>Allen Day wrote: > >> > >>>I don't know, is it? I have found the description of properties and the > >>>/property response document examples to be extremely unclear. I opted to > >>>URI escape here, but maybe it's not the right thing to do. > >>> > >>>-Allen > >>> > >>> > >>>On Thu, 21 Apr 2005, Ed Erwin wrote: > >>> > >>> > >>> > >>>>That response comes back with some "%20" things in it. That seems like > >>>>a bug to me, but is it? > >>>> > >>>> > >>>> > >>>> > >>>>Allen Day wrote: > >>>> > >>>> > >>>>>This is a good region. I just realized the type ids coming back in the > >>>>>/type response don't match the type ids given in the /feature response. > >>>>>I'm in the process of switching over from using table PKs for types to > >>>>>using type names, so rather than using a filter "type=9770" you can > >>>>>instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string > >>>>>is the term's namespace. SO has already changed it to "SOFA" in their > >>>>>most recent SOFA file in CVS, but I haven't updated yet). > >>>>> > >>>>>So, sorry about that. I just change thed /type response back to give > >>>>>primary keys until I get names working. All the types should now work > >>>>>with the type= filter. Here's a quick document you can grab, a subregion > >>>>>of what you asked for in your request that has something interesting > >>>>>inside: > >>>>> > >>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlaps=chr21/21292691:21324284;type=9688 > >>>>> > >>>>>-Allen > >>>>> > >>>>> > >>>>> > >>>>>On Thu, 21 Apr 2005, Helt,Gregg wrote: > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>>Cool, great to see a public server up and running! > >>>>>> > >>>>>>However, I'm having trouble with feature queries. For example: > >>>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > >>>>>>s=chr21/21500000:22500000;type=0000110 > >>>>>> > >>>>>>Which is an example of the most typical query IGB will be making -- one > >>>>>>overlap filter combined with one type filter. But this query returns > >>>>>>with a single line, no . I've tried it for > >>>>>>every type returned by the type query and get the same results. > >>>>>> > >>>>>>Is there something wrong with my query syntax? > >>>>>> > >>>>>>If this is not a good region to test, can you send me an example feature > >>>>>>query URL where you know what the response should be? > >>>>>> > >>>>>> thanks! > >>>>>> gregg > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>>>-----Original Message----- > >>>>>>>From: das2-bounces at portal.open-bio.org > >>>>>> > >>>>>>[mailto:das2-bounces at portal.open- > >>>>>> > >>>>>> > >>>>>> > >>>>>>>bio.org] On Behalf Of Allen Day > >>>>>>>Sent: Wednesday, April 20, 2005 10:42 PM > >>>>>>>To: das2 at portal.open-bio.org > >>>>>>>Subject: [DAS2] Human prototype server up > >>>>>>> > >>>>>>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > >>>>>> > >>>>>>sequences > >>>>>> > >>>>>> > >>>>>> > >>>>>>>aligned to the genome w/ > 90% identity. > >>>>>>> > >>>>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > >>>>>>> > >>>>>>>Here's a URL to get you going. This is running on 2 hosts: a dual > >>>>>>>opteron/8GB webserver frontend and a dual opteron/16GB database > >>>>>> > >>>>>>backend. > >>>>>> > >>>>>> > >>>>>> > >>>>>>>There are no other applications on these hosts right now, so hammer > >>>>>> > >>>>>>away. > >>>>>> > >>>>>> > >>>>>> > >>>>>>>You might want to use HTTP 1.1 keepalive though, the responses can be > >>>>>>>*really* slow for /feature requests on large regions. The reason for > >>>>>> > >>>>>>this > >>>>>> > >>>>>> > >>>>>> > >>>>>>>is the feature/part subpart requests are done feature by feature due > >>>>>> > >>>>>>to > >>>>>> > >>>>>> > >>>>>> > >>>>>>>the 1-many relationship... I'm still trying to figure out how to > >>>>>> > >>>>>>improve > >>>>>> > >>>>>> > >>>>>> > >>>>>>>this. > >>>>>>> > >>>>>>>Components that aren't working in even a basic way: > >>>>>>> > >>>>>>>* alignments > >>>>>>>* assemblies > >>>>>>>* individual /feature/ requests > >>>>>>>* versioned source response w/ supported formats > >>>>>>>* most feature filters (overlaps, type, and exacttype work) > >>>>>>> > >>>>>>>-Allen > >>>>>>>_______________________________________________ > >>>>>>>DAS2 mailing list > >>>>>>>DAS2 at portal.open-bio.org > >>>>>>>http://portal.open-bio.org/mailman/listinfo/das2 > >>>>>> > >>>>>_______________________________________________ > >>>>>DAS2 mailing list > >>>>>DAS2 at portal.open-bio.org > >>>>>http://portal.open-bio.org/mailman/listinfo/das2 > >>>> > From Gregg_Helt at affymetrix.com Thu Apr 21 23:04:10 2005 From: Gregg_Helt at affymetrix.com (Helt,Gregg) Date: Thu, 21 Apr 2005 16:04:10 -0700 Subject: [DAS2] Human prototype server up Message-ID: You need to do an anonymous check-out the genoviz CVS module at http://sourceforge.net/projects/genoviz/ and build the class files. This assumes you have a Java SDK installed, version 1.4 or later. You'll also need the xerces XML parser jar. The test UI I'm working on for the DAS2 client is in the class com.affymetrix.igb.view.Das2LoadView. Once you've compiled the source, you should be able to run it from the command line like this: [java executable] -classpath "[path for genoviz];[path for xerces]" com.affymetrix.igb.view.Das2LoadView On my machine this translates to: java -classpath "c:/projects/genoviz/classes;c:/projects/genoviz/lib/xerces.jar" com.affymetrix.igb.view.Das2LoadView You're presented with a cascading series of choices: DAS2 servers --> DAS2 sources --> DAS2 versions --> DAS2 regions, and your choice triggers the appropriate DAS2 query to populate the next choice in the cascade. Once a version is selected, a DAS2 types query populates a checkbox list of DAS2 types. You can also test feature queries with overlap and type filters by entering the min and max for the overlap filter, selecting which types to filter by, and clicking on "Load Features". This will bring up a rudimentary visualization of the sequence and the feature hierarchies returned by the query. By default it shows the whole length of the sequence, so you'll probably want to zoom in using the slider on the top and the scrollbar on the bottom. This viewer shows feature hierarchies as nested outlined boxes. For client-side data model, there are two additional levels of hierarchy added above the root features -- the parent of the root features is a request feature encompassing all the features returned by a single feature request, and the grandparent is a container for all feature requests of a given type on a single sequence (the model underneath this breaks requests for multiple types into separate request for each type). I just checked interaction of Das2LoadView with DAS2 test server at das.biopackages.net, and it looks like there are still some issues with correct handling of feature child/parent relationships. I'll try and figure out if it's a problem on the client end. thanks, gregg > -----Original Message----- > From: Allen Day [mailto:allenday at ucla.edu] > Sent: Thursday, April 21, 2005 2:31 PM > To: Helt,Gregg; Erwin, Ed > Cc: das2 at portal.open-bio.org; Andrew Dalke > Subject: RE: [DAS2] Human prototype server up > > Gregg, > > Please post your instructions for how to use your server test application. > Thanks. > > -Allen > > On Thu, 21 Apr 2005, Helt,Gregg wrote: > > > Cool, great to see a public server up and running! > > > > However, I'm having trouble with feature queries. For example: > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/feature?overlap > > s=chr21/21500000:22500000;type=0000110 > > > > Which is an example of the most typical query IGB will be making -- one > > overlap filter combined with one type filter. But this query returns > > with a single line, no . I've tried it for > > every type returned by the type query and get the same results. > > > > Is there something wrong with my query syntax? > > > > If this is not a good region to test, can you send me an example feature > > query URL where you know what the response should be? > > > > thanks! > > gregg > > > > > -----Original Message----- > > > From: das2-bounces at portal.open-bio.org > > [mailto:das2-bounces at portal.open- > > > bio.org] On Behalf Of Allen Day > > > Sent: Wednesday, April 20, 2005 10:42 PM > > > To: das2 at portal.open-bio.org > > > Subject: [DAS2] Human prototype server up > > > > > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > sequences > > > aligned to the genome w/ > 90% identity. > > > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > > > > > Here's a URL to get you going. This is running on 2 hosts: a dual > > > opteron/8GB webserver frontend and a dual opteron/16GB database > > backend. > > > There are no other applications on these hosts right now, so hammer > > away. > > > > > > You might want to use HTTP 1.1 keepalive though, the responses can be > > > *really* slow for /feature requests on large regions. The reason for > > this > > > is the feature/part subpart requests are done feature by feature due > > to > > > the 1-many relationship... I'm still trying to figure out how to > > improve > > > this. > > > > > > Components that aren't working in even a basic way: > > > > > > * alignments > > > * assemblies > > > * individual /feature/ requests > > > * versioned source response w/ supported formats > > > * most feature filters (overlaps, type, and exacttype work) > > > > > > -Allen > > > _______________________________________________ > > > DAS2 mailing list > > > DAS2 at portal.open-bio.org > > > http://portal.open-bio.org/mailman/listinfo/das2 > > > > On Thu, 21 Apr 2005, Allen Day wrote: > > > This is a good region. I just realized the type ids coming back in the > > /type response don't match the type ids given in the /feature response. > > I'm in the process of switching over from using table PKs for types to > > using type names, so rather than using a filter "type=9770" you can > > instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" string > > is the term's namespace. SO has already changed it to "SOFA" in their > > most recent SOFA file in CVS, but I haven't updated yet). > > > > So, sorry about that. I just change thed /type response back to give > > primary keys until I get names working. All the types should now work > > with the type= filter. Here's a quick document you can grab, a > subregion > > of what you asked for in your request that has something interesting > > inside: > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlaps=chr21/21292691:21324284;type=9688 > > > > -Allen > > > > > > > > On Thu, 21 Apr 2005, Helt,Gregg wrote: > > > > > Cool, great to see a public server up and running! > > > > > > However, I'm having trouble with feature queries. For example: > > > http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlap > > > s=chr21/21500000:22500000;type=0000110 > > > > > > Which is an example of the most typical query IGB will be making -- > one > > > overlap filter combined with one type filter. But this query returns > > > with a single line, no . I've tried it for > > > every type returned by the type query and get the same results. > > > > > > Is there something wrong with my query syntax? > > > > > > If this is not a good region to test, can you send me an example > feature > > > query URL where you know what the response should be? > > > > > > thanks! > > > gregg > > > > > > > -----Original Message----- > > > > From: das2-bounces at portal.open-bio.org > > > [mailto:das2-bounces at portal.open- > > > > bio.org] On Behalf Of Allen Day > > > > Sent: Wednesday, April 20, 2005 10:42 PM > > > > To: das2 at portal.open-bio.org > > > > Subject: [DAS2] Human prototype server up > > > > > > > > Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > > sequences > > > > aligned to the genome w/ > 90% identity. > > > > > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > > > > > > > Here's a URL to get you going. This is running on 2 hosts: a dual > > > > opteron/8GB webserver frontend and a dual opteron/16GB database > > > backend. > > > > There are no other applications on these hosts right now, so hammer > > > away. > > > > > > > > You might want to use HTTP 1.1 keepalive though, the responses can > be > > > > *really* slow for /feature requests on large regions. The reason > for > > > this > > > > is the feature/part subpart requests are done feature by feature due > > > to > > > > the 1-many relationship... I'm still trying to figure out how to > > > improve > > > > this. > > > > > > > > Components that aren't working in even a basic way: > > > > > > > > * alignments > > > > * assemblies > > > > * individual /feature/ requests > > > > * versioned source response w/ supported formats > > > > * most feature filters (overlaps, type, and exacttype work) > > > > > > > > -Allen > > > > _______________________________________________ > > > > DAS2 mailing list > > > > DAS2 at portal.open-bio.org > > > > http://portal.open-bio.org/mailman/listinfo/das2 > > > > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > That response comes back with some "%20" things in it. That seems like > > a bug to me, but is it? > > > > > > > > > > Allen Day wrote: > > > This is a good region. I just realized the type ids coming back in > the > > > /type response don't match the type ids given in the /feature response. > > > I'm in the process of switching over from using table PKs for types to > > > using type names, so rather than using a filter "type=9770" you can > > > instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" > string > > > is the term's namespace. SO has already changed it to "SOFA" in their > > > most recent SOFA file in CVS, but I haven't updated yet). > > > > > > So, sorry about that. I just change thed /type response back to give > > > primary keys until I get names working. All the types should now work > > > with the type= filter. Here's a quick document you can grab, a > subregion > > > of what you asked for in your request that has something interesting > > > inside: > > > > > > http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlaps=chr21/21292691:21324284;type=9688 > > > > > > -Allen > > > > > > > > > > > > On Thu, 21 Apr 2005, Helt,Gregg wrote: > > > > > > > > >>Cool, great to see a public server up and running! > > >> > > >>However, I'm having trouble with feature queries. For example: > > >>http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlap > > >>s=chr21/21500000:22500000;type=0000110 > > >> > > >>Which is an example of the most typical query IGB will be making -- > one > > >>overlap filter combined with one type filter. But this query returns > > >>with a single line, no . I've tried it for > > >>every type returned by the type query and get the same results. > > >> > > >>Is there something wrong with my query syntax? > > >> > > >>If this is not a good region to test, can you send me an example > feature > > >>query URL where you know what the response should be? > > >> > > >> thanks! > > >> gregg > > >> > > >> > > >>>-----Original Message----- > > >>>From: das2-bounces at portal.open-bio.org > > >> > > >>[mailto:das2-bounces at portal.open- > > >> > > >>>bio.org] On Behalf Of Allen Day > > >>>Sent: Wednesday, April 20, 2005 10:42 PM > > >>>To: das2 at portal.open-bio.org > > >>>Subject: [DAS2] Human prototype server up > > >>> > > >>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > >> > > >>sequences > > >> > > >>>aligned to the genome w/ > 90% identity. > > >>> > > >>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > >>> > > >>>Here's a URL to get you going. This is running on 2 hosts: a dual > > >>>opteron/8GB webserver frontend and a dual opteron/16GB database > > >> > > >>backend. > > >> > > >>>There are no other applications on these hosts right now, so hammer > > >> > > >>away. > > >> > > >>>You might want to use HTTP 1.1 keepalive though, the responses can be > > >>>*really* slow for /feature requests on large regions. The reason for > > >> > > >>this > > >> > > >>>is the feature/part subpart requests are done feature by feature due > > >> > > >>to > > >> > > >>>the 1-many relationship... I'm still trying to figure out how to > > >> > > >>improve > > >> > > >>>this. > > >>> > > >>>Components that aren't working in even a basic way: > > >>> > > >>> * alignments > > >>> * assemblies > > >>> * individual /feature/ requests > > >>> * versioned source response w/ supported formats > > >>> * most feature filters (overlaps, type, and exacttype work) > > >>> > > >>>-Allen > > >>>_______________________________________________ > > >>>DAS2 mailing list > > >>>DAS2 at portal.open-bio.org > > >>>http://portal.open-bio.org/mailman/listinfo/das2 > > >> > > > _______________________________________________ > > > DAS2 mailing list > > > DAS2 at portal.open-bio.org > > > http://portal.open-bio.org/mailman/listinfo/das2 > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > > > This is a question for Andrew Dalke. > > > > > > Allen Day wrote: > > > I don't know, is it? I have found the description of properties and > the > > > /property response document examples to be extremely unclear. I opted > to > > > URI escape here, but maybe it's not the right thing to do. > > > > > > -Allen > > > > > > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > > > > > > >>That response comes back with some "%20" things in it. That seems > like > > >>a bug to me, but is it? > > >> > > >> > > >> > > >> > > >>Allen Day wrote: > > >> > > >>>This is a good region. I just realized the type ids coming back in > the > > >>>/type response don't match the type ids given in the /feature > response. > > >>>I'm in the process of switching over from using table PKs for types > to > > >>>using type names, so rather than using a filter "type=9770" you can > > >>>instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" > string > > >>>is the term's namespace. SO has already changed it to "SOFA" in > their > > >>>most recent SOFA file in CVS, but I haven't updated yet). > > >>> > > >>>So, sorry about that. I just change thed /type response back to give > > >>>primary keys until I get names working. All the types should now > work > > >>>with the type= filter. Here's a quick document you can grab, a > subregion > > >>>of what you asked for in your request that has something interesting > > >>>inside: > > >>> > > >>>http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlaps=chr21/21292691:21324284;type=9688 > > >>> > > >>>-Allen > > >>> > > >>> > > >>> > > >>>On Thu, 21 Apr 2005, Helt,Gregg wrote: > > >>> > > >>> > > >>> > > >>>>Cool, great to see a public server up and running! > > >>>> > > >>>>However, I'm having trouble with feature queries. For example: > > >>>>http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlap > > >>>>s=chr21/21500000:22500000;type=0000110 > > >>>> > > >>>>Which is an example of the most typical query IGB will be making -- > one > > >>>>overlap filter combined with one type filter. But this query > returns > > >>>>with a single line, no . I've tried it for > > >>>>every type returned by the type query and get the same results. > > >>>> > > >>>>Is there something wrong with my query syntax? > > >>>> > > >>>>If this is not a good region to test, can you send me an example > feature > > >>>>query URL where you know what the response should be? > > >>>> > > >>>> thanks! > > >>>> gregg > > >>>> > > >>>> > > >>>> > > >>>>>-----Original Message----- > > >>>>>From: das2-bounces at portal.open-bio.org > > >>>> > > >>>>[mailto:das2-bounces at portal.open- > > >>>> > > >>>> > > >>>>>bio.org] On Behalf Of Allen Day > > >>>>>Sent: Wednesday, April 20, 2005 10:42 PM > > >>>>>To: das2 at portal.open-bio.org > > >>>>>Subject: [DAS2] Human prototype server up > > >>>>> > > >>>>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > >>>> > > >>>>sequences > > >>>> > > >>>> > > >>>>>aligned to the genome w/ > 90% identity. > > >>>>> > > >>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > >>>>> > > >>>>>Here's a URL to get you going. This is running on 2 hosts: a dual > > >>>>>opteron/8GB webserver frontend and a dual opteron/16GB database > > >>>> > > >>>>backend. > > >>>> > > >>>> > > >>>>>There are no other applications on these hosts right now, so hammer > > >>>> > > >>>>away. > > >>>> > > >>>> > > >>>>>You might want to use HTTP 1.1 keepalive though, the responses can > be > > >>>>>*really* slow for /feature requests on large regions. The reason > for > > >>>> > > >>>>this > > >>>> > > >>>> > > >>>>>is the feature/part subpart requests are done feature by feature > due > > >>>> > > >>>>to > > >>>> > > >>>> > > >>>>>the 1-many relationship... I'm still trying to figure out how to > > >>>> > > >>>>improve > > >>>> > > >>>> > > >>>>>this. > > >>>>> > > >>>>>Components that aren't working in even a basic way: > > >>>>> > > >>>>> * alignments > > >>>>> * assemblies > > >>>>> * individual /feature/ requests > > >>>>> * versioned source response w/ supported formats > > >>>>> * most feature filters (overlaps, type, and exacttype work) > > >>>>> > > >>>>>-Allen > > >>>>>_______________________________________________ > > >>>>>DAS2 mailing list > > >>>>>DAS2 at portal.open-bio.org > > >>>>>http://portal.open-bio.org/mailman/listinfo/das2 > > >>>> > > >>>_______________________________________________ > > >>>DAS2 mailing list > > >>>DAS2 at portal.open-bio.org > > >>>http://portal.open-bio.org/mailman/listinfo/das2 > > >> > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > My reasoning is that it ISN'T a URL yet, so you don't escape it until > > you turn it into a URL. But Andrew can figure out which is right and > > document it. > > > > Allen Day wrote: > > > exactly. my reasoning was that it was a relative url and needed to be > > > escaped. > > > > > > On Thu, 21 Apr 2005, Ed Erwin wrote: > > > > > > > > >>This is a question for Andrew Dalke. > > >> > > >> > > >>Allen Day wrote: > > >> > > >>>I don't know, is it? I have found the description of properties and > the > > >>>/property response document examples to be extremely unclear. I > opted to > > >>>URI escape here, but maybe it's not the right thing to do. > > >>> > > >>>-Allen > > >>> > > >>> > > >>>On Thu, 21 Apr 2005, Ed Erwin wrote: > > >>> > > >>> > > >>> > > >>>>That response comes back with some "%20" things in it. That seems > like > > >>>>a bug to me, but is it? > > >>>> > > >>>> > > >>>> > > >>>> > > >>>>Allen Day wrote: > > >>>> > > >>>> > > >>>>>This is a good region. I just realized the type ids coming back in > the > > >>>>>/type response don't match the type ids given in the /feature > response. > > >>>>>I'm in the process of switching over from using table PKs for types > to > > >>>>>using type names, so rather than using a filter "type=9770" you can > > >>>>>instead us a filter "type=sofa.ontology:CDS" (the "sofa.ontology" > string > > >>>>>is the term's namespace. SO has already changed it to "SOFA" in > their > > >>>>>most recent SOFA file in CVS, but I haven't updated yet). > > >>>>> > > >>>>>So, sorry about that. I just change thed /type response back to > give > > >>>>>primary keys until I get names working. All the types should now > work > > >>>>>with the type= filter. Here's a quick document you can grab, a > subregion > > >>>>>of what you asked for in your request that has something > interesting > > >>>>>inside: > > >>>>> > > >>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlaps=chr21/21292691:21324284;type=9688 > > >>>>> > > >>>>>-Allen > > >>>>> > > >>>>> > > >>>>> > > >>>>>On Thu, 21 Apr 2005, Helt,Gregg wrote: > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>>>Cool, great to see a public server up and running! > > >>>>>> > > >>>>>>However, I'm having trouble with feature queries. For example: > > >>>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa- > 17/feature?overlap > > >>>>>>s=chr21/21500000:22500000;type=0000110 > > >>>>>> > > >>>>>>Which is an example of the most typical query IGB will be making - > - one > > >>>>>>overlap filter combined with one type filter. But this query > returns > > >>>>>>with a single line, no . I've tried it > for > > >>>>>>every type returned by the type query and get the same results. > > >>>>>> > > >>>>>>Is there something wrong with my query syntax? > > >>>>>> > > >>>>>>If this is not a good region to test, can you send me an example > feature > > >>>>>>query URL where you know what the response should be? > > >>>>>> > > >>>>>> thanks! > > >>>>>> gregg > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>-----Original Message----- > > >>>>>>>From: das2-bounces at portal.open-bio.org > > >>>>>> > > >>>>>>[mailto:das2-bounces at portal.open- > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>bio.org] On Behalf Of Allen Day > > >>>>>>>Sent: Wednesday, April 20, 2005 10:42 PM > > >>>>>>>To: das2 at portal.open-bio.org > > >>>>>>>Subject: [DAS2] Human prototype server up > > >>>>>>> > > >>>>>>>Contains UCSC Hg17 knownGene, as well as all Affymetrix "target" > > >>>>>> > > >>>>>>sequences > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>aligned to the genome w/ > 90% identity. > > >>>>>>> > > >>>>>>>http://das.biopackages.net/das/genome/chado/chado-Hsa-17/region > > >>>>>>> > > >>>>>>>Here's a URL to get you going. This is running on 2 hosts: a > dual > > >>>>>>>opteron/8GB webserver frontend and a dual opteron/16GB database > > >>>>>> > > >>>>>>backend. > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>There are no other applications on these hosts right now, so > hammer > > >>>>>> > > >>>>>>away. > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>You might want to use HTTP 1.1 keepalive though, the responses > can be > > >>>>>>>*really* slow for /feature requests on large regions. The reason > for > > >>>>>> > > >>>>>>this > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>is the feature/part subpart requests are done feature by feature > due > > >>>>>> > > >>>>>>to > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>the 1-many relationship... I'm still trying to figure out how to > > >>>>>> > > >>>>>>improve > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>>this. > > >>>>>>> > > >>>>>>>Components that aren't working in even a basic way: > > >>>>>>> > > >>>>>>>* alignments > > >>>>>>>* assemblies > > >>>>>>>* individual /feature/ requests > > >>>>>>>* versioned source response w/ supported formats > > >>>>>>>* most feature filters (overlaps, type, and exacttype work) > > >>>>>>> > > >>>>>>>-Allen > > >>>>>>>_______________________________________________ > > >>>>>>>DAS2 mailing list > > >>>>>>>DAS2 at portal.open-bio.org > > >>>>>>>http://portal.open-bio.org/mailman/listinfo/das2 > > >>>>>> > > >>>>>_______________________________________________ > > >>>>>DAS2 mailing list > > >>>>>DAS2 at portal.open-bio.org > > >>>>>http://portal.open-bio.org/mailman/listinfo/das2 > > >>>> > > From dalke at dalkescientific.com Fri Apr 22 03:11:54 2005 From: dalke at dalkescientific.com (Andrew Dalke) Date: Thu, 21 Apr 2005 21:11:54 -0600 Subject: [DAS2] updated DAS/2 spec files Message-ID: <75d8f270590d2a0603e3926340743c15@dalkescientific.com> I've updated the spec in CVS. Here are the changes: - folded in the various changes I brought up last month ("PROPERTY-LIST" -> "PROPERTIES"; RFC dates -> ISO dates; etc.) Lincoln said yes to all the changes. - results of the template file are now saved to "das2_get.html" Given what Steve said today this means the web site should be updated to this version next time the cron is run - added a README describing how the spec is made from the template. Note: You can edit the spec directly as a normal HTML file. The templating language I use is ZopePageTemplates (also called "TAL") which extends HTML only through attributes. This are ignored by browsers so if Gregg wants to modify things, go right ahead. - checked ZopePageTemplates and ElementTree into CVS under the das/das2 directory. These are Python libraries used by the template processor program "convert_template.py" I checked them in so anyone can assemble the template from that directory doing either make or python convert_template.py das2_get_template.py das2_get.py I have verified that you can build the final "das2_get.py" from the template file on the open-bio machine "pub.open-bio.org" using the current setup. NOTE: If you change the RELAX-NG RNC files you'll need to rebuild the RNG and DTD files, and you'll need to configure the TRANG variable in the Makefile to point to your local copy of trang. I did not attempt to make this easy for someone who isn't on a unix machine. On unix the dependencies are rebuilt as part of the normal "make" NOTE: If you want to run the schema validation regression tests you must set the Makefile's JING variable to your local copy of java and jing. Then you can do "make verify" Andrew dalke at dalkescientific.com From Gregg_Helt at affymetrix.com Sat Apr 23 12:11:53 2005 From: Gregg_Helt at affymetrix.com (Helt,Gregg) Date: Sat, 23 Apr 2005 05:11:53 -0700 Subject: [DAS2] updated DAS/2 spec files Message-ID: I really like the links you've added out to RNC, RNG, and DTD schemas after each XML example. Very nice. One problem -- the DTDs aren't parsing as correct XML. The parser in my web browser is demanding that the " -----Original Message----- > From: das2-bounces at portal.open-bio.org [mailto:das2-bounces at portal.open- > bio.org] On Behalf Of Andrew Dalke > Sent: Thursday, April 21, 2005 8:12 PM > To: das2 at portal.open-bio.org > Subject: [DAS2] updated DAS/2 spec files > > I've updated the spec in CVS. Here are the changes: > > - folded in the various changes I brought up last month > ("PROPERTY-LIST" -> "PROPERTIES"; RFC dates -> ISO dates; etc.) > Lincoln said yes to all the changes. > > - results of the template file are now saved to "das2_get.html" > Given what Steve said today this means the web site should be > updated to this version next time the cron is run > > - added a README describing how the spec is made from the > template. > > Note: You can edit the spec directly as a normal HTML file. > The templating language I use is ZopePageTemplates (also > called "TAL") which extends HTML only through attributes. > This are ignored by browsers so if Gregg wants to modify > things, go right ahead. > > - checked ZopePageTemplates and ElementTree into CVS > under the das/das2 directory. These are Python libraries > used by the template processor program "convert_template.py" > I checked them in so anyone can assemble the template from > that directory doing either > > make > > or > > python convert_template.py das2_get_template.py das2_get.py > > I have verified that you can build the final "das2_get.py" > from the template file on the open-bio machine "pub.open-bio.org" > using the current setup. > > > NOTE: If you change the RELAX-NG RNC files you'll need > to rebuild the RNG and DTD files, and you'll need to > configure the TRANG variable in the Makefile to point to > your local copy of trang. I did not attempt to make this > easy for someone who isn't on a unix machine. On unix > the dependencies are rebuilt as part of the normal "make" > > NOTE: If you want to run the schema validation regression > tests you must set the Makefile's JING variable to your > local copy of java and jing. Then you can do "make verify" > > > Andrew > dalke at dalkescientific.com > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 From Gregg_Helt at affymetrix.com Mon Apr 25 16:52:10 2005 From: Gregg_Helt at affymetrix.com (Helt,Gregg) Date: Mon, 25 Apr 2005 09:52:10 -0700 Subject: [DAS2] DAS/2 abstract for BOSC 2005 Message-ID: Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. If you have time, please take a look and let me know if there needs to be any changes. If it's accepted I'll be giving a 20 minute presentation at BOSC. thanks! gregg -------------- next part -------------- A non-text attachment was scrubbed... Name: BOSC_2005_abstract.doc Type: application/msword Size: 32768 bytes Desc: BOSC_2005_abstract.doc URL: From edgrif at sanger.ac.uk Mon Apr 25 17:15:44 2005 From: edgrif at sanger.ac.uk (Ed Griffiths) Date: Mon, 25 Apr 2005 18:15:44 +0100 (BST) Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: References: Message-ID: gregg, > Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. > If you have time, please take a look and let me know if there needs to > be any changes. If it's accepted I'll be giving a 20 minute > presentation at BOSC. A couple of minor things: My name needs an "s": Ed Griffiths "Sanger Centre" should now read "Sanger Institute" otherwise it seems good to me, thanks for doing it. Ed -- ------------------------------------------------------------------------ | Ed Griffiths, Acedb development, Informatics Group, | | Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus, | | Hinxton, Cambridge CB10 1SA, UK | | | | email: edgrif at sanger.ac.uk Tel: +44-1223-494780 Fax: +44-1223-494919 | ------------------------------------------------------------------------ From lstein at cshl.edu Mon Apr 25 22:07:01 2005 From: lstein at cshl.edu (Lincoln Stein) Date: Mon, 25 Apr 2005 18:07:01 -0400 Subject: [DAS2] updated DAS/2 spec files In-Reply-To: <75d8f270590d2a0603e3926340743c15@dalkescientific.com> References: <75d8f270590d2a0603e3926340743c15@dalkescientific.com> Message-ID: <200504251807.04241.lstein@cshl.edu> Hi, I've done a chunk of work on the das2_put.html update document. I would be happy to migrate this over to Andrew's templating system, but I'd like to know where I can get the TRANG and JING jar files so that I can run "make." Is there anything else I need for the Python part of the conversion? Lincoln On Thursday 21 April 2005 11:11 pm, Andrew Dalke wrote: > I've updated the spec in CVS. Here are the changes: > > - folded in the various changes I brought up last month > ("PROPERTY-LIST" -> "PROPERTIES"; RFC dates -> ISO dates; etc.) > Lincoln said yes to all the changes. > > - results of the template file are now saved to "das2_get.html" > Given what Steve said today this means the web site should be > updated to this version next time the cron is run > > - added a README describing how the spec is made from the > template. > > Note: You can edit the spec directly as a normal HTML file. > The templating language I use is ZopePageTemplates (also > called "TAL") which extends HTML only through attributes. > This are ignored by browsers so if Gregg wants to modify > things, go right ahead. > > - checked ZopePageTemplates and ElementTree into CVS > under the das/das2 directory. These are Python libraries > used by the template processor program "convert_template.py" > I checked them in so anyone can assemble the template from > that directory doing either > > make > > or > > python convert_template.py das2_get_template.py das2_get.py > > I have verified that you can build the final "das2_get.py" > from the template file on the open-bio machine "pub.open-bio.org" > using the current setup. > > > NOTE: If you change the RELAX-NG RNC files you'll need > to rebuild the RNG and DTD files, and you'll need to > configure the TRANG variable in the Makefile to point to > your local copy of trang. I did not attempt to make this > easy for someone who isn't on a unix machine. On unix > the dependencies are rebuilt as part of the normal "make" > > NOTE: If you want to run the schema validation regression > tests you must set the Makefile's JING variable to your > local copy of java and jing. Then you can do "make verify" > > > Andrew > dalke at dalkescientific.com > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 -- Lincoln D. Stein Cold Spring Harbor Laboratory 1 Bungtown Road Cold Spring Harbor, NY 11724 NOTE: Please copy Sandra Michelsen on all emails regarding scheduling and other time-critical topics. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From Steve_Chervitz at affymetrix.com Tue Apr 26 00:43:39 2005 From: Steve_Chervitz at affymetrix.com (Chervitz, Steve) Date: Mon, 25 Apr 2005 17:43:39 -0700 Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: Message-ID: Looks good to me. Here's one minor change for the second to last sentence: "Client, server, and validation suite are all available as open source." to "Client, server, and validation suite are all available under open source licenses." (and all different ones at that!) (the parenthetical part is optional ;-). Steve > From: "Helt,Gregg" > Date: Mon, 25 Apr 2005 09:52:10 -0700 > To: > Subject: [DAS2] DAS/2 abstract for BOSC 2005 > > Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. > If you have time, please take a look and let me know if there needs to > be any changes. If it's accepted I'll be giving a 20 minute > presentation at BOSC. > > thanks! > gregg > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 From allenday at ucla.edu Tue Apr 26 03:26:45 2005 From: allenday at ucla.edu (Allen Day) Date: Mon, 25 Apr 2005 20:26:45 -0700 (PDT) Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: References: Message-ID: I'm willing to change my license to match others if it makes life easier for someone. -Allen On Mon, 25 Apr 2005, Chervitz, Steve wrote: > Looks good to me. Here's one minor change for the second to last sentence: > > "Client, server, and validation suite are all available as open source." > > to > > "Client, server, and validation suite are all available under open source > licenses." (and all different ones at that!) > > (the parenthetical part is optional ;-). > > Steve > > > > From: "Helt,Gregg" > > Date: Mon, 25 Apr 2005 09:52:10 -0700 > > To: > > Subject: [DAS2] DAS/2 abstract for BOSC 2005 > > > > Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. > > If you have time, please take a look and let me know if there needs to > > be any changes. If it's accepted I'll be giving a 20 minute > > presentation at BOSC. > > > > thanks! > > gregg > > > > _______________________________________________ > > DAS2 mailing list > > DAS2 at portal.open-bio.org > > http://portal.open-bio.org/mailman/listinfo/das2 > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 > From Steve_Chervitz at affymetrix.com Tue Apr 26 19:47:46 2005 From: Steve_Chervitz at affymetrix.com (Chervitz, Steve) Date: Tue, 26 Apr 2005 12:47:46 -0700 Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: Message-ID: I think it is something worth discussing. Having multiple licenses runs the risk of incompatibilities and could raise the barrier for involvement from the corporate world, where it would be easier for the legal department to evaluate a single license rather than multiple licenses before giving the OK to their developers. Steve > From: Allen Day > Date: Mon, 25 Apr 2005 20:26:45 -0700 (PDT) > To: "Chervitz, Steve" > Cc: Gregg Helt , > Subject: Re: [DAS2] DAS/2 abstract for BOSC 2005 > > I'm willing to change my license to match others if it makes life easier > for someone. > > -Allen > > On Mon, 25 Apr 2005, Chervitz, Steve wrote: > >> Looks good to me. Here's one minor change for the second to last sentence: >> >> "Client, server, and validation suite are all available as open source." >> >> to >> >> "Client, server, and validation suite are all available under open source >> licenses." (and all different ones at that!) >> >> (the parenthetical part is optional ;-). >> >> Steve >> >> >>> From: "Helt,Gregg" >>> Date: Mon, 25 Apr 2005 09:52:10 -0700 >>> To: >>> Subject: [DAS2] DAS/2 abstract for BOSC 2005 >>> >>> Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. >>> If you have time, please take a look and let me know if there needs to >>> be any changes. If it's accepted I'll be giving a 20 minute >>> presentation at BOSC. >>> >>> thanks! >>> gregg >>> >>> _______________________________________________ >>> DAS2 mailing list >>> DAS2 at portal.open-bio.org >>> http://portal.open-bio.org/mailman/listinfo/das2 >> >> _______________________________________________ >> DAS2 mailing list >> DAS2 at portal.open-bio.org >> http://portal.open-bio.org/mailman/listinfo/das2 >> From edgrif at sanger.ac.uk Tue Apr 26 19:50:39 2005 From: edgrif at sanger.ac.uk (Ed Griffiths) Date: Tue, 26 Apr 2005 20:50:39 +0100 (BST) Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: References: Message-ID: > I think it is something worth discussing. Having multiple licenses runs the > risk of incompatibilities and could raise the barrier for involvement from > the corporate world, where it would be easier for the legal department to > evaluate a single license rather than multiple licenses before giving the OK > to their developers. I'd just like to add that as someone who worked for IBM for 14 years I can tell you that this is undoubtedly the case ;-) Ed > > Steve > > > > From: Allen Day > > Date: Mon, 25 Apr 2005 20:26:45 -0700 (PDT) > > To: "Chervitz, Steve" > > Cc: Gregg Helt , > > Subject: Re: [DAS2] DAS/2 abstract for BOSC 2005 > > > > I'm willing to change my license to match others if it makes life easier > > for someone. > > > > -Allen > > > > On Mon, 25 Apr 2005, Chervitz, Steve wrote: > > > >> Looks good to me. Here's one minor change for the second to last sentence: > >> > >> "Client, server, and validation suite are all available as open source." > >> > >> to > >> > >> "Client, server, and validation suite are all available under open source > >> licenses." (and all different ones at that!) > >> > >> (the parenthetical part is optional ;-). > >> > >> Steve > >> > >> > >>> From: "Helt,Gregg" > >>> Date: Mon, 25 Apr 2005 09:52:10 -0700 > >>> To: > >>> Subject: [DAS2] DAS/2 abstract for BOSC 2005 > >>> > >>> Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. > >>> If you have time, please take a look and let me know if there needs to > >>> be any changes. If it's accepted I'll be giving a 20 minute > >>> presentation at BOSC. > >>> > >>> thanks! > >>> gregg > >>> > >>> _______________________________________________ > >>> DAS2 mailing list > >>> DAS2 at portal.open-bio.org > >>> http://portal.open-bio.org/mailman/listinfo/das2 > >> > >> _______________________________________________ > >> DAS2 mailing list > >> DAS2 at portal.open-bio.org > >> http://portal.open-bio.org/mailman/listinfo/das2 > >> > > _______________________________________________ > DAS2 mailing list > DAS2 at portal.open-bio.org > http://portal.open-bio.org/mailman/listinfo/das2 > > > -- ------------------------------------------------------------------------ | Ed Griffiths, Acedb development, Informatics Group, | | Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus, | | Hinxton, Cambridge CB10 1SA, UK | | | | email: edgrif at sanger.ac.uk Tel: +44-1223-494780 Fax: +44-1223-494919 | ------------------------------------------------------------------------ From lstein at cshl.edu Tue Apr 26 21:15:59 2005 From: lstein at cshl.edu (Lincoln Stein) Date: Tue, 26 Apr 2005 17:15:59 -0400 Subject: [DAS2] DAS/2 abstract for BOSC 2005 In-Reply-To: References: Message-ID: <200504261716.00047.lstein@cshl.edu> Hi Gregg, It looks great, aside from the little things that others noticed earlier. Lincoln On Monday 25 April 2005 12:52 pm, Helt,Gregg wrote: > Attached is the abstract I'm planning to submit tomorrow for BOSC 2005. > If you have time, please take a look and let me know if there needs to > be any changes. If it's accepted I'll be giving a 20 minute > presentation at BOSC. > > thanks! > gregg -- Lincoln D. Stein Cold Spring Harbor Laboratory 1 Bungtown Road Cold Spring Harbor, NY 11724 NOTE: Please copy Sandra Michelsen on all emails regarding scheduling and other time-critical topics. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From edgrif at sanger.ac.uk Wed Apr 27 15:06:25 2005 From: edgrif at sanger.ac.uk (Ed Griffiths) Date: Wed, 27 Apr 2005 16:06:25 +0100 (BST) Subject: [DAS2] jing, trang, python...the world... In-Reply-To: <45f24ce628766b59916de4e4a5ec05dc@dalkescientific.com> References: <4255C847.3090104@affymetrix.com> <45f24ce628766b59916de4e4a5ec05dc@dalkescientific.com> Message-ID: Andrew, I've got jing and trang and installed them and as far as I can tell they seem to be doing something....python is another matter though: cd /Users/edgrif/BIOINFORMATICS/das/das2/ make python ./convert_template.py das2_get_template.html das2_get.html Traceback (most recent call last): File "./convert_template.py", line 123, in ? main(sys.argv) File "./convert_template.py", line 120, in main open(output_filename, "w").write(pt(context=context)) File "./convert_template.py", line 34, in __call__ return self.pt_render(extra_context=context) File "/Users/edgrif/BIOINFORMATICS/das/das2/ZopePageTemplates/__init__.py", line 86, in pt_render raise PTRuntimeError, 'Page Template %s has errors.' % self.id ZopePageTemplates.PTRuntimeError: Page Template (unknown) has errors. make: *** [das2_get.html] Error 1 Compilation exited abnormally with code 2 at Wed Apr 27 16:02:01 Is this something that you came across, can you give any clues ? Is it the input to python that is mangled or is it a problem with the python code ? Ed -- ------------------------------------------------------------------------ | Ed Griffiths, Acedb development, Informatics Group, | | Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus, | | Hinxton, Cambridge CB10 1SA, UK | | | | email: edgrif at sanger.ac.uk Tel: +44-1223-494780 Fax: +44-1223-494919 | ------------------------------------------------------------------------ From Gregg_Helt at affymetrix.com Thu Apr 28 18:16:35 2005 From: Gregg_Helt at affymetrix.com (Helt,Gregg) Date: Thu, 28 Apr 2005 11:16:35 -0700 Subject: [DAS2] DAS/2 UML Class Diagram Message-ID: I've attached a jpeg of my latest attempt at making UML class diagrams based on the DAS/2 spec. I sent an earlier attempt a few weeks ago, but I think somehow the mailing list filters ate it. The zuml file (zipped up xmi) this is based on is checked into the genoviz sourceforge repository as documentation/GenometryWithDas.zuml. There's a lot of my "genometry" data modeling in the xmi, but the abstract DAS/2 model doesn't refer to the genometry stuff - I wanted to have an abstract model that doesn't depend on other stuff. My actual DAS/2 client implemention is a tighter integration between DAS and genometry models. Anyway, I've found having this diagram very useful when thinking about the spec. For example two of the spec changes I've been thinking about recently, removing the element from the versioned source info and adding a tag for types, I've highlighted in the model in red and green, respectively. If you have a chance please take a look at the diagram and let me know if anything looks wrong relative to the spec. gregg -------------- next part -------------- A non-text attachment was scrubbed... Name: Das2AbstractAPI.JPG Type: image/jpeg Size: 103971 bytes Desc: Das2AbstractAPI.JPG URL: From edgrif at sanger.ac.uk Thu Apr 28 19:15:23 2005 From: edgrif at sanger.ac.uk (Ed Griffiths) Date: Thu, 28 Apr 2005 20:15:23 +0100 (BST) Subject: [DAS2] das docs updates... In-Reply-To: References: Message-ID: All, while I've been over here with Lincoln working on the das2 write back spec I've followed Andrews good example and have put some new template style files into the repository. There are now 3 das2 spec files: das2_client_template.html das2_get_template.html das2_put_template.html These files get munged by a process set up by Andrew via a makefile into the actual spec docs: das2_client.html das2_get.html das2_put.html There are a couple of points to note here if you decide to alter the files: 1) You will need to get hold trang and jing (easy to find/download on the web) 2) You will need to edit the Makefiles first couple of lines to point at where you installed trang/jing on your system 3) You need to edit the files that are inputs, not the output files (!), e.g. you should edit das2_put_template.html, _not_ das2_put.html !! 4) You should also be aware that trang/jing/python scripts used to make the final spec docs are sensitive to bad html, you may find the html checking program "tiny" helpful in finding problems (again easily found/downloaded on the web). 5) If you want to read the specs then read the final versions: das2_client.html das2_get.html das2_put.html 6) The output files are currently stored in cvs, this is probably a good idea in fact otherwise we will all have to set up the environment to make them. Please do make comments/corrections etc., I expect I will be adding more over today and tomorrow but then its back off to "sunny" hinxton for acedb/zmap the world... Ed -- ------------------------------------------------------------------------ | Ed Griffiths, Acedb development, Informatics Group, | | Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus, | | Hinxton, Cambridge CB10 1SA, UK | | | | email: edgrif at sanger.ac.uk Tel: +44-1223-494780 Fax: +44-1223-494919 | ------------------------------------------------------------------------ From allenday at ucla.edu Sat Apr 30 07:58:28 2005 From: allenday at ucla.edu (Allen Day) Date: Sat, 30 Apr 2005 00:58:28 -0700 (PDT) Subject: [DAS2] alteration to /type response In-Reply-To: References: Message-ID: Hi, I'd like to have the element contain 0..N subelements. Consider the following subgraph of SOFA, described as das2xml: Showing the types/supertypes available in this way allows the client to infer that a request for "feature?type=SO:mRNA" is going to return both "SO:mRNA" and "SO:CDS" records -- even if the "SO:CDS" are not directly attached to a "SO:mRNA" parent (review "type" vs. "exacttype" feature filters to understand why). This information would also be useful for rendering a tree widget in a client for selecting types of interest. Also, for the "type" feature filter, we may only want to traverse "OBO_REL:is_a" edges and not "OBO_REL:part_of" edges, or other edge types such as "OBO_REL:adjacent_to"... this would prevent the server from returning "SO:CDS" features when "SO:mRNA" is requested, but not prevent the return of "SO:polycistronic_mRNA" features, because "SO:polycistronic_mRNA" "OBO_REL:is_a" "SO:mRNA". -Allen From allenday at ucla.edu Sat Apr 30 08:35:26 2005 From: allenday at ucla.edu (Allen Day) Date: Sat, 30 Apr 2005 01:35:26 -0700 (PDT) Subject: [DAS2] Human prototype server up In-Reply-To: References: Message-ID: More feature filters are now available. * contains * identical * inside * name Retrieve all tropomyosins and receptor tyrosine kinases by gene symbol: http://radius.genomics.ctrl.ucla.edu/das/genome/chado/chado-Hsa-17/feature?name=TPM*,*RTK* I'll probably have the "att" filter working tomorrow -- will post here again if so. -Allen