From Steve_Chervitz at affymetrix.com  Wed Dec  1 18:54:43 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Wed, 01 Dec 2004 15:54:43 -0800
Subject: [DAS2] New DAS2 list
In-Reply-To: <BDCA7B22.26FD%Steve_Chervitz@affymetrix.com>
Message-ID: <BDD399C3.298E%Steve_Chervitz@affymetrix.com>

As promised, we now have a discussion list for just the DAS/2 grant crew.
You should have received a welcome message from mailman already.
This list is unmoderated and will reject posts from non-members. I left the
archive public, so others could snoop if interested (and can guess the url).

Lincoln: How much overhead is moderation requiring for the original das
mailing list? Chris D made some improvements in the spam filters and I asked
if he thought they were good enough to go off-moderation. Here's what he
said:

> Our spam protection is pretty good but the stuff that leaks through is
> really hard to catch -- simple ASCII emails etc. that breeze right
> through our tests.
> 
> The current moderator web interface has a great new feature called
> "delete any messages marked as "defer" -- this lets you look at the
> moderation list and pick out only the good messages while not wasting
> time hitting checkboxes for spam. I'd recommend leaving it as is for now
> unnless you want to open the list up for a while as an experiment.

Steve


> From: <Chervitz>, Steve <Steve_Chervitz at affymetrix.com>
> Date: Wed, 24 Nov 2004 17:53:38 -0800
> Subject: Re: DAS 2 spec
> 
> Ed Griffiths <edgrif at sanger.ac.uk> wrote:
>> 
>> Hi all, 
>> 
>> Well its time to get back to doing something about the DAS2 spec for me...
>>  
>> a couple of questions......
>> 
>> where should discussion about issues take place ?
> 
> I'm working to get a mailman list set up on biodas.org that would include only
> the folks on the grant. Hopefully, this will happen soon. I'll let you know.
> I'm also looking into restricting the public biodas.org mailing list so that
> only registered users can post, to reduce junk and moderation overhead.
> 
> There may not be any action on this until next week since the US is having a
> major holiday (Thanksgiving) for the rest of this week.
>  
>> where is the current copy of the DAS2 spec ?
> 
> You can get it from the biodas cvs with these commands:
> 
> $ cvs -d :pserver:cvs at cvs.open-bio.org:/home/repository/biodas login
> password=cvs
> 
> $ cvs -d :pserver:cvs at cvs.open-bio.org:/home/repository/biodas co das
> 
> The spec is at das/das2/das2_protocol.html. The .txt version is not up to date
> (Lincoln: how about removing it to avoid confusion?). I will look into setting
> up a web page on biodas.org pointing at the latest version of
> das/das2/das2_protocol.html.
> 
> Steve
> 
> --
> Steve Chervitz | Bioinformatics Engineer | Software Development
> AFFYMETRIX, INC. | 6550 Vallejo St., Ste 100 | Emeryville, CA 94608
> Tel: 510-428-8530 | Fax: 510-428-8585 | Steve_Chervitz at affymetrix.com
> 



From Steve_Chervitz at affymetrix.com  Mon Dec 13 18:01:06 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Mon, 13 Dec 2004 15:01:06 -0800
Subject: [DAS2] September discussions
Message-ID: <BDE35F32.2F12%Steve_Chervitz@affymetrix.com>

I'm going to post some email discussions that took place during the Hinxton
Genome Informatics meeting in September, so they can be archived on this
list. Stay tuned.

Take a look at what I send, and if there are other relevant discussions,
please send them along.

Steve



From Steve_Chervitz at affymetrix.com  Mon Dec 13 20:23:05 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:23:05 -0800
Subject: [DAS2] Fwd: DAS2 server registry and discovery meeting,
	Friday at 10 AM
Message-ID: <B46F9150-4D6E-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Andrew Dalke <dalke at dalkescientific.com>
Date: September 23, 2004 9:49:59 PM PDT
Subject: Re: DAS2 server registry and discovery meeting, Friday at 10 AM

Steve:
> It would be helpful if someone could circulate notes summarizing
> the discussions, conclusions reached, and any action items (especially
> for us West-coast, non-morning types ;-).  Thanks!

Ughh!  I made it for the 10am BST conference call two days ago.
I'm still recovering ... :)

Here's my summary of the two conference calls.

The first day we went over the draft version of the protocol.
We made some minor changes, mostly to clear things up and fix
typos.  The biggest changes were in the document that returns
information about a data source, and we're getting rid of the
'sequences' section, to use 'region' instead.  (The document
had a bit of both.)

I think the only action item there is that Lincoln's going to
update the spec and redistribute it and I'll go through it with
my fine toothed comb.

This morning (my time) we talked about updates.  There are
many ways to do it.  One is via WebDAV, but none of us really
have any experience with it.  I've read the spec which made
me the most knowledgeable.  The thing about DAV is that it
requires several new HTTP level commands, like LOCK and
PROPFIND.  DAV seems to provide mechanism but not policy
and without some experience we decided to pass on it for
this version.

I personally would like to encourage people in the community
to experiment with that technology.

We talked about various ways to send updates, starting
with how people do it now.  I didn't follow this part well
enough to report it now but it seems that the client sends
basically a diff as one document.

The next topic was locking.  We can support optimistic
(CVS-style) or pessimistic (RCS-syle) locks.  In other words,
don't lock until it's time to commit changes vs. lock when
making the first change.  CVS-style is interesting, but
it makes lock conflicts harder to deal with.  We also
talked about tracking all change (just like a full-blown
version control system would) but decided again it was
too much to deal with now.

Though again I would like to encourage people to experiment
with, say, SVN as part of a back-end versioning system
layered on top of WebDAV.  The tricky parts are maintaining
internal consistency of all the independent branches and
handling branch merges in both client and server.

The conclusions in the locking discussion are:
   - pessimistic locking on a region specified by the curator.
      We will not support locks on a specific feature, feature
      type, etc.

   - the lock request is POSTed to a new URL, listed in the
       data source document and probably named something like
        .../lock/new

   - the request will include things like name (or is that
       provided via some other authentication means?), region
       to lock, and time for lock

   - if no error, the return include the lock URL (probably
       something like ".../lock/lock12345"), and length of
       time actually granted

   - a GET on the lock URL returns info about who locked it,
       the time remaining for the lock, and region

   - a POST of some sort to the URL is used to renew the lock,
       commit the changes or revert the transaction.  We think
       it should also be able to expand the locked region.

   - the data source document will also include a URL
       used to get the list of current locks

   - breaking locks is outside the scope of the spec.  Eg,
       it might only be done by people who can log into
       the database system.

There was also some discussion about how a client might
show to the curator which elements can be edited.  Currently
that's known through extra-protocol means (ie, by talking
to people and reading the documentation).  It might be
nice if the client tool could make some sort of indication.

We have not yet discussed how we're going to do the actual
editing.  It might be by PUT/DELETE calls on the elements,
or through a unified diff POSTed to the server which contains
a list of all the changes.

(Which reminds me, the DAV spec also support the HTTP
operations of MOVE and COPY.  These allow the versioning
system to keep version information even after splits and
renames.)

Did I omit or misstate anything?

					Andrew
					dalke at dalkescientific.com



From Steve_Chervitz at affymetrix.com  Mon Dec 13 20:33:38 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:33:38 -0800
Subject: [DAS2] Fwd: DAS2 meeting today (Sunday), 12:30-2:30
Message-ID: <2D91751F-4D70-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: "Helt,Gregg" <Gregg_Helt at affymetrix.com>
Date: September 25, 2004 11:55:45 PM PDT
Subject: DAS2 meeting today (Sunday), 12:30-2:30

Today we're having another in the ongoing series of DAS/2 meetings, from
12:30-2:30 PM in the Loft Room at the Wellcome Trust Conference Center.
I think we'll be covering a variety of topics with regard to the
specification, so BYOT (Bring Your Own Topic).

What I'm pretty sure we we will _not_ be covering is:
         registry and discovery of DAS/2 servers (already covered in
Saturday meeting, need to process results of that before discussing
again)
         writeback from DAS/2 clients to DAS/2 servers (there will be a
meeting Monday on this)
         implementation details of DAS/2 clients, servers, validation
suite (I want to keep focus on spec and logistics)

Topics I'd personally like to cover:
     Spec overview:
         General coherence
         Complexity comparison to DAS/1 and other alternatives
         Technology choices -- I'd like to have some coherent strategy
for explaining DAS/2 technology choices to others
                 For example, the decision to use URLs for most
identifiers rather than LSIDs
     Logistics:
         Timeline for meeting goals of grant
         Scheduling of next face-to-face meeting, as well as regular
teleconferences
         Summarize division of labor
         Getting feedback from bioinformatics community

Attendance is optional, but decisions _are_ getting made at these
meetings...

         thanks,
         gregg



From Steve_Chervitz at affymetrix.com  Mon Dec 13 20:34:37 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:34:37 -0800
Subject: [DAS2] Fwd: DAS2 meeting today (Sunday), 12:30-2:30
Message-ID: <50F49F32-4D70-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Andrew Dalke <dalke at dalkescientific.com>
Date: September 26, 2004 2:09:12 AM PDT
Subject: Re: DAS2 meeting today (Sunday), 12:30-2:30

BTW, when is Monday's meeting?

Gregg:
> Topics I'd personally like to cover:
> ????Spec overview:
> ????????General coherence

I found the spec somewhat confusing to read.  As written there are two
parts to the spec -- one an overview and the other the details of the
request.  But some of the details (like the use of /0 to mean the 
current
database, which I still don't like ;) are in the first part but not in
the second.  So there isn't one place to read to learn everything about
a given interface.

I would rather the overview section be much shorter and not contain
specification information.  I would also like it to include some
concrete information to show how it exists for a (hypothetical)
database.

   $entry                      -- main entry point to a DAS/2 server
   http://www.biodas.org/das2/das-genome

   $entry/$sourceid            -- data about a given source
   http://www.biodas.org/das2/das-genome/volvox

   $entry/$sourceid/$version  -- data about a given version of the source
   http://www.biodas.org/das2/das-genome/volvox/1

   $entry/$sourceid/$version/type -- description of various types of 
genomic features
   http://www.biodas.org/das2/das-genome/volvox/1

    ...

and perhaps with an abbreviation like $db = $entry/$sourceid/$version

The final version should support in-document hyperlinks to
the actual section.  That's a detail understandably left out
of the current version.

One topic I would like to revisit is the use of DTDs.  My
DAS/1 and my NCBI EUtils client both used DTDs to make parsers
for the data returned from the server.  That was part of
what made the client code easy to implement.  The problems
I ran into were 1) the old DTDs weren't correct, 2) DTDs
don't support non-text fields (like start/stop sequence
positions) and 3) the DTD-based parser I used was validating
and I ran into problems dealing with extensions and other
variances from the spec.

I want to investigate the use of RelaxNG instead.  That
supports datatypes, to say that the sequence start/stop
positions can be longs (64 bit integers).  It should help
handle these non-text fields better, so better for Java
people.  I might help with my validation. And I see there's
a way to annotate fields so it might be possible to
convert from a RelaxNG schema directly into documentation.

That may slide too much into implementation details.  I'll
restrict to the question "should the spec declare that DTDs
are going to be provided and will be considered normative?"


> ????????Complexity comparison to?DAS/1 and other alternatives

I only wrote a client to it, not a server.

Looking over my DAS/1 client code, I'm happy to see that we
no longer have "X-DAS-Capabilities".  My code basically ignored
it.  It also ignored the X-DAS-Version because just about
every server seemed to return a different version number.

The client-side implementation was pretty easy (except that
servers didn't meet the spec and several didn't even return
valid XML).

I don't know about other alternatives.

One of the fuzzy places in the spec is how to deal with issues
related to the underlying HTTP protocol.  I see that several
places didn't implement stylesheet so the HTTP request ended
up with a '404 Not Found' instead of using the DAS error code.

I see the DAS/1 spec talked about being able to leverage
things like content-encoding for sending compressed data back
and forth.  DAS/2 doesn't yet mention that.

> ????????Technology?choices -- I'd like to have some coherent
> ?strategy for explaining DAS/2 technology choices to others

What I just mentioned about interactions with HTTP might go
in this document.  I would also like to see comments like
"we considered WebDAV but because of its complexity and our
lack of experience ... " and "this locking scheme was chosen
because it is an easy one to conceptualize and implement.
Better possibilities exist, including ideas in source code
version control systems like CVS, subversion, and arch.
We hope people experiment with them to get experience that
may influence the next generation of DAS-like tools."

Okay, time for a 2.5hr nap before the meeting.  *yawn*


					Andrew
					dalke at dalkescientific.com




From Steve_Chervitz at affymetrix.com  Mon Dec 13 20:35:33 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:35:33 -0800
Subject: [DAS2] Fwd: thoughts on DAS/2 validation
Message-ID: <724755F0-4D70-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Andrew Dalke <dalke at dalkescientific.com>
Date: September 26, 2004 11:57:46 PM PDT
Subject: thoughts on DAS/2 validation

> ??writeback from DAS/2 clients to DAS/2 servers (there will be a 
> meeting Monday on this)
> ??implementation details?of DAS/2 clients, servers, validation suite 
> (I want to
>        keep focus on spec and logistics)

I don't think I need to attend that writeback meeting.  I
contributed my ideas during the locking/transaction meeting
from a couple of days ago, along with my followup email.

I want to mention a bit about my tentative plans for testing
and validation.

I've mentioned that I wanted to investigate using RelaxNG (RNG)
for the validation.  (www.relaxng.org).  I did that this evening.

Short history.  XML originally used DTDs for schema validation.
DTDs have various problems.  It doesn't handle data types (eg,
'start' must be a number) or unordered collections.  XML Schema
was supposed to improve on DTDs but it's a complicated standard
with lots of bits and pieces added to it in committee.

James Clark and a small group of other people worked on a few
alternates, which merged to become RNG.  This is definitely
easier to understand and use and (IMNSHO) more elegant than
XML Schema.

RNG has two representations, one in XML and the other using
a "compact" syntax which is easier for humans to work with.
The first one uses the filename suffix ".rng" and the
latter ".rnc".

There is a program called "trang" to convert between these
two formats.  It can also read/write to DTDs and write to
XML Schema.  Because it's so easy, I withdraw my complaint
about using DTDs.

To make things even more fun, there's a program called
"Examplotron" which is an XSLT (!) program to convert an
example XML document into an RNG for it.

I tested it out with the SOURCES example.  Here's the XML
from the spec, but without the DTD

<?xml version="1.0" standalone="no"?>
<SOURCES
     xmlns="http://www.biodas.org/ns/das-genome/2.00"
     xmlns:xlink="http://www.w3.org/1999/xlink"
     xml:base="http://dev.wormbase.org/das-genome/">
   <SOURCE id="volvox" description="Volvox Example Database"
           taxon="http://www.ncbi.nlm.nih.gov/taxon-browser?id=29118">
      <VERSION id="volvox/1" description="Build 1, October 2002" />
      <VERSION id="volvox/2" description="Build 2, January 2003" />
   ... truncated ...

Here's the RNG for it.  The "ega:example" is used to keep the
original data around.  This makes the RNG invertible back to
the example XML.  RNG validation ignores elements outside its
namespace.  This lets people add new fields, including documentation.
There's even software to turn an RNG plus documentation into
DocBook format.

<grammar xmlns="http://relaxng.org/ns/structure/1.0" 
xmlns:ega="http://examplotr
on.org/annotations/" xmlns:sch="http://www.ascc.net/xml/schematron" 
datatypeLibr
ary="http://www.w3.org/2001/XMLSchema-datatypes">
    <start>
       <element name="SOURCES" 
ns="http://www.biodas.org/ns/das-genome/2.00">
          <optional>
             <attribute name="base" 
ns="http://www.w3.org/XML/1998/namespace">
                <ega:example 
xml:base="http://dev.wormbase.org/das-genome/"/>
             </attribute>
          </optional>
          <oneOrMore>
             <element name="SOURCE">
                <optional>
                   <attribute name="id">
                      <ega:example id="volvox"/>
                   </attribute>
                </optional>
   ... truncated ...


The compact form

namespace ega = "http://examplotron.org/annotations/"
default namespace ns1 = "http://www.biodas.org/ns/das-genome/2.00"
namespace sch = "http://www.ascc.net/xml/schematron"
namespace xlink = "http://www.w3.org/1999/xlink"

start =
   element SOURCES {
     [ ega:example [ xml:base = "http://dev.wormbase.org/das-genome/" ] ]
     attribute base { text }?,
     (element SOURCE {
        [ ega:example [ id = "volvox" ] ] attribute id { text }?,
        [ ega:example [ description = "Volvox Example Database" ] ]
   ... truncated ...


and DTD

<!ELEMENT ns1:SOURCES (SOURCE)+>
<!ATTLIST ns1:SOURCES
   xmlns:ns1 CDATA #FIXED 'http://www.biodas.org/ns/das-genome/2.00'
   xml:base CDATA #IMPLIED>

<!ELEMENT ns1:SOURCE (VERSION)+>
<!ATTLIST ns1:SOURCE
   xmlns:ns1 CDATA #FIXED 'http://www.biodas.org/ns/das-genome/2.00'
   id CDATA #IMPLIED
   description CDATA #IMPLIED
   taxon CDATA #IMPLIED>
   ... truncated ...

What this means is that I should be able to get a working schema
up rather quickly.  This would then be used by the validator.

         ==========================

Here's my plan.  I'll continue to help with the spec in
finding places that need to be clarified or tweaked.  The
DAS/1 spec contained typos in the examples so I'll make sure
there's some way to extract all examples from the spec
so they can be tested.

I'll develop an RNG schema and come up with a set of test
documents, some legal and some illegal, to make sure the validator
is able to catch and report errors properly.

These sample documents will likely consist of two files, one for
the XML and one the HTTP headers.  This will let me run the
regression tests without a server.

I'll come up with a set of tests for data integrity.  For example,
get a sequence from a range and make sure the FASTA length is of
the right size.  Get all of the features in a range and test that
the server's filters work correctly.

The writeback tests will be harder.  I'll do a set of adds,
deletes, and modifies.  If we have transaction support on the
server I'll test that.  I'll test for proper lock semantics.

This is harder because it requires I be able to scribble on
a server's database, and that the code be given username/password
for how to get write permissions.  More likely two, to make
sure that locking works correctly.

What I would really like is be able to provide a known
data set that can be imported by a server specifically for
testing.  Then I could include tests that would know what
the answer is supposed to be, rather than simply testing
for internal integrity.  Some tests would include being able
to search for a feature with an attribute that includes a
& or / in it, and make sure that range searches don't all
have off-by-one errors.

This requires there be data files in a format easily
imported into a server.  Any thoughts?  I know Lincoln's
DAS/1 perl server had something like that but the file
format wasn't well defined (at least for my picky tastes ;)

As I recall, writeback support wasn't going to be until
year 2 of the grant so the testing for this year should
only be for format validation and basic integrity checks.


					Andrew
					dalke at dalkescientific.com




From Steve_Chervitz at affymetrix.com  Mon Dec 13 20:44:45 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:44:45 -0800
Subject: [DAS2] Fwd: DAS Writeback Summary
Message-ID: <BB42837B-4D71-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Ed Griffiths <edgrif at sanger.ac.uk>
Date: September 28, 2004 5:43:56 AM PDT
Subject: Todays meeting.

All,

I have attached a summary and a fuller set of notes about where we seem  
to have
got to. [inline below]

There is a lot of work to do in formalising and filling out the details  
but we
seem to be making some progress.

Ed
--  
   
------------------------------------------------------------------------
| Ed Griffiths, Acedb development, Informatics Group,                    
  |
|        Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus,  
  |
|               Hinxton, Cambridge CB10 1SA, UK                          
  |
|                                                                        
  |
| email: edgrif at sanger.ac.uk  Tel: +44-1223-494780  Fax:  
+44-1223-494919 |
   
------------------------------------------------------------------------


An aside:

we need to sort out the mailing lists, its clear that there is a lot I  
and
others haven't seen. Should we be using a restricted mailing list or....

actions: who is going to put the DAS2 spec together as a result of all  
these
meetings and by when ?

Is there a reference server set up that we can all try or copy to our  
own
locations ?


In this write up I've marked with a ** some of the issues we should  
discuss.


=================================================================
DAS 2.0 Write Back

HTTP protocol issues
--------------------

- we have decided not to use webdav in this DAS2 version.

- we need to decide how our DAS2 requests map to the http  
GET/POST/PUT/DELETE
commands.

e.g. we have not yet discussed how we're going to do the actual  
editing.  It might
be by PUT/DELETE calls on the elements, or through a unified diff  
POSTed to
the server which contains a list of all the changes.

- in particular we need to decide what we pass between server and  
client in the
way of authentification and state (e.g. locks).



Locking
-------

- we have decided that we will only allow locking on regions, not  
features.
The client can specify start/end coords for regions.

- Note that this implies that we only lock _locatable_ objects, this is
something we need to explore further, e.g. how do "types" get editted,  
how do
genes that not located get editted etc. etc. ?

- we decided that unless an object is fully contained within the locked  
region
then it is not locked. If the user wants to edit that object then they  
have to
extend the region that they want to lock to include that object.


Lock operations

- the lock request is POSTed to a new URL, listed in the data source  
document
and probably named something like .../lock/new. The request will include
things like name (or is that provided via some other authentication  
means?),
region to lock, and time for lock. If no error, the return includes the  
lock
URL (probably something like ".../lock/lock12345"), and length of time
actually granted.

- a GET on the lock URL returns info about who locked it, the time  
remaining
for the lock, and region

- There should be a request that allows the client to find out from the  
server
which regions are already locked so that annotators can be given this
information before they request a lock.  (the data source document will  
also
include a URL used to get the list of current locks)

- There will be a request which allows the client to find out which  
types are
edittable on a per user basis, this will allow the annotator to see in  
advance
what they are allowed/able to edit.

- There will have to be authorisation for each lock/edit operation. It  
was
assumed that all this will be straight forward as database  
authorisation and
authorisation via http are known and solved problems.

- a POST of some sort to the URL is used to renew the lock, commit the  
changes
or revert the transaction.  We think it should also be able to expand  
the
locked region.

- breaking locks is outside the scope of the spec.  Eg, it might only  
be done
by people who can log into the database system.



Versioning (but see later)
----------

We also talked about tracking all change (just like a full-blown version
control system would) but decided again it was too much to deal with  
now.

** What we do need to decide is on how a client asks for a specific  
version of
** a feature, or even if this is possible ?


Timeouts
--------

- locks will have timeouts (when a lock is granted, the response will  
tell
the client how long the lock will be held for).

- the timeout period is for "idle" time, not total time.


Transactions
------------
- it was agreed that transactions should be transparent to the client so
whether the database uses them to commit data will not be visible. This
implies that our protocol for updating must be via single, "atomic",  
http
requests.

- should be providing a mechanism for a client to verify that changes  
have
been successfully committed in this case ?


Proxies
-------

- Andrews investigations imply that caching will not be the problem we  
thought
it might be.

- But one thing to consider though is how we're going to do user
authentication.  Use an existing HTTP username/password scheme or  
something
outside of that?  If ours is independent of HTTP then we have to be more
careful about how to deal with caches.

If ours is based on HTTP and we want to play nice with caches then  
clients
shouldn't include password information to publicly accessible data as  
that
won't be cached.


Data issues
-----------

- versioning of features is currently responsibility of the database  
but we
may need to provide something to say "give me the latest version" ?

- the client can find out which feature types can be editted for any one
user and hence can show this to the annotator.

- the annotator can edit features on a sequence but not the sequence  
assembly
itself.



From Steve_Chervitz at affymetrix.com  Mon Dec 13 20:48:25 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:48:25 -0800
Subject: [DAS2] Fwd: DAS Write Followup
Message-ID: <3E81AE89-4D72-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Ed Griffiths <edgrif at sanger.ac.uk>
Date: September 28, 2004 5:43:56 AM PDT
Subject: Todays meeting.

All,

I have attached a summary and a fuller set of notes about where we seem 
to have
got to. [inline below]

There is a lot of work to do in formalising and filling out the details 
but we
seem to be making some progress.

Ed
-- 

[The following is Ed's "DAS-WRITE-FOLLOWUP.email" document.]

================================================================
DAS 2.0 Write Back


HTTP protocol issues
--------------------

There was an idea that we would try to map the basic http GET, POST, 
DELETE
etc.  to meaningful writeback/locking actions, we didn't discuss this in
detail except to say that no one was keen to artificially shoe-horn 
write-back
operations into this framework.

** We need to sort out which operations are GET/POST/PUT/DELETE.

We have not yet discussed how we're going to do the actual editing.  It 
might
be by PUT/DELETE calls on the elements, or through a unified diff 
POSTed to
the server which contains a list of all the changes.

(Which reminds me, the DAV spec also support the HTTP operations of 
MOVE and
COPY.  These allow the versioning system to keep version information 
even
after splits and renames.)



There was some discussion of how you implement write-back, which 
requires
state, within the stateless http protocol. Thomas raised webdav as a 
possible
way to go. Gregg said that Brian (surname ?) has investigated this to 
some
extent but is not currently working on it. No one else has great 
experience of
it but Thomas said that it provides a protocol on top of http and this 
made
the idea of using unpalatable because it would mean that all clients 
would
have to use it if they want to talk DAS2. Andrew added that "The thing 
about
DAV is that it requires several new HTTP level commands, like LOCK and
PROPFIND.  DAV seems to provide mechanism but not policy and without 
some
experience we decided to pass on it for this version."

The concensus was that we should not use webdav but produce our own 
protocol
for doing write-back.



Locking
-------

What should be locked ?

After quite a bit of discussion about the granularity of locking that 
should
be allowed the concensus was that the system would become far too 
complex (=
unusable/unimplementable) unless we went for a system like Otter where 
locking
was on the basis of regions. The client would ask for a lock on a 
region,
giving start/end coords and every feature located in that region would 
be
locked.

(Note that this implies that we only lock _locatable_ objects, this is
something we need to explore further, e.g. how do "types" get editted, 
how do
genes that not located get editted etc. etc. ?)

The policy that James adopts and seemed sensible to everyone was that 
unless
an object is fully contained within the locked region then it is not
locked. If the user wants to edit that object then they have to extend 
the
region that they want to lock to include that object.


Lock operations

There should be an operation that allows the client to find out from the
server which regions are already locked so that annotators can be given 
this
information before they request a lock.

We discussed when the client should get the lock, i.e. before the 
editting
starts or once the user has a set of edits they want to save. There were
strong advocates for both models but in the end it was thought that 
locking in
advance is better otherwise an annotator may spend some time editting 
only to
discover that they cannot save their edits, i.e. lock conflicts are 
hard to
deal with without locking in advance.


We also need a request which allows the client to find out which types 
are
edittable on a per user basis, this will allow the annotator to see in 
advance
what they are allowed/able to edit.

We discussed authorisation, which given that we are using http will 
have to be
done for each lock/edit operation. It was assumed that all this will be
straight forward as database authorisation and authorisation via http 
are
known and solved problems.


The conclusions in the locking discussion are:
    - pessimistic locking on a region specified by the curator.
       We will not support locks on a specific feature, feature
       type, etc.

    - the lock request is POSTed to a new URL, listed in the
        data source document and probably named something like
         .../lock/new

    - the request will include things like name (or is that
        provided via some other authentication means?), region
        to lock, and time for lock

    - if no error, the return include the lock URL (probably
        something like ".../lock/lock12345"), and length of
        time actually granted

    - a GET on the lock URL returns info about who locked it,
        the time remaining for the lock, and region

    - a POST of some sort to the URL is used to renew the lock,
        commit the changes or revert the transaction.  We think
        it should also be able to expand the locked region.

    - the data source document will also include a URL
        used to get the list of current locks

    - breaking locks is outside the scope of the spec.  Eg,
        it might only be done by people who can log into
        the database system.


Versioning (but see later)

We also talked about tracking all change (just like a full-blown version
control system would) but decided again it was too much to deal with 
now.

** What we do need to decide is on how a client asks for a specific 
version of
** a feature, or even if this is possible ?


Timeouts

Clearly we need to be able to timeout locks: clients may disappear 
without the
server being aware of this.

Andrew suggested that when the client requests a lock, the response 
will say
how long the lock will last and also returns a url which the client can 
use to
query/unlock the lock. Everyone agreed this seemed a sound idea.

It was agreed that the "timeout" period applies to how long the client 
has
been idle, not how long in total it has been connected. The lock url 
could be
used by the client to signal that it was still active without the need 
for
some artificial database operation.



Transactions

There was some discussion about whether these are necessary at all. 
James uses
transactions to write the annotators data back into the database and 
said that
this was a vital element as it enabled the annotator to be certain that 
their
edits had made it into the database.

In James system transactions are transparent to the client, they issue 
a save
and the save is done via a transaction. There is no prolonged 
transaction
spanning several http requests. This was seen as a good model otherwise 
the
client will have to understand too much of the locking/transaction 
model of
the underlying database.

I think everyone agreed that write back should be done via a single 
request so
that transactions did not get spread across multiple http request which 
looks
intractable.



Proxies

There is a problem with writing back data when there are proxy servers 
between
the client and the database. I'm not qualified to explain all of this, 
perhaps
Tony or someone could fill this section in. I assume the problem 
relates to
making sure the proxy actually sends the data back to the database and 
doesn't
cache it causing timeout problems etc.

Andrew said:

"It's the other way around.  A proxy may cache GET requests, because a 
GET is
not supposed to affect the server.  A POST to some other URL may affect 
the
first URL but a proxy won't know there was a change.

I mentioned this might be a problem.  I just did some research and 
found this
overview, http://www.mnot.net/cache_docs/

It describes when proxies are allowed to think something is cacheable.


1.  If the response's headers tell the cache not to keep it, it won't.
2.  If no validator (an ETag or Last-Modified header) is present
    on a response, it will be considered uncacheable.
3.  If the request is authenticated or secure, it won't be cached.
4.  A cached representation is considered fresh (that is, able to
    be sent to a client without checking with the origin server) if:
    o It has an expiry time or other age-controlling header set,
       and is still within the fresh period.
    o If a browser cache has already seen the representation, and
       has been  set to check once a session.
    o If a proxy cache has seen the representation recently, and it
       was modified relatively long ago.
   Fresh representations are served directly from the cache, without
   checking with the origin server
5. If an representation is stale, the origin server will be asked
   to validate it, or tell the cache whether the copy that it has
   is still good.

So it looks like we won't stumble into cache problems by accident.  
Someone
has to enable a validator tag before the cache comes into play.

One thing to consider though is how we're going to do user 
authentication.
Use an existing HTTP username/password scheme or something outside of 
that?
If ours is independent of HTTP then we have to be more careful about 
how to
deal with caches.

If ours is based on HTTP and we want to play nice with caches then 
clients
shouldn't include password information to publicly accessible data as 
that
won't be cached.

But I wouldn't worry about this.  It's most theoretical and best to 
wait until
it becomes an actual problem.


Data issues
-----------

We discussed version numbering of objects. James has unique ids for all
edittable objects with version numbers as well. This enables him to keep
numbered versions of an object and retrieve the latest version for 
editting
but also to go back to previous versions if required. It may be that we 
need
the client to be able to ask for previous versions.

How does the client know which things can be editted ?

In James system James system there are edittable and non-edittable data
streams.  It was felt that this is not flexible enough for DAS. 
Therefore the
client needs to be able to discover what objects within a region can be
editted.

Current suggestion is that when the server returns what types it can 
serve,
the type should include a flag to say whether this type from this 
server can
be editted. If the type can be editted then this says the annotator can 
edit
_all_ objects of that type from that server. This would obviate the 
need for
every object to be flagged with whether it was edittable, something 
that none
of us thought was a good idea. This mechanism will enable the client to 
show
the annotator which objects are edittable (note that the set of objects 
that
can be editted could change from user to user depending on their 
database
permissions).



How would the client lock stuff in practice ?

1) The annotator needs to know up front what they can edit so the 
client would
be able to issue a request to the server(s) which would tell it what 
objects
were edittable.

(Andrew made a suggestion here that said something like: two urls are 
supplied
for database information for POST, there are urls for features/regions, 
server
will say "yes I have a lock, no I don't" makes it extensible can anyone
remember what he said in more detail ?)


2) The annotator then indicates what they want to lock and the client 
issues a
request to the server that says "Am I allowed to lock these actual 
objects ?",
server replies yes or no.

3) If stage 2) was successful then the client can go on to issue a 
request to
actually lock the data.


So the mechanism that seemed to make sense was for the server to serve 
up
everything with _no_ indication at the object level whether something is
edittable. The client can then discover which of the objects it is 
displaying
can be editted.

It was agreed that the annotator would be allowed to edit objects 
located on
an assembly but not actually edit the assembly itself.  There is a clear
distinction between moving features around an assembly and actually 
changing
the assembly itself !

Conclusion
----------

We covered a lot of ground in the meeting and a lot of important points 
were
discussed. There are still a lot more things to firm up. There will be 
at
least one or two meetings today and over the weekend to do this. Gregg 
and Ed
can meet on Monday morning to try to write all this up so that we have a
proposal that everyone can then shoot down/applaud.



From Steve_Chervitz at affymetrix.com  Mon Dec 13 20:50:27 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:50:27 -0800
Subject: [DAS2] Fwd: Re: DAS Writeback Summary
Message-ID: <8786CB46-4D72-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Suzanna Lewis <suzi at fruitfly.org>
Date: September 29, 2004 11:22:23 AM PDT
Subject: Re: Todays meeting.

Hi Ed,

I could not read these for some reason (admittedly I didn't struggle)
but was wondering if you could send them as plain text.

I was also wondering if for a feature's authorization we could model
it like file system. World, group, and user with read and write  
permissions.
Just a whimsy, but might serve to help.

-S

On Sep 28, 2004, at 5:43 AM, Ed Griffiths wrote:

> All,
>
> I have attached a summary and a fuller set of notes about where we  
> seem to have
> got to.
>
> There is a lot of work to do in formalising and filling out the  
> details but we
> seem to be making some progress.
>
> Ed
> --   
> ----------------------------------------------------------------------- 
> -
> | Ed Griffiths, Acedb development, Informatics Group,                   
>   |
> |        Wellcome Trust Sanger Institute, Wellcome Trust Genome  
> Campus,  |
> |               Hinxton, Cambridge CB10 1SA, UK                         
>   |
> |                                                                       
>   |
> | email: edgrif at sanger.ac.uk  Tel: +44-1223-494780  Fax:  
> +44-1223-494919 |
>   
> ----------------------------------------------------------------------- 
> -
>
> <DAS-WRITEBACK-SUMMARY.email><DAS-WRITE-FOLLOWUP.email>



From Steve_Chervitz at affymetrix.com  Thu Dec 16 21:17:10 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Thu, 16 Dec 2004 18:17:10 -0800
Subject: [DAS2] FW: Semantic web position paper
In-Reply-To: <BDCA8136.26FE%Steve_Chervitz@affymetrix.com>
Message-ID: <BDE781A6.3118%Steve_Chervitz@affymetrix.com>



------ Forwarded Message
From: <Chervitz>, Steve <Steve_Chervitz at affymetrix.com>
Date: Wed, 24 Nov 2004 18:19:34 -0800
Subject: Semantic web position paper

Folks,

I attended a W3C workshop on the semantic web for life sciences last month
and wrote a position paper that discusses issues for the semantic web from a
DAS point of view. A big part of the discussion centers around LSID.

You can find the paper on this page, just do an in-page search for
'chervitz':
http://www.w3.org/2004/07/swls-agenda.html

Others at the meeting raised similar issues with LSID, but I'd say that my
paper had the most pointed assault (thanks in large part to Andrew). If you
have any comments or other thoughts, let me know.

Steve





From Steve_Chervitz at affymetrix.com  Thu Dec 16 21:22:12 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Thu, 16 Dec 2004 18:22:12 -0800
Subject: [DAS2] FW: Semantic web position paper
In-Reply-To: <BDE781A6.3118%Steve_Chervitz@affymetrix.com>
Message-ID: <BDE782D4.311A%Steve_Chervitz@affymetrix.com>


The W3C's report of the Workshop on Semantic Web for Life Sciences is now
online.  All position papers and presentations are archived in the report.

http://www.w3.org/2004/10/swls-workshop-report.html

Steve


> ------ Forwarded Message
> From: <Chervitz>, Steve <Steve_Chervitz at affymetrix.com>
> Date: Wed, 24 Nov 2004 18:19:34 -0800
> Subject: Semantic web position paper
> 
> Folks, 
> 
> I attended a W3C workshop on the semantic web for life sciences last month
> and wrote a position paper that discusses issues for the semantic web from a
> DAS point of view. A big part of the discussion centers around LSID.
> 
> You can find the paper on this page, just do an in-page search for
> 'chervitz': 
> http://www.w3.org/2004/07/swls-agenda.html
> 
> Others at the meeting raised similar issues with LSID, but I'd say that my
> paper had the most pointed assault (thanks in large part to Andrew). If you
> have any comments or other thoughts, let me know.
> 
> Steve 
> 
> 
> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2 



From Steve_Chervitz at affymetrix.com  Fri Dec 17 15:22:11 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Fri, 17 Dec 2004 12:22:11 -0800
Subject: [DAS2] RSS in Science Publishing
In-Reply-To: <E1CfKoB-0005w0-Ak@frink.w3.org>
Message-ID: <BDE87FF3.317E%Steve_Chervitz@affymetrix.com>

Here's an interesting article:

http://www.dlib.org/dlib/december04/hammond/12hammond.html

It makes me think about the applicability of RSS to DAS. DAS is in fact a
type of content syndication. Wouldn't it be cool if you could hook up an
off-the-shelf RSS reader to a DAS server and get an RSS feed for a specified
genomic region or feature or all features or a particular type?

Here's a DAS-RSS scenario: A curator adds or updates an annotation on a DAS
server, the server determines whether the modified region/feature/type is
syndicated and, if so, publishes an RSS entry describing what was changed
(or simply, "region/feature/type foo was changed"). The RSS entry would
contain a URL encoding a DAS request for the region/feature/type, which one
could then open in a DAS client for further exploration. Maybe the DAS
client itself would be the RSS reader -- presenting the user with a window
listing RSS entry titles and links to browse them graphically.

Perhaps more useful would be a DAS-RSS aggregator that listens to multiple
DAS servers and consolidates them so that a user could get RSS notifications
from multiple servers for a given region/feature/type. The DAS/RSS client
could provide this aggregation function, letting the user choose which
servers and data sources they want to aggregate.

The usefulness here is limited by the fact that you only get notified about
things that changed. However, one could in principle also hook the DAS-RSS
client into the world of scientific publications by pointing it at a
scientific publisher's RSS feed. When an article about
organism/region/feature/type of interest gets published, the client
generates the appropriate DAS request URL(s) so you can browse them in the
graphical viewer. 

If publishers include appropriate metadata about the articles using standard
ontologies and taxonomies, this scenario may not be such a leap. The "if" in
the previous statement may not be that big. The presence of Nature,
Elsevier, and the NLM at the W3C meeting indicates that publishers are
catching on.

I don't know how easy it would be to RSS-ifying the DAS spec. Doing this
might be too radical of a change from DAS/1. It might be better to layer the
DAS-RSS functionality on top of DAS, so it could work with both DAS/2 and
DAS/1 servers. Thoughts?

Steve

------ Forwarded Message
From: Ben Lund <b.lund at nature.com>
Date: Fri, 17 Dec 2004 08:18:23 -0800
To: <public-semweb-lifesci at w3.org>
Subject: Announcement:  D-Lib article on RSS in Science Publishing

Hi All, 

Apologies if any of you have seen this already, but I just wanted to
point you towards our new paper published a couple of days ago in D-Lib
Magazine (http://www.dlib.org/).  It goes into more depth on a number of
the issues that I talked about at the Semantic Web for Life Sciences
workshop in October:

The Role of RSS in Science Publishing:
Syndication and Annotation on the Web

by Tony Hammond, Timo Hannay and Ben Lund,
Nature Publishing Group

http://www.dlib.org/dlib/december04/hammond/12hammond.html

Hopefully it'll be of interest.

### 

D-Lib Magazine is a solely electronic publication with a primary focus
on digital library research and development, including but not limited
to new technologies, applications, and contextual social and economic
issues. D-Lib Magazine is produced by the Corporation for National
Research Initiatives (CNRI), has been sponsored by the Defense Advanced
Research Project Agency (DARPA), and is currently being funded by the
National Science Foundation (NSF).

### 

Ta, 
Ben 

--Ben Lund 
--New Technology 
--Nature Publishing Group

--- 



****************************************************************************
**** 
DISCLAIMER: This e-mail is confidential and should not be used by anyone who
is not the original intended recipient. If you have received this e-mail in
error please inform the sender and delete it from your mailbox or any other
storage mechanism. Neither Macmillan Publishers Limited nor any of its
agents accept liability for any statements made which are clearly the
sender's own and not expressly made on behalf of Macmillan Publishers
Limited or one of its agents. Please note that neither Macmillan Publishers
Limited nor any of its agents accept any responsibility for viruses that may
be contained in this e-mail or its attachments and it is your responsibility
to scan the e-mail and attachments (if any). No contracts may be concluded
on behalf of Macmillan Publishers Limited or its agents by means of e-mail
communication. Macmillan Publishers Limited Registered in England and Wales
with registered number 785998 Registered Office Brunel Road, Houndmills,
Basingstoke RG21 6XS

****************************************************************************
**** 

------ End of Forwarded Message



From Steve_Chervitz at affymetrix.com  Fri Dec 17 20:35:20 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Fri, 17 Dec 2004 17:35:20 -0800
Subject: [DAS2] Content type for DAS XML responses
Message-ID: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>

Currently, the DAS/2 spec dictates that responses should have content-types
like:

  application/x-das-source
  application/x-das-types
  application/x-das-region

Changing these to be 'text/xml' would be more convenient since the content
would then be viewable in a standard web browser. This would also be good
for backward compatibility with DAS/1 (where content-type is text/plain).

If we wanted, we could provide auxiliary information using attribute=value
notation:

  text/xml; dasformat=source
  text/xml; dasformat=types
  text/xml; dasformat=region

The corresponding compact content-types could be:

  text/plain; dasformat=compact-source
  text/plain; dasformat=compact-types
  text/plain; dasformat=compact-region

And other types could be:

  text/plain; dasformat=fasta
  text/plain; dasformat=gff3
  text/plain; dasformat=bed

Objections?

Steve

Refs: 
http://www.w3.org/TR/xhtml-media-types/#text-xml
http://www.iana.org/assignments/media-types/





From allenday at ucla.edu  Fri Dec 17 21:40:14 2004
From: allenday at ucla.edu (Allen Day)
Date: Fri, 17 Dec 2004 18:40:14 -0800 (PST)
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
Message-ID: <Pine.LNX.4.58.0412171838370.32735@sumo.ctrl.ucla.edu>



On Fri, 17 Dec 2004, Chervitz, Steve wrote:

> Currently, the DAS/2 spec dictates that responses should have content-types
> like:
> 
>   application/x-das-source
>   application/x-das-types
>   application/x-das-region
> 
> Changing these to be 'text/xml' would be more convenient since the content
> would then be viewable in a standard web browser. This would also be good
> for backward compatibility with DAS/1 (where content-type is text/plain).
> 
> If we wanted, we could provide auxiliary information using attribute=value
> notation:
> 
>   text/xml; dasformat=source
>   text/xml; dasformat=types
>   text/xml; dasformat=region

are browsers going to choke on this?  will browsers render text/x-das-* as 
text instead of prompting with "Save As..."?

> 
> The corresponding compact content-types could be:
> 
>   text/plain; dasformat=compact-source
>   text/plain; dasformat=compact-types
>   text/plain; dasformat=compact-region
> 
> And other types could be:
> 
>   text/plain; dasformat=fasta
>   text/plain; dasformat=gff3
>   text/plain; dasformat=bed
> 
> Objections?

Certainly one of these is better than what we currently have.

-Allen

> 
> Steve
> 
> Refs: 
> http://www.w3.org/TR/xhtml-media-types/#text-xml
> http://www.iana.org/assignments/media-types/
> 
> 
> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2
> 


From dalke at dalkescientific.com  Sat Dec 18 00:44:35 2004
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Fri, 17 Dec 2004 22:44:35 -0700
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
Message-ID: <E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>

Steve:
> Currently, the DAS/2 spec dictates that responses should have 
> content-types
> like:
>
>   application/x-das-source
>   application/x-das-types
>   application/x-das-region
>
> Changing these to be 'text/xml' would be more convenient since the 
> content
> would then be viewable in a standard web browser.

A browser should send something like

Accept: */*
   or
Accept: application/xml

or some other generic content type in its headers.  One solution
to at least consider is that IF a client does not request
application/x-das-whatever THEN return application/xml .  I do
not believe this is appropriate though.


>   text/xml; dasformat=source
>   text/xml; dasformat=types
>   text/xml; dasformat=region

I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )

    If an XML document -- that is, the unprocessed, source XML document
    -- is readable by casual users, text/xml is preferable to
    application/xml.  MIME user agents (and web user agents) that do not
    have explicit support for text/xml will treat it as text/plain, for
    example, by displaying the XML MIME entity as plain text.
    Application/xml is preferable when the XML MIME entity is unreadable
    by casual users.

That RFC is relevant to this discussion.  If I understand it
correctly it recommends using something more like

   text/x-das-source+xml

This would be understood by RFC 3023 compliant clients as
containing an XML document, by DAS clients to understand
which DAS XML format is used, and by generic clients to
contain an XML document.

> The corresponding compact content-types could be:
>
>   text/plain; dasformat=compact-source
>   text/plain; dasformat=compact-types
>   text/plain; dasformat=compact-region

There should be no need for this.  A client should use
the catch-all text/* handler for text/x-compact-source
and show it as normal text.

> And other types could be:
>
>   text/plain; dasformat=fasta
>   text/plain; dasformat=gff3
>   text/plain; dasformat=bed

Same for these.  What's the problem with
   text/x-fasta , etc.?

See also the discussion in appendix A.5 which describes
practical problems about using parameter in MIME types.

					Andrew
					dalke at dalkescientific.com



From allenday at ucla.edu  Sat Dec 18 02:14:54 2004
From: allenday at ucla.edu (Allen Day)
Date: Fri, 17 Dec 2004 23:14:54 -0800 (PST)
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
	<E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
Message-ID: <Pine.LNX.4.58.0412172314240.6215@sumo.ctrl.ucla.edu>

sounds good, can we get a patch to the spec so i can make the necessary 
implementation changes and test against browsers?  thanks!

-allen


On Fri, 17 Dec 2004, Andrew Dalke wrote:

> Steve:
> > Currently, the DAS/2 spec dictates that responses should have 
> > content-types
> > like:
> >
> >   application/x-das-source
> >   application/x-das-types
> >   application/x-das-region
> >
> > Changing these to be 'text/xml' would be more convenient since the 
> > content
> > would then be viewable in a standard web browser.
> 
> A browser should send something like
> 
> Accept: */*
>    or
> Accept: application/xml
> 
> or some other generic content type in its headers.  One solution
> to at least consider is that IF a client does not request
> application/x-das-whatever THEN return application/xml .  I do
> not believe this is appropriate though.
> 
> 
> >   text/xml; dasformat=source
> >   text/xml; dasformat=types
> >   text/xml; dasformat=region
> 
> I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
> RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )
> 
>     If an XML document -- that is, the unprocessed, source XML document
>     -- is readable by casual users, text/xml is preferable to
>     application/xml.  MIME user agents (and web user agents) that do not
>     have explicit support for text/xml will treat it as text/plain, for
>     example, by displaying the XML MIME entity as plain text.
>     Application/xml is preferable when the XML MIME entity is unreadable
>     by casual users.
> 
> That RFC is relevant to this discussion.  If I understand it
> correctly it recommends using something more like
> 
>    text/x-das-source+xml
> 
> This would be understood by RFC 3023 compliant clients as
> containing an XML document, by DAS clients to understand
> which DAS XML format is used, and by generic clients to
> contain an XML document.
> 
> > The corresponding compact content-types could be:
> >
> >   text/plain; dasformat=compact-source
> >   text/plain; dasformat=compact-types
> >   text/plain; dasformat=compact-region
> 
> There should be no need for this.  A client should use
> the catch-all text/* handler for text/x-compact-source
> and show it as normal text.
> 
> > And other types could be:
> >
> >   text/plain; dasformat=fasta
> >   text/plain; dasformat=gff3
> >   text/plain; dasformat=bed
> 
> Same for these.  What's the problem with
>    text/x-fasta , etc.?
> 
> See also the discussion in appendix A.5 which describes
> practical problems about using parameter in MIME types.
> 
> 					Andrew
> 					dalke at dalkescientific.com
> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2
> 


From lstein at cshl.edu  Mon Dec 20 12:04:23 2004
From: lstein at cshl.edu (Lincoln Stein)
Date: Mon, 20 Dec 2004 12:04:23 -0500
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
	<E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
Message-ID: <200412201204.23423.lstein@cshl.edu>

> Same for these.  What's the problem with
>    text/x-fasta , etc.?

I like this solution. I don't like returning text/xml because there is 
a growing tendency of browsers to try to parse the XML, returning 
uninterpretable results.

Lincoln

-- 
Lincoln D. Stein
Cold Spring Harbor Laboratory
1 Bungtown Road
Cold Spring Harbor, NY 11724

NOTE: Please copy Sandra Michelsen <michelse at cshl.edu> on
all emails regarding scheduling and other time-critical topics.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20041220/932350c8/attachment.bin>

From dalke at dalkescientific.com  Mon Dec 20 12:44:49 2004
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 20 Dec 2004 10:44:49 -0700
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <200412201204.23423.lstein@cshl.edu>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
	<E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
	<200412201204.23423.lstein@cshl.edu>
Message-ID: <D88F7C5E-52AE-11D9-8B02-000A956826C8@dalkescientific.com>

Lincoln:
>> Same for these.  What's the problem with
>>    text/x-fasta , etc.?
>
> I like this solution. I don't like returning text/xml because there is
> a growing tendency of browsers to try to parse the XML, returning
> uninterpretable results.

What is "this"?  The use of "+xml" at the end of the content type?

I ask because the snippet you quoted from me is what's already there
so isn't a new solution.

					Andrew
					dalke at dalkescientific.com



From lstein at cshl.edu  Mon Dec 20 15:09:50 2004
From: lstein at cshl.edu (Lincoln Stein)
Date: Mon, 20 Dec 2004 15:09:50 -0500
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <D88F7C5E-52AE-11D9-8B02-000A956826C8@dalkescientific.com>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
	<200412201204.23423.lstein@cshl.edu>
	<D88F7C5E-52AE-11D9-8B02-000A956826C8@dalkescientific.com>
Message-ID: <200412201509.50516.lstein@cshl.edu>

Doesn't the spec use application/* mime types? This is a text/* mime 
type.

Lincoln


On Monday 20 December 2004 12:44 pm, Andrew Dalke wrote:
> Lincoln:
> >> Same for these.  What's the problem with
> >>    text/x-fasta , etc.?
> >
> > I like this solution. I don't like returning text/xml because
> > there is a growing tendency of browsers to try to parse the XML,
> > returning uninterpretable results.
>
> What is "this"?  The use of "+xml" at the end of the content type?
>
> I ask because the snippet you quoted from me is what's already
> there so isn't a new solution.
>
> 					Andrew
> 					dalke at dalkescientific.com
>
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2

-- 
Lincoln D. Stein
Cold Spring Harbor Laboratory
1 Bungtown Road
Cold Spring Harbor, NY 11724

NOTE: Please copy Sandra Michelsen <michelse at cshl.edu> on
all emails regarding scheduling and other time-critical topics.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20041220/9e473462/attachment.bin>

From Steve_Chervitz at affymetrix.com  Mon Dec 20 21:42:43 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Mon, 20 Dec 2004 18:42:43 -0800
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <Pine.LNX.4.58.0412172314240.6215@sumo.ctrl.ucla.edu>
Message-ID: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>

I just committed a new version of das2_get.html with this change:

RCS file: /home/repository/biodas/das/das2/das2_get.html,v
----------------------------
revision 1.2
date: 2004/12/21 02:43:42;  author: sac;  state: Exp;  lines: +50 -49
Converted mime types from application/x-das-foo to text/x-das-foo+xml for
XML types. For non-XML types omitting the '+xml' and adding an 'x-' suffix
to the right of the slash as needed.

Steve


> From: Allen Day <allenday at ucla.edu>
> Date: Fri, 17 Dec 2004 23:14:54 -0800
> To: Andrew Dalke <dalke at dalkescientific.com>
> Cc: <das2 at portal.open-bio.org>
> Subject: Re: [DAS2] Content type for DAS XML responses
> 
> sounds good, can we get a patch to the spec so i can make the necessary
> implementation changes and test against browsers?  thanks!
> 
> -allen 
> 
> 
> On Fri, 17 Dec 2004, Andrew Dalke wrote:
> 
>> Steve: 
>>> Currently, the DAS/2 spec dictates that responses should have
>>> content-types 
>>> like: 
>>> 
>>>   application/x-das-source
>>>   application/x-das-types
>>>   application/x-das-region
>>> 
>>> Changing these to be 'text/xml' would be more convenient since the
>>> content 
>>> would then be viewable in a standard web browser.
>> 
>> A browser should send something like
>> 
>> Accept: */* 
>>    or 
>> Accept: application/xml
>> 
>> or some other generic content type in its headers.  One solution
>> to at least consider is that IF a client does not request
>> application/x-das-whatever THEN return application/xml .  I do
>> not believe this is appropriate though.
>> 
>> 
>>>   text/xml; dasformat=source
>>>   text/xml; dasformat=types
>>>   text/xml; dasformat=region
>> 
>> I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
>> RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )
>> 
>>     If an XML document -- that is, the unprocessed, source XML document
>>     -- is readable by casual users, text/xml is preferable to
>>     application/xml.  MIME user agents (and web user agents) that do not
>>     have explicit support for text/xml will treat it as text/plain, for
>>     example, by displaying the XML MIME entity as plain text.
>>     Application/xml is preferable when the XML MIME entity is unreadable
>>     by casual users.
>> 
>> That RFC is relevant to this discussion.  If I understand it
>> correctly it recommends using something more like
>> 
>>    text/x-das-source+xml
>> 
>> This would be understood by RFC 3023 compliant clients as
>> containing an XML document, by DAS clients to understand
>> which DAS XML format is used, and by generic clients to
>> contain an XML document.
>> 
>>> The corresponding compact content-types could be:
>>> 
>>>   text/plain; dasformat=compact-source
>>>   text/plain; dasformat=compact-types
>>>   text/plain; dasformat=compact-region
>> 
>> There should be no need for this.  A client should use
>> the catch-all text/* handler for text/x-compact-source
>> and show it as normal text.
>> 
>>> And other types could be:
>>> 
>>>   text/plain; dasformat=fasta
>>>   text/plain; dasformat=gff3
>>>   text/plain; dasformat=bed
>> 
>> Same for these.  What's the problem with
>>    text/x-fasta , etc.?
>> 
>> See also the discussion in appendix A.5 which describes
>> practical problems about using parameter in MIME types.
>> 
>>                                       Andrew
>>                                       dalke at dalkescientific.com
>> 
>> _______________________________________________
>> DAS2 mailing list
>> DAS2 at portal.open-bio.org
>> http://portal.open-bio.org/mailman/listinfo/das2
>> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2 



From allenday at ucla.edu  Mon Dec 20 23:32:42 2004
From: allenday at ucla.edu (Allen Day)
Date: Mon, 20 Dec 2004 20:32:42 -0800 (PST)
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
References: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
Message-ID: <Pine.LNX.4.58.0412202032140.24345@sumo.ctrl.ucla.edu>

changes made to das2 codebase, thanks.

On Mon, 20 Dec 2004, Chervitz, Steve wrote:

> I just committed a new version of das2_get.html with this change:
> 
> RCS file: /home/repository/biodas/das/das2/das2_get.html,v
> ----------------------------
> revision 1.2
> date: 2004/12/21 02:43:42;  author: sac;  state: Exp;  lines: +50 -49
> Converted mime types from application/x-das-foo to text/x-das-foo+xml for
> XML types. For non-XML types omitting the '+xml' and adding an 'x-' suffix
> to the right of the slash as needed.
> 
> Steve
> 
> 
> > From: Allen Day <allenday at ucla.edu>
> > Date: Fri, 17 Dec 2004 23:14:54 -0800
> > To: Andrew Dalke <dalke at dalkescientific.com>
> > Cc: <das2 at portal.open-bio.org>
> > Subject: Re: [DAS2] Content type for DAS XML responses
> > 
> > sounds good, can we get a patch to the spec so i can make the necessary
> > implementation changes and test against browsers?  thanks!
> > 
> > -allen 
> > 
> > 
> > On Fri, 17 Dec 2004, Andrew Dalke wrote:
> > 
> >> Steve: 
> >>> Currently, the DAS/2 spec dictates that responses should have
> >>> content-types 
> >>> like: 
> >>> 
> >>>   application/x-das-source
> >>>   application/x-das-types
> >>>   application/x-das-region
> >>> 
> >>> Changing these to be 'text/xml' would be more convenient since the
> >>> content 
> >>> would then be viewable in a standard web browser.
> >> 
> >> A browser should send something like
> >> 
> >> Accept: */* 
> >>    or 
> >> Accept: application/xml
> >> 
> >> or some other generic content type in its headers.  One solution
> >> to at least consider is that IF a client does not request
> >> application/x-das-whatever THEN return application/xml .  I do
> >> not believe this is appropriate though.
> >> 
> >> 
> >>>   text/xml; dasformat=source
> >>>   text/xml; dasformat=types
> >>>   text/xml; dasformat=region
> >> 
> >> I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
> >> RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )
> >> 
> >>     If an XML document -- that is, the unprocessed, source XML document
> >>     -- is readable by casual users, text/xml is preferable to
> >>     application/xml.  MIME user agents (and web user agents) that do not
> >>     have explicit support for text/xml will treat it as text/plain, for
> >>     example, by displaying the XML MIME entity as plain text.
> >>     Application/xml is preferable when the XML MIME entity is unreadable
> >>     by casual users.
> >> 
> >> That RFC is relevant to this discussion.  If I understand it
> >> correctly it recommends using something more like
> >> 
> >>    text/x-das-source+xml
> >> 
> >> This would be understood by RFC 3023 compliant clients as
> >> containing an XML document, by DAS clients to understand
> >> which DAS XML format is used, and by generic clients to
> >> contain an XML document.
> >> 
> >>> The corresponding compact content-types could be:
> >>> 
> >>>   text/plain; dasformat=compact-source
> >>>   text/plain; dasformat=compact-types
> >>>   text/plain; dasformat=compact-region
> >> 
> >> There should be no need for this.  A client should use
> >> the catch-all text/* handler for text/x-compact-source
> >> and show it as normal text.
> >> 
> >>> And other types could be:
> >>> 
> >>>   text/plain; dasformat=fasta
> >>>   text/plain; dasformat=gff3
> >>>   text/plain; dasformat=bed
> >> 
> >> Same for these.  What's the problem with
> >>    text/x-fasta , etc.?
> >> 
> >> See also the discussion in appendix A.5 which describes
> >> practical problems about using parameter in MIME types.
> >> 
> >>                                       Andrew
> >>                                       dalke at dalkescientific.com
> >> 
> >> _______________________________________________
> >> DAS2 mailing list
> >> DAS2 at portal.open-bio.org
> >> http://portal.open-bio.org/mailman/listinfo/das2
> >> 
> > _______________________________________________
> > DAS2 mailing list
> > DAS2 at portal.open-bio.org
> > http://portal.open-bio.org/mailman/listinfo/das2 
> 


From gilmanb at pantherinformatics.com  Tue Dec 21 11:03:32 2004
From: gilmanb at pantherinformatics.com (Brian Gilman)
Date: Tue, 21 Dec 2004 11:03:32 -0500
Subject: [DAS2] Full Spec?
In-Reply-To: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
References: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
Message-ID: <DCC0DFCD-5369-11D9-A785-000A95CA3D68@pantherinformatics.com>

Hello Everyone,

	Is there a full specification for DAS 2 somewhere? Can someone please 
send it to me? I'd like to read it again and cannot seem to find it in 
my e-mail anywhere.


						Best,

								-B
-- 
Brian Gilman
President Panther Informatics Inc.

01000010 01101001 01101111
01001001 01101110 01100110
01101111 01110010 01101101
01100001 01110100 01101001
01100011 01101001 01100001
01101110

On Dec 20, 2004, at 9:42 PM, Chervitz, Steve wrote:

> I just committed a new version of das2_get.html with this change:
>
> RCS file: /home/repository/biodas/das/das2/das2_get.html,v
> ----------------------------
> revision 1.2
> date: 2004/12/21 02:43:42;  author: sac;  state: Exp;  lines: +50 -49
> Converted mime types from application/x-das-foo to text/x-das-foo+xml 
> for
> XML types. For non-XML types omitting the '+xml' and adding an 'x-' 
> suffix
> to the right of the slash as needed.
>
> Steve
>
>
>> From: Allen Day <allenday at ucla.edu>
>> Date: Fri, 17 Dec 2004 23:14:54 -0800
>> To: Andrew Dalke <dalke at dalkescientific.com>
>> Cc: <das2 at portal.open-bio.org>
>> Subject: Re: [DAS2] Content type for DAS XML responses
>>
>> sounds good, can we get a patch to the spec so i can make the 
>> necessary
>> implementation changes and test against browsers?  thanks!
>>
>> -allen
>>
>>
>> On Fri, 17 Dec 2004, Andrew Dalke wrote:
>>
>>> Steve:
>>>> Currently, the DAS/2 spec dictates that responses should have
>>>> content-types
>>>> like:
>>>>
>>>>   application/x-das-source
>>>>   application/x-das-types
>>>>   application/x-das-region
>>>>
>>>> Changing these to be 'text/xml' would be more convenient since the
>>>> content
>>>> would then be viewable in a standard web browser.
>>>
>>> A browser should send something like
>>>
>>> Accept: */*
>>>    or
>>> Accept: application/xml
>>>
>>> or some other generic content type in its headers.  One solution
>>> to at least consider is that IF a client does not request
>>> application/x-das-whatever THEN return application/xml .  I do
>>> not believe this is appropriate though.
>>>
>>>
>>>>   text/xml; dasformat=source
>>>>   text/xml; dasformat=types
>>>>   text/xml; dasformat=region
>>>
>>> I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
>>> RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )
>>>
>>>     If an XML document -- that is, the unprocessed, source XML 
>>> document
>>>     -- is readable by casual users, text/xml is preferable to
>>>     application/xml.  MIME user agents (and web user agents) that do 
>>> not
>>>     have explicit support for text/xml will treat it as text/plain, 
>>> for
>>>     example, by displaying the XML MIME entity as plain text.
>>>     Application/xml is preferable when the XML MIME entity is 
>>> unreadable
>>>     by casual users.
>>>
>>> That RFC is relevant to this discussion.  If I understand it
>>> correctly it recommends using something more like
>>>
>>>    text/x-das-source+xml
>>>
>>> This would be understood by RFC 3023 compliant clients as
>>> containing an XML document, by DAS clients to understand
>>> which DAS XML format is used, and by generic clients to
>>> contain an XML document.
>>>
>>>> The corresponding compact content-types could be:
>>>>
>>>>   text/plain; dasformat=compact-source
>>>>   text/plain; dasformat=compact-types
>>>>   text/plain; dasformat=compact-region
>>>
>>> There should be no need for this.  A client should use
>>> the catch-all text/* handler for text/x-compact-source
>>> and show it as normal text.
>>>
>>>> And other types could be:
>>>>
>>>>   text/plain; dasformat=fasta
>>>>   text/plain; dasformat=gff3
>>>>   text/plain; dasformat=bed
>>>
>>> Same for these.  What's the problem with
>>>    text/x-fasta , etc.?
>>>
>>> See also the discussion in appendix A.5 which describes
>>> practical problems about using parameter in MIME types.
>>>
>>>                                       Andrew
>>>                                       dalke at dalkescientific.com
>>>
>>> _______________________________________________
>>> DAS2 mailing list
>>> DAS2 at portal.open-bio.org
>>> http://portal.open-bio.org/mailman/listinfo/das2
>>>
>> _______________________________________________
>> DAS2 mailing list
>> DAS2 at portal.open-bio.org
>> http://portal.open-bio.org/mailman/listinfo/das2
>
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2
>



From avc at sanger.ac.uk  Tue Dec 21 11:32:52 2004
From: avc at sanger.ac.uk (Tony Cox)
Date: Tue, 21 Dec 2004 16:32:52 +0000
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <200412201204.23423.lstein@cshl.edu>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>	<E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
	<200412201204.23423.lstein@cshl.edu>
Message-ID: <41C85034.3000303@sanger.ac.uk>

Hi Lincoln,

Just been reading the Get spec - I thought at our last meeting we agreed 
to add attributes/properties (doc_href"?) to features, sources etc. that 
allowed better documenting of the data being returned.

Tony



From lstein at cshl.edu  Tue Dec 21 16:25:21 2004
From: lstein at cshl.edu (Lincoln Stein)
Date: Tue, 21 Dec 2004 16:25:21 -0500
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <41C85034.3000303@sanger.ac.uk>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
	<200412201204.23423.lstein@cshl.edu> <41C85034.3000303@sanger.ac.uk>
Message-ID: <200412211625.21821.lstein@cshl.edu>

Hi Tony,

If the doc_href stuff isn't in there, then it was my error.  It is 
certainly in my notes.  I will add it forthwith (I've got another big 
chunk of time to work on the DAS spec coming up during xmas 
vacation).

Lincoln

On Tuesday 21 December 2004 11:32 am, Tony Cox wrote:
> Hi Lincoln,
>
> Just been reading the Get spec - I thought at our last meeting we
> agreed to add attributes/properties (doc_href"?) to features,
> sources etc. that allowed better documenting of the data being
> returned.
>
> Tony

-- 
Lincoln D. Stein
Cold Spring Harbor Laboratory
1 Bungtown Road
Cold Spring Harbor, NY 11724

NOTE: Please copy Sandra Michelsen <michelse at cshl.edu> on
all emails regarding scheduling and other time-critical topics.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20041221/a79aadc0/attachment.bin>

From lstein at cshl.edu  Tue Dec 21 16:27:20 2004
From: lstein at cshl.edu (Lincoln Stein)
Date: Tue, 21 Dec 2004 16:27:20 -0500
Subject: [DAS2] Full Spec?
In-Reply-To: <DCC0DFCD-5369-11D9-A785-000A95CA3D68@pantherinformatics.com>
References: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
	<DCC0DFCD-5369-11D9-A785-000A95CA3D68@pantherinformatics.com>
Message-ID: <200412211627.20463.lstein@cshl.edu>

It's in CVS:

repository = :ext:pub.open-bio.org:/home/repository/biodas
project     =  das

Lincoln

On Tuesday 21 December 2004 11:03 am, Brian Gilman wrote:
> Hello Everyone,
>
> 	Is there a full specification for DAS 2 somewhere? Can someone
> please send it to me? I'd like to read it again and cannot seem to
> find it in my e-mail anywhere.
>
>
> 						Best,
>
> 								-B

-- 
Lincoln D. Stein
Cold Spring Harbor Laboratory
1 Bungtown Road
Cold Spring Harbor, NY 11724

NOTE: Please copy Sandra Michelsen <michelse at cshl.edu> on
all emails regarding scheduling and other time-critical topics.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20041221/24734ca8/attachment.bin>

From lstein at cshl.edu  Tue Dec 21 16:50:35 2004
From: lstein at cshl.edu (Lincoln Stein)
Date: Tue, 21 Dec 2004 16:50:35 -0500
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
References: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
Message-ID: <200412211650.35407.lstein@cshl.edu>

Beautiful!  Thank you for doing that.

Lincoln

On Monday 20 December 2004 09:42 pm, Chervitz, Steve wrote:
> I just committed a new version of das2_get.html with this change:
>
> RCS file: /home/repository/biodas/das/das2/das2_get.html,v
> ----------------------------
> revision 1.2
> date: 2004/12/21 02:43:42;  author: sac;  state: Exp;  lines: +50
> -49 Converted mime types from application/x-das-foo to
> text/x-das-foo+xml for XML types. For non-XML types omitting the
> '+xml' and adding an 'x-' suffix to the right of the slash as
> needed.
>
> Steve
>
> > From: Allen Day <allenday at ucla.edu>
> > Date: Fri, 17 Dec 2004 23:14:54 -0800
> > To: Andrew Dalke <dalke at dalkescientific.com>
> > Cc: <das2 at portal.open-bio.org>
> > Subject: Re: [DAS2] Content type for DAS XML responses
> >
> > sounds good, can we get a patch to the spec so i can make the
> > necessary implementation changes and test against browsers? 
> > thanks!
> >
> > -allen
> >
> > On Fri, 17 Dec 2004, Andrew Dalke wrote:
> >> Steve:
> >>> Currently, the DAS/2 spec dictates that responses should have
> >>> content-types
> >>> like:
> >>>
> >>>   application/x-das-source
> >>>   application/x-das-types
> >>>   application/x-das-region
> >>>
> >>> Changing these to be 'text/xml' would be more convenient since
> >>> the content
> >>> would then be viewable in a standard web browser.
> >>
> >> A browser should send something like
> >>
> >> Accept: */*
> >>    or
> >> Accept: application/xml
> >>
> >> or some other generic content type in its headers.  One solution
> >> to at least consider is that IF a client does not request
> >> application/x-das-whatever THEN return application/xml .  I do
> >> not believe this is appropriate though.
> >>
> >>>   text/xml; dasformat=source
> >>>   text/xml; dasformat=types
> >>>   text/xml; dasformat=region
> >>
> >> I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
> >> RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )
> >>
> >>     If an XML document -- that is, the unprocessed, source XML
> >> document -- is readable by casual users, text/xml is preferable
> >> to application/xml.  MIME user agents (and web user agents) that
> >> do not have explicit support for text/xml will treat it as
> >> text/plain, for example, by displaying the XML MIME entity as
> >> plain text. Application/xml is preferable when the XML MIME
> >> entity is unreadable by casual users.
> >>
> >> That RFC is relevant to this discussion.  If I understand it
> >> correctly it recommends using something more like
> >>
> >>    text/x-das-source+xml
> >>
> >> This would be understood by RFC 3023 compliant clients as
> >> containing an XML document, by DAS clients to understand
> >> which DAS XML format is used, and by generic clients to
> >> contain an XML document.
> >>
> >>> The corresponding compact content-types could be:
> >>>
> >>>   text/plain; dasformat=compact-source
> >>>   text/plain; dasformat=compact-types
> >>>   text/plain; dasformat=compact-region
> >>
> >> There should be no need for this.  A client should use
> >> the catch-all text/* handler for text/x-compact-source
> >> and show it as normal text.
> >>
> >>> And other types could be:
> >>>
> >>>   text/plain; dasformat=fasta
> >>>   text/plain; dasformat=gff3
> >>>   text/plain; dasformat=bed
> >>
> >> Same for these.  What's the problem with
> >>    text/x-fasta , etc.?
> >>
> >> See also the discussion in appendix A.5 which describes
> >> practical problems about using parameter in MIME types.
> >>
> >>                                       Andrew
> >>                                       dalke at dalkescientific.com
> >>
> >> _______________________________________________
> >> DAS2 mailing list
> >> DAS2 at portal.open-bio.org
> >> http://portal.open-bio.org/mailman/listinfo/das2
> >
> > _______________________________________________
> > DAS2 mailing list
> > DAS2 at portal.open-bio.org
> > http://portal.open-bio.org/mailman/listinfo/das2
>
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2

-- 
Lincoln D. Stein
Cold Spring Harbor Laboratory
1 Bungtown Road
Cold Spring Harbor, NY 11724

NOTE: Please copy Sandra Michelsen <michelse at cshl.edu> on
all emails regarding scheduling and other time-critical topics.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20041221/6ac7dae4/attachment.bin>

From Steve_Chervitz at affymetrix.com  Tue Dec 21 19:16:24 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Tue, 21 Dec 2004 16:16:24 -0800
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <200412211650.35407.lstein@cshl.edu>
Message-ID: <BDEDFCD8.33DD%Steve_Chervitz@affymetrix.com>

Lincoln Stein <lstein at cshl.edu> wrote:
> 
> Beautiful!  Thank you for doing that.
>

No problem. 

BTW Lincoln, your email messages are being delivered in RFC822 attachments
making them difficult to view & reply to, at least with the mail readers I
use (Entourage, Apple Mail, pine, Outlook). I guess these readers aren't yet
clued into to this (probably superior) scheme. Could you send "non-attached"
versions of your messages in addition to the attached PGP-signed message?

Perhaps there's a configuration option I'm missing (though I did recently
figure out how to get pine from complaining about iso-8859-1 character
sets). 

Mailman deals with your attachments no problem, so I've actually been using
that as my reader: http://portal.open-bio.org/pipermail/das2/

Steve

PS. Here's a curiosity: When I exported the Message/RFC822 container in your
email, I got these two lines:

> From the-concourse-on-high Tue Dec 21 15:25:50 2004
> Date: Tue, 21 Dec 2004 13:51:37 -0800

What's 'the-concourse-on-high' about? Searching the web, it appears to have
a Baha'i association (without hyphens), or as a generic 'From' attribute in
emails (with hyphens).



> 
> On Monday 20 December 2004 09:42 pm, Chervitz, Steve wrote:
> 
> I just committed a new version of das2_get.html with this change:
>
> RCS file: /home/repository/biodas/das/das2/das2_get.html,v
> ---------------------------
> revision 1.2
> date: 2004/12/21 02:43:42;  author: sac;  state: Exp;  lines: +50
> -49 Converted mime types from application/x-das-foo to
> text/x-das-foo+xml for XML types. For non-XML types omitting the
> '+xml' and adding an 'x-' suffix to the right of the slash as
> needed.
>
> Steve
>
> > From: Allen Day <allenday at ucla.edu>
> > Date: Fri, 17 Dec 2004 23:14:54 -0800
> > To: Andrew Dalke <dalke at dalkescientific.com>
> > Cc: <das2 at portal.open-bio.org>
> > Subject: Re: [DAS2] Content type for DAS XML responses
> >
> > sounds good, can we get a patch to the spec so i can make the
> > necessary implementation changes and test against browsers?
> > thanks!
> >



From Steve_Chervitz at affymetrix.com  Wed Dec  1 23:54:43 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Wed, 01 Dec 2004 15:54:43 -0800
Subject: [DAS2] New DAS2 list
In-Reply-To: <BDCA7B22.26FD%Steve_Chervitz@affymetrix.com>
Message-ID: <BDD399C3.298E%Steve_Chervitz@affymetrix.com>

As promised, we now have a discussion list for just the DAS/2 grant crew.
You should have received a welcome message from mailman already.
This list is unmoderated and will reject posts from non-members. I left the
archive public, so others could snoop if interested (and can guess the url).

Lincoln: How much overhead is moderation requiring for the original das
mailing list? Chris D made some improvements in the spam filters and I asked
if he thought they were good enough to go off-moderation. Here's what he
said:

> Our spam protection is pretty good but the stuff that leaks through is
> really hard to catch -- simple ASCII emails etc. that breeze right
> through our tests.
> 
> The current moderator web interface has a great new feature called
> "delete any messages marked as "defer" -- this lets you look at the
> moderation list and pick out only the good messages while not wasting
> time hitting checkboxes for spam. I'd recommend leaving it as is for now
> unnless you want to open the list up for a while as an experiment.

Steve


> From: <Chervitz>, Steve <Steve_Chervitz at affymetrix.com>
> Date: Wed, 24 Nov 2004 17:53:38 -0800
> Subject: Re: DAS 2 spec
> 
> Ed Griffiths <edgrif at sanger.ac.uk> wrote:
>> 
>> Hi all, 
>> 
>> Well its time to get back to doing something about the DAS2 spec for me...
>>  
>> a couple of questions......
>> 
>> where should discussion about issues take place ?
> 
> I'm working to get a mailman list set up on biodas.org that would include only
> the folks on the grant. Hopefully, this will happen soon. I'll let you know.
> I'm also looking into restricting the public biodas.org mailing list so that
> only registered users can post, to reduce junk and moderation overhead.
> 
> There may not be any action on this until next week since the US is having a
> major holiday (Thanksgiving) for the rest of this week.
>  
>> where is the current copy of the DAS2 spec ?
> 
> You can get it from the biodas cvs with these commands:
> 
> $ cvs -d :pserver:cvs at cvs.open-bio.org:/home/repository/biodas login
> password=cvs
> 
> $ cvs -d :pserver:cvs at cvs.open-bio.org:/home/repository/biodas co das
> 
> The spec is at das/das2/das2_protocol.html. The .txt version is not up to date
> (Lincoln: how about removing it to avoid confusion?). I will look into setting
> up a web page on biodas.org pointing at the latest version of
> das/das2/das2_protocol.html.
> 
> Steve
> 
> --
> Steve Chervitz | Bioinformatics Engineer | Software Development
> AFFYMETRIX, INC. | 6550 Vallejo St., Ste 100 | Emeryville, CA 94608
> Tel: 510-428-8530 | Fax: 510-428-8585 | Steve_Chervitz at affymetrix.com
> 



From Steve_Chervitz at affymetrix.com  Mon Dec 13 23:01:06 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Mon, 13 Dec 2004 15:01:06 -0800
Subject: [DAS2] September discussions
Message-ID: <BDE35F32.2F12%Steve_Chervitz@affymetrix.com>

I'm going to post some email discussions that took place during the Hinxton
Genome Informatics meeting in September, so they can be archived on this
list. Stay tuned.

Take a look at what I send, and if there are other relevant discussions,
please send them along.

Steve



From Steve_Chervitz at affymetrix.com  Tue Dec 14 01:23:05 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:23:05 -0800
Subject: [DAS2] Fwd: DAS2 server registry and discovery meeting,
	Friday at 10 AM
Message-ID: <B46F9150-4D6E-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Andrew Dalke <dalke at dalkescientific.com>
Date: September 23, 2004 9:49:59 PM PDT
Subject: Re: DAS2 server registry and discovery meeting, Friday at 10 AM

Steve:
> It would be helpful if someone could circulate notes summarizing
> the discussions, conclusions reached, and any action items (especially
> for us West-coast, non-morning types ;-).  Thanks!

Ughh!  I made it for the 10am BST conference call two days ago.
I'm still recovering ... :)

Here's my summary of the two conference calls.

The first day we went over the draft version of the protocol.
We made some minor changes, mostly to clear things up and fix
typos.  The biggest changes were in the document that returns
information about a data source, and we're getting rid of the
'sequences' section, to use 'region' instead.  (The document
had a bit of both.)

I think the only action item there is that Lincoln's going to
update the spec and redistribute it and I'll go through it with
my fine toothed comb.

This morning (my time) we talked about updates.  There are
many ways to do it.  One is via WebDAV, but none of us really
have any experience with it.  I've read the spec which made
me the most knowledgeable.  The thing about DAV is that it
requires several new HTTP level commands, like LOCK and
PROPFIND.  DAV seems to provide mechanism but not policy
and without some experience we decided to pass on it for
this version.

I personally would like to encourage people in the community
to experiment with that technology.

We talked about various ways to send updates, starting
with how people do it now.  I didn't follow this part well
enough to report it now but it seems that the client sends
basically a diff as one document.

The next topic was locking.  We can support optimistic
(CVS-style) or pessimistic (RCS-syle) locks.  In other words,
don't lock until it's time to commit changes vs. lock when
making the first change.  CVS-style is interesting, but
it makes lock conflicts harder to deal with.  We also
talked about tracking all change (just like a full-blown
version control system would) but decided again it was
too much to deal with now.

Though again I would like to encourage people to experiment
with, say, SVN as part of a back-end versioning system
layered on top of WebDAV.  The tricky parts are maintaining
internal consistency of all the independent branches and
handling branch merges in both client and server.

The conclusions in the locking discussion are:
   - pessimistic locking on a region specified by the curator.
      We will not support locks on a specific feature, feature
      type, etc.

   - the lock request is POSTed to a new URL, listed in the
       data source document and probably named something like
        .../lock/new

   - the request will include things like name (or is that
       provided via some other authentication means?), region
       to lock, and time for lock

   - if no error, the return include the lock URL (probably
       something like ".../lock/lock12345"), and length of
       time actually granted

   - a GET on the lock URL returns info about who locked it,
       the time remaining for the lock, and region

   - a POST of some sort to the URL is used to renew the lock,
       commit the changes or revert the transaction.  We think
       it should also be able to expand the locked region.

   - the data source document will also include a URL
       used to get the list of current locks

   - breaking locks is outside the scope of the spec.  Eg,
       it might only be done by people who can log into
       the database system.

There was also some discussion about how a client might
show to the curator which elements can be edited.  Currently
that's known through extra-protocol means (ie, by talking
to people and reading the documentation).  It might be
nice if the client tool could make some sort of indication.

We have not yet discussed how we're going to do the actual
editing.  It might be by PUT/DELETE calls on the elements,
or through a unified diff POSTed to the server which contains
a list of all the changes.

(Which reminds me, the DAV spec also support the HTTP
operations of MOVE and COPY.  These allow the versioning
system to keep version information even after splits and
renames.)

Did I omit or misstate anything?

					Andrew
					dalke at dalkescientific.com



From Steve_Chervitz at affymetrix.com  Tue Dec 14 01:33:38 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:33:38 -0800
Subject: [DAS2] Fwd: DAS2 meeting today (Sunday), 12:30-2:30
Message-ID: <2D91751F-4D70-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: "Helt,Gregg" <Gregg_Helt at affymetrix.com>
Date: September 25, 2004 11:55:45 PM PDT
Subject: DAS2 meeting today (Sunday), 12:30-2:30

Today we're having another in the ongoing series of DAS/2 meetings, from
12:30-2:30 PM in the Loft Room at the Wellcome Trust Conference Center.
I think we'll be covering a variety of topics with regard to the
specification, so BYOT (Bring Your Own Topic).

What I'm pretty sure we we will _not_ be covering is:
         registry and discovery of DAS/2 servers (already covered in
Saturday meeting, need to process results of that before discussing
again)
         writeback from DAS/2 clients to DAS/2 servers (there will be a
meeting Monday on this)
         implementation details of DAS/2 clients, servers, validation
suite (I want to keep focus on spec and logistics)

Topics I'd personally like to cover:
     Spec overview:
         General coherence
         Complexity comparison to DAS/1 and other alternatives
         Technology choices -- I'd like to have some coherent strategy
for explaining DAS/2 technology choices to others
                 For example, the decision to use URLs for most
identifiers rather than LSIDs
     Logistics:
         Timeline for meeting goals of grant
         Scheduling of next face-to-face meeting, as well as regular
teleconferences
         Summarize division of labor
         Getting feedback from bioinformatics community

Attendance is optional, but decisions _are_ getting made at these
meetings...

         thanks,
         gregg



From Steve_Chervitz at affymetrix.com  Tue Dec 14 01:34:37 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:34:37 -0800
Subject: [DAS2] Fwd: DAS2 meeting today (Sunday), 12:30-2:30
Message-ID: <50F49F32-4D70-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Andrew Dalke <dalke at dalkescientific.com>
Date: September 26, 2004 2:09:12 AM PDT
Subject: Re: DAS2 meeting today (Sunday), 12:30-2:30

BTW, when is Monday's meeting?

Gregg:
> Topics I'd personally like to cover:
> ????Spec overview:
> ????????General coherence

I found the spec somewhat confusing to read.  As written there are two
parts to the spec -- one an overview and the other the details of the
request.  But some of the details (like the use of /0 to mean the 
current
database, which I still don't like ;) are in the first part but not in
the second.  So there isn't one place to read to learn everything about
a given interface.

I would rather the overview section be much shorter and not contain
specification information.  I would also like it to include some
concrete information to show how it exists for a (hypothetical)
database.

   $entry                      -- main entry point to a DAS/2 server
   http://www.biodas.org/das2/das-genome

   $entry/$sourceid            -- data about a given source
   http://www.biodas.org/das2/das-genome/volvox

   $entry/$sourceid/$version  -- data about a given version of the source
   http://www.biodas.org/das2/das-genome/volvox/1

   $entry/$sourceid/$version/type -- description of various types of 
genomic features
   http://www.biodas.org/das2/das-genome/volvox/1

    ...

and perhaps with an abbreviation like $db = $entry/$sourceid/$version

The final version should support in-document hyperlinks to
the actual section.  That's a detail understandably left out
of the current version.

One topic I would like to revisit is the use of DTDs.  My
DAS/1 and my NCBI EUtils client both used DTDs to make parsers
for the data returned from the server.  That was part of
what made the client code easy to implement.  The problems
I ran into were 1) the old DTDs weren't correct, 2) DTDs
don't support non-text fields (like start/stop sequence
positions) and 3) the DTD-based parser I used was validating
and I ran into problems dealing with extensions and other
variances from the spec.

I want to investigate the use of RelaxNG instead.  That
supports datatypes, to say that the sequence start/stop
positions can be longs (64 bit integers).  It should help
handle these non-text fields better, so better for Java
people.  I might help with my validation. And I see there's
a way to annotate fields so it might be possible to
convert from a RelaxNG schema directly into documentation.

That may slide too much into implementation details.  I'll
restrict to the question "should the spec declare that DTDs
are going to be provided and will be considered normative?"


> ????????Complexity comparison to?DAS/1 and other alternatives

I only wrote a client to it, not a server.

Looking over my DAS/1 client code, I'm happy to see that we
no longer have "X-DAS-Capabilities".  My code basically ignored
it.  It also ignored the X-DAS-Version because just about
every server seemed to return a different version number.

The client-side implementation was pretty easy (except that
servers didn't meet the spec and several didn't even return
valid XML).

I don't know about other alternatives.

One of the fuzzy places in the spec is how to deal with issues
related to the underlying HTTP protocol.  I see that several
places didn't implement stylesheet so the HTTP request ended
up with a '404 Not Found' instead of using the DAS error code.

I see the DAS/1 spec talked about being able to leverage
things like content-encoding for sending compressed data back
and forth.  DAS/2 doesn't yet mention that.

> ????????Technology?choices -- I'd like to have some coherent
> ?strategy for explaining DAS/2 technology choices to others

What I just mentioned about interactions with HTTP might go
in this document.  I would also like to see comments like
"we considered WebDAV but because of its complexity and our
lack of experience ... " and "this locking scheme was chosen
because it is an easy one to conceptualize and implement.
Better possibilities exist, including ideas in source code
version control systems like CVS, subversion, and arch.
We hope people experiment with them to get experience that
may influence the next generation of DAS-like tools."

Okay, time for a 2.5hr nap before the meeting.  *yawn*


					Andrew
					dalke at dalkescientific.com




From Steve_Chervitz at affymetrix.com  Tue Dec 14 01:35:33 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:35:33 -0800
Subject: [DAS2] Fwd: thoughts on DAS/2 validation
Message-ID: <724755F0-4D70-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Andrew Dalke <dalke at dalkescientific.com>
Date: September 26, 2004 11:57:46 PM PDT
Subject: thoughts on DAS/2 validation

> ??writeback from DAS/2 clients to DAS/2 servers (there will be a 
> meeting Monday on this)
> ??implementation details?of DAS/2 clients, servers, validation suite 
> (I want to
>        keep focus on spec and logistics)

I don't think I need to attend that writeback meeting.  I
contributed my ideas during the locking/transaction meeting
from a couple of days ago, along with my followup email.

I want to mention a bit about my tentative plans for testing
and validation.

I've mentioned that I wanted to investigate using RelaxNG (RNG)
for the validation.  (www.relaxng.org).  I did that this evening.

Short history.  XML originally used DTDs for schema validation.
DTDs have various problems.  It doesn't handle data types (eg,
'start' must be a number) or unordered collections.  XML Schema
was supposed to improve on DTDs but it's a complicated standard
with lots of bits and pieces added to it in committee.

James Clark and a small group of other people worked on a few
alternates, which merged to become RNG.  This is definitely
easier to understand and use and (IMNSHO) more elegant than
XML Schema.

RNG has two representations, one in XML and the other using
a "compact" syntax which is easier for humans to work with.
The first one uses the filename suffix ".rng" and the
latter ".rnc".

There is a program called "trang" to convert between these
two formats.  It can also read/write to DTDs and write to
XML Schema.  Because it's so easy, I withdraw my complaint
about using DTDs.

To make things even more fun, there's a program called
"Examplotron" which is an XSLT (!) program to convert an
example XML document into an RNG for it.

I tested it out with the SOURCES example.  Here's the XML
from the spec, but without the DTD

<?xml version="1.0" standalone="no"?>
<SOURCES
     xmlns="http://www.biodas.org/ns/das-genome/2.00"
     xmlns:xlink="http://www.w3.org/1999/xlink"
     xml:base="http://dev.wormbase.org/das-genome/">
   <SOURCE id="volvox" description="Volvox Example Database"
           taxon="http://www.ncbi.nlm.nih.gov/taxon-browser?id=29118">
      <VERSION id="volvox/1" description="Build 1, October 2002" />
      <VERSION id="volvox/2" description="Build 2, January 2003" />
   ... truncated ...

Here's the RNG for it.  The "ega:example" is used to keep the
original data around.  This makes the RNG invertible back to
the example XML.  RNG validation ignores elements outside its
namespace.  This lets people add new fields, including documentation.
There's even software to turn an RNG plus documentation into
DocBook format.

<grammar xmlns="http://relaxng.org/ns/structure/1.0" 
xmlns:ega="http://examplotr
on.org/annotations/" xmlns:sch="http://www.ascc.net/xml/schematron" 
datatypeLibr
ary="http://www.w3.org/2001/XMLSchema-datatypes">
    <start>
       <element name="SOURCES" 
ns="http://www.biodas.org/ns/das-genome/2.00">
          <optional>
             <attribute name="base" 
ns="http://www.w3.org/XML/1998/namespace">
                <ega:example 
xml:base="http://dev.wormbase.org/das-genome/"/>
             </attribute>
          </optional>
          <oneOrMore>
             <element name="SOURCE">
                <optional>
                   <attribute name="id">
                      <ega:example id="volvox"/>
                   </attribute>
                </optional>
   ... truncated ...


The compact form

namespace ega = "http://examplotron.org/annotations/"
default namespace ns1 = "http://www.biodas.org/ns/das-genome/2.00"
namespace sch = "http://www.ascc.net/xml/schematron"
namespace xlink = "http://www.w3.org/1999/xlink"

start =
   element SOURCES {
     [ ega:example [ xml:base = "http://dev.wormbase.org/das-genome/" ] ]
     attribute base { text }?,
     (element SOURCE {
        [ ega:example [ id = "volvox" ] ] attribute id { text }?,
        [ ega:example [ description = "Volvox Example Database" ] ]
   ... truncated ...


and DTD

<!ELEMENT ns1:SOURCES (SOURCE)+>
<!ATTLIST ns1:SOURCES
   xmlns:ns1 CDATA #FIXED 'http://www.biodas.org/ns/das-genome/2.00'
   xml:base CDATA #IMPLIED>

<!ELEMENT ns1:SOURCE (VERSION)+>
<!ATTLIST ns1:SOURCE
   xmlns:ns1 CDATA #FIXED 'http://www.biodas.org/ns/das-genome/2.00'
   id CDATA #IMPLIED
   description CDATA #IMPLIED
   taxon CDATA #IMPLIED>
   ... truncated ...

What this means is that I should be able to get a working schema
up rather quickly.  This would then be used by the validator.

         ==========================

Here's my plan.  I'll continue to help with the spec in
finding places that need to be clarified or tweaked.  The
DAS/1 spec contained typos in the examples so I'll make sure
there's some way to extract all examples from the spec
so they can be tested.

I'll develop an RNG schema and come up with a set of test
documents, some legal and some illegal, to make sure the validator
is able to catch and report errors properly.

These sample documents will likely consist of two files, one for
the XML and one the HTTP headers.  This will let me run the
regression tests without a server.

I'll come up with a set of tests for data integrity.  For example,
get a sequence from a range and make sure the FASTA length is of
the right size.  Get all of the features in a range and test that
the server's filters work correctly.

The writeback tests will be harder.  I'll do a set of adds,
deletes, and modifies.  If we have transaction support on the
server I'll test that.  I'll test for proper lock semantics.

This is harder because it requires I be able to scribble on
a server's database, and that the code be given username/password
for how to get write permissions.  More likely two, to make
sure that locking works correctly.

What I would really like is be able to provide a known
data set that can be imported by a server specifically for
testing.  Then I could include tests that would know what
the answer is supposed to be, rather than simply testing
for internal integrity.  Some tests would include being able
to search for a feature with an attribute that includes a
& or / in it, and make sure that range searches don't all
have off-by-one errors.

This requires there be data files in a format easily
imported into a server.  Any thoughts?  I know Lincoln's
DAS/1 perl server had something like that but the file
format wasn't well defined (at least for my picky tastes ;)

As I recall, writeback support wasn't going to be until
year 2 of the grant so the testing for this year should
only be for format validation and basic integrity checks.


					Andrew
					dalke at dalkescientific.com




From Steve_Chervitz at affymetrix.com  Tue Dec 14 01:44:45 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:44:45 -0800
Subject: [DAS2] Fwd: DAS Writeback Summary
Message-ID: <BB42837B-4D71-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Ed Griffiths <edgrif at sanger.ac.uk>
Date: September 28, 2004 5:43:56 AM PDT
Subject: Todays meeting.

All,

I have attached a summary and a fuller set of notes about where we seem  
to have
got to. [inline below]

There is a lot of work to do in formalising and filling out the details  
but we
seem to be making some progress.

Ed
--  
   
------------------------------------------------------------------------
| Ed Griffiths, Acedb development, Informatics Group,                    
  |
|        Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus,  
  |
|               Hinxton, Cambridge CB10 1SA, UK                          
  |
|                                                                        
  |
| email: edgrif at sanger.ac.uk  Tel: +44-1223-494780  Fax:  
+44-1223-494919 |
   
------------------------------------------------------------------------


An aside:

we need to sort out the mailing lists, its clear that there is a lot I  
and
others haven't seen. Should we be using a restricted mailing list or....

actions: who is going to put the DAS2 spec together as a result of all  
these
meetings and by when ?

Is there a reference server set up that we can all try or copy to our  
own
locations ?


In this write up I've marked with a ** some of the issues we should  
discuss.


=================================================================
DAS 2.0 Write Back

HTTP protocol issues
--------------------

- we have decided not to use webdav in this DAS2 version.

- we need to decide how our DAS2 requests map to the http  
GET/POST/PUT/DELETE
commands.

e.g. we have not yet discussed how we're going to do the actual  
editing.  It might
be by PUT/DELETE calls on the elements, or through a unified diff  
POSTed to
the server which contains a list of all the changes.

- in particular we need to decide what we pass between server and  
client in the
way of authentification and state (e.g. locks).



Locking
-------

- we have decided that we will only allow locking on regions, not  
features.
The client can specify start/end coords for regions.

- Note that this implies that we only lock _locatable_ objects, this is
something we need to explore further, e.g. how do "types" get editted,  
how do
genes that not located get editted etc. etc. ?

- we decided that unless an object is fully contained within the locked  
region
then it is not locked. If the user wants to edit that object then they  
have to
extend the region that they want to lock to include that object.


Lock operations

- the lock request is POSTed to a new URL, listed in the data source  
document
and probably named something like .../lock/new. The request will include
things like name (or is that provided via some other authentication  
means?),
region to lock, and time for lock. If no error, the return includes the  
lock
URL (probably something like ".../lock/lock12345"), and length of time
actually granted.

- a GET on the lock URL returns info about who locked it, the time  
remaining
for the lock, and region

- There should be a request that allows the client to find out from the  
server
which regions are already locked so that annotators can be given this
information before they request a lock.  (the data source document will  
also
include a URL used to get the list of current locks)

- There will be a request which allows the client to find out which  
types are
edittable on a per user basis, this will allow the annotator to see in  
advance
what they are allowed/able to edit.

- There will have to be authorisation for each lock/edit operation. It  
was
assumed that all this will be straight forward as database  
authorisation and
authorisation via http are known and solved problems.

- a POST of some sort to the URL is used to renew the lock, commit the  
changes
or revert the transaction.  We think it should also be able to expand  
the
locked region.

- breaking locks is outside the scope of the spec.  Eg, it might only  
be done
by people who can log into the database system.



Versioning (but see later)
----------

We also talked about tracking all change (just like a full-blown version
control system would) but decided again it was too much to deal with  
now.

** What we do need to decide is on how a client asks for a specific  
version of
** a feature, or even if this is possible ?


Timeouts
--------

- locks will have timeouts (when a lock is granted, the response will  
tell
the client how long the lock will be held for).

- the timeout period is for "idle" time, not total time.


Transactions
------------
- it was agreed that transactions should be transparent to the client so
whether the database uses them to commit data will not be visible. This
implies that our protocol for updating must be via single, "atomic",  
http
requests.

- should be providing a mechanism for a client to verify that changes  
have
been successfully committed in this case ?


Proxies
-------

- Andrews investigations imply that caching will not be the problem we  
thought
it might be.

- But one thing to consider though is how we're going to do user
authentication.  Use an existing HTTP username/password scheme or  
something
outside of that?  If ours is independent of HTTP then we have to be more
careful about how to deal with caches.

If ours is based on HTTP and we want to play nice with caches then  
clients
shouldn't include password information to publicly accessible data as  
that
won't be cached.


Data issues
-----------

- versioning of features is currently responsibility of the database  
but we
may need to provide something to say "give me the latest version" ?

- the client can find out which feature types can be editted for any one
user and hence can show this to the annotator.

- the annotator can edit features on a sequence but not the sequence  
assembly
itself.



From Steve_Chervitz at affymetrix.com  Tue Dec 14 01:48:25 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:48:25 -0800
Subject: [DAS2] Fwd: DAS Write Followup
Message-ID: <3E81AE89-4D72-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Ed Griffiths <edgrif at sanger.ac.uk>
Date: September 28, 2004 5:43:56 AM PDT
Subject: Todays meeting.

All,

I have attached a summary and a fuller set of notes about where we seem 
to have
got to. [inline below]

There is a lot of work to do in formalising and filling out the details 
but we
seem to be making some progress.

Ed
-- 

[The following is Ed's "DAS-WRITE-FOLLOWUP.email" document.]

================================================================
DAS 2.0 Write Back


HTTP protocol issues
--------------------

There was an idea that we would try to map the basic http GET, POST, 
DELETE
etc.  to meaningful writeback/locking actions, we didn't discuss this in
detail except to say that no one was keen to artificially shoe-horn 
write-back
operations into this framework.

** We need to sort out which operations are GET/POST/PUT/DELETE.

We have not yet discussed how we're going to do the actual editing.  It 
might
be by PUT/DELETE calls on the elements, or through a unified diff 
POSTed to
the server which contains a list of all the changes.

(Which reminds me, the DAV spec also support the HTTP operations of 
MOVE and
COPY.  These allow the versioning system to keep version information 
even
after splits and renames.)



There was some discussion of how you implement write-back, which 
requires
state, within the stateless http protocol. Thomas raised webdav as a 
possible
way to go. Gregg said that Brian (surname ?) has investigated this to 
some
extent but is not currently working on it. No one else has great 
experience of
it but Thomas said that it provides a protocol on top of http and this 
made
the idea of using unpalatable because it would mean that all clients 
would
have to use it if they want to talk DAS2. Andrew added that "The thing 
about
DAV is that it requires several new HTTP level commands, like LOCK and
PROPFIND.  DAV seems to provide mechanism but not policy and without 
some
experience we decided to pass on it for this version."

The concensus was that we should not use webdav but produce our own 
protocol
for doing write-back.



Locking
-------

What should be locked ?

After quite a bit of discussion about the granularity of locking that 
should
be allowed the concensus was that the system would become far too 
complex (=
unusable/unimplementable) unless we went for a system like Otter where 
locking
was on the basis of regions. The client would ask for a lock on a 
region,
giving start/end coords and every feature located in that region would 
be
locked.

(Note that this implies that we only lock _locatable_ objects, this is
something we need to explore further, e.g. how do "types" get editted, 
how do
genes that not located get editted etc. etc. ?)

The policy that James adopts and seemed sensible to everyone was that 
unless
an object is fully contained within the locked region then it is not
locked. If the user wants to edit that object then they have to extend 
the
region that they want to lock to include that object.


Lock operations

There should be an operation that allows the client to find out from the
server which regions are already locked so that annotators can be given 
this
information before they request a lock.

We discussed when the client should get the lock, i.e. before the 
editting
starts or once the user has a set of edits they want to save. There were
strong advocates for both models but in the end it was thought that 
locking in
advance is better otherwise an annotator may spend some time editting 
only to
discover that they cannot save their edits, i.e. lock conflicts are 
hard to
deal with without locking in advance.


We also need a request which allows the client to find out which types 
are
edittable on a per user basis, this will allow the annotator to see in 
advance
what they are allowed/able to edit.

We discussed authorisation, which given that we are using http will 
have to be
done for each lock/edit operation. It was assumed that all this will be
straight forward as database authorisation and authorisation via http 
are
known and solved problems.


The conclusions in the locking discussion are:
    - pessimistic locking on a region specified by the curator.
       We will not support locks on a specific feature, feature
       type, etc.

    - the lock request is POSTed to a new URL, listed in the
        data source document and probably named something like
         .../lock/new

    - the request will include things like name (or is that
        provided via some other authentication means?), region
        to lock, and time for lock

    - if no error, the return include the lock URL (probably
        something like ".../lock/lock12345"), and length of
        time actually granted

    - a GET on the lock URL returns info about who locked it,
        the time remaining for the lock, and region

    - a POST of some sort to the URL is used to renew the lock,
        commit the changes or revert the transaction.  We think
        it should also be able to expand the locked region.

    - the data source document will also include a URL
        used to get the list of current locks

    - breaking locks is outside the scope of the spec.  Eg,
        it might only be done by people who can log into
        the database system.


Versioning (but see later)

We also talked about tracking all change (just like a full-blown version
control system would) but decided again it was too much to deal with 
now.

** What we do need to decide is on how a client asks for a specific 
version of
** a feature, or even if this is possible ?


Timeouts

Clearly we need to be able to timeout locks: clients may disappear 
without the
server being aware of this.

Andrew suggested that when the client requests a lock, the response 
will say
how long the lock will last and also returns a url which the client can 
use to
query/unlock the lock. Everyone agreed this seemed a sound idea.

It was agreed that the "timeout" period applies to how long the client 
has
been idle, not how long in total it has been connected. The lock url 
could be
used by the client to signal that it was still active without the need 
for
some artificial database operation.



Transactions

There was some discussion about whether these are necessary at all. 
James uses
transactions to write the annotators data back into the database and 
said that
this was a vital element as it enabled the annotator to be certain that 
their
edits had made it into the database.

In James system transactions are transparent to the client, they issue 
a save
and the save is done via a transaction. There is no prolonged 
transaction
spanning several http requests. This was seen as a good model otherwise 
the
client will have to understand too much of the locking/transaction 
model of
the underlying database.

I think everyone agreed that write back should be done via a single 
request so
that transactions did not get spread across multiple http request which 
looks
intractable.



Proxies

There is a problem with writing back data when there are proxy servers 
between
the client and the database. I'm not qualified to explain all of this, 
perhaps
Tony or someone could fill this section in. I assume the problem 
relates to
making sure the proxy actually sends the data back to the database and 
doesn't
cache it causing timeout problems etc.

Andrew said:

"It's the other way around.  A proxy may cache GET requests, because a 
GET is
not supposed to affect the server.  A POST to some other URL may affect 
the
first URL but a proxy won't know there was a change.

I mentioned this might be a problem.  I just did some research and 
found this
overview, http://www.mnot.net/cache_docs/

It describes when proxies are allowed to think something is cacheable.


1.  If the response's headers tell the cache not to keep it, it won't.
2.  If no validator (an ETag or Last-Modified header) is present
    on a response, it will be considered uncacheable.
3.  If the request is authenticated or secure, it won't be cached.
4.  A cached representation is considered fresh (that is, able to
    be sent to a client without checking with the origin server) if:
    o It has an expiry time or other age-controlling header set,
       and is still within the fresh period.
    o If a browser cache has already seen the representation, and
       has been  set to check once a session.
    o If a proxy cache has seen the representation recently, and it
       was modified relatively long ago.
   Fresh representations are served directly from the cache, without
   checking with the origin server
5. If an representation is stale, the origin server will be asked
   to validate it, or tell the cache whether the copy that it has
   is still good.

So it looks like we won't stumble into cache problems by accident.  
Someone
has to enable a validator tag before the cache comes into play.

One thing to consider though is how we're going to do user 
authentication.
Use an existing HTTP username/password scheme or something outside of 
that?
If ours is independent of HTTP then we have to be more careful about 
how to
deal with caches.

If ours is based on HTTP and we want to play nice with caches then 
clients
shouldn't include password information to publicly accessible data as 
that
won't be cached.

But I wouldn't worry about this.  It's most theoretical and best to 
wait until
it becomes an actual problem.


Data issues
-----------

We discussed version numbering of objects. James has unique ids for all
edittable objects with version numbers as well. This enables him to keep
numbered versions of an object and retrieve the latest version for 
editting
but also to go back to previous versions if required. It may be that we 
need
the client to be able to ask for previous versions.

How does the client know which things can be editted ?

In James system James system there are edittable and non-edittable data
streams.  It was felt that this is not flexible enough for DAS. 
Therefore the
client needs to be able to discover what objects within a region can be
editted.

Current suggestion is that when the server returns what types it can 
serve,
the type should include a flag to say whether this type from this 
server can
be editted. If the type can be editted then this says the annotator can 
edit
_all_ objects of that type from that server. This would obviate the 
need for
every object to be flagged with whether it was edittable, something 
that none
of us thought was a good idea. This mechanism will enable the client to 
show
the annotator which objects are edittable (note that the set of objects 
that
can be editted could change from user to user depending on their 
database
permissions).



How would the client lock stuff in practice ?

1) The annotator needs to know up front what they can edit so the 
client would
be able to issue a request to the server(s) which would tell it what 
objects
were edittable.

(Andrew made a suggestion here that said something like: two urls are 
supplied
for database information for POST, there are urls for features/regions, 
server
will say "yes I have a lock, no I don't" makes it extensible can anyone
remember what he said in more detail ?)


2) The annotator then indicates what they want to lock and the client 
issues a
request to the server that says "Am I allowed to lock these actual 
objects ?",
server replies yes or no.

3) If stage 2) was successful then the client can go on to issue a 
request to
actually lock the data.


So the mechanism that seemed to make sense was for the server to serve 
up
everything with _no_ indication at the object level whether something is
edittable. The client can then discover which of the objects it is 
displaying
can be editted.

It was agreed that the annotator would be allowed to edit objects 
located on
an assembly but not actually edit the assembly itself.  There is a clear
distinction between moving features around an assembly and actually 
changing
the assembly itself !

Conclusion
----------

We covered a lot of ground in the meeting and a lot of important points 
were
discussed. There are still a lot more things to firm up. There will be 
at
least one or two meetings today and over the weekend to do this. Gregg 
and Ed
can meet on Monday morning to try to write all this up so that we have a
proposal that everyone can then shoot down/applaud.



From Steve_Chervitz at affymetrix.com  Tue Dec 14 01:50:27 2004
From: Steve_Chervitz at affymetrix.com (Steve_Chervitz)
Date: Mon, 13 Dec 2004 17:50:27 -0800
Subject: [DAS2] Fwd: Re: DAS Writeback Summary
Message-ID: <8786CB46-4D72-11D9-9358-000A95765236@affymetrix.com>


Begin forwarded message:

From: Suzanna Lewis <suzi at fruitfly.org>
Date: September 29, 2004 11:22:23 AM PDT
Subject: Re: Todays meeting.

Hi Ed,

I could not read these for some reason (admittedly I didn't struggle)
but was wondering if you could send them as plain text.

I was also wondering if for a feature's authorization we could model
it like file system. World, group, and user with read and write  
permissions.
Just a whimsy, but might serve to help.

-S

On Sep 28, 2004, at 5:43 AM, Ed Griffiths wrote:

> All,
>
> I have attached a summary and a fuller set of notes about where we  
> seem to have
> got to.
>
> There is a lot of work to do in formalising and filling out the  
> details but we
> seem to be making some progress.
>
> Ed
> --   
> ----------------------------------------------------------------------- 
> -
> | Ed Griffiths, Acedb development, Informatics Group,                   
>   |
> |        Wellcome Trust Sanger Institute, Wellcome Trust Genome  
> Campus,  |
> |               Hinxton, Cambridge CB10 1SA, UK                         
>   |
> |                                                                       
>   |
> | email: edgrif at sanger.ac.uk  Tel: +44-1223-494780  Fax:  
> +44-1223-494919 |
>   
> ----------------------------------------------------------------------- 
> -
>
> <DAS-WRITEBACK-SUMMARY.email><DAS-WRITE-FOLLOWUP.email>



From Steve_Chervitz at affymetrix.com  Fri Dec 17 02:17:10 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Thu, 16 Dec 2004 18:17:10 -0800
Subject: [DAS2] FW: Semantic web position paper
In-Reply-To: <BDCA8136.26FE%Steve_Chervitz@affymetrix.com>
Message-ID: <BDE781A6.3118%Steve_Chervitz@affymetrix.com>



------ Forwarded Message
From: <Chervitz>, Steve <Steve_Chervitz at affymetrix.com>
Date: Wed, 24 Nov 2004 18:19:34 -0800
Subject: Semantic web position paper

Folks,

I attended a W3C workshop on the semantic web for life sciences last month
and wrote a position paper that discusses issues for the semantic web from a
DAS point of view. A big part of the discussion centers around LSID.

You can find the paper on this page, just do an in-page search for
'chervitz':
http://www.w3.org/2004/07/swls-agenda.html

Others at the meeting raised similar issues with LSID, but I'd say that my
paper had the most pointed assault (thanks in large part to Andrew). If you
have any comments or other thoughts, let me know.

Steve





From Steve_Chervitz at affymetrix.com  Fri Dec 17 02:22:12 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Thu, 16 Dec 2004 18:22:12 -0800
Subject: [DAS2] FW: Semantic web position paper
In-Reply-To: <BDE781A6.3118%Steve_Chervitz@affymetrix.com>
Message-ID: <BDE782D4.311A%Steve_Chervitz@affymetrix.com>


The W3C's report of the Workshop on Semantic Web for Life Sciences is now
online.  All position papers and presentations are archived in the report.

http://www.w3.org/2004/10/swls-workshop-report.html

Steve


> ------ Forwarded Message
> From: <Chervitz>, Steve <Steve_Chervitz at affymetrix.com>
> Date: Wed, 24 Nov 2004 18:19:34 -0800
> Subject: Semantic web position paper
> 
> Folks, 
> 
> I attended a W3C workshop on the semantic web for life sciences last month
> and wrote a position paper that discusses issues for the semantic web from a
> DAS point of view. A big part of the discussion centers around LSID.
> 
> You can find the paper on this page, just do an in-page search for
> 'chervitz': 
> http://www.w3.org/2004/07/swls-agenda.html
> 
> Others at the meeting raised similar issues with LSID, but I'd say that my
> paper had the most pointed assault (thanks in large part to Andrew). If you
> have any comments or other thoughts, let me know.
> 
> Steve 
> 
> 
> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2 



From Steve_Chervitz at affymetrix.com  Fri Dec 17 20:22:11 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Fri, 17 Dec 2004 12:22:11 -0800
Subject: [DAS2] RSS in Science Publishing
In-Reply-To: <E1CfKoB-0005w0-Ak@frink.w3.org>
Message-ID: <BDE87FF3.317E%Steve_Chervitz@affymetrix.com>

Here's an interesting article:

http://www.dlib.org/dlib/december04/hammond/12hammond.html

It makes me think about the applicability of RSS to DAS. DAS is in fact a
type of content syndication. Wouldn't it be cool if you could hook up an
off-the-shelf RSS reader to a DAS server and get an RSS feed for a specified
genomic region or feature or all features or a particular type?

Here's a DAS-RSS scenario: A curator adds or updates an annotation on a DAS
server, the server determines whether the modified region/feature/type is
syndicated and, if so, publishes an RSS entry describing what was changed
(or simply, "region/feature/type foo was changed"). The RSS entry would
contain a URL encoding a DAS request for the region/feature/type, which one
could then open in a DAS client for further exploration. Maybe the DAS
client itself would be the RSS reader -- presenting the user with a window
listing RSS entry titles and links to browse them graphically.

Perhaps more useful would be a DAS-RSS aggregator that listens to multiple
DAS servers and consolidates them so that a user could get RSS notifications
from multiple servers for a given region/feature/type. The DAS/RSS client
could provide this aggregation function, letting the user choose which
servers and data sources they want to aggregate.

The usefulness here is limited by the fact that you only get notified about
things that changed. However, one could in principle also hook the DAS-RSS
client into the world of scientific publications by pointing it at a
scientific publisher's RSS feed. When an article about
organism/region/feature/type of interest gets published, the client
generates the appropriate DAS request URL(s) so you can browse them in the
graphical viewer. 

If publishers include appropriate metadata about the articles using standard
ontologies and taxonomies, this scenario may not be such a leap. The "if" in
the previous statement may not be that big. The presence of Nature,
Elsevier, and the NLM at the W3C meeting indicates that publishers are
catching on.

I don't know how easy it would be to RSS-ifying the DAS spec. Doing this
might be too radical of a change from DAS/1. It might be better to layer the
DAS-RSS functionality on top of DAS, so it could work with both DAS/2 and
DAS/1 servers. Thoughts?

Steve

------ Forwarded Message
From: Ben Lund <b.lund at nature.com>
Date: Fri, 17 Dec 2004 08:18:23 -0800
To: <public-semweb-lifesci at w3.org>
Subject: Announcement:  D-Lib article on RSS in Science Publishing

Hi All, 

Apologies if any of you have seen this already, but I just wanted to
point you towards our new paper published a couple of days ago in D-Lib
Magazine (http://www.dlib.org/).  It goes into more depth on a number of
the issues that I talked about at the Semantic Web for Life Sciences
workshop in October:

The Role of RSS in Science Publishing:
Syndication and Annotation on the Web

by Tony Hammond, Timo Hannay and Ben Lund,
Nature Publishing Group

http://www.dlib.org/dlib/december04/hammond/12hammond.html

Hopefully it'll be of interest.

### 

D-Lib Magazine is a solely electronic publication with a primary focus
on digital library research and development, including but not limited
to new technologies, applications, and contextual social and economic
issues. D-Lib Magazine is produced by the Corporation for National
Research Initiatives (CNRI), has been sponsored by the Defense Advanced
Research Project Agency (DARPA), and is currently being funded by the
National Science Foundation (NSF).

### 

Ta, 
Ben 

--Ben Lund 
--New Technology 
--Nature Publishing Group

--- 



****************************************************************************
**** 
DISCLAIMER: This e-mail is confidential and should not be used by anyone who
is not the original intended recipient. If you have received this e-mail in
error please inform the sender and delete it from your mailbox or any other
storage mechanism. Neither Macmillan Publishers Limited nor any of its
agents accept liability for any statements made which are clearly the
sender's own and not expressly made on behalf of Macmillan Publishers
Limited or one of its agents. Please note that neither Macmillan Publishers
Limited nor any of its agents accept any responsibility for viruses that may
be contained in this e-mail or its attachments and it is your responsibility
to scan the e-mail and attachments (if any). No contracts may be concluded
on behalf of Macmillan Publishers Limited or its agents by means of e-mail
communication. Macmillan Publishers Limited Registered in England and Wales
with registered number 785998 Registered Office Brunel Road, Houndmills,
Basingstoke RG21 6XS

****************************************************************************
**** 

------ End of Forwarded Message



From Steve_Chervitz at affymetrix.com  Sat Dec 18 01:35:20 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Fri, 17 Dec 2004 17:35:20 -0800
Subject: [DAS2] Content type for DAS XML responses
Message-ID: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>

Currently, the DAS/2 spec dictates that responses should have content-types
like:

  application/x-das-source
  application/x-das-types
  application/x-das-region

Changing these to be 'text/xml' would be more convenient since the content
would then be viewable in a standard web browser. This would also be good
for backward compatibility with DAS/1 (where content-type is text/plain).

If we wanted, we could provide auxiliary information using attribute=value
notation:

  text/xml; dasformat=source
  text/xml; dasformat=types
  text/xml; dasformat=region

The corresponding compact content-types could be:

  text/plain; dasformat=compact-source
  text/plain; dasformat=compact-types
  text/plain; dasformat=compact-region

And other types could be:

  text/plain; dasformat=fasta
  text/plain; dasformat=gff3
  text/plain; dasformat=bed

Objections?

Steve

Refs: 
http://www.w3.org/TR/xhtml-media-types/#text-xml
http://www.iana.org/assignments/media-types/





From allenday at ucla.edu  Sat Dec 18 02:40:14 2004
From: allenday at ucla.edu (Allen Day)
Date: Fri, 17 Dec 2004 18:40:14 -0800 (PST)
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
Message-ID: <Pine.LNX.4.58.0412171838370.32735@sumo.ctrl.ucla.edu>



On Fri, 17 Dec 2004, Chervitz, Steve wrote:

> Currently, the DAS/2 spec dictates that responses should have content-types
> like:
> 
>   application/x-das-source
>   application/x-das-types
>   application/x-das-region
> 
> Changing these to be 'text/xml' would be more convenient since the content
> would then be viewable in a standard web browser. This would also be good
> for backward compatibility with DAS/1 (where content-type is text/plain).
> 
> If we wanted, we could provide auxiliary information using attribute=value
> notation:
> 
>   text/xml; dasformat=source
>   text/xml; dasformat=types
>   text/xml; dasformat=region

are browsers going to choke on this?  will browsers render text/x-das-* as 
text instead of prompting with "Save As..."?

> 
> The corresponding compact content-types could be:
> 
>   text/plain; dasformat=compact-source
>   text/plain; dasformat=compact-types
>   text/plain; dasformat=compact-region
> 
> And other types could be:
> 
>   text/plain; dasformat=fasta
>   text/plain; dasformat=gff3
>   text/plain; dasformat=bed
> 
> Objections?

Certainly one of these is better than what we currently have.

-Allen

> 
> Steve
> 
> Refs: 
> http://www.w3.org/TR/xhtml-media-types/#text-xml
> http://www.iana.org/assignments/media-types/
> 
> 
> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2
> 


From dalke at dalkescientific.com  Sat Dec 18 05:44:35 2004
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Fri, 17 Dec 2004 22:44:35 -0700
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
Message-ID: <E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>

Steve:
> Currently, the DAS/2 spec dictates that responses should have 
> content-types
> like:
>
>   application/x-das-source
>   application/x-das-types
>   application/x-das-region
>
> Changing these to be 'text/xml' would be more convenient since the 
> content
> would then be viewable in a standard web browser.

A browser should send something like

Accept: */*
   or
Accept: application/xml

or some other generic content type in its headers.  One solution
to at least consider is that IF a client does not request
application/x-das-whatever THEN return application/xml .  I do
not believe this is appropriate though.


>   text/xml; dasformat=source
>   text/xml; dasformat=types
>   text/xml; dasformat=region

I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )

    If an XML document -- that is, the unprocessed, source XML document
    -- is readable by casual users, text/xml is preferable to
    application/xml.  MIME user agents (and web user agents) that do not
    have explicit support for text/xml will treat it as text/plain, for
    example, by displaying the XML MIME entity as plain text.
    Application/xml is preferable when the XML MIME entity is unreadable
    by casual users.

That RFC is relevant to this discussion.  If I understand it
correctly it recommends using something more like

   text/x-das-source+xml

This would be understood by RFC 3023 compliant clients as
containing an XML document, by DAS clients to understand
which DAS XML format is used, and by generic clients to
contain an XML document.

> The corresponding compact content-types could be:
>
>   text/plain; dasformat=compact-source
>   text/plain; dasformat=compact-types
>   text/plain; dasformat=compact-region

There should be no need for this.  A client should use
the catch-all text/* handler for text/x-compact-source
and show it as normal text.

> And other types could be:
>
>   text/plain; dasformat=fasta
>   text/plain; dasformat=gff3
>   text/plain; dasformat=bed

Same for these.  What's the problem with
   text/x-fasta , etc.?

See also the discussion in appendix A.5 which describes
practical problems about using parameter in MIME types.

					Andrew
					dalke at dalkescientific.com



From allenday at ucla.edu  Sat Dec 18 07:14:54 2004
From: allenday at ucla.edu (Allen Day)
Date: Fri, 17 Dec 2004 23:14:54 -0800 (PST)
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
	<E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
Message-ID: <Pine.LNX.4.58.0412172314240.6215@sumo.ctrl.ucla.edu>

sounds good, can we get a patch to the spec so i can make the necessary 
implementation changes and test against browsers?  thanks!

-allen


On Fri, 17 Dec 2004, Andrew Dalke wrote:

> Steve:
> > Currently, the DAS/2 spec dictates that responses should have 
> > content-types
> > like:
> >
> >   application/x-das-source
> >   application/x-das-types
> >   application/x-das-region
> >
> > Changing these to be 'text/xml' would be more convenient since the 
> > content
> > would then be viewable in a standard web browser.
> 
> A browser should send something like
> 
> Accept: */*
>    or
> Accept: application/xml
> 
> or some other generic content type in its headers.  One solution
> to at least consider is that IF a client does not request
> application/x-das-whatever THEN return application/xml .  I do
> not believe this is appropriate though.
> 
> 
> >   text/xml; dasformat=source
> >   text/xml; dasformat=types
> >   text/xml; dasformat=region
> 
> I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
> RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )
> 
>     If an XML document -- that is, the unprocessed, source XML document
>     -- is readable by casual users, text/xml is preferable to
>     application/xml.  MIME user agents (and web user agents) that do not
>     have explicit support for text/xml will treat it as text/plain, for
>     example, by displaying the XML MIME entity as plain text.
>     Application/xml is preferable when the XML MIME entity is unreadable
>     by casual users.
> 
> That RFC is relevant to this discussion.  If I understand it
> correctly it recommends using something more like
> 
>    text/x-das-source+xml
> 
> This would be understood by RFC 3023 compliant clients as
> containing an XML document, by DAS clients to understand
> which DAS XML format is used, and by generic clients to
> contain an XML document.
> 
> > The corresponding compact content-types could be:
> >
> >   text/plain; dasformat=compact-source
> >   text/plain; dasformat=compact-types
> >   text/plain; dasformat=compact-region
> 
> There should be no need for this.  A client should use
> the catch-all text/* handler for text/x-compact-source
> and show it as normal text.
> 
> > And other types could be:
> >
> >   text/plain; dasformat=fasta
> >   text/plain; dasformat=gff3
> >   text/plain; dasformat=bed
> 
> Same for these.  What's the problem with
>    text/x-fasta , etc.?
> 
> See also the discussion in appendix A.5 which describes
> practical problems about using parameter in MIME types.
> 
> 					Andrew
> 					dalke at dalkescientific.com
> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2
> 


From lstein at cshl.edu  Mon Dec 20 17:04:23 2004
From: lstein at cshl.edu (Lincoln Stein)
Date: Mon, 20 Dec 2004 12:04:23 -0500
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
	<E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
Message-ID: <200412201204.23423.lstein@cshl.edu>

> Same for these.  What's the problem with
>    text/x-fasta , etc.?

I like this solution. I don't like returning text/xml because there is 
a growing tendency of browsers to try to parse the XML, returning 
uninterpretable results.

Lincoln

-- 
Lincoln D. Stein
Cold Spring Harbor Laboratory
1 Bungtown Road
Cold Spring Harbor, NY 11724

NOTE: Please copy Sandra Michelsen <michelse at cshl.edu> on
all emails regarding scheduling and other time-critical topics.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20041220/932350c8/attachment.sig>

From dalke at dalkescientific.com  Mon Dec 20 17:44:49 2004
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 20 Dec 2004 10:44:49 -0700
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <200412201204.23423.lstein@cshl.edu>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
	<E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
	<200412201204.23423.lstein@cshl.edu>
Message-ID: <D88F7C5E-52AE-11D9-8B02-000A956826C8@dalkescientific.com>

Lincoln:
>> Same for these.  What's the problem with
>>    text/x-fasta , etc.?
>
> I like this solution. I don't like returning text/xml because there is
> a growing tendency of browsers to try to parse the XML, returning
> uninterpretable results.

What is "this"?  The use of "+xml" at the end of the content type?

I ask because the snippet you quoted from me is what's already there
so isn't a new solution.

					Andrew
					dalke at dalkescientific.com



From lstein at cshl.edu  Mon Dec 20 20:09:50 2004
From: lstein at cshl.edu (Lincoln Stein)
Date: Mon, 20 Dec 2004 15:09:50 -0500
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <D88F7C5E-52AE-11D9-8B02-000A956826C8@dalkescientific.com>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
	<200412201204.23423.lstein@cshl.edu>
	<D88F7C5E-52AE-11D9-8B02-000A956826C8@dalkescientific.com>
Message-ID: <200412201509.50516.lstein@cshl.edu>

Doesn't the spec use application/* mime types? This is a text/* mime 
type.

Lincoln


On Monday 20 December 2004 12:44 pm, Andrew Dalke wrote:
> Lincoln:
> >> Same for these.  What's the problem with
> >>    text/x-fasta , etc.?
> >
> > I like this solution. I don't like returning text/xml because
> > there is a growing tendency of browsers to try to parse the XML,
> > returning uninterpretable results.
>
> What is "this"?  The use of "+xml" at the end of the content type?
>
> I ask because the snippet you quoted from me is what's already
> there so isn't a new solution.
>
> 					Andrew
> 					dalke at dalkescientific.com
>
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2

-- 
Lincoln D. Stein
Cold Spring Harbor Laboratory
1 Bungtown Road
Cold Spring Harbor, NY 11724

NOTE: Please copy Sandra Michelsen <michelse at cshl.edu> on
all emails regarding scheduling and other time-critical topics.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20041220/9e473462/attachment.sig>

From Steve_Chervitz at affymetrix.com  Tue Dec 21 02:42:43 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Mon, 20 Dec 2004 18:42:43 -0800
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <Pine.LNX.4.58.0412172314240.6215@sumo.ctrl.ucla.edu>
Message-ID: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>

I just committed a new version of das2_get.html with this change:

RCS file: /home/repository/biodas/das/das2/das2_get.html,v
----------------------------
revision 1.2
date: 2004/12/21 02:43:42;  author: sac;  state: Exp;  lines: +50 -49
Converted mime types from application/x-das-foo to text/x-das-foo+xml for
XML types. For non-XML types omitting the '+xml' and adding an 'x-' suffix
to the right of the slash as needed.

Steve


> From: Allen Day <allenday at ucla.edu>
> Date: Fri, 17 Dec 2004 23:14:54 -0800
> To: Andrew Dalke <dalke at dalkescientific.com>
> Cc: <das2 at portal.open-bio.org>
> Subject: Re: [DAS2] Content type for DAS XML responses
> 
> sounds good, can we get a patch to the spec so i can make the necessary
> implementation changes and test against browsers?  thanks!
> 
> -allen 
> 
> 
> On Fri, 17 Dec 2004, Andrew Dalke wrote:
> 
>> Steve: 
>>> Currently, the DAS/2 spec dictates that responses should have
>>> content-types 
>>> like: 
>>> 
>>>   application/x-das-source
>>>   application/x-das-types
>>>   application/x-das-region
>>> 
>>> Changing these to be 'text/xml' would be more convenient since the
>>> content 
>>> would then be viewable in a standard web browser.
>> 
>> A browser should send something like
>> 
>> Accept: */* 
>>    or 
>> Accept: application/xml
>> 
>> or some other generic content type in its headers.  One solution
>> to at least consider is that IF a client does not request
>> application/x-das-whatever THEN return application/xml .  I do
>> not believe this is appropriate though.
>> 
>> 
>>>   text/xml; dasformat=source
>>>   text/xml; dasformat=types
>>>   text/xml; dasformat=region
>> 
>> I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
>> RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )
>> 
>>     If an XML document -- that is, the unprocessed, source XML document
>>     -- is readable by casual users, text/xml is preferable to
>>     application/xml.  MIME user agents (and web user agents) that do not
>>     have explicit support for text/xml will treat it as text/plain, for
>>     example, by displaying the XML MIME entity as plain text.
>>     Application/xml is preferable when the XML MIME entity is unreadable
>>     by casual users.
>> 
>> That RFC is relevant to this discussion.  If I understand it
>> correctly it recommends using something more like
>> 
>>    text/x-das-source+xml
>> 
>> This would be understood by RFC 3023 compliant clients as
>> containing an XML document, by DAS clients to understand
>> which DAS XML format is used, and by generic clients to
>> contain an XML document.
>> 
>>> The corresponding compact content-types could be:
>>> 
>>>   text/plain; dasformat=compact-source
>>>   text/plain; dasformat=compact-types
>>>   text/plain; dasformat=compact-region
>> 
>> There should be no need for this.  A client should use
>> the catch-all text/* handler for text/x-compact-source
>> and show it as normal text.
>> 
>>> And other types could be:
>>> 
>>>   text/plain; dasformat=fasta
>>>   text/plain; dasformat=gff3
>>>   text/plain; dasformat=bed
>> 
>> Same for these.  What's the problem with
>>    text/x-fasta , etc.?
>> 
>> See also the discussion in appendix A.5 which describes
>> practical problems about using parameter in MIME types.
>> 
>>                                       Andrew
>>                                       dalke at dalkescientific.com
>> 
>> _______________________________________________
>> DAS2 mailing list
>> DAS2 at portal.open-bio.org
>> http://portal.open-bio.org/mailman/listinfo/das2
>> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2 



From allenday at ucla.edu  Tue Dec 21 04:32:42 2004
From: allenday at ucla.edu (Allen Day)
Date: Mon, 20 Dec 2004 20:32:42 -0800 (PST)
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
References: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
Message-ID: <Pine.LNX.4.58.0412202032140.24345@sumo.ctrl.ucla.edu>

changes made to das2 codebase, thanks.

On Mon, 20 Dec 2004, Chervitz, Steve wrote:

> I just committed a new version of das2_get.html with this change:
> 
> RCS file: /home/repository/biodas/das/das2/das2_get.html,v
> ----------------------------
> revision 1.2
> date: 2004/12/21 02:43:42;  author: sac;  state: Exp;  lines: +50 -49
> Converted mime types from application/x-das-foo to text/x-das-foo+xml for
> XML types. For non-XML types omitting the '+xml' and adding an 'x-' suffix
> to the right of the slash as needed.
> 
> Steve
> 
> 
> > From: Allen Day <allenday at ucla.edu>
> > Date: Fri, 17 Dec 2004 23:14:54 -0800
> > To: Andrew Dalke <dalke at dalkescientific.com>
> > Cc: <das2 at portal.open-bio.org>
> > Subject: Re: [DAS2] Content type for DAS XML responses
> > 
> > sounds good, can we get a patch to the spec so i can make the necessary
> > implementation changes and test against browsers?  thanks!
> > 
> > -allen 
> > 
> > 
> > On Fri, 17 Dec 2004, Andrew Dalke wrote:
> > 
> >> Steve: 
> >>> Currently, the DAS/2 spec dictates that responses should have
> >>> content-types 
> >>> like: 
> >>> 
> >>>   application/x-das-source
> >>>   application/x-das-types
> >>>   application/x-das-region
> >>> 
> >>> Changing these to be 'text/xml' would be more convenient since the
> >>> content 
> >>> would then be viewable in a standard web browser.
> >> 
> >> A browser should send something like
> >> 
> >> Accept: */* 
> >>    or 
> >> Accept: application/xml
> >> 
> >> or some other generic content type in its headers.  One solution
> >> to at least consider is that IF a client does not request
> >> application/x-das-whatever THEN return application/xml .  I do
> >> not believe this is appropriate though.
> >> 
> >> 
> >>>   text/xml; dasformat=source
> >>>   text/xml; dasformat=types
> >>>   text/xml; dasformat=region
> >> 
> >> I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
> >> RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )
> >> 
> >>     If an XML document -- that is, the unprocessed, source XML document
> >>     -- is readable by casual users, text/xml is preferable to
> >>     application/xml.  MIME user agents (and web user agents) that do not
> >>     have explicit support for text/xml will treat it as text/plain, for
> >>     example, by displaying the XML MIME entity as plain text.
> >>     Application/xml is preferable when the XML MIME entity is unreadable
> >>     by casual users.
> >> 
> >> That RFC is relevant to this discussion.  If I understand it
> >> correctly it recommends using something more like
> >> 
> >>    text/x-das-source+xml
> >> 
> >> This would be understood by RFC 3023 compliant clients as
> >> containing an XML document, by DAS clients to understand
> >> which DAS XML format is used, and by generic clients to
> >> contain an XML document.
> >> 
> >>> The corresponding compact content-types could be:
> >>> 
> >>>   text/plain; dasformat=compact-source
> >>>   text/plain; dasformat=compact-types
> >>>   text/plain; dasformat=compact-region
> >> 
> >> There should be no need for this.  A client should use
> >> the catch-all text/* handler for text/x-compact-source
> >> and show it as normal text.
> >> 
> >>> And other types could be:
> >>> 
> >>>   text/plain; dasformat=fasta
> >>>   text/plain; dasformat=gff3
> >>>   text/plain; dasformat=bed
> >> 
> >> Same for these.  What's the problem with
> >>    text/x-fasta , etc.?
> >> 
> >> See also the discussion in appendix A.5 which describes
> >> practical problems about using parameter in MIME types.
> >> 
> >>                                       Andrew
> >>                                       dalke at dalkescientific.com
> >> 
> >> _______________________________________________
> >> DAS2 mailing list
> >> DAS2 at portal.open-bio.org
> >> http://portal.open-bio.org/mailman/listinfo/das2
> >> 
> > _______________________________________________
> > DAS2 mailing list
> > DAS2 at portal.open-bio.org
> > http://portal.open-bio.org/mailman/listinfo/das2 
> 


From gilmanb at pantherinformatics.com  Tue Dec 21 16:03:32 2004
From: gilmanb at pantherinformatics.com (Brian Gilman)
Date: Tue, 21 Dec 2004 11:03:32 -0500
Subject: [DAS2] Full Spec?
In-Reply-To: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
References: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
Message-ID: <DCC0DFCD-5369-11D9-A785-000A95CA3D68@pantherinformatics.com>

Hello Everyone,

	Is there a full specification for DAS 2 somewhere? Can someone please 
send it to me? I'd like to read it again and cannot seem to find it in 
my e-mail anywhere.


						Best,

								-B
-- 
Brian Gilman
President Panther Informatics Inc.

01000010 01101001 01101111
01001001 01101110 01100110
01101111 01110010 01101101
01100001 01110100 01101001
01100011 01101001 01100001
01101110

On Dec 20, 2004, at 9:42 PM, Chervitz, Steve wrote:

> I just committed a new version of das2_get.html with this change:
>
> RCS file: /home/repository/biodas/das/das2/das2_get.html,v
> ----------------------------
> revision 1.2
> date: 2004/12/21 02:43:42;  author: sac;  state: Exp;  lines: +50 -49
> Converted mime types from application/x-das-foo to text/x-das-foo+xml 
> for
> XML types. For non-XML types omitting the '+xml' and adding an 'x-' 
> suffix
> to the right of the slash as needed.
>
> Steve
>
>
>> From: Allen Day <allenday at ucla.edu>
>> Date: Fri, 17 Dec 2004 23:14:54 -0800
>> To: Andrew Dalke <dalke at dalkescientific.com>
>> Cc: <das2 at portal.open-bio.org>
>> Subject: Re: [DAS2] Content type for DAS XML responses
>>
>> sounds good, can we get a patch to the spec so i can make the 
>> necessary
>> implementation changes and test against browsers?  thanks!
>>
>> -allen
>>
>>
>> On Fri, 17 Dec 2004, Andrew Dalke wrote:
>>
>>> Steve:
>>>> Currently, the DAS/2 spec dictates that responses should have
>>>> content-types
>>>> like:
>>>>
>>>>   application/x-das-source
>>>>   application/x-das-types
>>>>   application/x-das-region
>>>>
>>>> Changing these to be 'text/xml' would be more convenient since the
>>>> content
>>>> would then be viewable in a standard web browser.
>>>
>>> A browser should send something like
>>>
>>> Accept: */*
>>>    or
>>> Accept: application/xml
>>>
>>> or some other generic content type in its headers.  One solution
>>> to at least consider is that IF a client does not request
>>> application/x-das-whatever THEN return application/xml .  I do
>>> not believe this is appropriate though.
>>>
>>>
>>>>   text/xml; dasformat=source
>>>>   text/xml; dasformat=types
>>>>   text/xml; dasformat=region
>>>
>>> I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
>>> RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )
>>>
>>>     If an XML document -- that is, the unprocessed, source XML 
>>> document
>>>     -- is readable by casual users, text/xml is preferable to
>>>     application/xml.  MIME user agents (and web user agents) that do 
>>> not
>>>     have explicit support for text/xml will treat it as text/plain, 
>>> for
>>>     example, by displaying the XML MIME entity as plain text.
>>>     Application/xml is preferable when the XML MIME entity is 
>>> unreadable
>>>     by casual users.
>>>
>>> That RFC is relevant to this discussion.  If I understand it
>>> correctly it recommends using something more like
>>>
>>>    text/x-das-source+xml
>>>
>>> This would be understood by RFC 3023 compliant clients as
>>> containing an XML document, by DAS clients to understand
>>> which DAS XML format is used, and by generic clients to
>>> contain an XML document.
>>>
>>>> The corresponding compact content-types could be:
>>>>
>>>>   text/plain; dasformat=compact-source
>>>>   text/plain; dasformat=compact-types
>>>>   text/plain; dasformat=compact-region
>>>
>>> There should be no need for this.  A client should use
>>> the catch-all text/* handler for text/x-compact-source
>>> and show it as normal text.
>>>
>>>> And other types could be:
>>>>
>>>>   text/plain; dasformat=fasta
>>>>   text/plain; dasformat=gff3
>>>>   text/plain; dasformat=bed
>>>
>>> Same for these.  What's the problem with
>>>    text/x-fasta , etc.?
>>>
>>> See also the discussion in appendix A.5 which describes
>>> practical problems about using parameter in MIME types.
>>>
>>>                                       Andrew
>>>                                       dalke at dalkescientific.com
>>>
>>> _______________________________________________
>>> DAS2 mailing list
>>> DAS2 at portal.open-bio.org
>>> http://portal.open-bio.org/mailman/listinfo/das2
>>>
>> _______________________________________________
>> DAS2 mailing list
>> DAS2 at portal.open-bio.org
>> http://portal.open-bio.org/mailman/listinfo/das2
>
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2
>



From avc at sanger.ac.uk  Tue Dec 21 16:32:52 2004
From: avc at sanger.ac.uk (Tony Cox)
Date: Tue, 21 Dec 2004 16:32:52 +0000
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <200412201204.23423.lstein@cshl.edu>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>	<E63D1D14-50B7-11D9-8B02-000A956826C8@dalkescientific.com>
	<200412201204.23423.lstein@cshl.edu>
Message-ID: <41C85034.3000303@sanger.ac.uk>

Hi Lincoln,

Just been reading the Get spec - I thought at our last meeting we agreed 
to add attributes/properties (doc_href"?) to features, sources etc. that 
allowed better documenting of the data being returned.

Tony



From lstein at cshl.edu  Tue Dec 21 21:25:21 2004
From: lstein at cshl.edu (Lincoln Stein)
Date: Tue, 21 Dec 2004 16:25:21 -0500
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <41C85034.3000303@sanger.ac.uk>
References: <BDE8C958.31EA%Steve_Chervitz@affymetrix.com>
	<200412201204.23423.lstein@cshl.edu> <41C85034.3000303@sanger.ac.uk>
Message-ID: <200412211625.21821.lstein@cshl.edu>

Hi Tony,

If the doc_href stuff isn't in there, then it was my error.  It is 
certainly in my notes.  I will add it forthwith (I've got another big 
chunk of time to work on the DAS spec coming up during xmas 
vacation).

Lincoln

On Tuesday 21 December 2004 11:32 am, Tony Cox wrote:
> Hi Lincoln,
>
> Just been reading the Get spec - I thought at our last meeting we
> agreed to add attributes/properties (doc_href"?) to features,
> sources etc. that allowed better documenting of the data being
> returned.
>
> Tony

-- 
Lincoln D. Stein
Cold Spring Harbor Laboratory
1 Bungtown Road
Cold Spring Harbor, NY 11724

NOTE: Please copy Sandra Michelsen <michelse at cshl.edu> on
all emails regarding scheduling and other time-critical topics.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20041221/a79aadc0/attachment.sig>

From lstein at cshl.edu  Tue Dec 21 21:27:20 2004
From: lstein at cshl.edu (Lincoln Stein)
Date: Tue, 21 Dec 2004 16:27:20 -0500
Subject: [DAS2] Full Spec?
In-Reply-To: <DCC0DFCD-5369-11D9-A785-000A95CA3D68@pantherinformatics.com>
References: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
	<DCC0DFCD-5369-11D9-A785-000A95CA3D68@pantherinformatics.com>
Message-ID: <200412211627.20463.lstein@cshl.edu>

It's in CVS:

repository = :ext:pub.open-bio.org:/home/repository/biodas
project     =  das

Lincoln

On Tuesday 21 December 2004 11:03 am, Brian Gilman wrote:
> Hello Everyone,
>
> 	Is there a full specification for DAS 2 somewhere? Can someone
> please send it to me? I'd like to read it again and cannot seem to
> find it in my e-mail anywhere.
>
>
> 						Best,
>
> 								-B

-- 
Lincoln D. Stein
Cold Spring Harbor Laboratory
1 Bungtown Road
Cold Spring Harbor, NY 11724

NOTE: Please copy Sandra Michelsen <michelse at cshl.edu> on
all emails regarding scheduling and other time-critical topics.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20041221/24734ca8/attachment.sig>

From lstein at cshl.edu  Tue Dec 21 21:50:35 2004
From: lstein at cshl.edu (Lincoln Stein)
Date: Tue, 21 Dec 2004 16:50:35 -0500
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
References: <BDECCDA3.3340%Steve_Chervitz@affymetrix.com>
Message-ID: <200412211650.35407.lstein@cshl.edu>

Beautiful!  Thank you for doing that.

Lincoln

On Monday 20 December 2004 09:42 pm, Chervitz, Steve wrote:
> I just committed a new version of das2_get.html with this change:
>
> RCS file: /home/repository/biodas/das/das2/das2_get.html,v
> ----------------------------
> revision 1.2
> date: 2004/12/21 02:43:42;  author: sac;  state: Exp;  lines: +50
> -49 Converted mime types from application/x-das-foo to
> text/x-das-foo+xml for XML types. For non-XML types omitting the
> '+xml' and adding an 'x-' suffix to the right of the slash as
> needed.
>
> Steve
>
> > From: Allen Day <allenday at ucla.edu>
> > Date: Fri, 17 Dec 2004 23:14:54 -0800
> > To: Andrew Dalke <dalke at dalkescientific.com>
> > Cc: <das2 at portal.open-bio.org>
> > Subject: Re: [DAS2] Content type for DAS XML responses
> >
> > sounds good, can we get a patch to the spec so i can make the
> > necessary implementation changes and test against browsers? 
> > thanks!
> >
> > -allen
> >
> > On Fri, 17 Dec 2004, Andrew Dalke wrote:
> >> Steve:
> >>> Currently, the DAS/2 spec dictates that responses should have
> >>> content-types
> >>> like:
> >>>
> >>>   application/x-das-source
> >>>   application/x-das-types
> >>>   application/x-das-region
> >>>
> >>> Changing these to be 'text/xml' would be more convenient since
> >>> the content
> >>> would then be viewable in a standard web browser.
> >>
> >> A browser should send something like
> >>
> >> Accept: */*
> >>    or
> >> Accept: application/xml
> >>
> >> or some other generic content type in its headers.  One solution
> >> to at least consider is that IF a client does not request
> >> application/x-das-whatever THEN return application/xml .  I do
> >> not believe this is appropriate though.
> >>
> >>>   text/xml; dasformat=source
> >>>   text/xml; dasformat=types
> >>>   text/xml; dasformat=region
> >>
> >> I get confused; is it "text/xml" or "application/xml"?  Ah-ha!
> >> RFC 3023 says (  http://www.faqs.org/rfcs/rfc3023.html )
> >>
> >>     If an XML document -- that is, the unprocessed, source XML
> >> document -- is readable by casual users, text/xml is preferable
> >> to application/xml.  MIME user agents (and web user agents) that
> >> do not have explicit support for text/xml will treat it as
> >> text/plain, for example, by displaying the XML MIME entity as
> >> plain text. Application/xml is preferable when the XML MIME
> >> entity is unreadable by casual users.
> >>
> >> That RFC is relevant to this discussion.  If I understand it
> >> correctly it recommends using something more like
> >>
> >>    text/x-das-source+xml
> >>
> >> This would be understood by RFC 3023 compliant clients as
> >> containing an XML document, by DAS clients to understand
> >> which DAS XML format is used, and by generic clients to
> >> contain an XML document.
> >>
> >>> The corresponding compact content-types could be:
> >>>
> >>>   text/plain; dasformat=compact-source
> >>>   text/plain; dasformat=compact-types
> >>>   text/plain; dasformat=compact-region
> >>
> >> There should be no need for this.  A client should use
> >> the catch-all text/* handler for text/x-compact-source
> >> and show it as normal text.
> >>
> >>> And other types could be:
> >>>
> >>>   text/plain; dasformat=fasta
> >>>   text/plain; dasformat=gff3
> >>>   text/plain; dasformat=bed
> >>
> >> Same for these.  What's the problem with
> >>    text/x-fasta , etc.?
> >>
> >> See also the discussion in appendix A.5 which describes
> >> practical problems about using parameter in MIME types.
> >>
> >>                                       Andrew
> >>                                       dalke at dalkescientific.com
> >>
> >> _______________________________________________
> >> DAS2 mailing list
> >> DAS2 at portal.open-bio.org
> >> http://portal.open-bio.org/mailman/listinfo/das2
> >
> > _______________________________________________
> > DAS2 mailing list
> > DAS2 at portal.open-bio.org
> > http://portal.open-bio.org/mailman/listinfo/das2
>
> _______________________________________________
> DAS2 mailing list
> DAS2 at portal.open-bio.org
> http://portal.open-bio.org/mailman/listinfo/das2

-- 
Lincoln D. Stein
Cold Spring Harbor Laboratory
1 Bungtown Road
Cold Spring Harbor, NY 11724

NOTE: Please copy Sandra Michelsen <michelse at cshl.edu> on
all emails regarding scheduling and other time-critical topics.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20041221/6ac7dae4/attachment.sig>

From Steve_Chervitz at affymetrix.com  Wed Dec 22 00:16:24 2004
From: Steve_Chervitz at affymetrix.com (Chervitz, Steve)
Date: Tue, 21 Dec 2004 16:16:24 -0800
Subject: [DAS2] Content type for DAS XML responses
In-Reply-To: <200412211650.35407.lstein@cshl.edu>
Message-ID: <BDEDFCD8.33DD%Steve_Chervitz@affymetrix.com>

Lincoln Stein <lstein at cshl.edu> wrote:
> 
> Beautiful!  Thank you for doing that.
>

No problem. 

BTW Lincoln, your email messages are being delivered in RFC822 attachments
making them difficult to view & reply to, at least with the mail readers I
use (Entourage, Apple Mail, pine, Outlook). I guess these readers aren't yet
clued into to this (probably superior) scheme. Could you send "non-attached"
versions of your messages in addition to the attached PGP-signed message?

Perhaps there's a configuration option I'm missing (though I did recently
figure out how to get pine from complaining about iso-8859-1 character
sets). 

Mailman deals with your attachments no problem, so I've actually been using
that as my reader: http://portal.open-bio.org/pipermail/das2/

Steve

PS. Here's a curiosity: When I exported the Message/RFC822 container in your
email, I got these two lines:

> From the-concourse-on-high Tue Dec 21 15:25:50 2004
> Date: Tue, 21 Dec 2004 13:51:37 -0800

What's 'the-concourse-on-high' about? Searching the web, it appears to have
a Baha'i association (without hyphens), or as a generic 'From' attribute in
emails (with hyphens).



> 
> On Monday 20 December 2004 09:42 pm, Chervitz, Steve wrote:
> 
> I just committed a new version of das2_get.html with this change:
>
> RCS file: /home/repository/biodas/das/das2/das2_get.html,v
> ---------------------------
> revision 1.2
> date: 2004/12/21 02:43:42;  author: sac;  state: Exp;  lines: +50
> -49 Converted mime types from application/x-das-foo to
> text/x-das-foo+xml for XML types. For non-XML types omitting the
> '+xml' and adding an 'x-' suffix to the right of the slash as
> needed.
>
> Steve
>
> > From: Allen Day <allenday at ucla.edu>
> > Date: Fri, 17 Dec 2004 23:14:54 -0800
> > To: Andrew Dalke <dalke at dalkescientific.com>
> > Cc: <das2 at portal.open-bio.org>
> > Subject: Re: [DAS2] Content type for DAS XML responses
> >
> > sounds good, can we get a patch to the spec so i can make the
> > necessary implementation changes and test against browsers?
> > thanks!
> >