From dalke at dalkescientific.com  Mon May  1 02:14:10 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 1 May 2006 00:14:10 -0600
Subject: [DAS2] new stylesheet syntax
Message-ID: <8dfd12074a5404e2577db4547888d08a@dalkescientific.com>

I've been thinking about the stylesheet document format.  I've
changed it somewhat.  Here's what the latest version looks like

<STYLESHEET>
   <STYLE 
uri="http://biodas.org/das2/bos_taurus/Dec2006/type/five_prime_UTR"
          zoom="high">
     <BOX fgcolor="red" />
     <LABEL fgcolor="black" />
   </STYLE>

   <STYLE 
uri="http://biodas.org/das2/bos_taurus/Dec2006/type/five_prime_UTR"
          zoom="medium">
     <BOX fgcolor="red" />
   </STYLE>

   <STYLE 
uri="http://biodas.org/das2/bos_taurus/Dec2006/type/five_prime_UTR"
          zoom="low">
     <LINE fgcolor="red" />
   </STYLE>

   <STYLE 
uri="http://biodas.org/das2/bos_taurus/Dec2006/type/strong_gene">
     <BOX fgcolor="black" border_style="solid" />
   </STYLE>

   <STYLE 
uri="http://biodas.org/das2/bos_taurus/Dec2006/type/moderate_gene">
     <BOX fgcolor="black" border_style="dotted" />
   </STYLE>

   <STYLE uri="http://biodas.org/das2/bos_taurus/Dec2006/type/weak_gene">
     <BOX fgcolor="gray" border_style="dotted" />
   </STYLE>

</STYLESHEET>

The change is that the attributes in the <STYLE> element are only
used as selectors, that is, fields which select feature types.  I've
moved the 'width', 'height', and 'bump' elements to a <SET> element.

The core of the schema looks like this.

style = element STYLE {
   # These are selectors.
   attribute uri { text },
   attribute zoom { "high" | "medium" | "low" }?,

   set_element,

   # Must specify a glyph, may specify a label.  Can be in
   # either order.
   ((glyph, label?) | (label, glyph))

}

# Set default depiction properties
set_element = element SET {
   height?,
   width?,
   bump { "yes" | "no" }?
}

If you hadn't looked at the previous schema, the height, width,
and bump attributes were in the STYLE attributes along with 'fgcolor'
and 'bgcolor', which provided defaults for the glyphs and title, but
the latter two could override the defaults.

The main reason for the change is to separate selectors from
the actual style.  I got rid of the default fg/bg colors because

					Andrew
					dalke at dalkescientific.com



From Steve_Chervitz at affymetrix.com  Mon May  1 13:56:20 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 01 May 2006 10:56:20 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 5 May 2006
Message-ID: <C07B99D4.1E185%Steve_Chervitz@affymetrix.com>

Notes from the weekly DAS/2 teleconference, 5 May 2006

$Id: das2-teleconf-2006-05-01.txt,v 1.1 2006/05/01 17:52:48 sac Exp $

Note taker: Steve Chervitz

Attendees: 
  Affy: Steve Chervitz, Ed E., Gregg Helt
  Dalke Scientific: Andrew Dalke
        
Action items are flagged with '[A]'.

These notes are checked into the biodas.org CVS repository at
das/das2/notes/2006. Instructions on how to access this
repository are at http://biodas.org

DISCLAIMER: 
The note taker aims for completeness and accuracy, but these goals are
not always achievable, given the desire to get the notes out with a
rapid turnaround. So don't consider these notes as complete minutes
from the meeting, but rather abbreviated, summarized versions of what
was discussed. There may be errors of commission and omission.
Participants are welcome to post comments and/or corrections to these
as they see fit. 


Proposed agenda:
* Discuss feedback on Andrew's writeback spec and solution to splits
  and joins


Topic: Writeback
-----------------

[A] Collect feedback from UK folks at next mtg since today is UK holiday.

ad: likes the model where you have multiple, composed change
statements rather than a single document describing all changes.
if you have history tracking, you have the server understand
versioning. 

gh: if you do a merge by deleting a feat and then extending another
feat or delete 2 and create one, how do you get history tracking?

ad: you have a single delta, describing "delete this, extend
that". Lots of small deltas comprise a big delta --- wraps up all of
the curator's work in a single session.
gh: delta lets you figure out what came from what in the change element.

ad: when do people use the split and merge history trail?
gh: to ask, "what the hell happened to the gene that I'm interested
in?" when the thing it got merged into has a completely different id.

ad: what I'm proposing could capture this history,
gh: it allows the capturing as long as client is careful about how it
constructs the change and server is configured to track changes.
ad: client and server both need special care.
gh: I could see a more direct way of doing it that doesn't require
such carefulness. I.e., explicitly say what a new feature is derived
from.
ad: this could be a micro-delta, such as a comment or remark saying
"this is a merger of x and y" along with a controlled vocab of changes.
I'm hoping people had a chance to look at it.

gh: to me it looks sufficient, but the main question is, does this
meet requirements of existing Apollo and Otter systems?

[A] Andrew will talk directly with Nomi and Otter folks.

gh: so we need to get someone to implement. Allen and Brian.

[A] Allen and/Brian provide feedback to Andrew for writeback based on their
impl.

gh: there's still the question of having arbitrary xml in a feature,
editing it, and how to indicate what should/shouldn't be changed.
Also, the locking document isn't on-line.

ad: the locking document is on my todo list. as we discussed, locking
and writeback are orthogonal.

[A] Andrew will create a locking document.

Topic: Stylesheets
------------------

ad: changed stylesheets a bit.
ee: I like getting color etc. out of the main tag. still not sure how to
impl this in IGB. We may ignore stuff about hi/low zoom, box, line.
ad: what are units of height and width. pixels, em?
sc: we could specify units in the document.
ad: ...

ad: Also, does width/height include the label and where is the label
to be positioned? 
ee: could just let it be undefined. leave it up to the
browser. There's too much variation in what sort of vizualization
different viewers will be doing.

ad: do we need to specify height/width?
ee: shouldn't need width since that's based on chromosomal
coordinates. maybe need a minimum width.
ad: would be glad to drop width/height from stle element.

[A] Andrew will drop height/width from stylsheet until there's a clear need.


Topic Status
------------

ee: investigated bug for our old das servers are not returning unique
ids for certain things (alignments).  in our das/1 parser, if both
things come back with same id in one request, it merges, if they come
back in different requests, the second will be ignored (will say it's
already seen it). They need to modify server to return unique ids.

ad: seems to be a common problem.

gh: still on vacation. Need to set up teleconf to discuss resubmission
of renewal grant, and the possibility of bridge funding until we get
another grant.
ad: what about the no-cost funding extension?
gh: still figuring out what we have funding-wise.

sc: Fixed the cronjob on biodas.org to CVS update the spec
documentation, so the docs on the website will stay current now.
Other things: I've been playing with ruby on rails for rapid web app
development. so far I'm impressed. The URL-based nature of it's
request processing makes me think about hooking it up with das, making
a simple das server and/or viewer. E.g., table-based views of
sources, versions, types. We could possibly do web-based graphical
views as well.
ad: the das url structure should make this feasible. processing
queries would take some work. there's an analogous python approach too.
sc: might be nice to have sample das servers and/or web-based viewers
in python, ruby, perl, and java.

[A] steve will set up a demo das server/client using ruby on rails.



From Steve_Chervitz at affymetrix.com  Mon May  1 14:28:53 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 01 May 2006 11:28:53 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 5 May 2006
In-Reply-To: <C07B99D4.1E185%Steve_Chervitz@affymetrix.com>
Message-ID: <C07BA175.1E190%Steve_Chervitz@affymetrix.com>

Correction: The date for this teleconf was 1 May 2006 (today).


> From: Steve Chervitz <Steve_Chervitz at affymetrix.com>
> Date: Mon, 01 May 2006 10:56:20 -0700
> To: DAS/2 <das2 at lists.open-bio.org>
> Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 5 May 2006
> 
> Notes from the weekly DAS/2 teleconference, 5 May 2006
> 
> $Id: das2-teleconf-2006-05-01.txt,v 1.1 2006/05/01 17:52:48 sac Exp $

Notes from the weekly DAS/2 teleconference, 1 May 2006

$Id: das2-teleconf-2006-05-01.txt,v 1.2 2006/05/01 18:19:01 sac Exp $

Steve 




From dalke at dalkescientific.com  Tue May  2 10:43:09 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Tue, 2 May 2006 08:43:09 -0600
Subject: [DAS2] Fwd: DAS writeback
Message-ID: <f8ad58f58b4b9850ee2caa4b6279f247@dalkescientific.com>

I emailed Nomi regarding talking about writeback.  She suggested
I talk with Mark Gibson instead.  Here's the core part of his
response on splits&joins

[Andrew]
>> One option is to track exactly what happened.  For exampe, if
>> features F1 and F2 are merged into F3 then either
>>   DELETE F1
>>   DELETE F2
>>   CREATE F3
>>      <derives from F1>
>>      <derives from F2>
>> -or-
>>   DELETE F1
>>   MODIFY F2
>>      -change the range-
>>      <derives from F1>

[Mark]
> this is essentially what apollo does and i dont like it.
>
> in apollo the user chooses the name from a merge which can be F1-name, 
> F2-name or temp-name (new name from db) - if F1 is picked then f2 
> should be deleted & vice versa - apollo doesnt do this proper at the 
> moment (historical reasons - user didnt use to pick the name & the 
> name picking was hacked yada yada...)

[Andrew]
>> In this case the <derives from *> means the information is included
>> in the feature record.
>> I proposed an alternative, which I'm calling a microdelta.
>>   REASON: merge
>>   DELETE F1
>>   DELETE F2
>>   CREATE F3

[Mark]
> i like this one the best. I think its valuable to record in the record 
> that merge (&splits) have occured - and this makes it the most 
> obvious. Some databases like to track merge & splits (like TAIR - i 
> dont think chado tracks this but it should)
>
> i think its worthy on its own and its muddled to shove this in the 
> feature with derives from.


					Andrew
					dalke at dalkescientific.com



From Steve_Chervitz at affymetrix.com  Mon May  8 21:44:57 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 08 May 2006 18:44:57 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 8 May 2006
Message-ID: <C0854229.1E48F%Steve_Chervitz@affymetrix.com>

Notes from the weekly DAS/2 teleconference, 8 May 2006

$Id: das2-teleconf-2006-05-08.txt,v 1.1 2006/05/09 01:43:17 sac Exp $

Note taker: Steve Chervitz

Attendees: 
  Affy: Steve Chervitz, Ed E., Gregg Helt
  Dalke Scientific: Andrew Dalke
  Sanger: Andreas Prlic, Roy
  UC Berkeley: Suzi Lewis
  UCLA: Brian O'connor

Action items are flagged with '[A]'.

These notes are checked into the biodas.org CVS repository at
das/das2/notes/2006. Instructions on how to access this
repository are at http://biodas.org

DISCLAIMER: 
The note taker aims for completeness and accuracy, but these goals are
not always achievable, given the desire to get the notes out with a
rapid turnaround. So don't consider these notes as complete minutes
from the meeting, but rather abbreviated, summarized versions of what
was discussed. There may be errors of commission and omission.
Participants are welcome to post comments and/or corrections to these
as they see fit. 



Topic: Writeback
----------------

[A] Andrew will find out status of biopackages server re: writeback

Topic: Stylesheets  (we'll return to writeback later...)
------------------

ad: stylesheet spec is out. see recent email post.

roy: similar to z-map approach, not too much different.

ap: you have a link to type. below you provide how you want to
draw. if you have a line, you can also have a wireframe, cartoon,
spacefill. Allow a list of elements that you want. Client can choose.

ad: for structure we might want to have a set up where most of
residues are in one depiction, some subset of residues are in an
alternative depection (e.g., active site residues are
spacefilled). How would this be addressed? (currently an open question)

[suzi joins.]

gh: getting back to writeback. suzi's feedback. plans for apollo.

sl: no resources at the moment. we have a proposal just for apollo RSA
for development. chances are not as good for funding. P. good
recommends bundling with das and submit a das grant.

gh: pushing back application a month or two to give gh a chance to
revise it. a good idea.

sl: the plan is to give us a more time to reivse das grant.

gh: given that there's no work on apollo writeback, looking at adding
this functionality to igb.

sl: what about pirating some apollo stuff for editing for inclusion
into igb?

gh: more detail in apollo. merge, split, etc. are in igb now in
prototype form. Looking at working on this to get some primitive
writeback before end of current grant cycle.

we may need to put more emphasis on cross client interaction.

sl: love the idea, but reviewers didn't think it feasible.
if we do cross component communication, the bulk of the work is
getting enough forward progress to make a more persuasive case in the
proposal. 

gh: resubmitting date? july?

sl: talk to p. good. july will be difficult for me. out of country.

gh: plan is for suzi to be PI.

sc: date? 

gh: need to get in touch w/ peter about this. to get renewal status
there is a time frame.

Topic: writeback (continued)
----------------------------

ap: if a feature is deleted and new is created, how to know the
history linked to new feature? similar to discussion andrew fwd to
list. didn't see any conclusion.

ad: there is no conclusion for it. the details are server dependent. a
server the impl history tracking can post to the feature itself.

sl: if it's deleted, it's deleted. this isn't an undoable delete is
it?

gh: grouping the changes. within the post, you have a grouping so you
are merging two things, they are encapsulated w/in a change element.

ad: orig idea was one big delta doc, including all feats add, modify,
deleted. But you want to track things more closely. proposed a delta
doc that groups things. then proposed a micro delta - these features got
merged,... and a remark statement. why the merge happened. conceptual.
whole writeback is a list of microdeltas.

people that impl feature history do it as they want to. extension to
feature. could also do it via another request.

server could track of everything that happened on server. track all
history if it wants to.

[see andrew's email for more information about this.]

gh: andrew should touch base with allen and brian progress on
biopackages server. if you can impl something in a few days go for it.

ad: like the in-memory database approach.

gh/sl: fond of this approach.

roy: favored sending back a mapping document.

ap: this features follows up some features (uri) that has been
deleted. permits link from new to old feature.

roy: post data to server, server replies with same document, modified
according to what the server has done. has to do work to redraw,
rather than have mapping doc that goes from and to, that info is in
the features themselves.

ad: my mapping doc, forces client to figure out what the new feats
are. your approach means less client work.

roy: client can't toss everything if you're sending a delta across.

ad: just everything that's changed.

gh: did you send email about this?
roy: just to andrew. can post to the list.

[A] roy/andrew will forward message re: writeback to the list

ad: like the idea of keeping feat element exactly the same. willing to
be less pure about it.

roy: think of it as extending the read-only server. extending the
feature as well.

[A] andrew will change spec to respond with a full feat doc for feats that
changed.

roy: only changes that the client sent.

ad: if the db is handling intra link (xid) pointing else where in the
database, and the pointed to feature disappears, the server could
update other features that pointed to those disappeared features.

sc: that might confuse the client that gest feats back it doesn't know
about.

ad: but the client might then get out of sync with server.

roy/gh: worry about this

ad: we can defer until issue arises (if it does)

roy: error state for spamming a server, i.e. hitting a server again
and again with same info. indicating that the client isn't getting the
response from server.

ad: client has to detect error and reload from the db.

roy: nice to be stateless.

ad: ask server for url. can put or post to it. if fails try again,
then you can read from it to see if your submit worked.
Example: 
ask server for ftp url.
ftp it over.
check it.
tell server it's there, process it.
you never get duplicates.

roy: this touches on locking.

gh: other than locking, not sure what state we need to maintain.

ad: other issue: authentication.

ap: this could be done by web server.

ad: which http authentication are we going to standardize on? async,
basic (passwds sent in clear). I propose we talk about it next week.

[A] we will talk about authentication next week (5/15).

ad: issues: improperly coded client. network going down (client can
reload from server).

gh: client never got response, so it will know .

ad: inside header, have location, if it got a partial download, it can
go and re-fetch it.

gh: should definitely be optional if it is in there.

ad: spamming question: a poorly impl client, posting the same thing
again and again.

ap: shouldn't locking prevent this?
gh: if same client, then no. not sure how bad that would be if
nothing's changed. how would that mess up the server?

ad: would depend on how server is impl.

gh: possible multi-threading issues - the mapping info may get out of
sync. I would put in as a recommendation in the spec that a client
wait until it gets a response from server before it sends another one.

ad: so fifo order processing.

[A] andrew spec mod: client should wait for server response before sending
another update request

ad: error handling. the error response document will be handy
here. figuring out why something failed.

gh: to me it seems unlikely that there will be clients that do
anything other than pass through the error info. that a client will do
anything interesting with the error response other than just show it
to the user.

Other topics:
--------------

bo: questions about igb. we can talk later. don't know status of
writeback right now. Check with Allen.

gh: status of registry?

ap: have dev servers that aren't publically available. have been busy
with other projects. das/2 working now on his private dev
server. waiting for server box. works - to retrieve a list of das sources.

the machine has little memory.

[A] Steve will send andreas link to versioned sources command for Affy das/2
server

[A] next week prepare to cover more details on writeback impls.


                                   



From rds at sanger.ac.uk  Tue May  9 03:38:54 2006
From: rds at sanger.ac.uk (Roy Storey)
Date: Tue, 9 May 2006 08:38:54 +0100 (BST)
Subject: [DAS2] DAS writeback (fwd)
Message-ID: <Pine.LNX.4.62.0605090817220.6731@deskpro110.dynamic.sanger.ac.uk>

All,

As requested during the conference call here's the mail re full features 
in the response.  What this does do is make the mapping explicit, making 
immediate history a property of each feature.  During the writeback action 
a client should in theory be able to "remove" the features it's saving and 
use the response as if it were a GET/query.  There is less issue with 
having to look up features using one key/uri and change to another, as 
would happen using mapping.xml response.

Regards

Roy

---------- Forwarded message ----------
Date: Thu, 27 Apr 2006 11:37:32 +0100 (BST)
From: Roy Storey <rds at sanger.ac.uk>
To: Andrew Dalke <dalke at dalkescientific.com>
Cc: allenday at ucla.edu
Subject: Re: DAS writeback

Andrew,

Sorry for not getting back on this yesterday, and as I've not really been able 
to keep 100% up to date on the DAS2 state of the art, apologies if I'm 
repeating previous discussion.

On Tue, 25 Apr 2006, Andrew Dalke wrote:

> Have you had any thoughts on this?

A few, nothing concrete.

> Mine is that a single POST using a private namespace is the
> right solution.  It makes for simpler servers and the client
> overhead isn't really that much.  What I like the most is the
> support for multiple versioning models, like "modify in-place"
> vs. "keep old versions"
>

This is about what we'd agreed in Feb. An editing client can hold objects wich 
are new and unsaved keying them how ever it wishes.  On saving these to the 
server, the server can respond with a mapping document between client ids and 
server ids.  The client then has the simple task of moving the objects from 
it's new unsaved objects list to its up-to-date objects list, applying the 
correct key translation as it does so.  As you said this works well for 
modifying current features and adding versions as well.


So what on splitting and merging?

This brings me to Otter, which rather than just replying with a simple mapping 
document, actually sends the complete features which are the result of the 
modifications.  I just wonder what the feeling for this is.

So a feature starts life in an editor...

uri="das-private:$md5_feature_bits"

gets saved to the server

somewhere in response
<feature uri="http://das2server/feature" >
   <previously uri="das-private:$md5_feature_bits" />
   ...
</feature>

get merged/split & saved

somewhere in response
<feature uri="http://das2server/another_feature" >
   <previously uri="http://das2server/featureA" />
   <!-- possibly, in case of merging -->
   <previously uri="http://das2server/featureB" />
   ...
</feature>

Ok so there are likely some issues with this, but I thought I'd raise as it 
could unify rw/ro and remove the need for mapping.rnc, which although I like 
does feel a tiny bit dirty. The first obvious problem with the above is it's 
more extra work for the server to do to output a single feature, so just on 
this feel free to shout me down.  It's possibly open to abuse too.  I also have 
mixed feelings on whether the spec should dictate how to do these more complex 
operations.


So a couple of what ifs:

- The client fails to do the mapping/moving.

This will result in the new feature being kept as "new" although the server 
knows about the feature.  This is likely to result in duplicate features on the 
server, as next time the client saves it'll send a "das-private:identifier". 
If the server is keying features on enough information to distinguish this IS a 
duplicate then the writeback will fail and this isn't an issue, except for the 
client author.  However, if the "new" feature stays as new, gets modified and 
saved again, then the possibly wrong features will just accumulate.

- The server doesn't send a complete response (dies/gets rebooted).

Only some of the features get "moved" in the client (See above). In this case 
though the client should/will know that something isn't right and should 
probably do a re-fetch of all the features, which will result in having to 
remove from the "new list" features which have been saved, but this seems sane 
to me.


Roy




From nomi at fruitfly.org  Thu May 11 19:08:41 2006
From: nomi at fruitfly.org (Nomi Harris)
Date: Thu, 11 May 2006 16:08:41 -0700
Subject: [DAS2] Poster deadline for ISMB
Message-ID: <ryiody45fd2.fsf@spongecake.lbl.gov>

Darn, I had it in my calendar to remind everyone that May 10th was the
deadline for submitting poster abstracts to ISMB (wasn't there some vague
plan to submit a poster about DAS/2?) and here it is, the 11th already.
I bet they'd still accept the abstract if someone submitted it today or
tomorrow along with profuse apologies about the lateness.

BOSC abstracts (http://www.open-bio.org/wiki/BOSC_2006) aren't due until
June 19th.

         Nomi



From allenday at ucla.edu  Thu May 11 20:17:43 2006
From: allenday at ucla.edu (Allen Day)
Date: Thu, 11 May 2006 17:17:43 -0700
Subject: [DAS2] Poster deadline for ISMB
In-Reply-To: <ryiody45fd2.fsf@spongecake.lbl.gov>
References: <ryiody45fd2.fsf@spongecake.lbl.gov>
Message-ID: <5c24dcc30605111717g9f93212y12e8e3c6b7ad70b1@mail.gmail.com>

I submitted a talk to the PLoS track that will touch on what we've been
doing, although it will be Assay-centric (see attached abstract).  If I'm
not accepted for a talk I can submit a more general DAS2 poster, given that
no-one else wants to do it...

Thanks for the heads-up on the BOSC date, I had totally forgotten about
that.

-Allen

On 5/11/06, Nomi Harris <nomi at fruitfly.org> wrote:
>
> Darn, I had it in my calendar to remind everyone that May 10th was the
> deadline for submitting poster abstracts to ISMB (wasn't there some vague
> plan to submit a poster about DAS/2?) and here it is, the 11th already.
> I bet they'd still accept the abstract if someone submitted it today or
> tomorrow along with profuse apologies about the lateness.
>
> BOSC abstracts (http://www.open-bio.org/wiki/BOSC_2006) aren't due until
> June 19th.
>
>          Nomi
>
> _______________________________________________
> DAS2 mailing list
> DAS2 at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/das2
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ismb.pdf
Type: application/pdf
Size: 36085 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20060511/4ba783e6/attachment.pdf>

From dalke at dalkescientific.com  Mon May 15 10:23:03 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 15 May 2006 08:23:03 -0600
Subject: [DAS2] status report
Message-ID: <49f160831a69404d9c0d672d28ae4c25@dalkescientific.com>

I've managed to get surprisingly little done in this last
week.  I've been getting ready for my move on Wednesday
morning.  Here's what I did:

  * I modified my GFF3 parser to emit basic DAS2XML, to use
for the writeback server.

My TODO:
  * Update the writeback spec to have microdelta support and
      a modified feature element with a "id used to be ..." attribute.

  * get complex feature examples from Lincoln and include
     into the spec

  * propose that features which are part of a complex annotation
     must have a "root" attribute with the URI of the top-level
     feature

  * work more on the validator

  * implement a simple writeback server


					Andrew
					dalke at dalkescientific.com



From dalke at dalkescientific.com  Mon May 15 11:47:02 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 15 May 2006 09:47:02 -0600
Subject: [DAS2] authentication
In-Reply-To: <65d553032331955821027cff5fc558bd@dalkescientific.com>
References: <65d553032331955821027cff5fc558bd@dalkescientific.com>
Message-ID: <c4caeb7d29fd847f2ade0c5e09871983@dalkescientific.com>

> I looked at HTTP digest authentication
>    http://www.ietf.org/rfc/rfc2617.txt
>    http://en.wikipedia.org/wiki/Digest_access_authentication

Grr.  I came across
   http://bitworking.org/news/Problems_with_HTTP_Authentication_Interop
which is a parody of the Monty Python Cheeseshop sketch.  The
summary is that digest has a lot of options, different servers
and client libraries do different things, so the de facto spec
is different than the written one.

Another solution is through cookies.  That's useful for web
browsers because it supports logout, but for a specialized
client (like we have) it's less useful.

Grr.

Okay, looks like the solutions are:
   1. pick a subset of HTTP Digest authentication
   2. Basic auth over HTTPs

with 1. in the lead.

					Andrew
					dalke at dalkescientific.com



From dalke at dalkescientific.com  Mon May 15 11:35:37 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 15 May 2006 09:35:37 -0600
Subject: [DAS2] authentication
Message-ID: <65d553032331955821027cff5fc558bd@dalkescientific.com>

We said we were going to talk about DAS authentication today.

The goal is to know who did a writeback edit.  A secondary
goal is to allow/deny read access.

I think the solution is to use HTTP Basic authentication.
This puts the username and password in the request headers
and is easy to implement.
   http://en.wikipedia.org/wiki/Basic_authentication_scheme

It's open to various attacks (sniffing the wire, replay attacks.)
The usual response, if this is a problem, is HTTP Digest
authentication.

I looked at HTTP digest authentication
    http://www.ietf.org/rfc/rfc2617.txt
    http://en.wikipedia.org/wiki/Digest_access_authentication

It's more complicated because it exchanges a nonce.  That's
a sort of key from the server combined on the client with
the password and a counter to generate an MD5 checksum.
The checksum is sent to the server in future requests.

It's more complicated.  I've never needed to support it
and I don't have experience with it.  I only know that
web sites do widely support it, along with many libraries.


Another solution is basic authentication over HTTPS.  I've
never had to support HTTPS either.


Yet another is to include the authentication information
in the payload
   <USER>Andrew</USER>
   <PASSWORD>12345</PASSWORD>

At best this will be equal to an existing HTTP authentication
scheme, with no existing library support, so I say no.


					Andrew
					dalke at dalkescientific.com



From Steve_Chervitz at affymetrix.com  Mon May 15 20:28:24 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 15 May 2006 17:28:24 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 15 May 2006
Message-ID: <C08E6AB8.1E695%Steve_Chervitz@affymetrix.com>

Notes from the weekly DAS/2 teleconference, 15 May 2006

$Id: das2-teleconf-2006-05-15.txt,v 1.1 2006/05/16 00:25:27 sac Exp $

Note taker: Steve Chervitz

Attendees: 
  Affy: Steve Chervitz, Gregg Helt
  CSHL: Lincoln Stein
  Dalke Scientific: Andrew Dalke
  UCLA: Allen Day, Brian O'connor

Action items are flagged with '[A]'.

These notes are checked into the biodas.org CVS repository at
das/das2/notes/2006. Instructions on how to access this
repository are at http://biodas.org

DISCLAIMER: 
The note taker aims for completeness and accuracy, but these goals are
not always achievable, given the desire to get the notes out with a
rapid turnaround. So don't consider these notes as complete minutes
from the meeting, but rather abbreviated, summarized versions of what
was discussed. There may be errors of commission and omission.
Participants are welcome to post comments and/or corrections to these
as they see fit. 


Status reports
--------------

gh: have been working on adding curation ability to IGB. Can do basic
creation and editing of features based on other features. Can write
out the curations in bed format. working on writing out das2xml.

gh: grant renewal - plan is to submit new on in October. we can extend
unused funds from original grant for two months after first grant
ends (so thru july). we'll also request bridge funding after our
funding runs out. 
Three things to do:
* no cost extension (of remaining money)
* request for bridge funding.
* re-application in oct w/ Suzi as PI

ad: (see email for my report). gff to das xml. writeback spec,
validator work. Can get it back up and running today. sweden and
iceland travel for teaching courses.

[A] andrew will get his validator app back online.

ls: got married, had lab retreat. had genome biology meeting. now am
finishing off two grants due tomorrow. no das-related work to report.

bo: finished up what we talked about (w/ gregg). sourceforge is
fixed. server is up but beware: CVS root has changed.

gh: it has been frustrating not being able to commit to sf the past week.

sc: we'll have to consider other options if sf continues to be so
unreliable. 

gh: other options?

ls: open-bio.org -- much less downtime than sf.

sc: My status: I've been working on getting affy das server registered
with Andreas. found bugs in affy das server's features response
(xml:base). gregg has fix. also working on updating the data served up
by our das and quickload servers - new version of human and mouse
genome assemblies. 

aday: set up database for writeback. xml parser using dom, handles
features, not type writeback. working on code that coverts xml into
sql insert statements. then we'll have writeback.

had trouble getting apache to expose the post data to me, need to
explore this more. other problem: loss of information in the way the
current das server exposes the way ontology terms are attached on to a
feature, as properties. e.g., mol function, cell component, biol
process, and sequence types - key is not go id, value is not.

[A] andrew, allen will think about best way to encode properties

ad: as new xml elements, extensions to das, so they could say this
means that exactly. you can specify what goes into which element.

aday: a way to insert property types.

ad: you extend das xml to include your own non-das element.

[A] andrew will write up email description of using das xml extension for
ontology terms

gh: if you have non das xml embedded in an xml feature, client gets
that, edits and sends back in writeback, does it have to send it back?

ad: yes, must return it back, otherwise it will be assumed to be a delete.

gh: could you put an example of a property of a feature being deleted
by omission?

ad: how about an accompanying document describing interaction behaviors.

[A] andrew will include example of writeback deletion by omission.

ad: why would client send back human readable text?

aday: so person wouldn't have to parse the ontology.  would prefer
giving back an accession not the human readable term since that
changes. need something that is reliably unique.

ad: new xml element

aday: what do we do for properties then?

ad: properties are a way for humans to create annotations, a
catch-all. Use them because you don't want to define a new xml
element, perhaps.

aday: will then have GO annotated elements, return as properties later
in response to a get request. could be confusing for clients.

ad: property table is accessible to humans, editable, no constraints
on what can be in there.

To summarize my position: let's use the 'x' in xml.

[A] gregg/andrew get response from suzi re: urls for ontology terms

Topic: Authentication
----------------------
ad: did research, see email. Two options: basic auth or http-digest
auth. Have never used it before. Don't know best practices.

gh: how about saying it's outside the scope. server supported on a
server-specific basis. we have enough stuff to figure out.

ad: fine. call it das 2.1

Other topics:
--------------
sc: writeback issue: confirmation from server before it modifies
anything on the server side. "You have requested to delete 17
features, and modify 13 others. Confirm?"

ad: what if thre are hundreds of modifications after serveral hours of
editing? will the user be expected to confirm each one?

sc: user could commit changes in a more piecemeal fashion to avoid
that situation. 

gh: I would prefer to have this done on the client side. the editing
tool can provide this functionality more effectively, flexibly, etc.

sc: ok. seems reasonable. but will server send confirmation response
to client that the writeback request succeeded? and the client will be
unable to issue a request until after the previous request succeeded?

gh: yes.







From Steve_Chervitz at affymetrix.com  Tue May 16 13:10:18 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Tue, 16 May 2006 10:10:18 -0700
Subject: [DAS2] most up-to-date mouse das? (mm7)
In-Reply-To: <83722dde0605160152q200d9c0as5ad2b5cd665a72b0@mail.gmail.com>
Message-ID: <C08F558A.1E6D1%Steve_Chervitz@affymetrix.com>

Hi Ann,

The list address has changed. It's now this: das2 at lists.open-bio.org

As for your question, check out the DAS registry server at the Sanger:

http://das.sanger.ac.uk/registry/

I don't think the registry provides an indication of how current the
annotations on each registered server for a given data source, such as
Entrez Gene. It would be a good piece of data to see, though.

As for the Affymetrix DAS/2 server, the mm7 annotations were last updated on
April 19 2006:

http://netaffxdas.affymetrix.com/das2/sources

The available annotations come from the UCSC server, and derive from the
knownGene, all_mrna, genscan, and refFlat files (called 'refseq' on the das
server). Looks like the knownGene data was last updated by UCSC on 15 Dec
2005:
http://hgdownload.cse.ucsc.edu/goldenPath/mm7/database/

Technical note: The xml:base attribute in the das2xml features document
returned by the Affy DAS/2 server is currently incorrect. It should be

xml:base="http://netaffxdas.affymetrix.com/das2/M_musculus_Aug_2005/features
"

instead of

xml:base="http://127.0.0.1:9021/das2/M_musculus_Aug_2005/features"

This will be fixed in the near future.

Steve

> From: Ann Loraine <aloraine at gmail.com>
> Date: Tue, 16 May 2006 03:52:29 -0500
> To: Steve Chervitz <Steve_Chervitz at affymetrix.com>
> Subject: Fwd: most up-to-date mouse das? (mm7)
> 
> Hi Steve,
> 
> Would you post this to the DAS/2 list for me?
> 
> Sorry to bother you, but for some reason my message didn't appear on the list.
> 
> -Ann
> 
> ---------- Forwarded message ----------
> From: Ann Loraine <aloraine at gmail.com>
> Date: May 15, 2006 3:35 PM
> Subject: most up-to-date mouse das? (mm7)
> To: Andrew Dalke <dalke at dalkescientific.com>, DAS/2 <das2 at portal.open-bio.org>
> 
> 
> Hi!
> 
> I working on a QTL study and need to get all the genes mapping to
> various regions under peaks.
> 
> I have the genomic coordinates for the regions so it should be very
> simple for me to get all accessions (feature ids) underneath those
> regions using DAS.
> 
> My question is: what is the most up-to-date server for mm7?
> 
> Here, of course, is UCSC:
> 
> http://genome.cse.ucsc.edu/cgi-bin/das/mm7/features?segment=chr1:3000000,40000
> 00;type=knownGene
> 
> Ultimately, I'd like to get Entrez Gene ids for the genes under the
> peaks so that I can start sifting through the candidates using GO.
> 
> Any tips would be gratefully accepted!
> 
> All the best,
> 
> Ann
> 
> --
> Ann Loraine
> Assistant Professor
> Section on Statistical Genetics
> University of Alabama at Birmingham
> http://www.ssg.uab.edu
> http://www.transvar.org
> 
> 
> -- 
> Ann Loraine
> Assistant Professor
> Section on Statistical Genetics
> University of Alabama at Birmingham
> http://www.ssg.uab.edu
> http://www.transvar.org



From aloraine at gmail.com  Thu May 18 00:20:47 2006
From: aloraine at gmail.com (Ann Loraine)
Date: Wed, 17 May 2006 23:20:47 -0500
Subject: [DAS2] most up-to-date mouse das? (mm7)
In-Reply-To: <C08F558A.1E6D1%Steve_Chervitz@affymetrix.com>
References: <83722dde0605160152q200d9c0as5ad2b5cd665a72b0@mail.gmail.com>
	<C08F558A.1E6D1%Steve_Chervitz@affymetrix.com>
Message-ID: <83722dde0605172120t5853b30al3f931bd6d73092df@mail.gmail.com>

Hi Steve,

Thank you very much for the info!

Now I have another question...

I'd like to look up the genomic coordinates of a list of mouse genes
using their numeric Entrez Gene ids.

If it's not too much bother, do you think you'd be able to give me
some tips on how to do this using DAS?

btw, the DAS services have been hugely helpful to me in the last week.
We have already found some interesting results with minimal coding.
And the coding was actually fun because there was NO SCREEN-SCRAPING.
Pure bliss.

-Ann

On 5/16/06, Steve Chervitz <Steve_Chervitz at affymetrix.com> wrote:
> Hi Ann,
>
> The list address has changed. It's now this: das2 at lists.open-bio.org
>
> As for your question, check out the DAS registry server at the Sanger:
>
> http://das.sanger.ac.uk/registry/
>
> I don't think the registry provides an indication of how current the
> annotations on each registered server for a given data source, such as
> Entrez Gene. It would be a good piece of data to see, though.
>
> As for the Affymetrix DAS/2 server, the mm7 annotations were last updated on
> April 19 2006:
>
> http://netaffxdas.affymetrix.com/das2/sources
>
> The available annotations come from the UCSC server, and derive from the
> knownGene, all_mrna, genscan, and refFlat files (called 'refseq' on the das
> server). Looks like the knownGene data was last updated by UCSC on 15 Dec
> 2005:
> http://hgdownload.cse.ucsc.edu/goldenPath/mm7/database/
>
> Technical note: The xml:base attribute in the das2xml features document
> returned by the Affy DAS/2 server is currently incorrect. It should be
>
> xml:base="http://netaffxdas.affymetrix.com/das2/M_musculus_Aug_2005/features
> "
>
> instead of
>
> xml:base="http://127.0.0.1:9021/das2/M_musculus_Aug_2005/features"
>
> This will be fixed in the near future.
>
> Steve
>
> > From: Ann Loraine <aloraine at gmail.com>
> > Date: Tue, 16 May 2006 03:52:29 -0500
> > To: Steve Chervitz <Steve_Chervitz at affymetrix.com>
> > Subject: Fwd: most up-to-date mouse das? (mm7)
> >
> > Hi Steve,
> >
> > Would you post this to the DAS/2 list for me?
> >
> > Sorry to bother you, but for some reason my message didn't appear on the list.
> >
> > -Ann
> >
> > ---------- Forwarded message ----------
> > From: Ann Loraine <aloraine at gmail.com>
> > Date: May 15, 2006 3:35 PM
> > Subject: most up-to-date mouse das? (mm7)
> > To: Andrew Dalke <dalke at dalkescientific.com>, DAS/2 <das2 at portal.open-bio.org>
> >
> >
> > Hi!
> >
> > I working on a QTL study and need to get all the genes mapping to
> > various regions under peaks.
> >
> > I have the genomic coordinates for the regions so it should be very
> > simple for me to get all accessions (feature ids) underneath those
> > regions using DAS.
> >
> > My question is: what is the most up-to-date server for mm7?
> >
> > Here, of course, is UCSC:
> >
> > http://genome.cse.ucsc.edu/cgi-bin/das/mm7/features?segment=chr1:3000000,40000
> > 00;type=knownGene
> >
> > Ultimately, I'd like to get Entrez Gene ids for the genes under the
> > peaks so that I can start sifting through the candidates using GO.
> >
> > Any tips would be gratefully accepted!
> >
> > All the best,
> >
> > Ann
> >
> > --
> > Ann Loraine
> > Assistant Professor
> > Section on Statistical Genetics
> > University of Alabama at Birmingham
> > http://www.ssg.uab.edu
> > http://www.transvar.org
> >
> >
> > --
> > Ann Loraine
> > Assistant Professor
> > Section on Statistical Genetics
> > University of Alabama at Birmingham
> > http://www.ssg.uab.edu
> > http://www.transvar.org
>
>


-- 
Ann Loraine
Assistant Professor
Section on Statistical Genetics
University of Alabama at Birmingham
http://www.ssg.uab.edu
http://www.transvar.org



From dalke at dalkescientific.com  Mon May 22 06:14:44 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 22 May 2006 10:14:44 +0000
Subject: [DAS2] will miss today's meeting
Message-ID: <a36109163a8cd0fd4a7b2ddaf7b8efb5@dalkescientific.com>

Hey guys,

   I'm not going to be able to make today's meeting.  Won't
be able to get the free time for it.

					Andrew
					dalke at dalkescientific.com



From Gregg_Helt at affymetrix.com  Mon May 22 12:16:54 2006
From: Gregg_Helt at affymetrix.com (Helt,Gregg)
Date: Mon, 22 May 2006 09:16:54 -0700
Subject: [DAS2] today's DAS/2 meeting
Message-ID: <C71929195D04BF48BAECD499AF717B480198CADF@msex02.affymetrix.com>

 I think it will be a short meeting today.  Mostly I just want to get a
status report from everyone, especially how writeback implementation is
going.

	Thanks,
	Gregg

> -----Original Message-----
> From: das2-bounces at lists.open-bio.org [mailto:das2-bounces at lists.open-
> bio.org] On Behalf Of Andrew Dalke
> Sent: Monday, May 22, 2006 3:15 AM
> To: DAS/2
> Subject: [DAS2] will miss today's meeting
> 
> Hey guys,
> 
>    I'm not going to be able to make today's meeting.  Won't
> be able to get the free time for it.
> 
> 					Andrew
> 					dalke at dalkescientific.com
> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/das2



From Steve_Chervitz at affymetrix.com  Mon May 22 14:09:18 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 22 May 2006 11:09:18 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 22 May 2006
Message-ID: <C0974C5E.1E7FC%Steve_Chervitz@affymetrix.com>

Notes from the weekly DAS/2 teleconference, 22 May 2006

$Id: das2-teleconf-2006-05-22.txt,v 1.1 2006/05/22 18:07:09 sac Exp $

Note taker: Steve Chervitz

Attendees: 
  Affy: Steve Chervitz, Ed Erwin, Gregg Helt
  CSHL: Lincoln Stein
  U Alabama: Ann Loraine
  UCLA: Allen Day

Action items are flagged with '[A]'.

These notes are checked into the biodas.org CVS repository at
das/das2/notes/2006. Instructions on how to access this
repository are at http://biodas.org

DISCLAIMER: 
The note taker aims for completeness and accuracy, but these goals are
not always achievable, given the desire to get the notes out with a
rapid turnaround. So don't consider these notes as complete minutes
from the meeting, but rather abbreviated, summarized versions of what
was discussed. There may be errors of commission and omission.
Participants are welcome to post comments and/or corrections to these
as they see fit. 


Topic: status reports
---------------------

aday: working on converting xml to sql for the writeback
functionality. Need to bind to apache. should be done today.

gh: would like to test writeback into IGB.

aday: will give url to gregg. if you post an xml doc to that url. will
get back a response mapping provisional names to persisted names in
the db. 

gh: editing too?
aday: no, creation only, at least today. maybe editing later this week.
gh: I could get some writeback for creation in igb this week. with
effort. 

aday: currently writes yeast and human features to the same data
source.   
gh: fine for testing. will focus on human.

sc: probably should just send to url to gregg then to send to list
once we test it a bit.

[A] allen will send info to gregg re: accessing his writeback server

gh: turning on curation in igb. need to connect to das
writeback. previously wasn't connected. given I have a das writer it
should go smoothly. aiming to get something going this week.
Not checked in. Still struggling with sf cvs change.

other news: applying for no cost extension to current grant. have
100-200k left. still determining. need to get app for extension in
today. this is more than we though. should last at least another
couple of month. will let folks know when we know.

ls: pretty good.

gh: will contact p. good about bridge funding until new grant is
funded. working with suzi to re-submit in October with her as PI.

sc: sf cvs trouble?

gh: hangs my jbuilder ide now. I updated repository information in CVS
Root files using perl. JBuilder doesn't like it.

sc: I did the same sort of perl-based editing of CVS Root files and it
worked for me.

al: Doing overrepresentation analysis for QTL studies.
Usually this is for microarray analysis, where you want to see what
things the expressed genes they have in common. you collect GO
annotations, if you see some recurring in the list, you infer that that is
important/involved in the process.
We want to do overrepresentation alaysis for gene assoc studies.
look for go terms/ pathways represented in the list of gene
associations. having trouble getting a null model. plan to hammer on a
das server to get a null distribution. would that be ok? thinking
about using the UCSC das server.

gh: would be a great test to see how they handle lots of
queries. 

ls: also try the biomart interface at ensembl. das feeds into it
internally. there are api's for variety of languages, or can send sql
to it. try the ucsc das server first.

al: is there a contact person for ucsc das server?

ls: jim kent.

aday: write to genome mailing list. they're responsive.

al: top page of das server has message that implies it's not
well-supported. 

gh: the things you're interested in will do fine (txts, rna, gene
predictions). doesn't do well on extra stuff in their db (fastcons
conservation scores across regions of conservation, expression score,
etc). For basic annotations it works fine.

al: most interested in the "knownGene" data type. das server gives
back accessions. annotations are encoded. need to look up entrez gene id
from accession. using NCBI mapping files to get mRNA accessions.
then get ontology annotations, also distributed via NCBI.

gh: in das/2 this is handled better with xids (this das id = that
entrez id).  
al: will ucsc update their server to das/2?
gh: more likely after grant renewal.

al: affy server?
sc: we have knownGene annotations.

al: doing this for a statistical genetics class this fall. expect lots
of hits on the server then.

[A] Ann will test hammering on the affy das/2 server before turning her
students loose on it this Fall.

sc: no writeback related work, but attended the JavaOne meeting last
week. Some things of interest to our work:
1) Presentation on the ATOM protocol and REST in general. Mentioned
   something called WADL that is a REST-based web service to automate
   client-side interactions. Seemed more general that ATOM since one
   can use it to implement ATOM.
2) Eclipse RCP (rich client platform). Is a gutted version of the
   Eclipse Java IDE with all the java tools removed. You're left with
   lots of core functionality that can be of use for any java app. Was
   used by NASA when they needed to integrate lots of heterogeneous
   apps related to the Mars space mission. Seemed like we have a
   similar situation with IGB + Apollo integration. The Eclipse RCP
   gives you a framework for assembling tightly integrated components
   that are loosely coupled.

gh: do all the components need to be eclipse plugins?
sc: yes (believe so). not sure how much work that entails.

gh: I looked at using eclipse and netbeans back when starting to
develop IGB, but that was a long time ago. might be worth looking
again. 

sc: there's a new book on building Eclipse RCP apps. On my desk.

[A] gregg will look into the eclipse RCP as a possible app framework.

3) Extreme GUI makeover talk was interesting to see what you can do
   with Java 2D to dress up a "plain-jane" application. This year,
   they created a suped-up version of the Thunderbird email app. There
   are other enhancements coming with the next java release that
   improve java on the desktop (version 6 called  Mustang, due in
   October). 

4) java.util.concurrent package in java 5.0 has quite a lot of
   functionality for implementing things in your app that require
   multithreading support. There is a back port to JDK 1.4.

5) Scripting in java was very big. Using the groovy scripting
   language, with simplified syntax but access to all of java.
   AJAX was also very hot.

gh: I saw something using gbrowse and AJAX.
ls: this is an ian holmes project. It has google maps like
interface. change track order by tracking, smooth scroll and and
zooming. the idea is to supercede igb so people can do it on the web.

al: you need to do 1d not 2d zooming.
ls: it does that.
gh: the hard part is dealing with lots of data.

al: I always get questions from people who look at google maps and
say, "how come you genome people can't do something like this?"

ls: it's running now. go to http://genome.biowiki.org/ or look for
"gbowse ajax client" on biowiki.org.


Topic: Next meeting
--------------------
gh: next monday is US holiday (memorial day, 5/29). next meeting will
therefore be in two weeks. 



From Steve_Chervitz at affymetrix.com  Mon May 22 15:48:13 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 22 May 2006 12:48:13 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 22 May 2006
In-Reply-To: <C0974C5E.1E7FC%Steve_Chervitz@affymetrix.com>
Message-ID: <C097638D.1E807%Steve_Chervitz@affymetrix.com>

Forgot to mention one more item of interest from the JavaOne meeting:

Steve wrote:
> sc: no writeback related work, but attended the JavaOne meeting last
> week. Some things of interest to our work:
>...

6) Probably the most relevant to this group was a talk called,
   "How to write an API that will stand the test of time". While it
   was definitely java-centric, there are some nuggets that apply to
   any software or spec writing team:
    * Be use-case driven
        - focus on what people need now, not what might be useful later.
    * Expose only necessary functionality
        - minimize your API's "surface area". This means less for
          users to learn, and less for devs to implement and maintain.
    * Be predictable and consistent
        - an interface that is predictable serves better than one
          which is locally optimal but inconsistent within the
          framework.
    * Design to test
        - write tests to fully cover API functionality.
    * Always think about evolution
        - the first version is never perfect.
        - allow for the co-existence of multiple versions.
    * An anti-example: The JavaMail API, which is optimized for
      implementing new wire protocols, not for the bulk of potential
      users who just want to read/send mail.
    * Links: 
       - http://openide.netbeans.org/tutorial/api-design.html
       - http://www.artima.com/apidesign/index.html


Steve



From ed_erwin at affymetrix.com  Mon May 22 18:28:20 2006
From: ed_erwin at affymetrix.com (Ed Erwin)
Date: Mon, 22 May 2006 15:28:20 -0700
Subject: [DAS2] authentication
In-Reply-To: <c4caeb7d29fd847f2ade0c5e09871983@dalkescientific.com>
References: <65d553032331955821027cff5fc558bd@dalkescientific.com>
	<c4caeb7d29fd847f2ade0c5e09871983@dalkescientific.com>
Message-ID: <44723B04.4060005@affymetrix.com>

I vote for basic auth with optional HTTPS.  That seems easy to implement 
and HTTPS can be added or not depending on how paranoid the particular 
site is.

Just my few cents.


Andrew Dalke wrote:
>>I looked at HTTP digest authentication
>>   http://www.ietf.org/rfc/rfc2617.txt
>>   http://en.wikipedia.org/wiki/Digest_access_authentication
> 
> 
> Grr.  I came across
>    http://bitworking.org/news/Problems_with_HTTP_Authentication_Interop
> which is a parody of the Monty Python Cheeseshop sketch.  The
> summary is that digest has a lot of options, different servers
> and client libraries do different things, so the de facto spec
> is different than the written one.
> 
> Another solution is through cookies.  That's useful for web
> browsers because it supports logout, but for a specialized
> client (like we have) it's less useful.
> 
> Grr.
> 
> Okay, looks like the solutions are:
>    1. pick a subset of HTTP Digest authentication
>    2. Basic auth over HTTPs
> 
> with 1. in the lead.
> 
> 					Andrew
> 					dalke at dalkescientific.com
> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/das2


From Steve_Chervitz at affymetrix.com  Mon May 22 19:52:27 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 22 May 2006 16:52:27 -0700
Subject: [DAS2] BOSC 2006 2nd Call for Papers
In-Reply-To: <4471D9C0.1050003@duke.edu>
Message-ID: <C0979CCB.1E846%Steve_Chervitz@affymetrix.com>


On behalf of Darin London <darin.london at duke.edu>:

> BOSC 2006 2nd Call for Papers
> 
> This is the second and last official call for speakers to submit their
> abstracts to speak at  BOSC 2006
> in Fortaleza, Brasil.  In order to be considered as a potential speaker,
> an abstract must be recieved by
> Monday, June 5th, 2006.  We look forward to a great conference this
> year. Please consult
> The Official BOSC 2006 Website at:
> 
> http://www.open-bio.org/wiki/BOSC_2006
> 
> for more details and information.
> 
> 
> In addition, a BOSC weblog has been setup to make it easier to
> desiminate all BOSC
> related announcements:
> 
> http://wiki.open-bio.org/boscblog/
> 
> And if you have an ICAL compatible Calendar, there is an EventDB
> calendar set up with all
> BOSC related deadlines.
> 
> http://eventful.com/groups/G0-001-000014747-0
> 
> More information about ISMB can be found at the Official ISMB 2006 Website:
> 
> http://ismb2006.cbi.cnptia.embrapa.br/
> 
> Thank You, and we look forward to seeing you all,
> 
> The BOSC Organizing Committee.

 



From Steve_Chervitz at affymetrix.com  Tue May 23 15:33:29 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Tue, 23 May 2006 12:33:29 -0700
Subject: [DAS2] WADL
In-Reply-To: <C0974C5E.1E7FC%Steve_Chervitz@affymetrix.com>
Message-ID: <C098B199.1E8B8%Steve_Chervitz@affymetrix.com>


As I mentioned in the weekly meeting notes:

> sc: no writeback related work, but attended the JavaOne meeting last
> week. Some things of interest to our work:
>
> 1) Presentation on the ATOM protocol and REST in general. Mentioned
>    something called WADL that is a REST-based web service to automate
>    client-side interactions. Seemed more general that ATOM since one
>    can use it to implement ATOM.

WADL stands for Web Application Description Language and is a fairly recent
invention, so it's not in widespread use.

Documentation can be found at
http://research.sun.com/techrep/2006/abstract-153.html

Here's the abstract:

> An increasing number of Web-based enterprises (Google, Yahoo, Amazon, Flickr -
> to name but a few) are developing HTTP-based applications that provide access
> to their internal data using XML. Typically these applications are described
> using a combination of textual protocol descriptions combined with XML
> schema-based data format descriptions; WADL is designed to provide a machine
> processable protocol description format for use with such HTTP-based Web
> applications, especially those using XML.

I haven't looked at it enough to tell whether it would be useful for DAS
applications. The DAS protocol does some things that are targetted by WADL,
like defining resource relationships. But other parts of WADL might be of
use for DAS, such as describing relationships between DAS servers, or
between DAS servers and other non-DAS resources.

It may be possible to describe DAS using WADL, but I haven't tried.

Steve




From dalke at dalkescientific.com  Mon May  1 02:37:29 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Sun, 30 Apr 2006 20:37:29 -0600
Subject: [DAS2] May 1 is a UK holiday
Message-ID: <9c6cb86e3269238eb234bb4b2c6da293@dalkescientific.com>

Andreas write to my in private email saying

> here in england  1st of may is a public holiday...

The hope idea was to talk about writeback, but the UK people (and
most specifically Roy) won't be able to make it.

Does anyone have any feedback on the writeback spec or comments
on my solution to splits and joins?





					Andrew
					dalke at dalkescientific.com



From dalke at dalkescientific.com  Mon May  1 06:14:10 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 1 May 2006 00:14:10 -0600
Subject: [DAS2] new stylesheet syntax
Message-ID: <8dfd12074a5404e2577db4547888d08a@dalkescientific.com>

I've been thinking about the stylesheet document format.  I've
changed it somewhat.  Here's what the latest version looks like

<STYLESHEET>
   <STYLE 
uri="http://biodas.org/das2/bos_taurus/Dec2006/type/five_prime_UTR"
          zoom="high">
     <BOX fgcolor="red" />
     <LABEL fgcolor="black" />
   </STYLE>

   <STYLE 
uri="http://biodas.org/das2/bos_taurus/Dec2006/type/five_prime_UTR"
          zoom="medium">
     <BOX fgcolor="red" />
   </STYLE>

   <STYLE 
uri="http://biodas.org/das2/bos_taurus/Dec2006/type/five_prime_UTR"
          zoom="low">
     <LINE fgcolor="red" />
   </STYLE>

   <STYLE 
uri="http://biodas.org/das2/bos_taurus/Dec2006/type/strong_gene">
     <BOX fgcolor="black" border_style="solid" />
   </STYLE>

   <STYLE 
uri="http://biodas.org/das2/bos_taurus/Dec2006/type/moderate_gene">
     <BOX fgcolor="black" border_style="dotted" />
   </STYLE>

   <STYLE uri="http://biodas.org/das2/bos_taurus/Dec2006/type/weak_gene">
     <BOX fgcolor="gray" border_style="dotted" />
   </STYLE>

</STYLESHEET>

The change is that the attributes in the <STYLE> element are only
used as selectors, that is, fields which select feature types.  I've
moved the 'width', 'height', and 'bump' elements to a <SET> element.

The core of the schema looks like this.

style = element STYLE {
   # These are selectors.
   attribute uri { text },
   attribute zoom { "high" | "medium" | "low" }?,

   set_element,

   # Must specify a glyph, may specify a label.  Can be in
   # either order.
   ((glyph, label?) | (label, glyph))

}

# Set default depiction properties
set_element = element SET {
   height?,
   width?,
   bump { "yes" | "no" }?
}

If you hadn't looked at the previous schema, the height, width,
and bump attributes were in the STYLE attributes along with 'fgcolor'
and 'bgcolor', which provided defaults for the glyphs and title, but
the latter two could override the defaults.

The main reason for the change is to separate selectors from
the actual style.  I got rid of the default fg/bg colors because

					Andrew
					dalke at dalkescientific.com



From Steve_Chervitz at affymetrix.com  Mon May  1 17:56:20 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 01 May 2006 10:56:20 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 5 May 2006
Message-ID: <C07B99D4.1E185%Steve_Chervitz@affymetrix.com>

Notes from the weekly DAS/2 teleconference, 5 May 2006

$Id: das2-teleconf-2006-05-01.txt,v 1.1 2006/05/01 17:52:48 sac Exp $

Note taker: Steve Chervitz

Attendees: 
  Affy: Steve Chervitz, Ed E., Gregg Helt
  Dalke Scientific: Andrew Dalke
        
Action items are flagged with '[A]'.

These notes are checked into the biodas.org CVS repository at
das/das2/notes/2006. Instructions on how to access this
repository are at http://biodas.org

DISCLAIMER: 
The note taker aims for completeness and accuracy, but these goals are
not always achievable, given the desire to get the notes out with a
rapid turnaround. So don't consider these notes as complete minutes
from the meeting, but rather abbreviated, summarized versions of what
was discussed. There may be errors of commission and omission.
Participants are welcome to post comments and/or corrections to these
as they see fit. 


Proposed agenda:
* Discuss feedback on Andrew's writeback spec and solution to splits
  and joins


Topic: Writeback
-----------------

[A] Collect feedback from UK folks at next mtg since today is UK holiday.

ad: likes the model where you have multiple, composed change
statements rather than a single document describing all changes.
if you have history tracking, you have the server understand
versioning. 

gh: if you do a merge by deleting a feat and then extending another
feat or delete 2 and create one, how do you get history tracking?

ad: you have a single delta, describing "delete this, extend
that". Lots of small deltas comprise a big delta --- wraps up all of
the curator's work in a single session.
gh: delta lets you figure out what came from what in the change element.

ad: when do people use the split and merge history trail?
gh: to ask, "what the hell happened to the gene that I'm interested
in?" when the thing it got merged into has a completely different id.

ad: what I'm proposing could capture this history,
gh: it allows the capturing as long as client is careful about how it
constructs the change and server is configured to track changes.
ad: client and server both need special care.
gh: I could see a more direct way of doing it that doesn't require
such carefulness. I.e., explicitly say what a new feature is derived
from.
ad: this could be a micro-delta, such as a comment or remark saying
"this is a merger of x and y" along with a controlled vocab of changes.
I'm hoping people had a chance to look at it.

gh: to me it looks sufficient, but the main question is, does this
meet requirements of existing Apollo and Otter systems?

[A] Andrew will talk directly with Nomi and Otter folks.

gh: so we need to get someone to implement. Allen and Brian.

[A] Allen and/Brian provide feedback to Andrew for writeback based on their
impl.

gh: there's still the question of having arbitrary xml in a feature,
editing it, and how to indicate what should/shouldn't be changed.
Also, the locking document isn't on-line.

ad: the locking document is on my todo list. as we discussed, locking
and writeback are orthogonal.

[A] Andrew will create a locking document.

Topic: Stylesheets
------------------

ad: changed stylesheets a bit.
ee: I like getting color etc. out of the main tag. still not sure how to
impl this in IGB. We may ignore stuff about hi/low zoom, box, line.
ad: what are units of height and width. pixels, em?
sc: we could specify units in the document.
ad: ...

ad: Also, does width/height include the label and where is the label
to be positioned? 
ee: could just let it be undefined. leave it up to the
browser. There's too much variation in what sort of vizualization
different viewers will be doing.

ad: do we need to specify height/width?
ee: shouldn't need width since that's based on chromosomal
coordinates. maybe need a minimum width.
ad: would be glad to drop width/height from stle element.

[A] Andrew will drop height/width from stylsheet until there's a clear need.


Topic Status
------------

ee: investigated bug for our old das servers are not returning unique
ids for certain things (alignments).  in our das/1 parser, if both
things come back with same id in one request, it merges, if they come
back in different requests, the second will be ignored (will say it's
already seen it). They need to modify server to return unique ids.

ad: seems to be a common problem.

gh: still on vacation. Need to set up teleconf to discuss resubmission
of renewal grant, and the possibility of bridge funding until we get
another grant.
ad: what about the no-cost funding extension?
gh: still figuring out what we have funding-wise.

sc: Fixed the cronjob on biodas.org to CVS update the spec
documentation, so the docs on the website will stay current now.
Other things: I've been playing with ruby on rails for rapid web app
development. so far I'm impressed. The URL-based nature of it's
request processing makes me think about hooking it up with das, making
a simple das server and/or viewer. E.g., table-based views of
sources, versions, types. We could possibly do web-based graphical
views as well.
ad: the das url structure should make this feasible. processing
queries would take some work. there's an analogous python approach too.
sc: might be nice to have sample das servers and/or web-based viewers
in python, ruby, perl, and java.

[A] steve will set up a demo das server/client using ruby on rails.



From Steve_Chervitz at affymetrix.com  Mon May  1 18:28:53 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 01 May 2006 11:28:53 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 5 May 2006
In-Reply-To: <C07B99D4.1E185%Steve_Chervitz@affymetrix.com>
Message-ID: <C07BA175.1E190%Steve_Chervitz@affymetrix.com>

Correction: The date for this teleconf was 1 May 2006 (today).


> From: Steve Chervitz <Steve_Chervitz at affymetrix.com>
> Date: Mon, 01 May 2006 10:56:20 -0700
> To: DAS/2 <das2 at lists.open-bio.org>
> Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 5 May 2006
> 
> Notes from the weekly DAS/2 teleconference, 5 May 2006
> 
> $Id: das2-teleconf-2006-05-01.txt,v 1.1 2006/05/01 17:52:48 sac Exp $

Notes from the weekly DAS/2 teleconference, 1 May 2006

$Id: das2-teleconf-2006-05-01.txt,v 1.2 2006/05/01 18:19:01 sac Exp $

Steve 




From dalke at dalkescientific.com  Tue May  2 14:43:09 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Tue, 2 May 2006 08:43:09 -0600
Subject: [DAS2] Fwd: DAS writeback
Message-ID: <f8ad58f58b4b9850ee2caa4b6279f247@dalkescientific.com>

I emailed Nomi regarding talking about writeback.  She suggested
I talk with Mark Gibson instead.  Here's the core part of his
response on splits&joins

[Andrew]
>> One option is to track exactly what happened.  For exampe, if
>> features F1 and F2 are merged into F3 then either
>>   DELETE F1
>>   DELETE F2
>>   CREATE F3
>>      <derives from F1>
>>      <derives from F2>
>> -or-
>>   DELETE F1
>>   MODIFY F2
>>      -change the range-
>>      <derives from F1>

[Mark]
> this is essentially what apollo does and i dont like it.
>
> in apollo the user chooses the name from a merge which can be F1-name, 
> F2-name or temp-name (new name from db) - if F1 is picked then f2 
> should be deleted & vice versa - apollo doesnt do this proper at the 
> moment (historical reasons - user didnt use to pick the name & the 
> name picking was hacked yada yada...)

[Andrew]
>> In this case the <derives from *> means the information is included
>> in the feature record.
>> I proposed an alternative, which I'm calling a microdelta.
>>   REASON: merge
>>   DELETE F1
>>   DELETE F2
>>   CREATE F3

[Mark]
> i like this one the best. I think its valuable to record in the record 
> that merge (&splits) have occured - and this makes it the most 
> obvious. Some databases like to track merge & splits (like TAIR - i 
> dont think chado tracks this but it should)
>
> i think its worthy on its own and its muddled to shove this in the 
> feature with derives from.


					Andrew
					dalke at dalkescientific.com



From Steve_Chervitz at affymetrix.com  Tue May  9 01:44:57 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 08 May 2006 18:44:57 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 8 May 2006
Message-ID: <C0854229.1E48F%Steve_Chervitz@affymetrix.com>

Notes from the weekly DAS/2 teleconference, 8 May 2006

$Id: das2-teleconf-2006-05-08.txt,v 1.1 2006/05/09 01:43:17 sac Exp $

Note taker: Steve Chervitz

Attendees: 
  Affy: Steve Chervitz, Ed E., Gregg Helt
  Dalke Scientific: Andrew Dalke
  Sanger: Andreas Prlic, Roy
  UC Berkeley: Suzi Lewis
  UCLA: Brian O'connor

Action items are flagged with '[A]'.

These notes are checked into the biodas.org CVS repository at
das/das2/notes/2006. Instructions on how to access this
repository are at http://biodas.org

DISCLAIMER: 
The note taker aims for completeness and accuracy, but these goals are
not always achievable, given the desire to get the notes out with a
rapid turnaround. So don't consider these notes as complete minutes
from the meeting, but rather abbreviated, summarized versions of what
was discussed. There may be errors of commission and omission.
Participants are welcome to post comments and/or corrections to these
as they see fit. 



Topic: Writeback
----------------

[A] Andrew will find out status of biopackages server re: writeback

Topic: Stylesheets  (we'll return to writeback later...)
------------------

ad: stylesheet spec is out. see recent email post.

roy: similar to z-map approach, not too much different.

ap: you have a link to type. below you provide how you want to
draw. if you have a line, you can also have a wireframe, cartoon,
spacefill. Allow a list of elements that you want. Client can choose.

ad: for structure we might want to have a set up where most of
residues are in one depiction, some subset of residues are in an
alternative depection (e.g., active site residues are
spacefilled). How would this be addressed? (currently an open question)

[suzi joins.]

gh: getting back to writeback. suzi's feedback. plans for apollo.

sl: no resources at the moment. we have a proposal just for apollo RSA
for development. chances are not as good for funding. P. good
recommends bundling with das and submit a das grant.

gh: pushing back application a month or two to give gh a chance to
revise it. a good idea.

sl: the plan is to give us a more time to reivse das grant.

gh: given that there's no work on apollo writeback, looking at adding
this functionality to igb.

sl: what about pirating some apollo stuff for editing for inclusion
into igb?

gh: more detail in apollo. merge, split, etc. are in igb now in
prototype form. Looking at working on this to get some primitive
writeback before end of current grant cycle.

we may need to put more emphasis on cross client interaction.

sl: love the idea, but reviewers didn't think it feasible.
if we do cross component communication, the bulk of the work is
getting enough forward progress to make a more persuasive case in the
proposal. 

gh: resubmitting date? july?

sl: talk to p. good. july will be difficult for me. out of country.

gh: plan is for suzi to be PI.

sc: date? 

gh: need to get in touch w/ peter about this. to get renewal status
there is a time frame.

Topic: writeback (continued)
----------------------------

ap: if a feature is deleted and new is created, how to know the
history linked to new feature? similar to discussion andrew fwd to
list. didn't see any conclusion.

ad: there is no conclusion for it. the details are server dependent. a
server the impl history tracking can post to the feature itself.

sl: if it's deleted, it's deleted. this isn't an undoable delete is
it?

gh: grouping the changes. within the post, you have a grouping so you
are merging two things, they are encapsulated w/in a change element.

ad: orig idea was one big delta doc, including all feats add, modify,
deleted. But you want to track things more closely. proposed a delta
doc that groups things. then proposed a micro delta - these features got
merged,... and a remark statement. why the merge happened. conceptual.
whole writeback is a list of microdeltas.

people that impl feature history do it as they want to. extension to
feature. could also do it via another request.

server could track of everything that happened on server. track all
history if it wants to.

[see andrew's email for more information about this.]

gh: andrew should touch base with allen and brian progress on
biopackages server. if you can impl something in a few days go for it.

ad: like the in-memory database approach.

gh/sl: fond of this approach.

roy: favored sending back a mapping document.

ap: this features follows up some features (uri) that has been
deleted. permits link from new to old feature.

roy: post data to server, server replies with same document, modified
according to what the server has done. has to do work to redraw,
rather than have mapping doc that goes from and to, that info is in
the features themselves.

ad: my mapping doc, forces client to figure out what the new feats
are. your approach means less client work.

roy: client can't toss everything if you're sending a delta across.

ad: just everything that's changed.

gh: did you send email about this?
roy: just to andrew. can post to the list.

[A] roy/andrew will forward message re: writeback to the list

ad: like the idea of keeping feat element exactly the same. willing to
be less pure about it.

roy: think of it as extending the read-only server. extending the
feature as well.

[A] andrew will change spec to respond with a full feat doc for feats that
changed.

roy: only changes that the client sent.

ad: if the db is handling intra link (xid) pointing else where in the
database, and the pointed to feature disappears, the server could
update other features that pointed to those disappeared features.

sc: that might confuse the client that gest feats back it doesn't know
about.

ad: but the client might then get out of sync with server.

roy/gh: worry about this

ad: we can defer until issue arises (if it does)

roy: error state for spamming a server, i.e. hitting a server again
and again with same info. indicating that the client isn't getting the
response from server.

ad: client has to detect error and reload from the db.

roy: nice to be stateless.

ad: ask server for url. can put or post to it. if fails try again,
then you can read from it to see if your submit worked.
Example: 
ask server for ftp url.
ftp it over.
check it.
tell server it's there, process it.
you never get duplicates.

roy: this touches on locking.

gh: other than locking, not sure what state we need to maintain.

ad: other issue: authentication.

ap: this could be done by web server.

ad: which http authentication are we going to standardize on? async,
basic (passwds sent in clear). I propose we talk about it next week.

[A] we will talk about authentication next week (5/15).

ad: issues: improperly coded client. network going down (client can
reload from server).

gh: client never got response, so it will know .

ad: inside header, have location, if it got a partial download, it can
go and re-fetch it.

gh: should definitely be optional if it is in there.

ad: spamming question: a poorly impl client, posting the same thing
again and again.

ap: shouldn't locking prevent this?
gh: if same client, then no. not sure how bad that would be if
nothing's changed. how would that mess up the server?

ad: would depend on how server is impl.

gh: possible multi-threading issues - the mapping info may get out of
sync. I would put in as a recommendation in the spec that a client
wait until it gets a response from server before it sends another one.

ad: so fifo order processing.

[A] andrew spec mod: client should wait for server response before sending
another update request

ad: error handling. the error response document will be handy
here. figuring out why something failed.

gh: to me it seems unlikely that there will be clients that do
anything other than pass through the error info. that a client will do
anything interesting with the error response other than just show it
to the user.

Other topics:
--------------

bo: questions about igb. we can talk later. don't know status of
writeback right now. Check with Allen.

gh: status of registry?

ap: have dev servers that aren't publically available. have been busy
with other projects. das/2 working now on his private dev
server. waiting for server box. works - to retrieve a list of das sources.

the machine has little memory.

[A] Steve will send andreas link to versioned sources command for Affy das/2
server

[A] next week prepare to cover more details on writeback impls.


                                   



From rds at sanger.ac.uk  Tue May  9 07:38:54 2006
From: rds at sanger.ac.uk (Roy Storey)
Date: Tue, 9 May 2006 08:38:54 +0100 (BST)
Subject: [DAS2] DAS writeback (fwd)
Message-ID: <Pine.LNX.4.62.0605090817220.6731@deskpro110.dynamic.sanger.ac.uk>

All,

As requested during the conference call here's the mail re full features 
in the response.  What this does do is make the mapping explicit, making 
immediate history a property of each feature.  During the writeback action 
a client should in theory be able to "remove" the features it's saving and 
use the response as if it were a GET/query.  There is less issue with 
having to look up features using one key/uri and change to another, as 
would happen using mapping.xml response.

Regards

Roy

---------- Forwarded message ----------
Date: Thu, 27 Apr 2006 11:37:32 +0100 (BST)
From: Roy Storey <rds at sanger.ac.uk>
To: Andrew Dalke <dalke at dalkescientific.com>
Cc: allenday at ucla.edu
Subject: Re: DAS writeback

Andrew,

Sorry for not getting back on this yesterday, and as I've not really been able 
to keep 100% up to date on the DAS2 state of the art, apologies if I'm 
repeating previous discussion.

On Tue, 25 Apr 2006, Andrew Dalke wrote:

> Have you had any thoughts on this?

A few, nothing concrete.

> Mine is that a single POST using a private namespace is the
> right solution.  It makes for simpler servers and the client
> overhead isn't really that much.  What I like the most is the
> support for multiple versioning models, like "modify in-place"
> vs. "keep old versions"
>

This is about what we'd agreed in Feb. An editing client can hold objects wich 
are new and unsaved keying them how ever it wishes.  On saving these to the 
server, the server can respond with a mapping document between client ids and 
server ids.  The client then has the simple task of moving the objects from 
it's new unsaved objects list to its up-to-date objects list, applying the 
correct key translation as it does so.  As you said this works well for 
modifying current features and adding versions as well.


So what on splitting and merging?

This brings me to Otter, which rather than just replying with a simple mapping 
document, actually sends the complete features which are the result of the 
modifications.  I just wonder what the feeling for this is.

So a feature starts life in an editor...

uri="das-private:$md5_feature_bits"

gets saved to the server

somewhere in response
<feature uri="http://das2server/feature" >
   <previously uri="das-private:$md5_feature_bits" />
   ...
</feature>

get merged/split & saved

somewhere in response
<feature uri="http://das2server/another_feature" >
   <previously uri="http://das2server/featureA" />
   <!-- possibly, in case of merging -->
   <previously uri="http://das2server/featureB" />
   ...
</feature>

Ok so there are likely some issues with this, but I thought I'd raise as it 
could unify rw/ro and remove the need for mapping.rnc, which although I like 
does feel a tiny bit dirty. The first obvious problem with the above is it's 
more extra work for the server to do to output a single feature, so just on 
this feel free to shout me down.  It's possibly open to abuse too.  I also have 
mixed feelings on whether the spec should dictate how to do these more complex 
operations.


So a couple of what ifs:

- The client fails to do the mapping/moving.

This will result in the new feature being kept as "new" although the server 
knows about the feature.  This is likely to result in duplicate features on the 
server, as next time the client saves it'll send a "das-private:identifier". 
If the server is keying features on enough information to distinguish this IS a 
duplicate then the writeback will fail and this isn't an issue, except for the 
client author.  However, if the "new" feature stays as new, gets modified and 
saved again, then the possibly wrong features will just accumulate.

- The server doesn't send a complete response (dies/gets rebooted).

Only some of the features get "moved" in the client (See above). In this case 
though the client should/will know that something isn't right and should 
probably do a re-fetch of all the features, which will result in having to 
remove from the "new list" features which have been saved, but this seems sane 
to me.


Roy




From nomi at fruitfly.org  Thu May 11 23:08:41 2006
From: nomi at fruitfly.org (Nomi Harris)
Date: Thu, 11 May 2006 16:08:41 -0700
Subject: [DAS2] Poster deadline for ISMB
Message-ID: <ryiody45fd2.fsf@spongecake.lbl.gov>

Darn, I had it in my calendar to remind everyone that May 10th was the
deadline for submitting poster abstracts to ISMB (wasn't there some vague
plan to submit a poster about DAS/2?) and here it is, the 11th already.
I bet they'd still accept the abstract if someone submitted it today or
tomorrow along with profuse apologies about the lateness.

BOSC abstracts (http://www.open-bio.org/wiki/BOSC_2006) aren't due until
June 19th.

         Nomi



From allenday at ucla.edu  Fri May 12 00:17:43 2006
From: allenday at ucla.edu (Allen Day)
Date: Thu, 11 May 2006 17:17:43 -0700
Subject: [DAS2] Poster deadline for ISMB
In-Reply-To: <ryiody45fd2.fsf@spongecake.lbl.gov>
References: <ryiody45fd2.fsf@spongecake.lbl.gov>
Message-ID: <5c24dcc30605111717g9f93212y12e8e3c6b7ad70b1@mail.gmail.com>

I submitted a talk to the PLoS track that will touch on what we've been
doing, although it will be Assay-centric (see attached abstract).  If I'm
not accepted for a talk I can submit a more general DAS2 poster, given that
no-one else wants to do it...

Thanks for the heads-up on the BOSC date, I had totally forgotten about
that.

-Allen

On 5/11/06, Nomi Harris <nomi at fruitfly.org> wrote:
>
> Darn, I had it in my calendar to remind everyone that May 10th was the
> deadline for submitting poster abstracts to ISMB (wasn't there some vague
> plan to submit a poster about DAS/2?) and here it is, the 11th already.
> I bet they'd still accept the abstract if someone submitted it today or
> tomorrow along with profuse apologies about the lateness.
>
> BOSC abstracts (http://www.open-bio.org/wiki/BOSC_2006) aren't due until
> June 19th.
>
>          Nomi
>
> _______________________________________________
> DAS2 mailing list
> DAS2 at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/das2
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ismb.pdf
Type: application/pdf
Size: 36085 bytes
Desc: not available
URL: <http://lists.open-bio.org/pipermail/das2/attachments/20060511/4ba783e6/attachment-0001.pdf>

From dalke at dalkescientific.com  Mon May 15 14:23:03 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 15 May 2006 08:23:03 -0600
Subject: [DAS2] status report
Message-ID: <49f160831a69404d9c0d672d28ae4c25@dalkescientific.com>

I've managed to get surprisingly little done in this last
week.  I've been getting ready for my move on Wednesday
morning.  Here's what I did:

  * I modified my GFF3 parser to emit basic DAS2XML, to use
for the writeback server.

My TODO:
  * Update the writeback spec to have microdelta support and
      a modified feature element with a "id used to be ..." attribute.

  * get complex feature examples from Lincoln and include
     into the spec

  * propose that features which are part of a complex annotation
     must have a "root" attribute with the URI of the top-level
     feature

  * work more on the validator

  * implement a simple writeback server


					Andrew
					dalke at dalkescientific.com



From dalke at dalkescientific.com  Mon May 15 15:47:02 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 15 May 2006 09:47:02 -0600
Subject: [DAS2] authentication
In-Reply-To: <65d553032331955821027cff5fc558bd@dalkescientific.com>
References: <65d553032331955821027cff5fc558bd@dalkescientific.com>
Message-ID: <c4caeb7d29fd847f2ade0c5e09871983@dalkescientific.com>

> I looked at HTTP digest authentication
>    http://www.ietf.org/rfc/rfc2617.txt
>    http://en.wikipedia.org/wiki/Digest_access_authentication

Grr.  I came across
   http://bitworking.org/news/Problems_with_HTTP_Authentication_Interop
which is a parody of the Monty Python Cheeseshop sketch.  The
summary is that digest has a lot of options, different servers
and client libraries do different things, so the de facto spec
is different than the written one.

Another solution is through cookies.  That's useful for web
browsers because it supports logout, but for a specialized
client (like we have) it's less useful.

Grr.

Okay, looks like the solutions are:
   1. pick a subset of HTTP Digest authentication
   2. Basic auth over HTTPs

with 1. in the lead.

					Andrew
					dalke at dalkescientific.com



From dalke at dalkescientific.com  Mon May 15 15:35:37 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 15 May 2006 09:35:37 -0600
Subject: [DAS2] authentication
Message-ID: <65d553032331955821027cff5fc558bd@dalkescientific.com>

We said we were going to talk about DAS authentication today.

The goal is to know who did a writeback edit.  A secondary
goal is to allow/deny read access.

I think the solution is to use HTTP Basic authentication.
This puts the username and password in the request headers
and is easy to implement.
   http://en.wikipedia.org/wiki/Basic_authentication_scheme

It's open to various attacks (sniffing the wire, replay attacks.)
The usual response, if this is a problem, is HTTP Digest
authentication.

I looked at HTTP digest authentication
    http://www.ietf.org/rfc/rfc2617.txt
    http://en.wikipedia.org/wiki/Digest_access_authentication

It's more complicated because it exchanges a nonce.  That's
a sort of key from the server combined on the client with
the password and a counter to generate an MD5 checksum.
The checksum is sent to the server in future requests.

It's more complicated.  I've never needed to support it
and I don't have experience with it.  I only know that
web sites do widely support it, along with many libraries.


Another solution is basic authentication over HTTPS.  I've
never had to support HTTPS either.


Yet another is to include the authentication information
in the payload
   <USER>Andrew</USER>
   <PASSWORD>12345</PASSWORD>

At best this will be equal to an existing HTTP authentication
scheme, with no existing library support, so I say no.


					Andrew
					dalke at dalkescientific.com



From Steve_Chervitz at affymetrix.com  Tue May 16 00:28:24 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 15 May 2006 17:28:24 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 15 May 2006
Message-ID: <C08E6AB8.1E695%Steve_Chervitz@affymetrix.com>

Notes from the weekly DAS/2 teleconference, 15 May 2006

$Id: das2-teleconf-2006-05-15.txt,v 1.1 2006/05/16 00:25:27 sac Exp $

Note taker: Steve Chervitz

Attendees: 
  Affy: Steve Chervitz, Gregg Helt
  CSHL: Lincoln Stein
  Dalke Scientific: Andrew Dalke
  UCLA: Allen Day, Brian O'connor

Action items are flagged with '[A]'.

These notes are checked into the biodas.org CVS repository at
das/das2/notes/2006. Instructions on how to access this
repository are at http://biodas.org

DISCLAIMER: 
The note taker aims for completeness and accuracy, but these goals are
not always achievable, given the desire to get the notes out with a
rapid turnaround. So don't consider these notes as complete minutes
from the meeting, but rather abbreviated, summarized versions of what
was discussed. There may be errors of commission and omission.
Participants are welcome to post comments and/or corrections to these
as they see fit. 


Status reports
--------------

gh: have been working on adding curation ability to IGB. Can do basic
creation and editing of features based on other features. Can write
out the curations in bed format. working on writing out das2xml.

gh: grant renewal - plan is to submit new on in October. we can extend
unused funds from original grant for two months after first grant
ends (so thru july). we'll also request bridge funding after our
funding runs out. 
Three things to do:
* no cost extension (of remaining money)
* request for bridge funding.
* re-application in oct w/ Suzi as PI

ad: (see email for my report). gff to das xml. writeback spec,
validator work. Can get it back up and running today. sweden and
iceland travel for teaching courses.

[A] andrew will get his validator app back online.

ls: got married, had lab retreat. had genome biology meeting. now am
finishing off two grants due tomorrow. no das-related work to report.

bo: finished up what we talked about (w/ gregg). sourceforge is
fixed. server is up but beware: CVS root has changed.

gh: it has been frustrating not being able to commit to sf the past week.

sc: we'll have to consider other options if sf continues to be so
unreliable. 

gh: other options?

ls: open-bio.org -- much less downtime than sf.

sc: My status: I've been working on getting affy das server registered
with Andreas. found bugs in affy das server's features response
(xml:base). gregg has fix. also working on updating the data served up
by our das and quickload servers - new version of human and mouse
genome assemblies. 

aday: set up database for writeback. xml parser using dom, handles
features, not type writeback. working on code that coverts xml into
sql insert statements. then we'll have writeback.

had trouble getting apache to expose the post data to me, need to
explore this more. other problem: loss of information in the way the
current das server exposes the way ontology terms are attached on to a
feature, as properties. e.g., mol function, cell component, biol
process, and sequence types - key is not go id, value is not.

[A] andrew, allen will think about best way to encode properties

ad: as new xml elements, extensions to das, so they could say this
means that exactly. you can specify what goes into which element.

aday: a way to insert property types.

ad: you extend das xml to include your own non-das element.

[A] andrew will write up email description of using das xml extension for
ontology terms

gh: if you have non das xml embedded in an xml feature, client gets
that, edits and sends back in writeback, does it have to send it back?

ad: yes, must return it back, otherwise it will be assumed to be a delete.

gh: could you put an example of a property of a feature being deleted
by omission?

ad: how about an accompanying document describing interaction behaviors.

[A] andrew will include example of writeback deletion by omission.

ad: why would client send back human readable text?

aday: so person wouldn't have to parse the ontology.  would prefer
giving back an accession not the human readable term since that
changes. need something that is reliably unique.

ad: new xml element

aday: what do we do for properties then?

ad: properties are a way for humans to create annotations, a
catch-all. Use them because you don't want to define a new xml
element, perhaps.

aday: will then have GO annotated elements, return as properties later
in response to a get request. could be confusing for clients.

ad: property table is accessible to humans, editable, no constraints
on what can be in there.

To summarize my position: let's use the 'x' in xml.

[A] gregg/andrew get response from suzi re: urls for ontology terms

Topic: Authentication
----------------------
ad: did research, see email. Two options: basic auth or http-digest
auth. Have never used it before. Don't know best practices.

gh: how about saying it's outside the scope. server supported on a
server-specific basis. we have enough stuff to figure out.

ad: fine. call it das 2.1

Other topics:
--------------
sc: writeback issue: confirmation from server before it modifies
anything on the server side. "You have requested to delete 17
features, and modify 13 others. Confirm?"

ad: what if thre are hundreds of modifications after serveral hours of
editing? will the user be expected to confirm each one?

sc: user could commit changes in a more piecemeal fashion to avoid
that situation. 

gh: I would prefer to have this done on the client side. the editing
tool can provide this functionality more effectively, flexibly, etc.

sc: ok. seems reasonable. but will server send confirmation response
to client that the writeback request succeeded? and the client will be
unable to issue a request until after the previous request succeeded?

gh: yes.







From Steve_Chervitz at affymetrix.com  Tue May 16 17:10:18 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Tue, 16 May 2006 10:10:18 -0700
Subject: [DAS2] most up-to-date mouse das? (mm7)
In-Reply-To: <83722dde0605160152q200d9c0as5ad2b5cd665a72b0@mail.gmail.com>
Message-ID: <C08F558A.1E6D1%Steve_Chervitz@affymetrix.com>

Hi Ann,

The list address has changed. It's now this: das2 at lists.open-bio.org

As for your question, check out the DAS registry server at the Sanger:

http://das.sanger.ac.uk/registry/

I don't think the registry provides an indication of how current the
annotations on each registered server for a given data source, such as
Entrez Gene. It would be a good piece of data to see, though.

As for the Affymetrix DAS/2 server, the mm7 annotations were last updated on
April 19 2006:

http://netaffxdas.affymetrix.com/das2/sources

The available annotations come from the UCSC server, and derive from the
knownGene, all_mrna, genscan, and refFlat files (called 'refseq' on the das
server). Looks like the knownGene data was last updated by UCSC on 15 Dec
2005:
http://hgdownload.cse.ucsc.edu/goldenPath/mm7/database/

Technical note: The xml:base attribute in the das2xml features document
returned by the Affy DAS/2 server is currently incorrect. It should be

xml:base="http://netaffxdas.affymetrix.com/das2/M_musculus_Aug_2005/features
"

instead of

xml:base="http://127.0.0.1:9021/das2/M_musculus_Aug_2005/features"

This will be fixed in the near future.

Steve

> From: Ann Loraine <aloraine at gmail.com>
> Date: Tue, 16 May 2006 03:52:29 -0500
> To: Steve Chervitz <Steve_Chervitz at affymetrix.com>
> Subject: Fwd: most up-to-date mouse das? (mm7)
> 
> Hi Steve,
> 
> Would you post this to the DAS/2 list for me?
> 
> Sorry to bother you, but for some reason my message didn't appear on the list.
> 
> -Ann
> 
> ---------- Forwarded message ----------
> From: Ann Loraine <aloraine at gmail.com>
> Date: May 15, 2006 3:35 PM
> Subject: most up-to-date mouse das? (mm7)
> To: Andrew Dalke <dalke at dalkescientific.com>, DAS/2 <das2 at portal.open-bio.org>
> 
> 
> Hi!
> 
> I working on a QTL study and need to get all the genes mapping to
> various regions under peaks.
> 
> I have the genomic coordinates for the regions so it should be very
> simple for me to get all accessions (feature ids) underneath those
> regions using DAS.
> 
> My question is: what is the most up-to-date server for mm7?
> 
> Here, of course, is UCSC:
> 
> http://genome.cse.ucsc.edu/cgi-bin/das/mm7/features?segment=chr1:3000000,40000
> 00;type=knownGene
> 
> Ultimately, I'd like to get Entrez Gene ids for the genes under the
> peaks so that I can start sifting through the candidates using GO.
> 
> Any tips would be gratefully accepted!
> 
> All the best,
> 
> Ann
> 
> --
> Ann Loraine
> Assistant Professor
> Section on Statistical Genetics
> University of Alabama at Birmingham
> http://www.ssg.uab.edu
> http://www.transvar.org
> 
> 
> -- 
> Ann Loraine
> Assistant Professor
> Section on Statistical Genetics
> University of Alabama at Birmingham
> http://www.ssg.uab.edu
> http://www.transvar.org



From aloraine at gmail.com  Thu May 18 04:20:47 2006
From: aloraine at gmail.com (Ann Loraine)
Date: Wed, 17 May 2006 23:20:47 -0500
Subject: [DAS2] most up-to-date mouse das? (mm7)
In-Reply-To: <C08F558A.1E6D1%Steve_Chervitz@affymetrix.com>
References: <83722dde0605160152q200d9c0as5ad2b5cd665a72b0@mail.gmail.com>
	<C08F558A.1E6D1%Steve_Chervitz@affymetrix.com>
Message-ID: <83722dde0605172120t5853b30al3f931bd6d73092df@mail.gmail.com>

Hi Steve,

Thank you very much for the info!

Now I have another question...

I'd like to look up the genomic coordinates of a list of mouse genes
using their numeric Entrez Gene ids.

If it's not too much bother, do you think you'd be able to give me
some tips on how to do this using DAS?

btw, the DAS services have been hugely helpful to me in the last week.
We have already found some interesting results with minimal coding.
And the coding was actually fun because there was NO SCREEN-SCRAPING.
Pure bliss.

-Ann

On 5/16/06, Steve Chervitz <Steve_Chervitz at affymetrix.com> wrote:
> Hi Ann,
>
> The list address has changed. It's now this: das2 at lists.open-bio.org
>
> As for your question, check out the DAS registry server at the Sanger:
>
> http://das.sanger.ac.uk/registry/
>
> I don't think the registry provides an indication of how current the
> annotations on each registered server for a given data source, such as
> Entrez Gene. It would be a good piece of data to see, though.
>
> As for the Affymetrix DAS/2 server, the mm7 annotations were last updated on
> April 19 2006:
>
> http://netaffxdas.affymetrix.com/das2/sources
>
> The available annotations come from the UCSC server, and derive from the
> knownGene, all_mrna, genscan, and refFlat files (called 'refseq' on the das
> server). Looks like the knownGene data was last updated by UCSC on 15 Dec
> 2005:
> http://hgdownload.cse.ucsc.edu/goldenPath/mm7/database/
>
> Technical note: The xml:base attribute in the das2xml features document
> returned by the Affy DAS/2 server is currently incorrect. It should be
>
> xml:base="http://netaffxdas.affymetrix.com/das2/M_musculus_Aug_2005/features
> "
>
> instead of
>
> xml:base="http://127.0.0.1:9021/das2/M_musculus_Aug_2005/features"
>
> This will be fixed in the near future.
>
> Steve
>
> > From: Ann Loraine <aloraine at gmail.com>
> > Date: Tue, 16 May 2006 03:52:29 -0500
> > To: Steve Chervitz <Steve_Chervitz at affymetrix.com>
> > Subject: Fwd: most up-to-date mouse das? (mm7)
> >
> > Hi Steve,
> >
> > Would you post this to the DAS/2 list for me?
> >
> > Sorry to bother you, but for some reason my message didn't appear on the list.
> >
> > -Ann
> >
> > ---------- Forwarded message ----------
> > From: Ann Loraine <aloraine at gmail.com>
> > Date: May 15, 2006 3:35 PM
> > Subject: most up-to-date mouse das? (mm7)
> > To: Andrew Dalke <dalke at dalkescientific.com>, DAS/2 <das2 at portal.open-bio.org>
> >
> >
> > Hi!
> >
> > I working on a QTL study and need to get all the genes mapping to
> > various regions under peaks.
> >
> > I have the genomic coordinates for the regions so it should be very
> > simple for me to get all accessions (feature ids) underneath those
> > regions using DAS.
> >
> > My question is: what is the most up-to-date server for mm7?
> >
> > Here, of course, is UCSC:
> >
> > http://genome.cse.ucsc.edu/cgi-bin/das/mm7/features?segment=chr1:3000000,40000
> > 00;type=knownGene
> >
> > Ultimately, I'd like to get Entrez Gene ids for the genes under the
> > peaks so that I can start sifting through the candidates using GO.
> >
> > Any tips would be gratefully accepted!
> >
> > All the best,
> >
> > Ann
> >
> > --
> > Ann Loraine
> > Assistant Professor
> > Section on Statistical Genetics
> > University of Alabama at Birmingham
> > http://www.ssg.uab.edu
> > http://www.transvar.org
> >
> >
> > --
> > Ann Loraine
> > Assistant Professor
> > Section on Statistical Genetics
> > University of Alabama at Birmingham
> > http://www.ssg.uab.edu
> > http://www.transvar.org
>
>


-- 
Ann Loraine
Assistant Professor
Section on Statistical Genetics
University of Alabama at Birmingham
http://www.ssg.uab.edu
http://www.transvar.org



From dalke at dalkescientific.com  Mon May 22 10:14:44 2006
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Mon, 22 May 2006 10:14:44 +0000
Subject: [DAS2] will miss today's meeting
Message-ID: <a36109163a8cd0fd4a7b2ddaf7b8efb5@dalkescientific.com>

Hey guys,

   I'm not going to be able to make today's meeting.  Won't
be able to get the free time for it.

					Andrew
					dalke at dalkescientific.com



From Gregg_Helt at affymetrix.com  Mon May 22 16:16:54 2006
From: Gregg_Helt at affymetrix.com (Helt,Gregg)
Date: Mon, 22 May 2006 09:16:54 -0700
Subject: [DAS2] today's DAS/2 meeting
Message-ID: <C71929195D04BF48BAECD499AF717B480198CADF@msex02.affymetrix.com>

 I think it will be a short meeting today.  Mostly I just want to get a
status report from everyone, especially how writeback implementation is
going.

	Thanks,
	Gregg

> -----Original Message-----
> From: das2-bounces at lists.open-bio.org [mailto:das2-bounces at lists.open-
> bio.org] On Behalf Of Andrew Dalke
> Sent: Monday, May 22, 2006 3:15 AM
> To: DAS/2
> Subject: [DAS2] will miss today's meeting
> 
> Hey guys,
> 
>    I'm not going to be able to make today's meeting.  Won't
> be able to get the free time for it.
> 
> 					Andrew
> 					dalke at dalkescientific.com
> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/das2



From Steve_Chervitz at affymetrix.com  Mon May 22 18:09:18 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 22 May 2006 11:09:18 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 22 May 2006
Message-ID: <C0974C5E.1E7FC%Steve_Chervitz@affymetrix.com>

Notes from the weekly DAS/2 teleconference, 22 May 2006

$Id: das2-teleconf-2006-05-22.txt,v 1.1 2006/05/22 18:07:09 sac Exp $

Note taker: Steve Chervitz

Attendees: 
  Affy: Steve Chervitz, Ed Erwin, Gregg Helt
  CSHL: Lincoln Stein
  U Alabama: Ann Loraine
  UCLA: Allen Day

Action items are flagged with '[A]'.

These notes are checked into the biodas.org CVS repository at
das/das2/notes/2006. Instructions on how to access this
repository are at http://biodas.org

DISCLAIMER: 
The note taker aims for completeness and accuracy, but these goals are
not always achievable, given the desire to get the notes out with a
rapid turnaround. So don't consider these notes as complete minutes
from the meeting, but rather abbreviated, summarized versions of what
was discussed. There may be errors of commission and omission.
Participants are welcome to post comments and/or corrections to these
as they see fit. 


Topic: status reports
---------------------

aday: working on converting xml to sql for the writeback
functionality. Need to bind to apache. should be done today.

gh: would like to test writeback into IGB.

aday: will give url to gregg. if you post an xml doc to that url. will
get back a response mapping provisional names to persisted names in
the db. 

gh: editing too?
aday: no, creation only, at least today. maybe editing later this week.
gh: I could get some writeback for creation in igb this week. with
effort. 

aday: currently writes yeast and human features to the same data
source.   
gh: fine for testing. will focus on human.

sc: probably should just send to url to gregg then to send to list
once we test it a bit.

[A] allen will send info to gregg re: accessing his writeback server

gh: turning on curation in igb. need to connect to das
writeback. previously wasn't connected. given I have a das writer it
should go smoothly. aiming to get something going this week.
Not checked in. Still struggling with sf cvs change.

other news: applying for no cost extension to current grant. have
100-200k left. still determining. need to get app for extension in
today. this is more than we though. should last at least another
couple of month. will let folks know when we know.

ls: pretty good.

gh: will contact p. good about bridge funding until new grant is
funded. working with suzi to re-submit in October with her as PI.

sc: sf cvs trouble?

gh: hangs my jbuilder ide now. I updated repository information in CVS
Root files using perl. JBuilder doesn't like it.

sc: I did the same sort of perl-based editing of CVS Root files and it
worked for me.

al: Doing overrepresentation analysis for QTL studies.
Usually this is for microarray analysis, where you want to see what
things the expressed genes they have in common. you collect GO
annotations, if you see some recurring in the list, you infer that that is
important/involved in the process.
We want to do overrepresentation alaysis for gene assoc studies.
look for go terms/ pathways represented in the list of gene
associations. having trouble getting a null model. plan to hammer on a
das server to get a null distribution. would that be ok? thinking
about using the UCSC das server.

gh: would be a great test to see how they handle lots of
queries. 

ls: also try the biomart interface at ensembl. das feeds into it
internally. there are api's for variety of languages, or can send sql
to it. try the ucsc das server first.

al: is there a contact person for ucsc das server?

ls: jim kent.

aday: write to genome mailing list. they're responsive.

al: top page of das server has message that implies it's not
well-supported. 

gh: the things you're interested in will do fine (txts, rna, gene
predictions). doesn't do well on extra stuff in their db (fastcons
conservation scores across regions of conservation, expression score,
etc). For basic annotations it works fine.

al: most interested in the "knownGene" data type. das server gives
back accessions. annotations are encoded. need to look up entrez gene id
from accession. using NCBI mapping files to get mRNA accessions.
then get ontology annotations, also distributed via NCBI.

gh: in das/2 this is handled better with xids (this das id = that
entrez id).  
al: will ucsc update their server to das/2?
gh: more likely after grant renewal.

al: affy server?
sc: we have knownGene annotations.

al: doing this for a statistical genetics class this fall. expect lots
of hits on the server then.

[A] Ann will test hammering on the affy das/2 server before turning her
students loose on it this Fall.

sc: no writeback related work, but attended the JavaOne meeting last
week. Some things of interest to our work:
1) Presentation on the ATOM protocol and REST in general. Mentioned
   something called WADL that is a REST-based web service to automate
   client-side interactions. Seemed more general that ATOM since one
   can use it to implement ATOM.
2) Eclipse RCP (rich client platform). Is a gutted version of the
   Eclipse Java IDE with all the java tools removed. You're left with
   lots of core functionality that can be of use for any java app. Was
   used by NASA when they needed to integrate lots of heterogeneous
   apps related to the Mars space mission. Seemed like we have a
   similar situation with IGB + Apollo integration. The Eclipse RCP
   gives you a framework for assembling tightly integrated components
   that are loosely coupled.

gh: do all the components need to be eclipse plugins?
sc: yes (believe so). not sure how much work that entails.

gh: I looked at using eclipse and netbeans back when starting to
develop IGB, but that was a long time ago. might be worth looking
again. 

sc: there's a new book on building Eclipse RCP apps. On my desk.

[A] gregg will look into the eclipse RCP as a possible app framework.

3) Extreme GUI makeover talk was interesting to see what you can do
   with Java 2D to dress up a "plain-jane" application. This year,
   they created a suped-up version of the Thunderbird email app. There
   are other enhancements coming with the next java release that
   improve java on the desktop (version 6 called  Mustang, due in
   October). 

4) java.util.concurrent package in java 5.0 has quite a lot of
   functionality for implementing things in your app that require
   multithreading support. There is a back port to JDK 1.4.

5) Scripting in java was very big. Using the groovy scripting
   language, with simplified syntax but access to all of java.
   AJAX was also very hot.

gh: I saw something using gbrowse and AJAX.
ls: this is an ian holmes project. It has google maps like
interface. change track order by tracking, smooth scroll and and
zooming. the idea is to supercede igb so people can do it on the web.

al: you need to do 1d not 2d zooming.
ls: it does that.
gh: the hard part is dealing with lots of data.

al: I always get questions from people who look at google maps and
say, "how come you genome people can't do something like this?"

ls: it's running now. go to http://genome.biowiki.org/ or look for
"gbowse ajax client" on biowiki.org.


Topic: Next meeting
--------------------
gh: next monday is US holiday (memorial day, 5/29). next meeting will
therefore be in two weeks. 



From Steve_Chervitz at affymetrix.com  Mon May 22 19:48:13 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 22 May 2006 12:48:13 -0700
Subject: [DAS2] Notes from the weekly DAS/2 teleconference, 22 May 2006
In-Reply-To: <C0974C5E.1E7FC%Steve_Chervitz@affymetrix.com>
Message-ID: <C097638D.1E807%Steve_Chervitz@affymetrix.com>

Forgot to mention one more item of interest from the JavaOne meeting:

Steve wrote:
> sc: no writeback related work, but attended the JavaOne meeting last
> week. Some things of interest to our work:
>...

6) Probably the most relevant to this group was a talk called,
   "How to write an API that will stand the test of time". While it
   was definitely java-centric, there are some nuggets that apply to
   any software or spec writing team:
    * Be use-case driven
        - focus on what people need now, not what might be useful later.
    * Expose only necessary functionality
        - minimize your API's "surface area". This means less for
          users to learn, and less for devs to implement and maintain.
    * Be predictable and consistent
        - an interface that is predictable serves better than one
          which is locally optimal but inconsistent within the
          framework.
    * Design to test
        - write tests to fully cover API functionality.
    * Always think about evolution
        - the first version is never perfect.
        - allow for the co-existence of multiple versions.
    * An anti-example: The JavaMail API, which is optimized for
      implementing new wire protocols, not for the bulk of potential
      users who just want to read/send mail.
    * Links: 
       - http://openide.netbeans.org/tutorial/api-design.html
       - http://www.artima.com/apidesign/index.html


Steve



From ed_erwin at affymetrix.com  Mon May 22 22:28:20 2006
From: ed_erwin at affymetrix.com (Ed Erwin)
Date: Mon, 22 May 2006 15:28:20 -0700
Subject: [DAS2] authentication
In-Reply-To: <c4caeb7d29fd847f2ade0c5e09871983@dalkescientific.com>
References: <65d553032331955821027cff5fc558bd@dalkescientific.com>
	<c4caeb7d29fd847f2ade0c5e09871983@dalkescientific.com>
Message-ID: <44723B04.4060005@affymetrix.com>

I vote for basic auth with optional HTTPS.  That seems easy to implement 
and HTTPS can be added or not depending on how paranoid the particular 
site is.

Just my few cents.


Andrew Dalke wrote:
>>I looked at HTTP digest authentication
>>   http://www.ietf.org/rfc/rfc2617.txt
>>   http://en.wikipedia.org/wiki/Digest_access_authentication
> 
> 
> Grr.  I came across
>    http://bitworking.org/news/Problems_with_HTTP_Authentication_Interop
> which is a parody of the Monty Python Cheeseshop sketch.  The
> summary is that digest has a lot of options, different servers
> and client libraries do different things, so the de facto spec
> is different than the written one.
> 
> Another solution is through cookies.  That's useful for web
> browsers because it supports logout, but for a specialized
> client (like we have) it's less useful.
> 
> Grr.
> 
> Okay, looks like the solutions are:
>    1. pick a subset of HTTP Digest authentication
>    2. Basic auth over HTTPs
> 
> with 1. in the lead.
> 
> 					Andrew
> 					dalke at dalkescientific.com
> 
> _______________________________________________
> DAS2 mailing list
> DAS2 at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/das2


From Steve_Chervitz at affymetrix.com  Mon May 22 23:52:27 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Mon, 22 May 2006 16:52:27 -0700
Subject: [DAS2] BOSC 2006 2nd Call for Papers
In-Reply-To: <4471D9C0.1050003@duke.edu>
Message-ID: <C0979CCB.1E846%Steve_Chervitz@affymetrix.com>


On behalf of Darin London <darin.london at duke.edu>:

> BOSC 2006 2nd Call for Papers
> 
> This is the second and last official call for speakers to submit their
> abstracts to speak at  BOSC 2006
> in Fortaleza, Brasil.  In order to be considered as a potential speaker,
> an abstract must be recieved by
> Monday, June 5th, 2006.  We look forward to a great conference this
> year. Please consult
> The Official BOSC 2006 Website at:
> 
> http://www.open-bio.org/wiki/BOSC_2006
> 
> for more details and information.
> 
> 
> In addition, a BOSC weblog has been setup to make it easier to
> desiminate all BOSC
> related announcements:
> 
> http://wiki.open-bio.org/boscblog/
> 
> And if you have an ICAL compatible Calendar, there is an EventDB
> calendar set up with all
> BOSC related deadlines.
> 
> http://eventful.com/groups/G0-001-000014747-0
> 
> More information about ISMB can be found at the Official ISMB 2006 Website:
> 
> http://ismb2006.cbi.cnptia.embrapa.br/
> 
> Thank You, and we look forward to seeing you all,
> 
> The BOSC Organizing Committee.

 



From Steve_Chervitz at affymetrix.com  Tue May 23 19:33:29 2006
From: Steve_Chervitz at affymetrix.com (Steve Chervitz)
Date: Tue, 23 May 2006 12:33:29 -0700
Subject: [DAS2] WADL
In-Reply-To: <C0974C5E.1E7FC%Steve_Chervitz@affymetrix.com>
Message-ID: <C098B199.1E8B8%Steve_Chervitz@affymetrix.com>


As I mentioned in the weekly meeting notes:

> sc: no writeback related work, but attended the JavaOne meeting last
> week. Some things of interest to our work:
>
> 1) Presentation on the ATOM protocol and REST in general. Mentioned
>    something called WADL that is a REST-based web service to automate
>    client-side interactions. Seemed more general that ATOM since one
>    can use it to implement ATOM.

WADL stands for Web Application Description Language and is a fairly recent
invention, so it's not in widespread use.

Documentation can be found at
http://research.sun.com/techrep/2006/abstract-153.html

Here's the abstract:

> An increasing number of Web-based enterprises (Google, Yahoo, Amazon, Flickr -
> to name but a few) are developing HTTP-based applications that provide access
> to their internal data using XML. Typically these applications are described
> using a combination of textual protocol descriptions combined with XML
> schema-based data format descriptions; WADL is designed to provide a machine
> processable protocol description format for use with such HTTP-based Web
> applications, especially those using XML.

I haven't looked at it enough to tell whether it would be useful for DAS
applications. The DAS protocol does some things that are targetted by WADL,
like defining resource relationships. But other parts of WADL might be of
use for DAS, such as describing relationships between DAS servers, or
between DAS servers and other non-DAS resources.

It may be possible to describe DAS using WADL, but I haven't tried.

Steve