[DAS2] Dasypus status

Andrew Dalke dalke at dalkescientific.com
Thu May 5 16:42:35 UTC 2005

(See, I'm trying out one of the possible names)

I've implemented the core architecture over the last
few days.  It took some head-banging, and sadly not
the fun kind.

The result is somewhat sweet.  The difficulty was
because I wanted to test the various sorts of things
that could go wrong.  This include:

   - HTTP server doesn't respond, infinite loop, etc.
   - the wrong values in the HTTP headers (eg, wrong version)
   - invalid XML returned [1]
   - wrong elements in the XML [2]
   - wrong fields in the XML [3]

[1] We should add a new DAS header which is the server
version.  Eg, some of the DAS/1 servers returned invalid
XML, like <BLAH href="http://example.com/blah?a=1&b=2">
when the "&" attribute needs to be escaped.  I want
to allow a cleanup, triggered on the server type, that
fixes these sorts of errors.

[2] While the schema can catch these, some are errors
while others (like extending the XML) should be warnings.

[3] For example, the prototype server uses the wrong
namespace and wrong xml:base.  I've told Allen about
it but in the meanwhile I've put a host-based cleanup
filter in place.

In addition several layers of the architecture are
easily exchangeable with others.  For example:

   - the "URL opener" is replaceable.  This lets me
put in test code to check that the URL of a request
is as expected, lets me wrote a standalone proxy
system to use local files for http requests, and
potentially lets someone implement, say, an interface
that reads directly from a MySQL connection instead
of going through the web.  (Hmmm, interesting ... I
had just thought of that possibility.)

   - the "builder" layer is replaceable.  This is the
one that in most apps takes the input socket connection
and makes a usable data structure from it.  I have
one builder for Python data structures.  I'll soon have
one for the schema checker and another that allows
both builders to work.

   - the "build a Python data structure" knows that a
few data structures are special and could be converted
into something other than a string or integer.  One
example is the ISO date, which by default gets turned
into a Python datetime.  Another is a link from one
part of DAS to another.  By default it's a string but
I experimented with a version that fetches the given link
on request, which makes the data structure act like a
direct mirror of the DAS object network.

In doing this I noticed a couple of problems with the
DAS XML we have.  I've pointed out a couple.  I'm
in the process of cleaning up a few others.

I am able to read all of the XML in the DAS CVS
repository and Allen's SOURCES url.  The server has
been down the last few days and I haven't been able
to get any further.

The next steps for me are:
   - we were going to change the <PROP> section of the
spec, right?  What were the changes?

   - check Dasypus into CVS

   - implements builders for the RNG validation and to
let both the RNG and Python data structure builders work.

   - finish the error/warning handling code

   - clean up the example XML files

   - start developing test sets

   - work with Allen's prototype server

   - implement the handler for the compact formats
(Again, what's the reason for the compact formats?)

All that is to check that a request/response works.
The next stage is to check for system integrity.
I'm not close enough to that yet to describe the
steps involved.

					dalke at dalkescientific.com

More information about the DAS2 mailing list