[MOBY-dev] [moby] Re: RFC #1914 - change & call for vote

[Martin Senger]

> ...almost automatically imply what should be returned... but not
> explicitly enough for my tiny brain :-)  Explicitly, what do you want
> getData to return?
>
   Your brain is fine - I actually did not say what to return. :-)
   The idea is that if you put a version to something, you know to what
you are putting it, so you know what could be returned by getData().
   But let me step back a bit: I started to talk about getData() and
finished by talking about versioning. I admit that these two topics are
closely related but they do not need to be solved both in the same
time. What I really wish to have (sooner the better) is versioning.
 
> Versioning of LSID's creates added complexity at the database end of
> things (since we need to keep track of all objects that have been
> deleted... forever!)
>
   Yes and no. Definitely, some complexity must be added, but not perhaps
that much. Because we do not need to return old objects  - we can just
refuse to resolve them. LSID spec does not allow to reuse the same LSID
for something else, but does not require to resolve data with old version
forever.
   So, for example, my naive implementation would be soemthing like this
(I am probably still missing some situations):
   * add columns "version" and "active" to moby entities,
   * when a moby entity is being registered, find if you have it already
in database with the flag "active" set to false:
      - if yes, update its version (and replace all other fields with the
new registration data),
      - if not, create a new record, set "version" to 1 and "active" to
true,
   * when an entity is being deregistered, set its "active" to false",
   * when to return entities, return only "active" once.
    
(I guess that you can optimize by keping old (unregistered) entities in a
separate table just with their last-used version number - this will
substitute the "active" flag.)

(Or, perhaps much better and simpler: just assign time of registration as
a version field - so everytime you register an entity it gets new
version. This seems the bestsolution... where is the catch? Is there any?)
   
> What isn't clear to me, however, is how this solves the problem that you
> are saying it will solve.  If you have cached the RDF for all objects
> locally, how does a versioned LSID save you any time?
>
   The RDF is not fine grained - one RDF document contains, for example,
all data types. But I want to get only one data type - in case I know that
this data type has been changed. Perhapd the best explanation is to tell
you how the current caching works in jMoby:
   For each type of entities (data types, service types, services and
namespace) I keep one file (called "list") with all their names (and their
LSIDs). When a user want to update a cache, these files are always created
newly from a registry (there are four API methods to get them; eventhough
it would be faster to have one method for getting all of them in one go,
but that's another story).
   Then, the caching software reads the list and compare what it has in
local cache. The entities that are in local cache but not in the list are
removed from the local cache. The entities that are in the list but not in
the local cache are aksed for (by a normal API call for one entity). But,
this mechanism cannot recognize if an existing entity was changed. If the
LSIDs have versions, the caching mechanism can also compare the LSIDs of
the local entities and those in the list - and when the version in the
list is newer, it can fetch the updated version.

   Cheers,
   Martin   

-- 
Martin Senger
   email: martin.senger at gmail.com
   skype: martinsenger
consulting for:
   International Rice Research Institute
   Biometrics and Bioinformatics Unit
   DAPO BOX 7777, Metro Manila
   Philippines, phone: +63-2-580-5600 (ext.2324)