[MOBY-dev] Java developers, let's talk together

[Martin Senger]

[ I am sorry this is a bit longer email. I am abusing the reply to Nina to
address also answers and hints from other people's emails from yesterday.  
Thanks very much for all of them; especially those interesting suggestions
about how to address versioning conflicts of the third party libraries.
Thanks again. ]

> From my point of view the RDF Agent logically belongs to the registry.
> Will be the org.biomoby.registry.rdfagent as a RDF Agent’s base package
> name ok?
>
   I agree, perfect. 

> Agree. I’ll develop an ANT build file for the RDF
> Agent and will commit it to the CVS with source code.
>
   Sounds great. (I put some comments about Ant and third-libraries 
below).
 
> Location for the third-party libraries is real issue.
> I think we should discuss that altogether with
> subproject directory structure.
>
   I would say that if we use Ant then the directory structure is not that 
important (because Ant knows where to find things). The directory 
structure becomes more relevant if we decide to move sources together. 
Which is not so bad idea - if it is easy (e.g. I feel that moving S-MOBY 
current structure to the same directory with jMoby-MOBY-S would not be 
practical).
   My suggestion is this: I am going to write down (first here, later it
should appear also in the docs somewhere; most of it is already there,
btw) some conventions which I am using now (and which - believe me - I was
thinking about quite a lot before I started to use them). Every developer
will have then three options (any has few pros and cons):

   1) To keep her/his subproject separate, or
   2) To incorporate it into current structure in moby-live/Java, or
   3) To put in into moby-live/Java - bus as a totally separate 
subdirectory.

What are the pros and cons?

   Ad 1) is preferable (unless there are reasons against it) because it
will force us to solve problems with versioning of the third-party
libraries as soon as they appear (if they appear; perhaps we will be lucky
:-)). Simply it means to share third-party libraries (sitting in the 'lib'
subdirectory).
   This is also quite good for the end-users using our libraries/code - 
because it is clear to them what to put on their classpath (usually we 
already have 'run' scripts doing it).
   The cons are: You need to conform with the existing structure and
existing policies which may not be always the best for your subproject. If
you want to change such policy it needs more discussion and time because
there are more people involved. Simply speaking: any change in structure
is costly.
   Also this does not solve the problem how the end-users can combine more 
subproject together. I will briefly address this problem at the end.

   Ad 3) is needed if you have special build requirements - so you can 
provide your own build.xml. I have never used nested build.xml so I have 
not yet experiences with this option. But it should work fine.

   If you choose ad 2) or ad 3) the suggested principles are simple:
   a) use Ant
   b) do not put Java generated documentation in the CVS (but make Ant to 
produce it)
   c) send me a link and few words about your subproject that I can
include in the jMoby main page

   If you choose ad 1) the principles above are the same, but there are 
also others (I am not only listing them but I am trying to put some 
rationale that lead me to have them. I am also prepared to discuss how to 
change them if you wish.):
   [ Everything below can be seen in moby-live/Java/build.xml, and in 
scripts moby-live/Java/build.* ]

   d) The sources are in 'src', but most of them are actually in
'src/main'. This is how to make possible to include also sources that do
not belong to any package (such as some testing classes). Simple speaking,
if your sources are in package (most of them, if not all, are) they should
go to 'src/main').

   e) Run scripts (the scripts starting various Java programs) are now in 
the main directory. This is going to be changed because we may have more 
scripts and it should polute the main level too much. There are two 
options:
   e1) To use 'run' directory.
   e2) To use 'build/run' directory.
   Using 'run' directory is easier for users. When they look into the top
level, they see 'run' so they look there and find appropriate script. But 
the script will not work until they actually build everything, and the 
script must always be started from the top level directory (because it 
must have relative path to 'build' and 'lib' directories with classes).
   Using 'build/run' is not so obvious but has otherwise quite a lot
advantages: the scripts are generated by Ant, so they have the full paths
to classes, so they can be called from anywhere; they cannot be started
before building them - because they do not exist before that :-), and they
can be customized (using standard 'build.properties' file) for local needs
(if there are such needs, hopefully as little as possible - but for
example for scripts accessing locations out of the CVS space, like Tomcat, 
it is needed).
   Saying this I vote for 'build/run'...

   f) The probably most contentious topic is how to get third-party
libraries. The basic question is: should they be part of the CVS or be
downloaded when you first build it? After many considerations, and after
discussing with people around here, I have taken the second option
(downloading them). (The moby-live/Java/README explains how it works.) If
we continue to use this option, I will change the place where I am getting
these libraries from to somewhere where other developers can put their
libraries as well (it must be a place accessible by HTTP - at the moment I
have it in my space at EBI).
   This email is too long now - so we may discuss this topic separately if 
we wish.

   That's all regarding the policies.

   Nina also suggested:
>   + doc
>   +config
>
   * moby-live/Java uses 'docs' instead of 'doc', sorry for that;
   * I usually have 'config' under 'src' - but there are no special 
reasons for that.

> This structure provides good subproject encapsulation
> and resolves libraries’ version conflict problem. Main
> disadvantage –duplicate some libraries. As a result
> less efficient use of  the server disk space and some
> extra traffic.
>
   As I tried to explain above, the problem is not with having duplicates,
but with the versioning conflicts that can happen when a user starts using
two - until now separated - subprojects. He has to pick up a library from
one of these subproject - which may lead to failure of the other one. If
we share the libraries from the beginning we know about it during the
developing and testing phase already.

> To put all libraries in the same place could become a nightmare for the
> project support/administration
>
   It may be well true - but better have this nightmare for administrators 
and developers than for our users. Just my 2c.

   So I finish now... Again sorry for the extend of this email.
   Cheers,
   Martin

-- 
Martin Senger

EMBL Outstation - Hinxton                Senger at EBI.ac.uk     
European Bioinformatics Institute        Phone: (+44) 1223 494636      
Wellcome Trust Genome Campus             (Switchboard:     494444)
Hinxton                                  Fax  : (+44) 1223 494468
Cambridge CB10 1SD
United Kingdom                           http://industry.ebi.ac.uk/~senger