[Bio-packaging] Ruby gems/bundler support

[David Thompson]

Pjotr Prins <pjotr.public66 at thebird.nl> writes:

> On Sun, Jun 14, 2015 at 11:09:09AM -0400, David Thompson wrote:
>> > David, currently in guix/build/ruby-build-system.scm you create a gem from the
>> > tgz file and then run gem install. Would it be logical to download the gem file
>> > directly?
>> 
>> No, because the gems are equivalent to a binary.  It's not the complete
>> corresponding source code for the gem.
>
> Aren't we making it to hard on ourselves? Almost all gems contain
> source code and docs - though the creator may (indeed) decide to limit
> the contents. Gems are easy to find, very standardized and tested by
> the authors, and we have a direct installer at hand. We only need to
> teach it where to put stuff.  Seems silly not to use it.

We need to find out for sure if a Ruby gem qualifies as complete
corresponding source code.  If they do not, then we cannot use them.  I
seem to recall a gem that didn't include it's test suite and other
things, thus I went for the tarball approach.

Test suites make things more complicated, anyhow.  In order to test a
gem file, we much extract the gem somewhere and run 'rake test' or
whatever.  Verifying that software passes its test suite is very
important.  We cannot skip this crucial step.

> I would like to have expressions that basically duplicate a gem
> install with the download version and hash value.
>
>> > *** TODO Make the GNU Guix gem path automatically available
>> >
>> > At this point the GEM_PATH is not set properly.
>> 
>> 'guix package --search-paths' will output the correct gem path for your
>> user profile, and 'guix environment' will automatically set the correct
>> qenvironment variables.  I highly recommend using 'guix environment' for
>> development tasks.  I consider this TODO to be complete.
>
> Hmmm. I am not completely happy with this. So after installing
> software users still need to set the environment. I am not worried
> about developers - I am worried about end-users.

That is a bigger UX issue that applies to Guix on the whole.  Right now,
this is the way things are done.  Besides, an end-user shouldn't be
manually installing gems from rubygems.org.  If they know how to do that
then they can figure out how set a few environment variables.

>> > *** TODO Provide a build system that uses simple gem descriptions
>> >
>> > We could extract Gem information from Rubygems - something Nix was
>> > capable of doing - and create expressions from those (even on the
>> > fly).
>> 
>> 'guix import rubygems' is something I started tinkering with.  The
>> current blocker is that, unlike PyPi, it's *really* hard to get the
>> complete corresponding source code.  However, since most gem source code
>> is hosted on github, I guess we can detect URIs in the rubygems home
>> page and source code metadata and then use the github API to fetch the
>> source tarball.  That would probably be good enough for most things.
>
> Not sure. I went through a few gems to find a useful one to experiment
> with and it is a mess. Only the actualy versioned gems can reliably be
> found on rubygems.

OK.

>> > To move forward I think we should create a 'rubygems' package that
>> > includes the most important build tools and a 'rubygems-extra' for
>> > some of the rarer tools.
>> 
>> I don't know if I like this idea.  The libraries required vary from gem
>> to gem.  I think it would be better to simply document somewhere how to
>> build native extensions.  
>
> I can't agree. Really, installing software should be as easy as
> running 'gem install target' for end-users. Having to deal with an
> environment and figuring out how to build native is not good enough.

GuixSD users shouldn't be installing things with gem.  If they want to
do that, they'll have to handle getting the right dependencies for
native extensions, just like any other distro.

> Ultimately we need 'one-click' installation. And I am sure we can do
> it.

And that will be by packaging gems for Guix.

> Anyway, even the intermediate phase of doing it by hand would be a
> good first step. Automation can come later.
>
>> In any case, here's the base environment
>> needed for bundling:
>  
>>    guix environment --ad-hoc ruby bundler make gcc glibc linux-libre-headers binutils
>
> You know my comment already. 
>
> Ok, let's focus on making compiled extensions work first.
>
>> > ** Bundler
>> >
>> > Bundler works in GNU Guix, both installed though Guix and through
>> > gems, but at this point there are no native builds for the same reason.
>> >
>> > *** TODO Support native gems for bundler
>> >
>> > * Nokogiri
>> 
>> I was under the impression that native extensions would build when
>> running 'gem install' on the local gem.  I'm going to find a simple gem
>> with native extensions to mess around with.
>
> Nokogiri fails because it needs some system include files and
> libraries. A cleaner native extension will probably compile with the
> standard build tools.

Just include libxml2, libxslt, etc. in the build environment of the package.

-- 
David Thompson
GPG Key: 0FF1D807