[Bio-packaging] Ruby gems/bundler support

[David Thompson]

Pjotr Prins <pjotr.public66 at thebird.nl> writes:

> We are starting on supporting Ruby gems in GNU Guix. So far I have written up
> (a recent copy at
> https://github.com/pjotrp/guix-notes/blob/master/RUBYGEMS-Nokogiri.org).
>
> David, currently in guix/build/ruby-build-system.scm you create a gem from the
> tgz file and then run gem install. Would it be logical to download the gem file
> directly?

No, because the gems are equivalent to a binary.  It's not the complete
corresponding source code for the gem.

> Below you can see that I propose to give the Ruby gem command a full build
> environment so it can do native builds. We may even hack gem itself (a little)
> to provide the correct PATHs by default.

Comments below.

> * Introduction
>
> Ruby developers have multiple needs when running a system when it
> comes to Ruby versions and modules (called gems). GNU Guix has great
> support for versioning and control of the dependency graph. Here we
> discuss gem support in GNU Guix (this is a work in progress).
>
> Basically we are aware of the following use cases after installing
> a version of Ruby (say 2.1.6):
>
> 1. A user/sysadmin wants to install a gem in the GNU Guix store 
> 2. A user wants to install a gem using rubygems outside GNU Guix and
>    local to $HOME
> 3. A developer wants to use bundler which installs gems in the source
>    tree for development and testing
>
> In all cases, when Ruby code needs to include a file, resolving the
> location of the gem should start in 3, next 2, then 1. As an example
> of supporting a complex gem we use the example of Nokogiri which has
> over 1,000 installation related questions on Stack Overflow.
>
> * Current state
>
> ** GNU Guix installed gem
>
> GNU Guix now build gems from a tar-ball using a Ruby build
> environment. Installing ruby-hoe
>
> : guix package -i ruby-hoe
>
> creates a symlink to the executable sow in
>
> : /home/wrk/.guix-profile/bin/sow -> /gnu/store/ccx2ri5l8y6qbrzkw4p0gq8l647kxn0g-ruby-hoe-3.13.1/bin/sow
>
> and the libraries are available through the symlink in
>
> : /home/wrk/.guix-profile/lib/ruby/gems/2.2.0/gems/hoe-3.13.1/ -> /gnu/store/ccx2ri5l8y6qbrzkw4p0gq8l647kxn0g-ruby-hoe-3.13.1/lib/ruby/gems/2.2.0/gems/hoe-3.13.1
>
> Running sow means you'll need to add the GNU Guix gem locations to the
> GEM_PATH, e.g.,
>
> : env GEM_PATH=~/.guix-profile/lib/ruby/gems/2.2.0 sow
>
> *** TODO Make the GNU Guix gem path automatically available
>
> At this point the GEM_PATH is not set properly.

'guix package --search-paths' will output the correct gem path for your
user profile, and 'guix environment' will automatically set the correct
qenvironment variables.  I highly recommend using 'guix environment' for
development tasks.  I consider this TODO to be complete.

> *** TODO Provide a build system that uses simple gem descriptions
>
> We could extract Gem information from Rubygems - something Nix was
> capable of doing - and create expressions from those (even on the
> fly).

'guix import rubygems' is something I started tinkering with.  The
current blocker is that, unlike PyPi, it's *really* hard to get the
complete corresponding source code.  However, since most gem source code
is hosted on github, I guess we can detect URIs in the rubygems home
page and source code metadata and then use the github API to fetch the
source tarball.  That would probably be good enough for most things.

> ** Gem in $HOME
>
> The gem tool comes with the GNU Guix Ruby installation. This means you can run
> gem after tweaking the local PATHs. An example can be found [[https://github.com/pjotrp/guix-notes/blob/master/scripts/ruby-guix-env][here]]. Run it as
>
> : . ruby-guix-env
>
> Note: it also sets the path for GNU Guix global gems.

Your script looks most robust than my simple method.  I just create a
new directory to house the gems for a particular project, set $GEM_HOME
as needed, and then prepend $GEM_HOME/bin to $PATH.  Has worked for me
thus far.

> *** TODO Supporting native gems
>
> Native gems include some C or C++ code.
>
> Currently the GNU Guix gem tool does not build native gems. The reason
> is that the build environment is not complete for the gem tool that
> comes with the installed Ruby. It is possible to force native builds
> by tweaking C_INCLUDE_PATH, LD_LIBRARY_PATH and friends, but that is
> not encouraged.
>
> To move forward I think we should create a 'rubygems' package that
> includes the most important build tools and a 'rubygems-extra' for
> some of the rarer tools.

I don't know if I like this idea.  The libraries required vary from gem
to gem.  I think it would be better to simply document somewhere how to
build native extensions.  In any case, here's the base environment
needed for bundling:

   guix environment --ad-hoc ruby bundler make gcc glibc linux-libre-headers binutils

> ** Bundler
>
> Bundler works in GNU Guix, both installed though Guix and through
> gems, but at this point there are no native builds for the same reason.
>
> *** TODO Support native gems for bundler
>
> * Nokogiri

I was under the impression that native extensions would build when
running 'gem install' on the local gem.  I'm going to find a simple gem
with native extensions to mess around with.

-- 
David Thompson
GPG Key: 0FF1D807