The way in which Nixpks builds Ruby gems means that certain operations
by bundler *will not work*, namely `bundle install --redownload`.
According to the source the _cache/_ directory should have been kept,
however it seems through revisions to the file it has been purged.
Here was the comment from the original commit that introduced
buildRubyGem:
```
# Note:
# We really do need to keep the $out/${ruby.gemPath}/cache.
# This is very important in order for many parts of RubyGems/Bundler to not blow up.
# See https://github.com/bundler/bundler/issues/3327
```
Why is the _cache_ directory needed?
Bundler and RubyGems uses the cache as a source of truth.
When bundler executes `bundler install --redownload`, any gems it
discovers in the _GEM_PATH_ it assums must have their _.gem_ file
present in the cache (unaware it was installed from Nix).
Rather than downloading the gem from RubyGems the bundler code forcibly
re-installs the gem from the cache directory instead and **fails** if it
does not exist.
I've opened https://github.com/rubygems/rubygems/issues/4088 to see if
this failure should be soft and not so explicit; or fallback to fetching
the gem from scratch.
Without this change the following is the error:
```bash
> [nix-shell:~/code/nix/playground/jruby-bundler-rake]$ bundle install --force
[DEPRECATED] The `--force` option has been renamed to `--redownload`
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/nix/store/fis6nzrpw9pmcivr84qh5byfgm07qn10-jruby-9.2.13.0/lib/ruby/stdlib/jopenssl.jar) to field java.security.MessageDigest.provider
WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Fetching gem metadata from https://rubygems.org/.
Using bundler 2.1.4
Installing hello-world 1.2.0
Bundler::GemNotFound: Could not find hello-world-1.2.0.gem for installation
An error occurred while installing hello-world (1.2.0), and Bundler
cannot continue.
Make sure that `gem install hello-world -v '1.2.0' --source
'https://rubygems.org/'` succeeds before bundling.
```
Wth the fix the following no woccurs:
```bash
[nix-shell:~/code/nix/playground/jruby-bundler-rake]$ bundle install --redownload
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/nix/store/69wjlj4yirp48rv1q03zxgd4xvf0150d-jruby-9.2.13.0/lib/ruby/stdlib/jopenssl.jar) to field java.security.MessageDigest.provider
WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Fetching gem metadata from https://rubygems.org/.
Using bundler 2.1.4
Installing hello-world 1.2.0
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
```
```
[nix-shell:~/code/nix/playground/jruby-bundler-rake]$ ls -l /nix/store/cwl9n5073hqgpfhnw4wic13nrrgg9dn8-gem-env/lib/jruby/gems/2.5.0/cache/
total 8
lrwxrwxrwx 1 fmzakari primarygroup 102 Dec 31 1969 bundler-2.1.4.gem -> /nix/store/ifc8a0gsfkrhkv953rd4rz8bcspahi8y-bundler-2.1.4/lib/jruby/gems/2.5.0/cache/bundler-2.1.4.gem
lrwxrwxrwx 1 fmzakari primarygroup 110 Dec 31 1969 hello-world-1.2.0.gem -> /nix/store/xi9ln6n1mz2is5ppykjxqhhkpjq9zm6i-hello-world-1.2.0/lib/jruby/gems/2.5.0/cache/hello-world-1.2.0.gem
```
I have a minimal project that demonstrates this issue at https://github.com/fzakaria/jruby-bundler-nix-failure
In building a gem whose native extension is a Rakefile, the previous
version of this code will call essentially `rake ""`, when it means to
call `rake`.
This change converts `""` into `[]` rather than `[""]`.
Go beyond the obvious setup hooks now, with a bit of sed, with a skipped case:
- cc-wrapper's `dontlink`, because it already is handled.
Also, in nix files escaping was manually added.
EMP
In case of the gem type 'git', nix-bundle-install.rb was called with
wrong path to the git checkout.
${src} does contain the sources, but not the newly generated .git dir,
which is created in the buildPhase of pkgs/development/ruby-modules/gem/default.nix
In some rare cases, this .git dir is needed at installPhase.
Don't default `doCheck` to false,
and use the default set of phases
so the phases list does not need to be overriden
in order to add checkPhase or installCheckPhase.
This just defers to `fetchurl` for fetching.
Additionally, update the `nix-bundle-install.rb` script to handle gems
installed from a path, i.e. those with a `url` source.
Some parts of that script have been disabled in the `path` case
that likely shouldn't be, but cause errors and aren't necessary to get
`vagrant` to work.
Keep the `source` attrset distinct to prevent its entries from merging
with the top level attrs.
Since each type of source has a different set of entries for `source`,
this is the easiest way to keep them together.
This will pave the way for a new `url` type of source.
This is a mass-rebuild of many ruby packages,
notably those using `git` type sources.
* Manage patches in git
* Fixes the hook invocation to be more safe. Thanks @Mic92
* Install gems as user by default
* Install gem binaries with the /usr/bin/env shebang
* Fixes a bug where the passthru.libPath and passthru.gemPath would
point to the wrong directory
* Overhaul ruby version heuristics
If a gemspec has UTF-8 characters in it, ruby will fail loading it with
invalid multibyte char (US-ASCII)
This change forces the encoding to be correct, we assume everyone now
uses UTF-8.
For some reason `gem install` unsets the GEM_PATH environment variable
internally unless the install dir is provided. This in turn means that
if it invokes extconf.rb and extconf.rb depends on a gem available on
the GEM_PATH (like pkg-config for nokogiri) then it's not available in
that context.
Proof: d8293c4729/lib/rubygems/commands/install_command.rb (L151)
Blame: 9ea600c9c2
This is a hack that sets the :install_dir to where we would install
anyways (the GEM_HOME is the default installation destination).
bundler for example needs to have the GEM_HOME being passed trough to
function properly.
For gems that are loading content dynamically, or can use plugins, use
buildRubyGem.
For executables that are wrapped in their own sealed thing use
bundlerEnv.
lib.escapeShellArgs doesn't work well when a null value is provided.
[] is also the correct value since it's really just an empty list of
arguments that we have.
FliegendeWurst