support --disable-static option on TruffleRuby

[flavorjones]

See for more context.

TruffleRuby, by default, sets the --use-system-libraries option (via these lines in mkmf.rb). Because of how Nokogiri's extconf.rb is currently implemented, this setting takes precedence over other options like --disable-static which would allow users to compile the packaged, patched versions of libxml2/libxslt and use them as shared libraries (instead of the system libraries).

I've proposed, in https://github.com/Shopify/oracle-truffleruby-collab/issues/11, to change Nokogiri's extconf.rb to essentially follow this logic:

static_p = enable_config("static", true) # defaults to true, overridden to false by `--disable-static`
if defined?(::TruffleRuby)
  if static_p
    # use system libraries - sane default to make installation easy
  else
    # use packaged libraries, built as shared objects - use the patched packaged versions, which might be hard for some systems to compile
  end
end

Essentially, this means that the environment variable NOKOGIRI_USE_SYSTEM_LIBRARIES would be presumed to be the default behavior on TruffleRuby; but allow it to be overridden.

[eregon]

I've made a small change locally to allow overriding NOKOGIRI_USE_SYSTEM_LIBRARIES with command-line arguments (PR incoming: #2193).

With that, it's possible to install nokogiri in all 3 variants. I'm using the latest TruffleRuby nightly build.

Current default, using system libraries:

$ time gem install --local pkg/nokogiri-1.11.1.gem    
Building native extensions. This could take a while...
Successfully installed nokogiri-1.11.1
1 gem installed
 34.24s user 2.19s system 424% cpu 8.579 total

Compiling the vendored libxml2/libxslt as shared libraries:

$ time gem install --local pkg/nokogiri-1.11.1.gem -- --use-system-libraries=false --disable-static 
Building native extensions with: '--use-system-libraries=false --disable-static'
This could take a while...
Successfully installed nokogiri-1.11.1
1 gem installed
99.45s user 12.71s system 281% cpu 39.789 total

Compiling the vendored libxml2/libxslt as static libraries (the default on CRuby):

$ time gem install --local pkg/nokogiri-1.11.1.gem -- --use-system-libraries=false                  
Building native extensions with: '--use-system-libraries=false'
This could take a while...
Successfully installed nokogiri-1.11.1
1 gem installed
136.03s user 13.37s system 223% cpu 1:06.99 total

The latter 2 include the libxml2 patches from nokogiri since they use the vendored version.
They are also compiled as bitcode (in addition to native code) and run on Sulong.

Running on Sulong might actually fix #1882, but the drawback is then libxml2/libxslt need to warm up on Sulong, which is slower than native code, and for large C libraries like those it's probably a better experience to run them natively when possible, and the flexibility from Sulong is not needed.

I think this is something that is probably best to solve on TruffleRuby or Sulong's side, where one could specify that some library dependencies should be run natively and not on Sulong, even if they have bitcode.

Regarding defaults, it seems vendored libxml2/libxslt as shared libaries is strictly better on TruffleRuby than vendored libxml2/libxslt as static libraries:

• Faster to compile
• Flexibility to run the vendored libraries as native code (this is very difficult with static libraries, but should be easy with shared libraries)
• Clear dependencies as seen with ldd/otool (minor, but nice for debugging).

Between shared libraries and system libraries it's less clear.

• --use-system-libraries installs faster (9s vs 40s, but 40s isn't really long either)
• but --use-system-libraries requires to have the libraries installed on the system which is inconvenient in CI, notably it's an issue on GitHub Actions and TravisCI which don't have them installed by default. On GitHub Actions, the libraries are automatically installed with ruby/setup-ruby and bundler-cache: true, but that takes an extra 13-15s.
• and --use-system-libraries has extra bugs and doesn't match the default behavior on CRuby
• however --use-system-libraries is probably slightly better for warmup until we have support in TruffleRuby/Sulong to always execute libxml2/libxslt as native. Early testing seems to show that running libxml2/libxslt is fast enough and warmup is OK, at least in TruffleRuby --native mode (the default).

[flavorjones]

Closing this issue in favor of the work at #2193