Add link flags ocamlmklib when using ctypes stubs.

[frejsoya]

Add link flags for external dependencies when using ctypes.

Required when the external lib is not statically linked, in which case
linker needs to be aware at runtime what libs to call. This is the case
for

• dune exec .bc
• dune utop
• mdx tests

That is add to ocamlmklib, link flags such as -l -L
from ctypes if info is available.

If dependent library are not installed in expected locations, env
variable such as LD_LIBRARY_PATH must be extended when required program
is run, depending on os. This may be fixed by adding relocatable paths to
link flags.

[frejsoya]

This change simply copies what is done in exe_rules.ml

It took some time to figure out that lib_rules.ml has to check explicitly what ctypes believes are the link flags vs. it being contained within in ctype_rules.ml:gen_rules

In the initial PR a test-case for #7146 is a repro about link failures/path expanding. While the test now seemingly "works", The actual problem is that this PR to discards existing c_library_flags stanza.

[lukstafi]

I face the problem of ocamlmklib not getting the library that it's supposed to link with for both the manual "vendored" setup, and using pkg_config (where I generate the .pc file manually).

[rgrinberg]

@lukstafi does this PR fix the problem for you?

[lukstafi]

@lukstafi does this PR fix the problem for you?

Yes! For (build_flags_resolver pkg_config), this PR solves the problem -completely-. For (build_flags_resolver (vendored ...)), with this PR (not synced/rebased) I regress to issue #108. Then, I can ameliorate that one with e.g. CAML_LD_LIBRARY_PATH=./_build/default/lib:$CAML_LD_LIBRARY_PATH dune exec examples/tut01_hello_world.bc.
Strangely/unfortunately, (env (_ (env-vars (CAML_LD_LIBRARY_PATH "./_build/default/lib:...")))) does not work. Edit: vendored also works with just this PR and dune clean; dune build; dune exec -- but I double-checked that dune clean; CAML_LD_LIBRARY_PATH=... dune exec is an alternative way to get it working.

In my case, I would go ahead using (build_flags_resolver pkg_config) (it's cleaner despite the extra complexity).

Edit: Another complication! dune clean; dune exec does not work with PR + (build_flags_resolver pkg_config). But dune clean; dune build; dune exec works; dune clean; CAML_LD_LIBRARY_PATH=... dune exec also works.

[frejsoya]

Note i have little clue about dune internals and even more so with ctypes. Similar its been decades since i dealt with linkers/shared libraries.

There are a few things that need to be correct if i recall correctly.

1. dllstub.so must be locatable by ocamlrun https://v2.ocaml.org/manual/runtime.html#s:ocamlrun-dllpath
2. afaik ocamlrun ends with dlopen(dllstub.so). So any dependencies on libraries needed by dllstub needs to be resolvable as well.

For the vendored libraries (non pkg-config) either manually setting LD_LIBRARY_PATH at runtime or adding the right rolcoatable (rpath) linker flags could resolve that part.

(See rpath discussion below wrt to CMAKE.

https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/RPATH-handling

[lukstafi]

For the vendored libraries (non pkg-config) either manually setting LD_LIBRARY_PATH at runtime or adding the right rolcoatable (rpath) linker flags could resolve that part.

So it looks to me rpath is not helpful in case of dllstubs because it would require different build flags / binaries for development and for install.

[lukstafi]

Another complication, perhaps not related to this PR: bytecode expect-test framework tests: (library (inline_tests) (modes byte) ...) are not run by dune test -- it only triggers a build.

[rgrinberg]

Another complication, perhaps not related to this PR: bytecode expect-test framework tests: (library (inline_tests) (modes byte) ...) are not run by dune test -- it only triggers a build.

Would you like to report a separate issue for this? Preferably with a minimal example that reproduces the issue.