-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problems with musl's ldso on 1.6 #40556
Comments
Also likely related is #36458 |
Can you use |
I can set them to match (and I think they do by default), but it doesn't appear that musl actually looks at SONAME, which explains why fiddling with SONAME doesn't really help. All it looks at is NEEDED, and whatever paths were originally passed to |
Building the library with the right SONAME (see the patch in Alpine) then should help because we'd open the library in the JLL package with the same name as the NEEDED one. |
I'm doing some printf debugging with ldso and julia, and also asking around on the musl IRC. It seems like we may also be doing something "wrong", in that we do a |
Ok, so I've spoken a bit with the musl devs, and here's the gist of it:
Some relevant quotes:
|
The musl devs have refused my proposal to patch ldso to look at SONAME and compare it with NEEDED entries. Therefore, I have prepared a patch which adventurous users may apply to their ldso to workaround this issue. WARNING: I do not make any claims about how safe this patch is to use in general; I recommend only applying this as your Julia interpreter, and that's it ( --- ldso/dynlink.c 2021-01-14 20:26:00.000000000 -0600
+++ ldso/dynlink.c 2021-04-22 14:21:02.230108318 -0500
@@ -93,6 +93,7 @@
struct td_index *td_index;
struct dso *fini_next;
char *shortname;
+ char *soname;
#if DL_FDPIC
unsigned char *base;
#else
@@ -1047,6 +1048,8 @@
for (p=head->next; p; p=p->next) {
if (p->shortname && !strcmp(p->shortname, name)) {
return p;
+ } else if (p->soname && !strcmp(p->soname, name)) {
+ return p;
}
}
if (strlen(name) > NAME_MAX) return 0;
@@ -1153,6 +1156,12 @@
return 0;
}
memcpy(p, &temp_dso, sizeof temp_dso);
+ for (int i=0; p->dynv[i]; i+=2) {
+ if (p->dynv[i] == DT_SONAME) {
+ p->soname = p->strings + p->dynv[i+1];
+ break;
+ }
+ }
p->dev = st.st_dev;
p->ino = st.st_ino;
p->needed_by = needed_by;
@@ -1885,6 +1894,11 @@
reclaim_gaps(&app);
reclaim_gaps(&ldso);
+ /* Set initial soname */
+ app.soname = NULL;
+ ldso.soname = NULL;
+ vdso.soname = NULL;
+
/* Load preload/needed libraries, add symbols to global namespace. */
ldso.deps = (struct dso **)no_deps;
if (env_preload) load_preload(env_preload); License is MIT, same as musl. |
So if I'm understanding correctly, musl already has a fast-path for previously-loaded libraries, which is the The fast-path: https://github.com/ifduyue/musl/blob/aad50fcd791e009961621ddfbe3d4c245fd689a3/ldso/dynlink.c#L1048-L1052 Where It seems to me that possibly, we don't get |
Pinging @richfelker, who may have a better idea on how to solve this. Julia, as a language, makes use of the behavior on glibc Linux, macOS, FreeBSD and Windows, that if This allows us to distribute libraries independently (e.g. we do a The difficulty on musl is that when we I'm curious to hear your thoughts about the best way to achieve feature parity here. As far as I can tell, there is no way for a process to satisfy the dynamic linker when loading libraries if it doesn't know the location of all the libraries beforehand, because in musl the only way for us to influence the search locations are by embedding paths into objects (e.g. rpath/runpath, not usable here because we don't know the relative paths between dependencies at compile-time) or setting environment variables (e.g. Thank you for your time! |
Can you explain what you're trying to achieve? The current behavior is intentional and serves to prevent modules loaded with an explicit pathname from shadowing libraries in the search path just because they happen to have the same name. If libfoo depends on a version of libbar at a particular location (absolute or relative to libfoo), the intended way to represent that is via rpath in libfoo, using |
We ship libraries to users outside of normal distribution channels; they do not get installed into e.g.
This makes sense, however I believe the idea behind the |
@richfelker just checking in with you to see if you have any suggestions on how we can get
Thank you again for you time! |
Note that I did not in fact submit the patch to musl, since I was told on official IRC by a mod that they were not interested in SONAME support and would not accept the patch. |
I still don't understand why it's as complicated as it is, but a really stupid solution is just putting a temp dir in the library path and instead of |
The fundamental constraint is that our libraries are pre-built and immutable (no patching them after installation) and you cannot know, at build time, what the exact path where they will be installed will be, yet they need to be able to depend on each other. This is why it's complicated. Although, I would argue this is how dynamic libraries should work, but that's a matter of opinion. Your workaround is interesting—having to symlink a library into a temp directory for library loading seems... unfortunate, although I suppose at least we can probably count on tempdirs and symlinks working reasonably well on any musl system. |
Also, would you mind giving some rationale for the lack of SONAME support in musl? Is there some reason that feature would be problematic? |
I don't see it as just "SONAME support" but as the intersection of 2 behaviors: SONAME based lookup and insertion of libraries loaded via explicit pathnames into the set of names searched for dependencies. The latter of these was intentionally not done, as explained before. Doing it just for SONAMEs might be a reasonable behavior, but it's something with potentially far-reaching consequences that would call for a proposal, analysis, input from community, etc. This is a big ask for something that has only come up with one project using |
I posted https://www.openwall.com/lists/musl/2021/12/16/1 "Satisfying DT_NEEDED from previous dlopens with explicit path" to the musl mailing list about this issue. |
Hello, I've bumped into the dlopen issue while testing a _jll I produced (issue report). Thanks @giordano to have pointed me to this thread. Was a decision taken? If it's not possible to change the linker behaviour, what about including in the linker search path a directory, where a link is set to each provided library (including not dlopen'ed ones) when a _jll is imported? Links to libraries loaded by a dlopen called in another context can also be added in order to have similar behaviours with musl and glibc linkers . Alternatively it can be links to artifact directories and libraries rpaths set accordingly. Links to executables provided by the _jll's can also be included, saving the need of a long LD_LIBRARY_PATH to run them. The difference with Rich's suggestion is that the links are kept during the whole Julia session and the solution does not rely on the linker skipping dependencies previously dlopen'd (using a relative or absolute path). Philippe. |
Jameson, Mosè, Valentin and I talked about this, and we came up with three ways forward:
Honorable mention to the idea of building glibc's |
For someone willing to implement solution 3, to get inspiration you can look at |
Thanks Elliot, Jameson, Mosè, and Valentin to have looked at the issue. Did you discuss about the symbolic link solution? Did you rule out, and if so, what would be the issues with this option ? |
In order to add a new directory onto the search path, we'd need to have it as part of |
The links could be set when loading the _jll modules. With a single directory tree |
You need one per Julia environment, which cannot be done with just RPATH, and you need to be able to change it when you activate a different environment, which means it must be per-process and cannot be shared. |
My idea was to still use the full path of symbolic link for the If the links are created when loading the Note that this will add the possibility for a |
This does not work because the dynamic linker sees absolute paths, not symlinks; everything is dereferenced by the time the dynamic linker gets ahold of it. You can see this in action with this testing repository, where |
I'm not clear how this would help. If you just want your own calls to
glibc's |
Yes, we want that. We also want to have the same logic applied (e.g. short-circuiting the search for a library if another library with the same |
OK, in that case, if I understand things correctly, the solution is not to use ELF-level |
We're not loading libraries that we manage here; we're loading things like GTK. We're more or less running a userspace distribution, but for more than just Linux. We install our libraries into content-addressed storage (similar to Nix), and manually call
|
For some reason, musl's loader refuses to reuse libraries that it's already loaded to service later requests for those same libraries. For example,
libbz2.so
from Bzip2_jll can be successfullydlopen
'd on its own, but when trying todlopen
libfreetype.so
from FreeType2_jll, we can see (throughstrace
) musl's loader looking through entries inLD_LIBRARY_PATH
instead. Trying to modify JLLWrappers.jl to do awithenv("LD_LIBRARY_PATH"=>join(LIBPATH_list, ':')) do ...
around thedlopen
does not seem to affect the search path, but I can setLD_LIBRARY_PATH
manually outside of Julia as usual (so I hypothesize that the environment is cached whenld
is first loaded?).Of course, this doesn't always happen. For me on Alpine Linux, JLLs that do not depend on Bzip2_jll (and a few other problematic JLLs) seem to work fine. @giordano hypothesizes that it's because of a difference between the SONAME and NEEDED entries between dependent libraries, but I've found that even by using
patchelf
to change these entries, nothing of benefit happens (when it does work, it's only because the loader instead found a matching system library). However, it's totally possible I'm doing this wrong.If anyone has suggestions on other things to look at or try, please let me know!
The text was updated successfully, but these errors were encountered: