-
Notifications
You must be signed in to change notification settings - Fork 6.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow CONFIG_POSIX_API on CONFIG_ARCH_POSIX #45100
Comments
For background, this topic was discussed, between others, here: #13054, and the PR that disabled the combination was #13540 About the purposes of native_posix: those are listed in the documentation. The "run Zephyr inside of a POSIX process in a production system (†)" is not one, and I very strongly discourage it for several reasons. @cfriedt, please, remove that from the description as that may mislead those who read it. Please note that there is a (very common) misconception in the description that would be good fixing: "native_posix" is a target board, which is not the same (although it uses) the "posix architecture". It would be nice if you corrected it. This matter does in any case not block the other PR you refer to, as you can simply leave the combination of POSIX ARCH and POSIX API shim disabled. About the "both dynamic and static proposals rely on visibility attribute (ELF-specific)", I did explain that this is not the case for the static library version proposal, but you seem to reiterate on it being so, so I would hope you can either back your statement or remove it. Regarding your other listed concerns, I do not agree with them: The 2nd line is not pointing at anything in particular. About the 3rd point , I do not see why the static library version would have per se any problem with macOS, but maybe you'd be so kind as to elaborate. Regarding AMP on simulated targets, that is something that some people have interest on, and that is enabled/made easier by that proposal; you may not have no interest in it, but I hope it does not prevent from understanding the proposal in itself, which can be applied fully ignoring the AMP use case. |
@aescolar Thanks for the links - I'll add them to the summary.
I strongly discourage it as well. It has been suggested verbally in the past, which is why I mentioned it as a hypothetical.
Done.
I'll consider it. I would rather just solve this issue here and then allow the other PR to move forward.
I re-reviewed the static proposal and it still mentions "localizing hidden symbols (those not declared with default visibility)". Symbol locality, prelinking, objdump capabilities. A lot of that is ELF-specific.
For the reasons mentioned above. |
In the static case the output of the "pre-link" stage is an object file. Local symbols are just not global symbols, meaning |
By default, symbols are global unless declared static, in terms of plain-old C. Symbol "visibility" (at the compiler level) is an ELF construct (it is not available for Mach-O targets, iirc). However, Mach-O does support tooling to categorize symbols as "exported" and "unexported" and the object format also supports differentiating between global and local. #define N_GSYM 0x20
#define N_LSYM 0x80 https://opensource.apple.com/source/xnu/xnu-2050.18.24/EXTERNAL_HEADERS/mach-o/stab.h
The rest of this does sound very application-specific. A similar concept that went in recently (or will be going in recently) is that of multi-target images - building a firmware image that includes e.g. application core, radio, or coprocessor firmware, all in one output file. The AMP aspect of what you've proposed sounds like it has the same goals. E.g. with a bootloader deciding how to load code for each (co-) processor. I think for the purposes of this issue, I would prefer to focus on just the Zephyr aspect of running |
Currently, the build depends on zephyrproject-rtos/zephyr#43987 That did originally include a change to allow CONFIG_POSIX_ARCH and CONFIG_POSIX_API to co-exist but the arch/posix maintainer rejected the changes. A workaround is required until the issue below is resolved. zephyrproject-rtos/zephyr#45100 Fixes #81 Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
Currently, the build depends on zephyrproject-rtos/zephyr#43987 That did originally include a change to allow CONFIG_POSIX_ARCH and CONFIG_POSIX_API to co-exist but the arch/posix maintainer rejected the changes. A workaround is required until the issue below is resolved. zephyrproject-rtos/zephyr#45100 Fixes #81 Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
This is a temporary workaround until the mountain of tech-debt is flattened w.r.t the following: zephyrproject-rtos/zephyr#43987 zephyrproject-rtos/gsoc-2022-thrift#62 zephyrproject-rtos/zephyr#43998 zephyrproject-rtos/zephyr#46910 zephyrproject-rtos/zephyr#45100 but also around the mountain of tech-debt as a result of poor arch/posix software architecture, CONFIG_ARCH_POSIX and CONFIG_POSIX_API incompatibility, and the resultant spillover of moving POSIX definitions into the network subsystem as a result. Fixes zephyrproject-rtos/gsoc-2022-thrift#129 Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
This is a temporary workaround until the mountain of tech-debt is flattened w.r.t the following: zephyrproject-rtos/zephyr#43987 #62 zephyrproject-rtos/zephyr#43998 zephyrproject-rtos/zephyr#46910 zephyrproject-rtos/zephyr#45100 but also around the mountain of tech-debt as a result of poor arch/posix software architecture, CONFIG_ARCH_POSIX and CONFIG_POSIX_API incompatibility, and the resultant spillover of moving POSIX definitions into the network subsystem as a result. Fixes #129 Signed-off-by: Christopher Friedt <cfriedt@fb.com>
This is a temporary workaround until the mountain of tech-debt is flattened w.r.t the following: zephyrproject-rtos/zephyr#43987 #62 zephyrproject-rtos/zephyr#43998 zephyrproject-rtos/zephyr#46910 zephyrproject-rtos/zephyr#45100 but also around the mountain of tech-debt as a result of poor arch/posix software architecture, CONFIG_ARCH_POSIX and CONFIG_POSIX_API incompatibility, and the resultant spillover of moving POSIX definitions into the network subsystem as a result. Fixes #129 Signed-off-by: Christopher Friedt <cfriedt@fb.com>
This is a temporary workaround until the mountain of tech-debt is flattened w.r.t the following: zephyrproject-rtos/zephyr#43987 #62 zephyrproject-rtos/zephyr#43998 zephyrproject-rtos/zephyr#46910 zephyrproject-rtos/zephyr#45100 but also around the mountain of tech-debt as a result of poor arch/posix software architecture, CONFIG_ARCH_POSIX and CONFIG_POSIX_API incompatibility, and the resultant spillover of moving POSIX definitions into the network subsystem as a result. Fixes #129 Signed-off-by: Christopher Friedt <cfriedt@fb.com>
This is a temporary workaround until the mountain of tech-debt is flattened w.r.t the following: zephyrproject-rtos/zephyr#43987 #62 zephyrproject-rtos/zephyr#43998 zephyrproject-rtos/zephyr#46910 zephyrproject-rtos/zephyr#45100 but also around the mountain of tech-debt as a result of poor arch/posix software architecture, CONFIG_ARCH_POSIX and CONFIG_POSIX_API incompatibility, and the resultant spillover of moving POSIX definitions into the network subsystem as a result. Fixes #129 Signed-off-by: Christopher Friedt <cfriedt@fb.com>
This is a temporary workaround until the mountain of tech-debt is flattened w.r.t the following: zephyrproject-rtos/zephyr#43987 zephyrproject-rtos/gsoc-2022-thrift#62 zephyrproject-rtos/zephyr#43998 zephyrproject-rtos/zephyr#46910 zephyrproject-rtos/zephyr#45100 but also around the mountain of tech-debt as a result of poor arch/posix software architecture, CONFIG_ARCH_POSIX and CONFIG_POSIX_API incompatibility, and the resultant spillover of moving POSIX definitions into the network subsystem as a result. Fixes zephyrproject-rtos/gsoc-2022-thrift#129 Signed-off-by: Christopher Friedt <cfriedt@meta.com>
This is a temporary workaround until the mountain of tech-debt is flattened w.r.t the following: zephyrproject-rtos/zephyr#43987 zephyrproject-rtos/gsoc-2022-thrift#62 zephyrproject-rtos/zephyr#43998 zephyrproject-rtos/zephyr#46910 zephyrproject-rtos/zephyr#45100 but also around the mountain of tech-debt as a result of poor arch/posix software architecture, CONFIG_ARCH_POSIX and CONFIG_POSIX_API incompatibility, and the resultant spillover of moving POSIX definitions into the network subsystem as a result. Fixes zephyrproject-rtos/gsoc-2022-thrift#129 Signed-off-by: Christopher Friedt <cfriedt@meta.com>
This is a temporary workaround until the mountain of tech-debt is flattened w.r.t the following: zephyrproject-rtos/zephyr#43987 zephyrproject-rtos/gsoc-2022-thrift#62 zephyrproject-rtos/zephyr#43998 zephyrproject-rtos/zephyr#46910 zephyrproject-rtos/zephyr#45100 but also around the mountain of tech-debt as a result of poor arch/posix software architecture, CONFIG_ARCH_POSIX and CONFIG_POSIX_API incompatibility, and the resultant spillover of moving POSIX definitions into the network subsystem as a result. Fixes zephyrproject-rtos/gsoc-2022-thrift#129 Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
This is a temporary workaround until the mountain of tech-debt is flattened w.r.t the following: zephyrproject-rtos/zephyr#43987 zephyrproject-rtos/gsoc-2022-thrift#62 zephyrproject-rtos/zephyr#43998 zephyrproject-rtos/zephyr#46910 zephyrproject-rtos/zephyr#45100 but also around the mountain of tech-debt as a result of poor arch/posix software architecture, CONFIG_ARCH_POSIX and CONFIG_POSIX_API incompatibility, and the resultant spillover of moving POSIX definitions into the network subsystem as a result. Fixes zephyrproject-rtos/gsoc-2022-thrift#129 Signed-off-by: Christopher Friedt <cfriedt@meta.com>
@carlescufi yep. |
Does that mean we can test POSIX_API on ARCH_POSIX? |
Yes, with native_sim. The tests are already enabled in CI. |
@aescolar - amazing!! Thank you! |
Reopening, because this does not seem to be finished yet. |
Sorry -for clarification, this works, but with not with |
Is your enhancement proposal related to a problem? Please describe.
Problem:
CONFIG_ARCH_POSIX
andCONFIG_POSIX_API
should not be mutually exclusiveIntent:
With the POSIX arch, we want to abstract away the host OS system calls (and parts of the system C library) in order to
For further details on intent see the docs
Clarification on
CONFIG_POSIX_API
:i. not strictly adhering the the embedded POSIX profile
CONFIG_ARCH_POSIX
, The Zephyr application should not interface directly to the host OS system calls / or libc implementationsThe
native_posix
port used to rely onposix_cheats.h
to perform some questionable preprocessor redefinitions of standard function names in order to avoid conflicts in the C global symbol namespace. This was obviously not portable and somewhat fragile.After some time, it was decided that
CONFIG_ARCH_POSIX
andCONFIG_POSIX_API
were mutually exclusive options (see #13054, #13540).This proposal is intended to show that
CONFIG_ARCH_POSIX
andCONFIG_POSIX_API
do not need to be (and really should not be) mutually exclusive, and that functionality can be achieved with conventional OOP methods and very mature, standard APIs.Moreover, the proposed solution is completely scalable, maintainable, and portable to any OS (likely even including non-POSIX operating systems).
Describe the solution you'd like
native_posix
i. for now this can be limited to C function & symbol definitions
i. portable and standard C
i. standard POSIX APIs (e.g.
dlopen()
,dlsym()
, can be extended to e.g. Windows API)i. conventional OOP methods (encapsulation / abstraction)
i. leverage
Kconfig
to (optionally) provide the name or path of the host libcii. leverage
Kconfig
to (optionally) require a strong checksum for the host libcW.r.t. host libc name, on GNU systems, the default fault is obtainable programmatically with
#include <gnu/lib-names.h>
andLIBC_SO
. However, it should be possible to link to other libc's as well.It would probably make sense to encapsulate this abstraction on a per-OS bases by having separate compilation units (e.g. posix-arch-if-linux.c, posix-arch-macos.c).
Describe alternatives you've considered
Proposal in #24685
There may be other RFCs on this subject
Additional context
This enhancement is a requirement / blocks #43987 which
CONFIG_POSIX_API
andCONFIG_ARCH_POSIX
are not mutually exclusivePR on its way
The text was updated successfully, but these errors were encountered: