Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

get kselftests target building #1698

Open
nickdesaulniers opened this issue Aug 25, 2022 · 12 comments
Open

get kselftests target building #1698

nickdesaulniers opened this issue Aug 25, 2022 · 12 comments
Labels
kselftests This issue affects building the Linux kernel selftests (tools/testing/selftests)

Comments

@nickdesaulniers
Copy link
Member

Looking at #906 , I noticed that make LLVM=1 -j72 kselftest fails with tons of warnings and errors; maybe we can use this as a metabug, since there will probably be lots of fixes necessary for this.
@nickdesaulniers
Copy link
Member Author

cc @gctucker , looking at #374 I thought we had kselftests building with clang...

@gctucker
Copy link

Well there are a bunch of errors but some tests are building OK as far as I can tell:
https://storage.staging.kernelci.org/kernelci/staging-next/staging-next-20220830.0/x86_64/x86_64_defconfig+kselftest/clang-16/logs/kselftest.log

@gctucker
Copy link

Note that there are 2 fixes applied on this KernelCI staging-next branch, for when the arch is x86_64 or i386 rather than just the ambiguous x86:
kernelci@839f7af
kernelci@bbda7f9

They've been reviewed on the mailing list by @nathanchance so in principle it's just a matter of time until they get applied to the kselftest tree. Not sure if that's related to the issues you're seeing, but worth mentioning anyway.

@JustinStitt
Copy link
Collaborator

Here's a tentative series from the mailing list that Benjamin and I have worked on that solves (at least) my kselftest build issues:

https://lore.kernel.org/all/20230825182316.m2ksjoxe4s7dsapn@google.com/

After discussion, Benjamin proposed a 3-patch series which fixed some of the issues I was experiencing. I, then, added a 4th patch which should be applied on top of this series. Hopefully Benjamin can re-roll this into an n=4 patch series so that b4 shazam applies it cleanly.

Once this is re-rolled into a single n=4 series (and subsequently applied to next) I think we can close this issue 😸

@nickdesaulniers nickdesaulniers mentioned this issue Aug 25, 2023
Open
@JustinStitt
Copy link
Collaborator

Update: I pinged Benjamin off-list regarding the state of this series. Fingers crossed 🤞

intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Sep 8, 2023
@JustinStitt @intel-lab-lkp
selftests/rseq: fix kselftest Clang build warnings
ab94009 
Hi,

I am experiencing many warnings when trying to build tools/testing/selftests.

Here's one such example from rseq tree:
|  param_test.c:1234:10: error: address argument to atomic operation must be a pointer to _Atomic type ('intptr_t *' (aka 'long *') invalid)
|   1234 |         while (!atomic_load(&args->percpu_list_ptr)) {}
|        |                 ^           ~~~~~~~~~~~~~~~~~~~~~~
|  /usr/local/google/home/justinstitt/repos/tc-build/build/llvm/final/lib/clang/18/include/stdatomic.h:140:29: note: expanded from macro 'atomic_load'
|    140 | #define atomic_load(object) __c11_atomic_load(object, __ATOMIC_SEQ_CST)
|        |                             ^                 ~~~~~~

I added the _Atomic type in various locations to silence _all_ (10) of these
warnings. I'm wondering, though, perhaps the absence of these _Atomic
types in the first place is on purpose? Am I on the right track to fix
these warnings without damaging the legitimacy of the tests at hand?

I'd like some feedback about where to go from here and if others are
experiencing the same issues. Thanks!

FWIW here's my specific build incantation on Clang-18 (49d41de57896e935cd5726719c5208bce22694ae):
$ make LLVM=1 -j128 ARCH=x86_64 mrproper headers defconfig kselftest-merge
$ make LLVM=1 ARCH=x86_64 -C tools/testing/selftests

Link: ClangBuiltLinux#1698
Link: ClangBuiltLinux/continuous-integration2#61
Signed-off-by: Justin Stitt <justinstitt@google.com>
@JustinStitt
Copy link
Collaborator

Just to keep this issue up-to-date...

Here are some patches in the works:

intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Sep 12, 2023
@JustinStitt @intel-lab-lkp
selftests/rseq: fix kselftest Clang build warnings
2bf503d 
When building with Clang, I am getting many warnings from the selftests/rseq tree.

Here's one such example from rseq tree:
|  param_test.c:1234:10: error: address argument to atomic operation must be a pointer to _Atomic type ('intptr_t *' (aka 'long *') invalid)
|   1234 |         while (!atomic_load(&args->percpu_list_ptr)) {}
|        |                 ^           ~~~~~~~~~~~~~~~~~~~~~~
|  /usr/local/google/home/justinstitt/repos/tc-build/build/llvm/final/lib/clang/18/include/stdatomic.h:140:29: note: expanded from macro 'atomic_load'
|    140 | #define atomic_load(object) __c11_atomic_load(object, __ATOMIC_SEQ_CST)
|        |                             ^                 ~~~~~~

Use compiler builtins `__atomic_load_n()` and `__atomic_store_n()` with
accompanying __ATOMIC_ACQUIRE and __ATOMIC_RELEASE, respectively. This
will fix the warnings because the compiler builtins do not expect their
arguments to have _Atomic type. This should also make TSAN happier.

Link: ClangBuiltLinux#1698
Link: ClangBuiltLinux/continuous-integration2#61
Suggested-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Justin Stitt <justinstitt@google.com>
akiyks pushed a commit to akiyks/linux that referenced this issue Sep 27, 2023
@JustinStitt @shuahkh
selftests/rseq: fix kselftest Clang build warnings
078a2ea 
When building with Clang, I am getting many warnings from the selftests/rseq tree.

Here's one such example from rseq tree:
|  param_test.c:1234:10: error: address argument to atomic operation must be a pointer to _Atomic type ('intptr_t *' (aka 'long *') invalid)
|   1234 |         while (!atomic_load(&args->percpu_list_ptr)) {}
|        |                 ^           ~~~~~~~~~~~~~~~~~~~~~~
|  /usr/local/google/home/justinstitt/repos/tc-build/build/llvm/final/lib/clang/18/include/stdatomic.h:140:29: note: expanded from macro 'atomic_load'
|    140 | #define atomic_load(object) __c11_atomic_load(object, __ATOMIC_SEQ_CST)
|        |                             ^                 ~~~~~~

Use compiler builtins `__atomic_load_n()` and `__atomic_store_n()` with
accompanying __ATOMIC_ACQUIRE and __ATOMIC_RELEASE, respectively. This
will fix the warnings because the compiler builtins do not expect their
arguments to have _Atomic type. This should also make TSAN happier.

Link: ClangBuiltLinux#1698
Link: ClangBuiltLinux/continuous-integration2#61
Suggested-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Justin Stitt <justinstitt@google.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
@JustinStitt
Copy link
Collaborator

Another update, Benjamin has the hid/bpf fixes applied: https://lore.kernel.org/all/169703194378.3126081.13251971976785038026.b4-ty@kernel.org/

@JustinStitt
Copy link
Collaborator

The plot thickens... It looked like we were finally going to get some clean builds with an incantation like:

$ make LLVM=1 -j128 headers && make LLVM=1 -j128 -C tools/testing/selftests

... but

Commit 14571ab ("kselftest: Add new test for detecting unprobed Devicetree devices")

which is in v6.7-rc1 has introduced some problems.

For me, the build just hangs when it gets to the dt selftest. For others, it seems some errors are produced: https://lore.kernel.org/all/e90cb52f-d55b-d3ba-3933-6cc7b43fcfbc@arm.com/ regarding infinite symlinks.

@gctucker
Copy link

gctucker commented Nov 3, 2023

Commit 14571ab ("kselftest: Add new test for detecting unprobed Devicetree devices")

which is in v6.7-rc1 has introduced some problems.

Yes I saw it got stuck in an infinite loop in the Python script.

@nfraprado Looks like it wasn't just me seeing this problem, do you need some help to fix it or reproduce it?

@nfraprado
Copy link

@gctucker Hi, I just got back from vacation and saw that report in the list regarding the infinite symlinks. I'll look into it and report back.

@nfraprado
Copy link

I've replied to the thread on the mailing list, but just to close the loop, I've sent this patch fixing the issue:
https://lore.kernel.org/all/20231107225624.9811-1-nfraprado@collabora.com/

@nickdesaulniers nickdesaulniers added the kselftests This issue affects building the Linux kernel selftests (tools/testing/selftests) label May 23, 2024
@johnhubbard
Copy link

Hi everyone, @nickdesaulniers asked me to respond here. I recently posted 30 or 35 patches to fix all clang errors and warnings in the kselftests. Status:

11 patches are already merged into Linux main:
commit 14d28ec (selftests/resctrl: fix clang build warnings related to abs(), labs() calls)
commit eb709b5 (selftests/net: fix uninitialized variables)
commit 41b307a (bpftool, selftests/hid/bpf: Fix 29 clang warnings)
commit d8171aa (selftests/resctrl: fix clang build failure: use LOCAL_HDRS)
commit 019baf6 (selftests/binderfs: use the Makefile's rules, not Make's implicit rules)
commit 8f6d24a (selftests/cgroup: fix uninitialized variables in test_zswap.c)
commit 3309ca6 (selftests/cgroup: cpu_hogger init: use {} instead of {NULL})
commit 0515089 (selftests/cgroup: fix clang warnings: uninitialized fd variable)
commit 1da2363 (selftests/cgroup: fix clang build failures for abs() calls)
commit 580ea35 (selftests/mm: fix additional build errors for selftests)
commit e076eac (selftests: break the dependency upon local header files)

These patches are pending. Most have been reviewed, with the notable exception of the x86 subsystem. I'll rebase on top of 6.10-rc1 and repost at that time (Monday):

[PATCH 0/7] selftests/x86: fix build errors and warnings found via clang [1]
[PATCH] selftests/vDSO: fix clang build errors and warnings
[PATCH] selftest/timerns: fix clang build failures for abs() calls
[PATCH] selftests/timers: remove unused irqcount variable
[PATCH] selftests/user_events: silence a clang warning: address of packed member
[PATCH] selftests/alsa: make dump_config_tree() as void function
[PATCH v2] selftests/netfilter: return a value for several "int" functions
[PATCH v2] selftests/mqueue: fix 5 warnings about signed/unsigned mismatches
[PATCH 1/2] selftests/openat2: fix clang build failures: -static-libasan, LOCAL_HDRS
[PATCH 2/2] selftests/fchmodat2: fix clang build failure due to -static-libasan
[PATCH 0/3] selftests/futex: clang-inspired fixes
[PATCH 1/2] selftests/net: suppress clang's "variable-sized type not at the end" warning

Here's a few more that are most likely resolved by other authors' patches:

[PATCH v2] selftests/alsa: missing a return value in unused dump_config_tree()
[PATCH] selftests/exec: build with -fPIE instead of -pie, to make clang happy
Re: [PATCH v2] selftests: exec: make binaries position independent
[PATCH 0/3] binfmt_elf: Honor PT_LOAD alignment for static PIE

[1] https://lore.kernel.org/20240503030214.86681-1-jhubbard@nvidia.com
...I can post the other links if requested, but everything is Cc'd to linux-selftests already.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kselftests This issue affects building the Linux kernel selftests (tools/testing/selftests)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@nickdesaulniers @johnhubbard @gctucker @JustinStitt @nfraprado