Use kernel headers from CONFIG_IKHEADERS

[alban]

I would like kubectl-trace to be able to use kernel headers from CONFIG_IKHEADERS. I am interested in supporting Minikube.
I added CONFIG_IKHEADERS in Minikube (PR#8582; it is available in release v1.12.0-beta.1 but not in release v1.12.0, see regression in kubernetes/minikube#8556 (comment)).

If we get it to work, we can update the doc PR#8607 with examples from kubectl-trace in addition to the bcc & Inspektor Gadget examples.

bpftrace has PR#768 "utils: unpack kheaders.tar.xz if necessary" and it is included in kubectl-trace in commit 2e97e64 but it is unreleased yet.

Currently, kheaders.tar.xz cannot be read because xz is not installed:

if your program has maps to print, send a SIGINT using Ctrl-C, if you want to interrupt the execution send SIGINT two times
tar (child): xz: Cannot exec: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
/bpftrace/include/clang_workarounds.h:14:10: fatal error: 'linux/types.h' file not found
exit status 1

I tried to install it with the patch:

--- a/build/Dockerfile.tracerunner
+++ b/build/Dockerfile.tracerunner
@@ -13,6 +13,8 @@ RUN make _output/bin/trace-runner
 
 FROM ubuntu:19.10
 
+RUN apt-get install -y xz-utils
+
 COPY --from=gobuilder /go/src/github.com/iovisor/kubectl-trace/_output/bin/trace-runner /bin/trace-runner
 COPY --from=bpftrace /usr/bin/bpftrace /usr/bin/bpftrace
 

But it seems it still fail, none of the tracepoints work:

if your program has maps to print, send a SIGINT using Ctrl-C, if you want to interrupt the execution send SIGINT two times
ERROR: tracepoint not found: syscalls:sys_enter_open
exit status 1

Tasks:

• Add CONFIG_IKHEADERS in Minikube again, see regression in Kernel with CONFIG_IKHEADERS for BPF tools on Kubernetes kubernetes/minikube#8556 (comment)
• Install xz in the container image (feat: add xz-utils in container image for CONFIG_IKHEADERS support #123)
• Fix CONFIG_FTRACE_SYSCALLS in Minikube Kernel with CONFIG_FTRACE_SYSCALLS kubectl-trace kubernetes/minikube#8637
• Investigate next issues
• Make a new release of kubectl-trace
• Update Minikube documentation in PR#8607

cc @marga-kinvolk @mauriciovasquezbernal

[alban]

Some events are available with Minikube but not syscalls:

$ ls -l /sys/kernel/debug/tracing/events/syscalls/
ls: cannot access '/sys/kernel/debug/tracing/events/syscalls/': No such file or directory
$ ls -l /sys/kernel/debug/tracing/events/|wc -l
94
$ zcat /proc/config.gz |grep -i CONFIG_FTRACE_SYSCALLS  
# CONFIG_FTRACE_SYSCALLS is not set

[leodido]

Strong +1