Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perfparser/Hotspot cannot unwind the stack for fork() of created processes #241

Closed
vtoropov opened this issue May 19, 2020 · 11 comments
Closed
Assignees
Labels

Comments

@vtoropov
Copy link

The problem was discovered while working with the coremark test.
It concerns multi-threaded mode and USE_FORK model of the test as well as any fork() application.

There is large discrepancy between the values of events costs values for symbols on the Hotspot's Caller/Callee tab and corresponded values of 'perf report'.

The following is 'perf report' output:

image

And hotspot:

image

As you can see the events cost are very different not in favor of hotspot.
coremark.exe was built by the command (make XCFLAGS = "- DMULTITHREAD = 4 -DUSE_FORK = 1")

@vtoropov vtoropov added the bug label May 19, 2020
@vtoropov
Copy link
Author

Hi Milian, could you please appoint me for fixing this bug.
Thanks a lot,
-- Valentin.

@milianw
Copy link
Member

milianw commented May 19, 2020

Hey! Thanks for the report.

First of all: what do you mean by "appoint me for fixing this bug" - do you want to look into this issue yourself and submit a patch eventually? I.e. should I assign this issue to you?

I'd obviously appreciate that :) But for starters I suggest you create a smaller example that shows this issue which doesn't depend on a larger application like coremark (https://www.eembc.org/coremark/ ?)

@vtoropov
Copy link
Author

Thank you for the response!

Yes, you are right. I’ve meant assigning the issue to me.
I have already prepared a fix for it in my sandbox.

Below is the output of Hotspot for the same perf.data file after my fix.
image

A little bit later I will prepare a small sample.
Thanks,
--Valentin.

@milianw
Copy link
Member

milianw commented May 20, 2020

cool, looking foward to the merge request

but since this will most probably target perfparser, could you please consider to push this patch upstream to the qt gerrit perfparser too?

@vtoropov
Copy link
Author

Milian,
Yes, I would like to consider to push this patch upstream to the qt gerrit perfparser too.
Could you please point me the following steps to looking forward?

@milianw
Copy link
Member

milianw commented May 23, 2020

maybe start with a merge request for kdab/perfparser and then once that's done upstream it - see https://www.qt.io/contribute-to-qt

@vtoropov
Copy link
Author

I created a merge request for KDAB/perfparser.
Should I create a merge request for KDAB/hotspot perfparser too.

@vtoropov
Copy link
Author

vtoropov commented Jun 1, 2020

Hi, Milian.
I have fix for #178. dwfl_attach_state should be called only for user-level CPU register state and user level stack, allowing stack unwinding.
And what about my merge request for KDAB/perfparser?

@milianw
Copy link
Member

milianw commented Jun 5, 2020

hey there @vtoropov, sorry for the delay - I've now reviewed your other change. Regarding #178 - please sent the patch as a merge request and then I can review it there.

@milianw
Copy link
Member

milianw commented Jun 12, 2020

I've started on a fix for this issue now and will submit it upstream too.

milianw added a commit to KDAB/perfparser that referenced this issue Jun 13, 2020
A forked process inherits the elfmap of the parent, thus we need to
copy that data when we encounter a fork event. To do so reliably,
we need to flush the event buffer, as otherwise the mmmap state may
not be accurate.

Furthermore, pass the ppid through the ThreadStart event, to allow
client applications to inherit the comm for the newly created process.

Fixes: KDAB/hotspot#241
Change-Id: I5de13644e12def6704c5f622428a815fd87d2af4
@milianw
Copy link
Member

milianw commented Jun 13, 2020

this issue isn't fully resolved yet, I found a way to reproduce this in a bigger application where the order in which we process events is important - will reopen

@milianw milianw reopened this Jun 13, 2020
milianw added a commit to KDAB/perfparser that referenced this issue Jun 13, 2020
A forked process inherits the elfmap of the parent, thus we need to
copy that data when we encounter a fork event. This is complicated by
the fact that the task events and mmap events need to be sorted first.
Thus, we need to find the fork event when we deplete the buffers after
sorting and then initialize the child process elf map with the one
from the parent process.

Furthermore, pass the ppid through the ThreadStart event, to allow
client applications to inherit the comm for the newly created process.

Fixes: KDAB/hotspot#241
Change-Id: I5de13644e12def6704c5f622428a815fd87d2af4
milianw added a commit to KDAB/perfparser that referenced this issue Jun 13, 2020
A forked process inherits the elfmap of the parent, thus we need to
copy that data when we encounter a fork event. This is complicated by
the fact that the task events and mmap events need to be sorted first.
Thus, we need to find the fork event when we deplete the buffers after
sorting and then initialize the child process elf map with the one
from the parent process.

Furthermore, pass the ppid through the ThreadStart event, to allow
client applications to inherit the comm for the newly created process.

Fixes: KDAB/hotspot#241
Change-Id: I5de13644e12def6704c5f622428a815fd87d2af4
milianw added a commit to KDAB/perfparser that referenced this issue Jun 13, 2020
A forked process inherits the elfmap of the parent, thus we need to
copy that data when we encounter a fork event. This is complicated by
the fact that the task events and mmap events need to be sorted first.
Thus, we need to find the fork event when we deplete the buffers after
sorting and then initialize the child process elf map with the one
from the parent process.

Furthermore, pass the ppid through the ThreadStart event, to allow
client applications to inherit the comm for the newly created process.

Fixes: KDAB/hotspot#241
Change-Id: I5de13644e12def6704c5f622428a815fd87d2af4
milianw added a commit to KDAB/perfparser that referenced this issue Jun 14, 2020
A forked process inherits the elfmap of the parent, thus we need to
copy that data when we encounter a fork event. This is complicated by
the fact that the task events and mmap events need to be sorted first.
Thus, we need to find the fork event when we deplete the buffers after
sorting and then initialize the child process elf map with the one
from the parent process.

Furthermore, pass the ppid through the ThreadStart event, to allow
client applications to inherit the comm for the newly created process.

Fixes: KDAB/hotspot#241
Change-Id: I5de13644e12def6704c5f622428a815fd87d2af4
milianw added a commit to KDAB/perfparser that referenced this issue Dec 29, 2020
A forked process inherits the elfmap of the parent, thus we need to
copy that data when we encounter a fork event. This is complicated by
the fact that the task events and mmap events need to be sorted first.
Thus, we need to find the fork event when we deplete the buffers after
sorting and then initialize the child process elf map with the one
from the parent process.

Furthermore, pass the ppid through the ThreadStart event, to allow
client applications to inherit the comm for the newly created process.

Fixes: KDAB/hotspot#241
Change-Id: I5de13644e12def6704c5f622428a815fd87d2af4
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants