Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a generic palette to highlight kernel / jitted code #203

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

asherkin
Copy link
Contributor

@asherkin asherkin commented Dec 27, 2020

The "java" and "js" colour palettes have a very useful feature of being able to highlight which frames are jitted or kernel functions based on the annotations added by the stackcollapse programs. Unfortunately they also make a number of assumptions based on the function names which are very specific to those languages, and cause nonsensical results with other runtimes.

This PR adds a new generic "annotated" colour palette that only uses the function name annotations, and can thus be used regardless of the runtime being profiled.

The "java" and "js" colour palettes have a very useful feature of
being able to highlight which frames are jitted or kernel functions
based on the annotations added to the function makes by the
stackcollapse programs. Unfortunately they also make a number of
assumptions based on the function names which are very specific to
those languages, and cause nonsensical results with other runtimes.

This PR adds a new generic "annotated" colour palette that only uses
the function name annotations, and can thus be used regardless of
the runtime being profiled.
@codecov
Copy link

codecov bot commented Dec 27, 2020

Codecov Report

Merging #203 (4caf2f3) into master (8fd0289) will decrease coverage by 0.68%.
The diff coverage is 75.60%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #203      +/-   ##
==========================================
- Coverage   90.08%   89.40%   -0.69%     
==========================================
  Files          17       17              
  Lines        2239     2312      +73     
==========================================
+ Hits         2017     2067      +50     
- Misses        222      245      +23     
Impacted Files Coverage Δ
src/collapse/mod.rs 52.63% <0.00%> (-14.04%) ⬇️
src/collapse/common.rs 59.66% <50.00%> (-0.61%) ⬇️
src/flamegraph/color/palettes.rs 98.64% <86.66%> (-0.89%) ⬇️
src/collapse/guess.rs 69.23% <100.00%> (-8.82%) ⬇️
src/collapse/matcher.rs 100.00% <100.00%> (ø)
src/collapse/perf.rs 95.45% <100.00%> (-1.91%) ⬇️
src/flamegraph/color/mod.rs 84.07% <100.00%> (+0.10%) ⬆️
src/flamegraph/mod.rs 97.54% <100.00%> (+1.08%) ⬆️
src/collapse/sample.rs 90.90% <0.00%> (-5.06%) ⬇️
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 43ed780...16e2494. Read the comment docs.

@jonhoo
Copy link
Owner

jonhoo commented Dec 27, 2020

Just out of interest, could you post a flamegraph with this color mode enabled? Would be interesting to see what it looks like!

jonhoo
jonhoo previously approved these changes Dec 27, 2020
@asherkin
Copy link
Contributor Author

Just out of interest, could you post a flamegraph with this color mode enabled? Would be interesting to see what it looks like!

https://lemon.limetech.io/srcds-startup-sp-only.svg

@jonhoo
Copy link
Owner

jonhoo commented Dec 27, 2020

For posterity:
2020-12-27-122618_3840x2160_scrot

That's really neat! I almost wonder if we should make this the default behavior for the Hot palette, and then have a LegacyHot that only uses red as today. @jasonrhansen @bcmyers @itamarst @AnderEnder what do you think?

@jasonrhansen
Copy link
Collaborator

@jonhoo I wouldn't be opposed to that idea.

@itamarst
Copy link
Contributor

The output definitely looks useful. I would suggest documenting the heuristics, so people know what to expect.

@jonhoo
Copy link
Owner

jonhoo commented Dec 31, 2020

@asherkin I'm pretty happy making this the default. Could you update the PR so that Hot uses your new heuristic, and then also document that behavior for the hot color scheme? We'll also then want to add a LegacyHot color scheme that is what Hot does today.

Oh, and as in the other PR, could you please update the changelog at the end?

@asherkin
Copy link
Contributor Author

@jonhoo Apologies for the late response, I caught the virus that's been going around and have been out of action for a few weeks.

Would love to make this the default! I think the only concern I'd have with that myself is the massively reduced colour range of "red" used here compared to the "hot" palette. I only used these colours as they're what the java one was using, how would you feel about me crafting specifically a new default using these annotations? I'm thinking "hot" for the default (so the output will be the same as currently for anything not-annotated), something muted like greys for the kernel frames, and something bright (probably stick with the green/blue) for the jit/inline frames?

@jonhoo
Copy link
Owner

jonhoo commented Jan 15, 2021

Ouch, I'm sorry to hear that. Glad to have you back with us!

Absolutely, that sounds like an excellent idea 🎉

@asherkin
Copy link
Contributor Author

asherkin commented Feb 27, 2021

I've implemented the proposed new colour scheme and I think it's looking good. The default has changed to be the new annotated palette - let me know if you would prefer that to still be called "hot" from a CLI perspective and I'll make that change, right now I've kept it as its own new thing (internally it'll never be totally straightforward due to BasicPalette vs MultiPalette anyway). I'll update the changelog once that's cleared up.

I could do with some pointers regarding where you'd like documentation related to the annotation behaviour - the adding of annotations and the details around that are handled (and documented) on the stackcollapse side, so I'm not totally sure what's wanted over here on the flamegraph end.

EDIT: Whoops, need to sort the tests.

image

@asherkin asherkin force-pushed the generic-annotated-palette branch from 1dae0aa to 6d0fa59 Compare February 27, 2021 00:33
@asherkin asherkin force-pushed the generic-annotated-palette branch from 6d0fa59 to 16e2494 Compare February 27, 2021 00:34
@jonhoo
Copy link
Owner

jonhoo commented Feb 27, 2021

Nice! How about we call the old palette legacy, and leave the new one as hot?

As for documentation, I think the thing to do is mention in the stack collapsing docs how we detect JIT/kernel things, and then in the flamegraph docs how we expect names to be annotated in order to highlight them as JIT/kernel. How does that sound? I think we'll likely end up with a full API makeover (including docs) at some point in the future when we land a proper API as well 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants