Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Qualification tool support recognizing decimal operations #2928

Merged
merged 19 commits into from
Jul 15, 2021

Conversation

tgravescs
Copy link
Collaborator

This adds support for the qualification tool trying to recognize decimal operations and adds them to the potential problems columns. It also adds any sql ops with decimals to the sql duration for problematic.

This fixes a couple of issues found during testing, like if the sql start event is missing but end event was there, we were setting time to 0 rather than the sql end time.

Output now looks like:

csv:

App Name,App ID,Score,Potential Problems,SQL Dataframe Duration,App Duration,Executor CPU Time Percent,App Duration Estimated,SQL Duration with Potential Problems,SQL Ids with Failures
Spark shell,local-1626199951158,0.65,**UDF:DECIMAL**,1421,219771,60.49,false,**1121**,""

specifically note the Potential Problems and SQL Duration with Potential Problems

@tgravescs tgravescs added this to the July 5 - July 16 milestone Jul 14, 2021
@tgravescs tgravescs self-assigned this Jul 14, 2021
@tgravescs
Copy link
Collaborator Author

build

@tgravescs
Copy link
Collaborator Author

build

nartal1
nartal1 previously approved these changes Jul 15, 2021
@nartal1
Copy link
Collaborator

nartal1 commented Jul 15, 2021

LGTM. Please resolve conflicts.

@tgravescs
Copy link
Collaborator Author

build

@tgravescs tgravescs merged commit c8cc36a into NVIDIA:branch-21.08 Jul 15, 2021
@tgravescs tgravescs deleted the decimaldatatype branch July 15, 2021 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants