-
Notifications
You must be signed in to change notification settings - Fork 240
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Qualification tool: Parsing Execs to get the ExecInfo #2 #5426
Conversation
Signed-off-by: Niranjan Artal <nartal@nvidia.com>
build |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why are we adding a global_local_limit_eventlog, I thought those execs were already in the nds_q86_test eventlog? I would much prefer just to reuse an existing log
tools/src/test/scala/com/nvidia/spark/rapids/tool/planparser/SqlPlanParserSuite.scala
Outdated
Show resolved
Hide resolved
Signed-off-by: Niranjan Artal <nartal@nvidia.com>
GlobalLimit and LocalLimit are in LogicalPlan and not in PhysicalPlan of nds_q86_test eventlog. So I couldn't use that eventlog for testing those execs. |
build |
1 similar comment
build |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please compress ./tools/src/test/resources/spark-events-qualification/global_local_limit_eventlog using zstd and update path in test
Signed-off-by: Niranjan Artal <nartal@nvidia.com>
build |
This fixes #5361. It is a follow on PR of #5420.
Execs supported in this PR:
Have added eventlog for testing GlobalLimit and LocalLimitExec. These execs are not present in final physical plan if
collect
is called. So the eventlog is generated usingshow
(df.limit(10).show). For rest of the execs, we create eventlogs on thr fly in the test itself.