[FEAT] Ray shuffle experiment

[jaychia]

No description provided.

[codspeed-hq]

CodSpeed Performance Report

Merging #2883 will not alter performance

_{Comparing jay/shuffle-experiment (ade5e2f) with main (768f422)}

Summary

✅ 17 untouched benchmarks

[codecov]

Codecov Report

Attention: Patch coverage is 24.76636% with 322 lines in your changes missing coverage. Please review.

Project coverage is 78.39%. Comparing base (868bbfa) to head (37b1d8e).

Files with missing lines	Patch %	Lines
daft/runners/ray_runner.py	18.68%	161 Missing ⚠️
daft/execution/physical_plan.py	3.50%	55 Missing ⚠️
src/daft-scheduler/src/scheduler.rs	0.00%	34 Missing ⚠️
src/daft-plan/src/physical_planner/translate.rs	57.77%	19 Missing ⚠️
src/daft-plan/src/physical_ops/exchange_op.rs	0.00%	16 Missing ⚠️
daft/execution/physical_plan_shuffles.py	74.46%	12 Missing ⚠️
...sical_optimization/rules/reorder_partition_keys.rs	0.00%	12 Missing ⚠️
src/daft-plan/src/physical_plan.rs	0.00%	9 Missing ⚠️
daft/context.py	33.33%	2 Missing ⚠️
daft/runners/pyrunner.py	66.66%	1 Missing ⚠️
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2883      +/-   ##
==========================================
- Coverage   78.71%   78.39%   -0.33%     
==========================================
  Files         597      599       +2     
  Lines       69735    70128     +393     
==========================================
+ Hits        54895    54977      +82     
- Misses      14840    15151     +311     

Flag	Coverage Δ
	78.39% <24.76%> (-0.33%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
daft/runners/runner.py	73.52% <100.00%> (+1.65%)	⬆️
src/daft-plan/src/physical_ops/mod.rs	0.00% <ø> (ø)
daft/runners/pyrunner.py	85.07% <66.66%> (-0.18%)	⬇️
src/daft-plan/src/display.rs	80.00% <0.00%> (-0.33%)	⬇️
daft/context.py	78.61% <33.33%> (+1.05%)	⬆️
src/daft-plan/src/physical_plan.rs	51.21% <0.00%> (-1.38%)	⬇️
daft/execution/physical_plan_shuffles.py	74.46% <74.46%> (ø)
...sical_optimization/rules/reorder_partition_keys.rs	75.25% <0.00%> (-4.97%)	⬇️
src/daft-plan/src/physical_ops/exchange_op.rs	0.00% <0.00%> (ø)
src/daft-plan/src/physical_planner/translate.rs	91.97% <57.77%> (-1.88%)	⬇️
... and 3 more

... and 5 files with indirect coverage changes