Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization tracking issue #339

Open
15 of 20 tasks
tomwhite opened this issue Dec 19, 2023 · 2 comments
Open
15 of 20 tasks

Optimization tracking issue #339

tomwhite opened this issue Dec 19, 2023 · 2 comments

Comments

@tomwhite
Copy link
Member

tomwhite commented Dec 19, 2023

This is an umbrella issue for tracking the work on optimizations in Cubed.

Creation optimizations

Making creation operations more efficient, typically by not materializing unnecessary data.

Fusion optimizations

Currenly we fuse map blocks operations with one input, but there are more types of fusion we could implement.

Reduction optimizations

Reduction operations like sum and mean could be improved by minimising the amount of data transferred.

High-level query optimizations

Re-writing array expressions to an optimized form (before applying the optimizations above).

Benchmarking and runtime

Testing the effect of the optimizations above.

Documentation

@TomNicholas
Copy link
Member

Do any of the caveats on the Scaling docs page need updating after all these improvements? e.g. it currently says

In theory multiple blockwise operations can be fused together, enhancing the performance further. However this has not yet been implemented in Cubed.

@tomwhite
Copy link
Member Author

Do any of the caveats on the Scaling docs page need updating after all these improvements? e.g. it currently says

In theory multiple blockwise operations can be fused together, enhancing the performance further. However this has not yet been implemented in Cubed.

Yes, thanks for raising this. I've opened #381 to track documentation changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants