Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assert for invariants in tests and debug builds #594

Open
houqp opened this issue Jun 20, 2021 · 0 comments
Open

Assert for invariants in tests and debug builds #594

houqp opened this issue Jun 20, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@houqp
Copy link
Member

houqp commented Jun 20, 2021

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Our invariants are formally documented in https://github.com/apache/arrow-datafusion/blob/master/docs/specification/invariants.md, but it's not being fully enforced across the board. During the implementation of #55, I have found couple bugs in our existing code base that could have been prevented if check for invariants more aggressively during development and in CI.

Describe the solution you'd like

One thing we could do is to insert conditional compilation asserts for invariants in places that could break the invariants. For example:

  • add assert to compare logical plan schema before and after each optimizer run
  • add assert to compare fields between logical plan and physical plan after physical planing
  • add assert to make sure physical plan only deals with unqualified fields, for example: ParquetExec::try_from_path

These asserts should be turned of in production builds.

@houqp houqp added the enhancement New feature or request label Jun 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant