Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Determine if any limits we can place on AST size #27

Open
overlookmotel opened this issue May 29, 2024 · 1 comment
Open

Determine if any limits we can place on AST size #27

overlookmotel opened this issue May 29, 2024 · 1 comment

Comments

@overlookmotel
Copy link

overlookmotel commented May 29, 2024

I asked for opinions on Discord about whether we could place any limits on max AST size in order to limit size of arena to 4 GiB.

Upshot was definitely not in all cases, but maybe in majority. It needs research.

As starting point, Boshen suggested parsing all of https://github.com/elastic/kibana and measuring AST size. Do that!

@overlookmotel
Copy link
Author

overlookmotel commented Jul 13, 2024

Maybe 32 GiB is doable as a maximum, and "pointers" could still be stored in 32 bits. Because all AST types are aligned on 8, bottom 3 bits of pointers are always 0. So you can address up to 32 GiB with a u32. ptr = arena_base_ptr.add(u32_index << 3).

Edit: Actually, if "pointers" were u32s, most AST types would be aligned on 4, not 8. So then only 2 spare bottom bits = 16 GiB max arena size. Maybe that's still viable. Obviously with reduced pointer size, AST would take significantly less memory. So less memory to play with, but you can fit more into each GiB.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant