-
Notifications
You must be signed in to change notification settings - Fork 155
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Serialization to disk #10
Comments
A similar, but different feature is to allow to transparently spill rarely used values to disk. IntelliJ relies on similar feature heavily: when you open a multi-million line project with lots of dependencies, indices become really huge. Note that this is a significantly different setup from rustc, which operates on a crate at a time, and has a reasonable natural cap on the amount of data it must process simultaneously. |
A very wise observation from: rust-lang/rfcs#1317 (comment) In a strictly on-demand setting (IDE, not a compiler), serialization to disk creates more problems than it solves. |
Note that some popular IDEs like Visual Studio actually use a disk database. VS migrated a while ago from a custom format to a SQLite database: https://devblogs.microsoft.com/cppblog/introducing-c-experimental-editor-tools/. |
Hi! This would be a desirable feature for me. Is this being worked on? Not trying to rush you, just trying to evaluate how suitable this library is for my use-case. Thank you. :) |
No, this is not being actively worked on at the moment.
…On Thu, 5 Mar 2020 at 16:38, Louis Pilfold ***@***.***> wrote:
Hi! This would be a desirable feature for me. Is this being worked on?
Not trying to rush you, just trying to evaluate how suitable this library
is for my use-case. Thank you. :)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#10?email_source=notifications&email_token=AANB3M3YCOSNRH5FCEPGGTLRF7BPRA5CNFSM4FYGX55KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEN5XSAQ#issuecomment-595294466>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AANB3M56WYJNTWGXN5QGE73RF7BPRANCNFSM4FYGX55A>
.
|
Thank you |
I think serialization should be generally opt-in:
I think I already have a kind of usage scenario ("scenario" as in "salsa is currently not used, but I investigate potential usages") in Click to expandThat program first walks through a file list and computes the hash of each file. Then it iterates over the list of hashes, takes the first associated file, and calls a user-defined hook script on that file. It caches the hash list and the progress. It might not really fit the usual `salsa` usage scenario, but the target is similiar: avoid redoing work.
Currently, I just take the "session serialization approach", deserialize at startup, and serialize at shutdown/interrupt, but this may lose some progress. I think that the |
Thanks for creating salsa. It's an outstanding piece of software and an extremely valuable inspiration resource. We're exploring adding incremental computation to Ruff, a static analysis tool for Python that is preliminary used from the CLI but also comes with an LSP. We're intrigued by salsa's model. It's nice how it handles much of the caching complexity for you. However, we believe that a persistent cache is essential for us because subsequent check times are important when using the CLI locally or in CI. That's how I came across this issue. Is this a feature where active contributions would be welcomed? Are there ides on how this could be implemented in Salsa 2022 that I could explore further? |
We should support some way to serialize the state of our queries to disk and then reload them for a future session. This is a lot of work and we can learn from rustc, of course. We'd want to do the reloading lazilly, for example.
I definitely want to punt on this.
The text was updated successfully, but these errors were encountered: