How to determine whether a file is indexed by Rust Analyzer

[davidbarsky]

(While this is marked as a feature request, it's more of a design document/me laying my thoughts out as how to tackle this problem. Thoughts, if any, would be appreciated)

In my employer's monorepo, people are generally using rust-analyzer pretty successfully, but run into issues when they navigate/open a Rust file that isn't part of the workspace that rust-analyzer indexed, which results in—unsurprisingly—a lack of IDE functionality. That's not great, and people don't necessarily know why things aren't working. I'd like to fix this problem by detecting whether the file they've opened is/isn't indexed by rust-analyzer, and if it's not, asking if they'd like that file to be added to their workspace via our enterprisey companion server.

My initial attempt consisted changing handle_did_open_text_document to the following (as an experiment, not as a proper solution):

pub(crate) fn handle_did_open_text_document(
    state: &mut GlobalState,
    params: DidOpenTextDocumentParams,
) -> anyhow::Result<()> {
    let _p = profile::span("handle_did_open_text_document");

    if let Ok(path) = from_proto::vfs_path(&params.text_document.uri) {
        let already_exists = state
            .mem_docs
            .insert(path.clone(), DocumentData::new(params.text_document.version))
            .is_err();
        if already_exists {
            tracing::error!("duplicate DidOpenTextDocument: {}", path);
        }
        state
            .vfs
            .write()
            .0
            .set_file_contents(path.clone(), Some(params.text_document.text.into_bytes()));

        if state.is_quiescent() {
            if let Some(id) = state.vfs.read().0.file_id(&path) {
                if let Ok(crates) = state.snapshot().analysis.crates_for(id) {
                    if crates.is_empty() {
                        tracing::error!("rust-analyzer does not track this file")
                    } else {
                        tracing::error!("file exists in VFS!")
                    }
                }
            }
        }
    }
    Ok(())
}

This, unfortunately, results in panics coming inside of Salsa along the lines of:

thread 'LspServer' panicked at /Users/dbarsky/.cargo/registry/src/index.crates.io-6f17d22bba15001f/salsa-0.17.0-pre.2/src/input.rs:106:32:
no value set for FileSourceRootQuery(FileId(4412))
stack backtrace:
   0: rust_begin_unwind
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:595:5
   1: core::panicking::panic_fmt
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/panicking.rs:67:14
   2: <salsa::input::InputStorage<Q> as salsa::plumbing::QueryStorageOps<Q>>::try_fetch
   3: <DB as base_db::SourceDatabaseExt>::file_source_root::__shim
   4: <base_db::FileLoaderDelegate<&T> as base_db::FileLoader>::relevant_crates
   5: <ide_db::RootDatabase as base_db::FileLoader>::relevant_crates
   6: ide::parent_module::crates_for
   7: salsa::Cancelled::catch
   8: ide::Analysis::crates_for
   9: rust_analyzer::handlers::notification::handle_did_open_text_document
  10: rust_analyzer::dispatch::NotificationDispatcher::on_sync_mut
  11: rust_analyzer::main_loop::<impl rust_analyzer::global_state::GlobalState>::run
  12: rust_analyzer::main_loop::main_loop
  13: rust_analyzer::run_server
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

This particular panic occurred after rust-analyzer already indexed a workspace and I navigated to crate I knew wasn't indexed by it. Despite following in the footsteps of how handle_analyzer_status command works, it seems that it's not possible to read from the Salsa databases immediately after writing to the VFS. Here are my questions:

1. Can it be possible to read a Salsa database immediately after writing to the VFS?
2. Even if it were possible, is it a good idea to do this check inside of handle_did_open_text_document?

If the answer to any of the above is no, then the approach that I'll probably take is the following:

1. In the workspaces map in our companion server, store all source files for a given workspace.
2. In a handle_did_open_text_document, we'll check if that file is in the cratedb, and if it's not, do a buck2 uquery "owner(opened_file.rs)" (we can typically get a response back in ~200 milliseconds).
   • If the response indicates that it's part of a crate that we previously indexed, we'll add it our workspaces map. This means that the file was recently created by the user and rust-analyzer knows about it, but the companion server does not.
     • The alternative would be for rust-analyzer to emit its crate graph/deltas and the companion server can consume it. I don't know if this is strictly necessary, though, but if the performance of querying buck isn't acceptable, then I'll probably come back and open an issue asking for this capability.
   • If the response indicates it's part of a crate that wasn't indexed, we'll kick off a rust-project.json generation and reload rust-analyzer, so that new file is now indexed.

[lnicola]

Can it be possible to read a Salsa database immediately after writing to the VFS?

The VFS records a number of changes, returns them from Vfs::take_changes, then those get passed as inputs to salsa (via GlobalState::process_changes, Change::apply and set_file_text_with_durability). So between Vfs::set_file_contents and set_file_text_with_durability, the changes aren't visible.

I think fn load_crate_graph is one place where we force this to happen in a single step.

[davidbarsky]

Ah, thanks for the help. Setting the file and calling state.process_changes was enough for me to try out the thing I wanted to do successfully!