Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document all public items in rustc_incremental #90407

Merged
merged 1 commit into from
Dec 11, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions compiler/rustc_incremental/src/assert_dep_graph.rs
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ use std::env;
use std::fs::{self, File};
use std::io::{BufWriter, Write};

#[allow(missing_docs)]
pub fn assert_dep_graph(tcx: TyCtxt<'_>) {
tcx.dep_graph.with_ignore(|| {
if tcx.sess.opts.debugging_opts.dump_dep_graph {
Expand Down Expand Up @@ -262,6 +263,7 @@ fn dump_graph(query: &DepGraphQuery) {
}
}

#[allow(missing_docs)]
pub struct GraphvizDepGraph<'q>(FxHashSet<&'q DepNode>, Vec<(&'q DepNode, &'q DepNode)>);

impl<'a, 'q> dot::GraphWalk<'a> for GraphvizDepGraph<'q> {
Expand Down
1 change: 1 addition & 0 deletions compiler/rustc_incremental/src/assert_module_sources.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ use rustc_session::cgu_reuse_tracker::*;
use rustc_span::symbol::{sym, Symbol};
use std::collections::BTreeSet;

#[allow(missing_docs)]
pub fn assert_module_sources(tcx: TyCtxt<'_>) {
tcx.dep_graph.with_ignore(|| {
if tcx.sess.opts.incremental.is_none() {
Expand Down
1 change: 1 addition & 0 deletions compiler/rustc_incremental/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
//! Support for serializing the dep-graph and reloading it.

#![deny(missing_docs)]
#![doc(html_root_url = "https://doc.rust-lang.org/nightly/nightly-rustc/")]
#![feature(in_band_lifetimes)]
#![feature(let_else)]
Expand Down
34 changes: 26 additions & 8 deletions compiler/rustc_incremental/src/persist/fs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -133,21 +133,26 @@ const QUERY_CACHE_FILENAME: &str = "query-cache.bin";
// case-sensitive (as opposed to base64, for example).
const INT_ENCODE_BASE: usize = base_n::CASE_INSENSITIVE;

/// Returns the path to a session's dependency graph.
pub fn dep_graph_path(sess: &Session) -> PathBuf {
in_incr_comp_dir_sess(sess, DEP_GRAPH_FILENAME)
}
/// Returns the path to a session's staging dependency graph.
///
/// On the difference between dep-graph and staging dep-graph,
/// see `build_dep_graph`.
pub fn staging_dep_graph_path(sess: &Session) -> PathBuf {
in_incr_comp_dir_sess(sess, STAGING_DEP_GRAPH_FILENAME)
}

pub fn work_products_path(sess: &Session) -> PathBuf {
in_incr_comp_dir_sess(sess, WORK_PRODUCTS_FILENAME)
}

/// Returns the path to a session's query cache.
pub fn query_cache_path(sess: &Session) -> PathBuf {
in_incr_comp_dir_sess(sess, QUERY_CACHE_FILENAME)
}

/// Locks a given session directory.
pub fn lock_file_path(session_dir: &Path) -> PathBuf {
let crate_dir = session_dir.parent().unwrap();

Expand All @@ -166,23 +171,35 @@ pub fn lock_file_path(session_dir: &Path) -> PathBuf {
crate_dir.join(&directory_name[0..dash_indices[2]]).with_extension(&LOCK_FILE_EXT[1..])
}

/// Returns the path for a given filename within the incremental compilation directory
/// in the current session.
pub fn in_incr_comp_dir_sess(sess: &Session, file_name: &str) -> PathBuf {
in_incr_comp_dir(&sess.incr_comp_session_dir(), file_name)
}

/// Returns the path for a given filename within the incremental compilation directory,
/// not necessarily from the current session.
///
/// To ensure the file is part of the current session, use [`in_incr_comp_dir_sess`].
pub fn in_incr_comp_dir(incr_comp_session_dir: &Path, file_name: &str) -> PathBuf {
incr_comp_session_dir.join(file_name)
}

/// Allocates the private session directory. The boolean in the Ok() result
/// indicates whether we should try loading a dep graph from the successfully
/// initialized directory, or not.
/// The post-condition of this fn is that we have a valid incremental
/// compilation session directory, if the result is `Ok`. A valid session
/// Allocates the private session directory.
///
/// If the result of this function is `Ok`, we have a valid incremental
/// compilation session directory. A valid session
/// directory is one that contains a locked lock file. It may or may not contain
/// a dep-graph and work products from a previous session.
/// If the call fails, the fn may leave behind an invalid session directory.
///
/// This always attempts to load a dep-graph from the directory.
/// If loading fails for some reason, we fallback to a disabled `DepGraph`.
/// See [`rustc_interface::queries::dep_graph`].
///
/// If this function returns an error, it may leave behind an invalid session directory.
/// The garbage collection will take care of it.
///
/// [`rustc_interface::queries::dep_graph`]: ../../rustc_interface/struct.Queries.html#structfield.dep_graph
pub fn prepare_session_directory(
sess: &Session,
crate_name: &str,
Expand Down Expand Up @@ -661,6 +678,7 @@ fn is_old_enough_to_be_collected(timestamp: SystemTime) -> bool {
timestamp < SystemTime::now() - Duration::from_secs(10)
}

/// Runs garbage collection for the current session.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It garbage collects all incremental session directories except the current session, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I'll double check.

Copy link
Member Author

@pierwill pierwill Oct 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The garbage collection logic is complex enough here that I wouldn't feel comfortable discussing its behavior in the public doc comment without input from someone familiar with this code. (Although I suspect you are right!)

pub fn garbage_collect_session_directories(sess: &Session) -> io::Result<()> {
debug!("garbage_collect_session_directories() - begin");

Expand Down
17 changes: 15 additions & 2 deletions compiler/rustc_incremental/src/persist/load.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,24 @@ use super::work_product;
type WorkProductMap = FxHashMap<WorkProductId, WorkProduct>;

#[derive(Debug)]
/// Represents the result of an attempt to load incremental compilation data.
pub enum LoadResult<T> {
Ok { data: T },
/// Loading was successful.
Ok {
#[allow(missing_docs)]
data: T,
},
/// The file either didn't exist or was produced by an incompatible compiler version.
DataOutOfDate,
Error { message: String },
/// An error occured.
Error {
#[allow(missing_docs)]
message: String,
},
}

impl<T: Default> LoadResult<T> {
/// Accesses the data returned in [`LoadResult::Ok`].
pub fn open(self, sess: &Session) -> T {
// Check for errors when using `-Zassert-incremental-state`
match (sess.opts.assert_incr_state, &self) {
Expand Down Expand Up @@ -99,6 +110,7 @@ pub enum MaybeAsync<T> {
}

impl<T> MaybeAsync<LoadResult<T>> {
/// Accesses the data returned in [`LoadResult::Ok`] in an asynchronous way if possible.
pub fn open(self) -> LoadResult<T> {
match self {
MaybeAsync::Sync(result) => result,
Expand All @@ -109,6 +121,7 @@ impl<T> MaybeAsync<LoadResult<T>> {
}
}

/// An asynchronous type for computing the dependency graph.
pub type DepGraphFuture = MaybeAsync<LoadResult<(SerializedDepGraph, WorkProductMap)>>;

/// Launch a thread and load the dependency graph in the background.
Expand Down
15 changes: 13 additions & 2 deletions compiler/rustc_incremental/src/persist/save.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,13 @@ use super::file_format;
use super::fs::*;
use super::work_product;

/// Save and dump the DepGraph.
/// Saves and writes the [`DepGraph`] to the file system.
pierwill marked this conversation as resolved.
Show resolved Hide resolved
///
/// No query must be invoked after this function.
/// This function saves both the dep-graph and the query result cache,
/// and drops the result cache.
///
/// This function should only run after all queries have completed.
/// Trying to execute a query afterwards would attempt to read the result cache we just dropped.
pub fn save_dep_graph(tcx: TyCtxt<'_>) {
debug!("save_dep_graph()");
tcx.dep_graph.with_ignore(|| {
Expand Down Expand Up @@ -75,6 +79,7 @@ pub fn save_dep_graph(tcx: TyCtxt<'_>) {
})
}

/// Saves the work product index.
pub fn save_work_product_index(
sess: &Session,
dep_graph: &DepGraph,
Expand Down Expand Up @@ -139,6 +144,12 @@ fn encode_query_cache(tcx: TyCtxt<'_>, encoder: &mut FileEncoder) -> FileEncodeR
tcx.sess.time("incr_comp_serialize_result_cache", || tcx.serialize_query_result_cache(encoder))
}

/// Builds the dependency graph.
pierwill marked this conversation as resolved.
Show resolved Hide resolved
///
/// This function breates the *staging dep-graph*. When the dep-graph is modified by a query
/// execution, the new dependency information is not kept in memory but directly
/// output to this file. `save_dep_graph` then finalizes the staging dep-graph
/// and moves it to the permanent dep-graph path
pub fn build_dep_graph(
sess: &Session,
prev_graph: SerializedDepGraph,
Expand Down
6 changes: 5 additions & 1 deletion compiler/rustc_incremental/src/persist/work_product.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
//! This module contains files for saving intermediate work-products.
//! Functions for saving and removing intermediate [work products].
//!
//! [work products]: WorkProduct

use crate::persist::fs::*;
use rustc_fs_util::link_or_copy;
Expand All @@ -7,6 +9,7 @@ use rustc_session::Session;
use std::fs as std_fs;
use std::path::PathBuf;

/// Copies a CGU work product to the incremental compilation directory, so next compilation can find and reuse it.
pub fn copy_cgu_workproduct_to_incr_comp_cache_dir(
sess: &Session,
cgu_name: &str,
Expand Down Expand Up @@ -40,6 +43,7 @@ pub fn copy_cgu_workproduct_to_incr_comp_cache_dir(
Some((work_product_id, work_product))
}

/// Removes files for a given work product.
pub fn delete_workproduct_files(sess: &Session, work_product: &WorkProduct) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is never used outside the crate.

if let Some(ref file_name) = work_product.saved_file {
let path = in_incr_comp_dir_sess(sess, file_name);
Expand Down