diff --git a/rust/.gitignore b/rust/.gitignore index 9740f634a..bcc52fede 100644 --- a/rust/.gitignore +++ b/rust/.gitignore @@ -1,2 +1,2 @@ *~ -yarn-error.log +target/ diff --git a/rust/CONTRIBUTING.html b/rust/CONTRIBUTING.html new file mode 120000 index 000000000..64233a9e9 --- /dev/null +++ b/rust/CONTRIBUTING.html @@ -0,0 +1 @@ +index.html \ No newline at end of file diff --git a/rust/CONTRIBUTING.md b/rust/CONTRIBUTING.md new file mode 100644 index 000000000..de44526ca --- /dev/null +++ b/rust/CONTRIBUTING.md @@ -0,0 +1,92 @@ +TODO organize this + +Every md file has a html relative symlink to index.html: + +`ln -s index.html CONTRIBUTING.html` + +The static website loads the page content from markdown dynamically, +based on the URL of its HTML page. + +Project text doesn't link directly to documentation resources containing +solutions - students should learn where to get the answers from the +pre-requisites and lessons. + +Project text may include inline links to pages that offer explanatios of terms +and concepts. + +Keep the project summary (the **Task**, **Goals**, etc. text) synced between +plan.md and the project description. + +Common sections of project descriptions: + +- The project summary (directly after h1) +- "Introduction" +- "Project spec" + +Headers do not capitalize every word; words after a colon +are capitalized: + +> ## Project spec + +> ## Lesson: Tools and good bones + +In projects, be clear on when the student should start hacking, and what they +should be hacking, by writing an imperative statement. Format that command in +italics: + +> Use [crates.io](https://crates.io) to find the documentation +for the `clap` crate, and implement the command line interface +such that the `cli_*` test cases pass. + +> _Try it now._ + +## Developing a new section + +Each section provides a similar project to previous sections, while expanding +the scope and building off of learnings from previous sections. Each project +lends itself to being extended by the next section's project. + +New sections are developed in pairs, to benefit from the feedback. + +Approximate workflow: + +- Author 1 and 2 collaborate on an outline for the new section, adding it to + plan.md, following the format of existing sections. This includes supporting lessons + and readings. Submit PR. + +- Author 1 writes the entire project description; optionally with test cases if + that makes the writing easier. Add new background readings or otherwise modify + the section plan as needed. Submit PR. Author 2 reviews. + +- Author 2 writes the remaining test cases and an example solution project. Add + new background readings or otherwise modify the section plan as needed. Submit + PR. Author 1 reviews. + +- Author 2 makes changes to the project text for clarification, improvement, + scope expansion, etc. Add new background readings or otherwise modify the + section plan as needed. Submit PR. Author 1 reviews. + +## + +When writing a project, look for steps where the design could be specified differently, +where there are multiple solutions, where there is deeper understanding to be gained, +and ask questions formulated to get the reader to think more deeply. + +Projects should be written to take between 2 - 4 hours to implement. + +When writing a project, look for optional "extension" steps that teach +additional practical subjects, but which either aren't necessary to complete the +project or require more time and skill to implement. Extenion steps go at the +end of a project. + +## Contributor notes + +- each md file needs an html relative symlink to index.html +- lessons need a .slides.html symlink +- links are generally to markdown files, not html files + - exception: links to slides from plan go to the hosted website + - all links rewritten when browsed locally +- in markdown, write links as relative to current directory +- keep "task", "goals", etc in project intros in sync between "plan.md" and + project pages + diff --git a/rust/README.md b/rust/README.md index a7de9aa73..9ddd6a811 100644 --- a/rust/README.md +++ b/rust/README.md @@ -3,16 +3,14 @@ A training course on practical software construction in Rust. The final project in this course is a networked, persistent [key-value store] -with multithreading, asynchronous I/O (via [tokio] and [tower]), a -high-performance network protocol ([GRPC] via [prost] and [tower-grpc]), -backed by a simple [LSM-tree] that the student will write themselves. +with multithreading, asynchronous I/O (via [hyper]). Subjects covered include: - Structuring and maintaining Rust programs - Fun and foolproof parallel programming -- Asyncronous programming with [futures] and [tokio] -- High-performance networking with [GRPC] and [prost] +- Asyncronous programming with [futures] +- Networking with [hyper] - Benchmarking with [criterion] and [critcmp] - Robust and reliable error handling with [failure] - Serialization with [serde] @@ -119,7 +117,7 @@ recommended to teach this course. ## Contributing -TODO +See [CONTRIBUTING.md]. ## License @@ -148,4 +146,5 @@ TODO [Distributed Systems in Rust]: todo [The Rust Book]: https://doc.rust-lang.org/book/ [plan]: plan.md -[serde]: todo \ No newline at end of file +[serde]: todo +[CONTRIBUTING.md]: CONTRIBUTING.md diff --git a/rust/css/style.css b/rust/css/style.css index add94b2fa..a83814651 100644 --- a/rust/css/style.css +++ b/rust/css/style.css @@ -4,7 +4,7 @@ nav#site-nav { position: absolute; top: 0px; width: 100%; - background: #666666; + background: #555; color: white; padding: 1rem; line-height: 1.5rem @@ -20,8 +20,3 @@ nav#site-nav a:hover { color: orange; } -section#content { - margin: 1rem; - margin-top: 4.5rem; /* nav border * 2 + line-height + 1rem */ -} - diff --git a/rust/css/text.css b/rust/css/text.css new file mode 100644 index 000000000..a187f4b94 --- /dev/null +++ b/rust/css/text.css @@ -0,0 +1,20 @@ +/* Styles for non-slide pages */ + +section#content { + margin-top: 5.5rem; /* nav border * 2 + line-height + 2rem */ +} + +section#content { + padding-left: 100px; + padding-right: 100px; +} + +pre { + background-color: #555; + color: white; + padding: 1rem; +} + +body { + margin-bottom: 5rem; +} diff --git a/rust/js/common.js b/rust/js/common.js index b6b7bd48b..8d8fe6aaf 100644 --- a/rust/js/common.js +++ b/rust/js/common.js @@ -8,6 +8,8 @@ export function mdUrlFromUrl(url) { let mdUrl = url; if (url.indexOf("index.html") != -1) { mdUrl = url.replace("index.html", "README.md"); + } else if (url.endsWith("/")) { + mdUrl = url + "README.md"; } else if (url.indexOf(".slides.html") != -1) { mdUrl = url.replace(".slides.html", ".md"); } else if (url.indexOf(".html") != -1) { diff --git a/rust/js/init.js b/rust/js/init.js index 3f0087efe..0ac8fe8d4 100644 --- a/rust/js/init.js +++ b/rust/js/init.js @@ -25,6 +25,7 @@ function init() { contentElement: contentElement, }; + addCssForPageType(config); dispatchInitForPageType(config); } @@ -45,6 +46,19 @@ function getPageType(url) { return type; } +function addCssForPageType(config) { + if (config.pageType === "lesson-slides") { + return; + } + + let head = document.querySelector("head"); + console.log(head); + let link = document.createElement("link"); + link.setAttribute("rel", "stylesheet"); + link.setAttribute("href", "css/text.css"); + head.appendChild(link); +} + function dispatchInitForPageType(config) { let mainModule = null; if (config.pageType === "md") { diff --git a/rust/notes.md b/rust/notes.md index b500da506..c14479924 100644 --- a/rust/notes.md +++ b/rust/notes.md @@ -37,6 +37,9 @@ in the README. - Ana likes this - variable shadowing - DSTs +- configuring clippy / rustfmt +- scripting clippy / rustfmt for CI +- CI setup ## Sources @@ -51,20 +54,17 @@ in the README. ## Subjects to potentially cut -- Whirlwind rust lesson - Parallelism section - Formatting lesson - Build time lesson - Collections and iterators -## Contributor notes +## Grading -- each md file needs a relative symlink to index.html -- lessons need a .slides.html symlink -- links are generally to markdown files, not html files - - exception: links to slides from plan go to the hosted website - - all links rewritten when browsed locally -- in markdown, write links as relative to current directory +- automated text-based cheat detection +- automated grading of test cases +- automated grading of non-unit-test requirements via python script +- how to grade freeform answers? ## TODO @@ -72,4 +72,4 @@ in the README. - lessons and labs pose questions - have a chinese native identify and remove 'hard' words and phrases - mention somewhere that we're using rust 2018 only, how to verify -- change slides.html URLS to link to the hosted file \ No newline at end of file +- change slides.html URLS to link to the hosted file diff --git a/rust/plan.md b/rust/plan.md index 7c040b088..4957304c9 100644 --- a/rust/plan.md +++ b/rust/plan.md @@ -24,209 +24,240 @@ A suggested workflow: - if taking the course online, read the writeups that accompany the slides - Follow each project according to their own instructions, writing Rust programs that pass the projects' accompanying tests. -- (Optionally) Submit project for online grading via [TODO]. + +As you work through the course content, _please_ be on the lookout for things +you would improve about the course, and either [submit issues][si] explaining, +or [submit pull requests][spr] with improvements. _Each accepted pull request +counts toward extra credit during final evaluation_. Pull requests to any other +project used during this course count as well. This is an opportunity to gain +experience contributing to an open-source Rust project. Make this a better +course for the next student to take it than it was for you. That is how open +source projects evolve. + +[si]: https://github.com/pingcap/talent-plan/issues/new +[spr]: https://github.com/pingcap/talent-plan/compare ## Section 1 (Setup) -- [Project: Tools and good bones][p-tools] - - Task: create an in-memory key/value store that passes simple tests - and responds to command-line arguments. +### [Project: Tools and good bones][p-tools] + +**Task**: Create an in-memory key/value store that passes simple tests and responds +to command-line arguments. + +**Goals**: - - Goals: - - Get a toolchain and tools - - Use `cargo init / run / test` - - Use external crates - - Define a data type for a key-value store +- Install the Rust compiler and tools +- Learn the project structure used throughout this course +- Use `cargo build` / `run` / `test` / `clippy` / `fmt` +- Use external crates +- Define a data type for a key-value store - - Topics: clap, testing, CARGO_VERSION, clippy, rustfmt +**Topics**: clap, testing, `CARGO_VERSION`, clippy, rustfmt - - Extensions: structopt, log / slog +**Extensions**: `structopt`, `log` / `slog`, -- [Lesson: Whirlwind Rust][t-whirlwind] ([slides][s-whirlwind]) - - Readings: - - [The Book - Getting Started](https://doc.rust-lang.org/book/ch01-00-getting-started.html) - - [The Book - Programming a Guessing Game](https://doc.rust-lang.org/book/ch02-00-guessing-game-tutorial.html) - - [The Book - Common Programming Concepts](https://doc.rust-lang.org/book/ch03-00-common-programming-concepts.html) +### [Lesson: Data structures in Rust][t-data] ([slides][s-data]) - - Topics: Rust and cargo, ownership & borrowing / aliasing & mutability in - brief, resources (TODO: what else?) +**Topics**: when to use which struct types, impls, ctor patterns, dtors, reprs, +padding demo, packed structs, size and alignment in depth, enum +implementation and optimizations, -- Lesson: Data structures in Rust - - Topics: when to use which struct types, impls, ctor patterns, dtors, reprs, - padding demo, packed structs, size and alignment in depth, enum - implementation and optimizations, +### [Lesson: Crates and crates.io][t-crates] ([slides][s-crates]) -- Lesson: Crates and crates.io +**Topics**: importing crates, features, debugging and fixing dependencies, +std vs crate philosophy and history, finding crates - - Topics: importing crates, features, debugging and fixing dependencies, - std vs crate philosophy and history, finding crates -- Lesson: Rust tooling +### [Lesson: Rust tooling][t-tools] ([slides][s-tools]) - - Topics: `#[test]`, how does test work under the hood?, what does 'cargo run' - actually do?, clippy, rustfmt, controlling clippy and rustfmt, links to - other useful tools, cargo / rustc wrapping pattern in depth (ex rustup, - RUSTC_WRAPPER) +**Readings**: + - [`clippy` README](https://github.com/rust-lang/rust-clippy/blob/master/README.md) + - [`rustfmt` README](https://github.com/rust-lang/rustfmt/blob/master/README.md) -- Lesson: Formatting, println et. al, log, and slog +**Topics**: `#[test]`, how does test work?, what does `cargo run` actually do?, +clippy, rustfmt, controlling clippy and rustfmt, links to other useful tools, +cargo / rustc wrapping pattern in depth (ex rustup, `RUSTC_WRAPPER`) - - Readings: - - [The Book - Macros](https://doc.rust-lang.org/book/ch19-06-macros.html) - - [`std::fmt`](https://doc.rust-lang.org/std/fmt/index.html) - - Topics: formatting tips, derive Debug in depth, how does format! work?, log - demo, slog and structured logging, env_logger, APIs that take format - strings, +### [Lesson: Formatting, println et. al, log, and slog][t-fmt] ([slides][s-fmt]) - - TODO: This subject isn't _needed_ to do the project, but formatting - is a good deep-dive opportunity +**Readings**: + - [The Book - Macros](https://doc.rust-lang.org/book/ch19-06-macros.html) + - [`std::fmt`](https://doc.rust-lang.org/std/fmt/index.html) +**Topics**: formatting tips, derive Debug in depth, how does `format!` work?, +`log` demo, `slog` and structured logging, `env_logger`, APIs that take format +strings ## Section 2 (File I/O) -- Project: File I/O - - Task: create a persistent key/value store that can be accessed from the - command line +### [Project: File I/O][p-fs] + +**Task**: Create a persistent key/value store that can be accessed from the +command line + +**Goals**: + +- Handle and report errors robustly +- Write data to disk as a write-ahead log +- Read the state of the key/value store from disk +- Use serde for serialization +- Use file I/O APIs to binary search + +**Topics**: `failure` crate, `std::net::fs`, `Read` / `Write` traits, +serde - - Goals: - - Handle and report errors robustly - - Accept command line arguments - - Write data to disk as a write-ahead log - - Read the state of the key/value store from disk - - Use serde for serialization - - Maintain consistency after crashes or data corruption? - - Use file I/O APIs to binary search +**Extensions**: range queries, store data using bitcast algo? - - Extensions: range queries, convert full WAL to binary-searchable file +### [Lesson: Proper error handling][t-errors] ([slides][s-errors]) -- Lesson: Proper error handling +**Readings**: - - Readings: - - [The Book - Error Handling](https://doc.rust-lang.org/book/ch09-00-error-handling.html) - - [Rust Error Handling](http://blog.burntsushi.net/rust-error-handling/) - - [The `failure` book](https://rust-lang-nursery.github.io/failure/) +- [The Book - Error Handling](https://doc.rust-lang.org/book/ch09-00-error-handling.html) +- [Rust Error Handling](http://blog.burntsushi.net/rust-error-handling/) +- [The `failure` book](https://rust-lang-nursery.github.io/failure/) - - Topics: TODO, `fn main() -> Result` +**Topics**: error patterns, `failure` crate, `fn main() -> Result`, `panic!` and +unwinding -- Lesson: Collections and iterators +### [Lesson: Collections and iterators][t-coll] ([slides][s-coll]) ## Section 3 (Networking) -- Project: Networking - - Task: create a single-threaded, persistent key/value store server and client - with synchronous networking over the HTTP and GRPC protocols. +### [Project: Networking][p-net] -- Lesson: Basic network APIs +**Task**: Create a single-threaded, persistent key/value store server and client +with synchronous networking over the HTTP protocol. - - Topics: std networking, TCP vs UDP?, reqwest, blocking HTTP serving w/ Iron/Hyper? +**Goals**: -- Lesson: Build-time Rust +- Use hyper for synchronous networking - - Topics: build scripts, protobuf compilation example, getting rustc version - and other useful info (TODO crates for this?), in-depth examples of crates - that rely on build scripts (e.g. skeptic, TODO), (TODO what else?) -- Lesson: GPRC and prost +### [Lesson: Basic network APIs][t-net] ([slides][s-net]) + +**Topics**: `std` networking, TCP vs UDP, `reqwest`, blocking HTTP serving w/ Iron + + +### [Lesson: Build-time Rust][t-build] ([slides][s-build]) + +**Topics**: build scripts, protobuf compilation example, getting rustc version +and other useful info, in-depth examples of crates that rely on build scripts ## Section 4 (Parallelism) -- Project: Parallelism - - Task: create a multi-threaded, persistent key/value store server and client - with synchronous networking via the GRPC protocol. +### [Project: Parallelism][p-par] + +**Task**: Create a multi-threaded, persistent key/value store server and client +with synchronous networking via HTTP. + +**Goals**: + +- Write a simple thread-pool +- Use crossbeam channels +- Benchmark single-threaded vs multi-threaded - - Goals: - - TODO - - Benchmark single-threaded vs multi-threaded -- Lesson: The big problem — aliasing and mutability +### [Lesson: The big problem — aliasing and mutability][t-alias] ([slides][s-alias]) - - Readings: - - [The Book - Understanding Ownership](https://doc.rust-lang.org/book/ch04-00-understanding-ownership.html) - - [The Nomicon - Aliasing](https://doc.rust-lang.org/nomicon/aliasing.html) - - [The Problem with Single-threaded Shared Mutability](https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-mutability) +**Readings**: - - Topics: mutable aliasing bugs, how ownership prevents mutable aliasing, Sync - / Sync, uniq / shared vs immutable / mutable, Rc and Arc, interior - mutability in depth, +- [The Book - Understanding Ownership](https://doc.rust-lang.org/book/ch04-00-understanding-ownership.html) +- [The Nomicon - Aliasing](https://doc.rust-lang.org/nomicon/aliasing.html) +- [The Problem with Single-threaded Shared Mutability](https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-mutability) -- Lesson: Ownership and borrowing in practice +**Topics**: mutable aliasing bugs, how ownership prevents mutable aliasing, Sync +/ Sync, uniq / shared vs immutable / mutable, `Rc` and `Arc`, interior +mutability in depth, - - Readings: - - [Rust - A Unique Perspective](https://limpet.net/mbrubeck/2019/02/07/rust-a-unique-perspective.html) - - [Too Many Linked Lists](https://cglab.ca/~abeinges/blah/too-many-lists/book) - - Topics: when to use pass-by-value, the performance impact of moves, - reference-bearing structs, (TODO: what other ownership and borrowing - topics?) +### [Lesson: Ownership and borrowing in practice][t-own] ([slides][s-own]) -- Lesson: Parallel Rust +**Readings**: - - Topics: sharing vs message passing, thread pools, (TODO what else?) +- [Rust - A Unique Perspective](https://limpet.net/mbrubeck/2019/02/07/rust-a-unique-perspective.html) +- [Too Many Linked Lists](https://cglab.ca/~abeinges/blah/too-many-lists/book) -- Lesson: Benchmarking, profiling, and debugging +**Topics**: when to use pass-by-value, the performance impact of moves, +reference-bearing structs - - Topics: println debugging, RUST_BACKTRACE, perf, callgrind?, bpftrace?, criterion and critcmp, gdb and - Rust, blocking/ (http://www.brendangregg.com/offcpuanalysis.html), (TODO which - profiling / bloat tools?), std bench vs criterion, black_box and - alternatives, how does rust benchmarking work?, + +### [Lesson: Parallel Rust][t-par] ([slides][s-par]) + +**Topics**: sharing vs message passing, thread pools + + +### [Lesson: Benchmarking, profiling, and debugging][t-prof] ([slides][s-prof]) + +**Topics**: println debugging, RUST_BACKTRACE, perf, gdb and Rust, std bench vs +criterion, black_box and alternatives, how does rust benchmarking work?, other +tools ## Section 5 (Async) -- Project: Async I/O - - Task: create a multi-threaded, persistent key/value store server and client - with asynchronous networking via the GRPC protocol. +### [Project: Async I/O][p-async] + +**Task**: Create a multi-threaded, persistent key/value store server and client +with asynchronous networking via HTTP. + +**Goals**: + +- Use async hyper and futures +- Support range queries +- Understand the distinction between concurrency and parallelism +- Use a thread pool to prevent "blocking" - - Goals: - - Define and compile a GRPC protocol - - Use tokio, prost and tower-grpc for networking - - Support range queries - - Understand the distinction between concurrency and parallelism - - Use a thread pool to prevent "blocking" +**Extensions**: crash recovery, async/await - - Extensions: tokio-file (or whatever), crash recovery, async/await, LSM - compaction -- Lesson: Basic futures +### [Lesson: Basic futures][t-fut] ([slides][s-fut]) - - Readings - - [The What and How of Futures and async/await in Rust](https://www.youtube.com/watch?v=9_3krAQtD2k) +**Readings**: - - Topics: what are futures?, how to think in futures, futures patterns, - mio? (probably should go somewhere before tokio) +- [The What and How of Futures and async/await in Rust](https://www.youtube.com/watch?v=9_3krAQtD2k) -- Lesson: Async I/O with Tokio and Tower +**Topics**: what are futures?, how to think in futures, futures patterns, mio - - Readings: - - [The Tokio docs](https://tokio.rs/docs/) - todo maybe list specific sections - - Topics: (TODO need to research further), alternatives to tokio +### [Lesson: `async` / `await`][t-async-await] ([slides][s-async-await]) -- Lesson: `async` / `await` +**Topics**: futures vs async/await, how does async / await work, +async borrowing, Pin - - Topics: futures vs async/await, how does async / await work, - async borrowing, Pin + + + + @@ -239,4 +270,49 @@ A suggested workflow: [p-tools]: projects/tools/project.md [t-whirlwind]: lessons/whirlwind.md [s-whirlwind]: lessons/whirlwind.slides.html - +[t-data]: lessons/data-structures.md +[s-data]: lessons/data-structures.slides.html +[t-crates]: lessons/crates.md +[s-crates]: lessons/crates.slides.html +[t-tools]: lessons/tools.md +[s-tools]: lessons/tools.slides.html +[t-fmt]: lessons/formatting.md +[s-fmt]: lessons/formatting.slides.html + + + +[p-fs]: projects/file-io/project.md +[t-errors]: lessons/error-handling.md +[s-errors]: lessons/error-handling.slides.html +[t-coll]: lessons/collections-and-iterators.md +[s-coll]: lessons/collections-and-iterators.slides.html + + + +[p-net]: projects/networking/project.md +[t-net]: lessons/networking.md +[s-net]: lessons/networking.slides.html +[t-build]: lessons/build-time.md +[s-build]: lessons/build-time.slides.html +[t-grpc]: lessons/grpc.md +[s-grpc]: lessons/gprc.slides.html + + + +[p-par]: projects/parallelism/project.md +[t-alias]: lessons/aliasing-and-mutability.md +[s-alias]: lessons/aliasing-and-mutability.slides.html +[t-own]: lessons/ownership-and-borrowing.md +[s-own]: lessons/ownership-and-borrowing.slides.html +[t-par]: lessons/parallelism.md +[s-par]: lessons/parallelism.slides.html +[t-prof]: lessons/profiling.md +[s-prof]: lessons/profiling.slides.html + + + +[p-async]: projects/async-io/project.md +[t-fut]: lessons/futures.md +[s-fut]: lessons/futures.slides.html +[t-async-await]: lessons/async-await.md +[s-async-await]: lessons/async-await.slides.html diff --git a/rust/projects/tools/Cargo.lock b/rust/projects/tools/Cargo.lock new file mode 100644 index 000000000..a0afef956 --- /dev/null +++ b/rust/projects/tools/Cargo.lock @@ -0,0 +1,4 @@ +[[package]] +name = "kvs" +version = "0.1.0" + diff --git a/rust/projects/tools/Cargo.toml b/rust/projects/tools/Cargo.toml new file mode 100644 index 000000000..3d8d08a50 --- /dev/null +++ b/rust/projects/tools/Cargo.toml @@ -0,0 +1,8 @@ +[package] +name = "kvs" +version = "0.1.0" +authors = ["Brian Anderson "] +description = "A key-value store" +edition = "2018" + +[dependencies] diff --git a/rust/projects/tools/project.md b/rust/projects/tools/project.md index f76aeec51..612df5799 100644 --- a/rust/projects/tools/project.md +++ b/rust/projects/tools/project.md @@ -1,25 +1,121 @@ # Project: Tools and good bones +**Task**: Create an in-memory key/value store that passes simple tests and responds +to command-line arguments. + +**Goals**: + +- Install the Rust compiler and tools +- Learn the project structure used throughout this course +- Use `cargo init` / `run` / `test` / `clippy` / `fmt` +- Learn how to find and import crates from crates.io +- Define an appropriate data type for a key-value store + +**Topics**: testing, clap, `CARGO_VERSION` etc., clippy, rustfmt + +**Extensions**: structopt + + ## Introduction +In this project you will create a simple in-memory key/value store that passes +some tests and responds to command line arguments. The focus of this project is +on the tooling and setup that goes into a typical Rust project. + + +## Project spec + +The cargo project, `kvs`, builds a command-line key-value store client called +`kvs`, which in turn calls into a library called `kvs`. + +The `kvs` executable supports the following command line arguments: + +- `kvs set ` + + Set the value of a string key to a string + +- `kvs get ` + + Get the string value of a given string key + +- `kvs -V` + + Print the version + +The `kvs` library contains a type, `KvStore`, that supports the following +methods: + +- `KvStore::set(key: String, value: String)` + + Set the value of a string key to a string + +- `KvStore::get(key: String) -> Option` + + Get the string value of the a string key. If the key does not exist, + return `None`. + +The `KvStore` type stores values in-memory, and thus the command-line client can +do little more than print the version. The `get`/ `set` commands will return an +"unimplemented" error when run from the command line. Future projects will store +values on disk and have a working command line interface. + + ## How to treat these projects -TODO: +The more you find the answers for yourself, the more you learn. So even though +this course comes with solutions and answers to every project, don't just go +read the answers and move on. Read the material, do the projects, and answer the +questions yourself. + +It is ok to get help though. In particular, you are encouraged +to ask questions an the #beginners channel in [the Rust Discord][rd] +or the #rust-beginners channel or [Mozilla IRC][mi]. + +[rd]: https://discord.gg/rust-lang +[mi]: https://chat.mibbit.com/?server=irc.mozilla.org&channel=%23rust + +For each project, your task is to make the provided tests pass, as described +below. If you are submitting the project for evaluation, then this is the part +that will be evaluated. In addition, the projects pose questions periodically. +These are to encourage you to think deeper about the subject matter, though your +answers won't be evaluated. -re answers to questions, solutions to problems, collaboration -## Setup +## Installation -You will do the work for this project (and all projects in this course) in your -own git repository, with your own Cargo project. You will import the test -cases for the project from the [source repository for this course][course]. +At this point in your Rust programming experience you should know +how to install Rust via [rustup]. + +[rustup]: https://www.rustup.rs + +If you haven't already, do so now by running + +``` +curl https://sh.rustup.rs -sSf | sh +``` + +(If you are running Windows then follow the instructions on rustup.rs. Note +though that you will face more challenges than others during this course, as it +was developed on Unix. In general, Rust development on Windows is as less +polished experience than on Unix). + +Verify that the toolchain works by typing `rustc -V`. If that doesn't work, log +out and log in again so that changes to the login profile made during +installation can take effect. + + +## Project setup + +You will do the work for this project in your own git repository, with your own +Cargo project. You will import the test cases for the project from the [source +repository for this course][course]. [course]: https://github.com/pingcap/talent-plan Note that within that repository, all content related to this course is within the `rust` subdirectory. You may ignore any other directories. -The projects in this course are both libraries and executables. They are +The projects in this course contain both libraries and executables. They are executables because we are developing an application that can be run. They are libraries because the supplied test cases must link to them. @@ -37,8 +133,11 @@ Cargo.toml tests.rs ``` -**Question A**: what does this directory layout suggest about how `lib.rs`, +**Question A**: What does this directory layout suggest about how `lib.rs`, `main.rs`, and `tests.rs` are compiled, and how they are linked to each other? +How many libraries and executables will this project build? Which source files +are compiled into each library and executable? Which executables link to which +libraries? [Answers](answers.md#question-a). The `Cargo.toml`, `lib.rs` and `main.rs` files look as follows: @@ -47,16 +146,17 @@ The `Cargo.toml`, `lib.rs` and `main.rs` files look as follows: ```toml [package] -name = "mypackage" +name = "kvs" version = "0.1.0" authors = ["Brian Anderson "] +description = "A key-value store" edition = "2018" ``` `lib.rs`: ```rust -pub fn main() { +pub fn run() { println!("Hello, world!"); } ``` @@ -65,13 +165,14 @@ pub fn main() { ```rust fn main() { - mypackage::main() + kvs::run() } ``` -The `name` and `authors` values can be whatever you like, Note though that the -contents of `main.rs` are affected by the package name, which is also the name -of the library within the package. +The `name` and `authors` values can be whatever you like, and the author should +be yourself. Note though that the contents of `main.rs` are affected by the +package name, which is also the name of the library within the package. (TODO +clarify) Finally, the `tests.rs` file is copied from the course materials. In this case, copy from the course repository the file `rust/project/tools/tests/tests.rs` @@ -81,25 +182,45 @@ You may set up this project with `cargo new --lib`, `cargo init --lib`, or manually. You'll probably also want to initialize a git repository in the same directory. -**Question B**: this is the simplest project setup that accomplishes our goals. In +**Question B**: This is the simplest project setup that accomplishes our goals. In practice we might not name `src/bin/main.rs` as `main.rs`. Why not? What's the name of our binary? What are two ways we could change the name of that binary? Try it yourself. [Answers](answers.md#question-b). -At this point you should be able to run the program with `cargo run`, and the -tests with `cargo test`. All the tests will fail, like +At this point you should be able to run the program with `cargo run`. It should + +_Try it now._ + +You are set up for this project and ready to start hacking. + + +## Part 1: Make the tests compile + +You've been provided with a suite of unit tests in `tests/tests.rs`. Open it up +and take a look. + +Try to run the tests with `cargo test`. What happens? Why? + +Your first task for this project is to make the tests _compile_. In `src/lib.rs` +write the type and method definitions necessary to make `cargo test --no-run` +complete successfully. Don't write any method bodies yet — +instead write `panic!()`. + +_Do that now before moving on._ + +Once that is done, if you run `cargo test` (without `--no-run`), +you should see that some of your tests are failing, like ``` -TODO +TODO insert after we have a sample project ``` -**Question C**: notice that there are _four_ different set of tests running (each could be -called a "test suite"). Where do each of those test suites come from? +**Question C**: Notice that there are _four_ different set of tests running +(each could be called a "test suite"). Where do each of those test suites come +from? [Answers](answers.md#question-c). -(TODO: use "test suite" like this or pick different terminology?) - In practice, particularly with large projects, you won't run the entire set of test suites while developing a single feature. To narrow down the set of tests to the ones we care about, run the following: @@ -114,19 +235,201 @@ this case the tests in `tests.rs` (`--test tests`). We can even narrow our testi down to a single test case within the `tests` test suite, like ``` -cargo test --test tests -- TODO +cargo test --test tests -- cli_no_args ``` -And that's probably how you will be running the tests yourself as you work +or, by matching multiple test cases, a few, like: + + +``` +cargo test --test tests -- cli +``` + +_Try it now._ + +That's probably how you will be running the tests yourself as you work through the project, otherwise you will be distracted by the many failing tests -that you are not currently working on fixing. +that you have not yet fixed. + +**Question D**: Why might we not want to run empty test suites? Besides issuing +the above command, how could we permanently disable the three test suites we +don't care about by editing the project manifest (`Cargo.toml`)? +[Answers](answers.md#question-d). + + +## Part 2: Accept command line arguments + +The key / value stores throughout this course are all controlled through a +command-line client. In this project the command-line client is very simple +because the state of the key-value store is only stored in memory, not persisted +to disk. + +In this part you will make the `cli_*` test cases pass. Notice that these test +cases all call the `main` function from your `kvs` library. This is +why we've put the "real" `main` function in `lib.rs` (the `kvs` library), +instead of `main.rs` (the `kvs` CLI). + +**Question E**: This pattern of having the executable do nothing but call into +the library's `main` function is often a reasonable thing to do, though it is +not often used for production libraries. What are some downsides of placing a +program's user interface into a library instead of directly into the executable? +[Answers](answers.md#question-e). + +Recall how to run individual test cases from previous sections +of + +Again, the interface for the CLI is: + +- `kvs set ` + + Set the value of a string key to a string + +- `kvs get ` + + Get the string value of a given string key + +- `kvs -V` + + Print the version + +In this iteration though, the `get` and `set` commands will print to stderr the +string, "unimplemented", and exiting with a non-zero exit code, indicating an +error. + +You will use the `clap` crate to handle command-line arguments. + +Use [crates.io](https://crates.io) to find the documentation +for the `clap` crate, and implement the command line interface +such that the `cli_*` test cases pass. + + +## Part 3: Cargo environment variables + +When you set up `clap` to parse your command line arguments, you probably set +the name, version, authors, and description (if not, do so). This information is +redundant w/ values provided in `Cargo.toml`. Cargo sets environment variables +that can be accessed through Rust source code, at build time. + +_Modify your clap setup to set these values from standard cargo environment +variables._ + + + + +## Part 3: Store values in memory + +Now that your command line scaffolding is done, let's turn to the implementation +of `KvStore`, and make the remaining test cases pass. + +The behavior of `KvStore`'s methods are fully-defined through the test cases +themselves — you don't need any further description to complete the +code for this project. + +_Make the remaining test cases pass by implementing methods on `KvStore`._ + + +## Part 4: Documentation + +You have implemented the project's functionality, but there are still a few more +things to do before it is a polished piece of Rust software, ready for +contributions or publication. + +First, public items should generally have doc comments. + +Doc comments are displayed in a crate's API documentation. API documentation can +be generated with the command, `cargo doc`, which will render them as HTML to +the `target/doc` folder. Note though that `target/doc` folder does not contain +an `index.html`. Your crate's documentation will be located at +`target/doc/kvs/index.html`. You can launch a web browser at that location with +`cargo doc --open`. + +[Good doc comments][gdc] do not just repeat the name of the function, nor repeat +information gathered from the type signature. They explain why and how one would +use a function, what the return value is on both success and failure, error and +panic conditions. The library you have written is very simple so the +documentation can be simple as well. If you truly cannot think of anything +useful to add through doc comments then it can be ok to not add a doc comment +(this is a matter of preference). With no doc comments it should be obvious how +the type or function is used from the name and type signature alone. + +Doc comments contain examples, and those examples can be tested with `cargo test +--doc`. + +You may want to add `#![deny(missing_docs)]` to the top of `src/lib.rs` +to enforce that all public items have doc comments. + +_Add doc comments to the types and methods in your library. Follow the +[documentatine guidelines][gdc]. Give each an example and make sure they pass +`cargo test --doc`._ + +[gdc]: https://rust-lang-nursery.github.io/api-guidelines/documentation.html + + +## Part 5: Ensure good style with `clippy` and `rustfmt` + +`clippy` and `rustfmt` are tools for enforcing common Rust style. `clippy` helps +ensure that code uses modern idioms, and prevents patterns that commonly lead to +errors. `rustfmt` enforces that code is formatted consistently. + +Both tools are included in the Rust toolchain, but not installed by default. +They can be installed with the following commands: + +``` +rustup component add clippy +rustup component add rustfix +``` + +_Do that now._ + +Both tools are invoked as cargo subcommands, `clippy` as `cargo clippy` and +`rustfmt` as `cargo fmt`. Note that `cargo fmt` modifies your source code, so +you might want to check in any changes before running it to avoid accidentally +making unwanted changes, after which you can include the changes as part of the +previous commit with `git commit --amend`. + +_Run `clippy` against your project and make any suggested changes. Run `rustfmt` +against yur project and commit any changes it makes._ + +Congratulations, you are done with project 1! If you like you +may complete the remaining "extensions". They are optional. + + +## Extension 1: `structopt` + +In this project we used `clap` to parse command line arguments. It's typical to +represent a program's parsed command line arguments as a struct, perhaps named +`Config` or `Options`. Doing so requires calling the appropriate methods on +`clap`'s `ArgMatches` type. Both steps, for larger programs, require _a lot_ of +boilerplate code. The `structopt` crate greatly reduces boilerplate by allowing +you to define a `Config` struct, annotated to automatically produce a `clap` +command line parser that produces that struct. Some find this approach nicer +than writing the `clap` code explicitly. + +_Modify your program to use `structopt` for parsing command line +arguments instead of using `clap` directly._ + -**Question D**: even if a given test suite doesn't contain any tests, why -might we not want them to run? Besides issuing the above command, how could -we permanently disable the three test suites we don't care about by -editing the project manifest (`Cargo.toml`)? [Answers](answers.md#question-d). + diff --git a/rust/projects/tools/src/bin/main.rs b/rust/projects/tools/src/bin/main.rs new file mode 100644 index 000000000..c435ec402 --- /dev/null +++ b/rust/projects/tools/src/bin/main.rs @@ -0,0 +1,3 @@ +fn main() { + kvs::run() +} diff --git a/rust/projects/tools/src/lib.rs b/rust/projects/tools/src/lib.rs new file mode 100644 index 000000000..d7e09afe0 --- /dev/null +++ b/rust/projects/tools/src/lib.rs @@ -0,0 +1,3 @@ +pub fn run() { + println!("Hello, world!"); +} diff --git a/rust/projects/tools/tests/tests.rs b/rust/projects/tools/tests/tests.rs new file mode 100644 index 000000000..399e3b459 --- /dev/null +++ b/rust/projects/tools/tests/tests.rs @@ -0,0 +1,8 @@ +#[test] +fn cli_no_args() {} + +#[test] +fn cli_2() {} + +#[test] +fn bar() {}