Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Docker image for cargo-pgo #52

Merged
merged 6 commits into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*
35 changes: 35 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: Publish Docker image
Kobzol marked this conversation as resolved.
Show resolved Hide resolved

on:
push:
tags:
- 'v*'

jobs:
push_to_registry:
name: Push Docker image to Docker Hub
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v4

- name: Log in to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ysun1993/cargo-pgo

- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
22 changes: 22 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
FROM rust:1.76-slim

RUN apt update \
&& apt install -y wget gnupg \
&& wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - \
&& echo "deb http://apt.llvm.org/bookworm/ llvm-toolchain-bookworm-18 main" > /etc/apt/sources.list.d/llvm-toolchain.list \
&& echo "deb-src http://apt.llvm.org/bookworm/ llvm-toolchain-bookworm-18 main" >> /etc/apt/sources.list.d/llvm-toolchain.list \
&& apt-get update \
&& apt install -y \
bolt-18 \
musl-tools \
&& ln -s /usr/bin/merge-fdata-18 /usr/bin/merge-fdata \
&& ln -s /usr/bin/llvm-bolt-18 /usr/bin/llvm-bolt \
&& ln -s /usr/lib/llvm-18/lib/libbolt_rt_instr.a /usr/lib/libbolt_rt_instr.a \
&& apt autoremove -y wget gnupg \
&& rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/llvm-toolchain.list

RUN rustup component add llvm-tools-preview \
&& rustup target add x86_64-unknown-linux-musl \
&& cargo install cargo-pgo
Kobzol marked this conversation as resolved.
Show resolved Hide resolved

WORKDIR /workdir
21 changes: 17 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ and [BOLT](https://github.com/llvm/llvm-project/tree/main/bolt) to optimize Rust

For an example on how to use `cargo-pgo` to optimize a binary on GitHub Actions CI, see [this workflow](ci/pgo.yml).

# Installation
## Installation

```bash
$ cargo install cargo-pgo
```
Expand All @@ -24,10 +25,22 @@ with `rustup`:
$ rustup component add llvm-tools-preview
```

For BOLT, it's unfortunately more complicated. See [below](#bolt-installation) for BOLT installation
guide.
For BOLT, it is highly recommended to use [Docker](#docker).
See [below](#bolt-installation) for BOLT installation guide.

> BOLT and Docker support is currently *experimental*.

## Docker

`cargo-pgo` has builds deployed to Docker Hub, to run `cargo-pgo` on any system that has Docker, run this in your project directory:
Kobzol marked this conversation as resolved.
Show resolved Hide resolved

```bash
docker run -v $(pwd):/workdir --rm -it kobzol/cargo-pgo
```

Then in container you can run `cargo-pgo` as you would on your system.

> BOLT support is currently *experimental*.
Note that with `--rm` argument, the container will be removed after you exit.

## PGO/BOLT workflow
It is important to understand the workflow of using feedback-directed optimizations. Put simply, it
Expand Down
12 changes: 6 additions & 6 deletions tests/integration/bolt.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@ fn test_bolt_optimize() -> anyhow::Result<()> {
let project = init_cargo_project()?;

project.run(&["bolt", "build"])?.assert_ok();
run_command(&project.bolt_instrumented_binary())?;
run_command(project.bolt_instrumented_binary())?;
project.run(&["bolt", "optimize"])?.assert_ok();
run_command(&project.bolt_optimized_binary())?;
run_command(project.bolt_optimized_binary())?;

Ok(())
}
Expand All @@ -55,15 +55,15 @@ fn test_bolt_pgo_optimize() -> anyhow::Result<()> {
let project = init_cargo_project()?;

project.run(&["build"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;

project.run(&["bolt", "build", "--with-pgo"])?.assert_ok();
run_command(&project.bolt_instrumented_binary())?;
run_command(project.bolt_instrumented_binary())?;

project
.run(&["bolt", "optimize", "--with-pgo"])?
.assert_ok();
run_command(&project.bolt_optimized_binary())?;
run_command(project.bolt_optimized_binary())?;

Ok(())
}
Expand Down Expand Up @@ -134,7 +134,7 @@ fn test_bolt_optimize_bolt_args() -> anyhow::Result<()> {
let project = init_cargo_project()?;

project.run(&["bolt", "build"])?.assert_ok();
run_command(&project.bolt_instrumented_binary())?;
run_command(project.bolt_instrumented_binary())?;

project
.run(&[
Expand Down
26 changes: 13 additions & 13 deletions tests/integration/pgo.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ fn test_instrument_run_instrumented_binary() -> anyhow::Result<()> {
let project = init_cargo_project()?;
project.run(&["build"])?.assert_ok();

run_command(&project.main_binary())?;
run_command(project.main_binary())?;

assert!(!get_dir_files(&project.default_pgo_profile_dir())?.is_empty());

Expand All @@ -38,9 +38,9 @@ fn test_build_optimize() -> anyhow::Result<()> {
let project = init_cargo_project()?;

project.run(&["build"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;
project.run(&["optimize"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;

Ok(())
}
Expand Down Expand Up @@ -76,7 +76,7 @@ mod tests {

project.run(&["test"])?.assert_ok();
project.run(&["optimize"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;

Ok(())
}
Expand All @@ -86,7 +86,7 @@ fn test_run_optimize() -> anyhow::Result<()> {
let project = init_cargo_project()?;
project.run(&["run"])?.assert_ok();
project.run(&["optimize"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;

Ok(())
}
Expand All @@ -96,7 +96,7 @@ fn test_bench_optimize() -> anyhow::Result<()> {
let project = init_cargo_project()?;
project.run(&["bench"])?.assert_ok();
project.run(&["optimize"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;

Ok(())
}
Expand All @@ -106,9 +106,9 @@ fn test_instrument_optimize() -> anyhow::Result<()> {
let project = init_cargo_project()?;

project.run(&["instrument"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;
project.run(&["optimize"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;

Ok(())
}
Expand All @@ -118,9 +118,9 @@ fn test_instrument_build() -> anyhow::Result<()> {
let project = init_cargo_project()?;

project.run(&["instrument", "build"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;
project.run(&["optimize"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;

Ok(())
}
Expand Down Expand Up @@ -156,7 +156,7 @@ mod tests {

project.run(&["instrument", "test"])?.assert_ok();
project.run(&["optimize"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;

Ok(())
}
Expand All @@ -166,7 +166,7 @@ fn test_instrument_run() -> anyhow::Result<()> {
let project = init_cargo_project()?;
project.run(&["instrument", "run"])?.assert_ok();
project.run(&["optimize"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;

Ok(())
}
Expand All @@ -176,7 +176,7 @@ fn test_instrument_bench() -> anyhow::Result<()> {
let project = init_cargo_project()?;
project.run(&["instrument", "bench"])?.assert_ok();
project.run(&["optimize"])?.assert_ok();
run_command(&project.main_binary())?;
run_command(project.main_binary())?;

Ok(())
}
Expand Down
2 changes: 1 addition & 1 deletion tests/integration/utils/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ pub fn init_cargo_project() -> anyhow::Result<CargoProject> {

let name = "foo";
let status = Command::new("cargo")
.args(&["new", "--bin", name])
.args(["new", "--bin", name])
.current_dir(dir.path())
.status()?;
assert!(status.success());
Expand Down