Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: cache task #6

Draft
wants to merge 18 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
# these are just here to test the Action
test/** -linguist-detectable
test/** -linguist-detectable
# ignore in diffs
dist/** -linguist-generated -diff
34 changes: 34 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Build

on:
push:
branches: [main]
pull_request:

jobs:
main:
runs-on: ubuntu-latest

permissions:
contents: write

steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}

- name: Setup Deno
uses: ./

- name: Lint
run: deno lint

- name: Bundle
run: deno run -A ./bundle.ts

- name: Push changes
uses: stefanzweifel/git-auto-commit-action@v5
with:
file_pattern: "dist/*"
commit_message: "chore: build action bundle"
commit_author: "github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>"
26 changes: 2 additions & 24 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,30 +21,8 @@ jobs:

- name: Setup Deno
uses: ./
with:
deno-config-path: "test/basic/deno.json"

- name: Test basic usage
run: deno run -A ./test/basic/main.ts

customized:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]

name: Customized on ${{ matrix.os }}
runs-on: ${{ matrix.os }}

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Test action
uses: ./
with:
deno-version: 1.x
deno-lock-path: "test/customized/deno.lock"
directory: "test/customized"

- name: Test customized usage
working-directory: test/customized
run: deno run -A main.ts
5 changes: 4 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
{
"deno.enable": true
"deno.enable": true,
"[typescript]": {
"editor.defaultFormatter": "denoland.vscode-deno"
}
}
72 changes: 58 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,62 @@

### Deno Setup action with integrated cache.

- Based on:
- Based on:
- [`denoland/setup-deno@v1`](https://github.com/denoland/setup-deno),
- [`actions/cache@v3`](https://github.com/actions/cache)
- Handles restoring and caching to `DENO_DIR` for you.
- Annotates your source code from `deno lint --compact` output.\
See the summary of the most recent [Problem Matcher worflow](https://github.com/nekowinston/setup-deno/actions/workflows/problem-matcher.yml) for an example.
See the summary of the most recent
[Problem Matcher worflow](https://github.com/nekowinston/setup-deno/actions/workflows/problem-matcher.yml)
to see it in action.
- Works on Ubuntu, macOS & Windows runners.

### Usage
### A note on Deno caching

Deno only caches the deps it needs to run a single script, and does so on the
fly. This is great UX/DX! But it does make effective caching in GitHub actions
harder.

[v1](https://github.com/nekowinston/setup-deno/tree/v1) of this repo used to run
`find . -regex '.*\.[jt]sx*' -exec deno cache {} \;` to prefetch all
dependencies for all scripts more reliably. In case of a non-standard repo
layout (e.g. monorepo), the action had another input to specify a single
directory to cache.

[v2](https://github.com/nekowinston/setup-deno/tree/v2) now offers an extension
to the `deno.json` config! Simply specify which paths to _include_ or _exclude_:

```jsonc
{
"tasks": {
// add a task to easily call `deno task cache`
"cache": "deno run --no-lock --allow-read --allow-run=deno --allow-env=CI,DENO_DIR https://esm.sh/gh/nekowinston/setup-deno/dist/cache.js"
},
// the task will respect both `.exclude`
"exclude": ["dist", "foo/*"],
// and new `.cache.include` & `.cache.exclude` fields
"cache": {
"include": "foo/bar.ts",
"exclude": "scripts/some_rarely_run_script.ts"
}
}
```

These fields get combined, and cached accordingly:

```console
$ deno task cache -v
caching main.ts
ignored dist/main.js
ignored foo/qux.ts
caching foo/bar.ts
ignored foo/baz.ts
ignored scripts/some_rarely_run_script.ts
```

> [!NOTE] This script is completely optional for this action.

### Action Usage

#### Basic:

Expand All @@ -27,26 +74,23 @@
with:
deno-version: "~1.38"
deno-json-path: ./subdirectory/deno.json
deno-lock-path: ./subdirectory/deno.lock
directory: ./subdirectory
```

### Inputs

- `deno-version`:\
The Deno version to install. Can be a semver version of a stable release, `'canary'` for the latest canary, or the Git hash of a specific canary release.\
See [`setup-deno`](https://github.com/marketplace/actions/setup-deno) for examples.\
The Deno version to install. Can be a semver version of a stable release,
`'canary'` for the latest canary, or the Git hash of a specific canary
release.\
See [`setup-deno`](https://github.com/marketplace/actions/setup-deno) for
examples.\
Defaults to `1.x`.
- `deno-json-path`:\
The path to the Deno config file to use for caching.\
Defaults to an empty string, using the built-in CLI default.
- `deno-lock-path`:\
The path to the lock file to use for caching.\
Defaults to `./deno.lock`.
- `directory`:\
The path to the scripts to cache. This can be useful if Deno is only part of your repo, and stored in a subdirectory.\
Defaults to the repo root.

### Outputs:

- `deno-version`: The Deno version that was installed.
- `is-canary`: If the installed Deno version was a canary version.
- `cache-hit`: A boolean value to indicate an exact match was found for the key.
- `cache-hit`: A boolean value to indicate an exact match was found for the key.
8 changes: 1 addition & 7 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ inputs:
deno-json-path:
description: "The path to the Deno config file to use for caching. Defaults to an empty string, using the built-in CLI default (`deno.json` and `deno.jsonc`)."
default: ""
deno-lock-path:
description: "The path to the lock file to use for caching. Defaults to `./deno.lock`."
default: "./deno.lock"
directory:
description: "The path to the scripts to cache. Defaults to the repo root."
default: "."

outputs:
deno-version:
Expand Down Expand Up @@ -56,4 +50,4 @@ runs:
- name: Restore Deno dependencies
shell: bash
run: |
find ${{ inputs.directory }} -regex '.*\.[jt]sx*' -exec deno cache ${{ inputs.deno-json-path != '' && '--config=' || '' }}${{ inputs.deno-json-path }} {} \;
"${{ github.action_path }}/dist/cache.js" --config "${{ inputs.deno-json-path }}"
21 changes: 21 additions & 0 deletions bundle.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/usr/bin/env -S deno run --allow-env=HOME,DENO_AUTH_TOKENS,DENO_DIR --allow-read --allow-write=./dist
import { bundle } from "https://deno.land/x/emit@0.32.0/mod.ts";
import { ensureDir } from "https://deno.land/std@0.209.0/fs/ensure_dir.ts";

const shebang = await Deno.readTextFileSync("./cache.ts").split("\n")[0];
const bundled = await bundle(
new URL("./cache.ts", import.meta.url),
{ minify: true },
);

ensureDir("./dist");
await Deno.writeTextFile(
"./dist/cache.js",
[
shebang.startsWith("#!") ? shebang : "#!/usr/bin/env -S deno run -A",
bundled.code,
].join("\n"),
{
mode: 0o744,
},
);
123 changes: 123 additions & 0 deletions cache.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
#!/usr/bin/env -S deno run --no-lock --allow-read --allow-run=deno --allow-env=CI,DENO_DIR
import { parseArgs } from "https://deno.land/std@0.210.0/cli/parse_args.ts";
import { encodeHex } from "https://deno.land/std@0.210.0/encoding/hex.ts";
import { exists, walk } from "https://deno.land/std@0.210.0/fs/mod.ts";
import * as JSONC from "https://deno.land/std@0.210.0/jsonc/parse.ts";
import * as log from "https://deno.land/std@0.210.0/log/mod.ts";
import { relative } from "https://deno.land/std@0.210.0/path/relative.ts";
import ignore from "https://esm.sh/gh/nekowinston/deno-ignore@v5.3.0/index.js?pin=v135";
type DenoConfig = {
cache: { include?: string[]; exclude?: string[] };
exclude?: string[];
[key: string]: unknown;
};

if (import.meta.main) {
const args = parseArgs(Deno.args, {
boolean: ["dry-run", "verbose", "lock-write"],
negatable: ["lock-write"],
string: ["config"],
default: { "lock-write": true },
alias: { "dry-run": "n", verbose: "v" },
"--": true,
unknown: (arg) => {
console.log(`Unknown argument: ${arg}`);
console.log(
"Usage: cache [-n | --dry-run] [-v | --verbose] [--] [DENO_ARGS...]",
);
Deno.exit(0);
},
});
const denoDir = Deno.env.get("DENO_DIR");

const logLevel = ["1", "true"].includes(Deno.env.get("CI") ?? "")
? "DEBUG"
: (args.verbose ? "DEBUG" : "INFO");
log.setup({
handlers: {
console: new log.handlers.ConsoleHandler(logLevel, {
formatter: (logRecord) =>
Deno.noColor
? logRecord.levelName + " " + logRecord.msg
: logRecord.msg,
}),
},
loggers: { default: { handlers: ["console"], "level": logLevel } },
});

if (args["dry-run"]) log.warning("dry-run mode enabled");
if (args.config && !await exists(args.config)) {
log.error(`config file ${args.config} not found`);
Deno.exit(1);
}

// the two-step hash is intentional
const hash = await Deno.readFile("./deno.lock")
.then((data) => crypto.subtle.digest("SHA-256", data))
.then((data) => crypto.subtle.digest("SHA-256", data))
.then((data) => encodeHex(new Uint8Array(data)))
.catch((_) => "<no deno.lock found>");
log.debug(`GitHub actions deno.lock hash: ${hash}`);

// Deno implementation: `json` takes precedence over `jsonc`
const cfgs = args.config ? [args.config] : ["deno.json", "deno.jsonc"];
const denoCfg = await Promise.all(
cfgs.map((path) => Deno.readTextFile(path).catch((_) => undefined)),
).then((v) => JSONC.parse(v.filter(Boolean)[0] ?? "{}")) as DenoConfig;

// flattening so that a single string doesn't break too much
const patterns = [
...[denoCfg?.exclude].filter(Boolean).flat(),
...[denoCfg?.cache?.exclude].filter(Boolean).flat(),
...[denoCfg?.cache?.include].filter(Boolean).flatMap((p) => `!${p}`),
] as string[];
patterns.length > 0 && log.debug(`patterns:\n| ${patterns.join("\n| ")}`);
const ig = ignore().add(patterns);

const walkIterator = walk(".", {
exts: ["js", ".jsx", ".ts", ".tsx"],
includeDirs: false,
});

const paths = [];

for await (const entry of walkIterator) {
const relativePath = relative(".", entry.path);

// assuming that DENO_DIR is set inside the current directory,
// this would first fetch the deps of this script, store them there,
// then include them here. since the cache could be huge, don't log it.
if (denoDir && entry.path.startsWith(denoDir)) continue;
if (ig.ignores(relativePath)) {
log.debug(`ignored ${relativePath}`);
continue;
}

log.debug(`caching ${relativePath}`);
paths.push(relativePath);
}

const denoArgs = [
"cache",
args["lock-write"] && "--lock-write",
args.config && `--config=${args.config}`,
...args["--"], // everything after `--` is passed to Deno
...paths,
].filter(Boolean) as string[];
const cmd = new Deno.Command(Deno.execPath(), { args: denoArgs });
const cmdString = Deno.execPath() + " " + denoArgs.join(" ");

if (args["dry-run"]) {
log.info(`would run: ${cmdString}`);
Deno.exit(0);
}
log.debug(`running: ${cmdString}`);
const { stderr, code } = await cmd.output();

// deno cache outputs to stderr
for (const line of new TextDecoder().decode(stderr).split("\n")) {
if (line.length > 0) log.info("deno> " + line);
}

if (code === 0) log.debug(`finished caching ${paths.length} files`);
}
5 changes: 4 additions & 1 deletion deno.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
{
"tasks": {
"cache": "deno run --no-lock --allow-read --allow-run=deno --allow-env=CI,DENO_DIR ./cache.ts"
},
"imports": {
"std/": "https://deno.land/std@0.207.0/",
"cliffy/": "https://deno.land/x/cliffy@v1.0.0-rc.3/"
},
"exclude": ["test/problem-matcher"]
"exclude": ["dist", "test"]
}
Loading