Skip to content

Commit

Permalink
[Issue #1400] Close issues in done column (#1533)
Browse files Browse the repository at this point in the history
* feat: Adds a `.github/linters/` sub-directory 
Stores custom linters for the project
* feat: Adds a `.github/linters/scripts/`
Stores custom scripts used by the linters
* Creates a `.github/linters/queries/` sub-directory
Stores graphql queries used by the linters
* feat: Adds `linters/scripts/close-issues-in-done-col.sh`
Linter to close issues in the done column that are still open
* ci: Adds `.github/workflows/ci-project-linters.yml` 
Checks all linters by doing a dry run
* ci: Adds `.github/workflows/lint-close-done-issues.yml` 
Runs the close done issues linter each night at midnight
  • Loading branch information
widal001 committed Mar 26, 2024
1 parent c775ee7 commit 8bea3cb
Show file tree
Hide file tree
Showing 6 changed files with 238 additions and 0 deletions.
1 change: 1 addition & 0 deletions .github/linters/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
tmp
59 changes: 59 additions & 0 deletions .github/linters/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Custom project linters

## Introduction

This section of the codebase contains a set of custom linters that we run against our GitHub repo and the associated GitHub projects.

## Project directory structure

Outlines the structure of the linters codebase, relative to the root of the simpler-grants-gov repo.

```text
root
├── .github
│ └── linters
│ └── queries Contains graphql queries used by the custom linting scripts
│ └── scripts Contains scripts that lint the codebase, GitHub repo, or GitHub projects
│ └── tmp Git ignored directory that can store temporary outputs of scripts
```

## Usage

### Review automated linters

| Workflow name | Description | Interval |
| --------------------------------------------- | ----------------------------------------------------------------- | ------------------ |
| [Lint - Close done issues][close-done-issues] | Close issues that are marked as done in a project but still open. | Nightly at 12am ET |

### Manually run the linters

> [!NOTE]
> Only project maintainers can manually run the linters
1. Navigate to the [GitHub actions tab of the repo](https://github.com/HHS/simpler-grants-gov/actions).
2. Find the name of the workflow in the left hand side of the "Actions" menu. It should start with `Lint -`.
3. Click on the workflow you want to trigger manually.
4. On the next page, click the dropdown menu that says "Run workflow".
5. Choose the version of the workflow you want to run based on its branch. **Note:** In most cases this will be `Branch: main`.
6. Finally, click the green "Run workflow" button to trigger that linter.

### Add a new linter

1. Create a new linting script in `linters/scripts/`.
- **Note:** If you're script requires a long graphql query for the GitHub graphql API, pull that query out into its own `.graphql` file stored in `linters/queries/`.
- **Note:** If you're script changes any resources directly in GitHub, make sure you include a dry run option that skips over any write step if the `--dry-run` flag is passed during execution.
- For a reference please see [`linters/scripts/close-issues-in-done-col.sh`][close-done-issues-script] and its associated query [`linters/queries/get-project-items.graphql`][get-project-items-query]
2. Update the permissions on your script so it can be executed: `chmod 744 ./scripts/<path-to-script>`
3. Test your script locally `./scripts/<path-to-script> --dry-run`
4. Add your script to the [CI checks for the linters](../workflows/ci-project-linters.yml). Make sure you include any environment variables needed by your script and the `--dry-run` flag in the GitHub action `run` statement.
5. Create a new GitHub action workflow to run your linter.
- **Note:** Make sure the name of the yaml file is prefixed with `lint-`.
- **Note:** Make sure the workflow is run from the `linters/` sub-directory.
- **Note:** Make sure the workflow has a `workflow_dispatch:` trigger option to allow for manual triggers.
- For a reference, please see [`.github/workflows/lint-close-done-issues.yml`][close-done-issues]
6. Add your new linter to the table in the ["Review automated linters"](#review-automated-linters) section above


[close-done-issues]: ../workflows/lint-close-done-issues.yml
[close-done-issues-script]: ./scripts/close-issues-in-done-col.sh
[get-project-items-query]: ./queries/get-project-items.graphql
38 changes: 38 additions & 0 deletions .github/linters/queries/get-project-items.graphql
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
query ($endCursor: String, $login: String!, $project: Int!, $batch: Int!) {
# get the project by the user login and project number
organization(login: $login) {
projectV2(number: $project) {
# insert the projectFields fragment below
...projectFields
}
}
}

fragment projectFields on ProjectV2 {
# get project items in batches of 100, which is the match batch size
items(first: $batch, after: $endCursor) {
# allows us to use --paginate in the gh api call
pageInfo {
hasNextPage
endCursor
}

# fetch details per item in the list
nodes {
# fetch the value of the status column
status: fieldValueByName(name: "Status") {
... on ProjectV2ItemFieldSingleSelectValue {
name
}
}

# fetch the issue URL and open/closed state
issue: content {
... on Issue {
url
state
}
}
}
}
}
83 changes: 83 additions & 0 deletions .github/linters/scripts/close-issues-in-done-col.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
#! /bin/bash
# Usage: ./scripts/close-issues-in-done-col.sh --dry-run
# Closes issues that are in the "Done" column on a project but still open in the repo

# parse command line args with format `--option arg`
# see this stack overflow for more details:
# https://stackoverflow.com/a/14203146/7338319
while [[ $# -gt 0 ]]; do
case $1 in
--dry-run)
echo "Running in dry run mode"
DRY_RUN=YES
shift # past argument
;;
--batch)
BATCH="$2"
shift # past argument
shift # past value
;;
--org)
ORG="$2"
shift # past argument
shift # past value
;;
--project)
PROJECT="$2"
shift # past argument
shift # past value
;;
--status)
STATUS="$2"
shift # past argument
shift # past value
;;
-*|--*)
echo "Unknown option $1"
exit 1
;;
*)
POSITIONAL_ARGS+=("$1") # save positional arg
shift # past argument
;;
esac
done

# set script-specific variables
mkdir -p tmp
to_close_file="./tmp/open-issues-that-are-done.txt"
query=$(cat ./queries/get-project-items.graphql)

# print the parsed variables for debugging
echo "Finding open issues in the '${STATUS}' column of GitHub project: ${ORG}/${PROJECT}"

# get all tickets from the project with their
# URL, open/closed state in the repo, and status on the project
gh api graphql \
--paginate \
--field login="${ORG}" \
--field project="${PROJECT}" \
--field batch="${BATCH}" \
-f query="${query}" \
--jq ".data.organization.projectV2.items.nodes" |\
# combine results into a single array
jq --slurp 'add' |\
# isolate the URLs of the issues that are marked "Done" in the project
# but still open in the repo, and use --raw-ouput flag to remove quotes
jq --raw-output "
.[]
| select((.status.name == \"${STATUS}\") and (.issue.state == \"OPEN\"))
| .issue.url" > $to_close_file # write output to a file

# iterate through the list of URLs written to the to_close_file
# and close them with a comment indicating the reason for closing
while read URL; do
if [[ $DRY_RUN == "YES" ]];
then
echo "Would close issue with URL: ${URL}"
else
echo "Closing issue with URL: ${URL}"
gh issue close $URL \
--comment "Closing because issue was marked as '${STATUS}' in https://github.com/orgs/${ORG}/projects/${PROJECT}"
fi
done < $to_close_file
30 changes: 30 additions & 0 deletions .github/workflows/ci-project-linters.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: CI Project linters

on:
workflow_dispatch:
pull_request:
paths:
- .github/linters/**
- .github/workflows/ci-project-linters.yml

defaults:
run:
working-directory: ./.github/linters # ensures that this job runs from the ./linters sub-directory

jobs:
dry-run-project-linters:
name: Dry run GitHub project linters
runs-on: ubuntu-latest
env:
GH_TOKEN: ${{ secrets.GH_TOKEN_PROJECT_ACCESS }}
steps:
- uses: actions/checkout@v3

- name: Dry run - Close open issues marked as "Done" in Sprint Board
run: |
./scripts/close-issues-in-done-col.sh \
--org HHS \
--project 13 \
--status Done \
--batch 100 \
--dry-run
27 changes: 27 additions & 0 deletions .github/workflows/lint-close-done-issues.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: Lint - Close done issues

on:
workflow_dispatch:
schedule:
- cron: "00 5 * * 1-5"

defaults:
run:
working-directory: ./.github/linters # ensures that this job runs from the ./linters sub-directory

jobs:
run-project-linters:
name: Run GitHub project linters
runs-on: ubuntu-latest
env:
GH_TOKEN: ${{ secrets.GH_TOKEN_PROJECT_ACCESS }}
steps:
- uses: actions/checkout@v3

- name: Close open issues marked as "Done" in Sprint Board
run: |
./scripts/close-issues-in-done-col.sh \
--org HHS \
--project 13 \
--status Done \
--batch 100

0 comments on commit 8bea3cb

Please sign in to comment.