Automate the Config Creation Process #46

Abby-Wheelis · 2023-11-14T22:37:15Z

After some trial and error, I have begun a workflow which will automatically create a PR for new config files after someone files the templated issue form to create them.

Steps:

deployer fills out a templated issue
creation/update of issue launches workflow that:
- converts the issue to a json file
- submits a PR to add the json file to our repo
TBD workflow for ensuring the PR is valid
We check the PR for validity and merge into repo
- check that it is a valid deployment (not an attack)
- check for associated "promised" files like onboarding survey, labels, or trip survey
- may need to do some work by hand for some custom nitty gritty (still working out form templating for trip survey or custom reminder scheme cases)

This is in an early draft stage, as I still have things to work out:

make sure the workflow only goes when the templated issue is filed, not a normal issue
ask about config fields I was unsure about
handle custom trip surveys
handle custom reminder schemes
set up validation (currently some degree by nature of the parsing process (won't work if things are missing), but I want to investigate testing it out)
write documentation for workflow and potential need to update the workflow

Related Issue

using this template, new deployers who are interested in working with us can fill out specifically what they want to do, in a controlled form manner -- used as input to a workflow

When an issue is created or updated, a workflow is set off that converts the templated issue into a json file, and submits it as a pr if the issue is updated, so will the pr

adding a conditional to skip the job if it is not using the issue template

with these try/catch blocks, the goal is to catch any errors as they happen, and throw a failure message specific to that section of the process By indicating where the problem was encountered, it should be easier to fix The goal would be to enforce all of the controls we need through the form itself, such that people cannot submit invalid forms, but there are limitations to the issue template

Abby-Wheelis · 2023-11-16T17:33:09Z

More form updates as I practice using this in my local repo:

set a default of false to trip end notifications
set autogen and token_generate to both use the value of autogen (these need to match)

we've found that these notifications tend to annoy users, so it should certainly be something that must be enabled rather than something that must be enabled

these are commented out for now, as their values are hardcoded, but in case we decide to reintroduce them, fixing the formatting of these

this section will host trip and reminder scheme customization that is difficult to set up through a form -- I'm going to move forward with setting up with a form for it, but we will need to check the results against the description of what is desired more than likely

extrapolated, since they don't exist yet

shankari

I'm going to merge this for now so that we can see how it looks, but have some comments that can be addressed in a future PR

shankari · 2023-11-18T19:17:36Z

.github/ISSUE_TEMPLATE/add-new-config.yml

+  validations:
+    required: true
+- type: input
+  id: deployment_partner_name_lang1


is there no way to have this be an array instead of hardcoded lang1, lang2...? What if some programs want to have more than 2 languages? Most election material in my county, for example, has at least 4 languages - English, Spanish, Mandarin and Vietnamese.

If not, maybe we will need to create multiple templates for the number of languages

I end up compiling it into an array, but this was the best way I found to format the template - the options are relatively limited, multiple templates could work

shankari · 2023-11-18T19:25:26Z

.github/ISSUE_TEMPLATE/add-new-config.yml

+  id: trip_survey_path
+  attributes:
+    label: Custom Trip Survey
+    description: Please add a path to a trip survey, something like "https://raw.githubusercontent.com/sebastianbarry/nrel-openpath-deploy-configs/surveys-info-and-surveys-data/survey-resources/data-json/trip-end-survey-multiple-select.json"


we should really merge this so that the example is not in @sebastianbarry's repo
@sebastianbarry can you submit a PR?

shankari · 2023-11-18T19:35:18Z

.github/actions/convertIssue/parse-issue-body.js

+import { readFile } from "node:fs/promises";
+import { setFailed } from "@actions/core";
+import yaml from "js-yaml";
+


If this is inspired by some other file, make sure to link it here and ensure that the LICENSE is correct

shankari · 2023-11-18T22:42:26Z

Note that issues were not enabled on this repo, and I had to enable them to (hopefully) get this to work.

shankari · 2023-11-18T22:45:39Z

Seems to work now - I can at least create an issue...

Abby-Wheelis · 2023-11-18T23:23:10Z

For the actions to run, I think you'll need to make sure that workflows are enabled and that they have permission to create pull requests

shankari · 2023-11-18T23:28:31Z

I am pretty sure that workflows are enabled.
https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#enabling-workflows-for-private-repository-forks

wrt enabling creation of pull requests, I think you can use the `GitHub actions token" and enable pull requests on it
https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

Do you want to create a test issue and see what it does?

Abby-Wheelis · 2023-11-19T00:04:37Z

It works! But is overzealous with the commits, I'll include a fix for that in my cleanup.

see the PR here

Abby Wheelis added 9 commits November 14, 2023 15:24

initial draft of issue template

7032c86

using this template, new deployers who are interested in working with us can fill out specifically what they want to do, in a controlled form manner -- used as input to a workflow

workflow with custom action

253a8d4

When an issue is created or updated, a workflow is set off that converts the templated issue into a json file, and submits it as a pr if the issue is updated, so will the pr

only run the job if it's a new config

2ec2b70

adding a conditional to skip the job if it is not using the issue template

add note about automation to readme

9d512db

polish the boolean parsing function

e00f492

increase documentation in parsing file

e6ddc8a

remove unused dependency

f9e82a2

token generation values should match

0e33613

Abby Wheelis added 4 commits November 16, 2023 10:39

trip end notifications default to false

e18af08

we've found that these notifications tend to annoy users, so it should certainly be something that must be enabled rather than something that must be enabled

fix formatting

ab879d8

these are commented out for now, as their values are hardcoded, but in case we decide to reintroduce them, fixing the formatting of these

add trip survey path field

d4e59ce

This was referenced Nov 16, 2023

Automate Config Creation e-mission/e-mission-docs#1026

Closed

Docs Surrounding Automatic Config Process e-mission/e-mission-docs#1031

Merged

Abby-Wheelis marked this pull request as ready for review November 16, 2023 23:23

add links to issue and docs

d54549d

extrapolated, since they don't exist yet

shankari approved these changes Nov 18, 2023

View reviewed changes

shankari merged commit a34c7f4 into e-mission:main Nov 18, 2023

Abby-Wheelis mentioned this pull request Nov 20, 2023

Polishing Changes to the Config Automation #49

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automate the Config Creation Process #46

Automate the Config Creation Process #46

Abby-Wheelis commented Nov 14, 2023 •

edited

Loading

Abby-Wheelis commented Nov 16, 2023 •

edited

Loading

shankari left a comment

shankari Nov 18, 2023

shankari Nov 18, 2023

Abby-Wheelis Nov 18, 2023

shankari Nov 18, 2023

shankari Nov 18, 2023

shankari commented Nov 18, 2023

shankari commented Nov 18, 2023

Abby-Wheelis commented Nov 18, 2023

shankari commented Nov 18, 2023 •

edited

Loading

Abby-Wheelis commented Nov 19, 2023

Automate the Config Creation Process #46

Automate the Config Creation Process #46

Conversation

Abby-Wheelis commented Nov 14, 2023 • edited Loading

Abby-Wheelis commented Nov 16, 2023 • edited Loading

shankari left a comment

Choose a reason for hiding this comment

shankari Nov 18, 2023

Choose a reason for hiding this comment

shankari Nov 18, 2023

Choose a reason for hiding this comment

Abby-Wheelis Nov 18, 2023

Choose a reason for hiding this comment

shankari Nov 18, 2023

Choose a reason for hiding this comment

shankari Nov 18, 2023

Choose a reason for hiding this comment

shankari commented Nov 18, 2023

shankari commented Nov 18, 2023

Abby-Wheelis commented Nov 18, 2023

shankari commented Nov 18, 2023 • edited Loading

Abby-Wheelis commented Nov 19, 2023

Abby-Wheelis commented Nov 14, 2023 •

edited

Loading

Abby-Wheelis commented Nov 16, 2023 •

edited

Loading

shankari commented Nov 18, 2023 •

edited

Loading