Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update circleci config #4093

Merged
merged 5 commits into from
Oct 11, 2020
Merged

Conversation

pkfec
Copy link
Contributor

@pkfec pkfec commented Oct 1, 2020

Summary (required)

  1. Update python docker image version to match the python version given in the runtime.txt file
  2. Update Postgres docker image version to match the DB version in our cloud.gov (medium-gp-psql-redundant) environment
  3. Set POSTGRES_HOST_AUTH_METHOD: "trust" environment variable. Builds fail without this env variable.
    See:
    https://discuss.circleci.com/t/postgresql-image-password-not-specified-issue/34555?utm_medium=SEM&utm_source=gnb&utm_campaign=SEM-gb-DSA-Eng-uscan&utm_content=&utm_term=dynamicSearch-&gclid=CjwKCAjw_NX7BRA1EiwA2dpg0hsoFeu7z02id7KLIrJiImZpUfDShDzw7DbAwJJdL75bstnqP4914RoCoWAQAvD_BwE
  1. Python from 3.7.4-stretch-node-browsers to 3.7.9-stretch-node-browsers
  2. Postgres from 9.6.8 to 11.9
  3. Add script to accommodate node upgrades in circleci

Screenshots

Before upgrade:
Screen Shot 2020-10-02 at 10 06 47 AM

Screen Shot 2020-10-02 at 10 07 06 AM

After upgrade:
Screen Shot 2020-10-06 at 12 57 02 PM

Screen Shot 2020-10-02 at 10 05 46 AM

How to test the changes locally

  • Rebuild or Rerun feature/3978-update-circleci-config branch on circleci and notice Spin up environment and Container circleci tasks downloading the upgraded python and postgres docker images.

@pkfec pkfec force-pushed the feature/3978-update-circleci-config branch from 611fe28 to c00b0b0 Compare October 2, 2020 16:07
Copy link
Contributor

@johnnyporkchops johnnyporkchops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good Priya! This will make life easier for local CMS development!

@pkfec pkfec force-pushed the feature/3978-update-circleci-config branch from c00b0b0 to 971e2f2 Compare October 6, 2020 16:52
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
nvm install 10.16.0
nvm use 10.16.0
nvm alias default 10.16.0
sudo npm install -g npm
npm install webpack
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outside the scope of this ticket, but we don't need npm install webpack if we also have npm install (and it's on the next line)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rfultz thanks for the suggestion. I will go ahead and remove the npm install webpack from the script.

@@ -50,8 +51,16 @@ jobs:
pip install -r requirements.txt

- run:
name: Install Node.js dependencies
name: Install node dependencies
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nowhere near a dealbreaker but "Install Node dependencies" doesn't really describe what's happening here. The node packages are being installed, yes, but it's also building the entire front-end of the site (JavaScript, CSS/scss, image copying)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rfultz Its the task name that we see on circleci builds. Do you prefer any other name? Please suggest.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rfultz @pkfec What about something like this? "Install node and build web assets". Seems like a generic way of saying we're building everything from JS/scss/images

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@patphongs the proposed task name sounds good. I will go ahead and update it, unless there is any objection.

environment:
TZ: America/New_York
DATABASE_URL: postgres://postgres@0.0.0.0/cfdm_cms_test

# PostgreSQL
- image: circleci/postgres:9.6.8
- image: circleci/postgres:11.9
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the build, why did this section end with Build was canceled?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

environment:
POSTGRES_USER: postgres
POSTGRES_HOST_AUTH_METHOD: "trust"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build is giving us a warning about this line (POSTGRES_HOST_AUTH_METHOD). Is this something we want to do outside of local? (The warning looks important but I'd rely 100% on others' expertise)

Copy link
Contributor Author

@pkfec pkfec Oct 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rfultz - We had those warning prior to the postgres docker upgrade. With older docker version, the circleci would silently, ignore the warning and let the build run.
Unless the ENV variable is specified, the builds will fail.

I mentioned in my PR already on why the ENV variable needs to be set and posting here again for your reference:https://discuss.circleci.com/t/postgresql-image-password-not-specified-issue/34555?utm_medium=SEM&utm_source=gnb&utm_campaign=SEM-gb-DSA-Eng-uscan&utm_content=&utm_term=dynamicSearch-&gclid=CjwKCAjw_NX7BRA1EiwA2dpg0hsoFeu7z02id7KLIrJiImZpUfDShDzw7DbAwJJdL75bstnqP4914RoCoWAQAvD_BwE

@pkfec
Copy link
Contributor Author

pkfec commented Oct 8, 2020

@patphongs Like you suggested i update the Node task name to Install node and build web assets. Please review and let me know if you have any questions or concerns. Added the latest CMS build screenshot below for your reference:
cc @rfultz @johnnyporkchops

circlebuild: https://app.circleci.com/pipelines/github/fecgov/fec-cms/654/workflows/b48f1c2c-679f-427a-8046-5c6f1d313b2e/jobs/3892

Screen Shot 2020-10-08 at 12 10 36 PM

@codecov-io
Copy link

Codecov Report

Merging #4093 into develop will decrease coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #4093      +/-   ##
===========================================
- Coverage    75.47%   75.45%   -0.02%     
===========================================
  Files          121      121              
  Lines         7412     7412              
  Branches       596      596              
===========================================
- Hits          5594     5593       -1     
- Misses        1818     1819       +1     
Impacted Files Coverage Δ
fec/fec/static/js/modules/calendar.js 91.91% <0.00%> (-0.74%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 017055e...f3e2a56. Read the comment docs.

@jason-upchurch jason-upchurch merged commit 3364ff6 into develop Oct 11, 2020
@jason-upchurch
Copy link
Contributor

@pkfec @patphongs I'm merging this in preparation for innovation sprint as 2 of 3 reviewers have approved. Please let me know if I need to revert.

@lbeaufort lbeaufort deleted the feature/3978-update-circleci-config branch November 3, 2020 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update circle ci python image to be consistent with runtime version
6 participants