Skip to content
This repository has been archived by the owner on Aug 21, 2024. It is now read-only.

Ansible Configuration to Dockerfile #1333

Merged
merged 1 commit into from
Nov 3, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Dockerfile
49 changes: 49 additions & 0 deletions .github/workflows/push-docker-images.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
name: Build and Push Docker Images

on:
push:
branches:
- master
tags:
- open-release/*
jobs:
push:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v2

# Use the release name as the image tag if we're building an open release tag.
# Examples: if we're building 'open-release/maple.1', tag the image as 'maple.1'.
# Otherwise, we must be building from a push to master, so use 'latest'.
- name: Get tag name
id: get-tag-name
uses: actions/github-script@v5
with:
script: |
const releasePrefix = 'refs/tags/open-release/';
const tagName = context.ref.split(releasePrefix)[1] || 'latest';
console.log('Will use tag: ' + tagName);
return tagName;
result-encoding: string

- name: Build and push Dev Docker image
uses: docker/build-push-action@v1
with:
push: true
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
target: dev
repository: edxops/insights-dev
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, can this be in the openedx Dockerhub?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tags: ${{ steps.get-tag-name.outputs.result }},${{ github.sha }}

# - name: Build and push prod Docker image
# uses: docker/build-push-action@v1
# with:
# push: true
# username: ${{ secrets.DOCKERHUB_USERNAME }}
# password: ${{ secrets.DOCKERHUB_PASSWORD }}
# target: prod
# repository: edxops/insights
# tags: ${{ steps.get-tag-name.outputs.result }},${{ github.sha }}
84 changes: 63 additions & 21 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,33 +1,75 @@
FROM ubuntu:xenial as openedx
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are currently using Ubuntu 20.04 which is the focal release. This was outdated and causing issues hence updated this.


RUN apt update && \
apt-get install -y software-properties-common && \
apt-add-repository -y ppa:deadsnakes/ppa && apt-get update && \
apt-get install -y curl && \
apt-get upgrade -qy && \
apt install -y git-core language-pack-en build-essential python3.8-dev python3.8-distutils libmysqlclient-dev && \
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py && python3.8 -m pip install --upgrade pip setuptools && \
FROM ubuntu:focal as app

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && apt-get install --no-install-recommends -qy \
language-pack-en \
build-essential \
python3.8-dev \
python3-virtualenv \
python3.8-distutils \
libmysqlclient-dev \
libssl-dev && \
rm -rf /var/lib/apt/lists/*

RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
ENV ANALYTICS_DASHBOARD_CFG /edx/etc/insights.yml

WORKDIR /edx/app/analytics_dashboard
COPY requirements /edx/app/analytics_dashboard/requirements
RUN python3.8 -m pip install -r requirements/production.txt
# ENV variables lifetime is bound to the container whereas ARGS variables lifetime is bound to the image building process only
# Also ARGS provide us an option of compatibility of Path structure for Tutor and other OpenedX installations
ARG COMMON_CFG_DIR "/edx/etc"
ARG COMMON_APP_DIR="/edx/app"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These variables are just defined here so they can be reused throughout the Dockerfile. They have been defined as ARG instead of ENV because ARG has its lifecycle bound only to Dockerfile whereas ENV variables live throughout the life of Docker container in the form of environment variables As these variables have no usage as environment variables it made more sense to define them as ARG with default values.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this context should be added as a comment in the file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was way before we had an initial review round with Kyle. Meanwhile this is still one of the motivation/reasons to use ARG instead of ENV here, but the actual and important point behind using ARGS is the compatibility of Path structure with Tutor and other OpenedX installations as they can be provided during Image building process.

ARG INSIGHTS_APP_DIR="${COMMON_APP_DIR}/insights"
ARG INSIGHTS_VENV_DIR="${COMMON_APP_DIR}/insights/venvs/insights"
ARG INSIGHTS_CODE_DIR="${INSIGHTS_APP_DIR}/edx_analytics_dashboard"
ARG INSIGHTS_NODEENV_DIR="${COMMON_APP_DIR}/insights/nodeenvs/insights"

ENV PATH "${INSIGHTS_VENV_DIR}/bin:${INSIGHTS_NODEENV_DIR}/bin:$PATH"
ENV INSIGHTS_APP_DIR ${INSIGHTS_APP_DIR}
ENV THEME_SCSS "sass/themes/open-edx.scss"

# No need to activate insights virtualenv as it is already activated by putting in the path
RUN virtualenv -p python3.8 --always-copy ${INSIGHTS_VENV_DIR}

COPY requirements ${INSIGHTS_CODE_DIR}/requirements

ENV PATH="${INSIGHTS_CODE_DIR}/node_modules/.bin:$PATH"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added node_modules to path so packages are available globally in our Docker image and we can run them directly instead of accessing the whole path again and again


WORKDIR ${INSIGHTS_CODE_DIR}/

# insights service config commands below
RUN pip install --no-cache-dir -r ${INSIGHTS_CODE_DIR}/requirements/production.txt

RUN nodeenv ${INSIGHTS_NODEENV_DIR} --node=16.14.0 --prebuilt \
&& npm install -g npm@8.5.x

# Tried to cache the dependencies by copying related files after the npm install step but npm post install fails in that case.
COPY . ${INSIGHTS_CODE_DIR}/
RUN npm set progress=false && npm ci

EXPOSE 8110
CMD gunicorn -b 127.0.0.1:8110 --workers 2 --timeout=300 analytics_dashboard.wsgi:application
EXPOSE 18110

FROM app as dev

RUN pip install --no-cache-dir -r requirements/local.txt

ENV DJANGO_SETTINGS_MODULE "analytics_dashboard.settings.devstack"

# Backwards compatibility with devstack
RUN touch "${INSIGHTS_APP_DIR}/insights_env"

CMD while true; do python ./manage.py runserver 0.0.0.0:8110; sleep 2; done

RUN useradd -m --shell /bin/false app
USER app
COPY . /edx/app/analytics_dashboard
FROM app as prod

FROM openedx as edx.org
RUN python3.8 -m pip install newrelic
CMD newrelic-admin run-program gunicorn -b 127.0.0.1:8110 --workers 2 --timeout=300 analytics_dashboard.wsgi:application
ENV DJANGO_SETTINGS_MODULE "analytics_dashboard.settings.production"

CMD gunicorn \
--pythonpath=/edx/app/insights/edx_analytics_dashboard/analytics_dashboard \
--timeout=300 \
-b 0.0.0.0:8110 \
-w 2 \
- analytics_dashboard.wsgi:application