v14.0.0: upgrade to Nutmeg

- 💥 [Feature] Upgrade to Nutmeg: (by @regisb)
    - 💥 [Feature] Persistent grades are now enabled by default.
    - [Bugfix] Remove edX references from bulk emails ([issue](openedx/wg-build-test-release#100)).
    - [Improvement] For Tutor Nightly (and only Nightly), official plugins are now installed from their nightly branches on GitHub instead of a version range on PyPI. This will allow Nightly users to install all official plugins by running ``pip install -e ".[full]"``.
    - [Bugfix] Start MongoDB when running migrations, because a new data migration fails if MongoDB is not running

CHANGELOG.md

@@ -18,6 +18,14 @@ Every user-facing change should have an entry in this changelog. Please respect
 
 ## Unreleased
 
+## v14.0.0 (2022-06-09)
+
+- 💥 [Feature] Upgrade to Nutmeg: (by @regisb)
+    - 💥 [Feature] Persistent grades are now enabled by default.
+    - [Bugfix] Remove edX references from bulk emails ([issue](https://github.com/openedx/build-test-release-wg/issues/100)).
+    - [Improvement] For Tutor Nightly (and only Nightly), official plugins are now installed from their nightly branches on GitHub instead of a version range on PyPI. This will allow Nightly users to install all official plugins by running ``pip install -e ".[full]"``.
+    - [Bugfix] Start MongoDB when running migrations, because a new data migration fails if MongoDB is not running
+
 ## v13.2.3 (2022-05-30)
 
 - [Fix] Truncate site display name to 50 characters with a warning, fixing data too long error for long site names. (by @navinkarkera)

docs/install.rst

@@ -121,11 +121,11 @@ Major Open edX releases are published twice a year, in June and December, by the
 4. Test the new release in a sandboxed environment.
 5. If you are running edx-platform, or some other repository from a custom branch, then you should rebase (and test) your changes on top of the latest release tag (see :ref:`edx_platform_fork`).
 
-The process for upgrading from one major release to the next works similarly to any other upgrade, with the ``quickstart`` command (see above). The single difference is that if the ``quickstart`` command detects that your tutor environment was generated with an older release, it will perform a few release-specific upgrade steps. These extra upgrade steps will be performed just once. But they will be ignored if you updated your local environment (for instance: with ``tutor config save``) before running ``quickstart``. This situation typically occurs if you need to re-build some Docker images (see above). In such a case, you should make use of the ``upgrade`` command. For instance, to upgrade a local installation from Lilac to Maple and rebuild some Docker images, run::
+The process for upgrading from one major release to the next works similarly to any other upgrade, with the ``quickstart`` command (see above). The single difference is that if the ``quickstart`` command detects that your tutor environment was generated with an older release, it will perform a few release-specific upgrade steps. These extra upgrade steps will be performed just once. But they will be ignored if you updated your local environment (for instance: with ``tutor config save``) before running ``quickstart``. This situation typically occurs if you need to re-build some Docker images (see above). In such a case, you should make use of the ``upgrade`` command. For instance, to upgrade a local installation from Maple to Nutmeg and rebuild some Docker images, run::
 
     tutor config save
     tutor images build all # list the images that should be rebuilt here
-    tutor local upgrade --from=lilac
+    tutor local upgrade --from=maple
     tutor local quickstart
 
 .. _autocomplete:

requirements/plugins.txt

@@ -1,12 +1,12 @@
-# change version ranges when upgrading from maple
-tutor-android>=13.0.0,<14.0.0
-tutor-discovery>=13.0.0,<14.0.0
-tutor-ecommerce>=13.0.0,<14.0.0
-tutor-forum>=13.0.0,<14.0.0
-tutor-license>=13.0.0,<14.0.0
-tutor-mfe>=13.0.0,<14.0.0
-tutor-minio>=13.0.0,<14.0.0
-tutor-notes>=13.0.0,<14.0.0
-tutor-richie>=13.0.0,<14.0.0
-tutor-webui>=13.0.0,<14.0.0
-tutor-xqueue>=13.0.0,<14.0.0
+# change version ranges when upgrading from nutmeg
+tutor-android>=14.0.0,<15.0.0
+tutor-discovery>=14.0.0,<15.0.0
+tutor-ecommerce>=14.0.0,<15.0.0
+tutor-forum>=14.0.0,<15.0.0
+tutor-license>=14.0.0,<15.0.0
+tutor-mfe>=14.0.0,<15.0.0
+tutor-minio>=14.0.0,<15.0.0
+tutor-notes>=14.0.0,<15.0.0
+tutor-richie>=14.0.0,<15.0.0
+tutor-webui>=14.0.0,<15.0.0
+tutor-xqueue>=14.0.0,<15.0.0

tests/test_env.py

@@ -244,6 +244,6 @@ def test_current_version_in_latest_env(self) -> None:
             ) as f:
                 f.write(__version__)
             self.assertEqual(__version__, env.current_version(root))
-            self.assertEqual("maple", env.get_env_release(root))
+            self.assertEqual("nutmeg", env.get_env_release(root))
             self.assertIsNone(env.should_upgrade_from_release(root))
             self.assertTrue(env.is_up_to_date(root))

tutor/__about__.py

@@ -2,7 +2,7 @@
 
 # Increment this version number to trigger a new release. See
 # docs/tutor.html#versioning for information on the versioning scheme.
-__version__ = "13.2.3"
+__version__ = "14.0.0"
 
 # The version suffix will be appended to the actual version, separated by a
 # dash. Use this suffix to differentiate between the actual released version and

tutor/commands/k8s.py

@@ -468,7 +468,7 @@ def wait(context: K8sContext, name: str) -> None:
 @click.option(
     "--from",
     "from_release",
-    type=click.Choice(["ironwood", "juniper", "koa", "lilac"]),
+    type=click.Choice(["ironwood", "juniper", "koa", "lilac", "maple"]),
 )
 @click.pass_context
 def upgrade(context: click.Context, from_release: Optional[str]) -> None:

tutor/commands/local.py

@@ -154,7 +154,7 @@ def quickstart(
 @click.option(
     "--from",
     "from_release",
-    type=click.Choice(["ironwood", "juniper", "koa", "lilac"]),
+    type=click.Choice(["ironwood", "juniper", "koa", "lilac", "maple"]),
 )
 @click.pass_context
 def upgrade(context: click.Context, from_release: t.Optional[str]) -> None:

tutor/commands/upgrade/k8s.py

@@ -27,6 +27,9 @@ def upgrade_from(context: Context, from_release: str) -> None:
         upgrade_from_lilac(config)
         running_release = "maple"
 
+    if running_release == "maple":
+        running_release = "nutmeg"
+
 
 def upgrade_from_ironwood(config: Config) -> None:
     if not config["RUN_MONGODB"]:

tutor/commands/upgrade/local.py

@@ -31,6 +31,9 @@ def upgrade_from(context: click.Context, from_release: str) -> None:
         common_upgrade.upgrade_from_lilac(config)
         running_release = "maple"
 
+    if running_release == "maple":
+        running_release = "nutmeg"
+
 
 def upgrade_from_ironwood(context: click.Context, config: Config) -> None:
     click.echo(fmt.title("Upgrading from Ironwood"))

tutor/env.py

@@ -376,6 +376,7 @@ def get_release(version: str) -> str:
         "11": "koa",
         "12": "lilac",
         "13": "maple",
+        "14": "nutmeg",
     }[version.split(".", maxsplit=1)[0]]
 
 

tutor/templates/apps/openedx/config/lms.env.json

@@ -22,6 +22,7 @@
     "ENABLE_OAUTH2_PROVIDER": true,
     "ENABLE_THIRD_PARTY_AUTH": true,
     "MILESTONES_APP": true,
+    "PERSISTENT_GRADES_ENABLED_FOR_ALL_TESTS": true,
     "ENABLE_PREREQUISITE_COURSES": true
   },
   "LMS_ROOT_URL": "{{ "https" if ENABLE_HTTPS else "http" }}://{{ LMS_HOST }}",

tutor/templates/apps/openedx/settings/partials/common_lms.py

@@ -20,6 +20,7 @@
 
 # Email settings
 DEFAULT_EMAIL_LOGO_URL = LMS_ROOT_URL + "/theming/asset/images/logo.png"
+BULK_EMAIL_SEND_USING_EDX_ACE = True
 
 # Create folders if necessary
 for folder in [DATA_DIR, LOG_DIR, MEDIA_ROOT, STATIC_ROOT_BASE, ORA2_FILEUPLOAD_ROOT]:

tutor/templates/build/openedx/Dockerfile

@@ -47,15 +47,6 @@ RUN git config --global user.email "tutor@overhang.io" \
 {{ patch("openedx-dockerfile-git-patches-default") }}
 {%- else %}
 # Patch edx-platform
-# Fix forum notification for questions
-# https://github.com/openedx/edx-platform/pull/29611
-RUN git fetch --depth=2 https://github.com/open-craft/edx-platform/ 03731f19459e558f188c06aac5cc9ca1bbc675c2 && git cherry-pick 03731f19459e558f188c06aac5cc9ca1bbc675c2
-# SAML security fix
-# https://github.com/overhangio/edx-platform/tree/overhangio/sec-fix-saml-vulnerability
-RUN git fetch --depth=2 https://github.com/overhangio/edx-platform/ 3b985f207853e88090d68a81acd52866b71f5af7 && git cherry-pick 3b985f207853e88090d68a81acd52866b71f5af7
-# Rate limiting security fix
-# https://github.com/overhangio/edx-platform/tree/overhangio/sec-rate-limiting
-RUN git fetch --depth=2 https://github.com/overhangio/edx-platform/ b5723e416e628cac4fa84392ca13e1b72817674f && git cherry-pick b5723e416e628cac4fa84392ca13e1b72817674f
 {%- endif %}
 
 {# Example: RUN git fetch --depth=2 https://github.com/openedx/edx-platform <GITSHA1> && git cherry-pick <GITSHA1> #}
@@ -85,15 +76,20 @@ COPY --from=code /openedx/edx-platform /openedx/edx-platform
 WORKDIR /openedx/edx-platform
 
 # Install the right version of pip/setuptools
-RUN pip install setuptools==44.1.0 pip==20.0.2 wheel==0.34.2
+# https://pypi.org/project/setuptools/
+# https://pypi.org/project/pip/
+# https://pypi.org/project/wheel/
+RUN pip install setuptools==62.1.0 pip==22.0.4 wheel==0.37.1
 
 # Install base requirements
 RUN pip install -r ./requirements/edx/base.txt
 
 # Install django-redis for using redis as a django cache
-RUN pip install django-redis==4.12.1
+# https://pypi.org/project/django-redis/
+RUN pip install django-redis==5.2.0
 
 # Install uwsgi
+# https://pypi.org/project/uWSGI/
 RUN pip install uwsgi==2.0.20
 
 {{ patch("openedx-dockerfile-post-python-requirements") }}
@@ -156,7 +152,7 @@ RUN pip install -r requirements/edx/local.in
 RUN mkdir -p /openedx/config ./lms/envs/tutor ./cms/envs/tutor
 COPY --chown=app:app revisions.yml /openedx/config/
 ENV LMS_CFG /openedx/config/lms.env.json
-ENV STUDIO_CFG /openedx/config/cms.env.json
+ENV CMS_CFG /openedx/config/cms.env.json
 ENV REVISION_CFG /openedx/config/revisions.yml
 COPY --chown=app:app settings/lms/*.py ./lms/envs/tutor/
 COPY --chown=app:app settings/cms/*.py ./cms/envs/tutor/

tutor/templates/build/openedx/revisions.yml

@@ -1 +1 @@
-EDX_PLATFORM_REVISION: maple
+EDX_PLATFORM_REVISION: nutmeg

tutor/templates/config/defaults.yml

@@ -48,7 +48,7 @@ OPENEDX_LMS_UWSGI_WORKERS: 2
 OPENEDX_MYSQL_DATABASE: "openedx"
 OPENEDX_CSMH_MYSQL_DATABASE: "{{ OPENEDX_MYSQL_DATABASE }}_csmh"
 OPENEDX_MYSQL_USERNAME: "openedx"
-OPENEDX_COMMON_VERSION: "open-release/maple.3"
+OPENEDX_COMMON_VERSION: "open-release/nutmeg.master"
 OPENEDX_EXTRA_PIP_REQUIREMENTS:
   - "openedx-scorm-xblock<14.0.0,>=13.0.0"
 MYSQL_HOST: "mysql"

tutor/templates/hooks/lms/init

@@ -1,4 +1,5 @@
 dockerize -wait tcp://{{ MYSQL_HOST }}:{{ MYSQL_PORT }} -timeout 20s
+dockerize -wait tcp://{{ MONGODB_HOST }}:{{ MONGODB_PORT }} -timeout 20s
 
 echo "Loading settings $DJANGO_SETTINGS_MODULE"
 

tutor/templates/k8s/deployments.yml

@@ -131,7 +131,7 @@ spec:
       containers:
         - name: cms-worker
           image: {{ DOCKER_IMAGE_OPENEDX }}
-          args: ["celery", "worker", "--app=cms.celery", "--loglevel=info", "--hostname=edx.cms.core.default.%%h", "--maxtasksperchild", "100", "--exclude-queues=edx.lms.core.default"]
+          args: ["celery", "--app=cms.celery", "worker", "--loglevel=info", "--hostname=edx.cms.core.default.%%h", "--max-tasks-per-child", "100", "--exclude-queues=edx.lms.core.default"]
           env:
           - name: SERVICE_VARIANT
             value: cms
@@ -231,7 +231,7 @@ spec:
       containers:
         - name: lms-worker
           image: {{ DOCKER_IMAGE_OPENEDX }}
-          args: ["celery", "worker", "--app=lms.celery", "--loglevel=info", "--hostname=edx.lms.core.default.%%h", "--maxtasksperchild=100", "--exclude-queues=edx.cms.core.default"]
+          args: ["celery", "--app=lms.celery", "worker", "--loglevel=info", "--hostname=edx.lms.core.default.%%h", "--max-tasks-per-child=100", "--exclude-queues=edx.cms.core.default"]
           env:
           - name: SERVICE_VARIANT
             value: lms

tutor/templates/local/docker-compose.jobs.yml

@@ -14,7 +14,7 @@ services:
         - ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
         - ../apps/openedx/settings/cms:/openedx/edx-platform/cms/envs/tutor:ro
         - ../apps/openedx/config:/openedx/config:ro
-      depends_on: {{ [("mysql", RUN_MYSQL)]|list_if }}
+      depends_on: {{ [("mysql", RUN_MYSQL), ("mongodb", RUN_MONGODB)]|list_if }}
 
     cms-job:
       image: {{ DOCKER_IMAGE_OPENEDX }}

tutor/templates/local/docker-compose.yml

@@ -170,7 +170,7 @@ services:
     environment:
       SERVICE_VARIANT: lms
       DJANGO_SETTINGS_MODULE: lms.envs.tutor.production
-    command: celery worker --app=lms.celery --loglevel=info --hostname=edx.lms.core.default.%%h --maxtasksperchild=100 --exclude-queues=edx.cms.core.default
+    command: celery --app=lms.celery worker --loglevel=info --hostname=edx.lms.core.default.%%h --max-tasks-per-child=100 --exclude-queues=edx.cms.core.default
     restart: unless-stopped
     volumes:
       - ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
@@ -188,7 +188,7 @@ services:
     environment:
       SERVICE_VARIANT: cms
       DJANGO_SETTINGS_MODULE: cms.envs.tutor.production
-    command: celery worker --app=cms.celery --loglevel=info --hostname=edx.cms.core.default.%%h --maxtasksperchild 100 --exclude-queues=edx.lms.core.default
+    command: celery --app=cms.celery worker --loglevel=info --hostname=edx.cms.core.default.%%h --max-tasks-per-child 100 --exclude-queues=edx.lms.core.default
     restart: unless-stopped
     volumes:
       - ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro