Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: ADR for alembic migrations #56

Merged
merged 1 commit into from
Jul 5, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions docs/decisions/0007_clickhouse_migrations.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
7. Alembic migrations for Aspects
#################################

Status
******

Accepted

Context
*******

Alembic is a migration tool for SQLAlchemy. It is used to manage the Clickhouse database
schema for the Aspects project. Alembic creates a migrations table in the database to
keep track of the migrations that have been applied.

Alembic has support for:

#. Branching migrations.
#. Multiple databases.
#. Rollback migrations
#. Autogenerated migrations.
#. Multiple migration directories.

For more information about Alembic, see https://alembic.sqlalchemy.org/en/latest/

Decisions
*********

#. There should be a single initial SQL executed at initialization time. This will
create the databases, users and permissions, and any other necessary setup before
running migrations.
#. Migratiosn are separated by service (e.g. xAPI, event sink, vector, etc). Each service has its
own migration directory.
#. Migrations are executed with RAW SQL using the ``op.execute`` method. This allows
us to use ClickHouse-specific SQL syntax.
#. Migrations should always define a downgrade step, even if it is a no-op.
#. Migrations are generated using a Sequential ID. This allows us to easily identify
the order in which migrations were created and executed.
#. Migrations cannot be modified once they are released to production. If a migration
needs to be modified, a new migration should be created with a new sequential ID.
#. The alembic migrations table is highly coupled with the xAPI database, for this reason
the migrations table is stored in the xAPI database.

Consequences
************

* Users should not run migrations manually. They should be run automatically by Tutor
when the project is initialized and at upgrade time.

Rejected Alternatives
*********************

**Clickhouse SQL Alchemy Models**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also rejected Django for this


Model allows us to automagically create tables and columns in Clickhouse. However, it is
not well supported and we didn't find any benefits over raw SQL migrations.

**Django Clickhouse Models**

Even when this would be created in a separated Open edX plugin, we didn't wanted to open
the possibility to couple Open edX and Clickhouse. We want to keep Clickhouse as a
separated service.