Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

alembic migrations are broken because alembic doesn't support drop_column for sqlite #207

Closed
heartsucker opened this issue Dec 4, 2018 · 0 comments · Fixed by #241
Closed
Assignees
Labels
bug Something isn't working

Comments

@heartsucker
Copy link
Contributor

To reproduce:

rm -rf svs.sqlite
sqlite3 svs.sqlaite .databases
alembic upgrade +2
alembic downgrade -1

Output from the last command:

INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running downgrade cd0fbb73bf8e -> 5344fc3a3d3f, Capture document count
Traceback (most recent call last):
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: near "DROP": syntax error

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/bin/alembic", line 11, in <module>
    sys.exit(main())
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/config.py", line 502, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/config.py", line 496, in main
    self.run_cmd(cfg, options)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/config.py", line 479, in run_cmd
    **dict((k, getattr(options, k, None)) for k in kwarg)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/command.py", line 294, in downgrade
    script.run_env()
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/script/base.py", line 427, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/util/pyfiles.py", line 81, in load_python_file
    module = load_module_py(module_id, path)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/util/compat.py", line 82, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 673, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "alembic/env.py", line 70, in <module>
    run_migrations_online()
  File "alembic/env.py", line 64, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/runtime/environment.py", line 836, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/runtime/migration.py", line 330, in run_migrations
    step.migration_fn(**kw)
  File "/home/heartsucker/code/freedomofpress/securedrop-client/alembic/versions/cd0fbb73bf8e_capture_document_count.py", line 25, in downgrade
    op.drop_column('sources', 'document_count')
  File "<string>", line 8, in drop_column
  File "<string>", line 3, in drop_column
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/operations/ops.py", line 1667, in drop_column
    return operations.invoke(op)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/operations/base.py", line 319, in invoke
    return fn(self, operation)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/operations/toimpl.py", line 81, in drop_column
    **operation.kw
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/ddl/impl.py", line 172, in drop_column
    self._exec(base.DropColumn(table_name, column, schema=schema))
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/alembic/ddl/impl.py", line 115, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1009, in _execute_ddl
    compiled
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 248, in reraise
    raise value.with_traceback(tb)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/heartsucker/.local/share/virtualenvs/securedrop-client-GExBtLKv/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) near "DROP": syntax error [SQL: 'ALTER TABLE sources DROP COLUMN document_count'] (Background on this error at: http://sqlalche.me/e/e3q8)

Note that this would have been caught by #55

We can fix this by copying what we do in SecureDrop core like we do here with temp tables: https://github.com/freedomofpress/securedrop/blob/develop/securedrop/alembic/versions/2d0ce3ee5bdc_added_passphrase_hash_column_to_.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant