warehouse: initial Identity base, migration

[woodruffw]

See #12477. Follow-on from #11272.

Very WIP; pushing up for initial review and discussion.

Signed-off-by: William Woodruff william@trailofbits.com

[woodruffw]

These changes evaluate and initialize the DB correctly, but aren't fully mapped:

>>> from warehouse.identity.models import Identity
>>> db.query(Identity).count()
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<console>", line 1, in <module>
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2161, in query
    return self._query_cls(entities, self, **kwargs)
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 180, in __init__
    self._set_entities(entities)
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 187, in _set_entities
    self._raw_columns = [
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 188, in <listcomp>
    coercions.expect(
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 178, in expect
    insp = inspection.inspect(element, raiseerr=False)
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/inspection.py", line 64, in inspect
    ret = reg(subject)
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/orm/decl_api.py", line 1056, in _inspect_decl_meta
    _DeferredMapperConfig.raise_unmapped_for_cls(cls)
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py", line 1091, in raise_unmapped_for_cls
    class_._sa_raise_deferred_config()
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/ext/declarative/extensions.py", line 307, in _sa_raise_deferred_config
    raise orm_exc.UnmappedClassError(
sqlalchemy.orm.exc.UnmappedClassError: Class warehouse.identity.models.Identity is a subclass of AbstractConcreteBase and has a mapping pending until all subclasses are defined. Call the sqlalchemy.orm.configure_mappers() function after all subclasses have been defined to complete the mapping of this class.

This makes sense to me, but I'm not actually sure where to put the configure_mappers() or Base.registry.configure() call that it's expecting.

cc @dstufft for thoughts.

[woodruffw]

Separately, if I manually call Base.registry.configure(), I get this exception when trying to use the abstract concrete base:

>>> ModelBase.registry.configure()
>>> db.query(Identity).count()
<console>:1: SAWarning: UserDefinedType CIText() will not produce a cache key because the ``cache_ok`` attribute is not set to True.  This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)
Traceback (most recent call last):
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedObject: type "disablereason" does not exist
LINE 3: ...MP WITHOUT TIME ZONE) AS last_login, CAST(NULL AS disablerea...
                                                             ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<console>", line 1, in <module>
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 3167, in count
    return self._from_self(col).enable_eagerloads(False).scalar()
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2892, in scalar
    ret = self.one()
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2869, in one
    return self._iter().one()
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2907, in _iter
    result = self.session.execute(
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1714, in execute
    result = conn._execute_20(statement, params or {}, execution_options)
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
    ret = self._execute_context(
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
    self._handle_dbapi_exception(
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
    util.raise_(
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
    raise exception
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/opt/warehouse/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedObject) type "disablereason" does not exist
LINE 3: ...MP WITHOUT TIME ZONE) AS last_login, CAST(NULL AS disablerea...
                                                             ^

[SQL: SELECT count(*) AS count_1
FROM (SELECT pjoin.id AS pjoin_id, pjoin.discriminator AS pjoin_discriminator, pjoin.repository_name AS pjoin_repository_name, pjoin.repository_owner AS pjoin_repository_owner, pjoin.repository_owner_id AS pjoin_repository_owner_id, pjoin.workflow_filename AS pjoin_workflow_filename, pjoin.sitemap_bucket AS pjoin_sitemap_bucket, pjoin.username AS pjoin_username, pjoin.name AS pjoin_name, pjoin.password AS pjoin_password, pjoin.password_date AS pjoin_password_date, pjoin.is_active AS pjoin_is_active, pjoin.is_frozen AS pjoin_is_frozen, pjoin.is_superuser AS pjoin_is_superuser, pjoin.is_moderator AS pjoin_is_moderator, pjoin.is_psf_staff AS pjoin_is_psf_staff, pjoin.prohibit_password_reset AS pjoin_prohibit_password_reset, pjoin.hide_avatar AS pjoin_hide_avatar, pjoin.date_joined AS pjoin_date_joined, pjoin.last_login AS pjoin_last_login, pjoin.disabled_for AS pjoin_disabled_for, pjoin.totp_secret AS pjoin_totp_secret, pjoin.last_totp_value AS pjoin_last_totp_value, pjoin.type AS pjoin_type
FROM (SELECT oidc_providers.id AS id, oidc_providers.discriminator AS discriminator, CAST(NULL AS VARCHAR) AS repository_name, CAST(NULL AS VARCHAR) AS repository_owner, CAST(NULL AS VARCHAR) AS repository_owner_id, CAST(NULL AS VARCHAR) AS workflow_filename, CAST(NULL AS TEXT) AS sitemap_bucket, CAST(NULL AS CITEXT) AS username, CAST(NULL AS VARCHAR(100)) AS name, CAST(NULL AS VARCHAR(128)) AS password, CAST(NULL AS TIMESTAMP WITHOUT TIME ZONE) AS password_date, CAST(NULL AS BOOLEAN) AS is_active, CAST(NULL AS BOOLEAN) AS is_frozen, CAST(NULL AS BOOLEAN) AS is_superuser, CAST(NULL AS BOOLEAN) AS is_moderator, CAST(NULL AS BOOLEAN) AS is_psf_staff, CAST(NULL AS BOOLEAN) AS prohibit_password_reset, CAST(NULL AS BOOLEAN) AS hide_avatar, CAST(NULL AS TIMESTAMP WITHOUT TIME ZONE) AS date_joined, CAST(NULL AS TIMESTAMP WITHOUT TIME ZONE) AS last_login, CAST(NULL AS disablereason) AS disabled_for, CAST(NULL AS BYTEA) AS totp_secret, CAST(NULL AS VARCHAR) AS last_totp_value, 'oidc_providers' AS type
FROM oidc_providers UNION ALL SELECT github_oidc_providers.id AS id, CAST(NULL AS VARCHAR) AS discriminator, github_oidc_providers.repository_name AS repository_name, github_oidc_providers.repository_owner AS repository_owner, github_oidc_providers.repository_owner_id AS repository_owner_id, github_oidc_providers.workflow_filename AS workflow_filename, CAST(NULL AS TEXT) AS sitemap_bucket, CAST(NULL AS CITEXT) AS username, CAST(NULL AS VARCHAR(100)) AS name, CAST(NULL AS VARCHAR(128)) AS password, CAST(NULL AS TIMESTAMP WITHOUT TIME ZONE) AS password_date, CAST(NULL AS BOOLEAN) AS is_active, CAST(NULL AS BOOLEAN) AS is_frozen, CAST(NULL AS BOOLEAN) AS is_superuser, CAST(NULL AS BOOLEAN) AS is_moderator, CAST(NULL AS BOOLEAN) AS is_psf_staff, CAST(NULL AS BOOLEAN) AS prohibit_password_reset, CAST(NULL AS BOOLEAN) AS hide_avatar, CAST(NULL AS TIMESTAMP WITHOUT TIME ZONE) AS date_joined, CAST(NULL AS TIMESTAMP WITHOUT TIME ZONE) AS last_login, CAST(NULL AS disablereason) AS disabled_for, CAST(NULL AS BYTEA) AS totp_secret, CAST(NULL AS VARCHAR) AS last_totp_value, 'github_oidc_providers' AS type
FROM github_oidc_providers UNION ALL SELECT users.id AS id, CAST(NULL AS VARCHAR) AS discriminator, CAST(NULL AS VARCHAR) AS repository_name, CAST(NULL AS VARCHAR) AS repository_owner, CAST(NULL AS VARCHAR) AS repository_owner_id, CAST(NULL AS VARCHAR) AS workflow_filename, users.sitemap_bucket AS sitemap_bucket, users.username AS username, users.name AS name, users.password AS password, users.password_date AS password_date, users.is_active AS is_active, users.is_frozen AS is_frozen, users.is_superuser AS is_superuser, users.is_moderator AS is_moderator, users.is_psf_staff AS is_psf_staff, users.prohibit_password_reset AS prohibit_password_reset, users.hide_avatar AS hide_avatar, users.date_joined AS date_joined, users.last_login AS last_login, users.disabled_for AS disabled_for, users.totp_secret AS totp_secret, users.last_totp_value AS last_totp_value, 'users' AS type
FROM users) AS pjoin) AS anon_1]
(Background on this error at: https://sqlalche.me/e/14/f405)

...so I'm probably still doing something wrong here. Will continue to debug.

Edit: It also happens if I try counting any subclass, e.g. db.query(User).count().