Zipline v3 ingest fails due to SQLAlchemy 2.0 not releasing database connection

[RichardDale]

By default, SQLAlchemy v2 now uses connection pooling to improve performance for subsequent database connections.

However, the side effect of this is that the connection is not released for some time, and this interferes with Zipline's ingestion process whereby a temporary directory and database is created and then subsequently copied over, and the cleanup removes the temporary directory. The database file is still open and causes the temporary folder cleanup routines to fail on certain operating systems (eg. Windows).

Easy solution: Don't use connection pooling

utils/sqlite_utils.py

Add this at line 19:

from sqlalchemy.pool import NullPool

Line 46, change from:

    return sa.create_engine("sqlite:///" + path)

to:

    return sa.create_engine("sqlite:///" + path, poolclass=NullPool)

[stefan-jansen]

Closed by #210.

[RichardDale]

@stefan-jansen Could you please release this urgent fix to conda-forge?