Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intermittent sqlite error after upgrade from 0.5.5 to 0.5.7 #1419

Closed
06chaynes opened this issue Sep 1, 2021 · 14 comments · Fixed by #1450
Closed

Intermittent sqlite error after upgrade from 0.5.5 to 0.5.7 #1419

06chaynes opened this issue Sep 1, 2021 · 14 comments · Fixed by #1450
Labels
bug db:sqlite Related to SQLite

Comments

@06chaynes
Copy link

06chaynes commented Sep 1, 2021

I am seeing an odd intermittent issue after upgrading from version 0.5.5. I have a number of queries using the sqlx::query function that run in sequence and after the upgrade I see the following error seemingly at random:

thread '<unnamed>' panicked at 'unable to close due to unfinalized statements or unfinished backups', .cargo/registry/src/github.com-1ecc6299db9ec823/sqlx-core-0.5.7/src/sqlite/connection/handle.rs:68:17

If I downgrade back to version 0.5.5 I no longer experience this issue. I have the following features enabled: "runtime-tokio-native-tls", "sqlite", "migrate". Version of SQLite is 3.31.1-4ubuntu0.2 amd64, running on Ubuntu Focal.

For now running version 0.5.5 allows me to continue so it's not critical but I wanted to make sure an issue was opened. If any more details are required, or if I can help troubleshoot further please let me know what's needed and I'd be happy to help if possible.

@abonander abonander added db:sqlite Related to SQLite bug labels Sep 1, 2021
@abonander
Copy link
Collaborator

I actually have no idea what's causing this. Obviously it's a ConnectionHandle getting dropped while some StatementHandle still exists, but I have no idea how.

For one, we make sure to clear all the virtual statements in SqliteConnection::drop(): https://github.com/launchbadge/sqlx/blob/master/sqlx-core/src/sqlite/connection/mod.rs#L105

The comment reminded me that drop order is not defined, but either way it shouldn't be an issue:

If handle: ConnectionHandle is dropped first, that should be fine, as that uses an Arc and sends a strong reference to the statement worker thread to make sure it's dropped last: https://github.com/launchbadge/sqlx/blob/master/sqlx-core/src/sqlite/statement/worker.rs#L79

The crossbeam-channel docs specify that even if all the senders have gone away (so the Sender in the StatementWorker struct getting dropped), the receiver should still be able to empty the channel of messages and thus work through all the remaining StatementHandles that might be queued.

I could see this panic happening if for some reason the thread we spawn in StatementWorker::new() dies prematurely, as that would leave StatementHandles in the channel, and then ConnectionHandle getting dropped before StatementWorker will trigger the panic.

However, the only way the statement worker thread would die first without consuming all the messages in the channel (barring a bug in crossbeam-channel) is by itself panicking; can you look back through your logs to find a panic with a different message?

@06chaynes
Copy link
Author

I checked again to be sure and this is the only error I see at all unfortunately. I am going to try to recreate this in a stripped down project to make troubleshooting easier. I will update if I find anything out from that.

@abonander
Copy link
Collaborator

@06chaynes the full panic backtrace may also help. It couldn't hurt, at least.

@06chaynes
Copy link
Author

thread '<unnamed>' panicked at 'unable to close due to unfinalized statements or unfinished backups', /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/sqlx-core-0.5.7/src/sqlite/connection/handle.rs:68:17
stack backtrace:
   0:     0x558300684bfa - std::backtrace_rs::backtrace::libunwind::trace::h34055254b57d8e79
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
   1:     0x558300684bfa - std::backtrace_rs::backtrace::trace_unsynchronized::h8f1e3fbd9afff6ec
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x558300684bfa - std::sys_common::backtrace::_print_fmt::h3a99a796b770c360
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x558300684bfa - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h32d1f94a80615d18
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/sys_common/backtrace.rs:46:22
   4:     0x5583004f8ecc - core::fmt::write::h306731c068f7162c
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/fmt/mod.rs:1110:17
   5:     0x558300683644 - std::io::Write::write_fmt::hd2fa90334eee2a21
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/io/mod.rs:1588:15
   6:     0x558300683f25 - std::sys_common::backtrace::_print::h5abaa2601a852287
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/sys_common/backtrace.rs:49:5
   7:     0x558300683f25 - std::sys_common::backtrace::print::h8d81445442bb638f
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/sys_common/backtrace.rs:36:9
   8:     0x558300683f25 - std::panicking::default_hook::{{closure}}::hcfe804496a9fa747
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:208:50
   9:     0x558300683436 - std::panicking::default_hook::hbea8e3ccf2ba8901
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:225:9
  10:     0x558300683436 - std::panicking::rust_panic_with_hook::h7ee9e1a2d0f8975a
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:622:17
  11:     0x55830069c0cd - std::panicking::begin_panic_handler::{{closure}}::h8ab3b4491718b2c7
  12:     0x55830069c03c - std::sys_common::backtrace::__rust_end_short_backtrace::hd489062ffa586a9f
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/sys_common/backtrace.rs:141:18
  13:     0x55830069bfed - rust_begin_unwind
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:515:5
  14:     0x55830042bca0 - std::panicking::begin_panic_fmt::h5479575e112541f8
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:457:5
  15:     0x558300672a04 - alloc::sync::Arc<T>::drop_slow::h5e9b78887050d417
  16:     0x55830067a4b9 - std::sys_common::backtrace::__rust_begin_short_backtrace::h5e11ff00e096ce1d
  17:     0x558300678691 - core::ops::function::FnOnce::call_once{{vtable.shim}}::hafdf47ca0c764638
  18:     0x5583006ad6d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h7ece6cfefaff1005
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/alloc/src/boxed.rs:1575:9
  19:     0x5583006ad6d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hb8b48e55c21f193e
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/alloc/src/boxed.rs:1575:9
  20:     0x5583006ad6d5 - std::sys::unix::thread::Thread::new::thread_start::h8c7c4450dba62914
                               at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/sys/unix/thread.rs:71:17
  21:     0x7f04d3330609 - start_thread
  22:     0x7f04d3102293 - clone
  23:                0x0 - <unknown>
Aborted (core dumped)

@abonander
Copy link
Collaborator

abonander commented Sep 1, 2021

Frustratingly, that doesn't actually show where in non-stdlib code that the panic occurred. It looks like a bunch of inlining has happened here. Considering how shallow the stack is, though, I'm guessing that that is the worker thread panicking when it drops the ConnectionHandleRef: https://github.com/launchbadge/sqlx/blob/master/sqlx-core/src/sqlite/statement/worker.rs#L79

I'm still at a loss as to how this is happening, though. Any progress on that minimal reproduction? It would help to see how you're actually using SQLx.

@06chaynes
Copy link
Author

Before I share this really strange looking example think I should share a bit of what led up to this. I started out with a sqlite pool limited to 1 connection but was running into issues, specifically I was seeing connection timeouts. This had me kind of stumped because it seemed no matter what I tried it would run into this connection error a crazy number of times.

So I modified things a bit more and ended up wrapping a SqliteConnection in a mutex and that in an arc, then would clone and wait for an owned lock, do the query, then drop that clone after it finished. This ended up working, though I did notice that after weeks of continuous operation I would see the connection timeout error a few times on some days. This program cycles every 30 seconds, so having an error that rarely isn't an issue for what we are doing.

After moving past v0.5.5 though I started getting the new error. This example can recreate the error at least, https://github.com/06chaynes/sqlxerr, not quite what I'm doing but apparently close enough!

@abonander
Copy link
Collaborator

abonander commented Sep 2, 2021

With your example, I figured it out!

The issue is the books: Vec<SqliteRow>, as SqliteRow retains a strong reference to the StatementHandle which finalizes the inner sqlite3_stmt on-drop.

Since you're prematurely dropping the SqliteConnection with drop(conn), it's trying to finalize the sqlite3 instance before it finalizes the Vec<SqliteRow> and thus the sqlite3_stmts contained therin (as the normal drop order in a function is LIFO).

As a temporary fix on your end, make sure to drop books before conn. With that change, the panic no longer happens in your example.

The permanent fix here on our end is probably to retain a ConnectionHandleRef in the SqliteRow struct so the connection cannot be finalized prematurely, or even just in StatementHandle itself, maybe.

@06chaynes
Copy link
Author

Ah! I didn't consider that at all, thank you. I'll make those changes as soon as possible and test again.

abonander added a commit that referenced this issue Sep 22, 2021
abonander added a commit that referenced this issue Sep 22, 2021
abonander added a commit that referenced this issue Sep 22, 2021
abonander added a commit that referenced this issue Sep 22, 2021
…SqliteRow`s still exist (#1450)

* chore(sqlite): add repro for #1419

* fix(sqlite): hold a reference to the connection in `SqliteRow`

fixes #1419
@definitelycarter
Copy link

I hit this error on version 0.5.9.

thread '<unnamed>' panicked at 'unable to close due to unfinalized statements or unfinished backups', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/sqlx-core-0.5.9/src/sqlite/connection/handle.rs:69:17

image

@06chaynes
Copy link
Author

@definitelycarter do you have a code snippet that could reproduce the issue by chance?

placrosse pushed a commit to JetASAP/sqlx that referenced this issue Nov 19, 2021
* Fix error message about wildcard overrides (launchbadge#1276)

Co-authored-by: Austin Bonander <austin@launchbadge.com>

* feat(docs): add an FAQ (launchbadge#1319)

* doc(faq): mention how to invert `= ANY($1)`

* doc(faq): empty array cases for `ANY` and `ALL`

* doc(faq): fix wording for empty cases

* Update crc 1.8.1 -> 2.0.0 (launchbadge#1256)

* Support MACADDR in Postgres (launchbadge#1329)

* fix(mysql): implement type traits for `chrono::DateTime<Local>` (launchbadge#1335)

closes launchbadge#1222

* Keep track of column typing in SQLite EXPLAIN parsing (launchbadge#1323)

* NewRowid, Column opcodes, better pointer handling

* Implement tracking of column typing on sqlite explain parser

* fmt for sqlite column typing for explain parsing

Co-authored-by: marshoepial <marshoepial@gmail.com>

* fix(pool): reimplement pool internals with `futures-intrusive` (launchbadge#1320)

* fix(pg_money): handle negative values correctly in `PgMoney::from_decimal()` (launchbadge#1334)

closes launchbadge#1321

* fix(macros): tell the compiler about external files/env vars to watch (launchbadge#1332)

* fix(macros): tell the compiler about external files/env vars to watch

closes launchbadge#663
closes launchbadge#681

* feat(cli): add `migrate` subcommand for generating a build script

suggest embedding migrations on `sqlx migrate add` in a new project

* fix(macros): prefix generated variable names in `query_as!()` (launchbadge#1336)

closes launchbadge#1322

* Use tokio `spawn_blocking` instead of `block_in_place` (launchbadge#1333)

This fixes a panic when sharing an SQLite connection pool between tokio runtime and actix runtime

* Mark the original DatabaseError as source. (launchbadge#1197)

* Use postgres as maintenance db unless maintaining postgres itself (launchbadge#1339)

Fixes launchbadge#1283.

* Fix GitHub Actions and integration test (launchbadge#1346)

* fix test suite

* rustfmt

* need Row

* test: fix integration test scripts and update the upstream supported databases

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* ci(actions): update supported databases

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* ci(actions): use `pg_isready` instead of `sleep` to avoid error cause by database not ready

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* feat(core): add `trait PgConnectionInfo` for connection parameter status from server

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* test(postgres): fix integration test for postgres

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* test(mysql): fix integration tests

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* ci(actions): test database against the oldest and newest supported versions

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* docs(core): document `trait PgConnectionInfo`

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

Co-authored-by: Montana Low <montanalow@gmail.com>

* build(deps): bump git2 from 0.13.19 to 0.13.20 (launchbadge#1362)

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* sqlite: fix a couple segfaults (launchbadge#1351)

* sqlite: use Arc instead of Copy-able StatementHandle

This guarantees that StatementHandle is never used after calling
`sqlite3_finalize`. Now `sqlite3_finalize` is only called when
StatementHandle is dropped.

(cherry picked from commit 5eebc05)

* sqlite: use Weak poiter to StatementHandle in the worker

Otherwise some tests fail to close connection.

(cherry picked from commit 5461eee)

* Fix segfault due to race condition in sqlite (launchbadge#1300)

(cherry picked from commit bb62cf7)

* fix(sqlite): run `sqlite3_reset()` in `StatementWorker`

this avoids possible race conditions without using a mutex

* fix(sqlite): have `StatementWorker` keep a strong ref to `ConnectionHandle`

this should prevent the database handle from being finalized before all statement handles
have been finalized

* fix(sqlite/test): make `concurrent_resets_dont_segfault` runtime-agnostic

Co-authored-by: link2xt <link2xt@testrun.org>
Co-authored-by: Adam Cigánek <adam.ciganek@gmail.com>

* Add docs for fetch_all, example for postgres transactions (launchbadge#1255)

* reference fetch_all() in query macros

* add example for using transactions in postgres

* fix: Ignore __CARGO_FIX_PLZ when running "cargo metadata" in macro (launchbadge#1352)

* Fix bug for PostgreSQL if the statement has type holes (launchbadge#1363)

* fix: wait until ready after executing other helper queries while pg quering is executing

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* fix: use tls parameter for testing target

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* fix(cli): pin `clap_derive` version (launchbadge#1381)

When `clap_derive 3.0.0-beta.4` released, new invocations of `cargo install sqlx-cli` would try to compile that against `clap 3.0.0-beta.2` which caused some breakages.

Until `clap 3.0.0` proper is released, we need to pin both versions to insure against potential breakages from automatic upgrades.

closes launchbadge#1378

* cockroachdb fix for macro (launchbadge#1386)

* preparing 0.5.6 release (launchbadge#1382)

* fix(pool): reenable connection reaper

* fix warnings

* chore: bump published crates to 0.5.6

* chore: update CHANGELOG.md for 0.5.6

* chore: bump versions of sqlx-* in dependencies

* fix(macros): use `resolve_path` when getting path for `include_str!()` (launchbadge#1392)

fixes launchbadge#1387

* fix(postgres): avoid recursively spawning tasks in `PgListener::drop()` (launchbadge#1393)

refactor(pool): deprecate `PoolConnection::release()`, provide renamed alts

* chore: prepare 0.5.7 (hotfix) release (launchbadge#1394)

* fix(cli) move database_url launchbadge#1391 (launchbadge#1400)

* Use promptly instead of dialoguer (launchbadge#1410)

See launchbadge#1409

Co-authored-by: David James <davidcjames@gmail.com>

* Getting current Handle in Drop (launchbadge#1395)

Signed-off-by: Freyskeyd <simon.paitrault@gmail.com>

* Add more debugging info to unresolved type declaration panic (launchbadge#1416)

* Don't require cargo to build offline queries (launchbadge#1415)

In particular building with bazel and cargo-raze doesn't imply having
cargo at all, and deferring the lookup allows same-crate builds to
succeed with no change to semantics.

Fixes launchbadge#1414

Signed-off-by: Robert Collins <robert.collins@cognite.com>

* Add executor trait "aliases" (launchbadge#1412)

* feat: allow log level customization (launchbadge#1371)

* don't assert #[repr(...)] on `#[derive(sqlx::Type)]` unless needed (launchbadge#1305)

* Reduce futures-util features (launchbadge#1427)

* Reduce futures-util features

* Remove unused futures crate from sqlx-macros

* Fix issue launchbadge#1431 (launchbadge#1432)

Why: dotenv() must execute before clap's get_matches()

Co-authored-by: David James <davidcjames@gmail.com>

* Fix CIDR[] not being compatible with Vec<IpNetwork> (launchbadge#1433)

* Support the immutable option on SQLite connections (launchbadge#1289)

Co-authored-by: Austin Bonander <austin@launchbadge.com>

* Finish support for Postgres COPY (launchbadge#1345)

* feat(postgres): WIP implement `COPY FROM/TO STDIN`

Signed-off-by: Austin Bonander <austin@launchbadge.com>

* feat(postgres): WIP implement `COPY FROM/TO STDIN`

Signed-off-by: Austin Bonander <austin@launchbadge.com>

* test and complete support for postgres copy

Co-authored-by: Austin Bonander <austin@launchbadge.com>

* fix: run `cargo fmt`

* Support custom initial options for sqlite  (launchbadge#1295)

* Support custom initial options for sqlite

Apply suggestions from code review

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>

Apply suggestions from code review

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>

Use order-preserving map to set pragmas for an initial sqlite statement

 Use Cow<'static, str> instead of String

Co-authored-by: Austin Bonander <austin@launchbadge.com>

* docs: added a note about MySQL syntax in the README example (launchbadge#1445)

* fix(mysql): handle multiple waiting results correctly (launchbadge#1439)

* test(mysql): add test case for pending rows and dropped transaction

* fix(mysql): handle multiple waiting results correctly

* Fix a panic in the worker thread when dropping the connection while `SqliteRow`s still exist (launchbadge#1450)

* chore(sqlite): add repro for launchbadge#1419

* fix(sqlite): hold a reference to the connection in `SqliteRow`

fixes launchbadge#1419

* Upgrade hmac to 0.11 (launchbadge#1443)

* Update postgres 14 test to 14rc1 (launchbadge#1454)

* Shut down statement worker in Sqlite Connection::close (launchbadge#1453)

* add explicit shutdown of sqlite statement worker in Connection::close()

Signed-off-by: Andrew Whitehead <cywolf@gmail.com>

* test sqlite database close method

Signed-off-by: Andrew Whitehead <cywolf@gmail.com>

* await worker shutdown after dropping SqliteConnection

Signed-off-by: Andrew Whitehead <cywolf@gmail.com>

* restore explicit drop

Signed-off-by: Andrew Whitehead <cywolf@gmail.com>

* Revert "feat: allow log level customization (launchbadge#1371)" (launchbadge#1465)

This reverts commit 719d800.

* [SQLite] encoding & decoding `NaiveTime` with correct format (launchbadge#1459)

* Fix SQLite encoding format

* Update SQLite decoding format

* Update sqlx-core/src/sqlite/types/chrono.rs

* fixup: add `#[rustfmt::skip]`

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>
Co-authored-by: Austin Bonander <austin@launchbadge.com>

* Add reverting instructions to README (launchbadge#1468)

Instructions on reverting migrations are missing from the README. Here are some that others may find helpful.

* preparing 0.5.8 release (launchbadge#1466)

* preparing 0.5.8 release

* fix warnings before release

* prepare 0.5.9 hotfix release (launchbadge#1469)

* Update README.md (launchbadge#1479)

* fix(test): fix mismatched type error in MySQL type tests (launchbadge#1517)

* fix new warning about trailing semicolon in expression macros

* fix(test): fix mismatched type error in MySQL type tests

* Add support for serialized threading mode to sqlite (launchbadge#1514)

* Add support for serialized threading mode

* Typos

* Fix build

* Add persistent setter (launchbadge#1503) (launchbadge#1508)

* Reduce indexmap version to 1.6.2 (launchbadge#1501)

deno_core currently uses `=1.6.2` for indexmap, and Cargo refuses to resolve a version with sqlx's requirement of 1.7.0.

https://github.com/denoland/deno/blob/004d07dccd69c9fae8370665632d90401f770938/core/Cargo.toml#L18

* fix(postgres): allow rust_decimal::Decimal in PgRange (launchbadge#1523)

* fix(postgres): allow rust_decimal::Decimal in PgRange

* test(postgres): add tests for BigDecimal and Decimal in ranges

* Update FAQ to include building on docs.rs (launchbadge#1497)

Thanks to @jplatte in the sqlx Discord for providing this solution.

* docs: revise `Pool` docs in a couple places (launchbadge#1526)

* derive Clone, Copy for AnyKind (launchbadge#1474)

* fix panic when converting negative chrono::Duration into PgInterval (launchbadge#1475)

* fix PgInterval convert failure for negative chrono::Duration

* add unit tests for PgInterval

* Fix: remove redundancy because nanosecond overflow implies microsecond overflow

* Encode/Decode impl for Cow<'_, str> (launchbadge#1343)

* Encode/Decode impl for Cow<'_, str>

resolves launchbadge#1214

* --wip-- [skip ci]

* Add Cow decode/encode to other databases and fix build

* Include a reference to the shape of the migration files in the migrate macro documentation (launchbadge#1498)

* fix(readme): link to `dotenv` crate

closes launchbadge#1405

* fix(readme): show runtime features in quickstart

closes launchbadge#1492

* fix(readme): mention `offline` feature

closes launchbadge#1490

* fix(readme): mention examples

* fix(sqlx-cli/readme): mention `openssl-vendored`

* Copy API improvement (launchbadge#1536) (launchbadge#1537)

* expose PgCopyIn

* downgrade Pool<Posgres> copy_in_raw/out_row to take &self

* bump libsqlite3-sys to 0.23.1 (launchbadge#1535)

Co-authored-by: Jonas Platte <jplatte@users.noreply.github.com>
Co-authored-by: Austin Bonander <austin@launchbadge.com>
Co-authored-by: Akhil Velagapudi <4@4khil.com>
Co-authored-by: nomick <michael.rossberg@tu-ilmenau.de>
Co-authored-by: marshoepial <54693067+marshoepial@users.noreply.github.com>
Co-authored-by: marshoepial <marshoepial@gmail.com>
Co-authored-by: guylapid <53928652+guylapid@users.noreply.github.com>
Co-authored-by: Luca Palmieri <lucapalmieri1993@hotmail.com>
Co-authored-by: Daniel Faust <hessijames@mailbox.org>
Co-authored-by: Atkins <atkinschang@gmail.com>
Co-authored-by: Montana Low <montanalow@gmail.com>
Co-authored-by: link2xt <link2xt@testrun.org>
Co-authored-by: Adam Cigánek <adam.ciganek@gmail.com>
Co-authored-by: Rafael Epplée <raffomania@users.noreply.github.com>
Co-authored-by: Fredrik Østrem <1686349+frxstrem@users.noreply.github.com>
Co-authored-by: Altan Özlü <altanozlu7@gmail.com>
Co-authored-by: Evan Cameron <cameron.evan@gmail.com>
Co-authored-by: David James <4576329+xpe@users.noreply.github.com>
Co-authored-by: David James <davidcjames@gmail.com>
Co-authored-by: Simon Paitrault <freyskeyd@gmail.com>
Co-authored-by: Robert Collins <robertc@robertcollins.net>
Co-authored-by: Elise <elise@headpat.services>
Co-authored-by: Emil Gardström <emil.gardstrom@gmail.com>
Co-authored-by: Paolo Barbolini <paolo@paolo565.org>
Co-authored-by: David Marcin <david@metawork.com>
Co-authored-by: Montana Low <montanalow@users.noreply.github.com>
Co-authored-by: Ghass Mo <45806536+ghassmo@users.noreply.github.com>
Co-authored-by: Vlad Frolov <frolvlad@gmail.com>
Co-authored-by: Kohei Suzuki <github@wanko.cc>
Co-authored-by: Andrew Whitehead <cywolf@gmail.com>
Co-authored-by: Billy Chan <30400950+billy1624@users.noreply.github.com>
Co-authored-by: Austin Bonander <austin.bonander@gmail.com>
Co-authored-by: Erik <eraker@gmail.com>
Co-authored-by: Yerkebulan Tulibergenov <yerkebulan@gmail.com>
Co-authored-by: Lucille Blumire <llblumire@gmail.com>
Co-authored-by: Akira Hayakawa <ruby.wktk@gmail.com>
Co-authored-by: Daniel Imfeld <daniel@imfeld.dev>
Co-authored-by: meh <meh@schizofreni.co>
Co-authored-by: Russ Weas <russweas@gmail.com>
Co-authored-by: yuyawk <46620009+yuyawk@users.noreply.github.com>
Co-authored-by: Kirill Mironov <vetrokm@gmail.com>
Co-authored-by: Zbigniew Żołnierowicz <zbigniew.zolnierowicz@gmail.com>
Co-authored-by: Akira Hayakawa <akira.hayakawa@supership.jp>
Co-authored-by: David Yamnitsky <david@yamnitsky.com>
placrosse pushed a commit to JetASAP/sqlx that referenced this issue Nov 20, 2021
* release: v0.5.4

* Fix infinite compile loop regression from recursive Lazy reference

* release: v0.5.5

* Improve root README.md and sqlx-cli/README.md (launchbadge#1262)

* readme: Fix inconsistent list style

* readme: Improve text alignment

* readme: Fix missing links

* readme: Consistently use code formatting for runtime & TLS crates and dedup links

* readme: Add SQLx is not an ORM section

* readme: Improve documentation about offline mode

* Rename _expr to expr (launchbadge#1264)

* Fix error message about wildcard overrides (launchbadge#1276)

Co-authored-by: Austin Bonander <austin@launchbadge.com>

* Fix error message about wildcard overrides (launchbadge#1276) (#8)

Co-authored-by: Austin Bonander <austin@launchbadge.com>

Co-authored-by: Jonas Platte <jplatte@users.noreply.github.com>
Co-authored-by: Austin Bonander <austin@launchbadge.com>

* Fix error message about wildcard overrides (launchbadge#1276) (#10)

Co-authored-by: Austin Bonander <austin@launchbadge.com>

Co-authored-by: Jonas Platte <jplatte@users.noreply.github.com>
Co-authored-by: Austin Bonander <austin@launchbadge.com>

* feat(docs): add an FAQ (launchbadge#1319)

* doc(faq): mention how to invert `= ANY($1)`

* doc(faq): empty array cases for `ANY` and `ALL`

* doc(faq): fix wording for empty cases

* Update crc 1.8.1 -> 2.0.0 (launchbadge#1256)

* Support MACADDR in Postgres (launchbadge#1329)

* fix(mysql): implement type traits for `chrono::DateTime<Local>` (launchbadge#1335)

closes launchbadge#1222

* Keep track of column typing in SQLite EXPLAIN parsing (launchbadge#1323)

* NewRowid, Column opcodes, better pointer handling

* Implement tracking of column typing on sqlite explain parser

* fmt for sqlite column typing for explain parsing

Co-authored-by: marshoepial <marshoepial@gmail.com>

* fix(pool): reimplement pool internals with `futures-intrusive` (launchbadge#1320)

* fix(pg_money): handle negative values correctly in `PgMoney::from_decimal()` (launchbadge#1334)

closes launchbadge#1321

* fix(macros): tell the compiler about external files/env vars to watch (launchbadge#1332)

* fix(macros): tell the compiler about external files/env vars to watch

closes launchbadge#663
closes launchbadge#681

* feat(cli): add `migrate` subcommand for generating a build script

suggest embedding migrations on `sqlx migrate add` in a new project

* fix(macros): prefix generated variable names in `query_as!()` (launchbadge#1336)

closes launchbadge#1322

* Use tokio `spawn_blocking` instead of `block_in_place` (launchbadge#1333)

This fixes a panic when sharing an SQLite connection pool between tokio runtime and actix runtime

* Mark the original DatabaseError as source. (launchbadge#1197)

* Use postgres as maintenance db unless maintaining postgres itself (launchbadge#1339)

Fixes launchbadge#1283.

* Fix GitHub Actions and integration test (launchbadge#1346)

* fix test suite

* rustfmt

* need Row

* test: fix integration test scripts and update the upstream supported databases

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* ci(actions): update supported databases

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* ci(actions): use `pg_isready` instead of `sleep` to avoid error cause by database not ready

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* feat(core): add `trait PgConnectionInfo` for connection parameter status from server

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* test(postgres): fix integration test for postgres

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* test(mysql): fix integration tests

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* ci(actions): test database against the oldest and newest supported versions

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* docs(core): document `trait PgConnectionInfo`

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

Co-authored-by: Montana Low <montanalow@gmail.com>

* build(deps): bump git2 from 0.13.19 to 0.13.20 (launchbadge#1362)

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* sqlite: fix a couple segfaults (launchbadge#1351)

* sqlite: use Arc instead of Copy-able StatementHandle

This guarantees that StatementHandle is never used after calling
`sqlite3_finalize`. Now `sqlite3_finalize` is only called when
StatementHandle is dropped.

(cherry picked from commit 5eebc05)

* sqlite: use Weak poiter to StatementHandle in the worker

Otherwise some tests fail to close connection.

(cherry picked from commit 5461eee)

* Fix segfault due to race condition in sqlite (launchbadge#1300)

(cherry picked from commit bb62cf7)

* fix(sqlite): run `sqlite3_reset()` in `StatementWorker`

this avoids possible race conditions without using a mutex

* fix(sqlite): have `StatementWorker` keep a strong ref to `ConnectionHandle`

this should prevent the database handle from being finalized before all statement handles
have been finalized

* fix(sqlite/test): make `concurrent_resets_dont_segfault` runtime-agnostic

Co-authored-by: link2xt <link2xt@testrun.org>
Co-authored-by: Adam Cigánek <adam.ciganek@gmail.com>

* Add docs for fetch_all, example for postgres transactions (launchbadge#1255)

* reference fetch_all() in query macros

* add example for using transactions in postgres

* fix: Ignore __CARGO_FIX_PLZ when running "cargo metadata" in macro (launchbadge#1352)

* Fix bug for PostgreSQL if the statement has type holes (launchbadge#1363)

* fix: wait until ready after executing other helper queries while pg quering is executing

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* fix: use tls parameter for testing target

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* fix(cli): pin `clap_derive` version (launchbadge#1381)

When `clap_derive 3.0.0-beta.4` released, new invocations of `cargo install sqlx-cli` would try to compile that against `clap 3.0.0-beta.2` which caused some breakages.

Until `clap 3.0.0` proper is released, we need to pin both versions to insure against potential breakages from automatic upgrades.

closes launchbadge#1378

* cockroachdb fix for macro (launchbadge#1386)

* preparing 0.5.6 release (launchbadge#1382)

* fix(pool): reenable connection reaper

* fix warnings

* chore: bump published crates to 0.5.6

* chore: update CHANGELOG.md for 0.5.6

* chore: bump versions of sqlx-* in dependencies

* fix(macros): use `resolve_path` when getting path for `include_str!()` (launchbadge#1392)

fixes launchbadge#1387

* fix(postgres): avoid recursively spawning tasks in `PgListener::drop()` (launchbadge#1393)

refactor(pool): deprecate `PoolConnection::release()`, provide renamed alts

* chore: prepare 0.5.7 (hotfix) release (launchbadge#1394)

* fix(cli) move database_url launchbadge#1391 (launchbadge#1400)

* Use promptly instead of dialoguer (launchbadge#1410)

See launchbadge#1409

Co-authored-by: David James <davidcjames@gmail.com>

* Getting current Handle in Drop (launchbadge#1395)

Signed-off-by: Freyskeyd <simon.paitrault@gmail.com>

* Add more debugging info to unresolved type declaration panic (launchbadge#1416)

* Don't require cargo to build offline queries (launchbadge#1415)

In particular building with bazel and cargo-raze doesn't imply having
cargo at all, and deferring the lookup allows same-crate builds to
succeed with no change to semantics.

Fixes launchbadge#1414

Signed-off-by: Robert Collins <robert.collins@cognite.com>

* Add executor trait "aliases" (launchbadge#1412)

* feat: allow log level customization (launchbadge#1371)

* don't assert #[repr(...)] on `#[derive(sqlx::Type)]` unless needed (launchbadge#1305)

* Reduce futures-util features (launchbadge#1427)

* Reduce futures-util features

* Remove unused futures crate from sqlx-macros

* Fix issue launchbadge#1431 (launchbadge#1432)

Why: dotenv() must execute before clap's get_matches()

Co-authored-by: David James <davidcjames@gmail.com>

* Fix CIDR[] not being compatible with Vec<IpNetwork> (launchbadge#1433)

* Support the immutable option on SQLite connections (launchbadge#1289)

Co-authored-by: Austin Bonander <austin@launchbadge.com>

* Finish support for Postgres COPY (launchbadge#1345)

* feat(postgres): WIP implement `COPY FROM/TO STDIN`

Signed-off-by: Austin Bonander <austin@launchbadge.com>

* feat(postgres): WIP implement `COPY FROM/TO STDIN`

Signed-off-by: Austin Bonander <austin@launchbadge.com>

* test and complete support for postgres copy

Co-authored-by: Austin Bonander <austin@launchbadge.com>

* fix: run `cargo fmt`

* Support custom initial options for sqlite  (launchbadge#1295)

* Support custom initial options for sqlite

Apply suggestions from code review

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>

Apply suggestions from code review

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>

Use order-preserving map to set pragmas for an initial sqlite statement

 Use Cow<'static, str> instead of String

Co-authored-by: Austin Bonander <austin@launchbadge.com>

* docs: added a note about MySQL syntax in the README example (launchbadge#1445)

* fix(mysql): handle multiple waiting results correctly (launchbadge#1439)

* test(mysql): add test case for pending rows and dropped transaction

* fix(mysql): handle multiple waiting results correctly

* Fix a panic in the worker thread when dropping the connection while `SqliteRow`s still exist (launchbadge#1450)

* chore(sqlite): add repro for launchbadge#1419

* fix(sqlite): hold a reference to the connection in `SqliteRow`

fixes launchbadge#1419

* Upgrade hmac to 0.11 (launchbadge#1443)

* Update postgres 14 test to 14rc1 (launchbadge#1454)

* Shut down statement worker in Sqlite Connection::close (launchbadge#1453)

* add explicit shutdown of sqlite statement worker in Connection::close()

Signed-off-by: Andrew Whitehead <cywolf@gmail.com>

* test sqlite database close method

Signed-off-by: Andrew Whitehead <cywolf@gmail.com>

* await worker shutdown after dropping SqliteConnection

Signed-off-by: Andrew Whitehead <cywolf@gmail.com>

* restore explicit drop

Signed-off-by: Andrew Whitehead <cywolf@gmail.com>

* Revert "feat: allow log level customization (launchbadge#1371)" (launchbadge#1465)

This reverts commit 719d800.

* [SQLite] encoding & decoding `NaiveTime` with correct format (launchbadge#1459)

* Fix SQLite encoding format

* Update SQLite decoding format

* Update sqlx-core/src/sqlite/types/chrono.rs

* fixup: add `#[rustfmt::skip]`

Co-authored-by: Austin Bonander <austin.bonander@gmail.com>
Co-authored-by: Austin Bonander <austin@launchbadge.com>

* Add reverting instructions to README (launchbadge#1468)

Instructions on reverting migrations are missing from the README. Here are some that others may find helpful.

* preparing 0.5.8 release (launchbadge#1466)

* preparing 0.5.8 release

* fix warnings before release

* prepare 0.5.9 hotfix release (launchbadge#1469)

* Update README.md (launchbadge#1479)

* fix(test): fix mismatched type error in MySQL type tests (launchbadge#1517)

* fix new warning about trailing semicolon in expression macros

* fix(test): fix mismatched type error in MySQL type tests

* Add support for serialized threading mode to sqlite (launchbadge#1514)

* Add support for serialized threading mode

* Typos

* Fix build

* Add persistent setter (launchbadge#1503) (launchbadge#1508)

* Reduce indexmap version to 1.6.2 (launchbadge#1501)

deno_core currently uses `=1.6.2` for indexmap, and Cargo refuses to resolve a version with sqlx's requirement of 1.7.0.

https://github.com/denoland/deno/blob/004d07dccd69c9fae8370665632d90401f770938/core/Cargo.toml#L18

* fix(postgres): allow rust_decimal::Decimal in PgRange (launchbadge#1523)

* fix(postgres): allow rust_decimal::Decimal in PgRange

* test(postgres): add tests for BigDecimal and Decimal in ranges

* Update FAQ to include building on docs.rs (launchbadge#1497)

Thanks to @jplatte in the sqlx Discord for providing this solution.

* docs: revise `Pool` docs in a couple places (launchbadge#1526)

* derive Clone, Copy for AnyKind (launchbadge#1474)

* fix panic when converting negative chrono::Duration into PgInterval (launchbadge#1475)

* fix PgInterval convert failure for negative chrono::Duration

* add unit tests for PgInterval

* Fix: remove redundancy because nanosecond overflow implies microsecond overflow

* Encode/Decode impl for Cow<'_, str> (launchbadge#1343)

* Encode/Decode impl for Cow<'_, str>

resolves launchbadge#1214

* --wip-- [skip ci]

* Add Cow decode/encode to other databases and fix build

* Include a reference to the shape of the migration files in the migrate macro documentation (launchbadge#1498)

* fix(readme): link to `dotenv` crate

closes launchbadge#1405

* fix(readme): show runtime features in quickstart

closes launchbadge#1492

* fix(readme): mention `offline` feature

closes launchbadge#1490

* fix(readme): mention examples

* fix(sqlx-cli/readme): mention `openssl-vendored`

* Copy API improvement (launchbadge#1536) (launchbadge#1537)

* expose PgCopyIn

* downgrade Pool<Posgres> copy_in_raw/out_row to take &self

* bump libsqlite3-sys to 0.23.1 (launchbadge#1535)

* add #[cfg(not(target_arch = "wasm32"))] to fix compile errors

Co-authored-by: Ryan Leckey <ryan@launchbadge.com>
Co-authored-by: toshokan <toshokan@shojigate.net>
Co-authored-by: Jonas Platte <jplatte@users.noreply.github.com>
Co-authored-by: Rohan Sharma <rhnsharma5113@gmail.com>
Co-authored-by: Austin Bonander <austin@launchbadge.com>
Co-authored-by: Akhil Velagapudi <4@4khil.com>
Co-authored-by: nomick <michael.rossberg@tu-ilmenau.de>
Co-authored-by: marshoepial <54693067+marshoepial@users.noreply.github.com>
Co-authored-by: marshoepial <marshoepial@gmail.com>
Co-authored-by: guylapid <53928652+guylapid@users.noreply.github.com>
Co-authored-by: Luca Palmieri <lucapalmieri1993@hotmail.com>
Co-authored-by: Daniel Faust <hessijames@mailbox.org>
Co-authored-by: Atkins <atkinschang@gmail.com>
Co-authored-by: Montana Low <montanalow@gmail.com>
Co-authored-by: link2xt <link2xt@testrun.org>
Co-authored-by: Adam Cigánek <adam.ciganek@gmail.com>
Co-authored-by: Rafael Epplée <raffomania@users.noreply.github.com>
Co-authored-by: Fredrik Østrem <1686349+frxstrem@users.noreply.github.com>
Co-authored-by: Altan Özlü <altanozlu7@gmail.com>
Co-authored-by: Evan Cameron <cameron.evan@gmail.com>
Co-authored-by: David James <4576329+xpe@users.noreply.github.com>
Co-authored-by: David James <davidcjames@gmail.com>
Co-authored-by: Simon Paitrault <freyskeyd@gmail.com>
Co-authored-by: Robert Collins <robertc@robertcollins.net>
Co-authored-by: Elise <elise@headpat.services>
Co-authored-by: Emil Gardström <emil.gardstrom@gmail.com>
Co-authored-by: Paolo Barbolini <paolo@paolo565.org>
Co-authored-by: David Marcin <david@metawork.com>
Co-authored-by: Montana Low <montanalow@users.noreply.github.com>
Co-authored-by: Ghass Mo <45806536+ghassmo@users.noreply.github.com>
Co-authored-by: Vlad Frolov <frolvlad@gmail.com>
Co-authored-by: Kohei Suzuki <github@wanko.cc>
Co-authored-by: Andrew Whitehead <cywolf@gmail.com>
Co-authored-by: Billy Chan <30400950+billy1624@users.noreply.github.com>
Co-authored-by: Austin Bonander <austin.bonander@gmail.com>
Co-authored-by: Erik <eraker@gmail.com>
Co-authored-by: Yerkebulan Tulibergenov <yerkebulan@gmail.com>
Co-authored-by: Lucille Blumire <llblumire@gmail.com>
Co-authored-by: Akira Hayakawa <ruby.wktk@gmail.com>
Co-authored-by: Daniel Imfeld <daniel@imfeld.dev>
Co-authored-by: meh <meh@schizofreni.co>
Co-authored-by: Russ Weas <russweas@gmail.com>
Co-authored-by: yuyawk <46620009+yuyawk@users.noreply.github.com>
Co-authored-by: Kirill Mironov <vetrokm@gmail.com>
Co-authored-by: Zbigniew Żołnierowicz <zbigniew.zolnierowicz@gmail.com>
Co-authored-by: Akira Hayakawa <akira.hayakawa@supership.jp>
Co-authored-by: David Yamnitsky <david@yamnitsky.com>
@06chaynes
Copy link
Author

06chaynes commented Nov 23, 2021

@abonander looks like I am still encountering this error after all. After your comment I had made some changes so I was surprised to see the error pop up again in the logs.

Quick question on the matter, would active_recs in this example also retain a strong reference to the StatementHandle as you mentioned before?

let mut conn = db_conn.clone().lock_owned().await;
let active_recs = sqlx::query_as::<_, WatchModel>(&q)
     .fetch_all(&mut *conn)
     .await?;
drop(conn);

WatchModel here is a struct that derives FromRow

@abonander
Copy link
Collaborator

@06chaynes it shouldn't since the rows aren't kept alive past the connection.

I also have some major internal refactors on #1551 that also address this issue as well as some segfaults.

@06chaynes
Copy link
Author

That's what I was thinking but wanted to be sure.

Sounds great, really looking forward to these changes!

@06chaynes
Copy link
Author

06chaynes commented Nov 24, 2021

Just to note, after testing your refactor against two of the issues mentioned and seeing it fixed them, I went ahead and rolled it out to a couple deployments I was seeing lock errors on. So far not a single error, things are looking good!

Edit: Did end up with a couple lock errors so will review things again soon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug db:sqlite Related to SQLite
Projects
None yet
3 participants