Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 'database is locked' error #657

Closed
wants to merge 1 commit into from

Conversation

cerblue
Copy link
Contributor

@cerblue cerblue commented Jul 20, 2022

Fixes #652

Caused by concurrent access from deal goroutines to a single sqlite3 DB connection. By default, this will cause the 'database is locked' error, and it is the reason for failures in deal status updates and deal log appends. The fix comes from suggestions in https://github.com/mattn/go-sqlite3#:~:text=Error%3A%20database%20is%20locked and mattn/go-sqlite3#209.

I wrote a toy program to reproduce the error and confirmed that this change fixes the error, and does the same to my boost daemon instance.

@nonsense
Copy link
Member

nonsense commented Aug 2, 2022

Thank you for this fix @cerblue .

I think we should also add the ?cache=shared as per docs at https://github.com/mattn/go-sqlite3#faq , what do you think?


I think we need to add the ?cache=shared at two places:

diff --git a/node/modules/storageminer.go b/node/modules/storageminer.go
index 0747c80..de7256c 100644
--- a/node/modules/storageminer.go
+++ b/node/modules/storageminer.go
@@ -285,7 +285,7 @@ func StorageNetworkName(ctx helpers.MetricsCtx, a v1api.FullNode) (dtypes.Networ
 }

 func NewBoostDB(r lotus_repo.LockedRepo) (*sql.DB, error) {
-       dbPath := path.Join(r.Path(), "boost.db")
+       dbPath := path.Join(r.Path(), "boost.db?cache=shared")
        return db.SqlDB(dbPath)
 }

@@ -294,7 +294,7 @@ type LogSqlDB struct {
 }

 func NewLogsSqlDB(r repo.LockedRepo) (*LogSqlDB, error) {
-       dbPath := path.Join(r.Path(), "boost.logs.db")
+       dbPath := path.Join(r.Path(), "boost.logs.db?cache=shared")
        d, err := db.SqlDB(dbPath)
        if err != nil {
                return nil, err

@nonsense nonsense added the kind/bug Kind: Bug label Aug 2, 2022
@nonsense nonsense added this to the v1.3.0 milestone Aug 2, 2022
@jacobheun
Copy link
Contributor

Closing this in favor of #679, which includes this.

@jacobheun jacobheun closed this Aug 5, 2022
@cerblue cerblue deleted the db-error-fix branch August 19, 2022 18:26
gfanton added a commit to gfanton/berty that referenced this pull request May 16, 2023
fixes error "database is locked", caused by concurrent access from deal goroutines to a single sqlite3 db connection
see: https://github.com/mattn/go-sqlite3#:~:text=Error%3A%20database%20is%20locked
see: filecoin-project/boost#657

Signed-off-by: gfanton <8671905+gfanton@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Kind: Bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Deals randomly fail with 'failed to persist deal state: database is locked' error after publish
3 participants