Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After recovery on corrupted log, existing values might get overriden after unrelated commit processing #145

Closed
Tpt opened this issue Oct 12, 2022 · 0 comments · Fixed by #148
Labels
bug Something isn't working

Comments

@Tpt
Copy link
Contributor

Tpt commented Oct 12, 2022

Senario:

  1. The database is created with reference-counting.
  2. A first commit C1 is done, inserting a key K1, the commit is processed and the log is flushed
  3. A first commit C2 is done, inserting a key K2 is inserted, and the commit is processed.
  4. A third commit C3 is done (might be empty), but the I/Os fail during commit processing.
  5. The database is closed and open again. The second log file is invalid and removed.
  6. The database is closed and open a second time. K1 is still present in the DB.
  7. A fourth commit C4 is done and processed. When writing a Key mismatch at 2 log is printed.
  8. K1 is not in the database anymore.

As a test:

	#[test]
	fn test_from_fuzzer() {
		let tmp = tempdir().unwrap();
		let mut options = Options::with_columns(tmp.path(), 1);
		options.columns[0].ref_counted = true;
		options.columns[0].preimage = true;
		options.always_flush = true;
		options.with_background_thread = false;

		{
			let db = Db::open_or_create(&options).unwrap();
			db.commit_changes(vec![(0, Operation::Set(vec![59], vec![59]))]).unwrap();
			db.process_commits().unwrap();
			db.flush_logs().unwrap();
			db.commit_changes(vec![(0, Operation::Set(vec![0], vec![0]))]).unwrap();
			db.process_commits().unwrap();
			db.commit_changes(vec![]).unwrap();
			crate::set_number_of_allowed_io_operations(3);
			assert!(db.process_commits().is_err()); // Partial write
			crate::set_number_of_allowed_io_operations(usize::MAX);
		}

		{
			let db = Db::open(&options).unwrap();
			assert!(db.get(0, &[0]).unwrap().is_some()) // Key 0 is here
		}

		{
			let db = Db::open(&options).unwrap();
			assert!(db.get(0, &[0]).unwrap().is_some()); // Key 0 is here
			db.commit_changes(vec![(0, Operation::Set(vec![196], vec![196]))])
			.unwrap();
			db.process_commits().unwrap();
			assert!(db.get(0, &[0]).unwrap().is_none()) // Key 0 has been removed
		}
	}

Before C4 the database content is:

Index key: [22, 1f, f2, 5f, 13, 93, be, df, 22, cc, fd, 8e, 6b, 6b, bf, e1, bf, d6, 3f, 3c, 27, 2a, 9c, 1f, db, 53, be, a3, 18, 15, 32, e0]
        Rc: 1
Value: 3b
Index key: [f3, 71, 9b, 9e, 9f, be, 4, e3, 5f, 25, 5c, d7, a9, 29, c5, 24, d9, 8d, b1, 21, 19, 2f, 64, 6e, 52, c3, 3b, 9b, 1b, d8, 6d, 6f]
        Rc: 1
Value: 00

And after:

for 65536 chunks of column 0
Index key: [22, 1f, f2, 5f, 13, 93, be, df, 22, cc, fd, 8e, 6b, 6b, bf, e1, bf, d6, 3f, 3c, 27, 2a, 9c, 1f, db, 53, be, a3, 18, 15, 32, e0]
        Rc: 1
parity_db::column] Value: 3b
Index key: [a5, fa, ce, bb, 91, 75, b7, d7, c5, 4b, de, f6, 65, e1, a5, 2d, 82, b2, 29, 33, 2c, 60, ab, 29, ab, 57, 8, 15, b2, dc, c6, ed]
        Rc: 1
Value: c4
Index key: [f3, 71, 9b, 9e, 9f, be, b7, d7, c5, 4b, de, f6, 65, e1, a5, 2d, 82, b2, 29, 33, 2c, 60, ab, 29, ab, 57, 8, 15, b2, dc, c6, ed]
        Rc: 1
Value: c4

It seems like somehow the (0, 0) key-value partially overridden by the (196-196) key-value.

@Tpt Tpt added the bug Something isn't working label Oct 12, 2022
@Tpt Tpt changed the title After recovery on corrupted log, existing values might get override with unrelated commit processing After recovery on corrupted log, existing values might get overriden after unrelated commit processing Oct 12, 2022
Tpt added a commit to Tpt/parity-db that referenced this issue Oct 14, 2022
Allows to make sure no old log is staying after proper recovery

Fixes paritytech#145
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant