Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[unimpl-ed] Show insufficient purge_zero_lamport_account logic #8176

Closed
wants to merge 1 commit into from
Closed

[unimpl-ed] Show insufficient purge_zero_lamport_account logic #8176

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions runtime/src/accounts_db.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2126,6 +2126,57 @@ pub mod tests {
assert_load_account(&accounts, current_slot, pubkey, zero_lamport);
}

#[test]
fn test_accounts_purge_chained() {
solana_logger::setup();

let some_lamport = 223;
let zero_lamport = 0;
let dummy_lamport = 999;
let no_data = 0;
let owner = Account::default().owner;

let account = Account::new(some_lamport, no_data, &owner);
let account2 = Account::new(some_lamport + 100_001, no_data, &owner);
let account3 = Account::new(some_lamport + 100_002, no_data, &owner);
let zero_lamport_account = Account::new(zero_lamport, no_data, &owner);

let pubkey = Pubkey::new_rand();
let purged_pubkey1 = Pubkey::new_rand();
let purged_pubkey2 = Pubkey::new_rand();

let dummy_account = Account::new(dummy_lamport, no_data, &owner);
let dummy_pubkey = Pubkey::default();

let accounts = AccountsDB::new_single();

let mut current_slot = 1;
accounts.store(current_slot, &[(&pubkey, &account)]);
accounts.store(current_slot, &[(&purged_pubkey1, &account2)]);
accounts.add_root(current_slot);

current_slot += 1;
accounts.store(current_slot, &[(&purged_pubkey1, &zero_lamport_account)]);
accounts.store(current_slot, &[(&purged_pubkey2, &account3)]);
accounts.add_root(current_slot);

current_slot += 1;
accounts.store(current_slot, &[(&purged_pubkey2, &zero_lamport_account)]);
Copy link
Member Author

@ryoqun ryoqun Feb 8, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem of current purge_zero_lamport_account is that it cannot recognize these chained/dependent/cascading relationship of zero lamport accounts.

In this database settings, no storage should be removed. But currently that function incorrectly removes a storage entry for this line's store. So, the old version (=not yet purged) of purged_pubkey2 re-appears incorrectly, causing this assertion failure.

Copy link
Member Author

@ryoqun ryoqun Feb 8, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not an algorithm expert at all; however the cost of these DAG dependency aware purging logic should be costly. It think that logic at least require individual checks and/or recursive checks. And it basically will be light-weight stop-the-world GC. Also, this could be DOS-friendly.

Because I couldn't come up with a simple and encouraging logic, this PR doesn't come with actual implementation...

I think we have two paths:

  1. Implement that logic rigorously.
  2. Separate non-zero account stores and zero account stores into different ones (To cut indefinitely-long chains).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sakridge I think the path 2. sounds more reasonable. Could you share your thoughts on this bug? Also, is there any better solution in your mind? :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For #2, it isn't clear to me how that exactly simplifies the problem, because don't you still have to track whether these accounts are shielding others even if they are in different stores or not?

accounts.add_root(current_slot);

current_slot += 1;
accounts.store(current_slot, &[(&dummy_pubkey, &dummy_account)]);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed because of another bug

accounts.add_root(current_slot);

purge_zero_lamport_accounts(&accounts, current_slot);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commenting out this purge_zero_lamport_accounts() invocartion makes this test pass.

let accounts = reconstruct_accounts_db_via_serialization(&accounts, current_slot);

assert_load_account(&accounts, current_slot, pubkey, some_lamport);
assert_load_account(&accounts, current_slot, purged_pubkey1, 0);
assert_load_account(&accounts, current_slot, purged_pubkey2, 0);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion fails as follows:

---- accounts_db::tests::test_accounts_purge_chained stdout ----
thread 'accounts_db::tests::test_accounts_purge_chained' panicked at 'assertion failed: `(left == right)`
  left: `(100225, 2)`,
 right: `(0, 2)`', runtime/src/accounts_db.rs:1928:9
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
  ..: ...
             ...
  13: std::panicking::begin_panic_fmt
             at src/libstd/panicking.rs:334
  14: solana_runtime::accounts_db::tests::assert_load_account
             at runtime/src/accounts_db.rs:1928
  15: solana_runtime::accounts_db::tests::test_accounts_purge_chained
             at runtime/src/accounts_db.rs:2176

This should pass, while purged_pubkey2's lamport being 0.

assert_load_account(&accounts, current_slot, dummy_pubkey, dummy_lamport);
}

#[test]
#[ignore]
fn test_store_account_stress() {
Expand Down