Skip to content
This repository has been archived by the owner on Aug 18, 2020. It is now read-only.

CDEC-494: Part two of block consolidation #3491

Merged
merged 3 commits into from
Aug 29, 2018
Merged

CDEC-494: Part two of block consolidation #3491

merged 3 commits into from
Aug 29, 2018

Conversation

erikd
Copy link
Member

@erikd erikd commented Aug 27, 2018

Description

This is probably not documented as well as it could be. Please point out things that you think need documenting.

This PR adds new code, but does not drastically change the operation of the code, but adds a shim layer to the old block storage so that when consolidation is activated, blocks can be retrieved from either the consolidated epoch/index files (if they have been consolidated) or the blund files if they have not.

The consolidation code is really difficult to test because it needs a huge about or environment that is really difficult to set up. I do have a separate program to test consolidation and it does work, and it is possible to retrieve blunds/block/undos that have been consolidated.

Linked issue

https://iohk.myjetbrains.com/youtrack/issue/CDEC-494

Type of change

  • 🐞 Bug fix (non-breaking change which fixes an issue)
  • 🛠 New feature (non-breaking change which adds functionality)
  • ⚠️ Breaking change (fix or feature that would cause existing functionality to change)
  • 🏭 Refactoring that does not change existing functionality but does improve things like code readability, structure etc
  • 🔨 New or improved tests for existing code
  • ⛑ git-flow chore (backport, hotfix, etc)

Developer checklist

  • I have read the style guide document, and my code follows the code style of this project.
  • If my code deals with exceptions, it follows the guidelines.
  • I have updated any documentation accordingly, if needed. Documentation changes can be reflected in opening a PR on cardanodocs.com, amending the inline Haddock comments, any relevant README file or one of the document listed in the docs directory.
  • CHANGELOG entry has been added and is linked to the correct PR on GitHub.

Testing checklist

  • I have added tests to cover my changes.
  • All new and existing tests passed.

QA Steps

With the code in this PR:

  • Made sure the cardano-node could sync to mainnet from the original genesis block.
  • Ran a separate test program to test the block consolidation code.

Screenshots (if available)

@erikd erikd force-pushed the erikd/CDEC-494-2 branch 2 times, most recently from 3c7d1ec to 7189a4f Compare August 28, 2018 03:39
@erikd erikd requested a review from mhuesch August 28, 2018 06:02
Copy link
Contributor

@ruhatch ruhatch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really good 👍


-- This type is a pain in the neck, but we have to live with it forever
-- because this encoding is baked into the system. The best we can do is
-- provide helper functions to minimise the pain.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain the problems with using the type a bit further (here rather than in the code). How is it baked into the system? Couldn't we have an alternative in Haskell-land that is encoded the same way in the blocks?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Further comments added.

-- Consolidate from the check point until the specified 'endEpoch'. If
-- the 'endEpoch' has already been consolidated, return success.
-- On each loop, the check point in the 'MiscDB' is updated.
consoldateLoop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be consolidateLoop!

lift $ mapM_ deleteOldBlund sihs

when (ccpEpochIndex nextCcp < endEpoch) $
loop nextCcp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat :)

@erikd erikd merged commit f4d550d into develop Aug 29, 2018
@erikd erikd deleted the erikd/CDEC-494-2 branch August 29, 2018 02:04
erikd added a commit that referenced this pull request Aug 29, 2018
The changes in PRs #3491 and #3482 turned out to be incompatible.
To avoind things like this, it would be ideal if PRs were not
merged unless they were rebased and had a complete CI build first.
@erikd erikd mentioned this pull request Aug 29, 2018
12 tasks
erikd added a commit that referenced this pull request Aug 29, 2018
The changes in PRs #3491 and #3482 turned out to be incompatible.
To avoind things like this, it would be ideal if PRs were not
merged unless they were rebased and had a complete CI build first.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants