feat: Refactor storage using triggers #168
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## main #168 +/- ##
==========================================
- Coverage 74.37% 71.11% -3.27%
==========================================
Files 86 88 +2
Lines 3345 3389 +44
==========================================
- Hits 2488 2410 -78
- Misses 679 800 +121
- Partials 178 179 +1
Continue to review full report at Codecov.
|
gfyrag
force-pushed
the
feature/num-427-ledger-db-triggers
branch
from
17752af
to
83b4d45
Compare
gfyrag
changed the title
gfyrag
force-pushed
the
feature/num-427-ledger-db-triggers
branch
2 times, most recently
from
a7a430d
to
8eb5129
Compare
Add following tables : * log * account * volumes * balances Now, we only write on the "log" table. There is a set of triggers which are in charge of filling tables when a write is make on "log" table. Thus, the hash is now on logs instead of transactions. This way the hash take in account, metadata applied to transactions or accounts. The automatic migration fill this log table with the existing data. Since, the new code use some json function, the code must be compiled with "json1" build tag to enable support on sqlite.
gfyrag
force-pushed
the
feature/num-427-ledger-db-triggers
branch
from
8eb5129
to
f703ed6
Compare
flemzord
approved these changes
Mar 18, 2022
|
Revert this merge temporarily (updating docs beforehand as the migration process needs to be documented) |
flemzord
pushed a commit
that referenced
this pull request
Jun 7, 2023
flemzord
pushed a commit
that referenced
this pull request
Sep 20, 2023
flemzord
pushed a commit
that referenced
this pull request
Dec 4, 2023
flemzord
pushed a commit
that referenced
this pull request
Sep 25, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refactor storage.
This PR add triggers on "transactions" table to automatically compute balances and volumes of accounts in "balances" and "volumes" tables.
It also create entries in a new table called "accounts". This table will replace "addresses" table.
Now, transaction postings are directly persist as json in the transactions table, as well as metadata.
Account metadata are persisted in "accounts" table, also with json type.
Previously, the GET /accounts was not retrieving accounts which was not part of a transaction.
The new table "accounts" fix defacto this issue.
Also, there is another new table, called "log" which is designed to model an event based store. The code now always write to this table, then a set of triggers generates
entries in the transaction table, or updates metadata of either accounts or transactions following the log "type" column.
The hash on transactions was removed and add on "log" table. As a result we have a complete hash chain taking in account, metadata set on transactions or accounts.
To complete the modification on hash, the "id" property of transactions has been changed from id to a string, and is set to be a uuid by the code.
Old transactions will not have a uuid , just a int to string convert is done by the migration, but since the id is an opaque string, it should not cause any problem. A column named "ord" has been added to keep transactions ordering without the need to read all logs. It is generate by triggers, and not by the code.