Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement SQL lite storage #72

Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
8bf6b8d
feat: initial scaffolding
Stefan-Ethernal Sep 24, 2024
f3b7e20
feat: insert implementation
Stefan-Ethernal Sep 24, 2024
ccc8cd6
feat: change column types based on the SQL lite supported types
Stefan-Ethernal Sep 25, 2024
ee4ed66
feat: changed schema, first version of CRUD operations implementation
Stefan-Ethernal Sep 26, 2024
0d72a77
refactor: extract common parts to helper functions
Stefan-Ethernal Sep 27, 2024
86cb156
chore: add comments
Stefan-Ethernal Sep 27, 2024
5e6f3c3
fix: linter config warning
Stefan-Ethernal Sep 27, 2024
630487a
refactor: simplify in-memory storage and fix deadlocks in case of errors
Stefan-Ethernal Sep 27, 2024
4f23f0e
feat: instantiate either in memory or sql storage
Stefan-Ethernal Sep 27, 2024
d46a33f
chore: add comment
Stefan-Ethernal Sep 27, 2024
8fedf41
test: fully covered memstorage feat minor renames
Stefan-Ethernal Sep 27, 2024
693e97d
feat: add github.com/mattn/go-sqlite3 as direct dependency
Stefan-Ethernal Sep 27, 2024
3a2b2dd
fix: adapt the DB schema, fix issues and add unit tests for sql storage
Stefan-Ethernal Sep 27, 2024
b5eaccf
feat: increase coverage
Stefan-Ethernal Sep 28, 2024
ed8276b
test: add some more unit tests
Stefan-Ethernal Sep 28, 2024
257c81e
fix: group variables
Stefan-Ethernal Sep 28, 2024
463dfcc
refactor: rename PersistenceFilename config parameter
Stefan-Ethernal Sep 30, 2024
9223da9
fix: rename helper function (address comment part 1)
Stefan-Ethernal Sep 30, 2024
c10403b
feat: introduce meddler
Stefan-Ethernal Oct 1, 2024
d344af8
chore: remove useless comments
Stefan-Ethernal Oct 1, 2024
f0604dc
fix: revert update logic to manually constructing SQL and address com…
Stefan-Ethernal Oct 1, 2024
3ea3dc7
fix: change some columns types (address comment from @joanestebanr 2n…
Stefan-Ethernal Oct 1, 2024
aa54d6a
fix: use meddler to construct on fly update statement, address commen…
Stefan-Ethernal Oct 2, 2024
8196f5c
feat: remove in memory storage, change block_number to BIGINT (addres…
Stefan-Ethernal Oct 2, 2024
40c398d
fix: remove redundant code
Stefan-Ethernal Oct 2, 2024
5077963
feat: simplify building of UPDATE clause
Stefan-Ethernal Oct 2, 2024
db40fc9
rebase fix
goran-ethernal Oct 2, 2024
c009e71
refactor: build base select and base delete statement dynamically (ad…
Stefan-Ethernal Oct 2, 2024
669a8d7
test: concurrent write to the database unit test
Stefan-Ethernal Oct 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
---
run:
timeout: 5m
skip-dirs:
- test
- mocks


linters:
enable:
- whitespace # Tool for detection of leading and trailing whitespace
Expand Down Expand Up @@ -47,6 +44,8 @@ issues:
linters:
- gosec
- lll
exclude-dirs:
- test
include:
- EXC0012 # EXC0012 revive: Annoying issue about not having a comment. The rare codebase has such comments
- EXC0014 # EXC0014 revive: Annoying issue about not having a comment. The rare codebase has such comments
Expand Down
11 changes: 2 additions & 9 deletions .mockery.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,7 @@ filename: "{{.InterfaceName | lower }}.generated.go"
mockname: "{{.InterfaceName}}"
outpkg: "mocks"
packages:
github.com/0xPolygon/zkevm-ethtx-manager/ethtxmanager:
github.com/0xPolygon/zkevm-ethtx-manager/types:
interfaces:
EthermanInterface:
config:
StorageInterface:
config:
dir: "{{.InterfaceDir}}"
filename: "{{.InterfaceName | lower }}.generated.go"
mockname: "{{.InterfaceName}}Mock"
outpkg: "{{.PackageName}}"
inpackage: True
config:
33 changes: 33 additions & 0 deletions common/common.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,41 @@
package common

import "github.com/ethereum/go-ethereum/common"

const (
// Base10 decimal base
Base10 = 10
// Gwei represents 1000000000 wei
Gwei = 1000000000

// SQLLiteDriverName is the name for the SQL lite driver
SQLLiteDriverName = "sqlite3"
)

// ToAddressPtr converts a string to a common.Address pointer or returns nil if empty.
func ToAddressPtr(addr string) *common.Address {
if addr == "" {
return nil
}

address := common.HexToAddress(addr)
return &address
}

// ToUint64Ptr is a helper to create uint64 pointer
func ToUint64Ptr(v uint64) *uint64 {
return &v
}

// SlicePtrsToSlice converts a slice of pointers to a slice of values.
func SlicePtrsToSlice[T any](ptrSlice []*T) []T {
// Create a new slice to hold the values
res := make([]T, len(ptrSlice))
// Dereference each pointer and add the value to the result slice
for i, ptr := range ptrSlice {
if ptr != nil {
res[i] = *ptr
}
}
return res
}
6 changes: 3 additions & 3 deletions ethtxmanager/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,10 +56,10 @@ type Config struct {
// max gas price limit: 110
// tx gas price = 110
MaxGasPriceLimit uint64 `mapstructure:"MaxGasPriceLimit"`
// PersistenceFilename is the filename to store the memory storage
PersistenceFilename string `mapstructure:"PersistenceFilename"`
// DBPath is the path of the SQL database
DBPath string `mapstructure:"DBPath"`
// ReadPendingL1Txs is a flag to enable the reading of pending L1 txs
// It can only be enabled if PersistenceFilename is empty
// It can only be enabled if DBPath is empty
ReadPendingL1Txs bool `mapstructure:"ReadPendingL1Txs"`
// Etherman configuration
Etherman etherman.Config `mapstructure:"Etherman"`
Expand Down
Loading
Loading