Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge stacks signers into develop #700

Draft
wants to merge 30 commits into
base: develop
Choose a base branch
from
Draft
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
0a722b3
build: run redis tests using a docker integration environment (#655)
rafaelcr Oct 16, 2024
e44d84a
feat: parse `/stackerdb_chunks` Stacks node event (#653)
rafaelcr Oct 16, 2024
c0b6bc8
chore: merge develop
rafaelcr Oct 22, 2024
ff66bb2
feat(ts-client): add signer messages to stacks payloads (#657)
rafaelcr Oct 22, 2024
6a7a42c
Merge branch 'develop' into stacks-signers
zone117x Oct 23, 2024
aee14bc
feat: include signer messages in Stacks predicate payloads (#656)
rafaelcr Oct 23, 2024
081a101
Merge branch 'develop' into stacks-signers
rafaelcr Oct 23, 2024
cc93873
fix: signer pubkey calculation (#665)
rafaelcr Oct 24, 2024
b5ad4ba
feat: include recovered signer pubkeys in new block payload (#662)
zone117x Oct 24, 2024
71364c1
fix: update SignerMessage deserializing (#663)
zone117x Oct 24, 2024
d12acd9
feat: store signer messages in local sqlite database (#664)
rafaelcr Oct 25, 2024
44bd445
ci: publish stacks signers as beta release
rafaelcr Oct 25, 2024
a682ba0
chore(release): 1.9.0-stacks-signers.1 [skip ci]
semantic-release-bot Oct 25, 2024
34db714
ci: check out the correct git tag on release
rafaelcr Oct 25, 2024
279f648
Merge branch 'stacks-signers' of github.com:hirosystems/chainhook int…
rafaelcr Oct 25, 2024
ad5fd54
feat: add mock signer message support (#669)
rafaelcr Oct 25, 2024
b3a156f
chore(release): 1.9.0-stacks-signers.2 [skip ci]
semantic-release-bot Oct 25, 2024
c414d79
feat: add mock signature pubkey (#670)
rafaelcr Oct 25, 2024
dab83e1
chore(release): 1.9.0-stacks-signers.3 [skip ci]
semantic-release-bot Oct 25, 2024
3c117b5
fix: bump stacks codec
rafaelcr Oct 25, 2024
186d337
chore(release): 1.9.0-stacks-signers.4 [skip ci]
semantic-release-bot Oct 25, 2024
8bb59c1
fix: mock signature retrieval (#671)
rafaelcr Oct 25, 2024
c90091b
chore(release): 1.9.0-stacks-signers.5 [skip ci]
semantic-release-bot Oct 25, 2024
a01470e
fix: read stacks-core http event POST payloads for ignored events (#673)
zone117x Oct 31, 2024
367362a
chore(release): 1.9.0-stacks-signers.6 [skip ci]
semantic-release-bot Oct 31, 2024
1bc949a
fix: propagate error when trying to store peer info
csgui Nov 6, 2024
cb5a7c3
Merge pull request #682 from hirosystems/chore/warn_mock_proposals
csgui Nov 7, 2024
defd86f
chore(release): 1.9.0-stacks-signers.7 [skip ci]
semantic-release-bot Nov 7, 2024
1f9f37e
fix: only import stacks tsv if chainstate is empty (#684)
rafaelcr Nov 12, 2024
7b7cdf9
chore(release): 1.9.0-stacks-signers.8 [skip ci]
semantic-release-bot Nov 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
[alias]
chainhook-install = "install --path components/chainhook-cli --locked --force --features cli --features debug --no-default-features"

[env]
RUST_TEST_THREADS = "1"
34 changes: 25 additions & 9 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -4,7 +4,9 @@ on:
push:
branches:
- main
- beta
- develop
- stacks-signers
paths-ignore:
- '**/CHANGELOG.md'
pull_request:
@@ -20,7 +22,11 @@ jobs:
strategy:
fail-fast: false
matrix:
suite: [cli, sdk]
include:
- suite: cli
features: redis_tests
- suite: sdk
features: stacks-signers
defaults:
run:
working-directory: ./components/chainhook-${{ matrix.suite }}
@@ -35,12 +41,6 @@ jobs:
rustup toolchain install stable --profile minimal
echo "RUST_VERSION_HASH=$(rustc --version | sha256sum | awk '{print $1}')" >> $GITHUB_ENV

- name: Install redis
if: matrix.suite == 'cli'
run: |
sudo apt-get install -y redis-server
echo "TARPAULIN_FLAGS=--features redis_tests" >> $GITHUB_ENV

- name: Cache cargo
uses: actions/cache@v4
with:
@@ -59,16 +59,31 @@ jobs:
run: |
cargo install cargo-tarpaulin

- name: Setup integration environment
run: |
sudo ufw disable
docker compose -f ../../dockerfiles/docker-compose.dev.yml up -d
docker compose -f ../../dockerfiles/docker-compose.dev.yml logs -t -f --no-color &> docker-compose-logs.txt &
if: matrix.suite == 'cli'

- name: Run tests
run: |
cargo tarpaulin --skip-clean --out lcov ${{ env.TARPAULIN_FLAGS }} -- --test-threads=1
cargo tarpaulin --skip-clean --out lcov --features ${{ matrix.features }} -- --test-threads=1

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v4
env:
token: ${{ secrets.CODECOV_TOKEN }}
codecov_yml_path: .github/codecov.yml

- name: Print integration environment logs
run: cat docker-compose-logs.txt
if: matrix.suite == 'cli' && failure()

- name: Teardown integration environment
run: docker compose -f ../../dockerfiles/docker-compose.dev.yml down -v -t 0
if: matrix.suite == 'cli' && always()

distributions:
runs-on: ${{ matrix.os }}

@@ -270,6 +285,7 @@ jobs:
- distributions
outputs:
new_release_version: ${{ steps.semantic.outputs.new_release_version }}
new_release_git_tag: ${{ steps.semantic.outputs.new_release_git_tag }}
steps:
- uses: actions/checkout@v4
with:
@@ -325,7 +341,7 @@ jobs:
uses: actions/checkout@v4
with:
persist-credentials: false
ref: ${{ needs.semantic-release.outputs.new_release_version }}
ref: ${{ needs.semantic-release.outputs.new_release_git_tag }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
5 changes: 2 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -19,6 +19,5 @@ components/chainhook-types-js/dist
*.redb
cache/
Chainhook.toml

components/chainhook-cli/src/service/tests/fixtures/tmp
components/chainhook-cli/src/archive/tests/fixtures/tmp
**/src/service/tests/fixtures/tmp
**/src/archive/tests/fixtures/tmp
12 changes: 11 additions & 1 deletion .releaserc
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
{
"branches": [
"main"
"main",
{
"name": "beta",
"channel": "beta",
"prerelease": true
},
{
"name": "stacks-signers",
"channel": "stacks-signers",
"prerelease": true
}
],
"plugins": [
[
46 changes: 46 additions & 0 deletions .vscode/chainhook.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
[storage]
working_dir = "cache"

# The HTTP API allows you to register / deregister
# predicates dynamically.
# This is disabled by default.
#
[http_api]
http_port = 20456
database_uri = "redis://localhost:6379/"

[network]
mode = "testnet"
bitcoind_rpc_url = "http://localhost:18443"
bitcoind_rpc_username = "btc"
bitcoind_rpc_password = "btc"

# Chainhook must be able to receive Bitcoin block events.
# These events can originate from either a Stacks node or a Bitcoin node's ZeroMQ interface.

# By default, the service is set to receive Bitcoin block events from the Stacks node:
stacks_node_rpc_url = "http://localhost:20443"
stacks_events_ingestion_port = 20455

# However, events can also be received directly from a Bitcoin node.
# To achieve this, comment out the `stacks_node_rpc_url` line and uncomment the following line:
# bitcoind_zmq_url = "tcp://0.0.0.0:18543"

[limits]
max_number_of_bitcoin_predicates = 100
max_number_of_concurrent_bitcoin_scans = 100
max_number_of_stacks_predicates = 10
max_number_of_concurrent_stacks_scans = 10
max_number_of_processing_threads = 16
max_number_of_networking_threads = 16
max_caching_memory_size_mb = 32000

# The TSV file is required for downloading historical data for your predicates.
# If this is not a requirement, you can comment out the `tsv_file_url` line.
# [[event_source]]
# tsv_file_url = "https://archive.hiro.so/regtest/stacks-blockchain-api/regtest-stacks-blockchain-api-latest"

# Enables a server that provides metrics that can be scraped by Prometheus.
# This is disabled by default.
# [monitoring]
# prometheus_monitoring_port = 20457
68 changes: 67 additions & 1 deletion .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,76 @@
{
"version": "0.2.0",
"configurations": [
{
"type": "lldb",
"request": "launch",
"name": "Debug executable 'chainhook'",
"cargo": {
"args": [
"build",
"--bin=chainhook",
"--package=chainhook"
],
"filter": {
"name": "chainhook",
"kind": "bin"
}
},
"args": [
"service",
"start",
"--config-path=${workspaceFolder}/.vscode/chainhook.toml",
],
"cwd": "${workspaceFolder}",
"preLaunchTask": "redis:start",
"postDebugTask": "redis:stop"
},
{
"type": "lldb",
"request": "launch",
"name": "test: chainhook-sdk",
"cargo": {
"args": ["test", "--no-run", "--lib", "--package=chainhook-sdk"],
"filter": {
"name": "chainhook_sdk",
"kind": "lib"
}
},
"args": [],
"env": {
"RUST_TEST_THREADS": "1"
},
"cwd": "${workspaceFolder}"
},
{
"type": "lldb",
"request": "launch",
"name": "test: chainhook-cli",
"cargo": {
"args": [
"test",
"--no-run",
"--bin=chainhook",
"--package=chainhook",
"--features=redis_tests"
],
"filter": {
"name": "chainhook",
"kind": "bin"
}
},
"args": [],
"env": {
"RUST_TEST_THREADS": "1"
},
"cwd": "${workspaceFolder}",
"preLaunchTask": "redis:start",
"postDebugTask": "redis:stop"
},
{
"type": "node",
"request": "launch",
"name": "ts client: jest",
"name": "test: typescript client",
"localRoot": "${workspaceFolder}/components/client/typescript",
"program": "${workspaceFolder}/components/client/typescript/node_modules/jest/bin/jest",
"args": [
47 changes: 47 additions & 0 deletions .vscode/tasks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "redis:start",
"type": "shell",
"command": "docker compose -f dockerfiles/docker-compose.dev.yml up --force-recreate -V",
"isBackground": true,
"problemMatcher": {
"pattern": { "regexp": ".", "file": 1, "location": 2, "message": 3 },
"background": {
"activeOnStart": true,
"beginsPattern": ".",
"endsPattern": "."
}
},
"presentation": {
"echo": true,
"reveal": "always",
"focus": false,
"panel": "dedicated",
"clear": false
}
},
{
"label": "redis:stop",
"type": "shell",
"command": "docker compose -f dockerfiles/docker-compose.dev.yml down -v -t 0",
"isBackground": true,
"problemMatcher": {
"pattern": { "regexp": ".", "file": 1, "location": 2, "message": 3 },
"background": {
"activeOnStart": true,
"beginsPattern": ".",
"endsPattern": "."
}
},
"presentation": {
"echo": true,
"reveal": "always",
"focus": false,
"panel": "dedicated",
"clear": false
}
}
]
}
73 changes: 73 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,76 @@
## [1.9.0-stacks-signers.8](https://github.com/hirosystems/chainhook/compare/v1.9.0-stacks-signers.7...v1.9.0-stacks-signers.8) (2024-11-12)


### Bug Fixes

* only import stacks tsv if chainstate is empty ([#684](https://github.com/hirosystems/chainhook/issues/684)) ([1f9f37e](https://github.com/hirosystems/chainhook/commit/1f9f37ee8c9afa864fc618b761195b0d0711dfbe)), closes [#677](https://github.com/hirosystems/chainhook/issues/677)

## [1.9.0-stacks-signers.7](https://github.com/hirosystems/chainhook/compare/v1.9.0-stacks-signers.6...v1.9.0-stacks-signers.7) (2024-11-07)


### Bug Fixes

* propagate error when trying to store peer info ([1bc949a](https://github.com/hirosystems/chainhook/commit/1bc949a6ce17ab67c2c1af2d21fe330461689f49))

## [1.9.0-stacks-signers.6](https://github.com/hirosystems/chainhook/compare/v1.9.0-stacks-signers.5...v1.9.0-stacks-signers.6) (2024-10-31)


### Bug Fixes

* read stacks-core http event POST payloads for ignored events ([#673](https://github.com/hirosystems/chainhook/issues/673)) ([a01470e](https://github.com/hirosystems/chainhook/commit/a01470e34876dcf67969063e1fc900ae947a0a54))

## [1.9.0-stacks-signers.5](https://github.com/hirosystems/chainhook/compare/v1.9.0-stacks-signers.4...v1.9.0-stacks-signers.5) (2024-10-25)


### Bug Fixes

* mock signature retrieval ([#671](https://github.com/hirosystems/chainhook/issues/671)) ([8bb59c1](https://github.com/hirosystems/chainhook/commit/8bb59c1b4727a37df47e08b7a31456bcb6528289))

## [1.9.0-stacks-signers.4](https://github.com/hirosystems/chainhook/compare/v1.9.0-stacks-signers.3...v1.9.0-stacks-signers.4) (2024-10-25)


### Bug Fixes

* bump stacks codec ([3c117b5](https://github.com/hirosystems/chainhook/commit/3c117b562d02b8b5379fd4b0f3187567d7f5289b))

## [1.9.0-stacks-signers.3](https://github.com/hirosystems/chainhook/compare/v1.9.0-stacks-signers.2...v1.9.0-stacks-signers.3) (2024-10-25)


### Features

* add mock signature pubkey ([#670](https://github.com/hirosystems/chainhook/issues/670)) ([c414d79](https://github.com/hirosystems/chainhook/commit/c414d793e9a25c1b541914fee256d98d53bc08a3))

## [1.9.0-stacks-signers.2](https://github.com/hirosystems/chainhook/compare/v1.9.0-stacks-signers.1...v1.9.0-stacks-signers.2) (2024-10-25)


### Features

* add mock signer message support ([#669](https://github.com/hirosystems/chainhook/issues/669)) ([ad5fd54](https://github.com/hirosystems/chainhook/commit/ad5fd54b3d1d8f638fa44a531bca71306fbb8c6f))

## [1.9.0-stacks-signers.1](https://github.com/hirosystems/chainhook/compare/v1.8.0...v1.9.0-stacks-signers.1) (2024-10-25)


### Features

* add various new nakamoto block fields to `/new_block` ingestion and `StacksPayload` ([#659](https://github.com/hirosystems/chainhook/issues/659)) ([f48cda5](https://github.com/hirosystems/chainhook/commit/f48cda533dfc58bb630737fb29cf1bc9c966a638))
* include recovered signer pubkeys in new block payload ([#662](https://github.com/hirosystems/chainhook/issues/662)) ([b5ad4ba](https://github.com/hirosystems/chainhook/commit/b5ad4ba11dd72722bb6cbe936ec29411cde9a606))
* include signer messages in Stacks predicate payloads ([#656](https://github.com/hirosystems/chainhook/issues/656)) ([aee14bc](https://github.com/hirosystems/chainhook/commit/aee14bc693573f403f8a6d8eafe7b30d2ca76b54))
* parse `/stackerdb_chunks` Stacks node event ([#653](https://github.com/hirosystems/chainhook/issues/653)) ([e44d84a](https://github.com/hirosystems/chainhook/commit/e44d84a0d739921a5a3ccae6e9643bdb85005f71))
* store signer messages in local sqlite database ([#664](https://github.com/hirosystems/chainhook/issues/664)) ([d12acd9](https://github.com/hirosystems/chainhook/commit/d12acd9c99ce0d0721dfcb853e5cb236e45925b7))
* support tenure-height in block messages ([#661](https://github.com/hirosystems/chainhook/issues/661)) ([99e5a61](https://github.com/hirosystems/chainhook/commit/99e5a6119fb8e87e1e14d789fdb866eae230e8e8))
* **ts-client:** add persisted predicates and periodic health check ([#658](https://github.com/hirosystems/chainhook/issues/658)) ([535226a](https://github.com/hirosystems/chainhook/commit/535226a80480585aabfcf89d68b21d2ada1c50c1))
* **ts-client:** add signer messages to stacks payloads ([#657](https://github.com/hirosystems/chainhook/issues/657)) ([ff66bb2](https://github.com/hirosystems/chainhook/commit/ff66bb2eab771c16fead154607a280664eb6be4e))


### Bug Fixes

* add a configurable max timeout for outgoing predicate payload requests ([#642](https://github.com/hirosystems/chainhook/issues/642)) ([6c1dfa9](https://github.com/hirosystems/chainhook/commit/6c1dfa9f741041d0fac32e2a89c04a71b4a485cd)), closes [#643](https://github.com/hirosystems/chainhook/issues/643)
* increase stacks event server body size limit to 500MB ([#660](https://github.com/hirosystems/chainhook/issues/660)) ([21234c9](https://github.com/hirosystems/chainhook/commit/21234c978d4c49ea5526c2cf1f74d42a645a68c8))
* return 500 status code to Stacks node when event ingestion fails ([#648](https://github.com/hirosystems/chainhook/issues/648)) ([63c753c](https://github.com/hirosystems/chainhook/commit/63c753c1a1b043e271d1ebca1f90f154030a89e8)), closes [#646](https://github.com/hirosystems/chainhook/issues/646)
* signer pubkey calculation ([#665](https://github.com/hirosystems/chainhook/issues/665)) ([cc93873](https://github.com/hirosystems/chainhook/commit/cc93873dacc88e5df97d1006394b99744e63009b))
* update SignerMessage deserializing ([#663](https://github.com/hirosystems/chainhook/issues/663)) ([71364c1](https://github.com/hirosystems/chainhook/commit/71364c15a88a06d5c71b3f3d3c88c489ff4733d1))
* validate ts client predicates before registering ([#639](https://github.com/hirosystems/chainhook/issues/639)) ([2d08f72](https://github.com/hirosystems/chainhook/commit/2d08f72e44446e9c311f8a64f992e103f17a4ae7))

## [1.8.0](https://github.com/hirosystems/chainhook/compare/v1.7.0...v1.8.0) (2024-08-12)

### New Features
7 changes: 4 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading