Makefile: strip debug symbols by default #363

alessio · 2020-04-20T15:18:02Z

Look for build otpions in the make's GAIA_BUILD_OPTIONS
variable, e.g.:

$ make install GAIA_BUILD_OPTIONS=nostrip

would customise the build and remove all symbols-stripping flags
from both LDFLAGS and compiler's command line.

Replace WITH_CLEVELDB variable with a cleveldb build option.

For contributor use:

Targeted PR against correct branch (see CONTRIBUTING.md)
Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
Code follows the module structure standards.
Wrote unit and integration tests
Updated relevant documentation (docs/) or specification (x/<module>/spec/)
Added relevant godoc comments.
Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
Re-reviewed Files changed in the Github PR explorer

For admin use:

Added appropriate labels to PR (ex. WIP, R4R, docs, etc)
Reviewers assigned
Squashed all commits, uses message "Merge pull request #XYZ: [title]" (coding standards)

codecov · 2020-04-20T15:24:24Z

Codecov Report

Merging #363 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #363   +/-   ##
=======================================
  Coverage   67.65%   67.65%           
=======================================
  Files           4        4           
  Lines         504      504           
=======================================
  Hits          341      341           
  Misses        132      132           
  Partials       31       31

Look for build otpions in the make's GAIA_BUILD_OPTIONS variable, e.g.: $ make install GAIA_BUILD_OPTIONS=nostrip would customise the build and remove all symbols-stripping flags from both LDFLAGS and compiler's command line. Replace WITH_CLEVELDB variable with a cleveldb build option. Strip debug symbols by default.

tac0turtle

LGTM

Reduce the number of targets and make the buildsystem more flexible by parsing the TENDERMINT_BUILD_OPTIONS command line variable (a-la Debian, inspired by dpkg-buildpackage's DEB_BUILD_OPTIONS), e.g: $ make install TENDERMINT_BUILD_OPTIONS='cleveldb' replaces the old: $ make install_c Options can be mix&match'd, e.g.: $ make install TENDERMINT_BUILD_OPTIONS='cleveldb race nostrip' Three options are available: - nostrip: don't strip debugging symbols nor DWARF tables. - cleveldb: use cleveldb as db backend instead of goleveldb; it switches on the CGO_ENABLED Go environment variale. - race: pass -race to go build and enable data race detection. This changeset is a port of gaia pull request: cosmos/gaia#363

Reduce the number of targets and make the buildsystem more flexible by parsing the TENDERMINT_BUILD_OPTIONS command line variable (a-la Debian, inspired by dpkg-buildpackage's DEB_BUILD_OPTIONS), e.g: $ make install TENDERMINT_BUILD_OPTIONS='cleveldb' replaces the old: $ make install_c Options can be mix&match'd, e.g.: $ make install TENDERMINT_BUILD_OPTIONS='cleveldb race nostrip' Three options are available: - nostrip: don't strip debugging symbols nor DWARF tables. - cleveldb: use cleveldb as db backend instead of goleveldb; it switches on the CGO_ENABLED Go environment variale. - race: pass -race to go build and enable data race detection. This changeset is a port of gaia pull request: cosmos/gaia#363.

Reduce the number of targets and make the buildsystem more flexible by parsing the TENDERMINT_BUILD_OPTIONS command line variable (a-la Debian, inspired by dpkg-buildpackage's DEB_BUILD_OPTIONS), e.g: $ make install TENDERMINT_BUILD_OPTIONS='cleveldb' replaces the old: $ make install_c Options can be mix&match'd, e.g.: $ make install TENDERMINT_BUILD_OPTIONS='cleveldb race nostrip' Three options are available: - nostrip: don't strip debugging symbols nor DWARF tables. - cleveldb: use cleveldb as db backend instead of goleveldb; it switches on the CGO_ENABLED Go environment variale. - race: pass -race to go build and enable data race detection. This changeset is a port of gaia pull request: cosmos/gaia#363. Co-authored-by: Alessio Treglia <alessio@tendermint.com>

Look for build options in the make's COSMOS_BUILD_OPTIONS variable. This allows one to build simd with ledger support. References: * cosmos/gaia#363 * tendermint/tendermint#4738

Makefile: parse COSMOS_BUILD_OPTIONS Look for build options in the make's COSMOS_BUILD_OPTIONS variable. This allows one to build simd with ledger support. References: * cosmos/gaia#363 * tendermint/tendermint#4738 fix tiny typo in nolint: statement that causes golangci-lint to misbehave.

* lite2: improve auto update (#4334) * lite2: advance to latest header without any exponential steps rename autoUpdate to autoUpdateRoutine * lite2: wait in Cleanup until goroutines finished running * docs: minor doc fixes (#4335) * docs: minor doc fixes - minor doc fixes that i ran into while reading things - test if we have github actions Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * no github actions yet * add with * revert and change wording * adr: ADR-051: Double Signing Risk Reduction (#4262) * Add adr-051 to docs * add details * Update docs/architecture/adr-051-double-signing-protection-with-tendermint-mode.md Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * rename adr-051 for only double singing protection * remove contents about tendermint mode * change title to Double Signing Rist Reduction * rename adr md file * add a adr link to ToC Co-authored-by: b-harvest <38277329+dlguddus@users.noreply.github.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * Add IPv6 support for P2P integration tests (#4340) * lite2: add Start, TrustedValidatorSet funcs (#4337) * lite2: add Start method There are few reasons to do that: 1) separation of state and dynamics (some users will want to delay starting the light client; does not matter we should not allow them to create a light client object) 2) less important, but some users might not need autoUpdateRoutine and removeNoLongerTrustedHeadersRoutine routines * lite2: wait till routines are finished in Stop because they are started in Start, it feels more natural to wait for them to finish in Stop. * lite2: add TrustedValidatorSet func * rpc: add sort_order option to tx_search (#4342) I have added order_by which can be "asc" or "desc" (should be in string format) in the tx_search RPC method. Fixes: #3333 Author: @princesinha19 * lite2: rename alternative providers to witnesses (#4344) Closes #4341 * docs: update links to rpc (#4348) * docs: update links to rpc - links to rpc have not been updated. thank you @okwme Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * Update docs/app-dev/indexing-transactions.md * lite2: refactor cleanup() (#4343) * lite2: add Start method There are few reasons to do that: 1) separation of state and dynamics (some users will want to delay starting the light client; does not matter we should not allow them to create a light client object) 2) less important, but some users might not need autoUpdateRoutine and removeNoLongerTrustedHeadersRoutine routines * lite2: wait till routines are finished in Stop because they are started in Start, it feels more natural to wait for them to finish in Stop. * lite2: add TrustedValidatorSet func * refactor cleanup code * changed restore header and val function to handle negative height * reverted restoreTrustedHeaderAndNextVals() functionality Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * node: use GRPCMaxOpenConnections when creating the gRPC server (#4349) not MaxOpenConnections Fixes #4311 Also, set MaxBodyBytes, MaxHeaderBytes and WriteTimeout similar to HTTP server. * lite2: batch save & delete operations in DB store (#4345) Closes #4330 * build(deps): bump google.golang.org/grpc from 1.26.0 to 1.27.0 (#4355) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.26.0 to 1.27.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.26.0...v1.27.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * lite2: panic if witness is on another chain (#4356) Closes #4350 Checks that the chain ID of the witness and that of the lite client are the same before updating the witness list. * Add ADR-053: State Sync Prototype (#4352) * deps: bump github.com/golang/protobuf from 1.3.2 to 1.3.3 (#4359) Bumps [github.com/golang/protobuf](https://github.com/golang/protobuf) from 1.3.2 to 1.3.3. - [Release notes](https://github.com/golang/protobuf/releases) - [Commits](https://github.com/golang/protobuf/compare/v1.3.2...v1.3.3) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * docs: update npm dependencies (#4364) * lite2: make witnesses mandatory (#4358) * lite2: make witnesses mandatory at least one witness is required * lite2: return an error if there are no witnesses https://github.com/tendermint/tendermint/pull/4358#pullrequestreview-350635444 * cmd/lite: add witnesses flag * fix linter errors * abci: fix broken spec link (#4366) * docs: update guides proto paths (#4365) * update guides with correct path to libs/kv proto files * Apply suggestions from code review Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * format something to rerun ci Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * lite2: replace primary provider with alternative when unavailable (#4354) Closes issue #4338 Uses a wrapper function around both the signedHeader and validatorSet calls to the primary provider which attempts to retrieve the information 5 times before deeming the provider unavailable and replacing the primary provider with the first alternative before trying recursively again (until all alternatives are depleted) Employs a mutex lock for any operations involving the providers of the light client to ensure no operations occurs whilst the new primary is chosen. Commits: * created swapProvider function * eliminates old primary provider after replacement. Uses a mutex when changing providers * renamed to replaceProvider * created wrapped functions for signed header and val set * created test for primary provider replacement * implemented suggested revisions * created Witnesses() and Primary() * modified backoffAndJitterTime * modified backoffAndJitterTime * changed backoff base and jitter to functional arguments * implemented suggested changes * removed backoff function * changed exp function to match go version * halved the backoff time * removed seeding and added comments * fixed incorrect test * extract backoff timeout calc into a function Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * lite2: fetch missing headers (#4362) Closes #4328 When TrustedHeader(height) is called, if the height is less than the trusted height but the header is not in the trusted store then a function finds the previous lowest height with a trusted header and performs a forwards sequential verification to the header of the height that was given. If no error is found it updates the trusted store with the header and validator set for that height and can then return them to the user. Commits: * drafted trusted header * created function to find previous trusted height * updates missing headers less than the trusted height * minor cosmetic tweaks * incorporated suggestions * lite2: implement Backwards verification and add SignedHeaderAfter func to Store interface Refs https://github.com/tendermint/tendermint/issues/4328#issuecomment-581878549 * remove unused method * write tests * start with next height in SignedHeaderAfter func * fix linter errors * address Callum's comments Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * lite2: cross-check new header with all witnesses (#4373) As opposed to checking a random witness, all witnesses provided should be used as a reference against the header provided by the primary node. This increases security (at the tradeoff of speed) but also gives control to the user. The more witnesses provided, the more secure the lite client can be. * deps: bump google.golang.org/grpc from 1.27.0 to 1.27.1 (#4372) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.27.0 to 1.27.1. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.27.0...v1.27.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * Fix broken /docs/spec links (#4376) * lite2: validate TrustOptions, add NewClientFromTrustedStore (#4374) * validate trust options * add NewClientFromTrustedStore func * make maxRetryAttempts an option Closes #4370 * hash size should be equal to tmhash.Size * make maxRetryAttempts uint * make maxRetryAttempts uint16 maxRetryAttempts possible - 68 years * we do not store trustingPeriod * added test to create client from trusted store * remove header and vals from primary to make sure we're restoring them from the DB * docs: fix incorrect link (#4377) * lite2: return if there are no headers in RemoveNoLongerTrustedHeaders (#4378) * proto: add buf and protogen script (#4369) * proto: add buf and protogen script - add buf with minimal changes - add protogen script to easier generate proto files Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * add protoc needs * add some needed shell cmds * remove buf from tools as it is not needed everytime * add proto lint and breakage to ci * add section in changelog and upgrading files * address pr comments * remove space in circle config * remove spaces in makefile comment * add section on contributing on how to work with proto * bump buf to 0.7 * test bufbuild image * test install make in bufbuild image * revert to tendermintdev image * Update Makefile Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * lite2: manage witness dropout (#4380) * witnesses are dropped after no response * test witness dropout * corrected import structure * moved non responsiveness check to compare function * removed dropout test as witnesses are never dropped * created test to compare witnesses * docs: fix spec links (#4384) - erik fixed many of the broken links, just fixed two outstanding ones. - closes #4381 Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * proto: minor linting to proto files (#4386) * proto: minor linting minor linting after working with the proto files in the sdk. there is no logic change just spacing fixes Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * hardcore linting * refactored lite client tests * lite2: improve string output of all existing providers (#4387) before: &http{AFBSD743A...} after: http{https://127.0.0.1:26657} Co-authored-by: Marko <marbar3778@yahoo.com> * autofile: resolve relative paths (#4390) Fixes #2649 * make: remove sentry setup cmds (#4383) * make: remove sentry setup cmds removal of make comands for sentry setup. it was unclear if they were being maintained and there has not been a mention of people using them - closes #4379 Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * remove depreacted readme * add not being maintained section to docs * readme: fix link to original paper (#4391) Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * release: minor release 0.33.1 (#4401) * release: minor release 0.33.1 - minor release for 0.33.1 Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * remvoe wording * version bump * upgrade: update upgrade.md for protobuf changes Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * docs: update Light Client Protocol page (#4405) * docs: update Light Client Protocol page Closes #4331 * one way to get hash & height * adr: light client implementation (#4397) * adr: light client implementation Closes #2133 * note on chain IDs * explain why witnesses are required * if chain forks maliciously, chain ID stays the same * add a note about min witnesses while cross-checking * makefile: place phony markers after targets (#4408) The .PHONY targets in the Makefile are usually placed far away from the actual targets, and thus aren't always updated. Placing the .PHONY targets right next to the targets they cover make them more visible and thus more likely to be updated when necessary. * lite2: modified sequence method to match bisection (#4403) Currently the sequence function always starts from the trustedHeader and trustedNextVals stored in the lite client. Whereas the bisection one allows the method to be started from any combination of header and validator set. I opened up the sequence verification method to do the same * lite2: disconnect from bad nodes (#4388) Closes #4385 * extract TrustOptions into its own file * print trusted hash before asking whenever to rollback or not so the user could reset the light client with the trusted header * do not return an error if rollback is aborted reason: we trust the old header presumably, so can continue from it. * add note about time of initial header * improve logging and add comments * cross-check newHeader after LC verified it * check if header is not nil so we don't crash on the next line * remove witness if it sends us incorrect header * require at least one witness * fix build and tests * rename tests and assert for specific error * wrote a test * fix linter errors * only check 1/3 if headers diverge * Docs staging (#4407) * update theme * Update version * Updated Questions section in the footer * Remove links to Riot chat * Typo * Add Discord link Co-authored-by: Marko <marbar3778@yahoo.com> * rpc: fix issue with multiple subscriptions (#4406) Using the WebSocket server, when the same client calls multiple time the subscribe method, only the last subscription receives all the events of the previous ones. example: subscription1 = tm.event = 'NewBlock' subscription2 = tm.event = 'Tx' In this case, subscription2 will receive the new blocks but subscription1 will not. This came from the WebSocket handler that had the declaration of the rpcrequest moved and so overridden for every request and given in the JSONReq client context (so the id of the subscription was not the right one). This fixes the issue by simply declaring the rpcrequest inside the loop so every request will create a new object without overwriting the previous one. * lite2: divide verify functions (#4412) Closes #4398 * divided verify functions * extacted method * renamed functions. Created standard Verify function * checked non-adjacency. separated VerifyCommit * lint fixes * fix godoc documentation for VerifyAdjacent and VerifyNonAdjacent * add a comment about VerifyCommit being the last check Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * dep: bump gokit dep (#4424) * dep: bump gokitdep: bump gokitdep: bump gokitdep: bump gokitdep: bump gokitdep: bump gokitdep: bump gokitdep: bump gokitdep: bump gokit- Signedoff-by: Marko Baricevic <marbar3778@yahoo.com> * go mod tidy * test go version change * combination of multiple cmds Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * Use uint64 for consensus.Reactor.SwitchToConsensus() blocksSynced (#4433) * dep: maunally bump dep (#4436) replaces #4434 Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * rpc: fix tx_search pagination with ordered results (#4437) * lite: modified bisection to loop (#4400) refs #4329 As opposed to using recursion to implement the bisection method of verifying a header, which could have problems with memory allocation (especially for smaller devices), the bisection algorithm now uses a for loop. * modified bisection to loop * made lint changes * made lint changes * move note to VerifyHeader since it applies both for sequence and bisection * test bisection jumps to header signed by 1/3+ of old validator set * update labels in debug log calls * copy tc Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * rpc: fix txsearch tests (#4438) * blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * consensus: reduce log severity for ErrVoteNonDeterministicSignature (#4431) from error to debug Closes #3583 * deps: bump github.com/stretchr/testify from 1.4.0 to 1.5.0 (#4435) Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.4.0 to 1.5.0. - [Release notes](https://github.com/stretchr/testify/releases) - [Commits](https://github.com/stretchr/testify/compare/v1.4.0...v1.5.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * rpc: fix TxSearch test nits (#4446) * build(deps): bump github.com/stretchr/testify from 1.5.0 to 1.5.1 (#4441) Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.5.0 to 1.5.1. - [Release notes](https://github.com/stretchr/testify/releases) - [Commits](https://github.com/stretchr/testify/compare/v1.5.0...v1.5.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * build(deps): bump github.com/spf13/cobra from 0.0.3 to 0.0.6 (#4440) Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 0.0.3 to 0.0.6. - [Release notes](https://github.com/spf13/cobra/releases) - [Commits](https://github.com/spf13/cobra/compare/v0.0.3...v0.0.6) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * Fix some golangci-lint warnings (#4448) * lite2: return already verified headers and verify earlier headers (#4428) closes #4413 and #4419 When VerifyHeaderAtHeight() is called, TrustedHeader is initially run to check if the header has already been verified and returns the Header. If the new header height is less than the lite clients latestTrustedHeader height, than backwards verification is performed else either sequence or bisection Refactored a test to reflect the changes * use trustedHeader func for already verified Headers * remove fetch missing header from TrustedHeader * check for already trusted Header in VerifyHeaderAtHeight * replace updateTrustedHeaderAndVals to updateTrustedHeaderAndNextVals * rename trustedHeader and trustedNextVals * refactored backwards and included it in VerifyHeader * cleaned up test to match changes * lite2: fixes after my own review Refs https://github.com/tendermint/tendermint/pull/4428#pullrequestreview-361730169 * fix ineffectual assignment * lite2: check that header exists in VerifyHeader * extract function Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * p2p: use curve25519.X25519() instead of ScalarMult() (#4449) * p2p: use curve25519.X25519() instead of ScalarMult() * Renamed array to shrKeyArray * Updated CHANGELOG_PENDING * Revert "Updated CHANGELOG_PENDING" This reverts commit dbb72e0bf721287847ac136c99f385ce7456d1f7. * ADR-053: update with implementation plan after prototype (#4427) * circleci: run P2P IPv4 and IPv6 tests in parallel (#4459) * Fix unexported returns (#4450) * lite2: don't save intermediate headers (#4452) closes #4426 The sequence and bisection methods no longer save the intermediate headers and validator sets that they require to verify a currently untrusted header. ## Commits: * sequence and bisection don't save intermediate headers and vals * check the next validator hash matches the header * check expired header at start of backwards verification * added tests * handled cleanup warning * lint fix * removed redundant code * tweaked minor errors * avoided premature trusting of nextVals * fix test error * updated trustedHeader and Vals together * fixed bisection error * fixed sequence error for different vals and made test * fixes after my own review * reorder vars to be consistent with the rest of the code Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * types: VerifyCommitX return when +2/3 sigs are verified (#4445) Closes #4417 * lite2: store current validator set (#4472) Before we were storing trustedHeader (height=1) and trustedNextVals (height=2). After this change, we will be storing trustedHeader (height=1) and trustedVals (height=1). This a) simplifies the code b) fixes #4399 inconsistent pairing issue c) gives a relayer access to the current validator set #4470. The only downside is more jumps during bisection. If validator set changes between trustedHeader and the next header (by 2/3 or more), the light client will be forced to download the next header and check that 2/3+ signed the transition. But we don't expect validator set change too much and too often, so it's an acceptable compromise. Closes #4470 and #4399 * lite2: cross-check first header and update tests (#4471) closes #4464 * lite2: remove expiration checks on functions that don't require them (#4477) closes: #4455 Verifying backwards checks that the trustedHeader hasn't expired both before and after the loop in case of verifying many headers (a longer operation), but not during the loop itself. TrustedHeader() no longer checks whether the header saved in the store has expired. Tests have been updated to reflect the changes ## Commits: * verify headers backwards out of trust period * removed expiration check in trusted header func * modified tests to reflect changes * wrote new tests for backwards verification * modified TrustedHeader and TrustedValSet functions * condensed test functions * condensed test functions further * fix build error * update doc * add comments * remove unnecessary declaration * extract latestHeight check into a separate func Co-authored-by: Callum Waters <cmwaters19@gmail.com> * deps: bump github.com/tendermint/tm-db from 0.4.0 to 0.4.1 (#4476) Bumps [github.com/tendermint/tm-db](https://github.com/tendermint/tm-db) from 0.4.0 to 0.4.1. - [Release notes](https://github.com/tendermint/tm-db/releases) - [Changelog](https://github.com/tendermint/tm-db/blob/master/CHANGELOG.md) - [Commits](https://github.com/tendermint/tm-db/compare/v0.4.0...v0.4.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * blockchain (v2): comment out TestReactorTerminationScenarios (#4480) Refs #4442 * blockchain (v2): async testing notes (#4484) Add some inline notes for context of why test are commented out and not deleted. Ref. #4482 * github: edit templates for use in issues and pull requests (#4483) * github: edit templates for use in issues and pull requests Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * fix sdk to tendermint * lite2: prune-headers (#4478) closes #4469 Improved speed of cleanup by using SignedHeaderAfter instead of TrustedHeader to jump from header to header. Prune() is now called when a new header and validator set are saved and is a function dealt by the database itself ## Commits: * prune headers and vals * modified cleanup and tests * fixes after my own review * implement Prune func * make db ops concurrently safe * use Iterator in SignedHeaderAfter we should iterate from height+1, not from the end! * simplify cleanup Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * lite2: return height as 2nd return param in TrustedValidatorSet (#4479) Closes #4473 * lite2: actually run example tests + clock drift (#4487) Closes: #4488 ## Commits: * run example tests * introduce max clock drift clockDrift variable from the spec. 10s should cover most of the clients. References: - http://vancouver-webpages.com/time/web.html - https://blog.codinghorror.com/keeping-time-on-the-pc/ * fix ExampleClient_Update * add test * increase clock drift * fix linter warning * JSON tests related changes (#4461) * test functions take time.Now and other minor changes * updated remaining test files * Update validation_test.go * fix typo * go fmt * import time Co-authored-by: Marko <marbar3778@yahoo.com> * build(deps): bump github.com/golang/protobuf from 1.3.3 to 1.3.4 (#4485) Bumps [github.com/golang/protobuf](https://github.com/golang/protobuf) from 1.3.3 to 1.3.4. - [Release notes](https://github.com/golang/protobuf/releases) - [Commits](https://github.com/golang/protobuf/compare/v1.3.3...v1.3.4) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * rpc: stop txSearch result processing if context is done (#4418) * docs: adr-046 add bisection algorithm details (#4496) * docs: adr-046 add bisection algorithm details Closes #4329 * format fig. 1 title * docs: adr-046 we no longer download headers in TrustedHeader https://github.com/tendermint/tendermint/pull/4496#issuecomment-592446054 * swagger: update swagger port (#4498) Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * circleci: fix reproducible builds test (#4497) * tool: add Mergify (#4490) * mergify: remove unnecessary conditions (#4501) * mergify: use strict merges (#4502) * test: simplified txsearch cancellation test (#4500) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * rpc: keep the original subscription "id" field when new RPCs come in (#4493) * example/kvstore: return ABCI query height (#4509) * example/kvstore: return ABCI query height * CHANGELOG: added PR link * Updated tests * lite2: fix tendermint lite sub command (#4505) * lite2: fix tendermint lite sub command - better logging - chainID as an argument - more examples * one more log msg * lite2: fire update right away after start * turn off auto update in verification tests Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * docs: `tendermint node --help` dumps all supported flags (#4511) Closes #4211 * rpc: remove BlockStoreRPC in favor of BlockStore (#4510) * rpc: use BlockStoreRPC instead of BlockStore BlockStoreRPC is a limited version of BlockStore interface, which does not include SaveBlock method. Closes #4159 * remove BlockStoreRPC interface in favor of single BlockStore interface Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * cmd: show useful error when tm not initialised (#4512) Closes #4303 Before: ``` TMHOME="/Users/daniel/go/src/github.com/tendermint/thome" tendermint node --proxy_app=unix:///Users/daniel/go/src/github.com/kvstore/example.sock ERROR: 1 error(s) decoding: * error decoding 'consensus.create_empty_blocks_interval': time: invalid duration ``` After: ``` $ TMHOME=/tmp ./build/tendermint node --proxy_app=kvstore ERROR: failed to create node: Couldn't read GenesisDoc file: open /tmp/config/genesis.json: no such file or directory ``` * docs: write about debug kill and dump (#4516) * docs: write about debug kill and dump Closes #4325 * wrap file tree in code blocks * deps: bump github.com/Workiva/go-datastructures (#4519) Bumps [github.com/Workiva/go-datastructures](https://github.com/Workiva/go-datastructures) from 1.0.50 to 1.0.51. - [Release notes](https://github.com/Workiva/go-datastructures/releases) - [Commits](https://github.com/Workiva/go-datastructures/compare/v1.0.50...v1.0.51) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * lite: add helper functions for initiating the light client (#4486) * add new net client * hijack example test * lint fixes * reorganised file structure * renamed funcs and added documentation * implemented suggested changes * restored example tests * edited comments * Update lite2/setup.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * Update lite2/client.go * lite cmds use HTTP client * better naming * Delete go.sum * Delete go.mod * restore go mod * restore go.sum * fix double import Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * cmd: fix debug kill and change debug dump archive filename format (#4517) * cmd: change archive filename format before: Mar 3 08:44:43.zip after: 2020-03-03T08:44:43Z.zip Refs #4515 * cmd: kill pid, not the debug process itself Fixes #4515 * update changelog Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * build(deps): bump github.com/prometheus/client_golang (#4525) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.4.1 to 1.5.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/master/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.4.1...v1.5.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * .github: add markdown link checker (#4513) co-authored-by: Marko <marbar3778@yahoo.com> * adr-047: evidence handling (#4429) Closes #4213 and #4182 * fix: proto-breakage (#4506) * fix: fix proto-breakage - this is amed to fix proto breakage for consumers Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * fix for importing third_party everywhere * undo change * test breakage change * test ssh * test https * change ssh to https * fix phony * fix addrbook tests (#4526) Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> Co-authored-by: Erik Grinaker <erik@grinaker.org> * docs: fix links (#4531) * docs: fix links - fix broken links closes #4522 Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * fix more links * add enable and disable * remvoe deadlink * readme: add discord to readme (#4533) * readme: add discord to readme Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * add our id * lite2: remove auto update (#4535) We first introduced auto-update as a separate struct AutoClient, which was wrapping Client and calling Update periodically. // AutoClient can auto update itself by fetching headers every N seconds. type AutoClient struct { base *Client updatePeriod time.Duration quit chan struct{} trustedHeaders chan *types.SignedHeader errs chan error } // NewAutoClient creates a new client and starts a polling goroutine. func NewAutoClient(base *Client, updatePeriod time.Duration) *AutoClient { c := &AutoClient{ base: base, updatePeriod: updatePeriod, quit: make(chan struct{}), trustedHeaders: make(chan *types.SignedHeader), errs: make(chan error), } go c.autoUpdate() return c } // TrustedHeaders returns a channel onto which new trusted headers are posted. func (c *AutoClient) TrustedHeaders() <-chan *types.SignedHeader { return c.trustedHeaders } // Err returns a channel onto which errors are posted. func (c *AutoClient) Errs() <-chan error { return c.errs } // Stop stops the client. func (c *AutoClient) Stop() { close(c.quit) } func (c *AutoClient) autoUpdate() { ticker := time.NewTicker(c.updatePeriod) defer ticker.Stop() for { select { case <-ticker.C: lastTrustedHeight, err := c.base.LastTrustedHeight() if err != nil { c.errs <- err continue } if lastTrustedHeight == -1 { // no headers yet => wait continue } newTrustedHeader, err := c.base.Update(time.Now()) if err != nil { c.errs <- err continue } if newTrustedHeader != nil { c.trustedHeaders <- newTrustedHeader } case <-c.quit: return } } } Later we merged it into the Client itself with the assumption that most clients will want it. But now I am not sure. Neither IBC nor cosmos/relayer are using it. It increases complexity (Start/Stop methods). That said, I think it makes sense to remove it until we see a need for it (until we better understand usage behavior). We can always introduce it later 😅. Maybe in the form of AutoClient. * lite2: indicate success/failure of Update (#4536) error itself is not enough since it only signals if there were any errors. Either (types.SignedHeader) or (success bool) is needed to indicate the status of the operation. Returning a header is optimal since most of the clients will want to get a newly verified header anyway. * rpc: create buffered subscriptions on /subscribe (#4521) Closes #3935 * lite2: replace primary when providing invalid header (#4523) Closes: #4420 Created a new error ErrInvalidHeaderwhich can be formed during the verification process verifier.go and will result in the replacement of the primary provider with a witness by executing: replacePrimaryProvider() Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * adr: crypto encoding for proto (#4481) * adr: crypto encoding for proto work - this adr is meant to help with deciding on how to move forward with keys in tendermint. * minor change * fix gomod * add a third option * fix spelling * add first part of descision * breakdown keys and where they are used * add some wording * minor wording fix * question * change proto messages * minor update * undo go.mod changes * add a few things based on comemnts * push, push it real good * minor explanation on interface type * touch up * CONTRIBUTING: include instructions for installing protobuf * build(deps): bump github.com/Workiva/go-datastructures (#4545) Bumps [github.com/Workiva/go-datastructures](https://github.com/Workiva/go-datastructures) from 1.0.51 to 1.0.52. - [Release notes](https://github.com/Workiva/go-datastructures/releases) - [Commits](https://github.com/Workiva/go-datastructures/compare/v1.0.51...v1.0.52) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * mvp blacklist alg * move reinstatement after if statement in pex reactor * separate reinstatement of peers from discovery of new peers * lint fix * libs/kv: remove unused type KI64Pair (#4542) * rc1/v0.33.2(#4541) * rc1/v0.33.2 - release candiadte for 0.33.2 Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * build(deps): bump google.golang.org/grpc from 1.27.1 to 1.28.0 (#4551) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.27.1 to 1.28.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.27.1...v1.28.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * noop commit (#4553) should force CircleCI to update the code cache Found a cache from build 100135 at go-src-v1-b215e07ecdc6ad5c363c3b420580aa6b52913feb Size: 65 MiB Cached paths: * /go/src/github.com/tendermint/tendermint/.git because we have v0.33.1-dev3 tag, bump-semver.py script is thinking the next tag should be v0.33.1, not v0.33.2: if re.match('[0-9]+$',patch) is None: patchfound = re.match('([0-9]+)',patch) patch = int(patchfound.group(1)) else: patch = int(patch) + 1 The last tag it finds is v0.33.1-dev3. So it hits the first branch. But the first branch only fetches the number, and doesn't increment it, so the result is v0.33.1. if there exists a patch tag like 1-dev3, then the next version should be 0.33.1. But in this case, 0.33.1 has already been released. * build(deps): bump github.com/tendermint/tm-db from 0.4.1 to 0.5.0 (#4554) Bumps [github.com/tendermint/tm-db](https://github.com/tendermint/tm-db) from 0.4.1 to 0.5.0. - [Release notes](https://github.com/tendermint/tm-db/releases) - [Changelog](https://github.com/tendermint/tm-db/blob/master/CHANGELOG.md) - [Commits](https://github.com/tendermint/tm-db/compare/v0.4.1...v0.5.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * reconfigure mutexes * ban function requires a ban duration as an argument * make banTime an argument and set default in PEX reactor instead of AddrBook * basic test for banning peers * added banned address error * banned addresses can't be added again * added isBanned check in addrbook * added logs for more information * lint fix * deps: bump deps that bot cant (#4555) * deps: bump deps that bot cant - bumping deps that dependat bot does not do. Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * run go mod tidy * fix go.sum * tools/build: delete stale tools (#4558) * lite2: add benchmarking tests (#4514) Closes #4392 Refs #4504 * privval: return error on getpubkey (#4534) closes #3602 Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * made suggested changes * cannot decrease ban time * update Changelog with PR #4548 * moved to changelog_pending * Use docs-staging branch (#4561) Co-authored-by: Marko <marbar3778@yahoo.com> * build(deps): bump github.com/golang/protobuf from 1.3.4 to 1.3.5 (#4563) Bumps [github.com/golang/protobuf](https://github.com/golang/protobuf) from 1.3.4 to 1.3.5. - [Release notes](https://github.com/golang/protobuf/releases) - [Commits](https://github.com/golang/protobuf/compare/v1.3.4...v1.3.5) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * removal: remove build folder (#4565) - remove tools/build folder Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * Update docs website (#4564) * update theme * Update version * Updated Questions section in the footer * Remove links to Riot chat * Typo * Add Discord link * Update docs theme to the latest version * Use docs-staging branch for staging website * Resolve merge conflicts * Update version * Add google analytics Co-authored-by: Marko <marbar3778@yahoo.com> * fix: fix spelling of comment (#4566) - :P Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * build(deps): bump github.com/prometheus/client_golang (#4574) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.5.0 to 1.5.1. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/master/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.5.0...v1.5.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * rpc: fix panic when `Subscribe` is called (#4570) but HTTP client is not running. `Subscribe`, `Unsubscribe(All)` methods return an error now. Closes #4568 * Dockerfile updated with defaults (#4577) * build(deps): bump github.com/gorilla/websocket from 1.4.1 to 1.4.2 (#4584) Bumps [github.com/gorilla/websocket](https://github.com/gorilla/websocket) from 1.4.1 to 1.4.2. - [Release notes](https://github.com/gorilla/websocket/releases) - [Commits](https://github.com/gorilla/websocket/compare/v1.4.1...v1.4.2) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * deps: run go mod tidy (#4587) Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * Docker image port fix (#4589) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * format: add format cmd & goimport repo (#4586) * format: add format cmd & goimport repo - replaced format command - added goimports to format command - ran goimports Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * fix outliers & undo proto file changes * minor link fixing (#4598) * blockchain: enable v2 to be set (#4597) * blockchain: enable v2 to be set - enable v2 to be set via config params Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * replace tab with space * correctly spell usability * change link checker to run daily (#4601) * lite2: cache headers in bisection (#4562) Closes: #4546 The algorithm uses an array to store the headers and validators and populates it at every bisection (which is an unsuccessful verification). When a successful verification finally occurs it updates the new trusted header, trims that header from the cache (the array) and sets the depth pointer back to 0. Instead of retrieving new headers it will use the cached headers, incrementing in depth until it reaches the end of the cache which by then it will start to retrieve new headers from the provider. Mathematically, this method doesn't properly bisect after the first round but it will always choose a pivot header that is within 1/8th of the upper header's height. I.e. if we are trying to jump 128 headers, the maximum offset from bisection height (64) is 64 + 16(128/8) = 80, therefore a better heuristic would be to obtain the new pivot header height as the middle of these two numbers which would therefore mean to multiply it by 9/16ths instead of 1/2 (sorry this might be a bit more complicated in writing but I can try better explain if someone is interested). Therefore I would also, upon consensus, propose that we change the pivot height to 9/16th's of the previous height * tools: remove need to install buf (#4605) * tools: remove need to install buf - using buf docker image instead of needing devs to install it Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * fix ci for lint and break checking * create style guide and update contributors section (#4576) * docs: validator setup & Key info (#4604) - defined what variables needed to be changed in the `config.toml` in order to run a validator. - Briefly explained how a sentry node archtecture should look - add section explaing importance of key secruity Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * all: name reactors when they are initialized (#4608) Previously, many reactors were initialized with the name "Reactor," which made it difficult to log which reactor was doing what. This changes those reactors' names to something more descriptive. * build(deps): bump github.com/spf13/cobra from 0.0.6 to 0.0.7 (#4612) Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 0.0.6 to 0.0.7. - [Release notes](https://github.com/spf13/cobra/releases) - [Commits](https://github.com/spf13/cobra/compare/v0.0.6...0.0.7) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * build(deps): bump github.com/tendermint/tm-db from 0.5.0 to 0.5.1 (#4613) Bumps [github.com/tendermint/tm-db](https://github.com/tendermint/tm-db) from 0.5.0 to 0.5.1. - [Release notes](https://github.com/tendermint/tm-db/releases) - [Changelog](https://github.com/tendermint/tm-db/blob/master/CHANGELOG.md) - [Commits](https://github.com/tendermint/tm-db/compare/v0.5.0...v0.5.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * proto: use docker to generate stubs (#4615) * proto: use docker to generate stubs - provide an option to developers to use docker to generate proto stubs closes #4579 Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * rpc: add codespace to ResultBroadcastTx (#4611) Closes: #4606 * lite2: use bisection for some of backward verification (#4575) Closes: #4537 Uses SignedHeaderBefore to find header before unverified header and then bisection to verify the header. Only when header is between first and last trusted header height else if before the first trusted header height then regular backwards verification is used. * lite2: make maxClockDrift an option (#4616) Closes #4607 * rpc/client: split out client packages (#4628) * rpc/client: initial split into directories * lite2: split out test package * rpc/client: simplify client constructurs * updated docs * updated changelog * ADR-053: strengthen and simplify the state sync ABCI interface (#4610) * adr-053: strengthened and simplified ABCI interface * remove internal_error reason and open questions * add follow-up task for block backfill Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * mempool: reserve IDs in InitPeer instead of AddPeer * p2p: limit the number of incoming connections to p2p.max_num_inbound_peers + len(p2p.unconditional_peer_ids) * update changelog and version * upgrading: add note on rpc/client subpackages (#4636) * upgrading: add note on rpc/client subpackages * clarify constructors * config: allow fastsync.version = v2 (#4639) * lite2: Prevent falsely returned double voting error (#4620) * prevent faulty double voting error * create test * clean tests * clean tests Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * lite2: Default to http scheme in provider.New (#4649) Closes: #4643 * p2p: PEX message abuse should ban as well as disconnect (#4621) * mark unsolicited and too frequent messaged as bad * add tests * update changelog and fix error * revised error types Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * lint: add review dog (#4652) * lint: add review dog - golangci is being deprecated on the 15th Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * check evidence hasn't already been stored (#4632) Add Has function, create better handling of errors when adding evidence, usage of error types. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * build(deps): bump google.golang.org/grpc from 1.28.0 to 1.28.1 (#4653) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.28.0 to 1.28.1. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.28.0...v1.28.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * run go mod tidy * Docs website update (#4658) * mempool: reserve IDs in InitPeer instead of AddPeer * p2p: limit the number of incoming connections to p2p.max_num_inbound_peers + len(p2p.unconditional_peer_ids) * update changelog and version * README: specify supported versions (#4660) Co-authored-by: Alessio Treglia <alessio@tendermint.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * build(deps): bump github.com/spf13/viper from 1.6.2 to 1.6.3 (#4664) Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.6.2 to 1.6.3. - [Release notes](https://github.com/spf13/viper/releases) - [Commits](https://github.com/spf13/viper/compare/v1.6.2...v1.6.3) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * cli: add command to generate shell completion scripts (#4665) How to use it: ``` $ . <(tendermint completion) ``` Note that the completion command does not show up in the help screen, though it comes with its own --help option. This is a port of the feature provided by cosmos-sdk. * mergify: use PR title and body for squash merge commit (#4669) * blockchain/v2: allow setting nil switch, for CustomReactors()  Fixes an issue reported in https://github.com/tendermint/tendermint/issues/4595#issuecomment-612667441. Not sure if this is sufficient to fully remove the reactor, but it fixes the immediate problem. ______ For contributor use: - [x] Wrote tests - [x] ~Updated CHANGELOG_PENDING.md~ - [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [x] ~Updated relevant documentation (`docs/`) and code comments~ - [x] Re-reviewed `Files changed` in the Github PR explorer * evidence: both MaxAgeDuration and MaxAgeNumBlocks need to be surpassed (#4667) for evidence to be considered expired. otherwise, a cabal group can manipulate block time to make a particular evidence too old. Refs https://github.com/tendermint/tendermint/issues/2565#issuecomment-432896645 Refs https://github.com/tendermint/tendermint/issues/2653 spec PR: tendermint/spec#87 * dep: bump protobuf, cobra, btcutil & std lib deps (#4676) * docs/architecture: add adr-55 for proto repo design (#4623) * changelog: add entries from secruity releases ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [ ] Re-reviewed `Files changed` in the Github PR explorer * docs: amend adr-54 with changes in the sdk (#4684) * blockchain/v2: don't broadcast base if height is 0 ## Description Fixes a bug where the reactor would broadcast a base with height=0. ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [x] Re-reviewed `Files changed` in the Github PR explorer * build(deps): bump @vuepress/plugin-google-analytics in /docs (#4692) Bumps [@vuepress/plugin-google-analytics](https://github.com/vuejs/vuepress/tree/HEAD/packages/@vuepress/plugin-google-analytics) from 1.3.1 to 1.4.1. - [Release notes](https://github.com/vuejs/vuepress/releases) - [Changelog](https://github.com/vuejs/vuepress/blob/master/CHANGELOG.md) - [Commits](https://github.com/vuejs/vuepress/commits/v1.4.1/packages/@vuepress/plugin-google-analytics) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * minor release process change ## Description The minor release process is changing in order to not have major release changes sitting in the pull request tab. This changes from taking master and releasing from master to creating a branch that you cherry-pick commits into. There are two options on labeling which pull requests to include in a minor release: 1. Use the label `R:minor` to know which pull requests to include then remove the label when those pull requests have been included in a release. 2. Create an Issue where pull request numbers are added. then the issue is closed when the release is done. this process should be followed after 0.33.3 ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [ ] Re-reviewed `Files changed` in the Github PR explorer * lite2: verify ConsensusHash in rpc client ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [x] Re-reviewed `Files changed` in the Github PR explorer * docs/architecture: create adr 56: prove amnesia attack ## Description ADR to address the process for proving an amnesia attack (as a form of global evidence) from `PotentialAmnesiaEvidence` detected by light clients ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [ ] Re-reviewed `Files changed` in the Github PR explorer * bump ABCI version to 0.16.2 due to ResponseCommit.retain_height See #4588 for original change. I believe this is appropriate. Anything else that needs to be updated? ______ For contributor use: - [ ] ~Wrote tests~ - [x] Updated CHANGELOG_PENDING.md - [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] ~Updated relevant documentation (`docs/`) and code comments~ - [x] Re-reviewed `Files changed` in the Github PR explorer * fix linter warnings * clarify service logging The service logging can be a bit unclear. For example, with state sync it would log: ``` I[2020-04-20|08:40:47.366] Starting StateSync module=statesync impl=Reactor I[2020-04-20|08:40:47.834] Starting state sync module=statesync ``` Where the first message is the reactor service startup, and the second message is the start of the actual state sync process. This clarifies the first message by changing it to `Starting StateSync service`. ______ For contributor use: - [ ] ~Wrote tests~ - [ ] ~Updated CHANGELOG_PENDING.md~ - [ ] ~Linked to Github issue with discussion and accepted design OR link to spec that describes this work.~ - [ ] ~Updated relevant documentation (`docs/`) and code comments~ - [x] Re-reviewed `Files changed` in the Github PR explorer * blockstore: allow initial SaveBlock() at any height Followup from #4588. Allow the first `SaveBlock()` call in an empty block store to be at any height, to start from a truncated block history. Subsequent `SaveBlock()` calls must be for contiguous blocks. ______ For contributor use: - [x] Wrote tests - [ ] ~Updated CHANGELOG_PENDING.md~ - [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [x] Updated relevant documentation (`docs/`) and code comments - [x] Re-reviewed `Files changed` in the Github PR explorer * ci: transition some ci to github actions ## Description move tests for abci_cli, abci_app and app_tests to github actions ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [ ] Re-reviewed `Files changed` in the Github PR explorer * p2p/pex: fix DATA RACE in TestPEXReactorDialsPeerUpToMaxAttemptsInSeedMode Closes #4668 ______ For contributor use: - [x] Wrote tests - [ ] ~~Updated CHANGELOG_PENDING.md~~ - [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] ~~Updated relevant documentation (`docs/`) and code comments~~ - [x] Re-reviewed `Files changed` in the Github PR explorer * lite2/rpc: verify block results and validators (#4703) Closes: #4695 Verify /block_results and /validators responses from an HTTP client using the light client. Added count and total to /validators response. Refs #3113 * rc1/v0.33.4 Prepare release 0.33.4 * kick CI * abci/server: print panic & stack trace to STDERR if logger is not set Closes #4382 * test: fix p2p test build breakage caused by Debian testing (#4880) * types: implement Header#ValidateBasic (#4638) - Move core stateless validation of the Header type to a ValidateBasic method. - Call header.ValidateBasic during a SignedHeader validation. - Call header.ValidateBasic during a PhantomValidatorEvidence validation. - Call header.ValidateBasic during a LunaticValidatorEvidence validation. lite tests are skipped since the package is deprecated, no need to waste time on it closes: #4572 Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * lite2: fix TestVerifyAdjacentHeaders (#4884) * Makefile: parse TENDERMINT_BUILD_OPTIONS (#4738) (#4883) Reduce the number of targets and make the buildsystem more flexible by parsing the TENDERMINT_BUILD_OPTIONS command line variable (a-la Debian, inspired by dpkg-buildpackage's DEB_BUILD_OPTIONS), e.g: $ make install TENDERMINT_BUILD_OPTIONS='cleveldb' replaces the old: $ make install_c Options can be mix&match'd, e.g.: $ make install TENDERMINT_BUILD_OPTIONS='cleveldb race nostrip' Three options are available: - nostrip: don't strip debugging symbols nor DWARF tables. - cleveldb: use cleveldb as db backend instead of goleveldb; it switches on the CGO_ENABLED Go environment variale. - race: pass -race to go build and enable data race detection. This changeset is a port of gaia pull request: cosmos/gaia#363. Co-authored-by: Alessio Treglia <alessio@tendermint.com> * lite: fix HTTP provider error handling (#4882) * lite: fix HTTP provider error handling Fixes #4739, kind of. See #4740 for the proper fix. --- For contributor use: - [x] Wrote tests - [x] Updated CHANGELOG_PENDING.md - [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [x] Updated relevant documentation (`docs/`) and code comments - [x] Re-reviewed `Files changed` in the Github PR explorer - [x] Applied Appropriate Labels * adapt tests to missing pull request Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * privval: remove deprecated `OldFilePV` (#4889) * privval: remove deprecated `OldFilePV` The old format was deprecated in v0.28. It's time we remove it. * fix changelog * blockchain/v2: backport fixes (#4887) * blockchain/v2: fix excessive CPU usage due to spinning on closed channels (#4761) The event loop uses a `select` on multiple channels. However, reading from a closed channel in Go always yields the channel's zero value. The processor and scheduler close their channels when done, and since these channels are always ready to receive, the event loop keeps spinning on them. This changes `routine.terminate()` to not close the channel, and also removes `stopDemux` and instead uses `events` channel closure to signal event loop termination. Fixes #4687. * blockchain/v2: respect fast_sync option (#4772) Not thoroughly tested, but seems to work. Will do further testing as this is integrated with state sync. Fixes #4688. * types: return an error if voting power overflows in VerifyCommitTrusting Closes #4755 * rpc: handle panics during panic handling (#4888) Fixes #4802. The Go HTTP server has a global panic handler for requests, so it was not as severe as first thought. This fix can still panic, since we try to send a `500` response - if that happens, the Go HTTP server will terminate the connection. Otherwise, the client will get a 200 response, which we should avoid. I'm sort of torn on whether it's even necessary to include this fix, instead of just letting the HTTP server deal with it. * mempool: allow ReapX and CheckTx functions to run in parallel allow ReapX and CheckTx functions to run in parallel, making it not possible to block certain proposers from creating a new block. Closes: #2972 * pex: use highwayhash for pex bucket Closes #2998 Spec PR: https://github.com/tendermint/spec/pull/94 * privval: retry GetPubKey/SignVote/SignProposal N times before returning an error Closes #4707 * rpc: use a struct to wrap all the global objects Closes #3433 * rpc: refactor lib folder (#4836) Closes https://github.com/tendermint/tendermint/issues/3857 Moves `lib/` folder to `jsonrpc/`. Renames: **packages** `rpc` package -> `jsonrpc` package `rpcclient` package -> `client` package `rpcserver` package -> `server` package **structs and interfaces** ``` JSONRPCClient to Client JSONRPCRequestBatch to RequestBatch JSONRPCCaller to Caller ``` **functions** ``` StartHTTPServer to Serve StartHTTPAndTLSServer to ServeTLS rpc/jsonrpc/client: rename NewURIClient to NewURI NewJSONRPCClient to New NewJSONRPCClientWithHTTPClient to NewWithHTTPClient NewWSClient to NewWS ``` **misc** - unexpose `ResponseWriterWrapper…

* mvp blacklist alg * move reinstatement after if statement in pex reactor * separate reinstatement of peers from discovery of new peers * lint fix * build(deps): bump google.golang.org/grpc from 1.27.1 to 1.28.0 (#4551) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.27.1 to 1.28.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.27.1...v1.28.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * build(deps): bump github.com/tendermint/tm-db from 0.4.1 to 0.5.0 (#4554) Bumps [github.com/tendermint/tm-db](https://github.com/tendermint/tm-db) from 0.4.1 to 0.5.0. - [Release notes](https://github.com/tendermint/tm-db/releases) - [Changelog](https://github.com/tendermint/tm-db/blob/master/CHANGELOG.md) - [Commits](tendermint/tm-db@v0.4.1...v0.5.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * reconfigure mutexes * ban function requires a ban duration as an argument * make banTime an argument and set default in PEX reactor instead of AddrBook * basic test for banning peers * added banned address error * banned addresses can't be added again * added isBanned check in addrbook * added logs for more information * lint fix * deps: bump deps that bot cant (#4555) * deps: bump deps that bot cant - bumping deps that dependat bot does not do. Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * run go mod tidy * fix go.sum * tools/build: delete stale tools (#4558) * lite2: add benchmarking tests (#4514) Closes #4392 Refs #4504 * privval: return error on getpubkey (#4534) closes #3602 Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * made suggested changes * cannot decrease ban time * update Changelog with PR #4548 * moved to changelog_pending * Use docs-staging branch (#4561) Co-authored-by: Marko <marbar3778@yahoo.com> * build(deps): bump github.com/golang/protobuf from 1.3.4 to 1.3.5 (#4563) Bumps [github.com/golang/protobuf](https://github.com/golang/protobuf) from 1.3.4 to 1.3.5. - [Release notes](https://github.com/golang/protobuf/releases) - [Commits](golang/protobuf@v1.3.4...v1.3.5) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * removal: remove build folder (#4565) - remove tools/build folder Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * Update docs website (#4564) * update theme * Update version * Updated Questions section in the footer * Remove links to Riot chat * Typo * Add Discord link * Update docs theme to the latest version * Use docs-staging branch for staging website * Resolve merge conflicts * Update version * Add google analytics Co-authored-by: Marko <marbar3778@yahoo.com> * fix: fix spelling of comment (#4566) - :P Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * build(deps): bump github.com/prometheus/client_golang (#4574) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.5.0 to 1.5.1. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/master/CHANGELOG.md) - [Commits](prometheus/client_golang@v1.5.0...v1.5.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * rpc: fix panic when `Subscribe` is called (#4570) but HTTP client is not running. `Subscribe`, `Unsubscribe(All)` methods return an error now. Closes #4568 * Dockerfile updated with defaults (#4577) * build(deps): bump github.com/gorilla/websocket from 1.4.1 to 1.4.2 (#4584) Bumps [github.com/gorilla/websocket](https://github.com/gorilla/websocket) from 1.4.1 to 1.4.2. - [Release notes](https://github.com/gorilla/websocket/releases) - [Commits](gorilla/websocket@v1.4.1...v1.4.2) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * deps: run go mod tidy (#4587) Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * Docker image port fix (#4589) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * format: add format cmd & goimport repo (#4586) * format: add format cmd & goimport repo - replaced format command - added goimports to format command - ran goimports Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * fix outliers & undo proto file changes * minor link fixing (#4598) * blockchain: enable v2 to be set (#4597) * blockchain: enable v2 to be set - enable v2 to be set via config params Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * replace tab with space * correctly spell usability * change link checker to run daily (#4601) * lite2: cache headers in bisection (#4562) Closes: #4546 The algorithm uses an array to store the headers and validators and populates it at every bisection (which is an unsuccessful verification). When a successful verification finally occurs it updates the new trusted header, trims that header from the cache (the array) and sets the depth pointer back to 0. Instead of retrieving new headers it will use the cached headers, incrementing in depth until it reaches the end of the cache which by then it will start to retrieve new headers from the provider. Mathematically, this method doesn't properly bisect after the first round but it will always choose a pivot header that is within 1/8th of the upper header's height. I.e. if we are trying to jump 128 headers, the maximum offset from bisection height (64) is 64 + 16(128/8) = 80, therefore a better heuristic would be to obtain the new pivot header height as the middle of these two numbers which would therefore mean to multiply it by 9/16ths instead of 1/2 (sorry this might be a bit more complicated in writing but I can try better explain if someone is interested). Therefore I would also, upon consensus, propose that we change the pivot height to 9/16th's of the previous height * tools: remove need to install buf (#4605) * tools: remove need to install buf - using buf docker image instead of needing devs to install it Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * fix ci for lint and break checking * create style guide and update contributors section (#4576) * docs: validator setup & Key info (#4604) - defined what variables needed to be changed in the `config.toml` in order to run a validator. - Briefly explained how a sentry node archtecture should look - add section explaing importance of key secruity Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * all: name reactors when they are initialized (#4608) Previously, many reactors were initialized with the name "Reactor," which made it difficult to log which reactor was doing what. This changes those reactors' names to something more descriptive. * build(deps): bump github.com/spf13/cobra from 0.0.6 to 0.0.7 (#4612) Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 0.0.6 to 0.0.7. - [Release notes](https://github.com/spf13/cobra/releases) - [Commits](spf13/cobra@v0.0.6...0.0.7) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * build(deps): bump github.com/tendermint/tm-db from 0.5.0 to 0.5.1 (#4613) Bumps [github.com/tendermint/tm-db](https://github.com/tendermint/tm-db) from 0.5.0 to 0.5.1. - [Release notes](https://github.com/tendermint/tm-db/releases) - [Changelog](https://github.com/tendermint/tm-db/blob/master/CHANGELOG.md) - [Commits](tendermint/tm-db@v0.5.0...v0.5.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * proto: use docker to generate stubs (#4615) * proto: use docker to generate stubs - provide an option to developers to use docker to generate proto stubs closes #4579 Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * rpc: add codespace to ResultBroadcastTx (#4611) Closes: #4606 * lite2: use bisection for some of backward verification (#4575) Closes: #4537 Uses SignedHeaderBefore to find header before unverified header and then bisection to verify the header. Only when header is between first and last trusted header height else if before the first trusted header height then regular backwards verification is used. * lite2: make maxClockDrift an option (#4616) Closes #4607 * rpc/client: split out client packages (#4628) * rpc/client: initial split into directories * lite2: split out test package * rpc/client: simplify client constructurs * updated docs * updated changelog * ADR-053: strengthen and simplify the state sync ABCI interface (#4610) * adr-053: strengthened and simplified ABCI interface * remove internal_error reason and open questions * add follow-up task for block backfill Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * upgrading: add note on rpc/client subpackages (#4636) * upgrading: add note on rpc/client subpackages * clarify constructors * config: allow fastsync.version = v2 (#4639) * lite2: Prevent falsely returned double voting error (#4620) * prevent faulty double voting error * create test * clean tests * clean tests Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * lite2: Default to http scheme in provider.New (#4649) Closes: #4643 * p2p: PEX message abuse should ban as well as disconnect (#4621) * mark unsolicited and too frequent messaged as bad * add tests * update changelog and fix error * revised error types Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * lint: add review dog (#4652) * lint: add review dog - golangci is being deprecated on the 15th Signed-off-by: Marko Baricevic <marbar3778@yahoo.com> * check evidence hasn't already been stored (#4632) Add Has function, create better handling of errors when adding evidence, usage of error types. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * build(deps): bump google.golang.org/grpc from 1.28.0 to 1.28.1 (#4653) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.28.0 to 1.28.1. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.28.0...v1.28.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * run go mod tidy * Docs website update (#4658) * mempool: reserve IDs in InitPeer instead of AddPeer * p2p: limit the number of incoming connections to p2p.max_num_inbound_peers + len(p2p.unconditional_peer_ids) * update changelog and version * README: specify supported versions (#4660) Co-authored-by: Alessio Treglia <alessio@tendermint.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * build(deps): bump github.com/spf13/viper from 1.6.2 to 1.6.3 (#4664) Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.6.2 to 1.6.3. - [Release notes](https://github.com/spf13/viper/releases) - [Commits](spf13/viper@v1.6.2...v1.6.3) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * cli: add command to generate shell completion scripts (#4665) How to use it: ``` $ . <(tendermint completion) ``` Note that the completion command does not show up in the help screen, though it comes with its own --help option. This is a port of the feature provided by cosmos-sdk. * mergify: use PR title and body for squash merge commit (#4669) * blockchain/v2: allow setting nil switch, for CustomReactors()  Fixes an issue reported in tendermint/tendermint#4595 (comment). Not sure if this is sufficient to fully remove the reactor, but it fixes the immediate problem. ______ For contributor use: - [x] Wrote tests - [x] ~Updated CHANGELOG_PENDING.md~ - [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [x] ~Updated relevant documentation (`docs/`) and code comments~ - [x] Re-reviewed `Files changed` in the Github PR explorer * evidence: both MaxAgeDuration and MaxAgeNumBlocks need to be surpassed (#4667) for evidence to be considered expired. otherwise, a cabal group can manipulate block time to make a particular evidence too old. Refs tendermint/tendermint#2565 (comment) Refs tendermint/tendermint#2653 spec PR: tendermint/spec#87 * dep: bump protobuf, cobra, btcutil & std lib deps (#4676) * docs/architecture: add adr-55 for proto repo design (#4623) * changelog: add entries from secruity releases ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [ ] Re-reviewed `Files changed` in the Github PR explorer * docs: amend adr-54 with changes in the sdk (#4684) * blockchain/v2: don't broadcast base if height is 0 ## Description Fixes a bug where the reactor would broadcast a base with height=0. ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [x] Re-reviewed `Files changed` in the Github PR explorer * build(deps): bump @vuepress/plugin-google-analytics in /docs (#4692) Bumps [@vuepress/plugin-google-analytics](https://github.com/vuejs/vuepress/tree/HEAD/packages/@vuepress/plugin-google-analytics) from 1.3.1 to 1.4.1. - [Release notes](https://github.com/vuejs/vuepress/releases) - [Changelog](https://github.com/vuejs/vuepress/blob/master/CHANGELOG.md) - [Commits](https://github.com/vuejs/vuepress/commits/v1.4.1/packages/@vuepress/plugin-google-analytics) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * minor release process change ## Description The minor release process is changing in order to not have major release changes sitting in the pull request tab. This changes from taking master and releasing from master to creating a branch that you cherry-pick commits into. There are two options on labeling which pull requests to include in a minor release: 1. Use the label `R:minor` to know which pull requests to include then remove the label when those pull requests have been included in a release. 2. Create an Issue where pull request numbers are added. then the issue is closed when the release is done. this process should be followed after 0.33.3 ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [ ] Re-reviewed `Files changed` in the Github PR explorer * lite2: verify ConsensusHash in rpc client ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [x] Re-reviewed `Files changed` in the Github PR explorer * docs/architecture: create adr 56: prove amnesia attack ## Description ADR to address the process for proving an amnesia attack (as a form of global evidence) from `PotentialAmnesiaEvidence` detected by light clients ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [ ] Re-reviewed `Files changed` in the Github PR explorer * bump ABCI version to 0.16.2 due to ResponseCommit.retain_height See #4588 for original change. I believe this is appropriate. Anything else that needs to be updated? ______ For contributor use: - [ ] ~Wrote tests~ - [x] Updated CHANGELOG_PENDING.md - [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] ~Updated relevant documentation (`docs/`) and code comments~ - [x] Re-reviewed `Files changed` in the Github PR explorer * fix linter warnings * clarify service logging The service logging can be a bit unclear. For example, with state sync it would log: ``` I[2020-04-20|08:40:47.366] Starting StateSync module=statesync impl=Reactor I[2020-04-20|08:40:47.834] Starting state sync module=statesync ``` Where the first message is the reactor service startup, and the second message is the start of the actual state sync process. This clarifies the first message by changing it to `Starting StateSync service`. ______ For contributor use: - [ ] ~Wrote tests~ - [ ] ~Updated CHANGELOG_PENDING.md~ - [ ] ~Linked to Github issue with discussion and accepted design OR link to spec that describes this work.~ - [ ] ~Updated relevant documentation (`docs/`) and code comments~ - [x] Re-reviewed `Files changed` in the Github PR explorer * blockstore: allow initial SaveBlock() at any height Followup from #4588. Allow the first `SaveBlock()` call in an empty block store to be at any height, to start from a truncated block history. Subsequent `SaveBlock()` calls must be for contiguous blocks. ______ For contributor use: - [x] Wrote tests - [ ] ~Updated CHANGELOG_PENDING.md~ - [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [x] Updated relevant documentation (`docs/`) and code comments - [x] Re-reviewed `Files changed` in the Github PR explorer * ci: transition some ci to github actions ## Description move tests for abci_cli, abci_app and app_tests to github actions ______ For contributor use: - [ ] Wrote tests - [ ] Updated CHANGELOG_PENDING.md - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Updated relevant documentation (`docs/`) and code comments - [ ] Re-reviewed `Files changed` in the Github PR explorer * p2p/pex: fix DATA RACE in TestPEXReactorDialsPeerUpToMaxAttemptsInSeedMode Closes #4668 ______ For contributor use: - [x] Wrote tests - [ ] ~~Updated CHANGELOG_PENDING.md~~ - [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] ~~Updated relevant documentation (`docs/`) and code comments~~ - [x] Re-reviewed `Files changed` in the Github PR explorer * lite2/rpc: verify block results and validators (#4703) Closes: #4695 Verify /block_results and /validators responses from an HTTP client using the light client. Added count and total to /validators response. Refs #3113 * rc1/v0.33.4 Prepare release 0.33.4 * kick CI * abci/server: print panic & stack trace to STDERR if logger is not set Closes #4382 * test: fix p2p test build breakage caused by Debian testing (#4880) * types: implement Header#ValidateBasic (#4638) - Move core stateless validation of the Header type to a ValidateBasic method. - Call header.ValidateBasic during a SignedHeader validation. - Call header.ValidateBasic during a PhantomValidatorEvidence validation. - Call header.ValidateBasic during a LunaticValidatorEvidence validation. lite tests are skipped since the package is deprecated, no need to waste time on it closes: #4572 Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * lite2: fix TestVerifyAdjacentHeaders (#4884) * Makefile: parse TENDERMINT_BUILD_OPTIONS (#4738) (#4883) Reduce the number of targets and make the buildsystem more flexible by parsing the TENDERMINT_BUILD_OPTIONS command line variable (a-la Debian, inspired by dpkg-buildpackage's DEB_BUILD_OPTIONS), e.g: $ make install TENDERMINT_BUILD_OPTIONS='cleveldb' replaces the old: $ make install_c Options can be mix&match'd, e.g.: $ make install TENDERMINT_BUILD_OPTIONS='cleveldb race nostrip' Three options are available: - nostrip: don't strip debugging symbols nor DWARF tables. - cleveldb: use cleveldb as db backend instead of goleveldb; it switches on the CGO_ENABLED Go environment variale. - race: pass -race to go build and enable data race detection. This changeset is a port of gaia pull request: cosmos/gaia#363. Co-authored-by: Alessio Treglia <alessio@tendermint.com> * lite: fix HTTP provider error handling (#4882) * lite: fix HTTP provider error handling Fixes #4739, kind of. See #4740 for the proper fix. --- For contributor use: - [x] Wrote tests - [x] Updated CHANGELOG_PENDING.md - [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [x] Updated relevant documentation (`docs/`) and code comments - [x] Re-reviewed `Files changed` in the Github PR explorer - [x] Applied Appropriate Labels * adapt tests to missing pull request Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * privval: remove deprecated `OldFilePV` (#4889) * privval: remove deprecated `OldFilePV` The old format was deprecated in v0.28. It's time we remove it. * fix changelog * blockchain/v2: backport fixes (#4887) * blockchain/v2: fix excessive CPU usage due to spinning on closed channels (#4761) The event loop uses a `select` on multiple channels. However, reading from a closed channel in Go always yields the channel's zero value. The processor and scheduler close their channels when done, and since these channels are always ready to receive, the event loop keeps spinning on them. This changes `routine.terminate()` to not close the channel, and also removes `stopDemux` and instead uses `events` channel closure to signal event loop termination. Fixes #4687. * blockchain/v2: respect fast_sync option (#4772) Not thoroughly tested, but seems to work. Will do further testing as this is integrated with state sync. Fixes #4688. * types: return an error if voting power overflows in VerifyCommitTrusting Closes #4755 * rpc: handle panics during panic handling (#4888) Fixes #4802. The Go HTTP server has a global panic handler for requests, so it was not as severe as first thought. This fix can still panic, since we try to send a `500` response - if that happens, the Go HTTP server will terminate the connection. Otherwise, the client will get a 200 response, which we should avoid. I'm sort of torn on whether it's even necessary to include this fix, instead of just letting the HTTP server deal with it. * mempool: allow ReapX and CheckTx functions to run in parallel allow ReapX and CheckTx functions to run in parallel, making it not possible to block certain proposers from creating a new block. Closes: #2972 * pex: use highwayhash for pex bucket Closes #2998 Spec PR: tendermint/spec#94 * privval: retry GetPubKey/SignVote/SignProposal N times before returning an error Closes #4707 * rpc: use a struct to wrap all the global objects Closes #3433 * rpc: refactor lib folder (#4836) Closes tendermint/tendermint#3857 Moves `lib/` folder to `jsonrpc/`. Renames: **packages** `rpc` package -> `jsonrpc` package `rpcclient` package -> `client` package `rpcserver` package -> `server` package **structs and interfaces** ``` JSONRPCClient to Client JSONRPCRequestBatch to RequestBatch JSONRPCCaller to Caller ``` **functions** ``` StartHTTPServer to Serve StartHTTPAndTLSServer to ServeTLS rpc/jsonrpc/client: rename NewURIClient to NewURI NewJSONRPCClient to New NewJSONRPCClientWithHTTPClient to NewWithHTTPClient NewWSClient to NewWS ``` **misc** - unexpose `ResponseWriterWrapper` - remove unused http_params.go * rpc/core: do not lock ConsensusState mutex in /validators, /consensus_params and /status Closes #3161 * p2p: return masked IP (not the actual IP) in addrbook#groupKey Closes #4846 Spec tendermint/spec#96 * lite2: fix pivot height during bisection fix bug with assigning the new pivot height during bisection. PR: #4850 * proto: add proto files for ibc unblock (#4853) (#4906) these proto files are meant to help unblock ibc in their quest of migrating the ibc module to proto. * prepare v0.33.5 RC2 * types: verify commit fully Since the light client work introduced in v0.33 it appears full nodes are no longer fully verifying commit signatures during block execution - they stop after +2/3. See in VerifyCommit: https://github.com/tendermint/tendermint/blob/0c7fd316eb006c0afc13996c00ac8bde1078b32c/types/validator_set.go#L700-L703 This means proposers can propose blocks that contain valid +2/3 signatures and then the rest of the signatures can be whatever they want. They can claim that all the other validators signed just by including a CommitSig with arbitrary signature data. While this doesn't seem to impact safety of Tendermint per se, it means that Commits may contain a lot of invalid data. This is already true of blocks, since they can include invalid txs filled with garbage, but in that case the application knows they they are invalid and can punish the proposer. But since applications dont verify commit signatures directly (they trust tendermint to do that), they won't be able to detect it. This can impact incentivization logic in the application that depends on the LastCommitInfo sent in BeginBlock, which includes which validators signed. For instance, Gaia incentivizes proposers with a bonus for including more than +2/3 of the signatures. But a proposer can now claim that bonus just by including arbitrary data for the final -1/3 of validators without actually waiting for their signatures. There may be other tricks that can be played because of this. In general, the full node should be a fully verifying machine. While it's true that the light client can avoid verifying all signatures by stopping after +2/3, the full node can not. Thus the light client and full node should use distinct VerifyCommit functions if one is going to stop after +2/3 or otherwise perform less validation (for instance light clients can also skip verifying votes for nil while full nodes can not). See a commit with a bad signature that verifies here: 56367fd. From what I can tell, Tendermint will go on to think this commit is valid and forward this data to the app, so the app will think the second validator actually signed when it clearly did not. * consensus: Do not allow signatures for a wrong block in commits Closes #4926 The dump consensus state had this: "last_commit": { "votes": [ "Vote{0:04CBBF43CA3E 385085/00/2(Precommit) 1B73DA9FC4C8 42C97B86D89D @ 2020-05-27T06:46:51.042392895Z}", "Vote{1:055799E028FA 385085/00/2(Precommit) 652B08AD61EA 0D507D7FA3AB @ 2020-06-28T04:57:29.20793209Z}", "Vote{2:056024CFA910 385085/00/2(Precommit) 652B08AD61EA C8E95532A4C3 @ 2020-06-28T04:57:29.452696998Z}", "Vote{3:0741C95814DA 385085/00/2(Precommit) 652B08AD61EA 36D567615F7C @ 2020-06-28T04:57:29.279788593Z}", Note there's a precommit in there from the first val from May (2020-05-27) while the rest are from today (2020-06-28). It suggests there's a validator from an old instance of the network at this height (they're using the same chain-id!). Obviously a single bad validator shouldn't be an issue. But the Commit refactor work introduced a bug. When we propose a block, we get the block.LastCommit by calling MakeCommit on the set of precommits we saw for the last height. This set may include precommits for a different block, and hence the block.LastCommit we propose may include precommits that aren't actually for the last block (but of course +2/3 will be). Before v0.33, we just skipped over these precommits during verification. But in v0.33, we expect all signatures for a blockID to be for the same block ID! Thus we end up proposing a block that we can't verify. * update changelog and bump version * changelog: tweak 0.33.6 entry * privval: if remote signer errors, don't retry (#5140) Closes #5112 * prepare 0.33.7 release (#5202) * changelog: rc0/33.8 (#5222) * only retrieve pubkey once for all validators (partially fixes #4865) (#4895) in consensus/state.go, when calulating metrics, retrieve address (ergo, pubkey) once prior to iterating over validatorset to ensure we do not make excessive calls to signer. Partially closes: #4865 * consensus: only call privValidator.GetPubKey once per block (#5143) Closes #4865 * privval: increase read/write timeout to 5s and calculate ping interval based on it (#5638) Partially closes #5550 * bump version and update changelog * use fmt.Sprint for string to int conversion * chores: remove proto CI file * chores: upgrade CI tasks * chores: bump go version in CI tests * chores: restore lint version * fix: flaky test reporting invalid header time (#250) * fix #241 : flaky test reporting invalid header time Backported from v0.34.10 fix at tendermint/tendermint@4b99502 Co-authored-by: Callum Michael Waters <cmwaters19@gmail.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Tess Rinearson <tess.rinearson@gmail.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> Co-authored-by: Denis Fadeev <denis@fadeev.org> Co-authored-by: Greg Szabo <16846635+greg-szabo@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Yongwoo Lee <whylee259@gmail.com> Co-authored-by: Erik Grinaker <erik@interchain.berlin> Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Alessio Treglia <alessio@tendermint.com> Co-authored-by: Joe Bowman <joe@chorus.one>

Makefile: parse COSMOS_BUILD_OPTIONS Look for build options in the make's COSMOS_BUILD_OPTIONS variable. This allows one to build simd with ledger support. References: * cosmos/gaia#363 * tendermint/tendermint#4738 fix tiny typo in nolint: statement that causes golangci-lint to misbehave.

alessio force-pushed the alessio/dont-strip-debug-syms-by-default branch from 783d03d to 88234d5 Compare April 20, 2020 16:16

alessio marked this pull request as ready for review April 20, 2020 16:16

alessio requested review from alexanderbez, fedekunze, hschoenburg and rigelrozanski as code owners April 20, 2020 16:16

alessio added the R4R label Apr 20, 2020

alessio requested review from jackzampolin, jgimeno and tac0turtle April 20, 2020 16:17

tac0turtle approved these changes Apr 20, 2020

View reviewed changes

alessio merged commit 7eb955a into master Apr 20, 2020

alessio deleted the alessio/dont-strip-debug-syms-by-default branch April 20, 2020 19:37

alessio mentioned this pull request Apr 24, 2020

Makefile: parse TENDERMINT_BUILD_OPTIONS tendermint/tendermint#4738

Merged

6 tasks

alessio mentioned this pull request Jul 7, 2020

Makefile: build simd with ledger support cosmos/cosmos-sdk#6629

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Makefile: strip debug symbols by default #363

Makefile: strip debug symbols by default #363

alessio commented Apr 20, 2020 •

edited

Loading

codecov bot commented Apr 20, 2020 •

edited

Loading

tac0turtle left a comment

Makefile: strip debug symbols by default #363

Makefile: strip debug symbols by default #363

Conversation

alessio commented Apr 20, 2020 • edited Loading

codecov bot commented Apr 20, 2020 • edited Loading

Codecov Report

tac0turtle left a comment

Choose a reason for hiding this comment

alessio commented Apr 20, 2020 •

edited

Loading

codecov bot commented Apr 20, 2020 •

edited

Loading