v1.6.8

Note

This release will reject connections from clients that don't support GZIP compression. Use --substreams-tier1-enforce-compression=false to keep previous behavior, then check the logs for incoming Substreams Blocks request logs with the value compressed: false to track users who are not using compressed HTTP connections.

• Substreams: add --substreams-tier1-enforce-compression to reject connections from clients that do not support GZIP compression
• Substreams performance: reduced the number of mallocs (patching some third-party libraries)
• Substreams performance: removed heavy tracing (that wasn't exposed to the client)
• Fixed reader-node-line-buffer-size flag that was not being respected in reader-node-stdin app
• Well-known chains: change genesis block for near-mainnet from 9820214 to 9820210
• BlockPoller library: reworked logic to support more flexible balancing strategy

v1.6.7

• firehose-grpc-listen-addr and substreams-tier1-grpc-listen-addr flags now accepts comma-separated addresses (allows listening as plaintext and snakeoil-ssl at the same time or on specific ip addresses)
• removed old RegisterServiceExtension implementation (not used anywhere anymore)
• rpc-poller lib: fix fetching the first block on an endpoint (was not following the cursor, failing unnecessarily on non-archive nodes)

v1.6.6

• Bump substreams and dmetering to latest version adding the outputModuleHash to metering sender.

v1.6.5

Substreams fixes

Note All caches for stores using the updatePolicy set_sum (added in substreams v1.7.0) and modules that depend on them will need to be deleted, since they may contain bad data.

• Fix bad data in stores using set_sum policy: squashing of store segments incorrectly "summed" some values that should have been "set" if the last event for a key on this segment was a "sum"
• Fix small bug making some requests in development-mode slow to start (when starting close to the module initialBlock with a store that doesn't start on a boundary)

Others

• [Operator] Node Manager HTTP /v1/resume call now accepts extra-env=<key>=<value>&extra-env=<keyN>=<valueN> enabling to override environment variables for the next restart only. Use curl -XPOST "http://localhost:10011/v1/resume?sync=true&extra-env=NODE_DEBUG=true" (change localhost:10011 accordingly to your setup).

  This is not persistent upon restart!

• [Metering] Revert undesired Firehose metric Endpoint changes, the correct new value used is sf.firehose.v2.Firehose/Blocks (had been mistakenly set to sf.firehose.v2.Firehose/Block between version v1.6.1 and v1.6.4 inclusively).

v1.6.4

Substreams fixes

• Fixed an(other) issue where multiple stores running on the same stage with different initialBlocks will fail to proress (and hang)

v1.6.3

Substreams fixes

• Fix "cannot resolve 'old cursor' from files in passthrough mode" error on some requests with an old cursor
• Fix handling of 'special case' substreams module with only "params" as its input: should not skip this execution (used in graph-node for head tracking)
  -> empty files in module cache with hash d3b1920483180cbcd2fd10abcabbee431146f4c8 should be deleted for consistency
• Fix bug where some invalid cursors may be sent (with 'LIB' being above the block being sent) and add safeguard/loggin if the bug appears again
• Fix panic in the whole tier2 process when stores go above the size limit while being read from "kvops" cached changes

Core fixes

• fix: reader-node-stdin not shutting down after receiving an EOF

v1.6.2

Core

• [Operator] The flag --advertise-block-id-encoding now accepts shorter form: hex, base64, etc. The older longer form BLOCK_ID_ENCODING_HEX is still supported but we suggested using the shorter form from now on.

Substreams v1.10.2

Note Since a bug that affected substreams with "skipping blocks" was corrected in this release, any previously produced substreams cache should be considered as possibly corrupted and be eventually replaced

• Substreams: fix bad handling of modules with multiple inputs when only one of them is filtered, resulting in bad outputs in production-mode.
• Substreams: fix stalling on some substreams with stores and mappers with different start block numbers on the same stage
• Substreams: fix 'development mode' and LIVE mode executing some modules that should be skipped

v1.6.1

• Bump substreams to v1.10.0: Version 1.10.0 adds a new EndpointInfo/Info endpoint, introduces a 3-minute default execution timeout per block, updates metering metrics with a deprecation warning, enhances substreams init commands, and improves wasm module caching and Prometheus tool flexibility. Full changelog: https://github.com/streamingfast/substreams/releases/tag/v1.10.0
• Metering update: more detailed metering with addition of new metrics. DEPRECATION WARNING: bytes_read and bytes_written metrics will be removed in the future, please use the new metrics for metering instead

v1.6.0

• Add sf.firehose.v2.EndpointInfo/Info service on Firehose and sf.substreams.rpc.v2.EndpointInfo/Info to Substreams endpoints. This involves the following new flags:

  • advertise-chain-name Canonical name of the chain according to https://thegraph.com/docs/en/developing/supported-networks/ (required, unless it is in the "well-known" list)
  • advertise-chain-aliases Alternate names for that chain (optional)
  • advertise-block-features List of features describing the blocks (optional)
  • advertise-block-id-encoding Encoding format of the block ID [BLOCK_ID_ENCODING_BASE58, BLOCK_ID_ENCODING_BASE64, BLOCK_ID_ENCODING_BASE64URL, BLOCK_ID_ENCODING_HEX, BLOCK_ID_ENCODING_0X_HEX] (required, unless the block type is in the "well-known" list)
  • ignore-advertise-validation Runtime checks of chain name/features/encoding against the genesis block will no longer cause server to wait or fail.

• Add a well-known list of chains (hard-coded in wellknown/chains.go to help automatically determine the 'advertise' flag values). Users are encouraged to propose Pull Requests to add more chains to the list.

• The new info endpoint adds a mandatory fetching of the first streamable block on startup, with a failure if no block can be fetched after 3 minutes and you are running firehose or substreams-tier1 service.
  It validates the following on a well-known chain:

  • if the first-streamable-block Num/ID match the genesis block of a known chain, e.g. matic, it will refuse another value for advertise-chain-name than matic or one of its aliases (polygon)
  • If the first-streamable-block does not match any known chain, it will require the advertise-chain-name to be non-empty
  • If the first-streamable-block type is unknown (i.e. not ethereum, solana, near, cosmos, bitcoin...), it will require the user to provide advertise-chain-name as well as advertise-block-id-encoding

• Substreams: add --common-tmp-dir flag to activate local caching of pre-compiled WASM modules through wazero v1.8.0 feature (performance improvement on WASM compilation)

• Substreams: revert module hash calculation from v1.5.5, when using a non-zero firstStreamableBlock. Hashes will now be the same even if the chain's first streamable block affects the initialBlock of a module.

• Substreams: add --substreams-block-execution-timeout flag (default 3 minutes) to prevent requests stalling

v1.5.7

• Bump substreams to v1.9.3: fix high CPU usage on tier1 caused by a bad error handling