Bitcoin Satellite 0.2.4

Bitcoin Satellite version 0.2.4 based on Bitcoin Core version 25.0 is now available.

The easiest way to download the upgrade is via the Blockstream Satellite command-line interface (CLI). You can do so by running the following command:

blocksat-cli deps update --btc

The update is available for the latest versions of Ubuntu, Fedora, Debian, and Raspberry Pi OS.

Please refer to the release notes for further information.

Bitcoin Satellite 0.2.4

Bitcoin Satellite version 0.2.4 is now available. This release includes minor improvements and updates the underlying Bitcoin Core version to 23.0.

The easiest way to download the upgrade is via the Blockstream Satellite command-line interface (CLI). You can do so by running the following command:

blocksat-cli deps update --btc

The update is available for the latest versions of Ubuntu, Fedora, Debian, Raspberry Pi OS, and CentOS.

Please refer to the release notes for further information.

Bitcoin Satellite 0.2.3

Bitcoin Satellite version 0.2.3 is now available with various improvements and bug fixes.

The easiest way to upgrade is via the Blockstream Satellite command-line interface (CLI). You can download the new version by running the following command:

blocksat-cli deps update --btc

The update is readily available for Ubuntu, Fedora, Debian, Raspberry Pi OS, and CentOS. Also, note Bitcoin Satellite 0.2.3 is based on the latest Bitcoin Core 22.0 version.

For further information, including the changelog, please refer to the release notes.

Bitcoin Satellite v0.2.2

Bitcoin Satellite version 0.2.2 is now available for Ubuntu, Fedora, and CentOS at:

• Launchpad (Debian packages)
• Copr (RPM packages)

If you are running the Blockstream Satellite command-line interface, you can upgrade Bitcoin Satellite by running:

blocksat-cli deps update --btc

For further information, including the changelog, please refer to the release notes.

v0.2.1 (on Bitcoin Core 0.21.1)

Bitcoin Satellite v0.2.1

Bitcoin Satellite version 0.2.1 is now available. For the release notes please see the git repository:

https://github.com/Blockstream/bitcoinsatellite/blob/master/doc/release-notes/satellite/release-notes-0.2.1.md

Changelog

Bitcoin Core Version

• Rebase Bitcoin Satellite v0.2.1 on top of Bitcoin Core 0.21.1.

FEC

• Update the Wirehair and cm256 sources to their latest versions.
• Fix fixture incompatibility and various other sources of instability on the
  FEC unit tests.
• Remove FEC test dependencies on Boost 1.61.
• Move the FEC implementation to a dedicated static library and assign the
  specific instruction subsets required to build it.
• Fix compilation based on native architecture (march=native) used
  previously to compile the FEC sources.

UDP Multicast Tx

• Make ring buffer and throttle unit tests less prone to racing conditions.
• Fix detection of full state on the ring buffer used for multicast Tx.

v0.2.0 (on Bitcoin Core 0.19.1)

UDP Multicast Tx

• Refactor the UDP Tx packet scheduler implementation on udpnet.cpp.
• Implement a standalone ring buffer class and use it on the new scheduler.
• Add a class to handle throttling in the UDP packet scheduler and reuse it to
  implement the throttling of txns sent over UDP multicast.
• Remove the tracking of block Tx window cycles (with debug=udpmulticast) due
  to inaccuracies that derive from the chunk interleaving approach.
• Fix checksum initialization on the UDP message header.
• Fix the computation of physical and logical UDP multicast stream indexes.
• Configure option udpmulticasttx based on an independent configuration file
  instead of using comma-separated arguments.
• Support the independent enabling of the block and txn streams composing each
  UDP multicast Tx instance.
• Support control of FEC overhead applied to block repetition loops via the new
  udpmulticasttx configuration file.
• Fix rounding of variable overhead used on FEC encoding.

UDP Multicast Rx

• Fix the UDP multicast Rx bitrate measurements that were not considering the
  variable-length UDP packets used for txns.

FEC

• Add unit tests covering fec.cpp.
• Support FEC decoding in both memory and mmap modes. Use the memory mode for
  new blocks and the mmap approach for historic (repeated) blocks.
• Optimize the fetching of wirehair-decoded FEC objects that are not pre-filled
  (i.e., historic blocks and large mempool txns).
• Update the naming format adopted for memory-mapped files storing partial
  FEC-encoded block data.
• Support persistence of partially-received FEC-encoded blocks across
  sessions. Keep the partial block files on disk until they are entirely decoded
  and reload partially-received block files on startup.

RPCs

Tx RPCs

• Add RPC gettxqueueinfo to fetch information from the transmit queues active
  in the UDP Tx packet scheduler.
• Add RPC gettxwindowinfo to fetch the UDP multicast Tx block window status
  and remove this information from the previous periodic debug logs.
• Add RPC gettxntxinfo for monitoring of mempool txn transmissions.
• Add RPC txblock for re-transmission of any arbitrary block (specified by
  height) over the UDP multicast Tx instances.

Rx RPCs

• Add RPC getfechitratio to fetch the txn and FEC chunk hit ratios
  corresponding to the txn/chunk pre-filling mechanism.
• Add RPC getudpmulticastinfo to read UDP multicast Rx information, including
  the measured bit rates.
• Add RPC getoooblocks to obtain information regarding out-of-order blocks.

Debug options

• Drop the previous optional debug printing of FEC chunk stats, deprecated in
  favor of debugging using the getchunkstats RPC.

v0.1.2 (on Bitcoin Core 0.19.1)

Bitcoin Satellite v0.1.2 (on Bitcoin Core 0.19.1)

New Features

• Add RPC to fetch Tx block interleave window info.
• Relay transactions received via udpmulticast.
• Let udpmulticasttx disable FEC block transmissions.

Optimizations

• Don't process repeated blocks if already available.
• Don't hold header data on the partial block struct.

Improvements

• Get height of partial block as soon as the header FEC is checked.
• Minor review of getchunkstats RPC help.

Fixes

• Fix processing of blocks combining tip and non-tip FEC chunks.

v0.19.1-satellite0.1: Bitcoin Satellite v0.1 (on Bitcoin Core 0.19.1)

[UDP Multicast Transport]

• Support multicast-addressed UDP transport of forward error correction (FEC)
  chunks containing data from new blocks, old (repeated) blocks, and mempool
  transactions (txns).
• Add two command-line options: udpmulticast and udpmulticasttx. The former
  handles reception of FEC chunks sent over multicast-addressed UDP datagrams.
  The latter activates multicast-addressed transmission of UDP messages.
• Support multiple instances of udpmulticast and udpmulticasttx bound to
  independent network interfaces.
• UDP Multicast Rx:
  • Support configuration of multicast Rx stream as a trusted UDP peer on
    Fibre's protocol.
  • Set the Rx socket buffer size to avoid buffer overflows.
  • Add optional debugging of the number of FEC chunks that are received,
    versus the number of chunks effectively used for FEC-decoding of each
    block. Track chunks received per multicast socket (per instance of option
    udpmulticast).
  • Add RPC getchunkstats for fetching of statistics regarding the partial
    blocks received over UDP.
• UDP Multicast Tx:
  • Support transmission of new blocks, old blocks and mempoll txns,
    time-multiplexed over the same UDP stream. Use separate threads for
    each of them.
  • For old blocks, iterates over a configurable number of past blocks, which
    is named as the "backfill depth". When the depth is set to 0, iterates
    over the full block chain.
  • Support a configurable number of mempool txns transmitted per second.
  • Support independent logical streams with different configurations (such
    as depth and DSCP of IP packets) when multiple udpmulticasttx options are
    set for the same network interface.
  • Rely on blocking socket transmissions, such that throttling of tx streams
    can be handled externally.
  • Support block interleaving on transmission of old (repeated)
    blocks. Instead of transmitting one block at a time, send a window of
    consecutive old blocks in parallel, while interleaving their FEC
    chunks. The goal is to minimize the chances of losing many chunks from the
    same block due to error bursts. Make the interleaving configurable in
    terms of the number of blocks that are sent in parallel.
• Full chain sync:
  • Force processing of blocks (even the initial blocks of the chain) received
    from trusted UDP multicast peers to allow full-chain synchronization.
  • Force the storage and acceptation/processing of out-of-order blocks that
    are far into the future with respect to the current chain tip, as long as
    these were received from trusted UDP peers. The goal is to make full-chain
    sync more efficient, as most likely the user will operate on out-of-order
    blocks until the Tx node wraps around to the beginning of the chain.
  • Send the height of each block as complimentary info on the block header
    structure sent over UDP multicast. This is meant to support storage of
    out-of-order blocks (OOOBs) specifically for pre-BIP34 blocks.

[Compression]

• Implement more efficient data representation (compression) of txns. Send
  compressed txns by default over the multicast UDP transport, both on
  FEC-coded blocks and on isolated mempool txn messages.
• Add support for multiple compressor/decompressor (codec) versions on
  txns. Send the codec version within the UDP block header information.

[Forward Error Correction]

• Review the FEC strategy with respect to the one adopted in Bitcoin Fibre.
  • Stop sending "uncoded" FEC chunks (the systematic part of the FEC-coded
    object) for blocks and rely only on "coded" chunks (the non-systematic
    part). Do so both for new blocks as well as old (repeated) blocks. The
    goal is to always support the combination of different UDP multicast
    streams, which is only efficient with coded chunks and with different
    (random) chunk ids from separate streams. Also, for new blocks, prefer
    "coded" chunks as the receive-end can use them to effectively fill in the
    erasures (unlike uncoded/systematic chunks).
  • For txns, always send uncoded chunks, so as to eliminate the inefficient
    repetition coding.
  • Stop sending the "body" of the FEC-coded block for blocks that contain
    only the coinbase and no other txn. For these, send only the header of the
    FEC-coded block, as it contain all the necessary information, including
    the coinbase txn as prefilled txn (c.f. BIP 152).
• Support non-padded chunks for txns., i.e. variable packet lengths for
  UDP multicast messages containing mempool txns, in contrast to block
  packets, which are of constant size. Most txns are smaller than the default
  FEC chunk size. Hence, by removing the padding of txns, there is significant
  saving of bytes sent over the multicast stream.
• Review the FEC overhead configurations for blocks. Assign a fixed overhead
  (currently of 60 chunks) plus a variable overhead based on the block size,
  which is currently set to 5% of the original number of chunks from the
  uncoded block.

[Misc - Fixes and Optimizations]

• Change buffer used for storing partial blocks to be processed. Use
  FIFO (queue) instead of LIFO.
• Support reception of FEC-coded block body before the reception of the
  block header.
• Add separate priority queue for mempool txns, which were previously
  classified with the same transmit priority of old (repeated) blocks. Now,
  prioritize mempool txns over old blocks, as the mempool txns are helpful to
  speed up decoding of new (chain tip) blocks on the receive end.
• Fix repeated transmission of parent mempool txns on the mempool txn
  transmission loop.
• Defer the processing of the partial block header of old (repeated) blocks to
  when both the header and the body are ready. This avoid loading too much
  data in memory when there is a long time between completion of the header
  and the body (such as when the blocks are interleaved).