The StreamingFast platform

     StreamingFast is an Open Source suite of products that enables low-latency, real-time processing of blockchain data streams, allows for massively parallelizable operations over historical data, and provides the robustness and reliability required by the most demanding loads.

Features

It is protocol-aware, supports multiple chain protocols and can be used to build higher-order blockchain data services.

StreamingFast products include a powerful real-time search engine (StreamingFast Search), mempool management technologies (StreamingFast Lifecycle), historical state services and several specialized indexers.

Installation

Each blockchain protocol has its own installation method. See section below.

Protocols

Here's a list of blockchain protocols that integrate with the StreamingFast stack:

• EOSIO on StreamingFast
• Ethereum on StreamingFast

Overview

The StreamingFast platform is composed of multiple open-source repositories. Here is a breakdown.

Top-tier products / solutions:

• StreamingFast Search, distributed, real-time, fork-aware, general purpose search engine.
• StreamingFast FluxDB, state snapshotting service for EOSIO.
• EOSIO Websocket interface
• EOSIO Push Guarantee
• EOSIO Transaction Lifecycle
• EOSIO Blockchain node manager
• EOSIO Point-in-time recovery backup solution (for EOSIO state & blocks files)
• Ethereum Transaction Lifecycle service, coming as Open Source soon
• Ethereum Historical State service, coming as Open Source soon
• Ethereum Speculative EVM execution engine, coming as Open Source soon

Common interfaces:

• gRPC service definitions and common Protobuf data models.
• Common GraphQL service (see protocol-specific resolvers and schemas)
• REST + WS for EOSIO

Second-tier service mesh:

• merger, collecting all forks info, for high-availability deployments.
• relayer, streaming blocks relayer, for high-availaibility deployments.
• block metadata service, offers fast access to the state of the chain's blocks, aware of on-going forks, finality, etc.

Documentation:

• The StreamingFast docs rendered at https://docs.dfuse.io

Building blocks:

• The bstream blocks & transactions streaming library
• The kvdb key-value storage abstraction

More general purpose Go tooling provided by StreamingFast:

• Errors management library
• Object storage abstraction
• Binary file packer
• gRPC tools
• Authentication Plugins Interfaces
• Consumption & Metering Plugins Interface
• Mesh Service Discovery library (used by Search)
• Simple obfuscation library

License

All StreamingFast components are generally Apache 2.0 licensed. See repositories for details.