-
Notifications
You must be signed in to change notification settings - Fork 35
Goals
moulliet edited this page Sep 30, 2016
·
5 revisions
- Easy Access To Data
- Simple Protocol
- Replay and Auditing
- Data Agnostic
- Modular Design
- Fault Tolerance
- Loose Coupling
- Distributed Parallel Clients
We want to make sharing data between teams as easy as possible.
- The data a hub contains is easily discovered and consumed
- Channels are used to separate data from different sources
- Tags are used to group multiple related channels together
- All interactions with the hub use HTTP
- Clients can be written in any language
- We can leverage existing expertise with proxies, load balancers and authentication
The Hub can answer the following questions about a channel:
- What is the oldest item?
- What is the most recent item?
- What items where written during a time period (day, hour, minute, second)?
- What are the next or previous items relative to a specific time or item?
All data in the hub is immutable and indexed by time with consistent ordering
Users can store any type of data in the hub, from plain text to binary files. Content-type is used by data producers to indicate data type.
The simple queries mean all questions can be answered by using the union of all available data sources. todo talk about Spoke and both S3 versions, as well as the local version.
The modular design has allowed us to more easily add additional features.