almanac

A distributed log storage and serving system.

Design goals:

• Easy to deploy on cloud infrastructure such as GCP or AWS.
• Minimal operational burden, i.e., deployments should be easy to upgrade, restart, modify, etc.
• System cost scales with usage rather than uptime, making the system viable for small and large deployments.
• Does not require operating a resilient and fault-tolerant storage system.

Design

The design doc for the system can be found here. As parts of the design go from being under discussion to being more consolidated, the design will gradually move into markdown in this repo.

Building and running

If you have a working go environment, you will need to run the following as one-time setup:

• ./tools/fetch-deps.sh
• dep ensure

Running the demo

Run the demo binary by executing:

go run ./cmd/almanac/almanac.go

This will start a single-process cluster and will print the locations of a few relevant web pages which can be used to play around manually. By default, the demo runs against an in-memory storage implementation. In order to use an actual GCS bucket, execute:

GOOGLE_APPLICATION_CREDENTIALS=<path> go run ./cmd/almanac/almanac.go --storage=gcs --storage.gcs.bucket=<bucket>

Running tests

To run all the tests, execute:

go test ./...