This repository contains an example of using Hashicorp Raft v1.1 together with BadgerDB. It was forked from aalda/hashicorp-raft-example and updated for API changes, as well as project layout.
Assuming the compiled example
binary and a *nix system, you should be able
to form a working cluster by running the following three commands in three different terminals:
./example -id=server_0 -http="127.0.0.1:9000" -raft="127.0.0.1:10000" -datadir=/tmp/raft/server_0/data -raftdir=/tmp/raft/server_0/raft
./example -id=server_1 -http="127.0.0.1:9001" -raft="127.0.0.1:10001" -join="127.0.0.1:9000" -datadir=/tmp/raft/server_1/data -raftdir=/tmp/raft/server_1/raft
./example -id=server_2 -http="127.0.0.1:9002" -raft="127.0.0.1:10002" -join="127.0.0.1:9000" -datadir=/tmp/raft/server_2/data -raftdir=/tmp/raft/server_2/raft
Here, -datadir=...
specifies BadgerDB's data directory (provided to badger.Open(...)
),
whereas -raftdir=...
specifies the base directory of Raft's configuration, log and snapshot storage. Log and Config
directories are used by raftbadger.NewBadgerStore(...)
; the snapshot directory is unrelated to BadgerDB and created by
Raft.NewFileSnapshotStore().
To set keys, issue a POST
to /key
providing a JSON string map. The following example sets the keys
answer
to value42
andfoo
to valuebar
curl --location --request POST 'localhost:9000/key/' \
--header 'Content-Type: application/json' \
--data-raw '{
"answer": "41",
"foo": "bar"
}'
To get the values, issue a GET
to /key/{name}
, e.g.
curl --location --request GET 'localhost:9000/key/answer'
curl --location --request GET 'localhost:9000/key/foo'
To delete a key, issue a DELETE
to /key/{name}
, e.g.
curl --location --request DELETE 'localhost:9000/key/answer'
Note that while reading keys is possible on every node, creating and deleting keys is only possible on the leader node. Attempting to mutate state on a follower will result in a 500 Internal Server Error; it's a bit crude, but it gets the job done for an example.
This project follows the Standard Go Project Layout described here and here.
To get Go 1.15 on Ubuntu 20.04, run either:
sudo snap install --classic --channel=1.15/stable go
sudo snap refresh --classic --channel=1.15/stable go
Build the application with
go build -o example cmd/raft-example/main.go
To build a minimal image containing only the binary, run
docker build --target release -t raft-example .
or specify --target release
. Note that this build is without compiler
optimizations and inlining in order to help debugging (with Delve in particular).
Using Docker Compose for testing (see docker-compose.yaml) is a bit fiddly, but it is possible to get a configuration working by starting with
docker-compose up
in one terminal, then fiddling around with the other nodes in another terminal:
docker-compose stop node_1 node_2
docker-compose start node_1 node_2
To clean everything up, run
docker-compose stop
docker-compose rm -v
This may still leave the volume around, so use docker volume ls
to spy for it. You can then delete it
using a command similar to the following:
docker volume rm hashicorp-raft-example_raft-data
To build a Docker image containing both the sources and the binary, then shell into it, run:
docker build --target dev-env -t raft-example .
docker run --rm -it --entrypoint sh raft-example
To build for Delve:
docker build --target debug -t raft-example .
You can then run the application e.g. like so (here, passing the --help
command-line arguments).
docker run --rm -t -p 40000:40000 -t raft-example -- --help
From the host, connect using
dlv connect "localhost:40000"
To continue the application from there, run continue
in the debugger.
To run all tests, execute
go test ./...
To run the benchmarks, execute
go test ./... -run=XXX -bench=.