Welcome! 👋
This is an educational repository that includes a microservice written in Go. It is used as the principal example of my video series: Building Microservices in Go.
This repository is not a template nor a framework, it's a collection of patterns and guidelines I've successfully used to deliver enterprise microservices when using Go, and just like with everything in Software Development some trade-offs were made.
My end goal with this project is to help you learn another way to structure your Go project with 3 final goals:
- It is enterprise, meant to last for years,
- It allows a team to collaborate efficiently with little friction, and
- It is as idiomatic as possible.
Join the fun at https://youtube.com/MarioCarrion.
This project uses a lot of the ideas introduced by Eric Evans in his book Domain Driven Design, I do encourage reading that book but before I think reading Domain-Driven Design Distilled makes more sense, also there's a free to download DDD Reference available as well.
On YouTube I created a playlist that includes some of my favorite talks and webinars, feel free to explore that as well.
Talking specifically about microservices only, the structure I like to recommend is the following, everything using <
and >
depends on the domain being implemented and the bounded context being defined.
-
build/
: defines the code used for creating infrastructure as well as docker containers.-
<cloud-providers>/
: define concrete cloud provider. -
<executableN>/
: contains a Dockerfile used for building the binary.
-
-
cmd/
-
<primary-server>/
: uses primary database. -
<replica-server>/
: uses readonly databases. -
<binaryN>/
-
-
db/
-
migrations/
: contains database migrations. -
seeds/
: contains file meant to populate basic database values.
-
-
internal/
: defines the core domain.-
<datastoreN>/
: a concrete repository used by the domain, for examplepostgresql
-
http/
: defines HTTP Handlers. -
service/
: orchestrates use cases and manages transactions.
-
-
pkg/
public API meant to be imported by other Go package.
There are cases where requiring a new bounded context is needed, in those cases the recommendation would be to
define a package like internal/<bounded-context>
that then should follow the same structure, for example:
internal/<bounded-context>/
internal/<bounded-context>/<datastoreN>
internal/<bounded-context>/http
internal/<bounded-context>/service
Please refer to the documentation in internal/tools/.
Icons meaning:
In no particular order:
- Project Layout
- Dependency Injection
- Secure Configuration
- Using Hashicorp Vault
- Using AWS SSM
- Infrastructure as code
- Metrics, Traces and Logging using OpenTelemetry
- Caching
- Persistent Storage
- Repository Pattern
- Database migrations
- MySQL
- PostgreSQL
- REST APIs
- HTTP Handlers
- Custom JSON Types
- Versioning
- Error Handling
- OpenAPI 3 and Swagger-UI
- Authorization
- Events and Messaging
- Testing
- Type-safe mocks with
maxbrunsfeld/counterfeiter
- Equality with
google/go-cmp
- Integration tests for Datastores with
ory/dockertest
- REST APIs
- Type-safe mocks with
- Containerization using Docker
- Graceful Shutdown
- Search Engine using ElasticSearch
- Documentation
- Cloud Design Patterns
- Reliability
- Tools as Dependencies
- Whatever else I forgot to include
- 2016: Peter Bourgon's: Repository structure
- 2016: Ben Johnson's: Standard Package Layout
- 2017: William Kennedy's: Design Philosophy On Packaging
- 2017: Jaana Dogan's: Style guideline for Go packages
- 2018: Kat Zien - How Do You Structure Your Go Apps
Please notice in order to run this project locally you need to run a few programs in advance, if you use Docker please refer to the concrete instructions in docs/
for more details.
There's also a docker-compose.yml, covered in Building Microservices In Go: Containerization with Docker, however like I mentioned in the video you have to execute docker-compose
in multiple steps.
Notice that because of the way RabbitMQ and Kafka are being used they are sort of competing with each other, so at the moment we either have to enable Kafka and disable RabbitMQ or the other way around in both the code and the docker-compose.yml
file, in either case there are Dockerfiles and services defined that cover building and running them.
The following instructions are confirmed to work with docker compose v2.24.5-desktop.1.
- Run
docker-compose up
, if you're usingrabbitmq
orkafka
you may see the rest-server and elasticsearch-indexer services fail because those services take too long to start, in that case use any of the following instructions to manually start those services after the dependent server is ready:- If you're planning to use RabbitMQ, run
docker-compose up rest-server elasticsearch-indexer-rabbitmq
. - If you're planning to use Kafka, run
docker-compose up rest-server elasticsearch-indexer-kafka
.
- If you're planning to use RabbitMQ, run
- For building the service images you can use:
rest-server
image:docker-compose build rest-server
.elasticsearch-indexer-rabbitmq
image:docker-compose build elasticsearch-indexer-rabbitmq
.elasticsearch-indexer-kafka
image:docker-compose build elasticsearch-indexer-kafka
.elasticsearch-indexer-redis
image:docker-compose build elasticsearch-indexer-redis
.
- Run
docker-compose run rest-server tern migrate --migrations "/api/migrations/" --conn-string "postgres://user:password@postgres:5432/dbname?sslmode=disable"
to have everything working correctly. - Finally interact with the API using Swagger UI: http://127.0.0.1:9234/static/swagger-ui/
To start a local HTTP server that serves a graphical editor:
mdl serve github.com/MarioCarrion/todo-api/internal/doc -dir docs/diagrams/
To generate JSON artifact for uploading to structurizr:
stz gen github.com/MarioCarrion/todo-api/internal/doc