Project setup

.dockerignore

@@ -0,0 +1,12 @@
+example.env
+a
+*.puml
+*.png
+*.md
+.github
+.idea
+.gitignore
+docker/
+server_exec
+client_exec
+*.yaml

.github/workflows/build.yml

@@ -0,0 +1,41 @@
+name: build
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+jobs:
+  build:
+    name: Build
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Set up Go
+        uses: actions/setup-go@v2
+        with:
+          go-version: '^1.22'
+
+      - name: Check out code into the Go module directory
+        uses: actions/checkout@v2
+
+      - name: Build the client
+        run: |
+          cd src/cmd/client
+          go build -o clientexec
+
+      - name: Check if client executable was created
+        run: |
+          ls src/cmd/client/clientexec
+
+      - name: Build the consumerService
+        run: |
+          cd src/cmd/server
+          go build -o serverexec
+
+      - name: Check if consumerService executable was created
+        run: |
+          ls src/cmd/server/serverexec

.github/workflows/tests.yml

@@ -0,0 +1,29 @@
+name: tests
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+jobs:
+  test:
+    name: Run Go Tests
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Set up Go
+        uses: actions/setup-go@v2
+        with:
+          go-version: '^1.22'
+
+      - name: Check out code
+        uses: actions/checkout@v2
+
+      - name: Get dependencies
+        run: go mod tidy
+
+      - name: Run tests
+        run: go test -v ./...

.gitignore

@@ -0,0 +1,29 @@
+# Binaries for programs and plugins
+*.exe
+*.exe~
+*.dll
+*.so
+*.dylib
+
+# Test binary, built with `go test -c`
+*.test
+
+# Output of the go coverage tool, specifically when used with LiteIDE
+*.out
+
+# Dependency directories (remove the comment below to include it)
+# vendor/
+
+# Go workspace file
+go.work
+
+.DS_Store
+.idea/
+.env
+a
+
+*.json
+*.js
+node_modules/
+server_exec
+client_exec

LICENSE.md

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Stefan Crnojević
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,145 @@
+
+[//]: # (Badges)
+
+[![Author](https://img.shields.io/badge/author-zarnoevic-blue)]()
+[![Language](https://img.shields.io/badge/go-1.22.0-blue)](https://github.com/zarnoevic)
+[![License](https://img.shields.io/github/license/zarnoevic/go-rabbitmq?color=green)](https://github.com/zarnoevic/go-rabbitmq/blob/main/LICENSE.md)
+[![Go Report Card](https://goreportcard.com/badge/github.com/zarnoevic/go-rabbitmq)](https://goreportcard.com/report/github.com/zarnoevic/go-rabbitmq)
+[![Build](https://img.shields.io/github/actions/workflow/status/zarnoevic/go-rabbitmq/build.yml?branch=main&event=push&label=build)](https://github.com/zarnoevic/go-rabbitmq/actions/workflows/build.yml)
+[![Tests](https://img.shields.io/github/actions/workflow/status/zarnoevic/go-rabbitmq/tests.yml?branch=main&event=push&label=tests)](https://github.com/zarnoevic/go-rabbitmq/actions/workflows/tests.yml)
+
+# Golang RabbitMQ Server-Client Framework
+
+This is a simple framework for a RabbitMQ server-client communication. 
+The server is a consumer service that listens to a queue and processes the messages. 
+The client is a producer service that sends messages to the queue.
+Each message is of a predefined format and denotes a task over an ordered mapping, whose state is kept, accessed, and updated by the server.
+
+## Results
+
+On a local Macbook Pro with 16GB of RAM and an M1 chip, the processing of 100,000 commands took `6.46` seconds, with both
+the client and the server instances running on the same machine, with the settings as provided in the `example.env` file.
+This is from the moment the first record is read from the CSV file on the client until the moment the last record is read from the 
+RabbitMQ and processed by the server.
+
+The following image demonstrates the snapshot of the state of RabbitMQ during processing.
+
+![processing.png](processing.png)
+
+We notice that the consumption tightly follows the production, with the server processing the messages as soon as they 
+are produced by the client, as would be expected in a real-time system.
+
+## Specification
+
+This is a visual (UML) representation of the framework.
+
+![img.png](diagram.png)
+
+
+## Environment setup
+
+Make your own `.env` file from the `.env.example` file.
+
+## Running the Project
+
+The project can be run in two ways: with docker-compose or directly on your machine.
+A separate docker-compose file is provided for each service, as well as a dockerfile for each service.
+This allows for a scalable deployment of the services, as they can be run in parallel and independently of each other, on different instances.
+
+### Running RabbitMQ
+
+Run the following command to start the service:
+
+```shell
+docker-compose -f rabbitClient.docker-compose.yaml --env-file .env up -d
+```
+
+### Running the Client
+
+Run the following command to start the Client on your machine:
+
+```shell
+go run src/cmd/client/main.go
+```
+
+#### With Docker
+
+```shell
+docker build -t client -f client.Dockerfile .
+docker run --network="host" --env-file .env client
+```
+
+#### Scaling the Client
+
+Run the following command to start the Server in a scalable way with docker compose:
+
+```shell
+docker-compose -f client.docker-compose.yaml --env-file .env up -d --scale client=4
+```
+
+Change the `--scale client=4` parameter to any number of clients you'd want to run in parallel.
+
+### Running the Server
+
+Run the following command to start the Server on your machine:
+
+#### Without Docker
+
+```shell
+go run src/cmd/server/main.go
+```
+
+#### With Docker
+
+```shell
+docker build -t server -f server.Dockerfile .
+docker run --network="host" --env-file .env server
+```
+
+#### Scaling the Server
+
+Run the following command to start the Server in a scalable way with docker compose:
+
+```shell
+docker-compose -f server.docker-compose.yaml --env-file .env up -d --scale server=4
+```
+
+Change the `--scale server=4` parameter to any number of clients you'd want to run in parallel.
+
+## File structure
+
+A snapshot of the file structure is provided below using the `tree` command for quick reference.
+
+```
+.
+├── .env
+├── go.mod
+├── go.sum
+├── client.Dockerfile
+├── client.docker-compose.yaml
+├── rabbitmq.docker-compose.yaml
+├── server.Dockerfile
+├── server.docker-compose.yaml
+├── resources
+│   ├── commands100.csv
+│   ├── commands100k.csv
+│   ├── commands10k.csv
+│   └── commands1k.csv
+└── src
+    ├── cmd
+    │   ├── client
+    │   │   └── main.go
+    │   └── server
+    │       └── main.go
+    └── pkg
+        ├── orderedmap
+        │   ├── orderedmap.go
+        │   └── orderedmap_test.go
+        ├── rabbitClient
+        │   └── rabbitClient.go
+        └── services
+            ├── consumerService
+            │   └── consumerService.go
+            └── producerService
+                └── producerService.go
+```

client.Dockerfile

@@ -0,0 +1,11 @@
+FROM --platform=linux/amd64 golang:1.22 as builder
+
+WORKDIR /app
+
+COPY . .
+
+RUN go mod download
+RUN go mod tidy
+RUN go build -o client_exec ./src/cmd/client/
+
+CMD ["./client_exec"]

client.docker-compose.yaml

@@ -0,0 +1,17 @@
+version: '3.8'
+
+services:
+  client:
+    build:
+      context: .
+      dockerfile: client.Dockerfile
+    environment:
+      RABBITMQ_USER: ${RABBITMQ_USER}
+      RABBITMQ_PASSWORD: ${RABBITMQ_PASSWORD}
+      RABBITMQ_AMQP_PORT: ${RABBITMQ_AMQP_PORT}
+      RABBITMQ_AMQP_HOST: ${RABBITMQ_AMQP_HOST}
+      COMMANDS_PATH: ${COMMANDS_PATH}
+    network_mode: host
+    volumes:
+      - ./resources:/resources
+

diagram.puml

@@ -0,0 +1,49 @@
+@startuml
+' https://plantuml.com/sequence-diagram
+
+title __Producer-Consumer Model__
+
+autonumber
+
+' General Formatting
+skinparam responseMessageBelowArrow true
+skinparam lifelineStrategy solid
+hide footbox
+
+' Protocol Participants
+box System
+  database      CommandsSource          as cs       order 5
+  entity        Producer                as p        order 10
+  queue         RabbitMQ                as rq       order 40
+  entity        Consumer                as c        order 70
+  database      OrderedMap              as om       order 130
+end box
+
+' Protocol Context & Assumptions
+/ note over om
+  In-Memory store
+end note
+/ note over cs
+  Pre-generated file
+end note
+/ note over c
+  Server
+end note
+/ note over p
+  Client
+end note
+
+
+' Protocol
+
+cs <-> p : Producer reads the commands
+p -> rq : Commands are sent to the queue
+loop
+ rq <-> c : Consumer listens for commands
+ c->om : Processes get, add, delete, getAll
+end
+
+|||
+== Protocol Continues Indefinitely ==
+@enduml
+

example.env

@@ -0,0 +1,10 @@
+RABBITMQ_USER=admin
+RABBITMQ_PASSWORD=i3gW0n4suKr2Ojzi78ShL4pKY3IzZn
+RABBITMQ_QUEUE_NAME=commands
+RABBITMQ_AMQP_HOST=127.0.0.1
+RABBITMQ_AMQP_PORT=5672
+RABBITMQ_UI_PORT=15672
+
+COMMANDS_PATH=resources/commands100k.csv
+
+SERVER_WORKERS=100

go.mod

@@ -0,0 +1,8 @@
+module github.com/zarnoevic/go-rabbitmq
+
+go 1.22.0
+
+require (
+	github.com/joho/godotenv v1.5.1
+	github.com/rabbitmq/amqp091-go v1.9.0
+)