Skip to content

Latest commit

 

History

History
282 lines (190 loc) · 7.77 KB

README.md

File metadata and controls

282 lines (190 loc) · 7.77 KB
Raw

GWOS Logo

License Build Go Report Card GoDoc

The Transit Connection Generator (TCG). TCG contains two sub-systems/packages:

  1. Transit agent - connects and sends metrics to Groundwork Monitor
  2. Controller service - an http server for external control of agent

Table of Contents

  1. Dependencies
  2. Building
  3. Running
  4. Docker
  5. Testing
  6. Environment variables

Dependencies

The TCG project is built with Go Modules. See go.mod for a list of dependencies. Here are some main frameworks used by this project:

  1. Gin Web Framework

    Gin is a web framework written in Go (Golang). It features a martini-like API with much better performance, up to 40 times faster.

     github.com/gin-gonic/gin

  2. Sessions

    Gin middleware for session management with multi-backend support.

     github.com/gin-gonic/contrib/sessions

  3. NATS Streaming System

    About NATS

     github.com/nats-io/go-nats-streaming
 github.com/nats-io/nats-streaming-server/server
 github.com/nats-io/nats-streaming-server/stores

  4. Envconfig

    Package envconfig implements decoding of environment variables based on a user defined specification. A typical use is using environment variables for configuration settings.

     github.com/kelseyhightower/envconfig

  5. Go-Cache

    Go-Cache is an in-memory key:value store/cache similar to memcached that is suitable for applications running on a single machine. Its major advantage is that, being essentially a thread-safe map[string]interface{} with expiration times, it doesn't need to serialize or transmit its contents over the network. Any object can be stored, for a given duration or forever, and the cache can be safely used by multiple goroutines.

     github.com/patrickmn/go-cache

  6. Testify

    Go code (golang) set of packages that provide many tools for testifying that your code will behave as you intend.

     github.com/stretchr/testify

  7. Zerolog

    The zerolog package provides a fast and simple logger dedicated to JSON output.

     github.com/rs/zerolog

  8. Gopsutil

    The challenge is porting all psutil functions on some architectures.

     github.com/shirou/gopsutil

  9. Gin-Swagger

    Gin Gonic middleware to automatically generate RESTful API documentation with Swagger 2.0.

     github.com/swaggo/gin-swagger

    Generate 'docs.go' for Swagger UI

     $ swag init

    Swagger url:

     {host}:{port}/swagger/index.html

Building

$ cd tcg
$ go build .

Building tcg shared module:

$ go build -ldflags "-X 'github.com/gwos/tcg/config.buildTime=`date --rfc-3339=s`' -X 'github.com/gwos/tcg/config.buildTag=<TAG>'" -buildmode=c-shared -o libtransit/libtransit.so libtransit/libtransit.go

or use Makefiles

Building Connectors:

LINUX:

$ cd connectors/<connector>
$ go build -ldflags "-X 'github.com/gwos/tcg/config.buildTime=`date --rfc-3339=s`' -X 'github.com/gwos/tcg/config.buildTag=<TAG>'"

OS X:

$ cd connectors/<connector>
$ go build -ldflags "-X 'github.com/gwos/tcg/config.buildTime=`date -u +"%Y-%m-%dT%H:%M:%SZ"`' -X 'github.com/gwos/tcg/config.buildTag=<TAG>'"

Building Connectors for OS Targets (Cross Compiling)

env GOOS=linux GOARCH=386 go build -ldflags "-X 'github.com/gwos/tcg/config.buildTime=`date -u +"%Y-%m-%dT%H:%M:%SZ"`' -X 'github.com/gwos/tcg/config.buildTag=8.1.0.1'"

To view supported platforms use commands

go tool dist list
go tool dist list -json
go tool dist list -json | jq '.[] | select(.CgoSupported == true)'

Installing as a service

To enable:

sudo systemctl enable tcg-elastic

To start:

sudo systemctl start tcg-elastic

Show status:

sudo systemctl status tcg-elastic

To stop:

sudo systemctl stop tcg-elastic

To disable:

sudo systemctl disable tcg-elastic

To reconfigure:

sudo systemctl daemon-reload

To tail:

journalctl -f -u tcg-elastic

Running

$ cd tcg
$ go run .

Docker

Build image:

$ docker build -t groundworkdevelopment/tcg --build-arg GITHUB_TOKEN .

Testing

The gotests tool can generate Go tests.

Run package tests:

With logs:

$ go test -v ./<package_one>/ ./<package_two>/

Run integration tests:

For integration tests you must provide environment variables for Groundwork Connection. Also have to deal with TLS certs: get it in local trust storage or suppress check.

$ TCG_TLS_CLIENT_INSECURE=TRUE \
    TCG_GWCONNECTIONS_0_USERNAME=remote TCG_GWCONNECTIONS_0_PASSWORD=remote \
    go test -v ./integration/

Examples:

$ go test -v ./config/ ./services/

$ go test -v ./libtransit/

$ go test -v $(go list ./... | grep -v tcg/integration)

$ GOFLAGS="-count=1" \
    TCG_TLS_CLIENT_INSECURE=TRUE \
    TCG_GWCONNECTIONS_0_USERNAME=remote TCG_GWCONNECTIONS_0_PASSWORD=remote \
    TCG_CONNECTOR_LOGCOLORS=TRUE TCG_CONNECTOR_LOGLEVEL=3 \
    OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 \
    TCG_CONNECTOR_AGENTID=TEST11 \
    go test -failfast -v ./integration/

$ GOFLAGS="-count=1" \
    TCG_TLS_CLIENT_INSECURE=TRUE \
    TCG_GWCONNECTIONS_0_USERNAME=remote TCG_GWCONNECTIONS_0_PASSWORD=remote \
    TCG_CONNECTOR_LOGCOLORS=TRUE TCG_CONNECTOR_LOGLEVEL=3 \
    __OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 \
    __TCG_HTTP_CLIENT_TIMEOUT_GW=120s \
    __TEST_FLAG_CLIENT=true \
    TEST_RESOURCES_COUNT=40 TEST_SERVICES_COUNT=100 \
    TCG_CONNECTOR_BATCHMETRICS=1s TCG_CONNECTOR_BATCHMAXBYTES=204800  TCG_CONNECTOR_NATSMAXPAYLOAD=40920 \
    go test -benchtime=10x -benchmem -run=^$ -bench ^BenchmarkE2E$  ./integration/ \
    | grep _STATS

Environment variables

LIBTRANSIT

Defines the path to libtransit.so library in docker container and tests.

$ export LIBTRANSIT=/path/to/libtransit.so

TCG config variables

By default, the config file is looked for in the work directory as tcg_config.yaml.

The path to config file and any config option can be overridden with env vars:

$ export TCG_CONFIG=/path/to/tcg_config.yaml
$ export TCG_CONNECTOR_NATSSTORETYPE=MEMORY

For more info see package config and tests.

Other variables

There are additional variables supported:

* OTEL_EXPORTER_OTLP_ENDPOINT=http://jaegertracing:4317
* TCG_HTTP_CLIENT_TIMEOUT=10s
* TCG_HTTP_CLIENT_TIMEOUT_GW=120s
* TCG_SUPPRESS_DOWNTIMES=true
* TCG_SUPPRESS_EVENTS=true
* TCG_SUPPRESS_INVENTORY=true
* TCG_SUPPRESS_METRICS=true

Run golangci-lint locally:

$ go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.54.2
$ ~/go/bin/golangci-lint --config ./.golangci.yaml run ./... --deadline=2m