Compass is a search and discovery engine built for querying application deployments, datasets and meta resources. It can also optionally track data flow relationships between these resources and allow the user to view a representation of the data flow graph.
Discover why users choose Compass as their main data discovery and lineage service
- Full text search Faster and better search results powered by ElasticSearch full text search capability.
- Search Tuning Narrow down your search results by adding filters, getting your crisp results.
- Data Lineage Understand the relationship between metadata with data lineage interface.
- Scale: Compass scales in an instant, both vertically and horizontally for high performance.
- Extensibility: Add your own metadata types and resources to support wide variety of metadata.
- Runtime: Compass can run inside VMs or containers in a fully managed runtime environment like kubernetes.
Explore the following resources to get started with Compass:
- Guides provides guidance on ingesting and querying metadata from Compass.
- Concepts describes all important Compass concepts.
- Reference contains details about configurations, metrics and other aspects of Compass.
- Contribute contains resources for anyone who wants to contribute to Compass.
Install Compass on macOS, Windows, Linux, OpenBSD, FreeBSD, and on any machine.
Refer this for installations and configurations
Download the appropriate version for your platform from releases page. Once downloaded, the binary can be run from anywhere.
You don’t need to install it into a global location. This works well for shared hosts and other systems where you don’t have a privileged account.
Ideally, you should install it somewhere in your PATH for easy use. /usr/local/bin is the most probable location.
compass is available via a Homebrew Tap, and as downloadable binary from the releases page:
brew install goto/tap/compassTo upgrade to the latest version:
brew upgrade compass
Check for installed compass version
compass versioncompass is available as downloadable binaries from the releases page. Download the .deb or .rpm from the releases page and install with sudo dpkg -i and sudo rpm -i respectively.
compass is available via scoop, and as a downloadable binary from the releases page:
scoop bucket add compass https://github.com/goto/scoop-bucket.git
To upgrade to the latest version:
scoop update compass
We provide ready to use Docker container images. To pull the latest image:
docker pull gotocompany/compass:latest
To pull a specific version:
docker pull gotocompany/compass:v0.3.2
If you like to have a shell alias that runs the latest version of compass from docker whenever you type compass:
mkdir -p $HOME/.config/gotocompany
alias compass="docker run -e HOME=/tmp -v $HOME/.config/gotocompany:/tmp/.config/gotocompany --user $(id -u):$(id -g) --rm -it -p 3306:3306/tcp gotocompany/compass:latest"
Compass is purely API-driven. It is very easy to get started with Compass. It provides CLI, HTTP and GRPC APIs for simpler developer experience.
Compass CLI is fully featured and simple to use, even for those who have very limited experience working from the command line. Run compass --help to see list of all available commands and instructions to use.
List of commands
compass --help
Print command reference
compass referenceCompass provides a fully-featured GRPC and HTTP API to interact with Compass server. Both APIs adheres to a set of standards that are rigidly followed. Please refer to proton for GRPC API definitions.
Dependencies:
-
Compass is written in Golang, and requires go version >= 1.21. Please make sure that the go toolchain is available on your machine. See Golang’s documentation for installation instructions. Alternatively, you can use docker to build Compass as a docker image. More on this in the next section.
-
Compass uses PostgreSQL 13 as its main storage and Elasticsearch v7 as the secondary storage to power the search. In order to run compass locally, you’ll need to have an instance of postgres and elasticsearch running. You can either download them and run it manually, or you can run them inside docker by using
docker-composewithdocker-compose.yamlprovided in the root of this project. -
PostgreSQL details and Elasticsearch brokers can alternatively be specified via the environment variable,
ELASTICSEARCH_BROKERSfor elasticsearch andDB_HOST,DB_NAME,DB_USER,DB_PASSWORDfor postgres. -
If you use Docker to build compass, then configuring networking requires extra steps. Following is one of doing it by running postgres and elasticsearch inside with
docker-composefirst. -
Go to the root of this project and run
docker-compose.docker-compose up -
Once postgres and elasticsearch has been ready, we can run Compass by passing in the config of postgres and elasticsearch defined in
docker-compose.yamlfile.
Begin by cloning this repository then you have two ways in which you can build compass
- As a native executable
- As a docker image
To build compass as a native executable, run make inside the cloned repository.
make
This will create the compass binary in the root directory
Building compass' Docker image is just a simple, just run docker build command and optionally name the image
docker build . -t compass
Before serving Compass app, we need to run the migration first. Run this docker command to migrate Compass.
$ docker run --rm --net compass_storage -p 8080:8080 -e ELASTICSEARCH_BROKERS=http://es:9200 -e ELASTICSEARCH_USERNAME= -e ELASTICSEARCH_PASSWORD= -e DB_HOST=postgres -e DB_PORT=5432 -e DB_NAME=compass -e DB_USER=compass -e DB_PASSWORD=compass_password gotocompany/compass compass server migrate
If you are using Compass binary, you can run this command.
./compass -elasticsearch-brokers "http://<broker-host-name>" -elasticsearch-username "" -elasticsearch-password "" -db-host "<postgres-host-name>" -db-port 5432 -db-name "<postgres-db-name>" -db-user "<postgres-db-user>" -db-password "<postgres-db-password> server migrate"
Once the migration has been done, Compass server can be started with this command.
docker run --net compass_storage -p 8080:8080 -e ELASTICSEARCH_BROKERS=http://es:9200 -e ELASTICSEARCH_USERNAME= -e ELASTICSEARCH_PASSWORD= -e DB_HOST=postgres -e DB_PORT=5432 -e DB_NAME=compass -e DB_USER=compass -e DB_PASSWORD=compass_password gotocompany/compass compass server start
If you are using Compass binary, you can run this command.
./compass -elasticsearch-brokers "http://<broker-host-name>" -elasticsearch-username "" -elasticsearch-password "" -db-host "<postgres-host-name>" -db-port 5432 -db-name "<postgres-db-name>" -db-user "<postgres-db-user>" -db-password "<postgres-db-password> server start"
Before running unit test, be warned that it will spin up docker container behind the scene. If the engine being used is Lima-based (like Colima), ensure that the DOCKER_HOST environment variable is set to avoid spin up failure.
Running all unit tests
make test
The tests combine both unit and integration tests, the test suite requires docker to run elasticsearch. In case you wish to test against an existing
elasticsearch cluster, set the value of ES_TEST_SERVER_URL to the URL of the elasticsearch server.
Development of Compass happens in the open on GitHub, and we are grateful to the community for contributing bugfixes and improvements. Read below to learn how you can take part in improving Compass.
Read our contributing guide to learn about our development process, how to propose bugfixes and improvements, and how to build and test your changes to Compass.
To help you get your feet wet and get you familiar with our contribution process, we have a list of good first issues that contain bugs which have a relatively limited scope. This is a great place to get started.
This project exists thanks to all the contributors.
Compass is Apache 2.0 licensed.