Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(EnPRO): extend OTel-shop with MongoDB, add Ansible, Terraform and EDMM models #9

Merged
merged 49 commits into from
Oct 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
4dd1845
[productcatalogservice] Added MongoDB
heidrifx Jun 22, 2024
85b53ec
[productcatalogservice] Changed to use env vars
heidrifx Jun 23, 2024
159e4d7
[K8s] Added MongoDB
heidrifx Jun 23, 2024
6b4247d
build(deps): bump docker/build-push-action from 6.0.1 to 6.1.0 (#1620)
dependabot[bot] Jun 26, 2024
a2e1b1e
fix: emailservice docker compose image (#1630)
rogercoll Jun 26, 2024
ca23a78
add collector configs as environment variable (#1632)
rogercoll Jun 26, 2024
51e485a
fix(frontend): Use port 443 as default to connect to flagd if https i…
SvenKirschbaum Jun 26, 2024
9c1153b
Update recommendation flag to match flagd configuration (#1634)
flands Jun 27, 2024
31f2e1d
Change AccountService from Go to DotNet (auto) (#1538)
RassK Jun 27, 2024
56571ba
build(deps): bump docker/build-push-action from 6.1.0 to 6.2.0 (#1636)
dependabot[bot] Jun 27, 2024
8bf3505
[chore] clarify complete release process (#1638)
puckpuck Jun 28, 2024
089804d
update to 1.11.0 release (#1639)
puckpuck Jun 28, 2024
020a14b
update to 1.11.0 release (#1640)
puckpuck Jun 28, 2024
5e3c66b
[chore] update demo role memberships (#1649)
puckpuck Jul 1, 2024
96da6ca
[otel-col] Add docker stats receiver (#1650)
julianocosta89 Jul 2, 2024
1ea7b68
add Kafka Dockerfile env variable (#1652)
rogercoll Jul 2, 2024
c4fb5ef
build(deps): bump docker/build-push-action from 6.2.0 to 6.3.0 (#1656)
dependabot[bot] Jul 3, 2024
c921c1e
Simplify error scenario logic (#1657)
beeme1mr Jul 8, 2024
c85f722
[tests] - optimize trace testing (#1659)
puckpuck Jul 8, 2024
4a923bc
[chore] fix build-images workflow (#1661)
puckpuck Jul 9, 2024
84a4826
Set OTLP receiver endpoint (#1662)
julianocosta89 Jul 12, 2024
169d02d
feat: [K8s] Added MongoDB
heidrifx Jun 23, 2024
998c55a
feat: ES-9 add EDMM Model (K8s) of otel-shop
eliasmueller Jul 18, 2024
53de8ec
feat: ES-39 add Terrfaform Model of otel-shop
eliasmueller Jul 18, 2024
6776ead
feat: ES-44 add Ansible Model of otel-shop
eliasmueller Jul 18, 2024
ad4fe4e
fixed MongoDB not working and removed duplicate MongoDB service
heidrifx Jul 19, 2024
1f63679
feat: ES-9 add some refinement and connects_to
eliasmueller Jul 22, 2024
c4baf66
Merge pull request #4 from UST-DeMAF/ES-44
eliasmueller Jul 28, 2024
f1c6ceb
Merge pull request #3 from UST-DeMAF/ES-39
eliasmueller Jul 28, 2024
f968106
Update otel_store_k8s_translated.yaml
Arikuma97 Jul 28, 2024
41eb5dd
Update otel_store_k8s_translated.yaml
Arikuma97 Jul 28, 2024
1e1feb8
Merge remote-tracking branch 'origin/demaf' into ES-47
eliasmueller Jul 29, 2024
3bfff7d
fix(ES-47): fix K8s mongo-db deployment
eliasmueller Jul 29, 2024
6d0c610
Merge remote-tracking branch 'origin/demaf' into ES-9-2
eliasmueller Jul 29, 2024
9fd3ada
Terraform model: Deletetd testing services and fixed image declaratio…
schurpl Jul 29, 2024
26e747e
Merge branch 'refs/heads/demaf' into ES-9-2
eliasmueller Jul 29, 2024
62fe516
Merge pull request #2 from UST-DeMAF/ES-9-2
eliasmueller Aug 6, 2024
64d6ed3
feat: ES-32 minor fix and todo
eliasmueller Aug 8, 2024
2062518
Adds missing component types and fixes relations
365Bit Aug 11, 2024
7b45b7c
fix: ES-56 fix some missing stuff in EDMM
eliasmueller Aug 12, 2024
9296af8
Added health checks where applicable
heidrifx Aug 16, 2024
9ee9c23
fix: auto-formatted K8s file
heidrifx Aug 25, 2024
28b7ea0
Merge pull request #7 from UST-DeMAF/ES-57
heidrifx Sep 3, 2024
c312ce5
chore(ES-80): move and cleanup main.tf (#8)
eliasmueller Sep 22, 2024
d012b1d
ES-59 Fixed main.tf and added linux support
schurpl Sep 24, 2024
7faf00f
Merge remote-tracking branch 'origin' into demaf
eliasmueller Sep 27, 2024
5fdbacd
fix: fix merge issues
eliasmueller Sep 27, 2024
1ae3eef
chore(ES-86): keep models up-to-date and add target models for all te…
eliasmueller Sep 30, 2024
30016fa
Merge branch 'refs/heads/main' into demaf
eliasmueller Sep 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ OPENSEARCH_IMAGE=opensearchproject/opensearch:2.16.0
POSTGRES_IMAGE=postgres:16.4
PROMETHEUS_IMAGE=quay.io/prometheus/prometheus:v2.54.1
VALKEY_IMAGE=valkey/valkey:8.0-alpine
MONGO_IMAGE=mongo:8.0.0-rc9
# must also update the version arg in ./test/tracetesting/Dockerfile
TRACETEST_IMAGE=kubeshop/tracetest:v1.5.2

Expand Down Expand Up @@ -138,6 +139,11 @@ KAFKA_SERVICE_DOCKERFILE=./src/kafka/Dockerfile
VALKEY_PORT=6379
VALKEY_ADDR=valkey-cart:${VALKEY_PORT}

# MongoDB
MONGO_PORT=27017
MONGO_USERNAME=mongo
MONGO_PASSWORD=mongo_product_catalog

# ********************
# Telemetry Components
# ********************
Expand Down
45 changes: 45 additions & 0 deletions .github/workflows/buildAndPushImage-productcatalogservice.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: buildAndPushImage

on:
push:
branches:
- main

env:
IMAGE_URL: ghcr.io/ust-demaf
IMAGE_NAME: productcatalogservice
IMAGE_TAG: 1.11.1

jobs:
build-using-dockerfile-push-2-ghcr:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: linux/amd64, linux/arm64

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
platforms: linux/amd64, linux/arm64

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}

# Build and push the image
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: ./src/productcatalogservice/Dockerfile
platforms: linux/amd64, linux/arm64
push: true
tags: ${{ env.IMAGE_URL }}/demo:${{ env.IMAGE_TAG }}-${{ env.IMAGE_NAME }}
provenance: false
184 changes: 54 additions & 130 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,140 +1,64 @@
<!-- markdownlint-disable-next-line -->
# <img src="https://opentelemetry.io/img/logos/opentelemetry-logo-nav.png" alt="OTel logo" width="45"> OpenTelemetry Demo

[![Slack](https://img.shields.io/badge/slack-@cncf/otel/demo-brightgreen.svg?logo=slack)](https://cloud-native.slack.com/archives/C03B4CWV4DA)
[![Version](https://img.shields.io/github/v/release/open-telemetry/opentelemetry-demo?color=blueviolet)](https://github.com/open-telemetry/opentelemetry-demo/releases)
[![Commits](https://img.shields.io/github/commits-since/open-telemetry/opentelemetry-demo/latest?color=ff69b4&include_prereleases)](https://github.com/open-telemetry/opentelemetry-demo/graphs/commit-activity)
[![Downloads](https://img.shields.io/docker/pulls/otel/demo)](https://hub.docker.com/r/otel/demo)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg?color=red)](https://github.com/open-telemetry/opentelemetry-demo/blob/main/LICENSE)
[![Integration Tests](https://github.com/open-telemetry/opentelemetry-demo/actions/workflows/run-integration-tests.yml/badge.svg)](https://github.com/open-telemetry/opentelemetry-demo/actions/workflows/run-integration-tests.yml)
[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/opentelemetry-demo)](https://artifacthub.io/packages/helm/opentelemetry-helm/opentelemetry-demo)
[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/9247/badge)](https://www.bestpractices.dev/en/projects/9247)

## Welcome to the OpenTelemetry Astronomy Shop Demo
## Welcome to the OpenTelemetry Astronomy Shop Demo with DeMAF extension

This repository contains the OpenTelemetry Astronomy Shop, a microservice-based
distributed system intended to illustrate the implementation of OpenTelemetry in
a near real-world environment.

Our goals are threefold:
FOR THE ORIGINAL README [CLICK HERE](./README_original.md).

The purpose of this fork is to extend the Astronomy Shop in three ways:

- Add a database to showcase that a persistence layer can also be part of the architecture without causing any problems.
- Create a deployment model to deploy the shop using [Ansible](https://docs.ansible.com/) and [Terraform](https://developer.hashicorp.com/terraform/docs) besides the given [Docker-Compose](https://opentelemetry.io/docs/demo/docker_deployment/) and [Kubernetes](https://opentelemetry.io/docs/demo/kubernetes_deployment/).
- Add translated EDMM models for [Kubernetes](./edmm/otel-store_k8s_translated.yaml), [Ansible](./edmm/otel-store_ansible_translated.yaml) and [Terraform](./edmm/otel-store_terraform_translated.yaml) to showcase the transformation result of those three DeMAF plugins in [EDMM](https://github.com/UST-EDMM).


### Database Integration
The database integration ensures that also a persistence layer is used in the provided demo-application,
making the architecture more comprehensive.

Therefore, a [MongoDB](http://mongodb.com) is added, which adds the products of the shop into the database as JSON entries.
In the default variant of the astronomy shop, only a plain file is used to provide the products.

Thus, the logic of the `productcatalogservice` is adapted, to initialize and use the additionally provided MongoDB.

The changes of the `productcatalogservice` and the addition of the MongoDB is integrated into all deployment variants in this fork.

### Translation to Ansible

To verify the [Ansible MPS Plugin](https://github.com/UST-DeMAF/ansible-mps-plugin) the Astronomy Shop is available as Ansible installation.
It is based on the [Docker-Compose](https://opentelemetry.io/docs/demo/docker_deployment/) and installs all components of the shop as docker containers on the local Docker runtime.
For each component there is a dedicated Ansible Role representing the loose coupling of the components.

A usage guide can be found in the dedicated [README in /ansible](./ansible/README.md).

### Translation to Terraform

To verify the [Terraform MPS Plugin](https://github.com/UST-DeMAF/terraform-mps-plugin) the Astronomy Shop is available as Terraform installation.
It is based on the [Docker-Compose](https://opentelemetry.io/docs/demo/docker_deployment/) and installs all components of the shop as docker containers on the local Docker runtime.

A usage guide can be found in the dedicated [README in /terraform](./terraform/README.md).

### EDMM Models

For all three evaluated technologies (Ansible, Terraform, Kubernetes) there is a transformed target model available.

The target EDMM models can be found in the [/edmm directory](./edmm)

In each model the goal is to retrieve a component for each deployed Astronomy Shop component
and as many _connects-to_ and _hosted_on_ relations as possible.

This in common, there are also differences in the target models.

Due to the imperative character of Ansible, in the [Ansible target model](./edmm/otel-store_ansible_translated.yaml), each component has operations representing the installation of the Docker containers.

- Provide a realistic example of a distributed system that can be used to
demonstrate OpenTelemetry instrumentation and observability.
- Build a base for vendors, tooling authors, and others to extend and
demonstrate their OpenTelemetry integrations.
- Create a living example for OpenTelemetry contributors to use for testing new
versions of the API, SDK, and other components or enhancements.
In the [Terraform target model](./edmm/otel-store_terraform_translated.yaml), there are common component types for all _docker_containers_,
as they all use the same terraform ressource type. This is different in the other two evaluated technologies, where each component has a dedicated type.

We've already made [huge
progress](https://github.com/open-telemetry/opentelemetry-demo/blob/main/CHANGELOG.md),
and development is ongoing. We hope to represent the full feature set of
OpenTelemetry across its languages in the future.

If you'd like to help (**which we would love**), check out our [contributing
guidance](./CONTRIBUTING.md).

If you'd like to extend this demo or maintain a fork of it, read our
[fork guidance](https://opentelemetry.io/docs/demo/forking/).

## Quick start

You can be up and running with the demo in a few minutes. Check out the docs for
your preferred deployment method:

- [Docker](https://opentelemetry.io/docs/demo/docker_deployment/)
- [Kubernetes](https://opentelemetry.io/docs/demo/kubernetes_deployment/)

## Documentation

For detailed documentation, see [Demo Documentation][docs]. If you're curious
about a specific feature, the [docs landing page][docs] can point you in the
right direction.

## Demos featuring the Astronomy Shop

We welcome any vendor to fork the project to demonstrate their services and
adding a link below. The community is committed to maintaining the project and
keeping it up to date for you.

| | | |
|-----------------------------------------|-----------------------------|----------------------------------------------------------------|
| [AlibabaCloud LogService][AlibabaCloud] | [Elastic][Elastic] | [New Relic][NewRelic] |
| [AppDynamics][AppDynamics] | [Google Cloud][GoogleCloud] | [OpenSearch][OpenSearch] |
| [Aspecto][Aspecto] | [Grafana Labs][GrafanaLabs] | [Sentry][Sentry] |
| [Axiom][Axiom] | [Guance][Guance] | [ServiceNow Cloud Observability][ServiceNowCloudObservability] |
| [Axoflow][Axoflow] | [Helios][Helios] | [Splunk][Splunk] |
| [Azure Data Explorer][Azure] | [Honeycomb.io][Honeycombio] | [Sumo Logic][SumoLogic] |
| [Coralogix][Coralogix] | [Instana][Instana] | [TelemetryHub][TelemetryHub] |
| [Dash0][Dash0] | [Kloudfuse][Kloudfuse] | [Teletrace][Teletrace] |
| [Datadog][Datadog] | [Liatrio][Liatrio] | [Tracetest][Tracetest] |
| [Dynatrace][Dynatrace] | [Logz.io][Logzio] | [Uptrace][Uptrace] |

## Contributing

To get involved with the project see our [CONTRIBUTING](CONTRIBUTING.md)
documentation. Our [SIG Calls](CONTRIBUTING.md#join-a-sig-call) are every other
Monday at 8:30 AM PST and anyone is welcome.

## Project leadership

[Maintainers](https://github.com/open-telemetry/community/blob/main/guides/contributor/membership.md#maintainer)
([@open-telemetry/demo-maintainers](https://github.com/orgs/open-telemetry/teams/demo-maintainers)):

- [Juliano Costa](https://github.com/julianocosta89), Datadog
- [Mikko Viitanen](https://github.com/mviitane), Dynatrace
- [Pierre Tessier](https://github.com/puckpuck), Honeycomb

[Approvers](https://github.com/open-telemetry/community/blob/main/guides/contributor/membership.md#approver)
([@open-telemetry/demo-approvers](https://github.com/orgs/open-telemetry/teams/demo-approvers)):

- [Cedric Ziel](https://github.com/cedricziel) Grafana Labs
- [Penghan Wang](https://github.com/wph95), AppDynamics
- [Reiley Yang](https://github.com/reyang), Microsoft
- [Roger Coll](https://github.com/rogercoll), Elastic
- [Ziqi Zhao](https://github.com/fatsheep9146), Alibaba

Emeritus:

- [Austin Parker](https://github.com/austinlparker)
- [Carter Socha](https://github.com/cartersocha)
- [Michael Maxwell](https://github.com/mic-max)
- [Morgan McLean](https://github.com/mtwo)

### Thanks to all the people who have contributed

[![contributors](https://contributors-img.web.app/image?repo=open-telemetry/opentelemetry-demo)](https://github.com/open-telemetry/opentelemetry-demo/graphs/contributors)

[docs]: https://opentelemetry.io/docs/demo/

<!-- Links for Demos featuring the Astronomy Shop section -->

[AlibabaCloud]: https://github.com/aliyun-sls/opentelemetry-demo
[AppDynamics]: https://www.appdynamics.com/blog/cloud/how-to-observe-opentelemetry-demo-app-in-appdynamics-cloud/
[Aspecto]: https://github.com/aspecto-io/opentelemetry-demo
[Axiom]: https://play.axiom.co/axiom-play-qf1k/dashboards/otel.traces.otel-demo-traces
[Axoflow]: https://axoflow.com/opentelemetry-support-in-more-detail-in-axosyslog-and-syslog-ng/
[Azure]: https://github.com/Azure/Azure-kusto-opentelemetry-demo
[Coralogix]: https://coralogix.com/blog/configure-otel-demo-send-telemetry-data-coralogix
[Dash0]: https://github.com/dash0hq/opentelemetry-demo
[Datadog]: https://docs.datadoghq.com/opentelemetry/guide/otel_demo_to_datadog
[Dynatrace]: https://www.dynatrace.com/news/blog/opentelemetry-demo-application-with-dynatrace/
[Elastic]: https://github.com/elastic/opentelemetry-demo
[GoogleCloud]: https://github.com/GoogleCloudPlatform/opentelemetry-demo
[GrafanaLabs]: https://github.com/grafana/opentelemetry-demo
[Guance]: https://github.com/GuanceCloud/opentelemetry-demo
[Helios]: https://otelsandbox.gethelios.dev
[Honeycombio]: https://github.com/honeycombio/opentelemetry-demo
[Instana]: https://github.com/instana/opentelemetry-demo
[Kloudfuse]: https://github.com/kloudfuse/opentelemetry-demo
[Liatrio]: https://github.com/liatrio/opentelemetry-demo
[Logzio]: https://logz.io/learn/how-to-run-opentelemetry-demo-with-logz-io/
[NewRelic]: https://github.com/newrelic/opentelemetry-demo
[OpenSearch]: https://github.com/opensearch-project/opentelemetry-demo
[Sentry]: https://github.com/getsentry/opentelemetry-demo
[ServiceNowCloudObservability]: https://docs.lightstep.com/otel/quick-start-operator#send-data-from-the-opentelemetry-demo
[Splunk]: https://github.com/signalfx/opentelemetry-demo
[SumoLogic]: https://www.sumologic.com/blog/common-opentelemetry-demo-application/
[TelemetryHub]: https://github.com/TelemetryHub/opentelemetry-demo/tree/telemetryhub-backend
[Teletrace]: https://github.com/teletrace/opentelemetry-demo
[Tracetest]: https://github.com/kubeshop/opentelemetry-demo
[Uptrace]: https://github.com/uptrace/uptrace/tree/master/example/opentelemetry-demo
For the Kubernetes model there are two variants. The [first](./edmm/otel-store_k8s_translated.yaml) is the actual result of the translation, lacking _hosted_on_ relations.
That is, as the Kubernetes plugin only creates _hosted_on_ relations when a component named _container_runtime_ with a matching type is present in the TADM.
Therefore, there is a [second target model](./edmm/otel-store_k8s_translated_with_container_runtime.yaml) where this _container_runtime_ has been manually added, before creating the relations.
Thus, all components obtain a _hosted_on_ relation.
Loading
Loading