Skip to content

Commit

Permalink
feature/APPEALS-50882 - Open Telemetry configuration and implementati…
Browse files Browse the repository at this point in the history 
…on (#22833)

* feature/appeals-45972 (#21950)

* Remove UUID from attrs sent_to metrics. (#21630)

* Remove UUID from attrs sent_to metrics.

* pushing with some linting issues for visibility.

* Change config implementation.

* Remove UUID from spec

* remove uuid from update_appellant job

* diable rubocop for open_telemetry init

* OTEL fixes

* rescue, errors

---------

Co-authored-by: mikefinneran <110622959+mikefinneran@users.noreply.github.com>

* add otel reqs

* Remove throw

* comment out otel file loop

* try vendor suggested config

* update otel config to match vendor

* update pg gem

* otel include dt_host file

* otel require bundler and rubygems

* otel subset of instrumentation

* Revert "update pg gem"

This reverts commit fc1a45d.

* update redis and minimize otel instrumentation to just rails, rack, and activerecord

* otel add use_all except pg and redis

* otel require instruments

* otel fix typo

* otel fix typo

* otel comment out net_http

* otel silence aws sdk internals

* otel silence aws sdk internals

* otel add net http

* Individually use OTEL instruments (#22082)

* Individually use OTEL instruments
with options.

* disable AwsSdk only. Checking Rack options.

* re-require awssdk even while disabled.

* disable awssdk

* disable datadog for testing

* change sequence factory to properly seed

* updated opentelemetry and datadog configs

* rack context getter initalizer

* use one at a time

* add curly braces

* Revert change

* revert change

* Revert change

* Revert change

* Remove gemfile grouping

* Remove datadog.

* ActionPack and Actionview changes

* APPEALS-44287: Excluding disposition held and select that appeal for distribution  (#22277)

* first run at an SQL query removing duplicate appeals from distribution

* code refactor and excluding disposition held and select that appeal for distribution

* automated test for the duplicate hearing bug

* fix rubocop offense SpaceInsideBlockBraces

---------

Co-authored-by: Sean Parker <Sean.Parker3@va.gov>
Co-authored-by: samasudhirreddy <sudhirreddy.sama@va.gov>

---------

Co-authored-by: Noelle Adkin <98478937+NoelleAd@users.noreply.github.com>
Co-authored-by: Raymond Hughes <131811099+raymond-hughes@users.noreply.github.com>
Co-authored-by: Dani <danielle.coleman1@va.gov>
Co-authored-by: raymond-hughes <raymond.hughes2@va.gov>
Co-authored-by: Craig Reese <109101548+craigrva@users.noreply.github.com>
Co-authored-by: Sean Parker <Sean.Parker3@va.gov>
Co-authored-by: samasudhirreddy <sudhirreddy.sama@va.gov>

* Update bundler to 2.4.22

* Updating OTEL to use all and remove new relic gem

* Remove calls to new relic

* removed id attrs in metrics service calls that were causing dimension errors

* Update config for otel

* Reorder gems to fix linting issues

* Updating config and refactoring verbose gems to add all

* Reinstall gems

* Adding all instruments

* Update config to only include basic instruments

* Update Gemfile.lock

* Adding rake to gemfile

* Adding instruments

* Comment out PG and ActiveSupport

* Suppress AWS logs

* Remove redis and turn on actionview

* Turn actionview off

* Add Redis instrumentation back

* Turn ActionPack back on

* Disable Redis

* Remove mentions of Datadog

* Removed extra mentions of Datadog

* removed newrelic references and yml file

* Test updating workflow

* Revert workflow change

* Adding simplecov back

* Fixing linting error

* Removing by: attribute after removing keyword in metric service

* Adding simplecov lcov gem and updating workflow

* Update workflow

* Updating simplecov

* Revert simple cov

* Adding featureenvy skip for reek

* Update service name. (#22762)

---------

Co-authored-by: mikefinneran <110622959+mikefinneran@users.noreply.github.com>
Co-authored-by: Noelle Adkin <98478937+NoelleAd@users.noreply.github.com>
Co-authored-by: Dani <danielle.coleman1@va.gov>
Co-authored-by: Craig Reese <109101548+craigrva@users.noreply.github.com>
Co-authored-by: Sean Parker <Sean.Parker3@va.gov>
Co-authored-by: samasudhirreddy <sudhirreddy.sama@va.gov>
Co-authored-by: Andrew Hadley <hadley_andrew@bah.com>
Co-authored-by: alex-guanipatin <alex.guanipatin@va.gov>
Co-authored-by: Drew Hadley <50673809+VandelayUtd@users.noreply.github.com>
Co-authored-by: Ron Wabukenda <130374706+ronwabVa@users.noreply.github.com>
  • Loading branch information
@raymond-hughes @mikefinneran
@NoelleAd @dcoleman21 @craigrva @seanrpa @samasudhirreddy @alex-guanipatin @drew-hadley @ronwabVa @AdamShawBAH
11 people authored and AdamShawBAH committed Sep 20, 2024
1 parent c2c2279 commit b5b473f
Show file tree
Hide file tree
Showing 29 changed files with 265 additions and 192 deletions.
4 changes: 0 additions & 4 deletions Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,6 @@ RUN apt install -y ${CASEFLOW} && \
# install jemalloc
RUN apt install -y --no-install-recommends libjemalloc-dev


# install datadog agent
RUN DD_INSTALL_ONLY=true DD_AGENT_MAJOR_VERSION=7 DD_API_KEY=$(cat config/datadog.key) bash -c "$(curl -L https://raw.githubusercontent.com/DataDog/datadog-agent/master/cmd/agent/install_script.sh)"

RUN rm -rf /var/lib/apt/lists/*

# Installing the version of bundler that corresponds to the Gemfile.lock
Expand Down
29 changes: 24 additions & 5 deletions Gemfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,6 @@ gem "connect_mpi", git: "https://github.com/department-of-veterans-affairs/conne
gem "connect_vbms", git: "https://github.com/department-of-veterans-affairs/connect_vbms.git", ref: "9807d9c9f0f3e3494a60b6693dc4f455c1e3e922"
gem "console_tree_renderer", git: "https://github.com/department-of-veterans-affairs/console-tree-renderer.git", tag: "v0.1.1"
gem "countries"
gem "ddtrace"
gem "dogstatsd-ruby"
gem "dry-schema", "~> 1.4"
gem "fast_jsonapi"
gem "fuzzy_match"
Expand All @@ -33,8 +31,28 @@ gem "icalendar"
gem "kaminari"
gem "logstasher"
gem "moment_timezone-rails"
gem "newrelic_rpm"
gem "nokogiri", ">= 1.11.0.rc4"

gem "opentelemetry-exporter-otlp", require: false
gem "opentelemetry-sdk", require: false

gem "opentelemetry-instrumentation-action_pack", require: false
gem "opentelemetry-instrumentation-action_view", require: false
gem "opentelemetry-instrumentation-active_job", require: false
gem "opentelemetry-instrumentation-active_model_serializers", require: false
gem "opentelemetry-instrumentation-active_record", require: false
gem "opentelemetry-instrumentation-aws_sdk", require: false
gem "opentelemetry-instrumentation-concurrent_ruby", require: false
gem "opentelemetry-instrumentation-faraday", require: false
gem "opentelemetry-instrumentation-http", require: false
gem "opentelemetry-instrumentation-http_client", require: false
gem "opentelemetry-instrumentation-net_http", require: false
gem "opentelemetry-instrumentation-pg", require: false
gem "opentelemetry-instrumentation-rack", require: false
gem "opentelemetry-instrumentation-rails", require: false
gem "opentelemetry-instrumentation-rake", require: false
gem "opentelemetry-instrumentation-redis", require: false

gem "paper_trail", "~> 12.0"
# Used to speed up reporting
gem "parallel"
Expand All @@ -59,7 +77,7 @@ gem "rainbow"
# React
gem "react_on_rails", "11.3.0"
gem "redis-mutex"
gem "redis-namespace"
gem "redis-namespace", "~> 1.11.0"
gem "redis-rails", "~> 5.0.2"
gem "request_store"
gem "roo", "~> 2.7"
Expand Down Expand Up @@ -119,7 +137,8 @@ group :test, :development, :demo, :make_docs do
gem "rubocop-performance"
gem "rubocop-rails"
gem "scss_lint", require: false
gem "simplecov", git: "https://github.com/colszowka/simplecov.git", require: false
gem "simplecov", require: false
gem "simplecov-lcov", require: false
gem "single_cov"
gem "sniffybara", git: "https://github.com/department-of-veterans-affairs/sniffybara.git"
gem "sql_tracker"
Expand Down
159 changes: 128 additions & 31 deletions Gemfile.lock
View file
Original file line number Diff line number Diff line change
@@ -1,12 +1,3 @@
GIT
remote: https://github.com/colszowka/simplecov.git
revision: 783c9d7e9995f3ea9baf9fbb517c1d0ceb12acdb
specs:
simplecov (0.15.1)
docile (~> 1.1.0)
json (>= 1.8, < 3)
simplecov-html (~> 0.10.0)

GIT
remote: https://github.com/department-of-veterans-affairs/caseflow-commons
revision: 9bd3635fbd8094d25160669f38d8699e2f1d7a98
Expand Down Expand Up @@ -1458,7 +1449,7 @@ GEM
backport (1.2.0)
base64 (0.2.0)
benchmark (0.3.0)
benchmark-ips (2.7.2)
benchmark-ips (2.14.0)
bootsnap (1.7.5)
msgpack (~> 1.0)
brakeman (4.7.1)
Expand Down Expand Up @@ -1529,8 +1520,6 @@ GEM
database_cleaner-core (~> 2.0.0)
database_cleaner-core (2.0.1)
date (3.3.3)
ddtrace (0.34.1)
msgpack
dead_end (4.0.0)
debase (0.2.4.1)
debase-ruby_core_source (>= 0.10.2)
Expand All @@ -1548,8 +1537,7 @@ GEM
ruby-statistics (>= 2.1)
thor (>= 0.19, < 2)
diff-lcs (1.3)
docile (1.1.5)
dogstatsd-ruby (4.4.0)
docile (1.4.1)
dotenv (2.8.1)
dotenv-rails (2.8.1)
dotenv (= 2.8.1)
Expand Down Expand Up @@ -1614,13 +1602,16 @@ GEM
foreman (0.87.2)
formatador (0.2.5)
fuzzy_match (2.1.0)
get_process_mem (0.2.4)
get_process_mem (0.2.7)
ffi (~> 1.0)
git (1.13.2)
addressable (~> 2.8)
rchardet (~> 1.8)
globalid (1.2.1)
activesupport (>= 6.1)
google-protobuf (3.25.4)
googleapis-common-protos-types (1.15.0)
google-protobuf (>= 3.18, < 5.a)
govdelivery-tms (2.8.4)
activesupport
faraday
Expand All @@ -1644,7 +1635,8 @@ GEM
builder (>= 2.1.2)
hana (1.3.6)
hashdiff (1.0.0)
heapy (0.1.4)
heapy (0.2.0)
thor
holidays (6.6.1)
httpclient (2.8.3)
httpi (2.4.4)
Expand All @@ -1664,7 +1656,6 @@ GEM
rails-dom-testing (>= 1, < 3)
railties (>= 4.2.0)
thor (>= 0.14, < 2.0)
json (2.3.0)
json-schema (4.3.0)
addressable (>= 2.8)
json_schemer (0.2.16)
Expand Down Expand Up @@ -1709,7 +1700,7 @@ GEM
net-pop
net-smtp
marcel (1.0.4)
memory_profiler (0.9.14)
memory_profiler (1.0.2)
meta_request (0.7.2)
rack-contrib (>= 1.1, < 3)
railties (>= 3.0.0, < 7)
Expand All @@ -1719,7 +1710,6 @@ GEM
mime-types-data (3.2019.1009)
mini_histogram (0.3.1)
mini_mime (1.1.2)
mini_portile2 (2.8.5)
minitest (5.19.0)
moment_timezone-rails (0.5.0)
momentjs-rails (2.29.4.1)
Expand All @@ -1737,11 +1727,9 @@ GEM
timeout
net-smtp (0.3.3)
net-protocol
newrelic_rpm (6.5.0.357)
nio4r (2.5.9)
no_proxy_fix (0.1.2)
nokogiri (1.15.5)
mini_portile2 (~> 2.8.2)
nokogiri (1.15.5-x86_64-darwin)
racc (~> 1.4)
nori (2.6.0)
notiffany (0.1.1)
Expand All @@ -1751,6 +1739,92 @@ GEM
faraday (>= 0.9)
sawyer (~> 0.8.0, >= 0.5.3)
open4 (1.3.4)
opentelemetry-api (1.1.0)
opentelemetry-common (0.19.7)
opentelemetry-api (~> 1.0)
opentelemetry-exporter-otlp (0.24.2)
google-protobuf (~> 3.19)
googleapis-common-protos-types (~> 1.3)
opentelemetry-api (~> 1.1)
opentelemetry-common (~> 0.19.6)
opentelemetry-sdk (~> 1.2)
opentelemetry-semantic_conventions
opentelemetry-instrumentation-action_pack (0.5.0)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-rack (~> 0.21)
opentelemetry-instrumentation-action_view (0.4.0)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-active_support (~> 0.1)
opentelemetry-instrumentation-base (~> 0.20)
opentelemetry-instrumentation-active_job (0.4.0)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-active_model_serializers (0.19.1)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-active_record (0.5.0)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-base (~> 0.21.0)
ruby2_keywords
opentelemetry-instrumentation-active_support (0.3.0)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-aws_sdk (0.3.2)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-base (0.21.1)
opentelemetry-api (~> 1.0)
opentelemetry-registry (~> 0.1)
opentelemetry-instrumentation-concurrent_ruby (0.20.1)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-faraday (0.22.0)
opentelemetry-api (~> 1.0)
opentelemetry-common (~> 0.19.3)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-http (0.21.0)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-http_client (0.21.0)
opentelemetry-api (~> 1.0)
opentelemetry-common (~> 0.19.3)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-net_http (0.21.1)
opentelemetry-api (~> 1.0)
opentelemetry-common (~> 0.19.3)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-pg (0.23.0)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-rack (0.22.1)
opentelemetry-api (~> 1.0)
opentelemetry-common (~> 0.19.3)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-rails (0.25.0)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-action_pack (~> 0.5.0)
opentelemetry-instrumentation-action_view (~> 0.4.0)
opentelemetry-instrumentation-active_job (~> 0.4.0)
opentelemetry-instrumentation-active_record (~> 0.5.0)
opentelemetry-instrumentation-active_support (~> 0.3.0)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-rake (0.1.1)
opentelemetry-api (~> 1.0)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-instrumentation-redis (0.24.1)
opentelemetry-api (~> 1.0)
opentelemetry-common (~> 0.19.3)
opentelemetry-instrumentation-base (~> 0.21.0)
opentelemetry-registry (0.2.0)
opentelemetry-api (~> 1.1)
opentelemetry-sdk (1.2.1)
opentelemetry-api (~> 1.1)
opentelemetry-common (~> 0.19.3)
opentelemetry-registry (~> 0.2)
opentelemetry-semantic_conventions
opentelemetry-semantic_conventions (1.10.0)
opentelemetry-api (~> 1.0)
paper_trail (12.3.0)
activerecord (>= 5.2)
request_store (~> 1.1)
Expand Down Expand Up @@ -1840,8 +1914,8 @@ GEM
redis-namespace (~> 1.0)
redis-mutex (4.0.2)
redis-classy (~> 2.0)
redis-namespace (1.6.0)
redis (>= 3.0.4)
redis-namespace (1.11.0)
redis (>= 4)
redis-rack (2.0.4)
rack (>= 1.5, < 3)
redis-store (>= 1.2, < 2)
Expand Down Expand Up @@ -1919,6 +1993,7 @@ GEM
ruby-prof (1.4.1)
ruby-progressbar (1.13.0)
ruby-statistics (3.0.2)
ruby2_keywords (0.0.5)
ruby_dep (1.5.0)
ruby_parser (3.20.3)
sexp_processor (~> 4.16)
Expand Down Expand Up @@ -1963,7 +2038,13 @@ GEM
thor
shoulda-matchers (5.3.0)
activesupport (>= 5.2.0)
simplecov-html (0.10.2)
simplecov (0.22.0)
docile (~> 1.1)
simplecov-html (~> 0.11)
simplecov_json_formatter (~> 0.1)
simplecov-html (0.13.1)
simplecov-lcov (0.8.0)
simplecov_json_formatter (0.1.4)
single_cov (1.3.2)
sixarm_ruby_unaccent (1.2.0)
socksify (1.7.1)
Expand Down Expand Up @@ -2045,7 +2126,7 @@ GEM
ziptz (2.1.6)

PLATFORMS
ruby
x86_64-darwin-22

DEPENDENCIES
aasm (= 4.11.0)
Expand All @@ -2072,10 +2153,8 @@ DEPENDENCIES
countries
danger (~> 6.2.2)
database_cleaner-active_record (= 2.0.0)
ddtrace
debase
derailed_benchmarks
dogstatsd-ruby
dotenv-rails
dry-schema (~> 1.4)
factory_bot_rails (~> 5.2)
Expand All @@ -2095,8 +2174,25 @@ DEPENDENCIES
logstasher
meta_request
moment_timezone-rails
newrelic_rpm
nokogiri (>= 1.11.0.rc4)
opentelemetry-exporter-otlp
opentelemetry-instrumentation-action_pack
opentelemetry-instrumentation-action_view
opentelemetry-instrumentation-active_job
opentelemetry-instrumentation-active_model_serializers
opentelemetry-instrumentation-active_record
opentelemetry-instrumentation-aws_sdk
opentelemetry-instrumentation-concurrent_ruby
opentelemetry-instrumentation-faraday
opentelemetry-instrumentation-http
opentelemetry-instrumentation-http_client
opentelemetry-instrumentation-net_http
opentelemetry-instrumentation-pg
opentelemetry-instrumentation-rack
opentelemetry-instrumentation-rails
opentelemetry-instrumentation-rake
opentelemetry-instrumentation-redis
opentelemetry-sdk
paper_trail (~> 12.0)
parallel
paranoia (~> 2.2)
Expand All @@ -2115,7 +2211,7 @@ DEPENDENCIES
rb-readline
react_on_rails (= 11.3.0)
redis-mutex
redis-namespace
redis-namespace (~> 1.11.0)
redis-rails (~> 5.0.2)
request_store
roo (~> 2.7)
Expand All @@ -2139,7 +2235,8 @@ DEPENDENCIES
sentry-raven
shoryuken (= 3.1.11)
shoulda-matchers
simplecov!
simplecov
simplecov-lcov
single_cov
sniffybara!
solargraph
Expand Down
7 changes: 0 additions & 7 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -137,13 +137,6 @@ See debugging steps as well as more information about FACOLS in our [wiki](https
Review the [FACOLS documentation](docs/FACOLS.md) for details.

## Monitoring #######################################################
We use NewRelic to monitor the app. By default, it's disabled locally. To enable it, do:

```
NEW_RELIC_LICENSE_KEY='<key as displayed on NewRelic.com>' NEW_RELIC_AGENT_ENABLED=true bundle exec foreman start
```

You may wish to do this if you are debugging our NewRelic integration, for instance.

---

Expand Down
2 changes: 0 additions & 2 deletions app/controllers/api/application_controller.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@
class Api::ApplicationController < ActionController::Base
protect_from_forgery with: :null_session

include TrackRequestId

before_action :strict_transport_security

before_action :setup_fakes,
Expand Down
Loading

0 comments on commit b5b473f

Please sign in to comment.