From 96d54b1a90bd3adbd8913f4ea206d4948be3cd4b Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Wed, 27 Mar 2024 16:34:05 -0400 Subject: [PATCH 01/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20defaults=20for=20?= =?UTF-8?q?`config.action=5Fdispatch.use=5Fcookies=5Fwith=5Fmetadata`=20an?= =?UTF-8?q?d=20`config.action=5Fmailer.delivery=5Fjob`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The following config settings are not backwards compatible: - config.action_dispatch.use_cookies_with_metadata - config.action_mailer.delivery_job Now that Rails 6.0 is stable on production, we can assume their default values going forward. --- config/application.rb | 20 +------------------- 1 file changed, 1 insertion(+), 19 deletions(-) diff --git a/config/application.rb b/config/application.rb index 37e00da3554..b6643e43232 100644 --- a/config/application.rb +++ b/config/application.rb @@ -12,7 +12,7 @@ module CaseflowCertification class Application < Rails::Application # Initialize configuration defaults for originally generated Rails version. - config.load_defaults 5.2 + config.load_defaults 6.0 config.autoloader = :classic # Settings in config/environments/* take precedence over those specified here. @@ -96,28 +96,10 @@ class Application < Rails::Application # Default as of 6.0: false config.action_view.default_enforce_utf8 = true - # Embed purpose and expiry metadata inside signed and encrypted - # cookies for increased security. - # - # This option is not backwards compatible with earlier Rails versions. - # It's best enabled when your entire app is migrated and stable on 6.0. - # Default as of 6.0: true - config.action_dispatch.use_cookies_with_metadata = false - # Change the return value of `ActionDispatch::Response#content_type` to Content-Type header without modification. # Default as of 6.0: false config.action_dispatch.return_only_media_type_on_content_type = true - # Use ActionMailer::MailDeliveryJob for sending parameterized and normal mail. - # - # The default delivery jobs (ActionMailer::Parameterized::DeliveryJob, ActionMailer::DeliveryJob), - # will be removed in Rails 6.1. This setting is not backwards compatible with earlier Rails versions. - # If you send mail in the background, job workers need to have a copy of - # MailDeliveryJob to ensure all delivery jobs are processed properly. - # Make sure your entire app is migrated and stable on 6.0 before using this setting. - # Default as of 6.0: "ActionMailer::MailDeliveryJob" - config.action_mailer.delivery_job = nil - # Enable the same cache key to be reused when the object being cached of type # `ActiveRecord::Relation` changes by moving the volatile information (max updated at and count) # of the relation's cache key into the cache version to support recycling cache key. From fc5e7b003d50ded656f8d3a616c16a4d268e15dd Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Wed, 27 Mar 2024 18:43:45 -0400 Subject: [PATCH 02/69] =?UTF-8?q?=E2=9C=85=20Fix=20flakey=20spec?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- spec/feature/non_comp/report_page_spec.rb | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/spec/feature/non_comp/report_page_spec.rb b/spec/feature/non_comp/report_page_spec.rb index 6cf2c458097..b714a8d328e 100644 --- a/spec/feature/non_comp/report_page_spec.rb +++ b/spec/feature/non_comp/report_page_spec.rb @@ -331,7 +331,7 @@ def clear_filters end def change_history_csv_file - wait_for(5) + sleep 5 # Copied from Capybara setup download_directory = Rails.root.join("tmp/downloads_#{ENV['TEST_SUBCATEGORY'] || 'all'}") list_of_files = Dir.glob(File.join(download_directory, "*")).select { |f| File.file?(f) } @@ -340,11 +340,4 @@ def change_history_csv_file expect(latest_file).to_not eq(nil) latest_file end - - def wait_for(seconds) - start_time = Time.zone.now - while Time.zone.now - start_time < seconds - # Do nothing, just wait - end - end end From a1eaefdd1c7a2d10397381b0b7f5ac32645222ab Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 28 Mar 2024 16:20:25 -0400 Subject: [PATCH 03/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.action=5Fdispatch.use=5Fauthenticated=5Fcookie=5Fencrypt?= =?UTF-8?q?ion`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Since we are making other cookie configuration changes in this PR for Rails 6.0, this is an opportune time to migrate this Rails 5.2 cookie setting to its default value as well. --- config/application.rb | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/config/application.rb b/config/application.rb index b6643e43232..c9e1d5a9b19 100644 --- a/config/application.rb +++ b/config/application.rb @@ -65,16 +65,6 @@ class Application < Rails::Application # Default as of 5.2: true config.active_record.cache_versioning = false - # Use AES-256-GCM authenticated encryption for encrypted cookies. - # Also, embed cookie expiry in signed or encrypted cookies for increased security. - # - # This option is not backwards compatible with earlier Rails versions. - # It's best enabled when your entire app is migrated and stable on 5.2. - # - # Existing cookies will be converted on read then written with the new scheme. - # Default as of 5.2: true - config.action_dispatch.use_authenticated_cookie_encryption = false - # # Use AES-256-GCM authenticated encryption as default cipher for encrypting messages # instead of AES-256-CBC, when use_authenticated_message_encryption is set to true. # Default as of 5.2: true From c662d259c15ca0b37a8a85d6a9d4ab060e3d45ed Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 4 Apr 2024 14:23:09 -0400 Subject: [PATCH 04/69] =?UTF-8?q?=E2=9C=A8=20Add=20new=20utility=20module?= =?UTF-8?q?=20for=20adding=20DB=20indexes=20concurrently?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Introduces `Caseflow::Migrations::AddIndexConcurrently` as a replacement for `Caseflow::Migration` for migrations on ActiveRecord 6.0 and beyond, since `Caseflow::Migration` is forever coupled to ActiveRecord 5.1 due to its extensive use on legacy migrations and should be deprecated moving forward. --- .../migrations/add_index_concurrently.rb | 97 +++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 lib/caseflow/migrations/add_index_concurrently.rb diff --git a/lib/caseflow/migrations/add_index_concurrently.rb b/lib/caseflow/migrations/add_index_concurrently.rb new file mode 100644 index 00000000000..4a406530961 --- /dev/null +++ b/lib/caseflow/migrations/add_index_concurrently.rb @@ -0,0 +1,97 @@ +# frozen_string_literal: true + +# @note Usage: Include this module only in migrations where you need to add an index concurrently. +# Invoke `#add_safe_index` in place of `#add index`. +# +# @note Since this module necessarily disables transactions, you should avoid mixing in other DB changes when +# adding indexes concurrently. Prefer isolating index additions in their own migration. +# +# @see [PostgreSQL: Building Indexes Concurrently](https://www.postgresql.org/docs/14/sql-createindex.html#SQL-CREATEINDEX-CONCURRENTLY) +# @see [thoughtbot: How to Create Postgres Indexes Concurrently in ActiveRecord Migrations](https://thoughtbot.com/blog/how-to-create-postgres-indexes-concurrently-in) +# +# @example +# class YourMigrationName < ActiveRecord::Migration[6.0] +# include Caseflow::Migrations::AddIndexConcurrently +# +# change +# add_safe_index :some_table, :some_column +# end +# end +# +# @example +# # Alternatively, you can add in the requisite incantations yourself, without this module +# +# class YourMigrationName < ActiveRecord::Migration[6.0] +# disable_ddl_transaction! +# +# change +# add_index :some_table, :some_column, algorithm: :concurrently +# rescue StandardError => error +# remove_index :some_table, :some_column +# end +# end +module Caseflow + module Migrations + module AddIndexConcurrently + extend ActiveSupport::Concern + + EXTENDED_STATEMENT_TIMEOUT_DURATION = 30.minutes + + included do + # Disables the automatic transaction wrapping this migration. + # https://github.com/rails/rails/blob/28bb76d3efc39b2ef663dfe2346f7c2621343cd6/activerecord/lib/active_record/migration.rb#L508-L524 + disable_ddl_transaction! + + # @note Use this method in place of `#add_index` to add an index concurrently (i.e. without blocking DB writes). + # - Accommodates long-running index builds by extending the statement_timeout duration + # - Performs a rollback should an error occur during the migration, since transactions are disabled + # + # @note Inspired by {Caseflow::Migration} https://github.com/department-of-veterans-affairs/caseflow/blob/6fc9d26a5ae9417b69d7d1f30cc70bea57a0700d/lib/caseflow/migration.rb#L12-L29 + def add_safe_index(*args) + original_statement_timeout_duration = get_current_statement_timeout_duration + extend_statement_timeout + add_index_concurrently(args) + rescue StandardError => error + rollback_index(error, args) + raise error # re-raise to abort migration + ensure + restore_original_statement_timeout(original_statement_timeout_duration) + end + + private + + def get_current_statement_timeout_duration + ActiveSupport::Duration.build( + ActiveRecord::Base.connection.execute("SHOW statement_timeout").first["statement_timeout"].to_i + ) + end + + def extend_statement_timeout + say "Extending statement_timeout to #{EXTENDED_STATEMENT_TIMEOUT_DURATION.inspect}" + ActiveRecord::Base.connection.execute( + "SET statement_timeout = #{EXTENDED_STATEMENT_TIMEOUT_DURATION.in_milliseconds}" + ) + end + + def add_index_concurrently(args) + table, columns, options = *args + options ||= {} + options[:algorithm] ||= :concurrently + add_index(table, columns, options) + end + + def rollback_index(error, args) + say "Caught #{error}, rolling back index" + table, columns, options = *args + options[:column] = columns unless options[:name] + remove_index(table, options) + end + + def restore_original_statement_timeout(duration) + say "Restoring statement_timeout to #{duration.inspect}" + ActiveRecord::Base.connection.execute("SET statement_timeout = #{duration.in_milliseconds}") + end + end + end + end +end From df7e3ad1169f9e4dfdf3b360f4f1a2c7721fc492 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 4 Apr 2024 14:28:26 -0400 Subject: [PATCH 05/69] =?UTF-8?q?=F0=9F=97=91=EF=B8=8F=20Deprecate=20`Case?= =?UTF-8?q?flow::Migration`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../disallowed_deprecations.rb | 6 ++++++ lib/caseflow/migration.rb | 21 +++++++++++++++++-- 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/app/services/deprecation_warnings/disallowed_deprecations.rb b/app/services/deprecation_warnings/disallowed_deprecations.rb index df4f14a24b1..26ddc36eb36 100644 --- a/app/services/deprecation_warnings/disallowed_deprecations.rb +++ b/app/services/deprecation_warnings/disallowed_deprecations.rb @@ -6,6 +6,11 @@ module DisallowedDeprecations class ::DisallowedDeprecationError < StandardError; end + # Regular expressions for custom deprecation warnings that we have addressed in the codebase + CUSTOM_DEPRECATION_WARNING_REGEXES = [ + /Caseflow::Migration is deprecated/ + ].freeze + # Regular expressions for Rails 6.0 deprecation warnings that we have addressed in the codebase RAILS_6_0_FIXED_DEPRECATION_WARNING_REGEXES = [ /Dangerous query method \(method whose arguments are used as raw SQL\) called with non\-attribute argument\(s\)/, @@ -24,6 +29,7 @@ class ::DisallowedDeprecationError < StandardError; end # Regular expressions for deprecation warnings that should raise an exception on detection DISALLOWED_DEPRECATION_WARNING_REGEXES = [ + *CUSTOM_DEPRECATION_WARNING_REGEXES, *RAILS_6_0_FIXED_DEPRECATION_WARNING_REGEXES, *RAILS_6_1_FIXED_DEPRECATION_WARNING_REGEXES ].freeze diff --git a/lib/caseflow/migration.rb b/lib/caseflow/migration.rb index 7f8ad1b01db..9fcf092fae3 100644 --- a/lib/caseflow/migration.rb +++ b/lib/caseflow/migration.rb @@ -1,8 +1,25 @@ # frozen_string_literal: true -# Migration with built-in timeout extensions for adding indexes - +# @deprecated Use {Caseflow::Migrations::AddIndexConcurrently} instead, because descendants of this class are forever +# coupled to Active Record 5.1. +# This class should be preserved until all descendant migrations <= LAST_DESCENDANT_MIGRATION_VERSION are pruned. +# +# @note Migration with built-in timeout extensions for adding indexes class Caseflow::Migration < ActiveRecord::Migration[5.1] + # version of last migration that inherits from this class (across 'primary' and 'etl' databases) + LAST_DESCENDANT_MIGRATION_VERSION = 20_240_525_120_005 + + def initialize(*) + super + # Trigger deprecation warning to prevent re-introduction in migrations after version 20240227154315 + if version > LAST_DESCENDANT_MIGRATION_VERSION + ActiveSupport::Deprecation.warn( + "Caseflow::Migration is deprecated and should no longer be used.\n" \ + "If adding an index, see Caseflow::Migrations::AddIndexConcurrently." + ) + end + end + # hardcode this because setting via class method does not work in subclass def disable_ddl_transaction say "disable_ddl_transaction is true" From 6fbd3f78eacb55442e357dab56f90d428a1d35e9 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 4 Apr 2024 14:29:56 -0400 Subject: [PATCH 06/69] =?UTF-8?q?=F0=9F=94=A7=20Add=20instructive=20error?= =?UTF-8?q?=20message=20for=20non-concurrent=20`add=5Findex`=20migrations?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- config/initializers/strong_migrations.rb | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/config/initializers/strong_migrations.rb b/config/initializers/strong_migrations.rb index c079aa8007b..c6ebfb8c42d 100644 --- a/config/initializers/strong_migrations.rb +++ b/config/initializers/strong_migrations.rb @@ -1,2 +1,20 @@ StrongMigrations.start_after = 20190111000717 StrongMigrations.auto_analyze = true + +# Customized error message when a new migration uses `add_index` non-concurrently +StrongMigrations.error_messages[:add_index] = <<~TEXT + Adding a non-concurrent index locks the table for writes. + Instead, prefer using `Caseflow::Migrations::AddIndexConcurrently #add_safe_index`: + + class YourMigrationName < ActiveRecord::Migration%{migration_suffix} + include Caseflow::Migrations::AddIndexConcurrently + + def change + add_safe_index # add args here... + end + end + + Nota Bene: Since adding indexes concurrently must occur outside of a transaction, + one should avoid mixing in other DB changes when doing so. It is strongly recommended + to segregate index additions from other DB changes in a separate migration. + TEXT From 632cb359f51f5bc43efd478fa3d8c6183e6e6f74 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 4 Apr 2024 15:24:45 -0400 Subject: [PATCH 07/69] =?UTF-8?q?=F0=9F=9A=A8=20Address=20linter=20/=20cod?= =?UTF-8?q?eclimate=20complaints?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- lib/caseflow/migrations/add_index_concurrently.rb | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/lib/caseflow/migrations/add_index_concurrently.rb b/lib/caseflow/migrations/add_index_concurrently.rb index 4a406530961..fc3b7ac155e 100644 --- a/lib/caseflow/migrations/add_index_concurrently.rb +++ b/lib/caseflow/migrations/add_index_concurrently.rb @@ -48,7 +48,7 @@ module AddIndexConcurrently # # @note Inspired by {Caseflow::Migration} https://github.com/department-of-veterans-affairs/caseflow/blob/6fc9d26a5ae9417b69d7d1f30cc70bea57a0700d/lib/caseflow/migration.rb#L12-L29 def add_safe_index(*args) - original_statement_timeout_duration = get_current_statement_timeout_duration + original_statement_timeout_duration = current_statement_timeout_duration extend_statement_timeout add_index_concurrently(args) rescue StandardError => error @@ -60,7 +60,7 @@ def add_safe_index(*args) private - def get_current_statement_timeout_duration + def current_statement_timeout_duration ActiveSupport::Duration.build( ActiveRecord::Base.connection.execute("SHOW statement_timeout").first["statement_timeout"].to_i ) @@ -69,7 +69,8 @@ def get_current_statement_timeout_duration def extend_statement_timeout say "Extending statement_timeout to #{EXTENDED_STATEMENT_TIMEOUT_DURATION.inspect}" ActiveRecord::Base.connection.execute( - "SET statement_timeout = #{EXTENDED_STATEMENT_TIMEOUT_DURATION.in_milliseconds}" + ActiveRecord::Base.sanitize_sql(["SET statement_timeout = ?", + EXTENDED_STATEMENT_TIMEOUT_DURATION.in_milliseconds]) ) end @@ -87,9 +88,12 @@ def rollback_index(error, args) remove_index(table, options) end + # :reek:FeatureEnvy def restore_original_statement_timeout(duration) say "Restoring statement_timeout to #{duration.inspect}" - ActiveRecord::Base.connection.execute("SET statement_timeout = #{duration.in_milliseconds}") + ActiveRecord::Base.connection.execute( + ActiveRecord::Base.sanitize_sql(["SET statement_timeout = ?", duration.in_milliseconds]) + ) end end end From c6219d191ec6fedf9a70a3707af22c6fc67875d4 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Wed, 24 Apr 2024 15:03:07 -0400 Subject: [PATCH 08/69] =?UTF-8?q?=E2=9C=A8=20Introduce=20`SslRedirectExclu?= =?UTF-8?q?sionPolicy`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit To be used in the environment configuration settings for excluding exempt request paths from SSL redirects when `config. force_ssl = true` --- app/policies/ssl_redirect_exclusion_policy.rb | 27 +++++++ spec/requests/ssl_redirects_spec.rb | 75 +++++++++++++++++++ 2 files changed, 102 insertions(+) create mode 100644 app/policies/ssl_redirect_exclusion_policy.rb create mode 100644 spec/requests/ssl_redirects_spec.rb diff --git a/app/policies/ssl_redirect_exclusion_policy.rb b/app/policies/ssl_redirect_exclusion_policy.rb new file mode 100644 index 00000000000..41f19dba046 --- /dev/null +++ b/app/policies/ssl_redirect_exclusion_policy.rb @@ -0,0 +1,27 @@ +# frozen_string_literal: true + +# @note To be used in the environment configuration settings for excluding exempt request paths from SSL redirects +# when `config.force_ssl = true` +# +# @example config/environments/production.rb +# +# Rails.application.configure do +# config.force_ssl = true +# config.ssl_options = { redirect: { exclude: SslRedirectExclusionPolicy } } +# # etc. +class SslRedirectExclusionPolicy + EXEMPT_PATH_PATTERNS = [ + %r{^/api/docs/v3/}, + %r{^/api/metadata$}, + %r{^/health-check$}, + %r{^/idt/api/v1/}, + %r{^/idt/api/v2/}, + %r{^/pdfjs/} + ].freeze + + # @param [ActionDispatch::Request] request + # @return [TrueClass, FalseClass] true if request path is exempt from an SSL redirect + def self.call(request) + EXEMPT_PATH_PATTERNS.any? { |pattern| pattern =~ request.path } + end +end diff --git a/spec/requests/ssl_redirects_spec.rb b/spec/requests/ssl_redirects_spec.rb new file mode 100644 index 00000000000..33c1b3d8412 --- /dev/null +++ b/spec/requests/ssl_redirects_spec.rb @@ -0,0 +1,75 @@ +# frozen_string_literal: true + +describe "SSL Redirects" do + # `app` is what RSpec tests against in request specs, similar to `controller` in controller specs. + # Here, we override it with our modified app. + def app + # Since `CaseflowCertification::Application` is already loaded at this stage, we can't modify the middleware stack, + # so we subclass the application and adjust the relevant SSL config settings. + @app ||= Class.new(Rails.application.class) do + config.force_ssl = true + config.ssl_options = { redirect: { exclude: SslRedirectExclusionPolicy } } + end + end + + before { allow(SslRedirectExclusionPolicy).to receive(:call).and_call_original } + + context "when request is not SSL" do + context "when path matches '/api/docs/v3/'" do + it "is exempt from SSL redirect" do + get "/api/docs/v3/decision_reviews" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path is '/api/metadata'" do + it "is exempt from SSL redirect" do + get "/api/metadata" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path is '/health-check'" do + it "is exempt from SSL redirect" do + get "/health-check" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path matches '/idt/api/v1/'" do + it "is exempt from SSL redirect" do + get "/idt/api/v1/appeals" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path matches '/idt/api/v2/'" do + it "is exempt from SSL redirect" do + get "/idt/api/v2/appeals" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path matches '/pdfjs/'" do + it "is exempt from SSL redirect" do + get "/pdfjs/full?file=%2Fcertifications%2F2774535%2Fform9_pdf" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path is not exempt from SSL redirects" do + it "is redirected with SSL" do + get "/users" + expect(SslRedirectExclusionPolicy).to have_received(:call) + + expect(response).to redirect_to("https://#{request.host}/users") + end + end + end +end From 130791a87b94bc4c81fbc68f9e0e75e65341c7fa Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Wed, 24 Apr 2024 15:05:04 -0400 Subject: [PATCH 09/69] =?UTF-8?q?=E2=99=BB=EF=B8=8F=20Replace=20deprecated?= =?UTF-8?q?=20controller-level=20`force=5Fssl`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace deprecated controller-level `force_ssl` with equivalent configuration settings in preparation for the Rails 6.1 upgrade. --- app/controllers/api/application_controller.rb | 5 ----- app/controllers/application_base_controller.rb | 5 ----- app/services/deprecation_warnings/disallowed_deprecations.rb | 3 ++- config/environments/production.rb | 3 ++- 4 files changed, 4 insertions(+), 12 deletions(-) diff --git a/app/controllers/api/application_controller.rb b/app/controllers/api/application_controller.rb index ea12981e0fc..adb315f0d93 100644 --- a/app/controllers/api/application_controller.rb +++ b/app/controllers/api/application_controller.rb @@ -5,7 +5,6 @@ class Api::ApplicationController < ActionController::Base include TrackRequestId - force_ssl if: :ssl_enabled? before_action :strict_transport_security before_action :setup_fakes, @@ -51,10 +50,6 @@ def unauthorized render json: { status: "unauthorized" }, status: :unauthorized end - def ssl_enabled? - Rails.env.production? - end - def strict_transport_security response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains" if request.ssl? end diff --git a/app/controllers/application_base_controller.rb b/app/controllers/application_base_controller.rb index 41f55a7fe3b..f9b1304a804 100644 --- a/app/controllers/application_base_controller.rb +++ b/app/controllers/application_base_controller.rb @@ -8,7 +8,6 @@ class ApplicationBaseController < ActionController::Base include TrackRequestId - force_ssl if: :ssl_enabled? before_action :check_out_of_service before_action :strict_transport_security @@ -35,10 +34,6 @@ def check_out_of_service render "out_of_service", layout: "application" if Rails.cache.read("out_of_service") end - def ssl_enabled? - Rails.env.production? - end - def strict_transport_security response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains" if request.ssl? end diff --git a/app/services/deprecation_warnings/disallowed_deprecations.rb b/app/services/deprecation_warnings/disallowed_deprecations.rb index 26ddc36eb36..e365e45ef33 100644 --- a/app/services/deprecation_warnings/disallowed_deprecations.rb +++ b/app/services/deprecation_warnings/disallowed_deprecations.rb @@ -24,7 +24,8 @@ class ::DisallowedDeprecationError < StandardError; end /ActionView::Base instances must implement `compiled_method_container`/, /render file: should be given the absolute path to a file/, /`ActiveRecord::Result#to_hash` has been renamed to `to_a`/, - /Class level methods will no longer inherit scoping/ + /Class level methods will no longer inherit scoping/, + /Controller-level `force_ssl` is deprecated and will be removed from Rails 6\.1/ ].freeze # Regular expressions for deprecation warnings that should raise an exception on detection diff --git a/config/environments/production.rb b/config/environments/production.rb index c1b6847d036..aa0fb4423fe 100644 --- a/config/environments/production.rb +++ b/config/environments/production.rb @@ -44,7 +44,8 @@ # config.action_cable.allowed_request_origins = [ 'http://example.com', /http:\/\/example.*/ ] # Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies. - # config.force_ssl = true + config.force_ssl = true + config.ssl_options = { redirect: { exclude: SslRedirectExclusionPolicy } } # Use the lowest log level to ensure availability of diagnostic information # when problems arise. From be7a4c79511668e9262e0ce84296f12c208c3fb5 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Wed, 24 Apr 2024 22:05:08 -0400 Subject: [PATCH 10/69] =?UTF-8?q?=F0=9F=94=A5=20Remove=20SSL=20redirect=20?= =?UTF-8?q?exemptions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit It turns out that HSTS is being enforced at the `va.gov` domain level, so exempting these routes at the Rails middleware level is futile. --- app/policies/ssl_redirect_exclusion_policy.rb | 27 ------- config/environments/production.rb | 1 - spec/requests/ssl_redirects_spec.rb | 75 ------------------- 3 files changed, 103 deletions(-) delete mode 100644 app/policies/ssl_redirect_exclusion_policy.rb delete mode 100644 spec/requests/ssl_redirects_spec.rb diff --git a/app/policies/ssl_redirect_exclusion_policy.rb b/app/policies/ssl_redirect_exclusion_policy.rb deleted file mode 100644 index 41f19dba046..00000000000 --- a/app/policies/ssl_redirect_exclusion_policy.rb +++ /dev/null @@ -1,27 +0,0 @@ -# frozen_string_literal: true - -# @note To be used in the environment configuration settings for excluding exempt request paths from SSL redirects -# when `config.force_ssl = true` -# -# @example config/environments/production.rb -# -# Rails.application.configure do -# config.force_ssl = true -# config.ssl_options = { redirect: { exclude: SslRedirectExclusionPolicy } } -# # etc. -class SslRedirectExclusionPolicy - EXEMPT_PATH_PATTERNS = [ - %r{^/api/docs/v3/}, - %r{^/api/metadata$}, - %r{^/health-check$}, - %r{^/idt/api/v1/}, - %r{^/idt/api/v2/}, - %r{^/pdfjs/} - ].freeze - - # @param [ActionDispatch::Request] request - # @return [TrueClass, FalseClass] true if request path is exempt from an SSL redirect - def self.call(request) - EXEMPT_PATH_PATTERNS.any? { |pattern| pattern =~ request.path } - end -end diff --git a/config/environments/production.rb b/config/environments/production.rb index aa0fb4423fe..73bccf982ab 100644 --- a/config/environments/production.rb +++ b/config/environments/production.rb @@ -45,7 +45,6 @@ # Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies. config.force_ssl = true - config.ssl_options = { redirect: { exclude: SslRedirectExclusionPolicy } } # Use the lowest log level to ensure availability of diagnostic information # when problems arise. diff --git a/spec/requests/ssl_redirects_spec.rb b/spec/requests/ssl_redirects_spec.rb deleted file mode 100644 index 33c1b3d8412..00000000000 --- a/spec/requests/ssl_redirects_spec.rb +++ /dev/null @@ -1,75 +0,0 @@ -# frozen_string_literal: true - -describe "SSL Redirects" do - # `app` is what RSpec tests against in request specs, similar to `controller` in controller specs. - # Here, we override it with our modified app. - def app - # Since `CaseflowCertification::Application` is already loaded at this stage, we can't modify the middleware stack, - # so we subclass the application and adjust the relevant SSL config settings. - @app ||= Class.new(Rails.application.class) do - config.force_ssl = true - config.ssl_options = { redirect: { exclude: SslRedirectExclusionPolicy } } - end - end - - before { allow(SslRedirectExclusionPolicy).to receive(:call).and_call_original } - - context "when request is not SSL" do - context "when path matches '/api/docs/v3/'" do - it "is exempt from SSL redirect" do - get "/api/docs/v3/decision_reviews" - expect(SslRedirectExclusionPolicy).to have_received(:call) - expect(response).not_to have_http_status(:redirect) - end - end - - context "when path is '/api/metadata'" do - it "is exempt from SSL redirect" do - get "/api/metadata" - expect(SslRedirectExclusionPolicy).to have_received(:call) - expect(response).not_to have_http_status(:redirect) - end - end - - context "when path is '/health-check'" do - it "is exempt from SSL redirect" do - get "/health-check" - expect(SslRedirectExclusionPolicy).to have_received(:call) - expect(response).not_to have_http_status(:redirect) - end - end - - context "when path matches '/idt/api/v1/'" do - it "is exempt from SSL redirect" do - get "/idt/api/v1/appeals" - expect(SslRedirectExclusionPolicy).to have_received(:call) - expect(response).not_to have_http_status(:redirect) - end - end - - context "when path matches '/idt/api/v2/'" do - it "is exempt from SSL redirect" do - get "/idt/api/v2/appeals" - expect(SslRedirectExclusionPolicy).to have_received(:call) - expect(response).not_to have_http_status(:redirect) - end - end - - context "when path matches '/pdfjs/'" do - it "is exempt from SSL redirect" do - get "/pdfjs/full?file=%2Fcertifications%2F2774535%2Fform9_pdf" - expect(SslRedirectExclusionPolicy).to have_received(:call) - expect(response).not_to have_http_status(:redirect) - end - end - - context "when path is not exempt from SSL redirects" do - it "is redirected with SSL" do - get "/users" - expect(SslRedirectExclusionPolicy).to have_received(:call) - - expect(response).to redirect_to("https://#{request.host}/users") - end - end - end -end From ea1e1339d06e9999f7646a1c386d6a9b69237034 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 14 May 2024 14:45:34 -0400 Subject: [PATCH 11/69] =?UTF-8?q?=F0=9F=94=A5=20Remove=20deprecated=20conf?= =?UTF-8?q?ig=20setting=20`config.active=5Frecord.sqlite3.represent=5Fbool?= =?UTF-8?q?ean=5Fas=5Finteger`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This will have no implications for Caseflow, since we are only using the `sqlite3` adapter nominally for the `demo_vacols` database, which is not actually being used in our demo environments (demo environments are deployed as `development` envs). --- config/application.rb | 4 ---- 1 file changed, 4 deletions(-) diff --git a/config/application.rb b/config/application.rb index c9e1d5a9b19..301e611cfb3 100644 --- a/config/application.rb +++ b/config/application.rb @@ -74,10 +74,6 @@ class Application < Rails::Application # Default as of 5.2: true config.action_controller.default_protect_from_forgery = false - # Store boolean values in sqlite3 databases as 1 and 0 instead of 't' and 'f' after migrating old data. - # Default as of 5.2: true - config.active_record.sqlite3.represent_boolean_as_integer = false - # ------------------------------------------------------------------------------------------------------------------ # Rails 6.0 default overrides # ------------------------------------------------------------------------------------------------------------------ From e9ff41d32c2de026f480c044f9c948b06386bcd2 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Wed, 3 Jul 2024 13:19:53 -0400 Subject: [PATCH 12/69] =?UTF-8?q?=E2=AC=86=EF=B8=8F=20Update=20`caseflow-c?= =?UTF-8?q?ommons`=20to=20resolve=20sub-dependency=20conflicts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Removes unneeded gems `bourbon` and `neat`, which had a sub-dependency conflict on `thor`. --- Gemfile | 5 ++++- Gemfile.lock | 11 ++--------- 2 files changed, 6 insertions(+), 10 deletions(-) diff --git a/Gemfile b/Gemfile index 17eec027d43..7ed67bc55d8 100644 --- a/Gemfile +++ b/Gemfile @@ -17,7 +17,10 @@ gem "bgs", git: "https://github.com/department-of-veterans-affairs/ruby-bgs.git" gem "bootsnap", require: false gem "browser" gem "business_time", "~> 0.9.3" -gem "caseflow", git: "https://github.com/department-of-veterans-affairs/caseflow-commons", ref: "716b58caf2116da5fca21c3b3aeea6c9712f3b9d" + +# TODO: lock caseflow gem to latest ref on master after merging https://github.com/department-of-veterans-affairs/caseflow-commons/pull/225 +gem "caseflow", git: "https://github.com/department-of-veterans-affairs/caseflow-commons", branch: "AlecK/APPEALS-43566" + gem "connect_mpi", git: "https://github.com/department-of-veterans-affairs/connect-mpi.git", ref: "a3a58c64f85b980a8b5ea6347430dd73a99ea74c" gem "connect_vbms", git: "https://github.com/department-of-veterans-affairs/connect_vbms.git", ref: "9807d9c9f0f3e3494a60b6693dc4f455c1e3e922" gem "console_tree_renderer", git: "https://github.com/department-of-veterans-affairs/console-tree-renderer.git", tag: "v0.1.1" diff --git a/Gemfile.lock b/Gemfile.lock index 2239aa22953..f63b907a8e9 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -9,16 +9,14 @@ GIT GIT remote: https://github.com/department-of-veterans-affairs/caseflow-commons - revision: 716b58caf2116da5fca21c3b3aeea6c9712f3b9d - ref: 716b58caf2116da5fca21c3b3aeea6c9712f3b9d + revision: 4c04f19bd6703d9b8b240cadad36455db9cedae8 + branch: AlecK/APPEALS-43566 specs: caseflow (0.4.8) aws-sdk-s3 - bourbon (= 4.2.7) d3-rails jquery-rails momentjs-rails - neat rails (>= 4.2.7.1) redis-namespace redis-rails @@ -1458,9 +1456,6 @@ GEM benchmark-ips (2.7.2) bootsnap (1.7.5) msgpack (~> 1.0) - bourbon (4.2.7) - sass (~> 3.4) - thor (~> 0.19) brakeman (4.7.1) browser (5.3.1) builder (3.2.4) @@ -1725,8 +1720,6 @@ GEM activesupport (>= 4.2) railties (>= 4.2) nap (1.1.0) - neat (4.0.0) - thor (~> 0.19) nenv (0.3.0) net-imap (0.3.7) date From 85548e2a97a5247812562a139e0496a1927084ec Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 14 May 2024 18:10:02 -0400 Subject: [PATCH 13/69] =?UTF-8?q?=E2=AC=86=EF=B8=8F=20Update=20rails=20and?= =?UTF-8?q?=20other=20gems=20as=20necessary?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- Gemfile | 10 +-- Gemfile.lock | 203 +++++++++++++++++++++++++++------------------------ 2 files changed, 113 insertions(+), 100 deletions(-) diff --git a/Gemfile b/Gemfile index 7ed67bc55d8..86c4be0bbc7 100644 --- a/Gemfile +++ b/Gemfile @@ -5,7 +5,7 @@ source ENV["GEM_SERVER_URL"] || "https://rubygems.org" # State machine gem "aasm", "4.11.0" -gem "activerecord-import" +gem "activerecord-import", "1.0.3" gem "acts_as_tree" # amoeba gem for cloning appeals @@ -59,7 +59,7 @@ gem "pg", platforms: :ruby # Discussion: https://github.com/18F/college-choice/issues/597#issuecomment-139034834 gem "puma", "5.6.4" gem "rack", "~> 2.2.6.2" -gem "rails", "6.0.6.1" +gem "rails", "6.1.7.7" # Used to colorize output for rake tasks gem "rainbow" # React @@ -83,7 +83,7 @@ gem "stringex", require: false gem "strong_migrations" # print trees gem "tty-tree" -gem "tzinfo", "1.2.10" +gem "tzinfo", "~> 2.0" # Use Uglifier as compressor for JavaScript assets gem "uglifier", ">= 1.3.0" gem "validates_email_format_of" @@ -91,14 +91,14 @@ gem "ziptz" group :production, :staging, :ssh_forwarding, :development, :test do # Oracle DB - gem "activerecord-oracle_enhanced-adapter", "~> 6.0.0" + gem "activerecord-oracle_enhanced-adapter", "~> 6.1.0" gem "ruby-oci8", "~> 2.2" end group :test, :development, :demo do # Security scanners gem "brakeman" - gem "bullet" + gem "bullet", "~> 6.1.0" gem "bundler-audit" # Testing tools gem "capybara" diff --git a/Gemfile.lock b/Gemfile.lock index f63b907a8e9..9779c3d03c7 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -98,66 +98,70 @@ GEM remote: https://rubygems.org/ specs: aasm (4.11.0) - actioncable (6.0.6.1) - actionpack (= 6.0.6.1) + actioncable (6.1.7.7) + actionpack (= 6.1.7.7) + activesupport (= 6.1.7.7) nio4r (~> 2.0) websocket-driver (>= 0.6.1) - actionmailbox (6.0.6.1) - actionpack (= 6.0.6.1) - activejob (= 6.0.6.1) - activerecord (= 6.0.6.1) - activestorage (= 6.0.6.1) - activesupport (= 6.0.6.1) + actionmailbox (6.1.7.7) + actionpack (= 6.1.7.7) + activejob (= 6.1.7.7) + activerecord (= 6.1.7.7) + activestorage (= 6.1.7.7) + activesupport (= 6.1.7.7) mail (>= 2.7.1) - actionmailer (6.0.6.1) - actionpack (= 6.0.6.1) - actionview (= 6.0.6.1) - activejob (= 6.0.6.1) + actionmailer (6.1.7.7) + actionpack (= 6.1.7.7) + actionview (= 6.1.7.7) + activejob (= 6.1.7.7) + activesupport (= 6.1.7.7) mail (~> 2.5, >= 2.5.4) rails-dom-testing (~> 2.0) - actionpack (6.0.6.1) - actionview (= 6.0.6.1) - activesupport (= 6.0.6.1) - rack (~> 2.0, >= 2.0.8) + actionpack (6.1.7.7) + actionview (= 6.1.7.7) + activesupport (= 6.1.7.7) + rack (~> 2.0, >= 2.0.9) rack-test (>= 0.6.3) rails-dom-testing (~> 2.0) rails-html-sanitizer (~> 1.0, >= 1.2.0) - actiontext (6.0.6.1) - actionpack (= 6.0.6.1) - activerecord (= 6.0.6.1) - activestorage (= 6.0.6.1) - activesupport (= 6.0.6.1) + actiontext (6.1.7.7) + actionpack (= 6.1.7.7) + activerecord (= 6.1.7.7) + activestorage (= 6.1.7.7) + activesupport (= 6.1.7.7) nokogiri (>= 1.8.5) - actionview (6.0.6.1) - activesupport (= 6.0.6.1) + actionview (6.1.7.7) + activesupport (= 6.1.7.7) builder (~> 3.1) erubi (~> 1.4) rails-dom-testing (~> 2.0) rails-html-sanitizer (~> 1.1, >= 1.2.0) - activejob (6.0.6.1) - activesupport (= 6.0.6.1) + activejob (6.1.7.7) + activesupport (= 6.1.7.7) globalid (>= 0.3.6) - activemodel (6.0.6.1) - activesupport (= 6.0.6.1) - activerecord (6.0.6.1) - activemodel (= 6.0.6.1) - activesupport (= 6.0.6.1) - activerecord-import (1.0.2) + activemodel (6.1.7.7) + activesupport (= 6.1.7.7) + activerecord (6.1.7.7) + activemodel (= 6.1.7.7) + activesupport (= 6.1.7.7) + activerecord-import (1.0.3) activerecord (>= 3.2) - activerecord-oracle_enhanced-adapter (6.0.6) - activerecord (~> 6.0.0) + activerecord-oracle_enhanced-adapter (6.1.6) + activerecord (~> 6.1.0) ruby-plsql (>= 0.6.0) - activestorage (6.0.6.1) - actionpack (= 6.0.6.1) - activejob (= 6.0.6.1) - activerecord (= 6.0.6.1) + activestorage (6.1.7.7) + actionpack (= 6.1.7.7) + activejob (= 6.1.7.7) + activerecord (= 6.1.7.7) + activesupport (= 6.1.7.7) marcel (~> 1.0) - activesupport (6.0.6.1) + mini_mime (>= 1.1.0) + activesupport (6.1.7.7) concurrent-ruby (~> 1.0, >= 1.0.2) - i18n (>= 0.7, < 2) - minitest (~> 5.1) - tzinfo (~> 1.1) - zeitwerk (~> 2.2, >= 2.2.2) + i18n (>= 1.6, < 2) + minitest (>= 5.1) + tzinfo (~> 2.0) + zeitwerk (~> 2.3) acts_as_tree (2.9.0) activerecord (>= 3.0.0) addressable (2.8.0) @@ -1453,21 +1457,22 @@ GEM aws-eventstream (~> 1, >= 1.0.2) backport (1.2.0) base64 (0.2.0) + benchmark (0.3.0) benchmark-ips (2.7.2) bootsnap (1.7.5) msgpack (~> 1.0) brakeman (4.7.1) browser (5.3.1) builder (3.2.4) - bullet (6.0.1) + bullet (6.1.5) activesupport (>= 3.0.0) uniform_notifier (~> 1.11) bummr (0.5.0) rainbow thor - bundler-audit (0.6.1) + bundler-audit (0.9.1) bundler (>= 1.2.0, < 3) - thor (~> 0.18) + thor (~> 1.0) business_time (0.9.3) activesupport (>= 3.2.0) tzinfo @@ -1526,24 +1531,29 @@ GEM date (3.3.3) ddtrace (0.34.1) msgpack + dead_end (4.0.0) debase (0.2.4.1) debase-ruby_core_source (>= 0.10.2) debase-ruby_core_source (0.10.14) - derailed_benchmarks (1.3.6) + derailed_benchmarks (2.1.2) benchmark-ips (~> 2) + dead_end get_process_mem (~> 0) heapy (~> 0) - memory_profiler (~> 0) + memory_profiler (>= 0, < 2) + mini_histogram (>= 0.3.0) rack (>= 1) - rake (> 10, < 13) - thor (~> 0.19) + rack-test + rake (> 10, < 14) + ruby-statistics (>= 2.1) + thor (>= 0.19, < 2) diff-lcs (1.3) docile (1.1.5) dogstatsd-ruby (4.4.0) - dotenv (2.7.5) - dotenv-rails (2.7.5) - dotenv (= 2.7.5) - railties (>= 3.2, < 6.1) + dotenv (2.8.1) + dotenv-rails (2.8.1) + dotenv (= 2.8.1) + railties (>= 3.2) dry-configurable (0.11.5) concurrent-ruby (~> 1.0) dry-core (~> 0.4, >= 0.4.7) @@ -1575,6 +1585,7 @@ GEM dry-equalizer (~> 0.3) dry-inflector (~> 0.1, >= 0.1.2) dry-logic (~> 1.0, >= 1.0.2) + e2mmap (0.1.0) ecma-re-validator (0.2.1) regexp_parser (~> 1.2) erubi (1.12.0) @@ -1608,8 +1619,8 @@ GEM git (1.13.2) addressable (~> 2.8) rchardet (~> 1.8) - globalid (1.1.0) - activesupport (>= 5.0) + globalid (1.2.1) + activesupport (>= 6.1) govdelivery-tms (2.8.4) activesupport faraday @@ -1697,8 +1708,7 @@ GEM net-imap net-pop net-smtp - marcel (1.0.2) - maruku (0.7.3) + marcel (1.0.4) memory_profiler (0.9.14) meta_request (0.7.2) rack-contrib (>= 1.1, < 3) @@ -1707,6 +1717,7 @@ GEM mime-types (3.3) mime-types-data (~> 3.2015) mime-types-data (3.2019.1009) + mini_histogram (0.3.1) mini_mime (1.1.2) mini_portile2 (2.8.5) minitest (5.19.0) @@ -1748,8 +1759,8 @@ GEM activerecord (>= 5.2) request_store (~> 1.1) parallel (1.23.0) - paranoia (2.4.2) - activerecord (>= 4.0, < 6.1) + paranoia (2.6.3) + activerecord (>= 5.1, < 7.2) parser (2.7.2.0) ast (~> 2.4.1) pdf-forms (1.2.0) @@ -1775,20 +1786,20 @@ GEM rack (~> 2.0) rack-test (2.1.0) rack (>= 1.3) - rails (6.0.6.1) - actioncable (= 6.0.6.1) - actionmailbox (= 6.0.6.1) - actionmailer (= 6.0.6.1) - actionpack (= 6.0.6.1) - actiontext (= 6.0.6.1) - actionview (= 6.0.6.1) - activejob (= 6.0.6.1) - activemodel (= 6.0.6.1) - activerecord (= 6.0.6.1) - activestorage (= 6.0.6.1) - activesupport (= 6.0.6.1) - bundler (>= 1.3.0) - railties (= 6.0.6.1) + rails (6.1.7.7) + actioncable (= 6.1.7.7) + actionmailbox (= 6.1.7.7) + actionmailer (= 6.1.7.7) + actionpack (= 6.1.7.7) + actiontext (= 6.1.7.7) + actionview (= 6.1.7.7) + activejob (= 6.1.7.7) + activemodel (= 6.1.7.7) + activerecord (= 6.1.7.7) + activestorage (= 6.1.7.7) + activesupport (= 6.1.7.7) + bundler (>= 1.15.0) + railties (= 6.1.7.7) sprockets-rails (>= 2.0.0) rails-dom-testing (2.1.1) activesupport (>= 5.0.0) @@ -1802,12 +1813,12 @@ GEM rails-html-sanitizer (1.6.0) loofah (~> 2.21) nokogiri (~> 1.14) - railties (6.0.6.1) - actionpack (= 6.0.6.1) - activesupport (= 6.0.6.1) + railties (6.1.7.7) + actionpack (= 6.1.7.7) + activesupport (= 6.1.7.7) method_source - rake (>= 0.8.7) - thor (>= 0.20.3, < 2.0) + rake (>= 12.2) + thor (~> 1.0) rainbow (3.1.1) rake (12.3.3) rb-fsevent (0.11.2) @@ -1911,6 +1922,7 @@ GEM ruby-plsql (0.8.0) ruby-prof (1.4.1) ruby-progressbar (1.13.0) + ruby-statistics (3.0.2) ruby_dep (1.5.0) ruby_parser (3.20.3) sexp_processor (~> 4.16) @@ -1959,18 +1971,20 @@ GEM single_cov (1.3.2) sixarm_ruby_unaccent (1.2.0) socksify (1.7.1) - solargraph (0.38.0) + solargraph (0.40.1) backport (~> 1.1) + benchmark bundler (>= 1.17.2) + e2mmap jaro_winkler (~> 1.5) - maruku (~> 0.7, >= 0.7.3) - nokogiri (~> 1.9, >= 1.9.1) + kramdown (~> 2.3) + kramdown-parser-gfm (~> 1.1) parser (~> 2.3) - reverse_markdown (~> 1.0, >= 1.0.5) - rubocop (~> 0.52) - thor (~> 0.19, >= 0.19.4) + reverse_markdown (>= 1.0.5, < 3) + rubocop (>= 0.52) + thor (~> 1.0) tilt (~> 2.0) - yard (~> 0.9) + yard (~> 0.9, >= 0.9.24) sprockets (3.7.2) concurrent-ruby (~> 1.0) rack (> 1, < 3) @@ -1986,19 +2000,18 @@ GEM terminal-table (1.8.0) unicode-display_width (~> 1.1, >= 1.1.1) test-prof (0.10.1) - thor (0.20.3) - thread_safe (0.3.6) + thor (1.3.1) tilt (2.0.8) timecop (0.9.1) timeout (0.4.0) tty-tree (0.3.0) - tzinfo (1.2.10) - thread_safe (~> 0.1) + tzinfo (2.0.6) + concurrent-ruby (~> 1.0) uglifier (4.1.20) execjs (>= 0.3.0, < 3) unicode-display_width (1.8.0) unicode_utils (1.4.0) - uniform_notifier (1.12.1) + uniform_notifier (1.16.0) uri_template (0.7.0) validates_email_format_of (1.6.3) i18n @@ -2040,8 +2053,8 @@ PLATFORMS DEPENDENCIES aasm (= 4.11.0) - activerecord-import - activerecord-oracle_enhanced-adapter (~> 6.0.0) + activerecord-import (= 1.0.3) + activerecord-oracle_enhanced-adapter (~> 6.1.0) acts_as_tree amoeba anbt-sql-formatter @@ -2050,7 +2063,7 @@ DEPENDENCIES bootsnap brakeman browser - bullet + bullet (~> 6.1.0) bummr bundler-audit business_time (~> 0.9.3) @@ -2101,7 +2114,7 @@ DEPENDENCIES pry-byebug (~> 3.9) puma (= 5.6.4) rack (~> 2.2.6.2) - rails (= 6.0.6.1) + rails (= 6.1.7.7) rails-erd rainbow rb-readline @@ -2142,7 +2155,7 @@ DEPENDENCIES test-prof timecop tty-tree - tzinfo (= 1.2.10) + tzinfo (~> 2.0) uglifier (>= 1.3.0) validates_email_format_of webdrivers From 116663c375ae362195a0bbc4e4097f3cd2b0a2d8 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 14 May 2024 18:13:51 -0400 Subject: [PATCH 14/69] =?UTF-8?q?=F0=9F=90=9B=20Fix=20'uninitialized=20con?= =?UTF-8?q?stant'=20error=20when=20loading=20app?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- app/services/deprecation_warnings/development_handler.rb | 1 + app/services/deprecation_warnings/production_handler.rb | 2 ++ app/services/deprecation_warnings/test_handler.rb | 1 + 3 files changed, 4 insertions(+) diff --git a/app/services/deprecation_warnings/development_handler.rb b/app/services/deprecation_warnings/development_handler.rb index a61d262e5e6..50d6dd67737 100644 --- a/app/services/deprecation_warnings/development_handler.rb +++ b/app/services/deprecation_warnings/development_handler.rb @@ -1,5 +1,6 @@ # frozen_string_literal: true +require_relative "base_handler" require_relative "disallowed_deprecations" # @note For use with `ActiveSupport::Deprecation.behavior=`. diff --git a/app/services/deprecation_warnings/production_handler.rb b/app/services/deprecation_warnings/production_handler.rb index fe1c1310d78..e5e0c1ccf72 100644 --- a/app/services/deprecation_warnings/production_handler.rb +++ b/app/services/deprecation_warnings/production_handler.rb @@ -1,5 +1,7 @@ # frozen_string_literal: true +require_relative "base_handler" + # @note For use with `ActiveSupport::Deprecation.behavior=`. module DeprecationWarnings class ProductionHandler < BaseHandler diff --git a/app/services/deprecation_warnings/test_handler.rb b/app/services/deprecation_warnings/test_handler.rb index b87945be19c..2454ac6edf7 100644 --- a/app/services/deprecation_warnings/test_handler.rb +++ b/app/services/deprecation_warnings/test_handler.rb @@ -1,5 +1,6 @@ # frozen_string_literal: true +require_relative "base_handler" require_relative "disallowed_deprecations" # @note For use with `ActiveSupport::Deprecation.behavior=`. From a814438335343761dc7c1a8fe9f386a678eecb31 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Wed, 15 May 2024 15:43:23 -0400 Subject: [PATCH 15/69] =?UTF-8?q?=E2=AC=86=EF=B8=8F=20bin/rails=20app:upda?= =?UTF-8?q?te=20-=20Apply=20relevant=20changes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- bin/rails | 4 +- bin/rake | 4 +- bin/setup | 8 +-- bin/spring | 16 ----- bin/yarn | 12 +++- config.ru | 5 +- config/application.rb | 12 ++-- config/boot.rb | 4 +- config/environment.rb | 2 +- config/environments/development.rb | 20 +++++- config/environments/production.rb | 12 +++- config/environments/test.rb | 14 +++- config/initializers/backtrace_silencers.rb | 7 +- .../initializers/filter_parameter_logging.rb | 4 +- .../new_framework_defaults_6_1.rb | 67 +++++++++++++++++++ config/initializers/permissions_policy.rb | 11 +++ config/puma.rb | 7 +- config/routes.rb | 2 - 18 files changed, 157 insertions(+), 54 deletions(-) delete mode 100755 bin/spring create mode 100644 config/initializers/new_framework_defaults_6_1.rb create mode 100644 config/initializers/permissions_policy.rb diff --git a/bin/rails b/bin/rails index 07396602377..6fb4e4051c4 100755 --- a/bin/rails +++ b/bin/rails @@ -1,4 +1,4 @@ #!/usr/bin/env ruby APP_PATH = File.expand_path('../config/application', __dir__) -require_relative '../config/boot' -require 'rails/commands' +require_relative "../config/boot" +require "rails/commands" diff --git a/bin/rake b/bin/rake index 17240489f64..4fbf10b960e 100755 --- a/bin/rake +++ b/bin/rake @@ -1,4 +1,4 @@ #!/usr/bin/env ruby -require_relative '../config/boot' -require 'rake' +require_relative "../config/boot" +require "rake" Rake.application.run diff --git a/bin/setup b/bin/setup index 5853b5ea875..90700ac4f9a 100755 --- a/bin/setup +++ b/bin/setup @@ -1,5 +1,5 @@ #!/usr/bin/env ruby -require 'fileutils' +require "fileutils" # path to your application root. APP_ROOT = File.expand_path('..', __dir__) @@ -9,8 +9,8 @@ def system!(*args) end FileUtils.chdir APP_ROOT do - # This script is a way to setup or update your development environment automatically. - # This script is idempotent, so that you can run it at anytime and get an expectable outcome. + # This script is a way to set up or update your development environment automatically. + # This script is idempotent, so that you can run it at any time and get an expectable outcome. # Add necessary setup steps to this file. puts '== Installing dependencies ==' @@ -18,7 +18,7 @@ FileUtils.chdir APP_ROOT do system('bundle check') || system!('bundle install') # Install JavaScript dependencies - # system('bin/yarn') + system! 'bin/yarn' # puts "\n== Copying sample files ==" # unless File.exist?('config/database.yml') diff --git a/bin/spring b/bin/spring deleted file mode 100755 index a4759986d15..00000000000 --- a/bin/spring +++ /dev/null @@ -1,16 +0,0 @@ -#!/usr/bin/env ruby -# frozen_string_literal: true - -# This file loads spring without using Bundler, in order to be fast. -# It gets overwritten when you run the `spring binstub` command. - -unless defined?(Spring) - require "rubygems" - require "bundler" - - if (match = Bundler.default_lockfile.read.match(/^GEM$.*?^ (?: )*spring \((.*?)\)$.*?^$/m)) - Gem.paths = { "GEM_PATH" => [Bundler.bundle_path.to_s, *Gem.path].uniq } - gem "spring", match[1] - require "spring/binstub" - end -end diff --git a/bin/yarn b/bin/yarn index 460dd565b4a..9fab2c35079 100755 --- a/bin/yarn +++ b/bin/yarn @@ -1,9 +1,15 @@ #!/usr/bin/env ruby APP_ROOT = File.expand_path('..', __dir__) Dir.chdir(APP_ROOT) do - begin - exec "yarnpkg", *ARGV - rescue Errno::ENOENT + yarn = ENV["PATH"].split(File::PATH_SEPARATOR). + select { |dir| File.expand_path(dir) != __dir__ }. + product(["yarn", "yarn.cmd", "yarn.ps1"]). + map { |dir, file| File.expand_path(file, dir) }. + find { |file| File.executable?(file) } + + if yarn + exec yarn, *ARGV + else $stderr.puts "Yarn executable was not detected in the system." $stderr.puts "Download Yarn at https://yarnpkg.com/en/docs/install" exit 1 diff --git a/config.ru b/config.ru index cb504065454..265d93376ed 100644 --- a/config.ru +++ b/config.ru @@ -1,8 +1,6 @@ -# frozen_string_literal: true - # This file is used by Rack-based servers to start the application. -require ::File.expand_path("../config/environment", __FILE__) +require_relative "config/environment" require "rack" # rubocop:disable all @@ -69,3 +67,4 @@ end # rubocop:enable all run Rails.application +Rails.application.load_server diff --git a/config/application.rb b/config/application.rb index 301e611cfb3..0de6f174062 100644 --- a/config/application.rb +++ b/config/application.rb @@ -1,6 +1,6 @@ -require_relative 'boot' +require_relative "boot" -require 'rails/all' +require "rails/all" # Require the gems listed in Gemfile, including any gems # you've limited to :test, :development, or :production. @@ -15,10 +15,10 @@ class Application < Rails::Application config.load_defaults 6.0 config.autoloader = :classic - # Settings in config/environments/* take precedence over those specified here. - # Application configuration can go into files in config/initializers - # -- all .rb files in that directory are automatically loaded after loading - # the framework and any gems in your application. + # Configuration for the application, engines, and railties goes here. + # + # These settings can be overridden in specific environments using the files + # in config/environments, which are processed later. # ================================================================================================================== # Rails default overrides diff --git a/config/boot.rb b/config/boot.rb index 9833741ac57..54cda1cdcc7 100644 --- a/config/boot.rb +++ b/config/boot.rb @@ -1,5 +1,5 @@ ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../Gemfile', __dir__) ENV["NLS_LANG"] = "AMERICAN_AMERICA.UTF8" -require 'bundler/setup' # Set up gems listed in the Gemfile. -require 'bootsnap/setup' # Speed up boot time by caching expensive operations. +require "bundler/setup" # Set up gems listed in the Gemfile. +require "bootsnap/setup" # Speed up boot time by caching expensive operations. diff --git a/config/environment.rb b/config/environment.rb index 426333bb469..cac53157752 100644 --- a/config/environment.rb +++ b/config/environment.rb @@ -1,5 +1,5 @@ # Load the Rails application. -require_relative 'application' +require_relative "application" # Initialize the Rails application. Rails.application.initialize! diff --git a/config/environments/development.rb b/config/environments/development.rb index e27e16487b1..964efacc0dd 100644 --- a/config/environments/development.rb +++ b/config/environments/development.rb @@ -1,8 +1,10 @@ +require "active_support/core_ext/integer/time" + Rails.application.configure do # Settings specified here will take precedence over those in config/application.rb. - # In the development environment your application's code is reloaded on - # every request. This slows down response time but is perfect for development + # In the development environment your application's code is reloaded any time + # it changes. This slows down response time but is perfect for development # since you don't have to restart the web server when you make code changes. config.cache_classes = false @@ -49,6 +51,12 @@ require_relative "../../app/services/deprecation_warnings/development_handler" ActiveSupport::Deprecation.behavior = DeprecationWarnings::DevelopmentHandler + # Raise exceptions for disallowed deprecations. + # config.active_support.disallowed_deprecation = :raise + + # Tell Active Support which deprecation messages to disallow. + # config.active_support.disallowed_deprecation_warnings = [] + # Raise an error on page load if there are pending migrations. config.active_record.migration_error = :page_load @@ -64,7 +72,10 @@ config.assets.quiet = true # Raises error for missing translations. - # config.action_view.raise_on_missing_translations = true + # config.i18n.raise_on_missing_translations = true + + # Annotate rendered view with file names. + # config.action_view.annotate_rendered_view_with_filenames = true # Use an evented file watcher to asynchronously detect changes in source code, # routes, locales, etc. This feature depends on the listen gem. @@ -73,6 +84,9 @@ # Bypass DNS rebinding protection for all `demo` sub-domains config.hosts << ".demo.appeals.va.gov" + # Uncomment if you wish to allow Action Cable access from any origin. + # config.action_cable.disable_request_forgery_protection = true + #===================================================================================================================== # Please keep custom config settings below this comment. # This will ensure cleaner diffs when generating config file changes during Rails upgrades. diff --git a/config/environments/production.rb b/config/environments/production.rb index 73bccf982ab..692632b2979 100644 --- a/config/environments/production.rb +++ b/config/environments/production.rb @@ -1,3 +1,5 @@ +require "active_support/core_ext/integer/time" + Rails.application.configure do # Settings specified here will take precedence over those in config/application.rb. @@ -29,7 +31,7 @@ config.assets.compile = false # Enable serving of images, stylesheets, and JavaScripts from an asset server. - # config.action_controller.asset_host = 'http://assets.example.com' + # config.asset_host = 'http://assets.example.com' # Specifies the header that your server uses for sending files. # config.action_dispatch.x_sendfile_header = 'X-Sendfile' # for Apache @@ -82,11 +84,17 @@ require_relative "../../app/services/deprecation_warnings/production_handler" ActiveSupport::Deprecation.behavior = DeprecationWarnings::ProductionHandler + # Log disallowed deprecations. + # config.active_support.disallowed_deprecation = :log + + # Tell Active Support which deprecation messages to disallow. + # config.active_support.disallowed_deprecation_warnings = [] + # Use default logging formatter so that PID and timestamp are not suppressed. config.log_formatter = ::Logger::Formatter.new # Use a different logger for distributed setups. - # require 'syslog/logger' + # require "syslog/logger" # config.logger = ActiveSupport::TaggedLogging.new(Syslog::Logger.new 'app-name') if ENV["RAILS_LOG_TO_STDOUT"].present? diff --git a/config/environments/test.rb b/config/environments/test.rb index 83e3a687dd0..e77d05547dd 100644 --- a/config/environments/test.rb +++ b/config/environments/test.rb @@ -1,3 +1,4 @@ +require "active_support/core_ext/integer/time" require "fileutils" # The test environment is used exclusively to run your application's @@ -50,15 +51,22 @@ require_relative "../../app/services/deprecation_warnings/test_handler" ActiveSupport::Deprecation.behavior = DeprecationWarnings::TestHandler + # Raise exceptions for disallowed deprecations. + # config.active_support.disallowed_deprecation = :raise + + # Tell Active Support which deprecation messages to disallow. + # config.active_support.disallowed_deprecation_warnings = [] + unless ENV['RAILS_ENABLE_TEST_LOG'] config.logger = Logger.new(nil) config.log_level = :error end - config.action_mailer.delivery_method = :test - # Raises error for missing translations. - # config.action_view.raise_on_missing_translations = true + # config.i18n.raise_on_missing_translations = true + + # Annotate rendered view with file names. + # config.action_view.annotate_rendered_view_with_filenames = true #===================================================================================================================== # Please keep custom config settings below this comment. diff --git a/config/initializers/backtrace_silencers.rb b/config/initializers/backtrace_silencers.rb index 59385cdf379..33699c30910 100644 --- a/config/initializers/backtrace_silencers.rb +++ b/config/initializers/backtrace_silencers.rb @@ -1,7 +1,8 @@ # Be sure to restart your server when you modify this file. # You can add backtrace silencers for libraries that you're using but don't wish to see in your backtraces. -# Rails.backtrace_cleaner.add_silencer { |line| line =~ /my_noisy_library/ } +# Rails.backtrace_cleaner.add_silencer { |line| /my_noisy_library/.match?(line) } -# You can also remove all the silencers if you're trying to debug a problem that might stem from framework code. -# Rails.backtrace_cleaner.remove_silencers! +# You can also remove all the silencers if you're trying to debug a problem that might stem from framework code +# by setting BACKTRACE=1 before calling your invocation, like "BACKTRACE=1 ./bin/rails runner 'MyClass.perform'". +Rails.backtrace_cleaner.remove_silencers! if ENV["BACKTRACE"] diff --git a/config/initializers/filter_parameter_logging.rb b/config/initializers/filter_parameter_logging.rb index 4a994e1e7bb..4b34a036689 100644 --- a/config/initializers/filter_parameter_logging.rb +++ b/config/initializers/filter_parameter_logging.rb @@ -1,4 +1,6 @@ # Be sure to restart your server when you modify this file. # Configure sensitive parameters which will be filtered from the log file. -Rails.application.config.filter_parameters += [:password] +Rails.application.config.filter_parameters += [ + :passw, :secret, :token, :_key, :crypt, :salt, :certificate, :otp, :ssn +] diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb new file mode 100644 index 00000000000..9526b835ab6 --- /dev/null +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -0,0 +1,67 @@ +# Be sure to restart your server when you modify this file. +# +# This file contains migration options to ease your Rails 6.1 upgrade. +# +# Once upgraded flip defaults one by one to migrate to the new default. +# +# Read the Guide for Upgrading Ruby on Rails for more info on each option. + +# Support for inversing belongs_to -> has_many Active Record associations. +# Rails.application.config.active_record.has_many_inversing = true + +# Track Active Storage variants in the database. +# Rails.application.config.active_storage.track_variants = true + +# Apply random variation to the delay when retrying failed jobs. +# Rails.application.config.active_job.retry_jitter = 0.15 + +# Stop executing `after_enqueue`/`after_perform` callbacks if +# `before_enqueue`/`before_perform` respectively halts with `throw :abort`. +# Rails.application.config.active_job.skip_after_callbacks_if_terminated = true + +# Specify cookies SameSite protection level: either :none, :lax, or :strict. +# +# This change is not backwards compatible with earlier Rails versions. +# It's best enabled when your entire app is migrated and stable on 6.1. +# Rails.application.config.action_dispatch.cookies_same_site_protection = :lax + +# Generate CSRF tokens that are encoded in URL-safe Base64. +# +# This change is not backwards compatible with earlier Rails versions. +# It's best enabled when your entire app is migrated and stable on 6.1. +# Rails.application.config.action_controller.urlsafe_csrf_tokens = true + +# Specify whether `ActiveSupport::TimeZone.utc_to_local` returns a time with an +# UTC offset or a UTC time. +# ActiveSupport.utc_to_local_returns_utc_offset_times = true + +# Change the default HTTP status code to `308` when redirecting non-GET/HEAD +# requests to HTTPS in `ActionDispatch::SSL` middleware. +# Rails.application.config.action_dispatch.ssl_default_redirect_status = 308 + +# Use new connection handling API. For most applications this won't have any +# effect. For applications using multiple databases, this new API provides +# support for granular connection swapping. +# Rails.application.config.active_record.legacy_connection_handling = false + +# Make `form_with` generate non-remote forms by default. +# Rails.application.config.action_view.form_with_generates_remote_forms = false + +# Set the default queue name for the analysis job to the queue adapter default. +# Rails.application.config.active_storage.queues.analysis = nil + +# Set the default queue name for the purge job to the queue adapter default. +# Rails.application.config.active_storage.queues.purge = nil + +# Set the default queue name for the incineration job to the queue adapter default. +# Rails.application.config.action_mailbox.queues.incineration = nil + +# Set the default queue name for the routing job to the queue adapter default. +# Rails.application.config.action_mailbox.queues.routing = nil + +# Set the default queue name for the mail deliver job to the queue adapter default. +# Rails.application.config.action_mailer.deliver_later_queue_name = nil + +# Generate a `Link` header that gives a hint to modern browsers about +# preloading assets when using `javascript_include_tag` and `stylesheet_link_tag`. +# Rails.application.config.action_view.preload_links_header = true diff --git a/config/initializers/permissions_policy.rb b/config/initializers/permissions_policy.rb new file mode 100644 index 00000000000..00f64d71b03 --- /dev/null +++ b/config/initializers/permissions_policy.rb @@ -0,0 +1,11 @@ +# Define an application-wide HTTP permissions policy. For further +# information see https://developers.google.com/web/updates/2018/06/feature-policy +# +# Rails.application.config.permissions_policy do |f| +# f.camera :none +# f.gyroscope :none +# f.microphone :none +# f.usb :none +# f.fullscreen :self +# f.payment :self, "https://secure.example.com" +# end diff --git a/config/puma.rb b/config/puma.rb index b2c644daa93..b1b0f09e2da 100644 --- a/config/puma.rb +++ b/config/puma.rb @@ -8,9 +8,14 @@ min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count } threads min_threads_count, max_threads_count +# Specifies the `worker_timeout` threshold that Puma will use to wait before +# terminating a worker in development environments. +# +worker_timeout 3600 if ENV.fetch("RAILS_ENV", "development") == "development" + # Specifies the `port` that Puma will listen on to receive requests; default is 3000. # -port ENV.fetch("PORT") { 3000 } +port ENV.fetch("PORT") { 3000 } # Specifies the `environment` that Puma will run in. # diff --git a/config/routes.rb b/config/routes.rb index c9f14e7da8b..0de5ed0f529 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -2,8 +2,6 @@ mount Rswag::Ui::Engine => '/api-docs' mount Rswag::Api::Engine => '/api-docs' # For details on the DSL available within this file, see https://guides.rubyonrails.org/routing.html - # The priority is based upon order of creation: first created -> highest priority. - # See how all your routes lay out with "rake routes". resources :sessions, only: [:new, :update] resources :certifications, path_names: { new: "new/:vacols_id" } do From e87fc5ac6cbe82415acdf4dd3a5109c42fb5ec88 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 11:22:01 -0400 Subject: [PATCH 16/69] =?UTF-8?q?=F0=9F=94=A7=20Override=20default=20for?= =?UTF-8?q?=20`config.active=5Frecord.has=5Fmany=5Finversing`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- config/application.rb | 8 ++++++++ config/initializers/new_framework_defaults_6_1.rb | 3 --- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/config/application.rb b/config/application.rb index 0de6f174062..dd5d4f747e3 100644 --- a/config/application.rb +++ b/config/application.rb @@ -92,6 +92,14 @@ class Application < Rails::Application # Default as of 6.0: true config.active_record.collection_cache_versioning = false + # ------------------------------------------------------------------------------------------------------------------ + # Rails 6.1 default overrides + # ------------------------------------------------------------------------------------------------------------------ + + # Support for inversing belongs_to -> has_many Active Record associations. + # Default as of 6.1: true + config.active_record.has_many_inversing = false + # ================================================================================================================== # Set Time.zone default to the specified zone and make Active Record auto-convert to this zone. diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 9526b835ab6..0d25657af4d 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,9 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Support for inversing belongs_to -> has_many Active Record associations. -# Rails.application.config.active_record.has_many_inversing = true - # Track Active Storage variants in the database. # Rails.application.config.active_storage.track_variants = true From a3ce67014223368af555de0c178ffd8a524a74d8 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 11:26:13 -0400 Subject: [PATCH 17/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.active=5Fstorage.track=5Fvariants`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We're not currently using ActiveStorage in Caseflow, so it is safe to just assume the default here. --- config/initializers/new_framework_defaults_6_1.rb | 3 --- 1 file changed, 3 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 0d25657af4d..d86caf0e3d9 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,9 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Track Active Storage variants in the database. -# Rails.application.config.active_storage.track_variants = true - # Apply random variation to the delay when retrying failed jobs. # Rails.application.config.active_job.retry_jitter = 0.15 From 04790fd02ffc4f83752a116dbeb2c971c15c7b04 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 28 May 2024 12:04:57 -0400 Subject: [PATCH 18/69] =?UTF-8?q?=F0=9F=94=A7=20Override=20default=20for?= =?UTF-8?q?=20`config.active=5Fjob.retry=5Fjitter`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The default jitter is probably safe, however, I'm not 100% sure that we don't have any jobs that need to be requeued with exact wait times. So we let's override this for now to stay on the safe side. --- config/application.rb | 4 ++++ config/initializers/new_framework_defaults_6_1.rb | 3 --- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/config/application.rb b/config/application.rb index dd5d4f747e3..2331384a132 100644 --- a/config/application.rb +++ b/config/application.rb @@ -100,6 +100,10 @@ class Application < Rails::Application # Default as of 6.1: true config.active_record.has_many_inversing = false + # Apply random variation to the delay when retrying failed jobs. + # Default as of 6.1: 0.15 + config.active_job.retry_jitter = 0 + # ================================================================================================================== # Set Time.zone default to the specified zone and make Active Record auto-convert to this zone. diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index d86caf0e3d9..9c0d04cdcdb 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,9 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Apply random variation to the delay when retrying failed jobs. -# Rails.application.config.active_job.retry_jitter = 0.15 - # Stop executing `after_enqueue`/`after_perform` callbacks if # `before_enqueue`/`before_perform` respectively halts with `throw :abort`. # Rails.application.config.active_job.skip_after_callbacks_if_terminated = true From 54018356039fd4a90d427210cf4a16d0dc23dcb4 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 11:29:31 -0400 Subject: [PATCH 19/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.active=5Fjob.skip=5Fafter=5Fcallbacks=5Fif=5Fterminated`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We're not currently using `throw :abort` within any `before_enqueue`/`before_perform`  callbacks on existing Caseflow jobs, so the default should be fine here. For more background, see https://lilyreile.medium.com/rails-6-1-new-framework-defaults-what-they-do-and-how-to-safely-uncomment-them-c546b70f0c5e#4c60 --- config/initializers/new_framework_defaults_6_1.rb | 4 ---- 1 file changed, 4 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 9c0d04cdcdb..608bbd68d78 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,10 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Stop executing `after_enqueue`/`after_perform` callbacks if -# `before_enqueue`/`before_perform` respectively halts with `throw :abort`. -# Rails.application.config.active_job.skip_after_callbacks_if_terminated = true - # Specify cookies SameSite protection level: either :none, :lax, or :strict. # # This change is not backwards compatible with earlier Rails versions. From 0fcf28b2e18621148584499b7b1b80609497c790 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 11:37:12 -0400 Subject: [PATCH 20/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.action=5Fdispatch.cookies=5Fsame=5Fsite=5Fprotection`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This setting controls the `SameSite` optional attribute for the `Set-Cookie` header. `SameSite=Lax` means that the cookie is not sent on cross-site requests, such as on requests to load images or frames, but is sent when a user is navigating to the origin site from an external site (for example, when following a link). This is the default behavior if the SameSite attribute is not specified. `Lax` is currently the default assumed by both Chrome and Edge browsers when this attribute is left unspecified, so assuming this value should be sensible. It allows us to have our cake (blocking CSRF attacks) and eat it too (providing a logged-in experience when users navigate to Caseflow across origins). For more background, see - https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie#samesitesamesite-value - https://lilyreile.medium.com/rails-6-1-new-framework-defaults-what-they-do-and-how-to-safely-uncomment-them-c546b70f0c5e#1f15 --- config/initializers/new_framework_defaults_6_1.rb | 6 ------ 1 file changed, 6 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 608bbd68d78..35764dfeb34 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,12 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Specify cookies SameSite protection level: either :none, :lax, or :strict. -# -# This change is not backwards compatible with earlier Rails versions. -# It's best enabled when your entire app is migrated and stable on 6.1. -# Rails.application.config.action_dispatch.cookies_same_site_protection = :lax - # Generate CSRF tokens that are encoded in URL-safe Base64. # # This change is not backwards compatible with earlier Rails versions. From b6381ef080ca70e40dc7723b9b5a21a3b4275696 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 14:57:36 -0400 Subject: [PATCH 21/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.action=5Fcontroller.urlsafe=5Fcsrf=5Ftokens`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- config/initializers/new_framework_defaults_6_1.rb | 6 ------ 1 file changed, 6 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 35764dfeb34..adb08ac97f2 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,12 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Generate CSRF tokens that are encoded in URL-safe Base64. -# -# This change is not backwards compatible with earlier Rails versions. -# It's best enabled when your entire app is migrated and stable on 6.1. -# Rails.application.config.action_controller.urlsafe_csrf_tokens = true - # Specify whether `ActiveSupport::TimeZone.utc_to_local` returns a time with an # UTC offset or a UTC time. # ActiveSupport.utc_to_local_returns_utc_offset_times = true From 408eb1d3e6f5857f85202934e91d1cce3375b80d Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:06:32 -0400 Subject: [PATCH 22/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?ActiveSupport.utc=5Fto=5Flocal=5Freturns=5Futc=5Foffset=5Ftimes?= =?UTF-8?q?`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We're not using `ActiveSupport::TimeZone.utc_to_local` anywhere, so the default is safe to assume here. --- config/initializers/new_framework_defaults_6_1.rb | 4 ---- 1 file changed, 4 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index adb08ac97f2..80c3285e428 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,10 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Specify whether `ActiveSupport::TimeZone.utc_to_local` returns a time with an -# UTC offset or a UTC time. -# ActiveSupport.utc_to_local_returns_utc_offset_times = true - # Change the default HTTP status code to `308` when redirecting non-GET/HEAD # requests to HTTPS in `ActionDispatch::SSL` middleware. # Rails.application.config.action_dispatch.ssl_default_redirect_status = 308 From 7b9e0e1839cb22e2b2fc96f9bd32b0d67a96b98b Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:09:20 -0400 Subject: [PATCH 23/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.action=5Fdispatch.ssl=5Fdefault=5Fredirect=5Fstatus`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The default is safe to assume. For more background, see https://lilyreile.medium.com/rails-6-1-new-framework-defaults-what-they-do-and-how-to-safely-uncomment-them-c546b70f0c5e#4c3e --- config/initializers/new_framework_defaults_6_1.rb | 4 ---- 1 file changed, 4 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 80c3285e428..1d8ba6fdaea 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,10 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Change the default HTTP status code to `308` when redirecting non-GET/HEAD -# requests to HTTPS in `ActionDispatch::SSL` middleware. -# Rails.application.config.action_dispatch.ssl_default_redirect_status = 308 - # Use new connection handling API. For most applications this won't have any # effect. For applications using multiple databases, this new API provides # support for granular connection swapping. From 7c33f1038f70593a6749d7f60668e13361a20f94 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:16:33 -0400 Subject: [PATCH 24/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.active=5Frecord.legacy=5Fconnection=5Fhandling`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The default should be safe to assume here, as we do not do any role or shard switching on database connections. For more background, see https://lilyreile.medium.com/rails-6-1-new-framework-defaults-what-they-do-and-how-to-safely-uncomment-them-c546b70f0c5e#8007 --- config/initializers/new_framework_defaults_6_1.rb | 5 ----- 1 file changed, 5 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 1d8ba6fdaea..1eea1bece52 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,11 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Use new connection handling API. For most applications this won't have any -# effect. For applications using multiple databases, this new API provides -# support for granular connection swapping. -# Rails.application.config.active_record.legacy_connection_handling = false - # Make `form_with` generate non-remote forms by default. # Rails.application.config.action_view.form_with_generates_remote_forms = false From 8ae96ffe1b2fb4a13a780d089f51650a1f6679ec Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:23:16 -0400 Subject: [PATCH 25/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.action=5Fview.form=5Fwith=5Fgenerates=5Fremote=5Fforms`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We don't use the `form_with` helper anywhere, so this behavior change is inconsequential for us, and we can safely assume the new default. --- config/initializers/new_framework_defaults_6_1.rb | 3 --- 1 file changed, 3 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 1eea1bece52..00b04aeb582 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,9 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Make `form_with` generate non-remote forms by default. -# Rails.application.config.action_view.form_with_generates_remote_forms = false - # Set the default queue name for the analysis job to the queue adapter default. # Rails.application.config.active_storage.queues.analysis = nil From 5d036d058553ca55ec819714dd4b245444436fab Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:24:56 -0400 Subject: [PATCH 26/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.active=5Fstorage.queues.analysis`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We do not use ActiveStorage, so the default is safe to assume here. --- config/initializers/new_framework_defaults_6_1.rb | 3 --- 1 file changed, 3 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 00b04aeb582..b30c6060a12 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,9 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Set the default queue name for the analysis job to the queue adapter default. -# Rails.application.config.active_storage.queues.analysis = nil - # Set the default queue name for the purge job to the queue adapter default. # Rails.application.config.active_storage.queues.purge = nil From b08aa7fc55ddb474b5869d9ebbaa2538fdcb5e70 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:26:00 -0400 Subject: [PATCH 27/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.active=5Fstorage.queues.purge`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We do not use ActiveStorage, so the default is safe to assume here. --- config/initializers/new_framework_defaults_6_1.rb | 3 --- 1 file changed, 3 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index b30c6060a12..2c92bce64f6 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,9 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Set the default queue name for the purge job to the queue adapter default. -# Rails.application.config.active_storage.queues.purge = nil - # Set the default queue name for the incineration job to the queue adapter default. # Rails.application.config.action_mailbox.queues.incineration = nil From 5a5d66f2751e1668f019f07accd8f254e612b94f Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:28:05 -0400 Subject: [PATCH 28/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.action=5Fmailbox.queues.incineration`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We don't use ActionMailbox, so the new default is safe to assume here. --- config/initializers/new_framework_defaults_6_1.rb | 3 --- 1 file changed, 3 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 2c92bce64f6..0b4cc229a32 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,9 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Set the default queue name for the incineration job to the queue adapter default. -# Rails.application.config.action_mailbox.queues.incineration = nil - # Set the default queue name for the routing job to the queue adapter default. # Rails.application.config.action_mailbox.queues.routing = nil From 01a7c41104fe56986693889a1c689821c55acf98 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:30:34 -0400 Subject: [PATCH 29/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.action=5Fmailbox.queues.routing`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We do not use ActionMailbox, so the default is safe to assume here. --- config/initializers/new_framework_defaults_6_1.rb | 3 --- 1 file changed, 3 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 0b4cc229a32..d415da704f9 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,9 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Set the default queue name for the routing job to the queue adapter default. -# Rails.application.config.action_mailbox.queues.routing = nil - # Set the default queue name for the mail deliver job to the queue adapter default. # Rails.application.config.action_mailer.deliver_later_queue_name = nil From a2d02436ad72af4594c8aeea2341c4efa4cea72d Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:33:01 -0400 Subject: [PATCH 30/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.action=5Fmailer.deliver=5Flater=5Fqueue=5Fname`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We're not using `ActionMailer::MessageDelivery #deliver_later` anywhere, so the default is safe to assume. --- config/initializers/new_framework_defaults_6_1.rb | 3 --- 1 file changed, 3 deletions(-) diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index d415da704f9..0ddf5b132a7 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -6,9 +6,6 @@ # # Read the Guide for Upgrading Ruby on Rails for more info on each option. -# Set the default queue name for the mail deliver job to the queue adapter default. -# Rails.application.config.action_mailer.deliver_later_queue_name = nil - # Generate a `Link` header that gives a hint to modern browsers about # preloading assets when using `javascript_include_tag` and `stylesheet_link_tag`. # Rails.application.config.action_view.preload_links_header = true From 5e41632d22c933ec2c3989379390e07fc30dc84e Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:38:52 -0400 Subject: [PATCH 31/69] =?UTF-8?q?=F0=9F=94=A7=20Assume=20default=20for=20`?= =?UTF-8?q?config.action=5Fview.preload=5Flinks=5Fheader`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This flag can be safely uncommented. Browsers that support Link headers will get a performance boost. Browsers that don’t will ignore them. We override in `development` environments to avoid an edge case leading to an HTTP response header overflow. For more background, see https://lilyreile.medium.com/rails-6-1-new-framework-defaults-what-they-do-and-how-to-safely-uncomment-them-c546b70f0c5e#3679 --- config/environments/development.rb | 5 +++++ config/initializers/new_framework_defaults_6_1.rb | 4 ---- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/config/environments/development.rb b/config/environments/development.rb index 964efacc0dd..3651fc6c2a1 100644 --- a/config/environments/development.rb +++ b/config/environments/development.rb @@ -3,6 +3,11 @@ Rails.application.configure do # Settings specified here will take precedence over those in config/application.rb. + # When `config.assets.debug == true`, there is an edge case where the length of the Link header could become + # exceptionally long, due to the way concatenated assets are split and included separately, thus exceeding the + # maximum 8192 bytes for HTTP response headers. To preclude this from happening, we override the default here: + config.action_view.preload_links_header = false + # In the development environment your application's code is reloaded any time # it changes. This slows down response time but is perfect for development # since you don't have to restart the web server when you make code changes. diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb index 0ddf5b132a7..d5951253808 100644 --- a/config/initializers/new_framework_defaults_6_1.rb +++ b/config/initializers/new_framework_defaults_6_1.rb @@ -5,7 +5,3 @@ # Once upgraded flip defaults one by one to migrate to the new default. # # Read the Guide for Upgrading Ruby on Rails for more info on each option. - -# Generate a `Link` header that gives a hint to modern browsers about -# preloading assets when using `javascript_include_tag` and `stylesheet_link_tag`. -# Rails.application.config.action_view.preload_links_header = true From 8e8c0704716273a06c0c030d08e4e29df0911f29 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:53:36 -0400 Subject: [PATCH 32/69] =?UTF-8?q?=F0=9F=94=A5=20Remove=20'new=5Fframework?= =?UTF-8?q?=5Fdefaults=5F6=5F1.rb'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- config/initializers/new_framework_defaults_6_1.rb | 7 ------- 1 file changed, 7 deletions(-) delete mode 100644 config/initializers/new_framework_defaults_6_1.rb diff --git a/config/initializers/new_framework_defaults_6_1.rb b/config/initializers/new_framework_defaults_6_1.rb deleted file mode 100644 index d5951253808..00000000000 --- a/config/initializers/new_framework_defaults_6_1.rb +++ /dev/null @@ -1,7 +0,0 @@ -# Be sure to restart your server when you modify this file. -# -# This file contains migration options to ease your Rails 6.1 upgrade. -# -# Once upgraded flip defaults one by one to migrate to the new default. -# -# Read the Guide for Upgrading Ruby on Rails for more info on each option. From cf8a03349ecaf8bc919555f89abd9293fc3b8934 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 16 May 2024 15:55:36 -0400 Subject: [PATCH 33/69] =?UTF-8?q?=F0=9F=94=A7=20Load=20defaults=20for=20Ra?= =?UTF-8?q?ils=206.1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- config/application.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config/application.rb b/config/application.rb index 2331384a132..c900992294f 100644 --- a/config/application.rb +++ b/config/application.rb @@ -12,7 +12,7 @@ module CaseflowCertification class Application < Rails::Application # Initialize configuration defaults for originally generated Rails version. - config.load_defaults 6.0 + config.load_defaults 6.1 config.autoloader = :classic # Configuration for the application, engines, and railties goes here. From c99784a7842e2d67c7f91bae5bddb17ed8c7fb88 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 17 May 2024 13:58:33 -0400 Subject: [PATCH 34/69] =?UTF-8?q?=E2=99=BB=EF=B8=8F=20Extract=20constant?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- app/services/deprecation_warnings/production_handler.rb | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/app/services/deprecation_warnings/production_handler.rb b/app/services/deprecation_warnings/production_handler.rb index e5e0c1ccf72..4bb31c34882 100644 --- a/app/services/deprecation_warnings/production_handler.rb +++ b/app/services/deprecation_warnings/production_handler.rb @@ -6,6 +6,7 @@ module DeprecationWarnings class ProductionHandler < BaseHandler APP_NAME = "caseflow" + SLACK_ALERT_TITLE = "Deprecation Warning - #{APP_NAME} (#{ENV['DEPLOY_ENV']})" SLACK_ALERT_CHANNEL = "#appeals-deprecation-alerts" class << self @@ -46,9 +47,7 @@ def emit_warning_to_sentry(message, callstack, deprecation_horizon, gem_name) end def emit_warning_to_slack_alerts_channel(message) - slack_alert_title = "Deprecation Warning - #{APP_NAME} (#{ENV['DEPLOY_ENV']})" - - SlackService.new.send_notification(message, slack_alert_title, SLACK_ALERT_CHANNEL) + SlackService.new.send_notification(message, SLACK_ALERT_TITLE, SLACK_ALERT_CHANNEL) end end end From 1a37f4b817d34b6c9816dfd2b8ea41ff5ddeb72f Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 17 May 2024 14:23:11 -0400 Subject: [PATCH 35/69] =?UTF-8?q?=E2=99=BB=EF=B8=8F=20Migrate=20to=20new?= =?UTF-8?q?=20Rails=20deprecation=20config=20where=20applicable?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../development_handler.rb | 25 ---------- .../production_handler.rb | 7 +-- .../deprecation_warnings/test_handler.rb | 25 ---------- config/environments/development.rb | 9 ++-- config/environments/production.rb | 12 ++--- config/environments/test.rb | 9 ++-- .../initializers/deprecation_warnings.rb | 14 +----- .../development_handler_spec.rb | 46 ------------------- .../production_handler_spec.rb | 6 --- .../deprecation_warnings/test_handler_spec.rb | 38 --------------- 10 files changed, 16 insertions(+), 175 deletions(-) delete mode 100644 app/services/deprecation_warnings/development_handler.rb delete mode 100644 app/services/deprecation_warnings/test_handler.rb rename app/services/deprecation_warnings/disallowed_deprecations.rb => config/initializers/deprecation_warnings.rb (71%) delete mode 100644 spec/services/deprecation_warnings/development_handler_spec.rb delete mode 100644 spec/services/deprecation_warnings/test_handler_spec.rb diff --git a/app/services/deprecation_warnings/development_handler.rb b/app/services/deprecation_warnings/development_handler.rb deleted file mode 100644 index 50d6dd67737..00000000000 --- a/app/services/deprecation_warnings/development_handler.rb +++ /dev/null @@ -1,25 +0,0 @@ -# frozen_string_literal: true - -require_relative "base_handler" -require_relative "disallowed_deprecations" - -# @note For use with `ActiveSupport::Deprecation.behavior=`. -module DeprecationWarnings - class DevelopmentHandler < BaseHandler - extend DisallowedDeprecations - - class << self - # :reek:LongParameterList - def call(message, _callstack, _deprecation_horizon, _gem_name) - raise_if_disallowed_deprecation!(message) - emit_warning_to_application_logs(message) - end - - private - - def emit_warning_to_application_logs(message) - Rails.logger.warn(message) - end - end - end -end diff --git a/app/services/deprecation_warnings/production_handler.rb b/app/services/deprecation_warnings/production_handler.rb index 4bb31c34882..248922f74e2 100644 --- a/app/services/deprecation_warnings/production_handler.rb +++ b/app/services/deprecation_warnings/production_handler.rb @@ -2,7 +2,7 @@ require_relative "base_handler" -# @note For use with `ActiveSupport::Deprecation.behavior=`. +# @note For use with `ActiveSupport::Deprecation.behavior=` or `Rails.application.config.active_support.deprecation=` module DeprecationWarnings class ProductionHandler < BaseHandler APP_NAME = "caseflow" @@ -12,7 +12,6 @@ class ProductionHandler < BaseHandler class << self # :reek:LongParameterList def call(message, callstack, deprecation_horizon, gem_name) - emit_warning_to_application_logs(message) emit_warning_to_sentry(message, callstack, deprecation_horizon, gem_name) emit_warning_to_slack_alerts_channel(message) rescue StandardError => error @@ -21,10 +20,6 @@ def call(message, callstack, deprecation_horizon, gem_name) private - def emit_warning_to_application_logs(message) - Rails.logger.warn(message) - end - # :reek:LongParameterList def emit_warning_to_sentry(message, callstack, deprecation_horizon, gem_name) # Pre-emptive bugfix for future versions of the `sentry-raven` gem: diff --git a/app/services/deprecation_warnings/test_handler.rb b/app/services/deprecation_warnings/test_handler.rb deleted file mode 100644 index 2454ac6edf7..00000000000 --- a/app/services/deprecation_warnings/test_handler.rb +++ /dev/null @@ -1,25 +0,0 @@ -# frozen_string_literal: true - -require_relative "base_handler" -require_relative "disallowed_deprecations" - -# @note For use with `ActiveSupport::Deprecation.behavior=`. -module DeprecationWarnings - class TestHandler < BaseHandler - extend DisallowedDeprecations - - class << self - # :reek:LongParameterList - def call(message, _callstack, _deprecation_horizon, _gem_name) - raise_if_disallowed_deprecation!(message) - emit_error_to_stderr(message) - end - - private - - def emit_error_to_stderr(message) - ActiveSupport::Logger.new($stderr).error(message) - end - end - end -end diff --git a/config/environments/development.rb b/config/environments/development.rb index 3651fc6c2a1..804eeaa15c0 100644 --- a/config/environments/development.rb +++ b/config/environments/development.rb @@ -1,4 +1,5 @@ require "active_support/core_ext/integer/time" +require_relative "../../config/initializers/deprecation_warnings" Rails.application.configure do # Settings specified here will take precedence over those in config/application.rb. @@ -52,15 +53,13 @@ end # Print deprecation notices to the Rails logger. - # config.active_support.deprecation = :log - require_relative "../../app/services/deprecation_warnings/development_handler" - ActiveSupport::Deprecation.behavior = DeprecationWarnings::DevelopmentHandler + config.active_support.deprecation = :log # Raise exceptions for disallowed deprecations. - # config.active_support.disallowed_deprecation = :raise + config.active_support.disallowed_deprecation = :raise # Tell Active Support which deprecation messages to disallow. - # config.active_support.disallowed_deprecation_warnings = [] + config.active_support.disallowed_deprecation_warnings = DeprecationWarnings::DISALLOWED_DEPRECATION_WARNING_REGEXES # Raise an error on page load if there are pending migrations. config.active_record.migration_error = :page_load diff --git a/config/environments/production.rb b/config/environments/production.rb index 692632b2979..310cd90b1fc 100644 --- a/config/environments/production.rb +++ b/config/environments/production.rb @@ -1,4 +1,6 @@ require "active_support/core_ext/integer/time" +require_relative "../../app/services/deprecation_warnings/production_handler" +require_relative "../../config/initializers/deprecation_warnings" Rails.application.configure do # Settings specified here will take precedence over those in config/application.rb. @@ -79,16 +81,14 @@ # the I18n.default_locale when a translation cannot be found). config.i18n.fallbacks = true - # Send deprecation notices to registered listeners. - # config.active_support.deprecation = :notify - require_relative "../../app/services/deprecation_warnings/production_handler" - ActiveSupport::Deprecation.behavior = DeprecationWarnings::ProductionHandler + # Send deprecation notices to handler. + config.active_support.deprecation = DeprecationWarnings::ProductionHandler # Log disallowed deprecations. - # config.active_support.disallowed_deprecation = :log + config.active_support.disallowed_deprecation = :log # Tell Active Support which deprecation messages to disallow. - # config.active_support.disallowed_deprecation_warnings = [] + config.active_support.disallowed_deprecation_warnings = DeprecationWarnings::DISALLOWED_DEPRECATION_WARNING_REGEXES # Use default logging formatter so that PID and timestamp are not suppressed. config.log_formatter = ::Logger::Formatter.new diff --git a/config/environments/test.rb b/config/environments/test.rb index e77d05547dd..49d5e11344f 100644 --- a/config/environments/test.rb +++ b/config/environments/test.rb @@ -1,5 +1,6 @@ require "active_support/core_ext/integer/time" require "fileutils" +require_relative "../../config/initializers/deprecation_warnings" # The test environment is used exclusively to run your application's # test suite. You never need to work with it otherwise. Remember that @@ -47,15 +48,13 @@ config.action_mailer.delivery_method = :test # Print deprecation notices to the stderr. - # config.active_support.deprecation = :stderr - require_relative "../../app/services/deprecation_warnings/test_handler" - ActiveSupport::Deprecation.behavior = DeprecationWarnings::TestHandler + config.active_support.deprecation = :stderr # Raise exceptions for disallowed deprecations. - # config.active_support.disallowed_deprecation = :raise + config.active_support.disallowed_deprecation = :raise # Tell Active Support which deprecation messages to disallow. - # config.active_support.disallowed_deprecation_warnings = [] + config.active_support.disallowed_deprecation_warnings = DeprecationWarnings::DISALLOWED_DEPRECATION_WARNING_REGEXES unless ENV['RAILS_ENABLE_TEST_LOG'] config.logger = Logger.new(nil) diff --git a/app/services/deprecation_warnings/disallowed_deprecations.rb b/config/initializers/deprecation_warnings.rb similarity index 71% rename from app/services/deprecation_warnings/disallowed_deprecations.rb rename to config/initializers/deprecation_warnings.rb index e365e45ef33..f19ee0b8884 100644 --- a/app/services/deprecation_warnings/disallowed_deprecations.rb +++ b/config/initializers/deprecation_warnings.rb @@ -1,11 +1,6 @@ # frozen_string_literal: true -# @note Temporary solution for disallowed deprecation warnings. -# To be replaced by ActiveSupport Disallowed Deprecations after upgrading to Rails 6.1: -# https://rubyonrails.org/2020/12/9/Rails-6-1-0-release#disallowed-deprecation-support -module DisallowedDeprecations - class ::DisallowedDeprecationError < StandardError; end - +module DeprecationWarnings # Regular expressions for custom deprecation warnings that we have addressed in the codebase CUSTOM_DEPRECATION_WARNING_REGEXES = [ /Caseflow::Migration is deprecated/ @@ -34,11 +29,4 @@ class ::DisallowedDeprecationError < StandardError; end *RAILS_6_0_FIXED_DEPRECATION_WARNING_REGEXES, *RAILS_6_1_FIXED_DEPRECATION_WARNING_REGEXES ].freeze - - # @param message [String] deprecation warning message to be checked against disallow list - def raise_if_disallowed_deprecation!(message) - if DISALLOWED_DEPRECATION_WARNING_REGEXES.any? { |re| re.match?(message) } - fail DisallowedDeprecationError, message - end - end end diff --git a/spec/services/deprecation_warnings/development_handler_spec.rb b/spec/services/deprecation_warnings/development_handler_spec.rb deleted file mode 100644 index 0848fd5b3ae..00000000000 --- a/spec/services/deprecation_warnings/development_handler_spec.rb +++ /dev/null @@ -1,46 +0,0 @@ -# frozen_string_literal: true - -module DeprecationWarnings - describe DevelopmentHandler do - context ".call" do - subject(:call) do - described_class.call(message, _callstack = [], _deprecation_horizon = "6.0", _gem_name = "Rails") - end - - let(:message) { "dummy deprecation message" } - let(:rails_logger) { Rails.logger } - - before do - allow(Rails).to receive(:logger).and_return(rails_logger) - allow(rails_logger).to receive(:warn) - end - - it "emits a warning to the application logs" do - call - - expect(rails_logger).to have_received(:warn).with(message) - end - - context "when deprecation is allowed" do - let(:message) { "allowed deprecation message" } - - it "does not raise error" do - expect { call }.not_to raise_error - end - end - - context "when deprecation is disallowed" do - let(:message) { "disallowed deprecation message" } - - before do - stub_const("DisallowedDeprecations::DISALLOWED_DEPRECATION_WARNING_REGEXES", - [Regexp.new(Regexp.escape(message))]) - end - - it "raises DisallowedDeprecationError" do - expect { call }.to raise_error(::DisallowedDeprecationError) - end - end - end - end -end diff --git a/spec/services/deprecation_warnings/production_handler_spec.rb b/spec/services/deprecation_warnings/production_handler_spec.rb index e28cf27f0e1..a34c219d584 100644 --- a/spec/services/deprecation_warnings/production_handler_spec.rb +++ b/spec/services/deprecation_warnings/production_handler_spec.rb @@ -25,12 +25,6 @@ module DeprecationWarnings allow(slack_service).to receive(:send_notification) end - it "emits a warning to the application logs" do - call - - expect(rails_logger).to have_received(:warn).with(message) - end - it "emits a warning to Sentry" do call diff --git a/spec/services/deprecation_warnings/test_handler_spec.rb b/spec/services/deprecation_warnings/test_handler_spec.rb deleted file mode 100644 index 0ddf9bec0ad..00000000000 --- a/spec/services/deprecation_warnings/test_handler_spec.rb +++ /dev/null @@ -1,38 +0,0 @@ -# frozen_string_literal: true - -module DeprecationWarnings - describe TestHandler do - context ".call" do - subject(:call) do - described_class.call(message, _callstack = [], _deprecation_horizon = "6.0", _gem_name = "Rails") - end - - let(:message) { "dummy deprecation message" } - - it "logs message to stderr" do - expect { call }.to output("#{message}\n").to_stderr - end - - context "when deprecation is allowed" do - let(:message) { "allowed deprecation message" } - - it "does not raise error" do - expect { call }.not_to raise_error - end - end - - context "when deprecation is disallowed" do - let(:message) { "disallowed deprecation message" } - - before do - stub_const("DisallowedDeprecations::DISALLOWED_DEPRECATION_WARNING_REGEXES", - [Regexp.new(Regexp.escape(message))]) - end - - it "raises DisallowedDeprecationError" do - expect { call }.to raise_error(::DisallowedDeprecationError) - end - end - end - end -end From 63de9e9ebf841f202a4e6db82eadaae1511503ce Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 17 May 2024 14:29:00 -0400 Subject: [PATCH 36/69] =?UTF-8?q?=E2=99=BB=EF=B8=8F=20Push=20members=20dow?= =?UTF-8?q?n=20now=20that=20there=20is=20only=20one=20subclass?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../deprecation_warnings/base_handler.rb | 23 ------------------- .../production_handler.rb | 13 ++++++++--- 2 files changed, 10 insertions(+), 26 deletions(-) delete mode 100644 app/services/deprecation_warnings/base_handler.rb diff --git a/app/services/deprecation_warnings/base_handler.rb b/app/services/deprecation_warnings/base_handler.rb deleted file mode 100644 index 65b473bfb38..00000000000 --- a/app/services/deprecation_warnings/base_handler.rb +++ /dev/null @@ -1,23 +0,0 @@ -# frozen_string_literal: true - -# @abstract Subclass and override {.call} to implement a custom DeprecationWarnings handler class. -# @note For use with `ActiveSupport::Deprecation.behavior=`. -module DeprecationWarnings - class BaseHandler - class << self - # Subclasses must respond to `.call` to play nice with `ActiveSupport::Deprecation.behavior=`. - # https://github.com/rails/rails/blob/a4581b53aae93a8dd3205abae0630398cbce9204/activesupport/lib/active_support/deprecation/behaviors.rb#L70-L71 - # :reek:LongParameterList - def call(_message, _callstack, _deprecation_horizon, _gem_name) - fail NotImplementedError - end - - # Subclasses must respond to `.arity` to play nice with `ActiveSupport::Deprecation.behavior=`. - # Must return number of arguments accepted by `.call`. - # https://github.com/rails/rails/blob/a4581b53aae93a8dd3205abae0630398cbce9204/activesupport/lib/active_support/deprecation/behaviors.rb#L101 - def arity - method(:call).arity - end - end - end -end diff --git a/app/services/deprecation_warnings/production_handler.rb b/app/services/deprecation_warnings/production_handler.rb index 248922f74e2..7e4f70d0743 100644 --- a/app/services/deprecation_warnings/production_handler.rb +++ b/app/services/deprecation_warnings/production_handler.rb @@ -1,15 +1,15 @@ # frozen_string_literal: true -require_relative "base_handler" - # @note For use with `ActiveSupport::Deprecation.behavior=` or `Rails.application.config.active_support.deprecation=` module DeprecationWarnings - class ProductionHandler < BaseHandler + class ProductionHandler APP_NAME = "caseflow" SLACK_ALERT_TITLE = "Deprecation Warning - #{APP_NAME} (#{ENV['DEPLOY_ENV']})" SLACK_ALERT_CHANNEL = "#appeals-deprecation-alerts" class << self + # Adhere to `.call` signature expected by `ActiveSupport::Deprecation.behavior=`. + # https://github.com/rails/rails/blob/a4581b53aae93a8dd3205abae0630398cbce9204/activesupport/lib/active_support/deprecation/behaviors.rb#L70-L71 # :reek:LongParameterList def call(message, callstack, deprecation_horizon, gem_name) emit_warning_to_sentry(message, callstack, deprecation_horizon, gem_name) @@ -18,6 +18,13 @@ def call(message, callstack, deprecation_horizon, gem_name) Raven.capture_exception(error) end + # Must respond to `.arity` to play nice with `ActiveSupport::Deprecation.behavior=` + # and return number of arguments accepted by `.call`. + # https://github.com/rails/rails/blob/a4581b53aae93a8dd3205abae0630398cbce9204/activesupport/lib/active_support/deprecation/behaviors.rb#L101 + def arity + method(:call).arity + end + private # :reek:LongParameterList From 61a2715ab42fab453737fab4a9bb39687139d4f9 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 17 May 2024 14:46:40 -0400 Subject: [PATCH 37/69] =?UTF-8?q?=F0=9F=A9=B9=20Add=20forgotten=20disallow?= =?UTF-8?q?ed=20deprecation=20warning?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This deprecation warning was addressed by the following PR, but we forgot to add it to the list of disallowed deprecation warnings: https://github.com/department-of-veterans-affairs/caseflow/pull/21614 --- config/initializers/deprecation_warnings.rb | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/config/initializers/deprecation_warnings.rb b/config/initializers/deprecation_warnings.rb index f19ee0b8884..4d102c4e702 100644 --- a/config/initializers/deprecation_warnings.rb +++ b/config/initializers/deprecation_warnings.rb @@ -20,7 +20,8 @@ module DeprecationWarnings /render file: should be given the absolute path to a file/, /`ActiveRecord::Result#to_hash` has been renamed to `to_a`/, /Class level methods will no longer inherit scoping/, - /Controller-level `force_ssl` is deprecated and will be removed from Rails 6\.1/ + /Controller-level `force_ssl` is deprecated and will be removed from Rails 6\.1/, + /NOT conditions will no longer behave as NOR in Rails 6\.1/ ].freeze # Regular expressions for deprecation warnings that should raise an exception on detection From 9b9c23d2ffc75c719f2c7fd752a53dcbe3bff208 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 17 May 2024 15:06:24 -0400 Subject: [PATCH 38/69] =?UTF-8?q?=F0=9F=92=A1=20Update=20comment?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task `rake routes` has been replaced with `rails routes` --- app/controllers/users_controller.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/controllers/users_controller.rb b/app/controllers/users_controller.rb index 437047dd68b..63b65843dd6 100644 --- a/app/controllers/users_controller.rb +++ b/app/controllers/users_controller.rb @@ -88,7 +88,7 @@ def filter_by_organization end # Depending on the route and the requested resource, the requested user's id could be sent as :id or :user_id - # ex from rake routes: user GET /users/:id or user_task_pages GET /users/:user_id/task_pages + # ex from rails routes: user GET /users/:id or user_task_pages GET /users/:user_id/task_pages def id @id ||= params[:id] || params[:user_id] end From eeab8e2713fafcc8a5f16684a941c3944388e2fe Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 17 May 2024 15:12:30 -0400 Subject: [PATCH 39/69] =?UTF-8?q?=E2=9C=85=20Update=20test=20to=20account?= =?UTF-8?q?=20for=20change=20to=20`ActionDispatch::Response#content=5Ftype?= =?UTF-8?q?`=C2=A0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ActionDispatch::Response#content_type` now returns the full Content-Type header --- spec/controllers/decision_reviews_controller_spec.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/controllers/decision_reviews_controller_spec.rb b/spec/controllers/decision_reviews_controller_spec.rb index e16465f0362..4280faa4987 100644 --- a/spec/controllers/decision_reviews_controller_spec.rb +++ b/spec/controllers/decision_reviews_controller_spec.rb @@ -942,7 +942,7 @@ params = { business_line_slug: non_comp_org.url }.merge(generate_report_filters.except("report_type")) get :generate_report, format: :csv, params: params expect(response).to have_http_status(:bad_request) - expect(response.content_type).to eq("application/json") + expect(response.content_type).to eq("application/json; charset=utf-8") json_response = JSON.parse(response.body) expect(json_response["error"]).to eq("param is missing or the value is empty: reportType") end From 413d84b617369c24dc33483a815cfa9c253b86bc Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 17 May 2024 16:07:52 -0400 Subject: [PATCH 40/69] =?UTF-8?q?=F0=9F=9A=A8=20Exclude=20'config.ru'=20fr?= =?UTF-8?q?om=20Rubocop=20cops?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .rubocop.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.rubocop.yml b/.rubocop.yml index 8773441a13f..68a4a88ff03 100644 --- a/.rubocop.yml +++ b/.rubocop.yml @@ -6,6 +6,7 @@ AllCops: Exclude: - 'bin/**/*' - 'db/**/*' + - 'config.ru' - 'config/**/*' - 'script/**/*' - 'vendor/**/*' From d4cf2333b097a25289009e6177bda2854ce1558a Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 30 Apr 2024 10:59:47 -0400 Subject: [PATCH 41/69] =?UTF-8?q?=F0=9F=9A=9A=20Move=20'db/etl/migrate'=20?= =?UTF-8?q?to=20'db/etl=5Fmigrate'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .codeclimate.yml | 2 ++ .reek.yml | 2 +- Dangerfile | 2 +- .../20191111164808_create_etl_appeals.rb | 0 .../migrate => etl_migrate}/20191119204827_create_etl_users.rb | 0 .../migrate => etl_migrate}/20191120184256_create_etl_tasks.rb | 0 .../migrate => etl_migrate}/20191120213930_create_etl_people.rb | 0 .../20191120213941_create_etl_organizations.rb | 0 .../20191126164353_remove_users_vacols_not_null.rb | 0 .../20191126203511_add_appeal_timestamps.rb | 0 .../20191209162326_add_appeal_status_sort.rb | 0 .../migrate => etl_migrate}/20191212221851_org_status_change.rb | 0 .../20191212222617_org_status_default.rb | 0 .../migrate => etl_migrate}/20191213194037_add_task_id_index.rb | 0 .../20191217193046_create_attorney_case_reviews.rb | 0 .../20191217202614_create_decision_issues.rb | 0 .../20191218214351_add_more_cached_user_attributes.rb | 0 .../20191223213954_add_email_address_to_person.rb | 0 .../20200114201144_add_aod_due_to_age.rb | 0 .../20200116215235_add_claimant_dob_to_appeals.rb | 0 .../20200121221718_add_appeals_claimant_indexes.rb | 0 .../20200212205344_add_etl_build_tables.rb | 0 .../20200303223155_add_etl_users_index.rb | 0 .../20200429152715_create_vha_decision_reviews.rb | 0 .../migrate => etl_migrate}/20200429164923_create_hearings.rb | 0 .../migrate => etl_migrate}/20200430152234_add_hearing_type.rb | 0 .../20200501211351_change_hearing_day_id_null.rb | 0 .../migrate => etl_migrate}/20200504133723_change_css_limit.rb | 0 .../20200504204154_remove_null_hearings.rb | 0 .../20200508141923_drop_legacy_hearing_scheduled_for.rb | 0 .../20200517021335_create_etl_remand_reason.rb | 0 ...519213755_change_vha_decision_reviews_to_decision_reviews.rb | 0 .../20200519214159_change_decision_reviews_comments.rb | 0 .../20200519214329_change_decision_reviews_indices.rb | 0 .../20200519221508_add_decision_issue_benefit_index.rb | 0 .../migrate => etl_migrate}/20200520220924_add_person_ssn.rb | 0 ...515_add_subject_text_and_percent_number_to_decision_issue.rb | 0 ...034_add_accepts_priority_pushed_cases_to_etl_organization.rb | 0 ...3_add_accepts_priority_pushed_cases_index_on_organization.rb | 0 .../20210729124530_remove_etl_decision_issues_index.rb | 0 ...9130353_add_etl_decision_issues_rating_issue_ref_id_index.rb | 0 .../20210810115309_create_etl_judge_case_review.rb | 0 .../20210810115459_create_etl_decision_document.rb | 0 .../20210818172716_add_ama_only_to_organization.rb | 0 ...07125705_remove_non_null_constraint_on_appeal_docket_type.rb | 0 ...0143457_remove_non_null_constraint_on_appeal_receipt_date.rb | 0 ...200527_add_exclude_appeals_from_affinity_to_organizations.rb | 0 47 files changed, 4 insertions(+), 2 deletions(-) rename db/{etl/migrate => etl_migrate}/20191111164808_create_etl_appeals.rb (100%) rename db/{etl/migrate => etl_migrate}/20191119204827_create_etl_users.rb (100%) rename db/{etl/migrate => etl_migrate}/20191120184256_create_etl_tasks.rb (100%) rename db/{etl/migrate => etl_migrate}/20191120213930_create_etl_people.rb (100%) rename db/{etl/migrate => etl_migrate}/20191120213941_create_etl_organizations.rb (100%) rename db/{etl/migrate => etl_migrate}/20191126164353_remove_users_vacols_not_null.rb (100%) rename db/{etl/migrate => etl_migrate}/20191126203511_add_appeal_timestamps.rb (100%) rename db/{etl/migrate => etl_migrate}/20191209162326_add_appeal_status_sort.rb (100%) rename db/{etl/migrate => etl_migrate}/20191212221851_org_status_change.rb (100%) rename db/{etl/migrate => etl_migrate}/20191212222617_org_status_default.rb (100%) rename db/{etl/migrate => etl_migrate}/20191213194037_add_task_id_index.rb (100%) rename db/{etl/migrate => etl_migrate}/20191217193046_create_attorney_case_reviews.rb (100%) rename db/{etl/migrate => etl_migrate}/20191217202614_create_decision_issues.rb (100%) rename db/{etl/migrate => etl_migrate}/20191218214351_add_more_cached_user_attributes.rb (100%) rename db/{etl/migrate => etl_migrate}/20191223213954_add_email_address_to_person.rb (100%) rename db/{etl/migrate => etl_migrate}/20200114201144_add_aod_due_to_age.rb (100%) rename db/{etl/migrate => etl_migrate}/20200116215235_add_claimant_dob_to_appeals.rb (100%) rename db/{etl/migrate => etl_migrate}/20200121221718_add_appeals_claimant_indexes.rb (100%) rename db/{etl/migrate => etl_migrate}/20200212205344_add_etl_build_tables.rb (100%) rename db/{etl/migrate => etl_migrate}/20200303223155_add_etl_users_index.rb (100%) rename db/{etl/migrate => etl_migrate}/20200429152715_create_vha_decision_reviews.rb (100%) rename db/{etl/migrate => etl_migrate}/20200429164923_create_hearings.rb (100%) rename db/{etl/migrate => etl_migrate}/20200430152234_add_hearing_type.rb (100%) rename db/{etl/migrate => etl_migrate}/20200501211351_change_hearing_day_id_null.rb (100%) rename db/{etl/migrate => etl_migrate}/20200504133723_change_css_limit.rb (100%) rename db/{etl/migrate => etl_migrate}/20200504204154_remove_null_hearings.rb (100%) rename db/{etl/migrate => etl_migrate}/20200508141923_drop_legacy_hearing_scheduled_for.rb (100%) rename db/{etl/migrate => etl_migrate}/20200517021335_create_etl_remand_reason.rb (100%) rename db/{etl/migrate => etl_migrate}/20200519213755_change_vha_decision_reviews_to_decision_reviews.rb (100%) rename db/{etl/migrate => etl_migrate}/20200519214159_change_decision_reviews_comments.rb (100%) rename db/{etl/migrate => etl_migrate}/20200519214329_change_decision_reviews_indices.rb (100%) rename db/{etl/migrate => etl_migrate}/20200519221508_add_decision_issue_benefit_index.rb (100%) rename db/{etl/migrate => etl_migrate}/20200520220924_add_person_ssn.rb (100%) rename db/{etl/migrate => etl_migrate}/20200526054515_add_subject_text_and_percent_number_to_decision_issue.rb (100%) rename db/{etl/migrate => etl_migrate}/20200715110034_add_accepts_priority_pushed_cases_to_etl_organization.rb (100%) rename db/{etl/migrate => etl_migrate}/20200715110603_add_accepts_priority_pushed_cases_index_on_organization.rb (100%) rename db/{etl/migrate => etl_migrate}/20210729124530_remove_etl_decision_issues_index.rb (100%) rename db/{etl/migrate => etl_migrate}/20210729130353_add_etl_decision_issues_rating_issue_ref_id_index.rb (100%) rename db/{etl/migrate => etl_migrate}/20210810115309_create_etl_judge_case_review.rb (100%) rename db/{etl/migrate => etl_migrate}/20210810115459_create_etl_decision_document.rb (100%) rename db/{etl/migrate => etl_migrate}/20210818172716_add_ama_only_to_organization.rb (100%) rename db/{etl/migrate => etl_migrate}/20220107125705_remove_non_null_constraint_on_appeal_docket_type.rb (100%) rename db/{etl/migrate => etl_migrate}/20220110143457_remove_non_null_constraint_on_appeal_receipt_date.rb (100%) rename db/{etl/migrate => etl_migrate}/20240129200527_add_exclude_appeals_from_affinity_to_organizations.rb (100%) diff --git a/.codeclimate.yml b/.codeclimate.yml index ff4076a1126..07f4b7837d2 100644 --- a/.codeclimate.yml +++ b/.codeclimate.yml @@ -52,6 +52,7 @@ plugins: mass_threshold: 81 exclude_patterns: - 'db/migrate/*' + - 'db/etl_migrate/*' - 'app/controllers/idt/api/v2/appeals_controller.rb' - 'spec/controllers/idt/api/v2/appeals_controller_spec.rb' - 'spec/controllers/idt/api/appeals_controller_spec.rb' @@ -79,6 +80,7 @@ plugins: enabled: false exclude_patterns: - 'db/migrate/*' + - 'db/etl_migrate/*' - 'app/jobs/dispatch_email_job.rb' - 'app/mailers/dispatch_mailer.rb' rubocop: diff --git a/.reek.yml b/.reek.yml index 727b53e049b..cd12f9cc576 100644 --- a/.reek.yml +++ b/.reek.yml @@ -291,5 +291,5 @@ directories: # Directories and files below will not be scanned at all exclude_paths: - db/migrate - - db/etl/migrate + - db/etl_migrate - spec diff --git a/Dangerfile b/Dangerfile index 21b9e8bb4c3..d5efa6b3e1c 100644 --- a/Dangerfile +++ b/Dangerfile @@ -40,7 +40,7 @@ if git.modified_files.grep(/db\/etl\/schema.rb/).any? end new_db_migrations = git.modified_files.grep(/db\/migrate\//).any? -new_etl_migrations = git.modified_files.grep(/db\/etl\/migrate\//).any? +new_etl_migrations = git.modified_files.grep(/db\/etl_migrate\//).any? # migration without migrating if new_db_migrations && git.modified_files.grep(/db\/schema.rb/).none? diff --git a/db/etl/migrate/20191111164808_create_etl_appeals.rb b/db/etl_migrate/20191111164808_create_etl_appeals.rb similarity index 100% rename from db/etl/migrate/20191111164808_create_etl_appeals.rb rename to db/etl_migrate/20191111164808_create_etl_appeals.rb diff --git a/db/etl/migrate/20191119204827_create_etl_users.rb b/db/etl_migrate/20191119204827_create_etl_users.rb similarity index 100% rename from db/etl/migrate/20191119204827_create_etl_users.rb rename to db/etl_migrate/20191119204827_create_etl_users.rb diff --git a/db/etl/migrate/20191120184256_create_etl_tasks.rb b/db/etl_migrate/20191120184256_create_etl_tasks.rb similarity index 100% rename from db/etl/migrate/20191120184256_create_etl_tasks.rb rename to db/etl_migrate/20191120184256_create_etl_tasks.rb diff --git a/db/etl/migrate/20191120213930_create_etl_people.rb b/db/etl_migrate/20191120213930_create_etl_people.rb similarity index 100% rename from db/etl/migrate/20191120213930_create_etl_people.rb rename to db/etl_migrate/20191120213930_create_etl_people.rb diff --git a/db/etl/migrate/20191120213941_create_etl_organizations.rb b/db/etl_migrate/20191120213941_create_etl_organizations.rb similarity index 100% rename from db/etl/migrate/20191120213941_create_etl_organizations.rb rename to db/etl_migrate/20191120213941_create_etl_organizations.rb diff --git a/db/etl/migrate/20191126164353_remove_users_vacols_not_null.rb b/db/etl_migrate/20191126164353_remove_users_vacols_not_null.rb similarity index 100% rename from db/etl/migrate/20191126164353_remove_users_vacols_not_null.rb rename to db/etl_migrate/20191126164353_remove_users_vacols_not_null.rb diff --git a/db/etl/migrate/20191126203511_add_appeal_timestamps.rb b/db/etl_migrate/20191126203511_add_appeal_timestamps.rb similarity index 100% rename from db/etl/migrate/20191126203511_add_appeal_timestamps.rb rename to db/etl_migrate/20191126203511_add_appeal_timestamps.rb diff --git a/db/etl/migrate/20191209162326_add_appeal_status_sort.rb b/db/etl_migrate/20191209162326_add_appeal_status_sort.rb similarity index 100% rename from db/etl/migrate/20191209162326_add_appeal_status_sort.rb rename to db/etl_migrate/20191209162326_add_appeal_status_sort.rb diff --git a/db/etl/migrate/20191212221851_org_status_change.rb b/db/etl_migrate/20191212221851_org_status_change.rb similarity index 100% rename from db/etl/migrate/20191212221851_org_status_change.rb rename to db/etl_migrate/20191212221851_org_status_change.rb diff --git a/db/etl/migrate/20191212222617_org_status_default.rb b/db/etl_migrate/20191212222617_org_status_default.rb similarity index 100% rename from db/etl/migrate/20191212222617_org_status_default.rb rename to db/etl_migrate/20191212222617_org_status_default.rb diff --git a/db/etl/migrate/20191213194037_add_task_id_index.rb b/db/etl_migrate/20191213194037_add_task_id_index.rb similarity index 100% rename from db/etl/migrate/20191213194037_add_task_id_index.rb rename to db/etl_migrate/20191213194037_add_task_id_index.rb diff --git a/db/etl/migrate/20191217193046_create_attorney_case_reviews.rb b/db/etl_migrate/20191217193046_create_attorney_case_reviews.rb similarity index 100% rename from db/etl/migrate/20191217193046_create_attorney_case_reviews.rb rename to db/etl_migrate/20191217193046_create_attorney_case_reviews.rb diff --git a/db/etl/migrate/20191217202614_create_decision_issues.rb b/db/etl_migrate/20191217202614_create_decision_issues.rb similarity index 100% rename from db/etl/migrate/20191217202614_create_decision_issues.rb rename to db/etl_migrate/20191217202614_create_decision_issues.rb diff --git a/db/etl/migrate/20191218214351_add_more_cached_user_attributes.rb b/db/etl_migrate/20191218214351_add_more_cached_user_attributes.rb similarity index 100% rename from db/etl/migrate/20191218214351_add_more_cached_user_attributes.rb rename to db/etl_migrate/20191218214351_add_more_cached_user_attributes.rb diff --git a/db/etl/migrate/20191223213954_add_email_address_to_person.rb b/db/etl_migrate/20191223213954_add_email_address_to_person.rb similarity index 100% rename from db/etl/migrate/20191223213954_add_email_address_to_person.rb rename to db/etl_migrate/20191223213954_add_email_address_to_person.rb diff --git a/db/etl/migrate/20200114201144_add_aod_due_to_age.rb b/db/etl_migrate/20200114201144_add_aod_due_to_age.rb similarity index 100% rename from db/etl/migrate/20200114201144_add_aod_due_to_age.rb rename to db/etl_migrate/20200114201144_add_aod_due_to_age.rb diff --git a/db/etl/migrate/20200116215235_add_claimant_dob_to_appeals.rb b/db/etl_migrate/20200116215235_add_claimant_dob_to_appeals.rb similarity index 100% rename from db/etl/migrate/20200116215235_add_claimant_dob_to_appeals.rb rename to db/etl_migrate/20200116215235_add_claimant_dob_to_appeals.rb diff --git a/db/etl/migrate/20200121221718_add_appeals_claimant_indexes.rb b/db/etl_migrate/20200121221718_add_appeals_claimant_indexes.rb similarity index 100% rename from db/etl/migrate/20200121221718_add_appeals_claimant_indexes.rb rename to db/etl_migrate/20200121221718_add_appeals_claimant_indexes.rb diff --git a/db/etl/migrate/20200212205344_add_etl_build_tables.rb b/db/etl_migrate/20200212205344_add_etl_build_tables.rb similarity index 100% rename from db/etl/migrate/20200212205344_add_etl_build_tables.rb rename to db/etl_migrate/20200212205344_add_etl_build_tables.rb diff --git a/db/etl/migrate/20200303223155_add_etl_users_index.rb b/db/etl_migrate/20200303223155_add_etl_users_index.rb similarity index 100% rename from db/etl/migrate/20200303223155_add_etl_users_index.rb rename to db/etl_migrate/20200303223155_add_etl_users_index.rb diff --git a/db/etl/migrate/20200429152715_create_vha_decision_reviews.rb b/db/etl_migrate/20200429152715_create_vha_decision_reviews.rb similarity index 100% rename from db/etl/migrate/20200429152715_create_vha_decision_reviews.rb rename to db/etl_migrate/20200429152715_create_vha_decision_reviews.rb diff --git a/db/etl/migrate/20200429164923_create_hearings.rb b/db/etl_migrate/20200429164923_create_hearings.rb similarity index 100% rename from db/etl/migrate/20200429164923_create_hearings.rb rename to db/etl_migrate/20200429164923_create_hearings.rb diff --git a/db/etl/migrate/20200430152234_add_hearing_type.rb b/db/etl_migrate/20200430152234_add_hearing_type.rb similarity index 100% rename from db/etl/migrate/20200430152234_add_hearing_type.rb rename to db/etl_migrate/20200430152234_add_hearing_type.rb diff --git a/db/etl/migrate/20200501211351_change_hearing_day_id_null.rb b/db/etl_migrate/20200501211351_change_hearing_day_id_null.rb similarity index 100% rename from db/etl/migrate/20200501211351_change_hearing_day_id_null.rb rename to db/etl_migrate/20200501211351_change_hearing_day_id_null.rb diff --git a/db/etl/migrate/20200504133723_change_css_limit.rb b/db/etl_migrate/20200504133723_change_css_limit.rb similarity index 100% rename from db/etl/migrate/20200504133723_change_css_limit.rb rename to db/etl_migrate/20200504133723_change_css_limit.rb diff --git a/db/etl/migrate/20200504204154_remove_null_hearings.rb b/db/etl_migrate/20200504204154_remove_null_hearings.rb similarity index 100% rename from db/etl/migrate/20200504204154_remove_null_hearings.rb rename to db/etl_migrate/20200504204154_remove_null_hearings.rb diff --git a/db/etl/migrate/20200508141923_drop_legacy_hearing_scheduled_for.rb b/db/etl_migrate/20200508141923_drop_legacy_hearing_scheduled_for.rb similarity index 100% rename from db/etl/migrate/20200508141923_drop_legacy_hearing_scheduled_for.rb rename to db/etl_migrate/20200508141923_drop_legacy_hearing_scheduled_for.rb diff --git a/db/etl/migrate/20200517021335_create_etl_remand_reason.rb b/db/etl_migrate/20200517021335_create_etl_remand_reason.rb similarity index 100% rename from db/etl/migrate/20200517021335_create_etl_remand_reason.rb rename to db/etl_migrate/20200517021335_create_etl_remand_reason.rb diff --git a/db/etl/migrate/20200519213755_change_vha_decision_reviews_to_decision_reviews.rb b/db/etl_migrate/20200519213755_change_vha_decision_reviews_to_decision_reviews.rb similarity index 100% rename from db/etl/migrate/20200519213755_change_vha_decision_reviews_to_decision_reviews.rb rename to db/etl_migrate/20200519213755_change_vha_decision_reviews_to_decision_reviews.rb diff --git a/db/etl/migrate/20200519214159_change_decision_reviews_comments.rb b/db/etl_migrate/20200519214159_change_decision_reviews_comments.rb similarity index 100% rename from db/etl/migrate/20200519214159_change_decision_reviews_comments.rb rename to db/etl_migrate/20200519214159_change_decision_reviews_comments.rb diff --git a/db/etl/migrate/20200519214329_change_decision_reviews_indices.rb b/db/etl_migrate/20200519214329_change_decision_reviews_indices.rb similarity index 100% rename from db/etl/migrate/20200519214329_change_decision_reviews_indices.rb rename to db/etl_migrate/20200519214329_change_decision_reviews_indices.rb diff --git a/db/etl/migrate/20200519221508_add_decision_issue_benefit_index.rb b/db/etl_migrate/20200519221508_add_decision_issue_benefit_index.rb similarity index 100% rename from db/etl/migrate/20200519221508_add_decision_issue_benefit_index.rb rename to db/etl_migrate/20200519221508_add_decision_issue_benefit_index.rb diff --git a/db/etl/migrate/20200520220924_add_person_ssn.rb b/db/etl_migrate/20200520220924_add_person_ssn.rb similarity index 100% rename from db/etl/migrate/20200520220924_add_person_ssn.rb rename to db/etl_migrate/20200520220924_add_person_ssn.rb diff --git a/db/etl/migrate/20200526054515_add_subject_text_and_percent_number_to_decision_issue.rb b/db/etl_migrate/20200526054515_add_subject_text_and_percent_number_to_decision_issue.rb similarity index 100% rename from db/etl/migrate/20200526054515_add_subject_text_and_percent_number_to_decision_issue.rb rename to db/etl_migrate/20200526054515_add_subject_text_and_percent_number_to_decision_issue.rb diff --git a/db/etl/migrate/20200715110034_add_accepts_priority_pushed_cases_to_etl_organization.rb b/db/etl_migrate/20200715110034_add_accepts_priority_pushed_cases_to_etl_organization.rb similarity index 100% rename from db/etl/migrate/20200715110034_add_accepts_priority_pushed_cases_to_etl_organization.rb rename to db/etl_migrate/20200715110034_add_accepts_priority_pushed_cases_to_etl_organization.rb diff --git a/db/etl/migrate/20200715110603_add_accepts_priority_pushed_cases_index_on_organization.rb b/db/etl_migrate/20200715110603_add_accepts_priority_pushed_cases_index_on_organization.rb similarity index 100% rename from db/etl/migrate/20200715110603_add_accepts_priority_pushed_cases_index_on_organization.rb rename to db/etl_migrate/20200715110603_add_accepts_priority_pushed_cases_index_on_organization.rb diff --git a/db/etl/migrate/20210729124530_remove_etl_decision_issues_index.rb b/db/etl_migrate/20210729124530_remove_etl_decision_issues_index.rb similarity index 100% rename from db/etl/migrate/20210729124530_remove_etl_decision_issues_index.rb rename to db/etl_migrate/20210729124530_remove_etl_decision_issues_index.rb diff --git a/db/etl/migrate/20210729130353_add_etl_decision_issues_rating_issue_ref_id_index.rb b/db/etl_migrate/20210729130353_add_etl_decision_issues_rating_issue_ref_id_index.rb similarity index 100% rename from db/etl/migrate/20210729130353_add_etl_decision_issues_rating_issue_ref_id_index.rb rename to db/etl_migrate/20210729130353_add_etl_decision_issues_rating_issue_ref_id_index.rb diff --git a/db/etl/migrate/20210810115309_create_etl_judge_case_review.rb b/db/etl_migrate/20210810115309_create_etl_judge_case_review.rb similarity index 100% rename from db/etl/migrate/20210810115309_create_etl_judge_case_review.rb rename to db/etl_migrate/20210810115309_create_etl_judge_case_review.rb diff --git a/db/etl/migrate/20210810115459_create_etl_decision_document.rb b/db/etl_migrate/20210810115459_create_etl_decision_document.rb similarity index 100% rename from db/etl/migrate/20210810115459_create_etl_decision_document.rb rename to db/etl_migrate/20210810115459_create_etl_decision_document.rb diff --git a/db/etl/migrate/20210818172716_add_ama_only_to_organization.rb b/db/etl_migrate/20210818172716_add_ama_only_to_organization.rb similarity index 100% rename from db/etl/migrate/20210818172716_add_ama_only_to_organization.rb rename to db/etl_migrate/20210818172716_add_ama_only_to_organization.rb diff --git a/db/etl/migrate/20220107125705_remove_non_null_constraint_on_appeal_docket_type.rb b/db/etl_migrate/20220107125705_remove_non_null_constraint_on_appeal_docket_type.rb similarity index 100% rename from db/etl/migrate/20220107125705_remove_non_null_constraint_on_appeal_docket_type.rb rename to db/etl_migrate/20220107125705_remove_non_null_constraint_on_appeal_docket_type.rb diff --git a/db/etl/migrate/20220110143457_remove_non_null_constraint_on_appeal_receipt_date.rb b/db/etl_migrate/20220110143457_remove_non_null_constraint_on_appeal_receipt_date.rb similarity index 100% rename from db/etl/migrate/20220110143457_remove_non_null_constraint_on_appeal_receipt_date.rb rename to db/etl_migrate/20220110143457_remove_non_null_constraint_on_appeal_receipt_date.rb diff --git a/db/etl/migrate/20240129200527_add_exclude_appeals_from_affinity_to_organizations.rb b/db/etl_migrate/20240129200527_add_exclude_appeals_from_affinity_to_organizations.rb similarity index 100% rename from db/etl/migrate/20240129200527_add_exclude_appeals_from_affinity_to_organizations.rb rename to db/etl_migrate/20240129200527_add_exclude_appeals_from_affinity_to_organizations.rb From 0ecbb0475083f944c0a7805ca23a1dbdf7163359 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 30 Apr 2024 11:31:21 -0400 Subject: [PATCH 42/69] =?UTF-8?q?=F0=9F=9A=9A=20Move=20'db/etl/schema.rb'?= =?UTF-8?q?=20to=20'db/etl=5Fschema.rb'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .codeclimate.yml | 1 + Dangerfile | 4 ++-- db/{etl/schema.rb => etl_schema.rb} | 0 3 files changed, 3 insertions(+), 2 deletions(-) rename db/{etl/schema.rb => etl_schema.rb} (100%) diff --git a/.codeclimate.yml b/.codeclimate.yml index 07f4b7837d2..f16d7d860f0 100644 --- a/.codeclimate.yml +++ b/.codeclimate.yml @@ -117,6 +117,7 @@ plugins: exclude_patterns: - 'db/schema.rb' + - 'db/etl_schema.rb' - 'db/seeds.rb' - 'db/scripts/*' - 'node_modules/**/*' diff --git a/Dangerfile b/Dangerfile index d5efa6b3e1c..1fb8280ddae 100644 --- a/Dangerfile +++ b/Dangerfile @@ -35,7 +35,7 @@ if git.modified_files.grep(/db\/schema.rb/).any? warn("This PR changes the schema. Please use the PR template checklist.") end -if git.modified_files.grep(/db\/etl\/schema.rb/).any? +if git.modified_files.grep(/db\/etl_schema.rb/).any? warn("This PR changes the etl schema. Please use the PR template checklist.") end @@ -47,7 +47,7 @@ if new_db_migrations && git.modified_files.grep(/db\/schema.rb/).none? warn("This PR contains db migrations, but the schema.rb is not modified. Did you forget to run 'make migrate'?") end -if new_etl_migrations && git.modified_files.grep(/db\/etl\/schema.rb/).none? +if new_etl_migrations && git.modified_files.grep(/db\/etl_schema.rb/).none? warn("This PR contains etl migrations, but the etl schema.rb is not modified. Did you forget to run 'make migrate'?") end diff --git a/db/etl/schema.rb b/db/etl_schema.rb similarity index 100% rename from db/etl/schema.rb rename to db/etl_schema.rb From 6aed2403d3272b3dc554ded16395c744e8ff3e84 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 30 Apr 2024 13:57:49 -0400 Subject: [PATCH 43/69] =?UTF-8?q?=E2=99=BB=EF=B8=8F=20Arrange=20'database.?= =?UTF-8?q?yml'=20configs=20by=20environment?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Group DB configs by environment in anticipation of reformatting for Rails 6+ multi-DB configuration. --- config/database.yml | 110 ++++++++++++++++++++++---------------------- 1 file changed, 55 insertions(+), 55 deletions(-) diff --git a/config/database.yml b/config/database.yml index 6a2e9dcf41e..6c407c044c8 100644 --- a/config/database.yml +++ b/config/database.yml @@ -18,26 +18,6 @@ default: &default password: <%= ENV['POSTGRES_PASSWORD'] %> <% end %> -demo: - <<: *default - database: caseflow_certification_demo - -ssh_forwarding: - <<: *default - database: caseflow_certification_ssh_forwarding - -development: - <<: *default - database: caseflow_certification_development - -staging: - <<: *default - database: caseflow_certification_staging - -production: - <<: *default - url: <%= ENV["POSTGRES_URL"] %> - # NOTE Rails 6 has native support for multiple databases, # and this configuration should change to nest "primary" and "etl" and "vacols" # under each env name, per @@ -52,18 +32,14 @@ etl_default: &etl_default password: <%= ENV["ETL_DB_PASSWORD"] || ENV["POSTGRES_PASSWORD"] %> <% end %> +development: + <<: *default + database: caseflow_certification_development + etl_development: <<: *etl_default database: caseflow_etl_development -etl_test: - <<: *etl_default - database: caseflow_etl_test<%= ENV["TEST_SUBCATEGORY"] %> - -etl_production: - <<: *etl_default - url: <%= ENV["ETL_DB_URL"] %> - development_vacols: adapter: oracle_enhanced username: VACOLS_DEV @@ -74,12 +50,51 @@ development_vacols: <% end %> database: <%= ENV['DEMO_DB'] || "BVAP" %> +# Warning: The database defined as "test" will be erased and +# re-generated from your development database when you run "rake". +# Do not set this db to the same as development or production. +test: + <<: *default + username: <%= ENV['POSTGRES_USER'] || `whoami` %> + database: caseflow_certification_test<%= ENV['TEST_SUBCATEGORY'] %> + <% if ENV['POSTGRES_HOST'] %> + host: <%= ENV['POSTGRES_HOST'] %> + <% end %> + <% if ENV['POSTGRES_PASSWORD'] %> + password: <%= ENV['POSTGRES_PASSWORD'] %> + <% end %> + +etl_test: + <<: *etl_default + database: caseflow_etl_test<%= ENV["TEST_SUBCATEGORY"] %> + +# Increase the receive timeout value due to ORA-12609: TNS: Receive timeout occurred +# Solution: https://www.fatihacar.com/blog/category/databases/oracle/oracle-errors-and-solutions/page/13/ +# Oracle timeout configuration: https://github.com/rsim/oracle-enhanced#timeouts +test_vacols: + adapter: oracle_enhanced + username: VACOLS_TEST + password: VACOLS_TEST + database: "(DESCRIPTION= + (ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=<%= ENV['TEST_VACOLS_HOST']%>)(PORT=1521))) + (RECV_TIMEOUT=120)(SEND_TIMEOUT=5) + (CONNECT_DATA=(SID=BVAP)) + )" + +demo: + <<: *default + database: caseflow_certification_demo + demo_vacols: adapter: sqlite3 pool: 5 timeout: 5000 database: db/demo-vacols.sqlite3 +ssh_forwarding: + <<: *default + database: caseflow_certification_ssh_forwarding + ssh_forwarding_vacols: adapter: oracle_enhanced host: localhost @@ -88,6 +103,10 @@ ssh_forwarding_vacols: password: <%= ENV["VACOLS_PASSWORD"] %> database: BVAP +staging: + <<: *default + database: caseflow_certification_staging + staging_vacols: adapter: oracle_enhanced username: <%= ENV["VACOLS_USERNAME"] %> @@ -96,6 +115,14 @@ staging_vacols: port: 1526 database: BVAP.VACO.VA.GOV +production: + <<: *default + url: <%= ENV["POSTGRES_URL"] %> + +etl_production: + <<: *etl_default + url: <%= ENV["ETL_DB_URL"] %> + production_vacols: adapter: oracle_enhanced username: <%= ENV["VACOLS_USERNAME"] %> @@ -104,30 +131,3 @@ production_vacols: port: <%= ENV["VACOLS_PORT"] %> database: <%= ENV["VACOLS_DATABASE"] %> pool: <%= ENV["DB_CONN_POOL_MAX_SIZE"] || 5 %> - -# Warning: The database defined as "test" will be erased and -# re-generated from your development database when you run "rake". -# Do not set this db to the same as development or production. -test: - <<: *default - username: <%= ENV['POSTGRES_USER'] || `whoami` %> - database: caseflow_certification_test<%= ENV['TEST_SUBCATEGORY'] %> - <% if ENV['POSTGRES_HOST'] %> - host: <%= ENV['POSTGRES_HOST'] %> - <% end %> - <% if ENV['POSTGRES_PASSWORD'] %> - password: <%= ENV['POSTGRES_PASSWORD'] %> - <% end %> - -# Increase the receive timeout value due to ORA-12609: TNS: Receive timeout occurred -# Solution: https://www.fatihacar.com/blog/category/databases/oracle/oracle-errors-and-solutions/page/13/ -# Oracle timeout configuration: https://github.com/rsim/oracle-enhanced#timeouts -test_vacols: - adapter: oracle_enhanced - username: VACOLS_TEST - password: VACOLS_TEST - database: "(DESCRIPTION= - (ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=<%= ENV['TEST_VACOLS_HOST']%>)(PORT=1521))) - (RECV_TIMEOUT=120)(SEND_TIMEOUT=5) - (CONNECT_DATA=(SID=BVAP)) - )" From c843692eb63c90657c8455edd73d11a1c47d3c73 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 9 May 2024 00:59:50 -0400 Subject: [PATCH 44/69] =?UTF-8?q?=F0=9F=94=A7=20Reformat=20'database.yml'?= =?UTF-8?q?=20to=20Rails=206+=20multi-DB=20conventions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- app/models/etl/record.rb | 2 +- app/models/vacols/record.rb | 2 +- config/database.yml | 167 +++++++++++++-------------- config/initializers/warmup_vacols.rb | 2 +- 4 files changed, 85 insertions(+), 88 deletions(-) diff --git a/app/models/etl/record.rb b/app/models/etl/record.rb index 79819d27973..a71604ee94c 100644 --- a/app/models/etl/record.rb +++ b/app/models/etl/record.rb @@ -16,7 +16,7 @@ class ETL::Record < ApplicationRecord self.abstract_class = true - establish_connection :"etl_#{Rails.env}" + establish_connection :etl class << self def sync_with_original(original) diff --git a/app/models/vacols/record.rb b/app/models/vacols/record.rb index cf6070a6c70..145c5b0466d 100644 --- a/app/models/vacols/record.rb +++ b/app/models/vacols/record.rb @@ -3,7 +3,7 @@ class VACOLS::Record < ApplicationRecord self.abstract_class = true - establish_connection "#{Rails.env}_vacols".to_sym + establish_connection :vacols ActiveSupport.run_load_hooks(:active_record_vacols, VACOLS::Record) # This method calculates the appropriate date & timezone diff --git a/config/database.yml b/config/database.yml index 6c407c044c8..3a536e28e6b 100644 --- a/config/database.yml +++ b/config/database.yml @@ -33,101 +33,98 @@ etl_default: &etl_default <% end %> development: - <<: *default - database: caseflow_certification_development - -etl_development: - <<: *etl_default - database: caseflow_etl_development - -development_vacols: - adapter: oracle_enhanced - username: VACOLS_DEV - password: VACOLS_DEV - <% if ! ENV['DOCKERIZED'] %> - host: localhost - port: 1521 - <% end %> - database: <%= ENV['DEMO_DB'] || "BVAP" %> + primary: + <<: *default + database: caseflow_certification_development + etl: + <<: *etl_default + database: caseflow_etl_development + vacols: + adapter: oracle_enhanced + username: VACOLS_DEV + password: VACOLS_DEV + <% if ! ENV['DOCKERIZED'] %> + host: localhost + port: 1521 + <% end %> + database: <%= ENV['DEMO_DB'] || "BVAP" %> # Warning: The database defined as "test" will be erased and # re-generated from your development database when you run "rake". # Do not set this db to the same as development or production. test: - <<: *default - username: <%= ENV['POSTGRES_USER'] || `whoami` %> - database: caseflow_certification_test<%= ENV['TEST_SUBCATEGORY'] %> - <% if ENV['POSTGRES_HOST'] %> - host: <%= ENV['POSTGRES_HOST'] %> - <% end %> - <% if ENV['POSTGRES_PASSWORD'] %> - password: <%= ENV['POSTGRES_PASSWORD'] %> - <% end %> - -etl_test: - <<: *etl_default - database: caseflow_etl_test<%= ENV["TEST_SUBCATEGORY"] %> - -# Increase the receive timeout value due to ORA-12609: TNS: Receive timeout occurred -# Solution: https://www.fatihacar.com/blog/category/databases/oracle/oracle-errors-and-solutions/page/13/ -# Oracle timeout configuration: https://github.com/rsim/oracle-enhanced#timeouts -test_vacols: - adapter: oracle_enhanced - username: VACOLS_TEST - password: VACOLS_TEST - database: "(DESCRIPTION= - (ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=<%= ENV['TEST_VACOLS_HOST']%>)(PORT=1521))) - (RECV_TIMEOUT=120)(SEND_TIMEOUT=5) - (CONNECT_DATA=(SID=BVAP)) - )" + primary: + <<: *default + username: <%= ENV['POSTGRES_USER'] || `whoami` %> + database: caseflow_certification_test<%= ENV['TEST_SUBCATEGORY'] %> + <% if ENV['POSTGRES_HOST'] %> + host: <%= ENV['POSTGRES_HOST'] %> + <% end %> + <% if ENV['POSTGRES_PASSWORD'] %> + password: <%= ENV['POSTGRES_PASSWORD'] %> + <% end %> + etl: + <<: *etl_default + database: caseflow_etl_test<%= ENV["TEST_SUBCATEGORY"] %> + # Increase the receive timeout value due to ORA-12609: TNS: Receive timeout occurred + # Solution: https://www.fatihacar.com/blog/category/databases/oracle/oracle-errors-and-solutions/page/13/ + # Oracle timeout configuration: https://github.com/rsim/oracle-enhanced#timeouts + vacols: + adapter: oracle_enhanced + username: VACOLS_TEST + password: VACOLS_TEST + database: "(DESCRIPTION= + (ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=<%= ENV['TEST_VACOLS_HOST']%>)(PORT=1521))) + (RECV_TIMEOUT=120)(SEND_TIMEOUT=5) + (CONNECT_DATA=(SID=BVAP)) + )" demo: - <<: *default - database: caseflow_certification_demo - -demo_vacols: - adapter: sqlite3 - pool: 5 - timeout: 5000 - database: db/demo-vacols.sqlite3 + primary: + <<: *default + database: caseflow_certification_demo + vacols: + adapter: sqlite3 + pool: 5 + timeout: 5000 + database: db/demo-vacols.sqlite3 ssh_forwarding: - <<: *default - database: caseflow_certification_ssh_forwarding - -ssh_forwarding_vacols: - adapter: oracle_enhanced - host: localhost - port: 1526 - username: <%= ENV["VACOLS_USERNAME"] %> - password: <%= ENV["VACOLS_PASSWORD"] %> - database: BVAP + primary: + <<: *default + database: caseflow_certification_ssh_forwarding + vacols: + adapter: oracle_enhanced + host: localhost + port: 1526 + username: <%= ENV["VACOLS_USERNAME"] %> + password: <%= ENV["VACOLS_PASSWORD"] %> + database: BVAP staging: - <<: *default - database: caseflow_certification_staging - -staging_vacols: - adapter: oracle_enhanced - username: <%= ENV["VACOLS_USERNAME"] %> - password: <%= ENV["VACOLS_PASSWORD"] %> - host: vacols.dev.vaco.va.gov - port: 1526 - database: BVAP.VACO.VA.GOV + primary: + <<: *default + database: caseflow_certification_staging + vacols: + adapter: oracle_enhanced + username: <%= ENV["VACOLS_USERNAME"] %> + password: <%= ENV["VACOLS_PASSWORD"] %> + host: vacols.dev.vaco.va.gov + port: 1526 + database: BVAP.VACO.VA.GOV production: - <<: *default - url: <%= ENV["POSTGRES_URL"] %> - -etl_production: - <<: *etl_default - url: <%= ENV["ETL_DB_URL"] %> - -production_vacols: - adapter: oracle_enhanced - username: <%= ENV["VACOLS_USERNAME"] %> - password: <%= ENV["VACOLS_PASSWORD"] %> - host: <%= ENV["VACOLS_HOST"] %> - port: <%= ENV["VACOLS_PORT"] %> - database: <%= ENV["VACOLS_DATABASE"] %> - pool: <%= ENV["DB_CONN_POOL_MAX_SIZE"] || 5 %> + primary: + <<: *default + url: <%= ENV["POSTGRES_URL"] %> + etl: + <<: *etl_default + url: <%= ENV["ETL_DB_URL"] %> + vacols: + adapter: oracle_enhanced + username: <%= ENV["VACOLS_USERNAME"] %> + password: <%= ENV["VACOLS_PASSWORD"] %> + host: <%= ENV["VACOLS_HOST"] %> + port: <%= ENV["VACOLS_PORT"] %> + database: <%= ENV["VACOLS_DATABASE"] %> + pool: <%= ENV["DB_CONN_POOL_MAX_SIZE"] || 5 %> diff --git a/config/initializers/warmup_vacols.rb b/config/initializers/warmup_vacols.rb index 68f5db2f460..053b49551f7 100644 --- a/config/initializers/warmup_vacols.rb +++ b/config/initializers/warmup_vacols.rb @@ -20,7 +20,7 @@ db_config = Rails.application.config.database_configuration[Rails.env] # use specified initial pool size, default to half the maximum size - initial_pool_size = (ENV['DB_CONN_POOL_INITIAL_SIZE'] || db_config['pool'] / 2).to_i + initial_pool_size = (ENV['DB_CONN_POOL_INITIAL_SIZE'] || db_config['primary']['pool'] / 2).to_i Rails.logger.info("creating #{initial_pool_size} initial connections...") unless ApplicationController.dependencies_faked? From e07e6361d1bdaffad12d9eae64b9a746b4319c64 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Mon, 20 May 2024 14:26:28 -0400 Subject: [PATCH 45/69] =?UTF-8?q?=F0=9F=94=A7=20Add=20etl=20migration=20pa?= =?UTF-8?q?ths=20to=20DB=20config?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- config/database.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/config/database.yml b/config/database.yml index 3a536e28e6b..f4af703a714 100644 --- a/config/database.yml +++ b/config/database.yml @@ -31,6 +31,7 @@ etl_default: &etl_default <% if ENV["ETL_DB_PASSWORD"] || ENV["POSTGRES_PASSWORD"] %> password: <%= ENV["ETL_DB_PASSWORD"] || ENV["POSTGRES_PASSWORD"] %> <% end %> + migrations_paths: db/etl_migrate development: primary: From 300399d26589d1caeef5c1d743e31dffabd9e70f Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Mon, 20 May 2024 19:28:15 -0400 Subject: [PATCH 46/69] =?UTF-8?q?=F0=9F=94=A7=20Update=20DB=20connection?= =?UTF-8?q?=20names=20in=20'database=5Fcleaner'=20config?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- spec/support/database_cleaner.rb | 40 ++++++++++++++------------------ 1 file changed, 18 insertions(+), 22 deletions(-) diff --git a/spec/support/database_cleaner.rb b/spec/support/database_cleaner.rb index bc90949aa75..9905b0e7f78 100644 --- a/spec/support/database_cleaner.rb +++ b/spec/support/database_cleaner.rb @@ -5,35 +5,31 @@ RSpec.configure do |config| config.use_transactional_fixtures = false - etl = "etl_#{Rails.env}".to_sym - vacols = "#{Rails.env}_vacols".to_sym - caseflow = Rails.env.to_s.to_sym - vacols_tables_to_preserve = %w[vftypes issref] - # IMPORTANT that in all these hook defs, the "caseflow" connection comes last. + # IMPORTANT that in all these hook defs, the "primary" connection comes last. config.before(:suite) do - DatabaseCleaner[:active_record, { db: etl }].clean_with(:truncation) - DatabaseCleaner[:active_record, { db: vacols }] + DatabaseCleaner[:active_record, { db: :etl }].clean_with(:truncation) + DatabaseCleaner[:active_record, { db: :vacols }] .clean_with(:deletion, except: vacols_tables_to_preserve) - DatabaseCleaner[:active_record, { db: caseflow }].clean_with(:truncation) + DatabaseCleaner[:active_record, { db: :primary }].clean_with(:truncation) end config.before(:each) do |example| # Allows seeded data to persist for use across threads # You will need to manually clean the data once the threads under test close. unless example.metadata[:bypass_cleaner] - DatabaseCleaner[:active_record, { db: vacols }].strategy = :transaction - DatabaseCleaner[:active_record, { db: caseflow }].strategy = :transaction + DatabaseCleaner[:active_record, { db: :vacols }].strategy = :transaction + DatabaseCleaner[:active_record, { db: :primary }].strategy = :transaction end end config.before(:each, db_clean: :truncation) do |example| unless example.metadata[:bypass_cleaner] - DatabaseCleaner[:active_record, { db: vacols }].strategy = + DatabaseCleaner[:active_record, { db: :vacols }].strategy = :deletion, { except: vacols_tables_to_preserve } - DatabaseCleaner[:active_record, { db: caseflow }].strategy = :truncation + DatabaseCleaner[:active_record, { db: :primary }].strategy = :truncation end end @@ -46,42 +42,42 @@ # Driver is probably for an external browser with an app # under test that does *not* share a database connection with the # specs, so use truncation strategy. - DatabaseCleaner[:active_record, { db: vacols }].strategy = + DatabaseCleaner[:active_record, { db: :vacols }].strategy = :deletion, { except: vacols_tables_to_preserve } - DatabaseCleaner[:active_record, { db: caseflow }].strategy = :truncation + DatabaseCleaner[:active_record, { db: :primary }].strategy = :truncation end end config.before(:each) do |example| unless example.metadata[:bypass_cleaner] - DatabaseCleaner[:active_record, { db: vacols }].start - DatabaseCleaner[:active_record, { db: caseflow }].start + DatabaseCleaner[:active_record, { db: :vacols }].start + DatabaseCleaner[:active_record, { db: :primary }].start end end config.append_after(:each) do - DatabaseCleaner[:active_record, { db: vacols }].clean - DatabaseCleaner[:active_record, { db: caseflow }].clean + DatabaseCleaner[:active_record, { db: :vacols }].clean + DatabaseCleaner[:active_record, { db: :primary }].clean clean_application! end # ETL is never used in feature tests and there are only a few, so we tag those with :etl # ETL db uses deletion strategy everywhere because syncing runs in a transaction. config.before(:each, :etl) do - DatabaseCleaner[:active_record, { db: etl }].strategy = :deletion + DatabaseCleaner[:active_record, { db: :etl }].strategy = :deletion end config.before(:each, :etl, db_clean: :truncation) do - DatabaseCleaner[:active_record, { db: etl }].strategy = :truncation + DatabaseCleaner[:active_record, { db: :etl }].strategy = :truncation end config.before(:each, :etl) do Rails.logger.info("DatabaseCleaner.start ETL") - DatabaseCleaner[:active_record, { db: etl }].start + DatabaseCleaner[:active_record, { db: :etl }].start end config.append_after(:each, :etl) do - DatabaseCleaner[:active_record, { db: etl }].clean + DatabaseCleaner[:active_record, { db: :etl }].clean Rails.logger.info("DatabaseCleaner.clean ETL") end end From 8fe0a64bccc3824538ac46e40252372b9df5044a Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 21 May 2024 13:14:25 -0400 Subject: [PATCH 47/69] =?UTF-8?q?=E2=99=BB=EF=B8=8F=20Use=20new=20database?= =?UTF-8?q?-specific=20rake=20tasks?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit After migrating to the Rails 6+ native multi-database configuration, the behavior of some DB management tasks, such as `rake db:migrate` changed such that they now act on ALL databases and not just the primary database. So we must replace the invocations of these tasks with their new, database-specific counterparts. --- .github/workflows/make-docs.yml | 4 ++-- .github/workflows/workflow.yml | 4 ++-- MAC_INTEL.md | 4 ++-- MAC_M1.md | 4 ++-- Makefile.example | 15 +++++++-------- Rakefile | 3 ++- WINDOWS_11.md | 4 ++-- docker-bin/startup.sh | 3 ++- lib/tasks/local/build.rake | 4 ++-- lib/tasks/local/destroy.rake | 2 +- lib/tasks/spec.rake | 2 +- scripts/squash.sh | 2 +- 12 files changed, 26 insertions(+), 25 deletions(-) diff --git a/.github/workflows/make-docs.yml b/.github/workflows/make-docs.yml index 3dccd6568ed..7039c6120be 100644 --- a/.github/workflows/make-docs.yml +++ b/.github/workflows/make-docs.yml @@ -79,11 +79,11 @@ jobs: POSTGRES_USER: postgres run: | echo "::group::Set up Caseflow DB" - bin/rails db:create && bin/rails db:schema:load + bin/rails db:create:primary && bin/rails db:schema:load:primary echo "::endgroup::" echo "::group::Set up Caseflow ETL DB" - DB=etl bundle exec rake db:create db:schema:load + bundle exec rake db:create:etl db:schema:load:etl echo "::endgroup::" # Skipping VACOLS since Oracle DB is not set up diff --git a/.github/workflows/workflow.yml b/.github/workflows/workflow.yml index 6e17eb068a6..88958d6a15c 100644 --- a/.github/workflows/workflow.yml +++ b/.github/workflows/workflow.yml @@ -192,8 +192,8 @@ jobs: - name: Database setup run: | - ./ci-bin/capture-log "DB=etl bundle exec rake db:create db:schema:load db:migrate" - ./ci-bin/capture-log "bundle exec rake db:create db:schema:load db:migrate" + ./ci-bin/capture-log "bundle exec rake db:create:etl db:schema:load:etl db:migrate:etl" + ./ci-bin/capture-log "bundle exec rake db:create:primary db:schema:load:primary db:migrate:primary" ./ci-bin/capture-log "make -f Makefile.example external-db-create" # added line to create external table(s) that are needed for tests diff --git a/MAC_INTEL.md b/MAC_INTEL.md index 18fc4ea6a06..ee480bdb68d 100644 --- a/MAC_INTEL.md +++ b/MAC_INTEL.md @@ -96,12 +96,12 @@ 27. Run `make reset` * If issues occur: - 1. Run `bundle exec rake db:create` + 1. Run `bundle exec rake db:create:primary` * If you get connection issues stating no file to be found, run the following: * `rm /opt/homebrew/var/postgres/postmaster.pid` or possibly `rm /usr/local/var/postgres/postmaster.pid` * `brew services restart postgresql` 2. Run `bundle exec rake local:vacols:seed` - 3. Run `bundle exec rake db:schema:load db:seed` + 3. Run `bundle exec rake db:schema:load:primary db:seed` 27. Open a new tab in terminal diff --git a/MAC_M1.md b/MAC_M1.md index 571b340cba5..a08d13f4c48 100644 --- a/MAC_M1.md +++ b/MAC_M1.md @@ -194,7 +194,7 @@ Running Caseflow 5. Start Vacols UTM VM (if not already running) 6. run `make up-m1` to create the docker containers and volumes 7. run `make reset` to (re)create and seed the database; this takes a while (~45 minutes) - 1. if you get a database not found error, run `bundle exec rake db:drop db:create db:schema:load`, and then run `make reset` again + 1. if you get a database not found error, run `bundle exec rake db:drop:primary db:create:primary db:schema:load:primary`, and then run `make reset` again 8. open a second terminal tab/window 9. run `make run-backend-m1` in one tab, and `make run-frontend` in the other 1. the backend should launch on `localhost:3000`. go there in your browser to access Caseflow @@ -252,7 +252,7 @@ The following steps are an alternative to step 7 of the Running Caseflow section * d. `make reset` * i. Resets caseflow and ETL database schemas, seeds databases, and enables feature flags * **If `make reset` returns database not found error: - * a. Run command `bundle exec rake db:drop` + * a. Run command `bundle exec rake db:drop:primary` * b. Download caseflow-db-backup.gz (not able to share this download via policy, ask in the slack channel) * c. Enter terminal, navigate to ~/Downloads * e. Run command diff --git a/Makefile.example b/Makefile.example index e27b6efcb99..2dcf61e3f96 100644 --- a/Makefile.example +++ b/Makefile.example @@ -222,19 +222,19 @@ c: ## Start rails console bundle exec rails console etl-migrate: ## Migrate ETL database - DB=etl bundle exec rake db:migrate + bundle exec rake db:migrate:etl etl-test-prepare: - DB=etl bundle exec rake db:test:prepare + bundle exec rake db:test:prepare:etl etl-rollback: ## Rollback ETL database - DB=etl bundle exec rake db:rollback + bundle exec rake db:rollback:etl db-migrate: ## Migrate main Caseflow db - bundle exec rake db:migrate + bundle exec rake db:migrate:primary db-rollback: ## Rollback main Caseflow db - bundle exec rake db:rollback + bundle exec rake db:rollback:primary migrate: external-db-remove etl-migrate etl-test-prepare db-migrate ## Migrate all non-external Rails databases @@ -246,8 +246,8 @@ fresh: ## https://github.com/imsky/git-fresh - updates local master to match or reset: reset-dbs seed-dbs enable-feature-flags ## Resets databases and enable feature flags reset-dbs: ## Resets Caseflow and ETL database schemas - DB=etl bundle exec rake db:drop db:create db:schema:load - bundle exec rake db:drop db:create db:schema:load + bundle exec rake db:drop:etl db:create:etl db:schema:load:etl + bundle exec rake db:drop:primary db:create:primary db:schema:load:primary make consolidated_db_scripts seed-vbms-ext-claim: ## Seed only vbms_ext_claim @@ -260,7 +260,6 @@ seed-dbs: ## Seed all databases bundle exec rake local:vacols:seed bundle exec rake spec:setup_vacols bundle exec rake db:seed - DB=etl bundle exec rake db:seed enable-feature-flags: ## enable all feature flags bundle exec rails runner scripts/enable_features_dev.rb diff --git a/Rakefile b/Rakefile index 838add5d4e9..dac0a4b9bb8 100644 --- a/Rakefile +++ b/Rakefile @@ -10,4 +10,5 @@ Dir[Rails.root.join("lib/tasks/support/**/*.rb")].sort.each { |f| require f } Rails.application.load_tasks -task "db:schema:dump": "strong_migrations:alphabetize_columns" +task "db:schema:dump:primary": "strong_migrations:alphabetize_columns" +task "db:schema:dump:etl": "strong_migrations:alphabetize_columns" diff --git a/WINDOWS_11.md b/WINDOWS_11.md index 2b8139175f4..8d7f388a3d9 100644 --- a/WINDOWS_11.md +++ b/WINDOWS_11.md @@ -134,9 +134,9 @@ Run each line is a separate command, run them one at a time 26. ```cd ~/appeals/caseflow in terminal``` -27. ```bin/rails db:migrate RAILS_ENV=development``` +27. ```bin/rails db:migrate:primary RAILS_ENV=development``` -28. Run ```bundle install``` to install missing gems and then ```bin/rails db:migrate``` ```RAILS_ENV=development``` command if you have too again. +28. Run ```bundle install``` to install missing gems and then ```bin/rails db:migrate:primary``` ```RAILS_ENV=development``` command if you have too again. 29. ```make reset``` (should have split_correlation_tables) diff --git a/docker-bin/startup.sh b/docker-bin/startup.sh index 97a73eeae4e..5689eecc68d 100644 --- a/docker-bin/startup.sh +++ b/docker-bin/startup.sh @@ -25,7 +25,8 @@ echo "Waiting for Vacols to be ready" rake local:vacols:wait_for_connection echo "Creating DB in PG" -rake db:setup +rake db:create:primary +rake db:schema:load:primary echo "Seeding Facols" rake local:vacols:seed diff --git a/lib/tasks/local/build.rake b/lib/tasks/local/build.rake index 15b08e42ada..7f202ba75f7 100644 --- a/lib/tasks/local/build.rake +++ b/lib/tasks/local/build.rake @@ -21,7 +21,7 @@ namespace :local do puts "" puts ">>> 04/08 Creating development and test caseflow databases" - system("RAILS_ENV=development bundle exec rake db:create") || abort + system("RAILS_ENV=development bundle exec rake db:create:primary") || abort puts ">>> 05/08 Seeding FACOLS" system("RAILS_ENV=development bundle exec rake local:vacols:seed") || abort @@ -30,7 +30,7 @@ namespace :local do system("RAILS_ENV=test bundle exec rake spec:setup_vacols") || abort puts ">>> 07/08 Loading schema and seeding local caseflow database" - system("RAILS_ENV=development bundle exec rake db:schema:load db:seed") || abort + system("RAILS_ENV=development bundle exec rake db:schema:load:primary db:seed") || abort puts ">>> 08/08 Enabling feature flags" system("bundle exec rails runner scripts/enable_features_dev.rb") || abort diff --git a/lib/tasks/local/destroy.rake b/lib/tasks/local/destroy.rake index e0873e20ffe..541f489e467 100644 --- a/lib/tasks/local/destroy.rake +++ b/lib/tasks/local/destroy.rake @@ -6,7 +6,7 @@ namespace :local do puts ">>> BEGIN local:destroy" puts ">>> 01/02 Dropping development and test caseflow databases" - system("RAILS_ENV=development bundle exec rake db:drop") || abort + system("RAILS_ENV=development bundle exec rake db:drop:primary") || abort puts ">>> 02/02 Tearing down docker volumes" # Note: In some cases, there may be dangling images or volumes that diff --git a/lib/tasks/spec.rake b/lib/tasks/spec.rake index aca6c9082c3..31dfc56f59a 100644 --- a/lib/tasks/spec.rake +++ b/lib/tasks/spec.rake @@ -25,7 +25,7 @@ namespace :spec do task :spec_name do # rubocop:disable Rails/RakeEnvironment envs = "export TEST_SUBCATEGORY=#{spec_name} && export RAILS_ENV=test &&" - ["rake db:create", "rake db:schema:load"].each do |cmd| + ["rake db:create:primary", "rake db:schema:load:primary"].each do |cmd| ShellCommand.run_and_batch_output("#{envs} #{cmd}") end end diff --git a/scripts/squash.sh b/scripts/squash.sh index bb631e46153..751e8463c34 100755 --- a/scripts/squash.sh +++ b/scripts/squash.sh @@ -5,5 +5,5 @@ git ls-files db/migrate/*.rb | sort | tail -1 | \ %x(git rm -f db/migrate/*.rb; mkdir db/migrate; git mv db/schema.rb #{init_schema}; - bundle exec rake db:migrate; + bundle exec rake db:migrate:primary; git add db/schema.rb; git commit -m 'Squashed migrations')" From 6f45766db52c6c22b5c9d416e224e454623d77df Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 21 May 2024 13:22:04 -0400 Subject: [PATCH 48/69] =?UTF-8?q?=E2=9E=96=20Remove=20'multiverse'=20gem?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Now that we have fiully transitioned to Rails-native multi-database support, we are no longer reliant on the 'multiverse' gem and can remove it. --- Gemfile | 3 --- Gemfile.lock | 5 ----- 2 files changed, 8 deletions(-) diff --git a/Gemfile b/Gemfile index 86c4be0bbc7..ca8f2733a19 100644 --- a/Gemfile +++ b/Gemfile @@ -36,9 +36,6 @@ gem "icalendar" gem "kaminari" gem "logstasher" gem "moment_timezone-rails" -# Rails 6 has native support for multiple dbs, so prefer that over multiverse after upgrade. -# https://github.com/ankane/multiverse#upgrading-to-rails-6 -gem "multiverse" gem "newrelic_rpm" gem "nokogiri", ">= 1.11.0.rc4" gem "paper_trail", "~> 12.0" diff --git a/Gemfile.lock b/Gemfile.lock index 9779c3d03c7..dd7640b96f9 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -1726,10 +1726,6 @@ GEM railties (>= 3.1) msgpack (1.4.2) multipart-post (2.1.1) - multiverse (0.2.2) - activerecord (>= 4.2) - activesupport (>= 4.2) - railties (>= 4.2) nap (1.1.0) nenv (0.3.0) net-imap (0.3.7) @@ -2099,7 +2095,6 @@ DEPENDENCIES logstasher meta_request moment_timezone-rails - multiverse newrelic_rpm nokogiri (>= 1.11.0.rc4) paper_trail (~> 12.0) From 93c54bce0341c8f0ccb65a3d30515d7909a6455e Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 21 May 2024 19:48:55 -0400 Subject: [PATCH 49/69] =?UTF-8?q?=F0=9F=97=83=EF=B8=8F=20Prohibit=20execut?= =?UTF-8?q?ion=20of=20vacols=20DB=20and=20non-DB-specific=20rake=20tasks?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit After transitioning to Rails-native multi-DB support, the behavior of some DB tasks changed such that they will now act on ALL databases and not just the primary database (ex. `rake db:migrate` will now migrate ALL databases). To avoid accidents, we re-define these tasks here to no-op and output a helpful message to redirect developers toward using their new database-specific counterparts instead. --- lib/tasks/db.rake | 300 ++++++++++++++++++++++++++++++++++++++ spec/lib/tasks/db_spec.rb | 295 +++++++++++++++++++++++++++++++++++++ 2 files changed, 595 insertions(+) create mode 100644 lib/tasks/db.rake create mode 100644 spec/lib/tasks/db_spec.rb diff --git a/lib/tasks/db.rake b/lib/tasks/db.rake new file mode 100644 index 00000000000..e56cb6d3404 --- /dev/null +++ b/lib/tasks/db.rake @@ -0,0 +1,300 @@ +# frozen_string_literal: true + +# Once on Rails 7+, we can add the `database_tasks: false` option to the `database.yml` to permit connection to the +# vacols database without generating database tasks for it: +# https://github.com/rails/rails/blob/984c3ef2775781d47efa9f541ce570daa2434a80/guides/source/active_record_multiple_databases.md?plain=1#L203-L219 +# +# After adopting and sufficiently testing the above setting setting, we can dispense with the workarounds in this file. +if Rails::VERSION::MAJOR >= 7 + ActiveSupport::Deprecation.warn( + "Use the new `database_tasks` DB config option to skip generation of database tasks for the VACOLS DB.\n" \ + "For further details, see https://github.com/rails/rails/blob/984c3ef2775781d47efa9f541ce570daa2434a80/guides/source/active_record_multiple_databases.md?plain=1#L203-L219" + ) +end + +# Explicitly clear any generated database tasks for the vacols DB. +%w[ + db:create:vacols + db:drop:vacols + db:migrate:vacols + db:migrate:status:vacols + db:migrate:up:vacols + db:migrate:down:vacols + db:migrate:redo:vacols + db:rollback:vacols + db:schema:dump:vacols + db:schema:load:vacols + db:structure:dump:vacols + db:structure:load:vacols +].each do |task_name| + Rake::Task[task_name].clear if Rake::Task.task_defined?(task_name) +end + +#----------------------------------------------------------------------------------------------------------------------- + +# The original definition of the `db:seed` task calls `db:abort_if_pending_migrations`, which would attempt to create a +# `schema_migrations` table on ALL configured databases, including the vacols DB: +# https://github.com/rails/rails/blob/ac87f58207cff18880593263be9d83456aa3a2ef/activerecord/lib/active_record/railties/databases.rake#L389-L393 +# +# This would result in the following error, since the Caseflow app does not have permissions to alter the VACOLS schema: +# +# ActiveRecord::StatementInvalid: OCIError: ORA-01031: insufficient privileges +# +# This re-definiton of `db:seed` will instead call the DB-specific `db:abort_if_pending_migrations` as a workaround. + +db_namespace = namespace :db do + Rake::Task["db:seed"].clear if Rake::Task.task_defined?("db:seed") + desc "Loads the seed data from db/seeds.rb" + task seed: :load_config do + db_namespace["abort_if_pending_migrations:primary"].invoke + + # skip for environments where ETL DB does not exist + db_namespace["abort_if_pending_migrations:etl"].invoke unless %w(demo prodtest).include?(ENV["DEPLOY_ENV"]) + + ActiveRecord::Base.establish_connection(ActiveRecord::Tasks::DatabaseTasks.env.to_sym) + + ActiveRecord::Tasks::DatabaseTasks.load_seed + end +end + +#----------------------------------------------------------------------------------------------------------------------- + +# After transitioning to Rails-native multi-DB support, the behavior of some DB tasks changed such that they will now +# act on ALL databases, including the vacols DB, and not just the primary database. +# +# To avoid accidents, we re-define these tasks here to no-op and output a helpful message to redirect developers toward +# using their new database-specific counterparts instead. + +# rubocop:disable Rails/RakeEnvironment, Layout/HeredocIndentation +namespace :db do + Rake::Task["db:create"].clear if Rake::Task.task_defined?("db:create") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :create do + puts <<~HEREDOC + + db:create is prohibited. + + Prefer using the appropriate database-specific task below: + + db:create:primary # Create primary database for current environment + db:create:etl # Create etl database for current environment + HEREDOC + end + + Rake::Task["db:drop"].clear if Rake::Task.task_defined?("db:drop") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :drop do + puts <<~HEREDOC + + db:drop is prohibited. + + Prefer using the appropriate database-specific task below: + + db:drop:primary # Drop primary database for current environment + db:drop:etl # Drop etl database for current environment + HEREDOC + end + + Rake::Task["db:migrate"].clear if Rake::Task.task_defined?("db:migrate") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :migrate do + puts <<~HEREDOC + + db:migrate is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:primary # Migrate primary database for current environment + db:migrate:etl # Migrate etl database for current environment + HEREDOC + end + + namespace :migrate do + Rake::Task["db:migrate:down"].clear if Rake::Task.task_defined?("db:migrate:down") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :down do + puts <<~HEREDOC + + db:migrate:down is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:down:primary # Runs the "down" for a given migration VERSION on the primary database + db:migrate:down:etl # Runs the "down" for a given migration VERSION on the etl database + HEREDOC + end + + Rake::Task["db:migrate:redo"].clear if Rake::Task.task_defined?("db:migrate:redo") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :redo do + puts <<~HEREDOC + + db:migrate:redo is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:redo:primary # Rolls back primary database one migration and re-migrates up (options: STEP=x, VERSION=x) + db:migrate:redo:etl # Rolls back etl database one migration and re-migrates up (options: STEP=x, VERSION=x) + HEREDOC + end + + Rake::Task["db:migrate:status"].clear if Rake::Task.task_defined?("db:migrate:status") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :status do + puts <<~HEREDOC + + db:migrate:status is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:status:primary # Display status of migrations for primary database + db:migrate:status:etl # Display status of migrations for etl database + HEREDOC + end + + Rake::Task["db:migrate:up"].clear if Rake::Task.task_defined?("db:migrate:up") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :up do + puts <<~HEREDOC + + db:migrate:up is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:up:primary # Runs the "up" for a given migration VERSION on the primary database + db:migrate:up:etl # Runs the "up" for a given migration VERSION on the etl database + HEREDOC + end + end + + Rake::Task["db:reset"].clear if Rake::Task.task_defined?("db:reset") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :reset do + puts <<~HEREDOC + + db:reset is prohibited. + + Prefer using the appropriate sequence of database-specific tasks below: + + db:drop:primary db:create:primary db:schema:load:primary db:seed # Reset the primary database + db:drop:etl db:create:etl db:schema:load:etl db:seed # Reset the etl database + HEREDOC + end + + Rake::Task["db:rollback"].clear if Rake::Task.task_defined?("db:rollback") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :rollback do + puts <<~HEREDOC + + db:rollback is prohibited. + + Prefer using the appropriate database-specific task below: + + db:rollback:primary # Rollback primary database for current environment (specify steps w/ STEP=n) + db:rollback:etl # Rollback etl database for current environment (specify steps w/ STEP=n) + HEREDOC + end + + namespace :schema do + Rake::Task["db:schema:dump"].clear if Rake::Task.task_defined?("db:schema:dump") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :dump do + puts <<~HEREDOC + + db:schema:dump is prohibited. + + Prefer using the appropriate database-specific task below: + + db:schema:dump:primary # Creates a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) for primary database + db:schema:dump:etl # Creates a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) for etl database + HEREDOC + end + + Rake::Task["db:schema:load"].clear if Rake::Task.task_defined?("db:schema:load") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :load do + puts <<~HEREDOC + + db:schema:load is prohibited. + + Prefer using the appropriate database-specific task below: + + db:schema:load:primary # Loads a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) into the primary database + db:schema:load:etl # Loads a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) into the etl database + HEREDOC + end + end + + Rake::Task["db:setup"].clear if Rake::Task.task_defined?("db:setup") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :setup do + puts <<~HEREDOC + + db:setup is prohibited. + + Prefer using the appropriate sequence of database-specific tasks below: + + db:create:primary db:schema:load:primary db:seed # Setup the primary database + db:create:etl db:schema:load:etl db:seed # Setup the etl database + HEREDOC + end + + namespace :test do + Rake::Task["db:test:load"].clear if Rake::Task.task_defined?("db:test:load") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :load do + puts <<~HEREDOC + + db:test:load is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:load:primary # Recreate the primary test database from the current schema + db:test:load:etl # Recreate the etl test database from the current schema + HEREDOC + end + + Rake::Task["db:test:load_schema"].clear if Rake::Task.task_defined?("db:test:load_schema") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :load_schema do + puts <<~HEREDOC + + db:test:load_schema is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:load_schema:primary # Recreate the primary test database from an existent schema file (schema.rb or structure.sql, depending on `config.active_record.schema_format`) + db:test:load_schema:etl # Recreate the etl test database from an existent schema file (schema.rb or structure.sql, depending on `config.active_record.schema_format`) + HEREDOC + end + + Rake::Task["db:test:prepare"].clear if Rake::Task.task_defined?("db:test:prepare") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :prepare do + puts <<~HEREDOC + + db:test:prepare is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:prepare:primary # Load the schema for the primary test database + db:test:prepare:etl # Load the schema for the etl test database + HEREDOC + end + + Rake::Task["db:test:purge"].clear if Rake::Task.task_defined?("db:test:purge") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :purge do + puts <<~HEREDOC + + db:test:purge is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:purge:primary # Empty the primary test database + db:test:purge:etl # Empty the etl test database + HEREDOC + end + end +end +# rubocop:enable Rails/RakeEnvironment, Layout/HeredocIndentation diff --git a/spec/lib/tasks/db_spec.rb b/spec/lib/tasks/db_spec.rb new file mode 100644 index 00000000000..fab7ca72e54 --- /dev/null +++ b/spec/lib/tasks/db_spec.rb @@ -0,0 +1,295 @@ +# frozen_string_literal: true + +# rubocop:disable Layout/HeredocIndentation +describe "db" do + include_context "rake" + + describe "create" do + subject { Rake::Task["db:create"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:create is prohibited. + + Prefer using the appropriate database-specific task below: + + db:create:primary # Create primary database for current environment + db:create:etl # Create etl database for current environment + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "drop" do + subject { Rake::Task["db:drop"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:drop is prohibited. + + Prefer using the appropriate database-specific task below: + + db:drop:primary # Drop primary database for current environment + db:drop:etl # Drop etl database for current environment + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "migrate" do + subject { Rake::Task["db:migrate"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:migrate is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:primary # Migrate primary database for current environment + db:migrate:etl # Migrate etl database for current environment + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "migrate:down" do + subject { Rake::Task["db:migrate:down"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:migrate:down is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:down:primary # Runs the "down" for a given migration VERSION on the primary database + db:migrate:down:etl # Runs the "down" for a given migration VERSION on the etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "migrate:redo" do + subject { Rake::Task["db:migrate:redo"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:migrate:redo is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:redo:primary # Rolls back primary database one migration and re-migrates up (options: STEP=x, VERSION=x) + db:migrate:redo:etl # Rolls back etl database one migration and re-migrates up (options: STEP=x, VERSION=x) + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "migrate:status" do + subject { Rake::Task["db:migrate:status"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:migrate:status is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:status:primary # Display status of migrations for primary database + db:migrate:status:etl # Display status of migrations for etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "migrate:up" do + subject { Rake::Task["db:migrate:up"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:migrate:up is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:up:primary # Runs the "up" for a given migration VERSION on the primary database + db:migrate:up:etl # Runs the "up" for a given migration VERSION on the etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "reset" do + subject { Rake::Task["db:reset"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:reset is prohibited. + + Prefer using the appropriate sequence of database-specific tasks below: + + db:drop:primary db:create:primary db:schema:load:primary db:seed # Reset the primary database + db:drop:etl db:create:etl db:schema:load:etl db:seed # Reset the etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "rollback" do + subject { Rake::Task["db:rollback"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:rollback is prohibited. + + Prefer using the appropriate database-specific task below: + + db:rollback:primary # Rollback primary database for current environment (specify steps w/ STEP=n) + db:rollback:etl # Rollback etl database for current environment (specify steps w/ STEP=n) + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "schema:dump" do + subject { Rake::Task["db:schema:dump"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:schema:dump is prohibited. + + Prefer using the appropriate database-specific task below: + + db:schema:dump:primary # Creates a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) for primary database + db:schema:dump:etl # Creates a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) for etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "schema:load" do + subject { Rake::Task["db:schema:load"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:schema:load is prohibited. + + Prefer using the appropriate database-specific task below: + + db:schema:load:primary # Loads a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) into the primary database + db:schema:load:etl # Loads a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) into the etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "setup" do + subject { Rake::Task["db:setup"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:setup is prohibited. + + Prefer using the appropriate sequence of database-specific tasks below: + + db:create:primary db:schema:load:primary db:seed # Setup the primary database + db:create:etl db:schema:load:etl db:seed # Setup the etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "test:load" do + subject { Rake::Task["db:test:load"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:test:load is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:load:primary # Recreate the primary test database from the current schema + db:test:load:etl # Recreate the etl test database from the current schema + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "test:load_schema" do + subject { Rake::Task["db:test:load_schema"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:test:load_schema is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:load_schema:primary # Recreate the primary test database from an existent schema file (schema.rb or structure.sql, depending on `config.active_record.schema_format`) + db:test:load_schema:etl # Recreate the etl test database from an existent schema file (schema.rb or structure.sql, depending on `config.active_record.schema_format`) + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "test:prepare" do + subject { Rake::Task["db:test:prepare"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:test:prepare is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:prepare:primary # Load the schema for the primary test database + db:test:prepare:etl # Load the schema for the etl test database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "test:purge" do + subject { Rake::Task["db:test:purge"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:test:purge is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:purge:primary # Empty the primary test database + db:test:purge:etl # Empty the etl test database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end +end +# rubocop:enable Layout/HeredocIndentation From 655f77b9f4e643804607db362f6a1a66f93d429a Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 23 May 2024 18:00:44 -0400 Subject: [PATCH 50/69] =?UTF-8?q?=E2=99=BB=EF=B8=8F=20Create=20new=20envir?= =?UTF-8?q?onment=20for=20GH=20workflow=20'Make-docs-to-webpage'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Instead of performing a bunch of hard-to-maintain `sed` gymnastics to modify the existing 'test' environment, let's create a new 'make_docs' environment (based off of 'test') and configure it appropriately for use by the 'Make-docs-to-webpage' GH workflow. --- .github/workflows/make-docs.yml | 30 +----- Gemfile | 2 +- config/database.yml | 23 +++++ config/environments/make_docs.rb | 152 +++++++++++++++++++++++++++++++ 4 files changed, 178 insertions(+), 29 deletions(-) create mode 100644 config/environments/make_docs.rb diff --git a/.github/workflows/make-docs.yml b/.github/workflows/make-docs.yml index 7039c6120be..7d4516d5621 100644 --- a/.github/workflows/make-docs.yml +++ b/.github/workflows/make-docs.yml @@ -40,32 +40,6 @@ jobs: - name: Checkout code uses: actions/checkout@v4 - # Since installing a client to Oracle DB is a pain and setting up the VACOLS DB is not necessary, let's skip it. - - name: Remove references to VACOLS Oracle DB - run: | - sed '/ruby-oci8/d' -i Gemfile - sed '/activerecord-oracle_enhanced-adapter/d' -i Gemfile - - # Would like to run `bundle lock --update` to update Gemfile.lock without installing gems (to be done in next step) - # but bundler is not installed yet, so manually modify it: - sed '/ruby-oci8/d' -i Gemfile.lock - sed '/^ activerecord-oracle_enhanced-adapter/,/^ [a-z]/{/^ activerecord-oracle_enhanced-adapter/!{/^ [a-z]/!d}}' -i Gemfile.lock - sed '/activerecord-oracle_enhanced-adapter/d' -i Gemfile.lock - sed '/ruby-plsql/d' -i Gemfile.lock - - # Remove VACOLS database configuration - # https://stackoverflow.com/questions/6287755/using-sed-to-delete-all-lines-between-two-matching-patterns - sed '/_vacols:$/,/^$/{/^_vacols:$/!{/^$/!d}}' -i config/database.yml - - # Even though it won't be used in this script, Rails expects a 'test_vacols' configuration due to "#{Rails.env}_vacols" - echo '# Copied from demo_vacols configuration - test_vacols: - adapter: postgresql - pool: 1 - timeout: 5000 - database: test-vacols - ' >> config/database.yml - - name: Setup Ruby and install gems uses: ruby/setup-ruby@v1 with: @@ -73,7 +47,7 @@ jobs: - name: Setup test database env: - RAILS_ENV: test + RAILS_ENV: make_docs POSTGRES_HOST: localhost POSTGRES_PASSWORD: postgres POSTGRES_USER: postgres @@ -95,7 +69,7 @@ jobs: - name: Create DB schema documentation env: - RAILS_ENV: test + RAILS_ENV: make_docs POSTGRES_HOST: localhost POSTGRES_PASSWORD: postgres POSTGRES_USER: postgres diff --git a/Gemfile b/Gemfile index ca8f2733a19..bddfeb97f36 100644 --- a/Gemfile +++ b/Gemfile @@ -92,7 +92,7 @@ group :production, :staging, :ssh_forwarding, :development, :test do gem "ruby-oci8", "~> 2.2" end -group :test, :development, :demo do +group :test, :development, :demo, :make_docs do # Security scanners gem "brakeman" gem "bullet", "~> 6.1.0" diff --git a/config/database.yml b/config/database.yml index f4af703a714..88e0cea7747 100644 --- a/config/database.yml +++ b/config/database.yml @@ -80,6 +80,29 @@ test: (CONNECT_DATA=(SID=BVAP)) )" +# For use by GH workflow `Make-docs-to-webpage` (See `.github/workflows/make-docs.yml`) +make_docs: + primary: + <<: *default + username: <%= ENV['POSTGRES_USER'] || `whoami` %> + database: caseflow_certification_test<%= ENV['TEST_SUBCATEGORY'] %> + <% if ENV['POSTGRES_HOST'] %> + host: <%= ENV['POSTGRES_HOST'] %> + <% end %> + <% if ENV['POSTGRES_PASSWORD'] %> + password: <%= ENV['POSTGRES_PASSWORD'] %> + <% end %> + etl: + <<: *etl_default + database: caseflow_etl_test<%= ENV["TEST_SUBCATEGORY"] %> + # Even though it won't be used, Rails expects a 'vacols' DB configuration due to + # VACOLS::Record `establish_connection :vacols` + vacols: + adapter: postgresql + pool: 1 + timeout: 5000 + database: test-vacols + demo: primary: <<: *default diff --git a/config/environments/make_docs.rb b/config/environments/make_docs.rb new file mode 100644 index 00000000000..fc7a8460dc3 --- /dev/null +++ b/config/environments/make_docs.rb @@ -0,0 +1,152 @@ +require "active_support/core_ext/integer/time" +require "fileutils" + +# The test environment is used exclusively to run your application's +# test suite. You never need to work with it otherwise. Remember that +# your test database is "scratch space" for the test suite and is wiped +# and recreated between test runs. Don't rely on the data there! + +Rails.application.configure do + # Settings specified here will take precedence over those in config/application.rb. + + config.cache_classes = true + + cache_dir = Rails.root.join("tmp", "cache", "test_#{ENV['TEST_SUBCATEGORY']}", $$.to_s) + FileUtils.mkdir_p(cache_dir) unless File.exists?(cache_dir) + config.cache_store = :file_store, cache_dir + + # Do not eager load code on boot. This avoids loading your whole application + # just for the purpose of running a single test. If you are using a tool that + # preloads Rails for running tests, you may have to set it to true. + config.eager_load = false + + # Configure public file server for tests with Cache-Control for performance. + config.public_file_server.enabled = true + config.public_file_server.headers = { + 'Cache-Control' => "public, max-age=#{1.hour.to_i}" + } + + # Show full error reports and disable caching. + config.consider_all_requests_local = false + config.action_controller.perform_caching = false + + # Raise exceptions instead of rendering exception templates. + config.action_dispatch.show_exceptions = true + + # Disable request forgery protection in test environment. + config.action_controller.allow_forgery_protection = false + + # Store uploaded files on the local file system in a temporary directory. + config.active_storage.service = :test + + config.action_mailer.perform_caching = false + + # Tell Action Mailer not to deliver emails to the real world. + # The :test delivery method accumulates sent emails in the + # ActionMailer::Base.deliveries array. + config.action_mailer.delivery_method = :test + + unless ENV['RAILS_ENABLE_TEST_LOG'] + config.logger = Logger.new(nil) + config.log_level = :error + end + + # Raises error for missing translations. + # config.i18n.raise_on_missing_translations = true + + # Annotate rendered view with file names. + # config.action_view.annotate_rendered_view_with_filenames = true + + #===================================================================================================================== + # Please keep custom config settings below this comment. + # This will ensure cleaner diffs when generating config file changes during Rails upgrades. + #===================================================================================================================== + + config.after_initialize do + Bullet.enable = false + Bullet.bullet_logger = true + Bullet.rails_logger = true + Bullet.raise = true + Bullet.unused_eager_loading_enable = false + end + + ENV["VA_DOT_GOV_API_URL"] = "https://sandbox-api.va.gov/" + + # For testing uncertification methods + ENV["TEST_USER_ID"] = "TEST_USER_ID" + ENV["TEST_APPEAL_IDS"] = "123C,456D,678E" + ENV["FULL_GRANT_IDS"] = "VACOLS123,VACOLS234,VACOLS345,VACOLS456" + ENV["PARTIAL_AND_REMAND_IDS"] = "VACOLS321,VACOLS432,VACOLS543,VACOLS654" + + ENV["AWS_ACCESS_KEY_ID"] ||= "dummykeyid" + ENV["AWS_SECRET_ACCESS_KEY"] ||= "dummysecretkey" + + # BatchProcess ENVs + # priority_ep_sync + ENV["BATCH_PROCESS_JOB_DURATION"] ||= "50" # Number of minutes the job will run for + ENV["BATCH_PROCESS_SLEEP_DURATION"] ||= "0" # Number of seconds between loop iterations + ENV["BATCH_PROCESS_BATCH_LIMIT"]||= "100" # Max number of records in a batch + ENV["BATCH_PROCESS_ERROR_DELAY"] ||= "3" # In number of hours + ENV["BATCH_PROCESS_MAX_ERRORS_BEFORE_STUCK"] ||= "3" # When record errors for X time, it's declared stuck + + # RequestIssue paginates_per offset (vbms intake) + ENV["REQUEST_ISSUE_PAGINATION_OFFSET"] ||= "10" + ENV["REQUEST_ISSUE_DEFAULT_UPPER_BOUND_PER_PAGE"] ||= "50" + + config.active_job.queue_adapter = :test + + # Disable SqlTracker from creating tmp/sql_tracker-*.json files -- https://github.com/steventen/sql_tracker/pull/10 + SqlTracker::Config.enabled = false + + # Setup S3 + config.s3_enabled = false + + config.vacols_db_name = "VACOLS_TEST" + + if ENV["TEST_SUBCATEGORY"] + assets_cache_path = Rails.root.join("tmp/cache/assets/#{ENV['TEST_SUBCATEGORY']}") + config.assets.configure do |env| + env.cache = Sprockets::Cache::FileStore.new(assets_cache_path) + end + end + + # Allows rake scripts to be run without querying VACOLS on startup + if ENV["DISABLE_FACTORY_BOT_INITIALIZERS"] + config.factory_bot.definition_file_paths = [] + end + + # VA Notify environment variables + ENV["VA_NOTIFY_API_URL"] ||= "https://staging-api.va.gov/vanotify" + ENV["VA_NOTIFY_API_KEY"] ||= "secret-key" + ENV["VA_NOTIFY_SERVICE_ID"] ||= "fake-service-id" + ENV["VA_NOTIFY_TOKEN_ALG"] ||= "my-secret-algorithm" + ENV["VA_NOTIFY_STATUS_UPDATE_BATCH_LIMIT"] ||= "650" + + # One time Appeal States migration for Legacy & AMA Appeal Batch Sizes + ENV["STATE_MIGRATION_JOB_BATCH_SIZE"] ||= "1000" + + # Quarterly Notifications Batch Sizes + ENV["QUARTERLY_NOTIFICATIONS_JOB_BATCH_SIZE"] ||= "1000" + + # Travel Board Sync Batch Size + ENV["TRAVEL_BOARD_HEARING_SYNC_BATCH_LIMIT"] ||= "250" + + # Time in seconds before the sync lock expires + LOCK_TIMEOUT = ENV["SYNC_LOCK_MAX_DURATION"] ||= "60" + + # Notifications page eFolder link + ENV["CLAIM_EVIDENCE_EFOLDER_BASE_URL"] ||= "https://vefs-claimevidence-ui-uat.stage.bip.va.gov" + + ENV['TEST_VACOLS_HOST'] ||= "localhost" + + # Pacman environment variables + ENV["PACMAN_API_TOKEN_ALG"] ||= "HS512" + ENV["PACMAN_API_URL"] ||= "https://pacman-uat.dev.bip.va.gov" + ENV["PACMAN_API_SAML_TOKEN"] ||= "our-saml-token" + ENV["PACMAN_API_TOKEN_SECRET"] ||= "client-secret" + ENV["PACMAN_API_TOKEN_ISSUER"] ||= "issuer-of-our-token" + ENV["PACMAN_API_SYS_ACCOUNT"] ||= "CSS_ID_OF_OUR_ACCOUNT" + + # Dynatrace variables + ENV["STATSD_ENV"] = "test" +end From d0cb5065271f2efb0339b682049bf7d6f15b4834 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 23 May 2024 18:40:56 -0400 Subject: [PATCH 51/69] =?UTF-8?q?=F0=9F=92=9A=20Remove=20redundant=20DB=20?= =?UTF-8?q?migrations=20from=20CI=20workflow?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task `db:schema:load` already loads the checked in schema, so there should be no need to run `db:migrate` afterwards. --- .github/workflows/workflow.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/workflow.yml b/.github/workflows/workflow.yml index 88958d6a15c..d7bd23611fa 100644 --- a/.github/workflows/workflow.yml +++ b/.github/workflows/workflow.yml @@ -192,8 +192,8 @@ jobs: - name: Database setup run: | - ./ci-bin/capture-log "bundle exec rake db:create:etl db:schema:load:etl db:migrate:etl" - ./ci-bin/capture-log "bundle exec rake db:create:primary db:schema:load:primary db:migrate:primary" + ./ci-bin/capture-log "bundle exec rake db:create:etl db:schema:load:etl" + ./ci-bin/capture-log "bundle exec rake db:create:primary db:schema:load:primary" ./ci-bin/capture-log "make -f Makefile.example external-db-create" # added line to create external table(s) that are needed for tests From a109182ff740b0cb7edc501b45a3fb859eba7f44 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 28 May 2024 13:55:36 -0400 Subject: [PATCH 52/69] =?UTF-8?q?=F0=9F=90=9B=20Fix=20`spec/mailers/hearin?= =?UTF-8?q?g=5Fmailer=5Fspec.rb`=20-=20`NoMethodError`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the following error: NoMethodError: undefined method `build_lookup_context' for ActionView::Base:Class --- app/services/hearings/calendar_service.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/app/services/hearings/calendar_service.rb b/app/services/hearings/calendar_service.rb index caf1936b7ab..aa27315fe58 100644 --- a/app/services/hearings/calendar_service.rb +++ b/app/services/hearings/calendar_service.rb @@ -100,8 +100,8 @@ def create_calendar_event(hearing) end def render_virtual_hearing_calendar_event_template(email_recipient_info, event_type, locals) - lookup_context = ActionView::Base.build_lookup_context(ActionController::Base.view_paths) - context = CalendarEventView.new(lookup_context) + lookup_context = ActionView::LookupContext.new(ActionController::Base.view_paths) + context = CalendarEventView.with_empty_template_cache.new(lookup_context, {}, nil) # Some *~ magic ~* here. The recipient title is used to determine which template to load: # From 4c16b293493fd9ef92574828fd551e058a9bdefb Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 28 May 2024 14:19:17 -0400 Subject: [PATCH 53/69] =?UTF-8?q?=F0=9F=90=9B=20Fix=20`spec/workflows/post?= =?UTF-8?q?=5Fdecision=5Fmotion=5Fupdater=5Fspec.rb`=20-=20`FrozenError`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the following error: FrozenError: can't modify frozen Hash: {} --- app/workflows/post_decision_motion_updater.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/app/workflows/post_decision_motion_updater.rb b/app/workflows/post_decision_motion_updater.rb index 70585da819e..45341485500 100644 --- a/app/workflows/post_decision_motion_updater.rb +++ b/app/workflows/post_decision_motion_updater.rb @@ -69,7 +69,7 @@ def create_motion end unless motion.valid? - errors.messages.merge!(motion.errors.messages) + errors.merge!(motion.errors) return end @@ -85,13 +85,13 @@ def handle_denial_or_dismissal abstract_task = create_abstract_task unless abstract_task.valid? - errors.messages.merge!(abstract_task.errors.messages) + errors.merge!(abstract_task.errors) return end new_task = create_denial_or_dismissal_task(abstract_task) unless new_task.valid? - errors.messages.merge!(new_task.errors.messages) + errors.merge!(new_task.errors) return end new_task.save @@ -132,7 +132,7 @@ def create_new_stream_tasks(stream) ) unless attorney_task.valid? - errors.messages.merge!(attorney_task.errors.messages) + errors.merge!(attorney_task.errors) return end From 225b777f272ac208a328630950a448ef1bb2364b Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Wed, 29 May 2024 15:37:51 -0400 Subject: [PATCH 54/69] =?UTF-8?q?=E2=9C=85=20Add=20test=20for=20`RoSchedul?= =?UTF-8?q?ePeriod`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- spec/factories/ro_schedule_period.rb | 2 +- spec/models/ro_schedule_period_spec.rb | 22 ++++++++++++++++++++++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/spec/factories/ro_schedule_period.rb b/spec/factories/ro_schedule_period.rb index b3ef91ac530..8746f031fa2 100644 --- a/spec/factories/ro_schedule_period.rb +++ b/spec/factories/ro_schedule_period.rb @@ -1,7 +1,7 @@ # frozen_string_literal: true FactoryBot.define do - before(:create) do + after(:build) do S3Service.store_file(SchedulePeriod::S3_SUB_BUCKET + "/" + "validRoSpreadsheet.xlsx", "spec/support/validRoSpreadsheet.xlsx", :filepath) S3Service.store_file(SchedulePeriod::S3_SUB_BUCKET + "/" + "blankRoSpreadsheet.xlsx", diff --git a/spec/models/ro_schedule_period_spec.rb b/spec/models/ro_schedule_period_spec.rb index 774902d3423..78af18d1563 100644 --- a/spec/models/ro_schedule_period_spec.rb +++ b/spec/models/ro_schedule_period_spec.rb @@ -7,6 +7,28 @@ subject { ro_schedule_period.validate_spreadsheet } it { is_expected.to be_truthy } + + context "when spreadsheet is invalid" do + let(:spreadsheet_errors) do + [ + HearingSchedule::ValidateRoSpreadsheet::RoDatesNotUnique.new("RoDatesNotUnique message"), + HearingSchedule::ValidateRoSpreadsheet::RoDatesNotInRange.new("RoDatesNotInRange message"), + HearingSchedule::ValidateRoSpreadsheet::RoDatesNotCorrectFormat.new("RoDatesNotCorrectFormat message"), + HearingSchedule::ValidateRoSpreadsheet::RoTemplateNotFollowed.new("RoTemplateNotFollowed message") + ] + end + + before do + allow_any_instance_of(HearingSchedule::ValidateRoSpreadsheet).to receive(:validate) { spreadsheet_errors } + end + + it "adds errors to model" do + model = build(:ro_schedule_period) + model.valid? + expect(model).to_not be_valid + expect(model.errors[:base]).to match_array([spreadsheet_errors]) + end + end end context "Allocate RO Days Per Given Schedule" do From fcaafbadb48eecf3c9ba6dadfb3bcdf413917c42 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Wed, 29 May 2024 17:59:18 -0400 Subject: [PATCH 55/69] =?UTF-8?q?=F0=9F=90=9B=20Fix=20`spec/models/schedul?= =?UTF-8?q?e=5Fperiod=5Fspec.rb`=20-=20`ActiveRecord::RecordInvalid`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Apparently, there were some changes to the inner workings of `ActiveModel::Errors` in Rails 6.1, causing a model to be considered invalid in the case that `errors[:base] == [[]]`. This makes sense, as `[[]]` is not considered "empty". Unfortunately, this was causing `RoSchedulePeriod #validate_spreadsheet` to inadvertently mark the model as invalid upon creation. `HearingSchedule::ValidateRoSpreadsheet #validate` returns an empty array (`[]`) when valid, which gets pushes onto the `RoSchedulePeriod` `errors[:base]` array, resulting in a non-empty array (`[[]]`) and an erroneously invalid disposition. Furthermore, calling `<<` to an `ActiveModel::Errors` message array in order to add an error is a deprecated, so we can take this opportunity to use the new `#add` API to hit two birds with one stone. The change implemented here is not a pure refactoring, however the end-user experience is unchanged in terms of how errors are presented when attempting to upload a spreadsheet with multiple non-conformities. Down the road, we may want to consider moving `HearingSchedule::ValidateRoSpreadsheet` toward using `ActiveModel::Validations` in order to leverage the full `ActiveModel::Errors` API and construct the errors object in the prescribed manner. For more details see - https://api.rubyonrails.org/v6.1.7.7/classes/ActiveModel/Validations.html - https://api.rubyonrails.org/v6.1.7.7/classes/ActiveModel/Errors.html --- app/models/ro_schedule_period.rb | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/app/models/ro_schedule_period.rb b/app/models/ro_schedule_period.rb index 3966864cb65..7fdbbf7cee0 100644 --- a/app/models/ro_schedule_period.rb +++ b/app/models/ro_schedule_period.rb @@ -17,8 +17,9 @@ class RoSchedulePeriod < SchedulePeriod # Run various validations on the uploaded spreadsheet and record errors def validate_spreadsheet - validate_spreadsheet = HearingSchedule::ValidateRoSpreadsheet.new(spreadsheet, start_date, end_date) - errors[:base] << validate_spreadsheet.validate + spreadsheet_errors = HearingSchedule::ValidateRoSpreadsheet.new(spreadsheet, start_date, end_date).validate + errors.add(:base, spreadsheet_errors) if spreadsheet_errors.any? + errors[:base] end # Create NonAvailibility records for ROs and CO and Allocation records for each RO From 7c6709439966bdaf57a480682c3d8ffb88cea7a4 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 30 May 2024 17:23:44 -0400 Subject: [PATCH 56/69] =?UTF-8?q?=F0=9F=90=9B=20Fix=20`spec/mailers/hearin?= =?UTF-8?q?g=5Fmailer=5Fspec.rb`=20-=20`ActionView::Template::Error`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- app/services/hearings/calendar_service.rb | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/app/services/hearings/calendar_service.rb b/app/services/hearings/calendar_service.rb index aa27315fe58..e84331fd085 100644 --- a/app/services/hearings/calendar_service.rb +++ b/app/services/hearings/calendar_service.rb @@ -114,8 +114,11 @@ def render_virtual_hearing_calendar_event_template(email_recipient_info, event_t template_name = "#{email_recipient_info.title.downcase}_#{event_type}_event_description" + absolute_filepath_for_template = + Dir.glob(Rails.root.join("app/views/hearing_mailer/calendar_events/*")).find { |p| p.match?(template_name) } + context.render( - template: "hearing_mailer/calendar_events/#{template_name}", + file: absolute_filepath_for_template, locals: locals ) end From 7812a537498c9b2574ba12683938aebd96f632b7 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Wed, 5 Jun 2024 09:50:41 -0400 Subject: [PATCH 57/69] =?UTF-8?q?=E2=9C=85=20Fix=20`spec/models/veteran=5F?= =?UTF-8?q?spec.rb`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- spec/models/veteran_spec.rb | 103 ++++++++++++++++++++++++------------ 1 file changed, 69 insertions(+), 34 deletions(-) diff --git a/spec/models/veteran_spec.rb b/spec/models/veteran_spec.rb index 7b3a4b5c875..c139c615f7e 100644 --- a/spec/models/veteran_spec.rb +++ b/spec/models/veteran_spec.rb @@ -316,18 +316,24 @@ end end - context "when veteran pay grade is invalid" do - subject { veteran.validate_veteran_pay_grade } - let(:service) do - [{ branch_of_service: "Army", - entered_on_duty_date: "06282002", - released_active_duty_date: "06282003", - pay_grade: "not valid", - char_of_svc_code: "TBD" }] - end + context "pay grade" do + context "when not in BGS pay grade list" do + let(:service) do + [{ branch_of_service: "Army", + entered_on_duty_date: "06282002", + released_active_duty_date: "06282003", + pay_grade: "not valid", + char_of_svc_code: "TBD" }] + end + + it "is invalid within 'bgs' context" do + expect(veteran.valid?(:bgs)).to eq(false) + expect(veteran.errors.messages_for(:pay_grades)).to eq ["invalid_pay_grade"] + end - it "pay grade invalid" do - expect(subject).to eq ["invalid_pay_grade"] + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end end end @@ -554,11 +560,18 @@ end end - context "when a zip code is invalid" do - let(:zip_code) { "1234" } + context "zip code" do + context "when not a valid zip code" do + let(:zip_code) { "1234" } + + it "is invalid within 'bgs' context" do + expect(veteran.valid?(:bgs)).to eq(false) + expect(veteran.errors.messages_for(:zip_code)).to eq ["invalid_zip_code"] + end - it "zip code has invalid characters" do - expect(veteran.validate_zip_code).to eq ["invalid_zip_code"] + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end end @@ -573,21 +586,35 @@ end end - context "given a military address with invalid city characters" do - let(:military_postal_type_code) { "AA" } - let(:city) { "ÐÐÐÐÐ" } - let(:state) { nil } + context "city" do + context "when a military address with invalid city characters" do + let(:military_postal_type_code) { "AA" } + let(:city) { "ÐÐÐÐÐ" } + let(:state) { nil } + + it "is invalid within 'bgs' context" do + expect(veteran.valid?(:bgs)).to eq(false) + expect(veteran.errors.messages_for(:city)).to eq ["invalid_characters"] + end - it "city is considered invalid" do - expect(veteran.validate_city).to eq ["invalid_characters"] + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end end - context "given date of birth is missing leading zeros" do - let(:date_of_birth) { "2/2/1956" } + context "date of birth" do + context "when missing leading zeros" do + let(:date_of_birth) { "2/2/1956" } + + it "is invalid within 'bgs' context" do + expect(veteran.valid?(:bgs)).to eq(false) + expect(veteran.errors.messages_for(:date_of_birth)).to eq ["invalid_date_of_birth"] + end - it "date_of_birth is considered invalid" do - expect(veteran.validate_date_of_birth).to eq ["invalid_date_of_birth"] + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end end @@ -599,22 +626,30 @@ end end - context "#validate_name_suffix" do - subject { veteran.validate_name_suffix } - let(:name_suffix) { "JR." } + context "name suffix" do + context "when containing a special character" do + let(:name_suffix) { "JR." } + + it "is invalid within 'bgs' context" do + expect(veteran.valid?(:bgs)).to eq false + expect(veteran.errors.messages_for(:name_suffix)).to eq(["invalid_character"]) + end - it "name_suffix is considered invalid" do - expect(subject).to eq ["invalid_character"] - expect(veteran.valid?(:bgs)).to eq false + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end - context "name_suffix nil" do + context "when nil" do let(:name_suffix) { nil } - it "name_suffix is considered valid" do - subject + it "is valid within 'bgs' context" do expect(veteran.valid?(:bgs)).to eq true end + + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end end From 27be82c5b5e33e1b170573777462e15130caa45e Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 7 Jun 2024 14:28:29 -0400 Subject: [PATCH 58/69] =?UTF-8?q?=E2=9C=85=20Fix=20`spec/sql/ama=5Fcases?= =?UTF-8?q?=5Fsql=5Fspec.rb`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses failures such as the below: 0) AMA Cases Tableau data source expected report calculates age and AOD based on person.dob Failure/Error: expect(aod_case["aod_veteran.age"]).to eq("76") expected: "76" got: 0.76e2 --- spec/sql/ama_cases_sql_spec.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/sql/ama_cases_sql_spec.rb b/spec/sql/ama_cases_sql_spec.rb index 6dbafeb8e1d..def32079c57 100644 --- a/spec/sql/ama_cases_sql_spec.rb +++ b/spec/sql/ama_cases_sql_spec.rb @@ -22,10 +22,10 @@ non_aod_case = result.find { |r| r["id"] == not_distributed.id } expect(aod_case["aod_is_advanced_on_docket"]).to eq(true) - expect(aod_case["aod_veteran.age"]).to eq("76") + expect(aod_case["aod_veteran.age"]).to eq(76) expect(non_aod_case["aod_is_advanced_on_docket"]).to eq(false) - expect(non_aod_case["aod_veteran.age"]).to eq("65") + expect(non_aod_case["aod_veteran.age"]).to eq(65) end end end From 01ee550157bd5b2592119310d7a3a5c620e395ac Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 7 Jun 2024 15:05:38 -0400 Subject: [PATCH 59/69] =?UTF-8?q?=E2=9C=85=20Fix=20multiple=20specs=20-=20?= =?UTF-8?q?`Minitest::UnexpectedError`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Test helper method `#perform_enqueued_jobs` now wraps exceptions in an `Minitest::UnexpectedError`: https://github.com/rails/rails/blob/914caca2d31bd753f47f9168f2a375921d9e91cc/activejob/lib/active_job/test_helper.rb#L591 So, to assert that a specific exception is raised during execution of the `#perform_enqueued_jobs` block, we must rescue the `Minitest::UnexpectedError` and make the assertion on its error message instead. --- spec/workflows/ama_appeal_dispatch_spec.rb | 5 ++++- spec/workflows/legacy_appeal_dispatch_spec.rb | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/spec/workflows/ama_appeal_dispatch_spec.rb b/spec/workflows/ama_appeal_dispatch_spec.rb index 62a1e336e61..425ab981695 100644 --- a/spec/workflows/ama_appeal_dispatch_spec.rb +++ b/spec/workflows/ama_appeal_dispatch_spec.rb @@ -70,7 +70,10 @@ it "does not call #perform_later on MailRequestJob" do allow(ProcessDecisionDocumentJob).to receive(:perform_later).and_raise(StandardError) expect(MailRequestJob).to_not receive(:perform_later) - expect { subject }.to raise_error(StandardError) + subject + rescue Minitest::UnexpectedError => error # rubocop:disable Lint/SuppressedException + ensure + expect(error.message).to match("StandardError") end end end diff --git a/spec/workflows/legacy_appeal_dispatch_spec.rb b/spec/workflows/legacy_appeal_dispatch_spec.rb index 81c40e35793..a1831d29f04 100644 --- a/spec/workflows/legacy_appeal_dispatch_spec.rb +++ b/spec/workflows/legacy_appeal_dispatch_spec.rb @@ -102,7 +102,10 @@ it "does not call #perform_later on MailRequestJob" do allow(ProcessDecisionDocumentJob).to receive(:perform_later).and_raise(StandardError) expect(MailRequestJob).to_not receive(:perform_later) - expect { subject }.to raise_error(StandardError) + subject + rescue Minitest::UnexpectedError => error # rubocop:disable Lint/SuppressedException + ensure + expect(error.message).to match("StandardError") end end end From fbd7682987ad7436ae712a5e2c4c8bd2e0ac5d99 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Mon, 10 Jun 2024 12:11:25 -0400 Subject: [PATCH 60/69] =?UTF-8?q?=E2=9C=85=20Fix=20`spec/lib/helpers/assoc?= =?UTF-8?q?iation=5Fwrapper=5Fspec.rb`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- spec/lib/helpers/association_wrapper_spec.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/lib/helpers/association_wrapper_spec.rb b/spec/lib/helpers/association_wrapper_spec.rb index 5acf5ae0cb1..86e157dc91b 100644 --- a/spec/lib/helpers/association_wrapper_spec.rb +++ b/spec/lib/helpers/association_wrapper_spec.rb @@ -91,7 +91,7 @@ [:attorney_case_reviews, false, false, "task_id", "attorney_case_review_id", nil], [:task_timers, false, false, "task_id", "task_timer_id", nil], # has_one declared in Task - [:cached_appeal, false, true, :appeal_id, "cached_appeal_id", nil] + [:cached_appeal, false, true, "appeal_id", "cached_appeal_id", nil] ] end end From d5fb249f7cbbfc3104fe89f68ea94aa5a028e598 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Mon, 10 Jun 2024 12:54:09 -0400 Subject: [PATCH 61/69] =?UTF-8?q?=E2=9C=85=20Fix=20`spec/controllers/api/v?= =?UTF-8?q?1/jobs=5Fcontroller=5Fspec.rb`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In Rails 6.1, `ActiveJob #perform_now` was changed to behave as it did once before (at the behest of GitHub), returning the value fo the job instead of true/false. See related GH issue: https://github.com/rails/rails/issues/38040 --- spec/controllers/api/v1/jobs_controller_spec.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/controllers/api/v1/jobs_controller_spec.rb b/spec/controllers/api/v1/jobs_controller_spec.rb index bd632f60f68..036aeb3bd4b 100644 --- a/spec/controllers/api/v1/jobs_controller_spec.rb +++ b/spec/controllers/api/v1/jobs_controller_spec.rb @@ -30,7 +30,7 @@ # needed to reach 90% test coverage it "should successfully run a job" do - expect(HeartbeatTasksJob.perform_now).to eq true + expect(HeartbeatTasksJob.perform_now).to eq 53 end end From 072c6e775dc2194c60cff71b9049ae0c16845742 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Mon, 10 Jun 2024 15:00:42 -0400 Subject: [PATCH 62/69] =?UTF-8?q?=F0=9F=90=9B=20Fix=20`spec/controllers/ap?= =?UTF-8?q?peals=5Fcontroller=5Fspec.rb`=20-=20`NoMethodError`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses error: NoMethodError: undefined method `workflow' for # 0) AppealsController GET appeals when current user is a System Admin when request header does not contain Veteran ID responds with an error Failure/Error: errors: errors.messages[:workflow], NoMethodError: undefined method `workflow' for # # ./app/workflows/case_search_results_base.rb:31:in `search_call' --- app/workflows/case_search_results_base.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/app/workflows/case_search_results_base.rb b/app/workflows/case_search_results_base.rb index ea30ccd739e..e0e400552be 100644 --- a/app/workflows/case_search_results_base.rb +++ b/app/workflows/case_search_results_base.rb @@ -3,6 +3,8 @@ class CaseSearchResultsBase include ActiveModel::Validations + attr_accessor :workflow + validate :vso_employee_has_access def initialize(user:) From 9a546f23b4d51003ea27a682cf0f649886e67058 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 11 Jun 2024 16:26:58 -0400 Subject: [PATCH 63/69] =?UTF-8?q?=F0=9F=90=9B=20Fix=20`CaseSearchResultsBa?= =?UTF-8?q?se`=20validations?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses test failures in `spec/controllers/appeals_controller_spec.rb` similar to the below: AppealsController GET appeals when current user is a System Admin when request header does not contain Veteran ID responds with an error Failure/Error: expect(response_body["errors"][0]["title"]).to eq "Veteran file number missing" expected: "Veteran file number missing" got: nil Using `ActiveModel::Errors` to store error data in an arbitrary format may have been somewhat permissible in the past, but it is an abuse of the object's intended use and is also proving incompatible with the more formalized `ActiveModels::Errors` API in Rails 6.1. In order to preserve the existing response shape of the affected JSON endpoints, we need to move away from the `ActiveModel::Validations` implementation on `CaseSearchResultsBase` (and its descendent classes) to a more bespoke method of performing validations and aggregating errors, since Rails 6.1 `ActiveModel::Errors` is no longer appropriate for our needs here. --- app/workflows/case_search_results_base.rb | 48 +++++++++++++++++-- ..._search_results_for_caseflow_veteran_id.rb | 6 ++- .../case_search_results_for_docket_number.rb | 6 ++- ..._search_results_for_veteran_file_number.rb | 10 ++-- 4 files changed, 57 insertions(+), 13 deletions(-) diff --git a/app/workflows/case_search_results_base.rb b/app/workflows/case_search_results_base.rb index e0e400552be..f8e4e1a0528 100644 --- a/app/workflows/case_search_results_base.rb +++ b/app/workflows/case_search_results_base.rb @@ -1,14 +1,35 @@ # frozen_string_literal: true class CaseSearchResultsBase - include ActiveModel::Validations + class AbstractMethodError < StandardError; end - attr_accessor :workflow + # @note Data Transfer Object (DTO) for encapsulating error data grouped by attribute + class Errors + attr_accessor :messages - validate :vso_employee_has_access + private :messages= + + delegate :none?, to: :messages + + def initialize + @messages = Hash.new([]) + end + + # @param attribute [Symbol] attribute the error is associated with + # @param title [String] summary of the error + # @param detail [String] detailed description of the error + # + # @example + # add(:workflow, title: "Access to Veteran file prohibited", + # detail: "You do not have access to this claims file number") + def add(attribute, title:, detail:) + messages[attribute.to_sym] += [{ title: title, detail: detail }] + end + end def initialize(user:) @user = user + @errors = Errors.new end def call @@ -101,8 +122,25 @@ def search_page_json_appeals(appeals) private + attr_accessor :errors attr_reader :success + def valid? + validate_vso_employee_has_access + validation_hook + errors.none? + end + + # @note (Optional) hook method to be implemented by subclasses for performing subclass-specific validations. + # @example + # def validation_hook + # unless foobar_has_bazbah? + # errors.add(:workflow, title: "Veteran file number missing", + # detail: "HTTP_CASE_SEARCH request header must include Veteran file number") + # end + # end + def validation_hook; end + def access?(file_number) !current_user_is_vso_employee? || bgs.can_access?(file_number) end @@ -111,7 +149,7 @@ def bgs @bgs ||= BGSService.new end - def veterans_exist + def validate_veterans_exist return unless veterans_user_can_access.empty? errors.add(:workflow, not_found_error) @@ -148,7 +186,7 @@ def case_search_results } end - def vso_employee_has_access + def validate_vso_employee_has_access return unless current_user_is_vso_employee? errors.add(:workflow, prohibited_error) if diff --git a/app/workflows/case_search_results_for_caseflow_veteran_id.rb b/app/workflows/case_search_results_for_caseflow_veteran_id.rb index 986b3dfdd4d..3fe2b5604df 100644 --- a/app/workflows/case_search_results_for_caseflow_veteran_id.rb +++ b/app/workflows/case_search_results_for_caseflow_veteran_id.rb @@ -1,8 +1,6 @@ # frozen_string_literal: true class CaseSearchResultsForCaseflowVeteranId < ::CaseSearchResultsBase - validate :veterans_exist - def initialize(caseflow_veteran_ids:, user:) super(user: user) @caseflow_veteran_ids = caseflow_veteran_ids @@ -18,6 +16,10 @@ def veterans attr_reader :caseflow_veteran_ids + def validation_hook + validate_veterans_exist + end + def not_found_error { "title": "Veteran not found", diff --git a/app/workflows/case_search_results_for_docket_number.rb b/app/workflows/case_search_results_for_docket_number.rb index af8ace8a2fb..6c16a50a46d 100644 --- a/app/workflows/case_search_results_for_docket_number.rb +++ b/app/workflows/case_search_results_for_docket_number.rb @@ -1,8 +1,6 @@ # frozen_string_literal: true class CaseSearchResultsForDocketNumber < ::CaseSearchResultsBase - validate :veterans_exist, if: :current_user_is_vso_employee? - def initialize(docket_number:, user:) super(user: user) @docket_number = docket_number.to_s if docket_number @@ -22,6 +20,10 @@ def appeals attr_reader :docket_number + def validation_hook + validate_veterans_exist if current_user_is_vso_employee? + end + def not_found_error { "title": "Docket ID not found", diff --git a/app/workflows/case_search_results_for_veteran_file_number.rb b/app/workflows/case_search_results_for_veteran_file_number.rb index 63cd79983b1..3f0f97fe466 100644 --- a/app/workflows/case_search_results_for_veteran_file_number.rb +++ b/app/workflows/case_search_results_for_veteran_file_number.rb @@ -1,9 +1,6 @@ # frozen_string_literal: true class CaseSearchResultsForVeteranFileNumber < ::CaseSearchResultsBase - validate :file_number_or_ssn_presence - validate :veterans_exist, if: :current_user_is_vso_employee? - def initialize(file_number_or_ssn:, user:) super(user: user) # Ensure we have a string made of solely numeric characters @@ -14,7 +11,12 @@ def initialize(file_number_or_ssn:, user:) attr_reader :file_number_or_ssn - def file_number_or_ssn_presence + def validation_hook + validate_file_number_or_ssn_presence + validate_veterans_exist if current_user_is_vso_employee? + end + + def validate_file_number_or_ssn_presence return if file_number_or_ssn errors.add(:workflow, missing_veteran_file_number_or_ssn_error) From 0062d25c01f93a23da0a50408d53eddd834d1737 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Wed, 12 Jun 2024 16:54:24 -0400 Subject: [PATCH 64/69] =?UTF-8?q?=E2=9C=85=20Fix=20`spec/controllers/appli?= =?UTF-8?q?cation=5Fcontroller=5Fspec.rb`=20--=20Cache-Control=20error?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the test failure below: ApplicationController no cache headers when toggle set sets Cache-Control etc Failure/Error: expect(response.headers["Cache-Control"]).to eq "no-cache, no-store" expected: "no-cache, no-store" got: "no-store" (compared using ==) # ./spec/controllers/application_controller_spec.rb:59:in `block (4 levels) in ' In Rails 6.1, the `no-store` directive is exclusive of any others that are set on the `Cache-Control` header, which makes sense given the specification https://datatracker.ietf.org/doc/html/rfc7234#section-3 This change was implemented in PR https://github.com/rails/rails/pull/39461 Since it no longer makese sense to set both `no-store` and `no-cache` directives, we will only set `no-store` here, as that is the stronger of the two. --- app/controllers/application_controller.rb | 2 +- spec/controllers/application_controller_spec.rb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/app/controllers/application_controller.rb b/app/controllers/application_controller.rb index c34519195e4..277121cf689 100644 --- a/app/controllers/application_controller.rb +++ b/app/controllers/application_controller.rb @@ -273,7 +273,7 @@ def set_no_cache_headers # https://stackoverflow.com/a/748646 def no_cache # :nocov: - response.headers["Cache-Control"] = "no-cache, no-store" + response.headers["Cache-Control"] = "no-store" response.headers["Pragma"] = "no-cache" response.headers["Expires"] = "Fri, 01 Jan 1990 00:00:00 GMT" # waaaay in the past # :nocov: diff --git a/spec/controllers/application_controller_spec.rb b/spec/controllers/application_controller_spec.rb index 44d392ed3cf..6feb3e44749 100644 --- a/spec/controllers/application_controller_spec.rb +++ b/spec/controllers/application_controller_spec.rb @@ -56,7 +56,7 @@ def index it "sets Cache-Control etc" do get :index - expect(response.headers["Cache-Control"]).to eq "no-cache, no-store" + expect(response.headers["Cache-Control"]).to eq "no-store" end end end From ca6a9d2ee92b6b621c9fb899111c7c87e5542126 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 21 Jun 2024 15:34:00 -0400 Subject: [PATCH 65/69] =?UTF-8?q?=F0=9F=90=9B=20Fix=20multiple=20specs=20-?= =?UTF-8?q?=20`ActiveRecord::EagerLoadPolymorphicError`?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses multiple test failures caused by the error below: QueueConfig.to_hash title when assigned to an org is formatted as expected Failure/Error: tasks.with_assignees.group("assignees.display_name").count(:all).each_pair.map do |option, count| label = self.class.format_option_label(option, count) self.class.filter_option_hash(option, label) end ActiveRecord::EagerLoadPolymorphicError: Cannot eagerly load the polymorphic association :appeal # ./app/models/queue_column.rb:110:in `assignee_options' --- app/models/queue_column.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/models/queue_column.rb b/app/models/queue_column.rb index b848c345925..750639de49a 100644 --- a/app/models/queue_column.rb +++ b/app/models/queue_column.rb @@ -107,7 +107,7 @@ def task_type_options(tasks) end def assignee_options(tasks) - tasks.with_assignees.group("assignees.display_name").count(:all).each_pair.map do |option, count| + tasks.with_assignees.group("display_name").count(:all).each_pair.map do |option, count| label = self.class.format_option_label(option, count) self.class.filter_option_hash(option, label) end From 5beba4a739ba09e4fcdcfca619e768a157e10f9b Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 25 Jun 2024 17:03:04 -0400 Subject: [PATCH 66/69] =?UTF-8?q?=F0=9F=90=9B=20Fix=20`spec/models/task=5F?= =?UTF-8?q?spec.rb`=20-=20`update=5Fall`=20clears=20query=20cache?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In Rails 6.1.7.7, the method `ActiveRecord::Relation #update_all` will now clear any records cached by the calling relation. This was altering the behavior of `Task #cancel_task_and_child_subtasks` and causing the following test failure: Task#cancel_task_and_child_subtasks cancels all tasks and child subtasks Failure/Error: expect(second_level_tasks[0].versions.count).to eq(initial_versions + 2) expected: 3 got: 2 (compared using ==) # ./spec/models/task_spec.rb:368:in `block (3 levels) in ' To remedy, we will now cache the necessary Task records in an Array, which can be used for generating PaperTrail versions both before and after the `update_all`. --- app/models/task.rb | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/app/models/task.rb b/app/models/task.rb index f74de948c52..b2244e0f1b0 100644 --- a/app/models/task.rb +++ b/app/models/task.rb @@ -724,15 +724,14 @@ def previous_task def cancel_task_and_child_subtasks # Cancel all descendants at the same time to avoid after_update hooks marking some tasks as completed. # it would be better if we could allow the callbacks to happen sanely - descendant_ids = descendants.pluck(:id) # by avoiding callbacks, we aren't saving PaperTrail versions # Manually save the state before and after. - tasks = Task.open.where(id: descendant_ids) + tasks = Task.open.where(id: descendants).to_a transaction do tasks.each { |task| task.paper_trail.save_with_version } - tasks.update_all( + Task.where(id: tasks).update_all( status: Constants.TASK_STATUSES.cancelled, cancelled_by_id: RequestStore[:current_user]&.id, closed_at: Time.zone.now From b1fd1fc8a520fa423dde668933480bbfb738c35d Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Thu, 27 Jun 2024 13:17:07 -0400 Subject: [PATCH 67/69] =?UTF-8?q?=F0=9F=90=9B=20Fix=20`spec/services/heari?= =?UTF-8?q?ngs/calendar=5Fservice=5Fspec.rb`=20-=20template=20rendering=20?= =?UTF-8?q?error?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the following test failure: Hearings::CalendarService.confirmation_calendar_invite returns appropriate iCalendar event Failure/Error: expect(ical_event.description).to eq(expected_description) expected: "You're scheduled for a virtual hearing with a Veterans Law Judge of the Board of Veterans' Appeals.\...to reschedule or cancel your virtual hearing, contact us by email at bvahearingteamhotline@va.gov\n" got: # --- app/services/hearings/calendar_service.rb | 30 +++-------------------- 1 file changed, 4 insertions(+), 26 deletions(-) diff --git a/app/services/hearings/calendar_service.rb b/app/services/hearings/calendar_service.rb index e84331fd085..8a9be76d602 100644 --- a/app/services/hearings/calendar_service.rb +++ b/app/services/hearings/calendar_service.rb @@ -8,11 +8,6 @@ # emails. class Hearings::CalendarService - class CalendarEventView < ActionView::Base - include Hearings::CalendarTemplateHelper - include Hearings::AppellantNameHelper - end - class << self # Sent when first switching a video hearing to a virtual hearing, # and also when the scheduled time for an existing virtual hearing @@ -100,27 +95,10 @@ def create_calendar_event(hearing) end def render_virtual_hearing_calendar_event_template(email_recipient_info, event_type, locals) - lookup_context = ActionView::LookupContext.new(ActionController::Base.view_paths) - context = CalendarEventView.with_empty_template_cache.new(lookup_context, {}, nil) - - # Some *~ magic ~* here. The recipient title is used to determine which template to load: - # - # judge_confirmation_event_description - # representative_confirmation_event_description - # veteran_confirmation_event_description - # - # representative_changed_to_video_event_description - # veteran_changed_to_video_event_description - - template_name = "#{email_recipient_info.title.downcase}_#{event_type}_event_description" - - absolute_filepath_for_template = - Dir.glob(Rails.root.join("app/views/hearing_mailer/calendar_events/*")).find { |p| p.match?(template_name) } - - context.render( - file: absolute_filepath_for_template, - locals: locals - ) + template_name = + "hearing_mailer/calendar_events/#{email_recipient_info.title.downcase}_#{event_type}_event_description" + + ApplicationController.render(template: template_name, locals: locals) end end end From 78408b5435842b4af75f277f01ff25cde3babe96 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Tue, 9 Jul 2024 12:27:06 -0400 Subject: [PATCH 68/69] =?UTF-8?q?=F0=9F=90=9B=20Fix=20YAML=20syntax=20erro?= =?UTF-8?q?r=20caused=20by=20whitespace=20in=20ENV=20var?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address the following error, found during demo deployment: rake aborted! Cannot load database configuration: YAML syntax error occurred while parsing /caseflow/config/database.yml. Please note that YAML must be consistently indented using spaces. Tabs are not allowed. Error: (): could not find expected ':' while scanning a simple key at line 49 column 5 --- docker-bin/env.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docker-bin/env.sh b/docker-bin/env.sh index 80735dd9ab2..6a1acb1d207 100644 --- a/docker-bin/env.sh +++ b/docker-bin/env.sh @@ -13,8 +13,7 @@ export REDIS_URL_SIDEKIQ=redis://appeals-redis:6379 # envs to make development work in docker without affecting other devs export DOCKERIZED=true export DEMO_PORT=1521 -export DEMO_DB="(DESCRIPTION= - (ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=VACOLS_DB-development)(PORT=1521)))(RECV_TIMEOUT=120)(SEND_TIMEOUT=5)(CONNECT_DATA=(SID=BVAP)))" +export DEMO_DB="(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=VACOLS_DB-development)(PORT=1521)))(RECV_TIMEOUT=120)(SEND_TIMEOUT=5)(CONNECT_DATA=(SID=BVAP)))" export PATH=/.yarn/bin:/.config/yarn/global/node_modules/.bin:/usr/local/nvm/versions/node/v16.16.0/bin:/usr/local/bundle/bin:/usr/local/bundle/gems/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/oracle/instantclient_12_2: From e45190aea668df054ad976e3078d5e416ef215b4 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 26 Jul 2024 13:05:32 -0400 Subject: [PATCH 69/69] =?UTF-8?q?=E2=AC=86=EF=B8=8F=20Update=20`caseflow-c?= =?UTF-8?q?ommons`=20dependency=20to=20latest=20ref?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Removes `bourbon` and `neat` dependencies. --- Gemfile | 5 +---- Gemfile.lock | 4 ++-- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/Gemfile b/Gemfile index bddfeb97f36..92e60188bc8 100644 --- a/Gemfile +++ b/Gemfile @@ -17,10 +17,7 @@ gem "bgs", git: "https://github.com/department-of-veterans-affairs/ruby-bgs.git" gem "bootsnap", require: false gem "browser" gem "business_time", "~> 0.9.3" - -# TODO: lock caseflow gem to latest ref on master after merging https://github.com/department-of-veterans-affairs/caseflow-commons/pull/225 -gem "caseflow", git: "https://github.com/department-of-veterans-affairs/caseflow-commons", branch: "AlecK/APPEALS-43566" - +gem "caseflow", git: "https://github.com/department-of-veterans-affairs/caseflow-commons", ref: "9bd3635fbd8094d25160669f38d8699e2f1d7a98" gem "connect_mpi", git: "https://github.com/department-of-veterans-affairs/connect-mpi.git", ref: "a3a58c64f85b980a8b5ea6347430dd73a99ea74c" gem "connect_vbms", git: "https://github.com/department-of-veterans-affairs/connect_vbms.git", ref: "9807d9c9f0f3e3494a60b6693dc4f455c1e3e922" gem "console_tree_renderer", git: "https://github.com/department-of-veterans-affairs/console-tree-renderer.git", tag: "v0.1.1" diff --git a/Gemfile.lock b/Gemfile.lock index dd7640b96f9..f2db3b52b7f 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -9,8 +9,8 @@ GIT GIT remote: https://github.com/department-of-veterans-affairs/caseflow-commons - revision: 4c04f19bd6703d9b8b240cadad36455db9cedae8 - branch: AlecK/APPEALS-43566 + revision: 9bd3635fbd8094d25160669f38d8699e2f1d7a98 + ref: 9bd3635fbd8094d25160669f38d8699e2f1d7a98 specs: caseflow (0.4.8) aws-sdk-s3