Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

APPEALS-56649, APPEALS-56527, APPEALS-51865: MST bug fixes and Developer Command Line Tools #22688

Conversation

HunJerBAH
Copy link
Contributor

@HunJerBAH HunJerBAH commented Sep 3, 2024

Resolves
APPEALS-56449
APPEALS-56527
APPEALS-51865

Description

APPEALS-56449: created schema migration that removes not-null constraints on the SpecialIssuesTable.

APPEALS-56527: sets MST/PACT status to false for unidentified issues

APPEALS-51865
Created command line development tools outlined in this Wiki

git clean-pr
git clean-push
git codeclimate

Acceptance Criteria

  • Code compiles correctly

Testing Plan

  1. Go to Jira Issue/Test Plan Link or list them below
  • For feature branches merging into master: Was this deployed to UAT?

Frontend

User Facing Changes

  • Screenshots of UI changes added to PR & Original Issue
BEFORE AFTER

Storybook Story

For Frontend (Presentation) Components

  • Add a Storybook file alongside the component file (e.g. create MyComponent.stories.js alongside MyComponent.jsx)
  • Give it a title that reflects the component's location within the overall Caseflow hierarchy
  • Write a separate story (within the same file) for each discrete variation of the component

Backend

Database Changes

Only for Schema Changes

  • Add typical timestamps (created_at, updated_at) for new tables
  • Update column comments; include a "PII" prefix to indicate definite or potential PII data content
  • Have your migration classes inherit from Caseflow::Migration, especially when adding indexes (use add_safe_index) (see Writing DB migrations)
  • Verify that migrate:rollback works as desired (change supported functions)
  • Perform query profiling (eyeball Rails log, check bullet and fasterer output)
  • For queries using raw sql was an explain plan run by System Team
  • Add appropriate indexes (especially for foreign keys, polymorphic columns, unique constraints, and Rails scopes)
  • Run make check-fks; add any missing foreign keys or add to config/initializers/immigrant.rb (see Record associations and Foreign Keys)
  • Add belongs_to for associations to enable the schema diagrams to be automatically updated
  • Document any non-obvious semantics or logic useful for interpreting database data at Caseflow Data Model and Dictionary

Integrations: Adding endpoints for external APIs

  • Check that Caseflow's external API code for the endpoint matches the code in the relevant integration repo
    • Request: Service name, method name, input field names
    • Response: Check expected data structure
    • Check that calls are wrapped in MetricService record block
  • Check that all configuration is coming from ENV variables
    • Listed all new ENV variables in description
    • Worked with or notified System Team that new ENV variables need to be set
  • Update Fakes
  • For feature branches: Was this tested in Caseflow UAT

Best practices

Code Documentation Updates

  • Add or update code comments at the top of the class, module, and/or component.

Tests

Test Coverage

Did you include any test coverage for your code? Check below:

  • RSpec
  • Jest
  • Other

Code Climate

Your code does not add any new code climate offenses? If so why?

  • No new code climate issues added

Monitoring, Logging, Auditing, Error, and Exception Handling Checklist

Monitoring

  • Are performance metrics (e.g., response time, throughput) being tracked?
  • Are key application components monitored (e.g., database, cache, queues)?
  • Is there a system in place for setting up alerts based on performance thresholds?

Logging

  • Are logs being produced at appropriate log levels (debug, info, warn, error, fatal)?
  • Are logs structured (e.g., using log tags) for easier querying and analysis?
  • Are sensitive data (e.g., passwords, tokens) redacted or omitted from logs?
  • Is log retention and rotation configured correctly?
  • Are logs being forwarded to a centralized logging system if needed?

Auditing

  • Are user actions being logged for audit purposes?
  • Are changes to critical data being tracked ?
  • Are logs being securely stored and protected from tampering or exposing protected data?

Error Handling

  • Are errors being caught and handled gracefully?
  • Are appropriate error messages being displayed to users?
  • Are critical errors being reported to an error tracking system (e.g., Sentry, ELK)?
  • Are unhandled exceptions being caught at the application level ?

Exception Handling

  • Are custom exceptions defined and used where appropriate?
  • Is exception handling consistent throughout the codebase?
  • Are exceptions logged with relevant context and stack trace information?
  • Are exceptions being grouped and categorized for easier analysis and resolution?

HunJerBAH and others added 4 commits August 28, 2024 14:01
* APPEALS-50693: Create Bash Script to Run Tests on Git Commit and PR Review

* APPEALS-51866: Added CodeClimate CLI command to custom git actions (#22193)

* added command line option to run codeclimate locally

* added code climate option to clean-pr command

* created code climate command to run tests

* added code climate UI output for user.

* APPEALS-52246: Enable Git Test Commands to Autofix Issues (#22457)

* created auto fix option for git clean-pr

* changed variable name for autofix

* created autofix option for clean push

* added way to run feature tests headless in git clean-pr

* fixed headless test command

* increased code climate timeout for entire project to 1 hour from 15 mins
@HunJerBAH HunJerBAH changed the base branch from master to release/FY24Q4.4.0 September 3, 2024 13:23
* Adding skips for tests

* Trigger Build

* Unskip tests for troubleshooting

* Have SqsService spec use its own queue

* Add back in DB persistence across threads in SyncDecidedAppealsHelper spec

* Set current_user in each parallel thread

* Wrap threads in executors

---------

Co-authored-by: Matthew Thornton <ThorntonMatthew@users.noreply.github.com>
@raymond-hughes raymond-hughes marked this pull request as ready for review September 3, 2024 13:26
@raymond-hughes raymond-hughes marked this pull request as draft September 3, 2024 13:27
@HunJerBAH HunJerBAH marked this pull request as ready for review September 3, 2024 13:27
craigrva and others added 2 commits September 3, 2024 08:27
* Calvin/APPEALS-43852-cavc-levers (#21441)

* enabled cavc affinity levers in UI

* updated rspec

* [APPEALS-43849]Update CAVC Affinity Implementation for AMA Dockets to… (#21456)

* [APPEALS-43849]Update CAVC Affinity Implementation for AMA Dockets to Account for Omit/Infinity

* Affinity rules applied to non genpop

* Addressed comments

* Addressed comments

* fixed rubocop issues + added clarity to where clause

---------

Co-authored-by: Calvin <Calvin.Costa@va.gov>

* APPEALS-44956: Add AppealAffinity model and database table (#21526)

* add migration for appeal_affinities

* add AppealAffinity model and associations, update migration for new column

* update index to be unique

* add factory, add tests

* add factory traits to appeal and case for appeal affinities

* add combination trait to appeal factory

* add appeal_affinity to skipped associations in ETL reporting

* add a validation, test

* Craig/appeals 44958 (#21564)

* add new job, update affinity model validation and after save hook

* add update from push job

* fix job extending distribution scopes

* add with appeal affinities to distribution scopes

* typo

* add error handling, add test file

* add distributed case factory, refactor naming in job

* fix factories, added tests

* fix migration for null affinity start date column

* fixes, added tests

* more test updates

* add return in job if no query results, tests for no query results

* add test for after_save hook adding dist task instructions

* set start dist job to queue affinity job after running

* fix update job and start dist job spec

* queue affinity update job from push job

* code clarity

* fix judge in seed file

* remove comment, fix hearing factory, disable some seeds for testing

* add more tests

* test refactor

* update appeals for dist query to add affinity start, add seed file, fix hearing factory, add stat to dist factory

* disable new seed on reset

* update seed file with vet names, add another seed category

* fix distirbuted case factory?

* actually fix GHA runs

* lint, test fixes

* change constants in new job

* [Appeals 43850] Update Legacy Docket Queries to Account for the Previous Decision Judge and Type Action  (#21556)

* test changes for seans ticket

* test changes

* added joins to all required methods

* fixed lint

* fixed column ambiguity errors

* cleaned up naming scheme

* Documentation for JOIN_PREVIOUS_APPEALS constant

---------

Co-authored-by: Calvin <Calvin.Costa@va.gov>

* APPEALS-44959: Modify affinity date checks to use appeal_affinity (#21611)

* swap distribution queries from distribution_task to appeal_affinities

* update seed files to use appeal affinities instead of distribution task

* clean up seed file method names

* add missing Timecop.return in ama affinity case seed

* fix name of a method in a seed file

* remove references to distribution task in distribution scopes

* fix push priority job tests

* fix naming of args in one of the seed files

* fix user seed, fix date format in distribution task instructions

* fix tests for date format update

* APPEALS-44187: Factory Bot Additions (#21438)

* AC1: values for bfddec and bfmpro

* AC2: case issues updated to '3'

* AC3-4: attorney and judge additions

* ac 5: bfdpdcn addition

* AC6: case type action addition

* ac7: new folder match to original

* ac8: case issues set to original

* AC 3/4: added associations to original

* ac3/4: updated logic to handle no args

* ac3/4: return sattyid

* ac7:updating folder assignment

* ac7: added bfkey to except block

* ac7/8: update to case issue list and validations dismissed

* removed byebug

* ac7: added 'ticknum' to except

* lint fixes

* lint fixes

* lint fixes

* lint fixes

* nested trait into form_9 factory

* new addtions

* added .save to case issues

* resolving correspondent and titrnum associations

* fixed bfdc typo

* factory additions

* added ssn to associated corr.

* removed transient and added .save

* added after create to corr factory

* veteran lookup check prior to create

* committing missed 'end'

* moved over veteran create to case fact.

* move corr. association field to case fact.

* lint issues + corres. save

* Calvin/APPEALS-44957-rake-affinity (#21577)

* grabbed receipt dates from distributed cases

* refactored for functionality + added method to grab appeals that match

* using receipt date, get all related appeals

* added update/creation plus cleaned prior imple.

* gets most recent distributed case receipt_date

* skips if receipt_date is nil for performance

* if appeal affinity is nil, it will now be updated

* created spec file

* fixed non ready appeals

* updated query to match new AC

* removing comment

* testing for each docket

* updated spec file

* added new tests to rspec

* updated start date to receipt date instead of Time.now

* fixed date/time rspec errors

* added rails logger to know when rake task has finished

* added tag for rails log

* removed nonpriority dockets for direct_review and evidence_submission

* fixed lint issue

* fixed flaky spec test

* limits distributed cases query to within the last week

* APPEALS-46016: Add Affinity Start Date to the Explain Page (#21660)

* add affinity start date to explain page

* add feature test to verify dates display

* Sudhir/appeals 43851 (#21613)

* Implement CAVC + AOD Affinity Lever for AMA Dockets

* addressed comments

* addressed comments

* Addressed comments

* added cavc_aod_affinity in case distribution lever model

* addressed comments

* addressed comments

* updated specs

* Updated specs

* specs changes reverted

* ama_aod and ama_non_aod queries updated

* change the assertion in docket spec

* Craig/appeals 46196 (#21689)

* fix query, tested locally

* add basic test to verify csv downloads aren't broken

* APPEALS-43851: Add test to validate CAVC+AOD behavior on hearing docket appeals affinities (#21678)

* add test to validate cavc+aod on hearing docket appeals

* lint, test case_docket_spec fix

* modify case_docket_spec again

* more test fix testing

* attempt to fix test again

* test removing prev appeals from nonpriority queries

* more test tests

* feature toggle change in test

* reorder new portions of query

* remove unused portions of queries in case_docket

* revert unneeded change to query order

* revert unneeded change to query order

* update rake task and spec (#21731)

* APPEALS-46325: Add Seeds for AOD Appeals and Update Dates to Match CAVC (#21730)

* add aod hearing cases to ama affinity cases seed

* fix lever spec

* APPEALS-45148: Hook to clear saved affinity date (#21623)

* initial imp. idea

* AC1: check for affinity_start_date on assignment

* AC2/3: update affinity start date  w/ instr.

* updates to naming, instructions, and hook logic

* updates after review

* rspec coverage and addtional condition

* removed unused identifier

* removed reduntant 'self's

* added update on actual AA record

* updated to save aa record and addtional rspec

* added change to assignment on no record test

* check for assignment

* addd update to 'on_hold' status

* public method to handle legacy affinity appeals

* added .reload to :with_affinity_appeal

* added .reload to :ready_for_distribution

* updates to pass explain_spec

* switched boolean values

* typo

* readujsted order on :create for affinity appeal

* removed after(:create)

* testing rspec by readding after :create

* reloading in assertation

* addressing lint errors

* fix seeds/users_spec

* add case dist lever to new tests (#21776)

* fix tests, add lever to factory, fix dist scopes (#21779)

* fix rubocop warning

* Acd/appeals 43853 43854 (#21971)

* Calvin/appeals 43853 (#21723)

* initial updates

* removing unnecessary variable

* focused in on priority

* removing non priority stuff

* added general comments

* added BFAC and AOD to cavc aod lever query

* adding judge vacols id to query

* aod affinity_start_date filter initial changes

* fixed sorting

* fixed rubocop issues

* updated filter method

* error handling

* added ineligibility to queries for PREV_DECIDING_JUDGE

* fixed SQL query + added comments

* added exclude from affinity check into the case docket queries

* error handling + fixing sql queries

* rejects appeals without affinity_start_dates and nonmatching judges

* fixing rubocop offenses

* fixed inconsistencies between methods

* fixed conditions for rejecting appeals

* refactored cavc aod affinity filter to make it much easier to read

* refactored code to account for AC6

* error handling for empty exclude from affinity

* reverted next if block to old logic to ensure it works

* added PREV_DEC_JUDGE is not null

* case.rb factory changes

* added more options to legacy_cavc_appeal creation

* cleaned up code for simpler reading

* fix for aod legacy cavc creation

* added tied to option to legacy cavc appeal factory

* limits are now handled correctly in query

* replaced return false to next if, as return false was causing unexpected behavior

* fix rspecs + one edge case

* added cavc aod lever creation to rspecs

* removed bfac and aod from nonpriority query

* cavc aod appeals w/excluded judges are now properly being filtered

* refactored excluded judges check

* added to old query to fix rspec errors

* modified case factory bot

* query now handles when prev_deciding_judge is nil

* removed unnecessary condition

* fixed case factory to now have tied_to attatched to orig appeal

* fixed next if block within filter

* handles omit scenarios + correctly rejects with next

* working on rspec (still failing)

* fix for ineligble VLJ when infinite

* fixed rspec suite for cavc aod filters

* fixed omit scenario in cavc aod affinity filter

* consolidation & readability refactor

* rubocop fixes

* fixed spec error

* Implement CAVC Affinity for Legacy Docket (#21706)

* Implement CAVC Affinity for Legacy Docket

* addressed comments

* Added BFAC in the query

* code changes for affinitty date

* Added affinity code

* code refactor and removed non priority code changes

* fixed syntax change

* Addressed comments

* refactor cavc affinity filter

* refactored code

* code refactor

* code refactor

* Updated existing specs

* code refactor

* Added new rspecs

* code refactor and added test  cases

* code refactor

* added test cases

* fixes push_priority_spec

* fixed rubocop issues

* rubocop issue fixed

* refactored code to make it easier to understand

* refactored + fixed rspec and lint errors

---------

Co-authored-by: Calvin <Calvin.Costa@va.gov>
Co-authored-by: calvincostaBAH <108481161+calvincostaBAH@users.noreply.github.com>

* basic creation of legacy affinity cases seed data

* bug fixes, added bfcorlid with veterans, fully runs now

* added new appeals for affinity_and_tied_to_judge

* made data have realistic bfcorlids

* changes document sequence to use less digits

* added new file numbers for tied_to cases to make them easier to identify

---------

Co-authored-by: samasudhirreddy <108430298+samasudhirreddy@users.noreply.github.com>

* APPEALS-50692: Update Appeals Ready to Distribute CSV to include CAVC remand original judge (#22070)

* CSV download functional

* add tests, fix CSV query in CaseDocket

* fix lint

* Calvin/appeals 44313 (#22119)

* initial seed data file

* added legacy cavc and cavc aod affinity cases

* update

* fixed tied to for legacy appeals

* added AOD versions of appeals

* small lint fixes

* ensured AOD cases for legacy hearings with exluded or ineligible judge

* added vacols staff record creation for users without it

* APPEALS-47741: Update the UpdateAppealAffinityDatesJob to add appeal_affinity records for Legacy Appeals (#22023)

* AC1: changes and respective tests

* adjustements after refactoring

* identifier mismatch

* name update

* name update

* added appeal affinity filter

* updated comment

* remove byebug

* update rspec to handle hash input

* added no start date test case

* updated process method test

* removed 'todo' comment

* dried up query string

* aligned conditions

* update to hash quotations

* update to hash quotations

* added legacy to priority receipt dates from dist.

* moved append to resulting list

* added legacy receipt date to push job hash

* uncomment call to legacy

* updated dist.id to @dist_id

* uncomment call to process legacy appeals

* handling update to legacy docket type

* current rspec status

* fixed typo

* fix rspec

* legacy spec additions

* legacy spec additions

* added legacy dist. case factory

* removed vacols_judge ref

* updates for spec

* final review

* removed comment

* rubocop fixes

* fix rubocop warnings (#22225)

* Fix rubocop and tests (#22231)

* Calvin/appeals 52551 (#22293)

* age_of_n_oldest_priority_appeals_available_to_judge time out changes

* fixed timeout issue for distribute_priority_appeals

* fixed lint errors

* Craig/case docket optimization (#22294)

* age_of_n_oldest_priority_appeals_available_to_judge time out changes

* fixed timeout issue for distribute_priority_appeals

* optimize case docket priority distribution methods

* fix das deprecation distribution spec

---------

Co-authored-by: Calvin <Calvin.Costa@va.gov>

* Update admin_ui_spec.rb for CAVC levers being enabled

* APPEALS-53993: Update Implementation of Legacy Appeals with Hearing Held (#22473)

* implement new hearing requirement and test

* first pass at seed data for testing

* add new case to test and seed, fix typo and comments

* update for new 8.2.1.1 requirement

* fix rubocop warning

* fix error in seed file

* add new affinity scenario to seed data

* make some seed methods private

* fix rubocop warnings

---------

Co-authored-by: calvincostaBAH <108481161+calvincostaBAH@users.noreply.github.com>
Co-authored-by: samasudhirreddy <108430298+samasudhirreddy@users.noreply.github.com>
Co-authored-by: Calvin <Calvin.Costa@va.gov>
Co-authored-by: seanrpa <155660052+seanrpa@users.noreply.github.com>
Co-authored-by: Isaiah Saucedo <irsaucedo5@gmail.com>
Copy link

codeclimate bot commented Sep 3, 2024

Code Climate has analyzed commit 29f35e8 and detected 15 issues on this pull request.

Here's the issue category breakdown:

Category Count
Complexity 9
Duplication 6

View more on Code Climate.

@raymond-hughes raymond-hughes merged commit 84c1651 into release/FY24Q4.4.0 Sep 3, 2024
16 of 17 checks passed
@HunJerBAH HunJerBAH deleted the combo/APPEALS-56649/APPEALS-56527/APPEALS-51865 branch October 15, 2024 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants