V4 baseline (#160)

* Fixed SDC loading to work with newer DRC measures. And cherry-pick work from master. (#39)

* Fixed SDC loading to work with newer DRC measures.
 - Fixed issues with most data criteria getting thrown out.

* Bring over dependabot nokogiri update and the simplexml_parser removal from #30.
[Security] Update nokogiri requirement from ~> 1.8.5 to >= 1.8.5, < 1.11.0
Updates the requirements on [nokogiri](https://github.com/sparklemotion/nokogiri) to permit the latest version.
- [Release notes](https://github.com/sparklemotion/nokogiri/releases)
- [Changelog](https://github.com/sparklemotion/nokogiri/blob/master/CHANGELOG.md)
- [Commits](sparklemotion/nokogiri@v1.8.5...v1.10.3)

* Add the hqmf identifier to a statement reference (#25)

* Port ratio/proportional cv fix from hds and add tests (#48)

* codeListId and hqmfOid are both needed for sdc uniqueness

* Add descriptive error message if model cannot be found

* 2019 standards update (#63)

* 2019 standards update entry point fix (#54)
* fixed gem entry point file to be named properly
* fix issue with loading api uploaded files (#55)
* [Security] Bump nokogiri from 1.10.3 to 1.10.4
* Bump cqm-models version to 3.0.0

* [Security] Update rubyzip requirement from ~> 1.2.2 to >= 1.2.2, < 2.1.0 (#67)

* [Security] Update rubyzip requirement from ~> 1.2.2 to >= 1.2.2, < 2.1.0

Updates the requirements on [rubyzip](https://github.com/rubyzip/rubyzip) to permit the latest version.
- [Release notes](https://github.com/rubyzip/rubyzip/releases)
- [Changelog](https://github.com/rubyzip/rubyzip/blob/master/Changelog.md)
- [Commits](rubyzip/rubyzip@v1.2.3...v2.0.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Updated rubyzip dependency to be less than version 2.x, which requires ruby 2.4

* BONNIE-593 Bonnie Unresponsive Message and Error Loading Measure Packages

* BONNIE-593 Bonnie Unresponsive Message and Error Loading Measure Packages

* BONNIE-593 Bonnie Unresponsive Message and Error Loading Measure Packages

* Bonnie-593(ONCJira) test case fix

* BONNIE-593 Bonnie Unresponsive Message and Error Loading Measure Packages
Fixed vulnerability: sparklemotion/nokogiri#1943

* BONNIE-587 Error loading VSAC value sets(ONC jira id)

* Updated version of bonnie_version cqm-parser branch (#71)

* Updated version of bonnie_version cqm-parser branch

* Updated cqm-parser (binnie_viersion branch)

* BONNIEMAT-614 Bonnie only processing one of two measure observations(oncjira)

* [Security] Bump nokogiri from 1.10.5 to 1.10.8 (#76)

Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.10.5 to 1.10.8. **This update includes a security fix.**
- [Release notes](https://github.com/sparklemotion/nokogiri/releases)
- [Changelog](https://github.com/sparklemotion/nokogiri/blob/master/CHANGELOG.md)
- [Commits](sparklemotion/nokogiri@v1.10.5...v1.10.8)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* [Security] Bump rake from 12.3.1 to 12.3.3 (#77)

Bumps [rake](https://github.com/ruby/rake) from 12.3.1 to 12.3.3. **This update includes a security fix.**
- [Release notes](https://github.com/ruby/rake/releases)
- [Changelog](https://github.com/ruby/rake/blob/master/History.rdoc)
- [Commits](ruby/rake@v12.3.1...v12.3.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* MAT1385_rails_upgrade_bv (#87)

* made necessary upgrades to line up with bonnie's rails upgrade -> 5.2

* removed deprecated require

* updated codecov (0.1.14 -> 0.2.5) and json (2.1.0 -> 2.3.1) to fix 'Title: json Gem for Ruby Unsafe Object Creation Vulnerability (additional fix)'

* MAT-1308_update_cqm_models (#107)

* cqm-models 3.0.0 -> 3.0.3

* ran bundle install

* Mat 1708 (#112)

* BONNIEMAT-623 & BONNIEMAT-629 cql-integration change

* cqm-models version upgrade

* [MAT-1757] Replacing VSAC Username/Password with API Key in VSAC Calls (#114)

* Replacing vsac username/password with api key.

* Checking for nil via safe navigation on single_code_concepts hash  before checking sub-hash.

When QDM datatype template has includeSubTemplate, the single_code_concepts hash will not have a related key to access the sub-hash.

* Replacing safe nav operator with Hash.dig to better handle the nested hashes.

* pull in cqm-model v3.0.6 version (#129)

* MAT-2803 (#143)

* MAT-2803 Update Bonnie to support UTF8 - QDM

* Ruby 2.7.2 warnings fix

* MAT-2647 Update cqm-parsers to QDM 5.6 (#148)

* MAT-2647 Update cqm-parsers to QDM 5.6

* MAT-2647 Update cqm-parsers to QDM 5.6

* added git actions

* removed travis.yaml

* Integrate from cqm-models MAT-2993 Failed to initialize Data element with class attribute in Bonnie (#149)

* ecurity vulnerabilty nokogiry (#150)

* MAT-2837: Using published version of cqm-models (v4.0.0) instead of branch

* bump nokogiri in case of security vulnerability

* Baseline for v4.0.0 release

* update Gemfile

* git ignore Gemfile.lock

* move ci workflow and add gitleaks.toml

* address rubocop concerns

* revert activesupport to support rails 5

* Bring mongoid back to 6

Co-authored-by: hossenlopp <hossenlopp@mitre.org>
Co-authored-by: Luke Osborne <lwosborne@mitre.org>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Ashok <ashok.dongare@semanticbits.com>
Co-authored-by: Daniel Mee <danmee10@gmail.com>
Co-authored-by: Joe Kotanchik <56264529+jkotanchik-SB@users.noreply.github.com>
Co-authored-by: Joe Kotanchik <joseph.kotanchik@semanticbits.com>
Co-authored-by: Andrew Bird <andrew.bird@semanticbits.com>

.github/workflows/ci.yml

@@ -1,8 +1,7 @@
 name: Continuous Integration
 on:
   pull_request:
-    branches:
-      - master
+    branches: [master, bonnie_version]
 
 jobs:
   build:
@@ -11,7 +10,7 @@ jobs:
 
     strategy:
       matrix:
-        ruby-version: [2.5]
+        ruby-version: [2.5, 2.7.2]
         mongodb-version: [4.0.18, 4.4]
 
     steps:

.gitignore

@@ -6,3 +6,4 @@ tmp
 .idea
 .rvmrc
 Gemfile.lock
+

.rubocop_todo.yml

@@ -527,6 +527,8 @@ Lint/UselessAssignment:
 # Offense count: 134
 Metrics/AbcSize:
   Max: 270
+  Exclude:
+    - 'test/unit/hqmf/1.0/nqf_0002_test.rb'
 
 # Offense count: 6
 # Configuration parameters: CountComments, ExcludedMethods.

Gemfile

@@ -2,9 +2,8 @@ source 'https://rubygems.org'
 
 gemspec :development_group => :test
 
-gem 'mongoid', '~> 7.0.5'
+gem 'mongoid', '~> 6.4.2'
 
-# gem 'cqm-models', '~> 3.0.0'
 # gem 'cqm-models', git: 'https://github.com/projecttacoma/cqm-models.git', branch: 'master'
 # gem 'cqm-models', :path => '../cqm-models'
 

README.md

@@ -42,22 +42,9 @@ bundle exec rake test
 
 ```
 
-
 ## Versioning
 
-Starting with version **2.0.0** released on 6/20/19, cqm-parsers versioning has the format **X.Y.Z**, where:
-
-* **X** maps to a version of the CQL-based HQMF IG. See the table below to see the existing mapping to CQL-based HQMF IG versions.
-
-  | X | CQL-based HQMF IG|
-  | --- | --- |
-  | 2 | R1 STU3 |
-
-* **Y** indicates major changes (incompatible API changes)
-
-* **Z** indicates minor changes (added functionality in a backwards-compatible manner) and patch changes (backwards-compatible bug fixes)
-
-For the versions available, see [tags on this repository](https://github.com/projecttacoma/cqm-parsers/tags).
+We use [SemVer](http://semver.org/) for versioning. For the versions available, see [tags on this repository](https://github.com/projecttacoma/cqm-parsers/tags). 
 
 ## License
 

cqm-parsers.gemspec

@@ -9,16 +9,15 @@ Gem::Specification.new do |s|
   s.authors = ["The MITRE Corporation"]
   s.license = 'Apache-2.0'
 
-  s.version = '3.2.0.1'
+  s.version = '4.0.0.0'
 
-  s.add_dependency 'cqm-models', '~> 3.0'
+  s.add_dependency 'cqm-models', '~> 4.0.0'
   s.add_dependency 'mustache'
   s.add_dependency 'erubis', '~> 2.7.0'
-  s.add_dependency 'mongoid', '~> 7.0.5'
-  s.add_dependency 'mongoid-tree', '~> 2.1'
-  s.add_dependency 'activesupport', '~> 6.0'
+  s.add_dependency 'mongoid', '~> 6.4.2'
+  s.add_dependency 'mongoid-tree', '~> 2.1.0'
+  s.add_dependency 'activesupport', '~> 5.2.1'
 
-  s.add_dependency 'protected_attributes_continued', '~> 1.4.0'
   s.add_dependency 'uuid', '~> 2.3.7'
   s.add_dependency 'builder', '~> 3.1'
   s.add_dependency 'nokogiri', '>= 1.8.5', '< 1.13.0'

lib/measure-loader/mat_measure_files.rb

@@ -58,7 +58,7 @@ def unzip_measure_zip_into_hash(zip_file)
             pn = Pathname(f.name)
             next if '__MACOSX'.in? pn.each_filename  # ignore anything in a __MACOSX folder
             next unless pn.basename.extname.in? ['.xml','.cql','.json','.html']
-            folders[pn.dirname][:files] << { basename: pn.basename, contents: f.get_input_stream.read }
+            folders[pn.dirname][:files] << { basename: pn.basename, contents: f.get_input_stream.read.force_encoding('UTF-8') }
             folders[pn.dirname][:depth] =  pn.each_filename.count # this is just a count of how many folders are in the path
           end
         end
@@ -135,4 +135,4 @@ def verify_library_versions_match_and_get_version(cql, elm, elm_annotation)
 
     end
   end
-end
+end

lib/measure-loader/source_data_criteria_loader.rb

@@ -46,7 +46,8 @@ def extract_fields_from_standard_data_criteria(criteria, entry)
     def extract_fields_from_single_code_reference_data_criteria(criteria)
       single_code_reference = criteria.at_css('value[codeSystem][code]') || criteria.at_css('code[codeSystem][code]')
       system_id = "#{single_code_reference['codeSystem']}_#{single_code_reference['codeSystemVersion']}".to_sym
-      concept = @single_code_concepts[system_id][single_code_reference['code'].to_sym] || get_concept_from_participation(criteria.at_css('participation'))
+      # concept = @single_code_concepts[system_id]&[single_code_reference['code'].to_sym] || get_concept_from_participation(criteria.at_css('participation'))
+      concept = @single_code_concepts&.dig(system_id, single_code_reference['code']&.to_sym) || get_concept_from_participation(criteria.at_css('participation'))
       value_set = concept._parent
       return {
         description: concept.display_name,

lib/measure-loader/vsac_value_set_loader.rb

@@ -7,8 +7,7 @@ def initialize(options)
       options.symbolize_keys!
       @vsac_options = options[:options]
       @vsac_ticket_granting_ticket = options[:ticket_granting_ticket]
-      @vsac_username = options[:username]
-      @vsac_password = options[:password]
+      @vsac_api_key = options[:api_key]
       @vs_model_cache = {}
     end
 
@@ -48,7 +47,7 @@ def retrieve_and_modelize_value_sets_from_vsac(value_sets)
 
     def load_api
       return @api if @api.present?
-      @api = Util::VSAC::VSACAPI.new(config: APP_CONFIG['vsac'], ticket_granting_ticket: @vsac_ticket_granting_ticket, username: @vsac_username, password: @vsac_password)
+      @api = Util::VSAC::VSACAPI.new(config: APP_CONFIG['vsac'], ticket_granting_ticket: @vsac_ticket_granting_ticket, api_key: @vsac_api_key)
       return @api
     end
 

lib/util/hqmfr2cql_template_oid_map.json

@@ -219,10 +219,6 @@
       "definition":"care_goal",
       "status":"",
       "negation":false},
-  "2.16.840.1.113883.10.20.28.4.13":{
-      "definition":"device",
-      "status":"applied",
-      "negation":false},
   "2.16.840.1.113883.10.20.28.4.11":{
       "definition":"device_adverse_event",
       "status":"",

lib/util/vsac_api.rb

@@ -7,7 +7,7 @@ module VSAC
     class VSACError < StandardError
     end
 
-    # Error represnting a not found response from the API. Includes OID for reporting to user.
+    # Error representing a not found response from the API. Includes OID for reporting to user.
     class VSNotFoundError < VSACError
       attr_reader :oid
       def initialize(oid)
@@ -23,15 +23,15 @@ def initialize
       end
     end
 
-    # Error represnting a program not found response from the API.
+    # Error representing a program not found response from the API.
     class VSACProgramNotFoundError < VSACError
       attr_reader :oid
       def initialize(program)
         super("VSAC Program #{program} does not exist.")
       end
     end
 
-    # Error represnting a response from the API that had no concepts.
+    # Error representing a response from the API that had no concepts.
     class VSEmptyError < VSACError
       attr_reader :oid
       def initialize(oid)
@@ -50,14 +50,14 @@ def initialize
     # Raised when the user credentials were invalid.
     class VSACInvalidCredentialsError < VSACError
       def initialize
-        super('VSAC ULMS credentials are invalid.')
+        super('VSAC UMLS credentials are invalid.')
       end
     end
 
     # Raised when a call requiring auth is attempted when no ticket_granting_ticket or credentials were provided.
     class VSACNoCredentialsError < VSACError
       def initialize
-        super('VSAC ULMS credentials were not provided.')
+        super('VSAC UMLS credentials were not provided.')
       end
     end
 
@@ -93,7 +93,7 @@ def initialize(options)
         end
 
         # if a ticket_granting_ticket was passed in, check it and raise errors if found
-        # username and password will be ignored
+        # VSAC API Key will be ignored
         if !options[:ticket_granting_ticket].nil?
           provided_ticket_granting_ticket = options[:ticket_granting_ticket]
           if provided_ticket_granting_ticket[:ticket].nil? || provided_ticket_granting_ticket[:expires].nil?
@@ -105,9 +105,9 @@ def initialize(options)
           @ticket_granting_ticket = { ticket: provided_ticket_granting_ticket[:ticket],
                                       expires: provided_ticket_granting_ticket[:expires] }
 
-        # if username and password were provided use them to get a ticket granting ticket
-        elsif !options[:username].nil? && !options[:password].nil?
-          @ticket_granting_ticket = get_ticket_granting_ticket(options[:username], options[:password])
+          # if api key was provided use it to get a ticket granting ticket
+        elsif !options[:api_key].nil?
+          @ticket_granting_ticket = get_ticket_granting_ticket(options[:api_key])
         end
       end
 
@@ -304,13 +304,11 @@ def create_service_ticket_request
                                      params: { service: TICKET_SERVICE_PARAM})
       end
 
-      # Use your username and password to retrive a ticket granting ticket from VSAC
-      def get_ticket_granting_ticket(username, password)
+      # Use your API Key to retrive a ticket granting ticket from VSAC
+      def get_ticket_granting_ticket(api_key)
         response = Typhoeus.post(
           "#{@config[:auth_url]}/Ticket",
-          # looks like typheous sometimes switches the order of username/password when encoding
-          # which vsac cant handle (!?), so encode first
-          body: URI.encode_www_form(username: username, password: password)
+          body: URI.encode_www_form(apikey: api_key)
         )
         raise VSACInvalidCredentialsError.new if response.response_code == 401
         validate_http_status(response.response_code)

test/fixtures/vcr_cassettes/measure__load_composite_measure.yml

@@ -5,7 +5,7 @@ http_interactions:
     uri: https://vsac.nlm.nih.gov/vsac/ws/Ticket
     body:
       encoding: US-ASCII
-      string: username=<VSAC_USERNAME>&password=<VSAC_PASSWORD>
+      string: apikey=<VSAC_API_KEY>
     headers:
       User-Agent:
       - Typhoeus - https://github.com/typhoeus/typhoeus