Replace macOS packages with new ones, which are expected to generate vulnerabilities.

[santipadilla]

Main issue #	#5074

Description

This is a sub-issue of Vulnerability Detection E2E: Replace macOS vulnerable packages by npm.

As outlined in Syscollector unable to gather installed pkg packages on macOS Sonoma, including Homebrew-installed packages, syscollector on macOS fails to correctly detect brew and pkg packages. This directly impacts the end-to-end (E2E) testing of vulnerability detection, rendering validation of the module refactor introduced in version 4.8.0 for macOS endpoints impossible.

To address this issue, it is proposed to replace the current pkg packages used in E2E tests for macOS with npm packages.

Task to perform

• Replace macOS packages with new ones, which are expected to generate vulnerabilities.

Validation

• All E2E Vulnerability detection tests works as expected for macOS agents

[santipadilla]

In this issue, we will carry out an investigation of packages installed using npm on macOS agent, to obtain their vulnerabilities.

First we configure the VD module in the manager:

<vulnerability-detection>
    <enabled>yes</enabled>
    <index-status>yes</index-status>
    <feed-update-interval>60m</feed-update-interval>
  </vulnerability-detection>

  <indexer>
    <enabled>yes</enabled>
    <hosts>
      <host>https://172.31.14.180:9200</host>
    </hosts>
    <ssl>
     <certificate_authorities>
      <ca>/etc/pki/filebeat/root-ca.pem</ca>
     </certificate_authorities>
     <certificate>/etc/pki/filebeat/node-2.pem</certificate>
     <key>/etc/pki/filebeat/node-2-key.pem</key>
    </ssl>
   </indexer>

[santipadilla]

Case	Packages	Download links	CVE info	CVE detected
E2E-VD-3: Installation of a vulnerable package	http-proxy 0.5.9	npm install -g http-proxy@0.5.9	http-proxy-0.5.9	CVE-2017-16014
E2E-VD-4: Updating a vulnerable package that remains vulnerable to the same CVE	http-proxy 0.5.9 -> 0.5.10	npm install -g http-proxy@0.5.10	http-proxy-0.5.10	CVE-2017-16014
E2E-VD-5: Updating a vulnerable package that becomes vulnerable to another CVE	systeminformation 4.34.23 -> 5.0.0	npm install -g systeminformation@4.34.23	systeminformation 4.34.23	CVE-2021-21388 CVE-2021-21315 CVE-2023-42810
E2E-VD-6: Updating a vulnerable package that becomes vulnerable to another CVE and retains the previous one	systeminformation 4.34.23 -> 5.0.0	npm install -g systeminformation@5.0.0	systeminformation 5.0.0	CVE-2021-21388 CVE-2021-21315 CVE-2023-42810
E2E-VD-7: Updating a vulnerable package that ceases to be vulnerable	http-proxy 0.5.10 -> 0.7.0	npm install -g http-proxy@0.7.0	-	-
E2E-VD-8: Deleting a vulnerable package	http-proxy 0.5.9	npm uninstall -g http-proxy	-	-
E2E-VD-9: Installation of a non-vulnerable package	http-proxy 0.7.0	npm install -g http-proxy@0.7.0	-	-
E2E-VD-10: Updating a non-vulnerable package that remains non-vulnerable	http-proxy 0.7.0 -> 0.7.2	npm install -g http-proxy@0.7.2	-	-
E2E-VD-11: Updating a non-vulnerable package that becomes vulnerable	luxon 2.5.2 -> 3.0.0	npm install -g luxon@3.0.0	luxon 3.0.0	CVE-2022-31129

Note:

It has also been tested with the react package, which has vulnerabilities but they do not appear because the package comes without some data such as the name of the vendor, in this case facebook.

React data

root@ip-172-31-14-180:/home/qa# curl -k -X GET "https://localhost:55000/syscollector/002/packages?search=react&pretty=true" -H "Authorization: Bearer ${TOKEN}"
{
   "data": {
      "affected_items": [
         {
            "scan": {
               "id": 0,
               "time": "2024-04-01T11:55:47+00:00"
            },
            "vendor": " ",
            "install_time": " ",
            "source": "https://facebook.github.io/react/",
            "description": "React is a JavaScript library for building user interfaces.",
            "name": "react",
            "section": " ",
            "architecture": " ",
            "format": "npm",
            "location": "/opt/homebrew/lib/node_modules/react/package.json",
            "size": 0,
            "version": "16.0.0",
            "priority": " ",
            "agent_id": "002"
         }
      ],
      "total_affected_items": 1,
      "total_failed_items": 0,
      "failed_items": []
   },
   "message": "All specified syscollector information was returned",
   "error": 0
}

[santipadilla]

Change to on hold due to this: #5128

[santipadilla]

These changes have been made in this branch:

5129-replace-macos-packages

In this PR: #5174

[santipadilla]

Update

Test result:

• Passed: 10
• Failed: 4
• It fails with cases where a package is already available and needs to be removed or updated as in the case of "remove_package", it removes the package from the macOS machine, but it should be investigated that it prevents the vulnerability from being detected.
  Manually it works fine by removing the package and resolves the vulnerabilities commented here.

[santipadilla]

Moved to on hold for suggested changes in #5130 and #5128

[santipadilla]

Update

Test result:

• Passed: 11

• Failed: 3

• Fixed failure in "remove_package", npm does not delete the package with the specified version e.g. "npm uninstall -g http-proxy@0.59.0", we have to use "npm uninstall -g http-proxy". Fixed in vuln_packages.json variable "uninstall_name".

• The three missing errors are due to the three cases of updates, the cause needs to be investigated.

  • upgrade_package_maintain_vulnerability
  • upgrade_package_maintain_add_vulnerability0
  • upgrade_package_maintain_add_vulnerability1

• Report:
  
  report.zip

Manually it works fine by upgrading the package and resolves the vulnerabilities commented here.

Manual check 🟢

sh-3.2# npm install -g http-proxy@0.5.9
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: 'http-proxy@0.5.9',
npm WARN EBADENGINE   required: { node: '0.4.x || 0.5.x' },
npm WARN EBADENGINE   current: { node: 'v21.7.1', npm: '10.5.0' }
npm WARN EBADENGINE }
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142

added 52 packages in 2s

2 packages are looking for funding
  run `npm fund` for details

sh-3.2# npm -g list
/usr/local/lib
├── corepack@0.25.2
├── http-proxy@0.5.9
└── npm@10.5.0

{"timestamp":"2024-04-04T13:54:59.703+0000","rule":{"level":7,"description":"CVE-2017-16014 affects http-proxy","id":"23504","firedtimes":9,"mail":false,"groups":["vulnerability-detector"],"gdpr":["IV_35.7.d"],"pci_dss":["11.2.1","11.2.3"],"tsc":["CC7.1","CC7.2"]},"agent":{"id":"002","name":"agent1","ip":"192.168.64.13"},"manager":{"name":"ip-172-31-6-78"},"id":"1712238899.1902044","cluster":{"name":"wazuh","node":"master"},"decoder":{"name":"json"},"data":{"vulnerability":{"assigner":"hackerone","cve":"CVE-2017-16014","cvss":{"cvss2":{"base_score":"5","vector":{"access_complexity":"LOW","authentication":"NONE","availability":"PARTIAL","confidentiality_impact":"NONE","integrity_impact":"NONE"}}},"cwe_reference":"CWE-388","enumeration":"CVE","package":{"architecture":" ","condition":"Package less than 0.7.0","name":"http-proxy","source":" ","version":"0.5.9"},"published":"2018-06-04T19:29:00Z","rationale":"Http-proxy is a proxying library. Because of the way errors are handled in versions before 0.7.0, an attacker that forces an error can crash the server, causing a denial of service.","reference":"https://github.com/nodejitsu/node-http-proxy/pull/101, https://nodesecurity.io/advisories/323","severity":"Medium","status":"Active","title":"CVE-2017-16014 affects http-proxy","type":"Packages","updated":"2019-10-09T23:24:36Z"}},"location":"vulnerability-detector"}


sh-3.2# npm install -g http-proxy@0.5.10
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: 'http-proxy@0.5.10',
npm WARN EBADENGINE   required: { node: '0.4.x || 0.5.x' },
npm WARN EBADENGINE   current: { node: 'v21.7.1', npm: '10.5.0' }
npm WARN EBADENGINE }
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142

changed 52 packages in 378ms

2 packages are looking for funding
  run `npm fund` for details

sh-3.2# npm -g list
/usr/local/lib
├── corepack@0.25.2
├── http-proxy@0.5.10
└── npm@10.5.0

{"timestamp":"2024-04-04T13:54:59.703+0000","rule":{"level":7,"description":"CVE-2017-16014 affects http-proxy","id":"23504","firedtimes":9,"mail":false,"groups":["vulnerability-detector"],"gdpr":["IV_35.7.d"],"pci_dss":["11.2.1","11.2.3"],"tsc":["CC7.1","CC7.2"]},"agent":{"id":"002","name":"agent1","ip":"192.168.64.13"},"manager":{"name":"ip-172-31-6-78"},"id":"1712238899.1902044","cluster":{"name":"wazuh","node":"master"},"decoder":{"name":"json"},"data":{"vulnerability":{"assigner":"hackerone","cve":"CVE-2017-16014","cvss":{"cvss2":{"base_score":"5","vector":{"access_complexity":"LOW","authentication":"NONE","availability":"PARTIAL","confidentiality_impact":"NONE","integrity_impact":"NONE"}}},"cwe_reference":"CWE-388","enumeration":"CVE","package":{"architecture":" ","condition":"Package less than 0.7.0","name":"http-proxy","source":" ","version":"0.5.9"},"published":"2018-06-04T19:29:00Z","rationale":"Http-proxy is a proxying library. Because of the way errors are handled in versions before 0.7.0, an attacker that forces an error can crash the server, causing a denial of service.","reference":"https://github.com/nodejitsu/node-http-proxy/pull/101, https://nodesecurity.io/advisories/323","severity":"Medium","status":"Active","title":"CVE-2017-16014 affects http-proxy","type":"Packages","updated":"2019-10-09T23:24:36Z"}},"location":"vulnerability-detector"}
{"timestamp":"2024-04-04T13:58:16.280+0000","rule":{"level":7,"description":"CVE-2017-16014 affects http-proxy","id":"23504","firedtimes":10,"mail":false,"groups":["vulnerability-detector"],"gdpr":["IV_35.7.d"],"pci_dss":["11.2.1","11.2.3"],"tsc":["CC7.1","CC7.2"]},"agent":{"id":"002","name":"agent1","ip":"192.168.64.13"},"manager":{"name":"ip-172-31-6-78"},"id":"1712239096.1904384","cluster":{"name":"wazuh","node":"master"},"decoder":{"name":"json"},"data":{"vulnerability":{"assigner":"hackerone","cve":"CVE-2017-16014","cvss":{"cvss2":{"base_score":"5","vector":{"access_complexity":"LOW","authentication":"NONE","availability":"PARTIAL","confidentiality_impact":"NONE","integrity_impact":"NONE"}}},"cwe_reference":"CWE-388","enumeration":"CVE","package":{"architecture":" ","condition":"Package less than 0.7.0","name":"http-proxy","source":" ","version":"0.5.10"},"published":"2018-06-04T19:29:00Z","rationale":"Http-proxy is a proxying library. Because of the way errors are handled in versions before 0.7.0, an attacker that forces an error can crash the server, causing a denial of service.","reference":"https://github.com/nodejitsu/node-http-proxy/pull/101, https://nodesecurity.io/advisories/323","severity":"Medium","status":"Active","title":"CVE-2017-16014 affects http-proxy","type":"Packages","updated":"2019-10-09T23:24:36Z"}},"location":"vulnerability-detector"}
{"timestamp":"2024-04-04T13:58:16.391+0000","rule":{"level":3,"description":"The CVE-2017-16014 that affected http-proxy was solved due to a package removal/update or a system upgrade","id":"23502","firedtimes":15,"mail":false,"groups":["vulnerability-detector"],"gdpr":["IV_35.7.d"],"pci_dss":["11.2.1","11.2.3"],"tsc":["CC7.1","CC7.2"]},"agent":{"id":"002","name":"agent1","ip":"192.168.64.13"},"manager":{"name":"ip-172-31-6-78"},"id":"1712239096.1906726","cluster":{"name":"wazuh","node":"master"},"decoder":{"name":"json"},"data":{"vulnerability":{"cve":"CVE-2017-16014","cvss":{"cvss2":{"base_score":"5"}},"enumeration":"CVE","package":{"architecture":" ","name":"http-proxy","version":"0.5.9"},"published":"2018-06-04T19:29:00Z","reference":"https://github.com/nodejitsu/node-http-proxy/pull/101, https://nodesecurity.io/advisories/323","severity":"Medium","status":"Solved","title":"CVE-2017-16014 affecting http-proxy was solved","type":"Packages","updated":"2019-10-09T23:24:36Z"}},"location":"vulnerability-detector"}

Dashboard

Manual test check 🟢

(test_VD) root@santipa-Lenovo-Legion-5-15IMH05:/home/santipa/Escritorio# python3 test_framework.py
Installing package 'http-proxy@0.5.9' on host 'agent1'...
Installation result: {'changed': True, 'cmd': 'PATH=/usr/local/bin:$PATH npm install -g http-proxy@0.5.9', 'delta': '0:00:03.023652', 'end': '2024-04-04 07:55:15.534977', 'msg': '', 'rc': 0, 'start': '2024-04-04 07:55:12.511325', 'stderr': "npm WARN EBADENGINE Unsupported engine {\nnpm WARN EBADENGINE   package: 'http-proxy@0.5.9',\nnpm WARN EBADENGINE   required: { node: '0.4.x || 0.5.x' },\nnpm WARN EBADENGINE   current: { node: 'v21.7.1', npm: '10.5.0' }\nnpm WARN EBADENGINE }\nnpm WARN deprecated har-validator@5.1.5: this library is no longer supported\nnpm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.\nnpm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142", 'stderr_lines': ['npm WARN EBADENGINE Unsupported engine {', "npm WARN EBADENGINE   package: 'http-proxy@0.5.9',", "npm WARN EBADENGINE   required: { node: '0.4.x || 0.5.x' },", "npm WARN EBADENGINE   current: { node: 'v21.7.1', npm: '10.5.0' }", 'npm WARN EBADENGINE }', 'npm WARN deprecated har-validator@5.1.5: this library is no longer supported', 'npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.', 'npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142'], 'stdout': '\nadded 52 packages in 3s\n\n2 packages are looking for funding\n  run `npm fund` for details', 'stdout_lines': ['', 'added 52 packages in 3s', '', '2 packages are looking for funding', '  run `npm fund` for details']}

(test_VD) root@santipa-Lenovo-Legion-5-15IMH05:/home/santipa/Escritorio# python3 test_framework.py
Installing package 'http-proxy@0.5.10' on host 'agent1'...
Installation result: {'changed': True, 'cmd': 'PATH=/usr/local/bin:$PATH npm install -g http-proxy@0.5.10', 'delta': '0:00:00.565811', 'end': '2024-04-04 07:56:47.092803', 'msg': '', 'rc': 0, 'start': '2024-04-04 07:56:46.526992', 'stderr': "npm WARN EBADENGINE Unsupported engine {\nnpm WARN EBADENGINE   package: 'http-proxy@0.5.10',\nnpm WARN EBADENGINE   required: { node: '0.4.x || 0.5.x' },\nnpm WARN EBADENGINE   current: { node: 'v21.7.1', npm: '10.5.0' }\nnpm WARN EBADENGINE }\nnpm WARN deprecated har-validator@5.1.5: this library is no longer supported\nnpm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.\nnpm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142", 'stderr_lines': ['npm WARN EBADENGINE Unsupported engine {', "npm WARN EBADENGINE   package: 'http-proxy@0.5.10',", "npm WARN EBADENGINE   required: { node: '0.4.x || 0.5.x' },", "npm WARN EBADENGINE   current: { node: 'v21.7.1', npm: '10.5.0' }", 'npm WARN EBADENGINE }', 'npm WARN deprecated har-validator@5.1.5: this library is no longer supported', 'npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.', 'npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142'], 'stdout': '\nchanged 52 packages in 445ms\n\n2 packages are looking for funding\n  run `npm fund` for details', 'stdout_lines': ['', 'changed 52 packages in 445ms', '', '2 packages are looking for funding', '  run `npm fund` for details']}

Manager alerts

{"timestamp":"2024-04-04T14:56:06.642+0000","rule":{"level":7,"description":"CVE-2017-16014 affects http-proxy","id":"23504","firedtimes":1,"mail":false,"groups":["vulnerability-detector"],"gdpr":["IV_35.7.d"],"pci_dss":["11.2.1","11.2.3"],"tsc":["CC7.1","CC7.2"]},"agent":{"id":"002","name":"agent1","ip":"192.168.64.13"},"manager":{"name":"ip-172-31-6-78"},"id":"1712242566.1909436","cluster":{"name":"wazuh","node":"master"},"decoder":{"name":"json"},"data":{"vulnerability":{"assigner":"hackerone","cve":"CVE-2017-16014","cvss":{"cvss2":{"base_score":"5","vector":{"access_complexity":"LOW","authentication":"NONE","availability":"PARTIAL","confidentiality_impact":"NONE","integrity_impact":"NONE"}}},"cwe_reference":"CWE-388","enumeration":"CVE","package":{"architecture":" ","condition":"Package less than 0.7.0","name":"http-proxy","source":" ","version":"0.5.9"},"published":"2018-06-04T19:29:00Z","rationale":"Http-proxy is a proxying library. Because of the way errors are handled in versions before 0.7.0, an attacker that forces an error can crash the server, causing a denial of service.","reference":"https://github.com/nodejitsu/node-http-proxy/pull/101, https://nodesecurity.io/advisories/323","severity":"Medium","status":"Active","title":"CVE-2017-16014 affects http-proxy","type":"Packages","updated":"2019-10-09T23:24:36Z"}},"location":"vulnerability-detector"}
{"timestamp":"2024-04-04T14:57:13.225+0000","rule":{"level":7,"description":"CVE-2017-16014 affects http-proxy","id":"23504","firedtimes":2,"mail":false,"groups":["vulnerability-detector"],"gdpr":["IV_35.7.d"],"pci_dss":["11.2.1","11.2.3"],"tsc":["CC7.1","CC7.2"]},"agent":{"id":"002","name":"agent1","ip":"192.168.64.13"},"manager":{"name":"ip-172-31-6-78"},"id":"1712242633.1911776","cluster":{"name":"wazuh","node":"master"},"decoder":{"name":"json"},"data":{"vulnerability":{"assigner":"hackerone","cve":"CVE-2017-16014","cvss":{"cvss2":{"base_score":"5","vector":{"access_complexity":"LOW","authentication":"NONE","availability":"PARTIAL","confidentiality_impact":"NONE","integrity_impact":"NONE"}}},"cwe_reference":"CWE-388","enumeration":"CVE","package":{"architecture":" ","condition":"Package less than 0.7.0","name":"http-proxy","source":" ","version":"0.5.10"},"published":"2018-06-04T19:29:00Z","rationale":"Http-proxy is a proxying library. Because of the way errors are handled in versions before 0.7.0, an attacker that forces an error can crash the server, causing a denial of service.","reference":"https://github.com/nodejitsu/node-http-proxy/pull/101, https://nodesecurity.io/advisories/323","severity":"Medium","status":"Active","title":"CVE-2017-16014 affects http-proxy","type":"Packages","updated":"2019-10-09T23:24:36Z"}},"location":"vulnerability-detector"}
{"timestamp":"2024-04-04T14:57:13.328+0000","rule":{"level":3,"description":"The CVE-2017-16014 that affected http-proxy was solved due to a package removal/update or a system upgrade","id":"23502","firedtimes":2,"mail":false,"groups":["vulnerability-detector"],"gdpr":["IV_35.7.d"],"pci_dss":["11.2.1","11.2.3"],"tsc":["CC7.1","CC7.2"]},"agent":{"id":"002","name":"agent1","ip":"192.168.64.13"},"manager":{"name":"ip-172-31-6-78"},"id":"1712242633.1914118","cluster":{"name":"wazuh","node":"master"},"decoder":{"name":"json"},"data":{"vulnerability":{"cve":"CVE-2017-16014","cvss":{"cvss2":{"base_score":"5"}},"enumeration":"CVE","package":{"architecture":" ","name":"http-proxy","version":"0.5.9"},"published":"2018-06-04T19:29:00Z","reference":"https://github.com/nodejitsu/node-http-proxy/pull/101, https://nodesecurity.io/advisories/323","severity":"Medium","status":"Solved","title":"CVE-2017-16014 affecting http-proxy was solved","type":"Packages","updated":"2019-10-09T23:24:36Z"}},"location":"vulnerability-detector"}

Dashboard