diff --git a/CHANGELOG.md b/CHANGELOG.md index f5bd9ea3..b266e54d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,9 +2,51 @@ ## Table of Contents +- [v0.3.0](#v030) - [v0.2.0](#v020) - [v0.1.0 - Initial version](#v010---initial-version) +# v0.3.0 + +**This version introduces API Testing Guidelines and initial linting ruleset and other changes to documents and artifacts approved since v0.2.0.** + +## Please note: +The content of the release includes: +* Commonalities approved deliverables in **[documentation](https://github.com/camaraproject/Commonalities/tree/release-0.3.0/documentation)** folder + - **New:** + - [API Testing Guidelines](https://github.com/camaraproject/Commonalities/blob/release-0.3.0/documentation/API-Testing-Guidelines.md) + - [API Linting Rules](https://github.com/camaraproject/Commonalities/blob/release-0.3.0/documentation/Linting-rules.md) + - [OpenAPI Linting Rules Implementaion Guideline](https://github.com/camaraproject/Commonalities/blob/release-0.3.0/documentation/API-linting-Implementation-Guideline.md) +* Commonalities approved artifacts in **[artifacts](https://github.com/camaraproject/Commonalities/tree/release-0.3.0/artifacts)** folder + - **New:** + - [CAMARA_common.yaml](https://github.com/camaraproject/Commonalities/blob/release-0.3.0/artifacts/CAMARA_common.yaml) + - [Linting rules implementation files](https://github.com/camaraproject/Commonalities/tree/release-0.3.0/artifacts/linting_rules) + +### Added +* API Testing Guidelines created by @shilpa-padgaonkar in https://github.com/camaraproject/Commonalities/pull/117 +* API Design Guidelines updated with scopes naming guidelines by @jlurien in https://github.com/camaraproject/Commonalities/pull/57 +* API Linting Rules - initial linting ruleset description by @rartych in https://github.com/camaraproject/Commonalities/pull/74 +* API linting implementation and guidelines by @ravindrapalaskar17 in https://github.com/camaraproject/Commonalities/pull/110 + +### Changed +* CAMARA_common.yaml includes the following changes: + - CAMARA_common.json was changed to CAMARA_common.yaml to be consistent with all CAMARA API specs + - Includes guidance for info object + - Adds the aligned device schema updated by @shilpa-padgaonkar in https://github.com/camaraproject/Commonalities/pull/107 + +* X-Correlator header as required in OAS definition, X-Version removed from API Design Guidelines by @jlurien in https://github.com/camaraproject/Commonalities/pull/115 +* Filtering criteria in API Design Guidelines changed by @RubenBG7 in https://github.com/camaraproject/Commonalities/pull/132 + +### Fixed +* CAMARA_common.yaml - error response codes updated by @RubenBG7 in https://github.com/camaraproject/Commonalities/pull/124 +* API Design Guidelines updated on response filtering by @rartych in https://github.com/camaraproject/Commonalities/pull/123 + +### Removed +* API-exposure-reference-solution.docx removed from documentation/SupportingDocuments by @jordonezlucena in https://github.com/camaraproject/Commonalities/pull/104 + +**Full Changelog**: https://github.com/camaraproject/Commonalities/compare/v0.2.0...v0.3.0 + + # v0.2.0 **This version introduces CloudEvents format to CAMARA Events and other changes to documents and artifacts approved since v0.1.0.** diff --git a/CODEOWNERS b/CODEOWNERS index f8138a0e..3bfdc4ff 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -5,4 +5,4 @@ # For more details, read the following article on GitHub: https://help.github.com/articles/about-codeowners/. # These are the default owners for the whole content of this repository. The default owners are automatically added as reviewers when you open a pull request, unless different owners are specified in the file. -* @shilpa-padgaonkar @rartych @patrice-conil @RubenBG7 @jordonezlucena +* @shilpa-padgaonkar @rartych @patrice-conil @RubenBG7 @PedroDiez diff --git a/README.md b/README.md index 68c7211a..af69f044 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,10 @@ Repository to describe and document common guidelines and assets for CAMARA APIs The documents that are relevant for CAMARA API Sub Projects are found in the `documentation` directory (at the top-level). The rest of the sub-directories are primarily for internal working of the Commonalities Working Group. -The `artifacts` directory contains templates for creating Github issues and Common data and error formats for CAMARA APIs. +The `artifacts` directory contains: +* templates for creating Github issues +* common data and error formats for CAMARA APIs in [CAMARA_common.yaml](artifacts/CAMARA_common.yaml) +* OAS definition of CAMARA Event using CloudEvents: [notification-as-cloud-event.yaml](artifacts/notification-as-cloud-event.yaml) ### Frequently-accessed output documents @@ -32,20 +35,21 @@ A list of some of the frequently accessed documents that are an output of the wo | [API-Readiness-Checklist.md](documentation/API-Readiness-Checklist.md) | A checklist that describes the minimum criteria for considering an API Ready | | [Camara_Versioning_Guidelines.md](documentation/Camara_Versioning_Guidelines.md) | Guidelines for API Subprojects on making releases | | [Glossary.md](documentation/Glossary.md) | A glossary of the common terms and their API parameter/field names for use in the APIs | - +| [API-Testing-Guidelines.md](documentation/API-Testing-Guidelines.md) | Guidelines for the API testing in CAMARA project | ## Status and released versions +* Release 0.3.0 of guidelines and assets for CAMARA APIs is available within the [release-0.3.0 branch](https://github.com/camaraproject/Commonalities/tree/release-0.3.0): * Release 0.2.0 of guidelines and assets for CAMARA APIs is available within the [release-0.2.0 branch](https://github.com/camaraproject/Commonalities/tree/release-0.2.0): * Release 0.1.0 of guidelines and assets for CAMARA APIs is available within the [release-0.1.0 branch](https://github.com/camaraproject/Commonalities/tree/release-0.1.0): For changes see [CHANGELOG.md](https://github.com/camaraproject/Commonalities/blob/main/CHANGELOG.md) ## Meetings -* Meetings are held virtually -* Schedule: bi-weekly, Monday, 4 PM CET/CEST (3 PM BST, 15:00 UTC) -* Meeting link: [MS Teams](https://teams.microsoft.com/l/meetup-join/19%3ameeting_M2E3ZmUxYWUtMDZkNi00YmM1LThiYWMtZjQzNWI0NWQxOGY0%40thread.v2/0?context=%7b%22Tid%22%3a%22bde4dffc-4b60-4cf6-8b04-a5eeb25f5c4f%22%2c%22Oid%22%3a%22a5cb7460-f2b0-42ec-b511-c642f83aa9a3%22%7d) +* Meetings are held virtually on the LF Platform: [Meeting Registration / Join](https://zoom-lfx.platform.linuxfoundation.org/meeting/91016460698?password=d031b0e3-8d49-49ae-958f-af3213b1e547) +* Schedule: bi-weekly, Monday, 4 PM CET/CEST (3 PM BST, 15:00 UTC). The date of the next meeting can be found in the previous [meeting minutes](https://wiki.camaraproject.org/display/CAM/2024+Commonalities+WG+Minutes). + ## Contributorship and mailing list * To subscribe / unsubscribe to the mailing list of this Sub Project and thus be / resign as Contributor please visit . diff --git a/artifacts/linting_rules/.github/workflows/megalinter.yml b/artifacts/linting_rules/.github/workflows/megalinter.yml new file mode 100644 index 00000000..6bda7009 --- /dev/null +++ b/artifacts/linting_rules/.github/workflows/megalinter.yml @@ -0,0 +1,78 @@ +--- +# MegaLinter GitHub Action configuration file +# More info at https://megalinter.io +# CAMARA Project - Github Action for Pull Reqests +# 31.01.2024 - initial version + +name: MegaLinter + +on: # yamllint disable-line rule:truthy + # Pull Requests to main + pull_request: + branches: [master, main] + +env: # Comment env block if you do not want to apply fixes + # Apply linter fixes configuration + APPLY_FIXES: all # When active, APPLY_FIXES must also be defined as environment variable (in github/workflows/mega-linter.yml or other CI tool) + APPLY_FIXES_EVENT: pull_request # Decide which event triggers application of fixes in a commit or a PR (pull_request, push, all) + APPLY_FIXES_MODE: commit # If APPLY_FIXES is used, defines if the fixes are directly committed (commit) or posted in a PR (pull_request) + +concurrency: + group: ${{ github.ref }}-${{ github.workflow }} + cancel-in-progress: true + +jobs: + build: + name: MegaLinter + runs-on: ubuntu-latest + permissions: + # Give the default GITHUB_TOKEN write permission to commit and push, comment issues & post new PR + # Remove the ones you do not need + contents: write + issues: write + pull-requests: write + steps: + # Git Checkout + - name: Checkout Code + uses: actions/checkout@v4 + with: + token: ${{ secrets.GITHUB_TOKEN }} + fetch-depth: 0 # If you use VALIDATE_ALL_CODEBASE = true, you can remove this line to improve performances + - name: Install Spectral + run: npm install -g @stoplight/spectral + - name: Install Spectral functions + run: npm install -g @stoplight/spectral-functions + # - name: Run spectral:oas Spectral Linting + # run: spectral lint code/API_definitions/*.yaml --verbose --ruleset .spectral.yml + # Replace openapi.yaml file with your API specification file + + # MegaLinter + - name: MegaLinter + id: ml + # You can override MegaLinter flavor used to have faster performances + # More info at https://megalinter.io/flavors/ + uses: oxsecurity/megalinter/flavors/java@v7.3.0 + env: + # All available variables are described in documentation + # https://megalinter.io/configuration/ + PRINT_ALPACA: false + # VALIDATE_ALL_CODEBASE: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }} # Validates all source when push on main, else just the git diff with main. Override with true if you always want to lint all sources + VALIDATE_ALL_CODEBASE: true + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + # ADD YOUR CUSTOM ENV VARIABLES HERE OR DEFINE THEM IN A FILE .mega-linter.yml AT THE ROOT OF YOUR REPOSITORY + DISABLE: COPYPASTE,MARKDOWN + DISABLE_LINTERS: SPELL_CSPELL,SPELL_LYCHEE,YAML_PRETTIER,REPOSITORY_GRYPE, REPOSITORY_SEMGREP,REPOSITORY_DEVSKIM,REPOSITORY_KICS,REPOSITORY_TRIVY,REPOSITORY_TRIVY_SBOM,REPOSITORY_TRUFFLEHOG,REPOSITORY_CHECKOV,REPOSITORY_GITLEAKS,YAML_V8R,JAVA_PMD,JAVA_CHECKSTYLE + YAML_YAMLLINT_CONFIG_FILE: ".yamllint.yaml" + OPENAPI_SPECTRAL_CONFIG_FILE: ".spectral.yml" + YAML_YAMLLINT_FILTER_REGEX_INCLUDE: "(code/)" + OPENAPI_SPECTRAL_FILTER_REGEX_INCLUDE: "(code/)" + + # Upload MegaLinter artifacts + - name: Archive production artifacts + if: ${{ success() }} || ${{ failure() }} + uses: actions/upload-artifact@v4 + with: + name: MegaLinter reports + path: | + megalinter-reports + mega-linter.log diff --git a/artifacts/linting_rules/.github/workflows/spectral_oas_lint.yml b/artifacts/linting_rules/.github/workflows/spectral_oas_lint.yml new file mode 100644 index 00000000..a828fd58 --- /dev/null +++ b/artifacts/linting_rules/.github/workflows/spectral_oas_lint.yml @@ -0,0 +1,36 @@ +--- +# CAMARA Project - workflow configuration to manually run CAMARA OAS rules +# see https://docs.github.com/en/actions/using-workflows/manually-running-a-workflow +# 31.01.2024 - initial version + +name: Spectral manual run + +on: workflow_dispatch + +concurrency: + group: ${{ github.ref }}-${{ github.workflow }} + cancel-in-progress: true + +jobs: + build: + name: Spectral linting + runs-on: ubuntu-latest + permissions: + # Give the default GITHUB_TOKEN write permission to commit and push, comment issues & post new PR + # Remove the ones you do not need + contents: write + issues: write + pull-requests: write + steps: + # Git Checkout + - name: Checkout Code + uses: actions/checkout@v4 + with: + token: ${{ secrets.GITHUB_TOKEN }} + fetch-depth: 0 # If you use VALIDATE_ALL_CODEBASE = true, you can remove this line to improve performances + - name: Install Spectral + run: npm install -g @stoplight/spectral + - name: Install Spectral functions + run: npm install -g @stoplight/spectral-functions + - name: Run Spectral linting + run: spectral lint code/API_definitions/*.yaml --verbose --ruleset .spectral.yml diff --git a/artifacts/linting_rules/.spectral.yml b/artifacts/linting_rules/.spectral.yml new file mode 100644 index 00000000..0b16508e --- /dev/null +++ b/artifacts/linting_rules/.spectral.yml @@ -0,0 +1,258 @@ +# CAMARA Project - linting ruleset - documentation avaialable here: +# https://github.com/camaraproject/Commonalities/blob/main/documentation/Linting-rules.md +# 31.01.2024 - initial version + +extends: "spectral:oas" +functions: + - camara-reserved-words + - camara-language-avoid-telco + - camara-security-no-secrets-in-path-or-query-parameters +functionsDir: "./lint_function" +rules: + # Built-in OpenAPI Specification ruleset. Each rule then can be enabled individually. + # The severity keyword is optional in rule definition and can be error, warn, info, hint, or off. The default value is warn. + contact-properties: false + duplicated-entry-in-enum: true + info-contact: true + info-description: true + info-license: true + license-url: true + no-$ref-siblings: error + no-eval-in-markdown: true + no-script-tags-in-markdown: true + openapi-tags: false + openapi-tags-alphabetical: false + openapi-tags-uniqueness: error + operation-description: true + operation-operationId: true + operation-operationId-unique: error + operation-operationId-valid-in-url: true + operation-parameters: true + operation-singular-tag: true + operation-success-response: true + operation-tags: true + operation-tag-defined: true + path-declarations-must-exist: true + path-keys-no-trailing-slash: true + path-not-include-query: true + path-params: error + tag-description: false + typed-enum: true + oas3-api-servers: true + oas3-examples-value-or-externalValue: true + oas3-operation-security-defined: false + oas3-parameter-description: false + oas3-schema: true + oas3-server-not-example.com: false + oas3-server-trailing-slash: true + oas3-unused-component: true + oas3-valid-media-example: true + oas3-valid-schema-example: true + # oas3-server-variables: true + + # Custom Rules Utilizing Spectral's Built-in Functions and JavaScript Implementations + + camara-language-avoid-telco: + message: "{{error}}" + severity: hint + description: | + This rule checks for telco-specific terminology in your API definitions and suggests more inclusive terms. + given: "$..*.*" + then: + function: camara-language-avoid-telco + recommended: false # Set to true/false to enable/disable this rule + + camara-oas-version: + message: "OpenAPI Version Error: The OpenAPI specification must adhere to version 3.0.3." + severity: error + description: | + This rule validates the OpenAPI version in your specification and requires compliance with version 3.0.3. + given: "$" + then: + field: openapi + function: pattern + functionOptions: + match: 3.0.3 + recommended: true # Set to true/false to enable/disable this rule + + camara-path-param-id: + message: "Path Parameter Naming Warning: Use 'resource_id' instead of just 'id' in path parameters." + severity: warn + description: | + This rule ensures consistent and descriptive naming for path parameters in your OpenAPI specification. + Please use 'resource_id' instead of just 'id' for your path parameters. + given: "$..parameters[?(@.in == 'path')]" + then: + field: name + function: pattern + functionOptions: + notMatch: \b(id|Id|ID|iD)\b + recommended: true # Set to true/false to enable/disable this rule + + camara-security-no-secrets-in-path-or-query-parameters: + message: "Sensitive data found in path: {{error}} Consider avoiding the use of Sesentive data " + severity: warn + description: | + This rule checks for sensitive data ('MSISDN' and 'IMSI') in API paths and suggests avoiding their use. + given: + - "$.paths" + then: + function: camara-security-no-secrets-in-path-or-query-parameters + recommended: true # Set to true/false to enable/disable this rule + + camara-http-methods: + description: "Ensure that all path URLs have valid HTTP methods (GET, PUT, POST, DELETE, PATCH, OPTIONS)." + message: "Invalid HTTP method for '{{path}}'. Must be one of get, put, post, delete, patch, options." + severity: error + given: $.paths[*][*]~ + then: + function: pattern + functionOptions: + match: "^(get|put|post|delete|patch|options)$" + recommended: true # Set to true/false to enable/disable this rule + + camara-get-no-request-body: + message: There must be no request body for Get and DELETE + severity: error + given: + - "$.paths.*.get" + - "$.paths.*.delete" + then: + field: requestBody + function: falsy + recommended: true # Set to true/false to enable/disable this rule + + camara-reserved-words: + message: "Reserved words found {{error}} Consider avoiding the use of reserved word " + severity: warn + description: | + This rule checks Reserved words must not be used in the following parts of an API specification [Paths, Request Body properties, Component, Operation Id, Security Schema] + given: + - "$.paths" # Paths + - "$..parameters[*]" # Path or Query Parameter Names: + - "$..components.schemas.*.properties.*" # Request and Response body parameter + - "$.paths.*." # Path and Operation Names: + - "$.components.securitySchemes" # Security Schemes: + - "$.components.*.*" # Component Names: + - "$.paths.*.*.operationId" # OperationIds: + then: + function: camara-reserved-words + recommended: true # Set to true/false to enable/disable this rule + + camara-routes-description: + message: "Functionality method description Warning: Each method should have description." + severity: warn + description: | + This rule checks if each operation (POST, GET, DELETE, PUT, PATCH, OPTIONS) in your API specification has a description. + Ensure that you have added a 'summary' field for each operation in your OpenAPI specification. + given: + - "$.paths.*.post" + - "$.paths.*.get" + - "$.paths.*.delete" + - "$.paths.*.put" + - "$.paths.*.patch" + - "$.paths.*.options" + then: + field: description + function: truthy + recommended: true # Set to true/false to enable/disable this rule + + camara-parameters-descriptions: + message: "Parameter description is missing or empty: {{error}}" + severity: warn + description: | + This Spectral rule ensures that each path parameter in the API specification has a descriptive and meaningful description. + given: + - "$.paths..parameters.*" + then: + field: description + function: truthy + recommended: true # Set to true/false to enable/disable this rule + + camara-response-descriptions: + message: "Parameter description is missing or empty: {{error}}" + severity: warn + description: | + This Spectral rule ensures that each responese object in the API specification has a descriptive and meaningful description. + given: + - "$.paths..responses.*" + then: + field: description + function: truthy + recommended: true # Set to true/false to enable/disable this rule + + camara-properties-descriptions: + message: "Property description is missing or empty: {{error}}" + severity: warn + description: | + This Spectral rule ensures that each propoerty within objects in the API specification has a descriptive and meaningful description. + given: + - "$.components.*.*" + - "$.components.*.*.properties.*" + then: + field: description + function: truthy + recommended: true # Set to true/false to enable/disable this rule + + camara-operation-summary: + message: "Operation Summary Warning: Each operation should include a short summary for better understanding." + severity: warn + description: | + This rule checks if each operation (POST, GET, DELETE, PUT, PATCH, OPTIONS) in your API specification has a meaningful summary. + Ensure that you have added a 'summary' field for each operation in your OpenAPI specification. + given: + - "$.paths.*.post" + - "$.paths.*.get" + - "$.paths.*.delete" + - "$.paths.*.put" + - "$.paths.*.patch" + - "$.paths.*.options" + then: + field: summary + function: truthy + recommended: true # Set to true/false to enable/disable this rule + + camara-discriminator-use: + description: | + Ensure that API definition YAML files with oneOf or anyOf sections include a discriminator object for serialization, deserialization, and validation. + severity: hint + given: "$..[?(@.oneOf || @.anyOf)]" + then: + field: discriminator + function: truthy + description: "Discriminator object is required when using oneOf or anyOf." + recommended: true # Set to true/false to enable/disable this rule + + camara-operationid-casing-convention: + message: Operation Id must be in Camel case "{{error}}" + severity: hint + description: | + This rule checks Operation ids should follow a specific case convention: camel case. + given: "$.paths.*.*.operationId" + then: + function: casing + functionOptions: + type: camel + recommended: true # Set to true/false to enable/disable this rule + + camara-schema-casing-convention: + description: This rule checks schema should follow a specific case convention pascal case. + message: "{{property}} should be pascal case (UppperCamelCase)" + severity: warn + given: $.components.schemas[*]~ + then: + function: casing + functionOptions: + type: pascal + recommended: true # Set to true/false to enable/disable this rule + + camara-parameter-casing-convention: + description: Paths should be kebab-case. + severity: error + message: "{{property}} is not kebab-case: {{error}}" + given: $.paths[*]~ + then: + function: pattern + functionOptions: + match: "^\/([a-z0-9]+(-[a-z0-9]+)*)?(\/[a-z0-9]+(-[a-z0-9]+)*|\/{.+})*$" # doesn't allow /asasd{asdas}sadas pattern or not closed braces + recommended: true # Set to true/false to enable/disable this rule diff --git a/artifacts/linting_rules/.yamllint.yaml b/artifacts/linting_rules/.yamllint.yaml new file mode 100644 index 00000000..081ef093 --- /dev/null +++ b/artifacts/linting_rules/.yamllint.yaml @@ -0,0 +1,35 @@ +--- +# CAMARA Project - YAML linting configuration for yamllint https://yamllint.readthedocs.io/en/latest/rules.html +# 31.01.2024 - initial version + +yaml-files: + - '*.yaml' + - '*.yml' + - '.yamllint' + +rules: + braces: enable + brackets: enable + colons: enable + commas: enable + comments: + min-spaces-from-content: 1 + level: error + comments-indentation: + level: error + document-end: disable + document-start: disable + empty-lines: enable + empty-values: disable + hyphens: enable + indentation: enable + key-duplicates: enable + key-ordering: disable + line-length: disable + new-line-at-end-of-file: enable + new-lines: disable + octal-values: disable + quoted-strings: disable + trailing-spaces: enable + truthy: + level: error diff --git a/artifacts/linting_rules/lint_function/camara-language-avoid-telco.js b/artifacts/linting_rules/lint_function/camara-language-avoid-telco.js new file mode 100644 index 00000000..061b5431 --- /dev/null +++ b/artifacts/linting_rules/lint_function/camara-language-avoid-telco.js @@ -0,0 +1,40 @@ +// CAMARA Project - support function for Spectral linter +// 31.01.2024 - initial version + +const replacements = [ + { original: 'UE', recommended: 'device' }, + { original: 'MSISDN', recommended: 'phone number' }, + { original: 'mobile network', recommended: 'network' } +]; + +export default async function (input) { + const errors = []; + const suggestions = []; + + // Iterate over properties of the input object + for (const path in input) { + const value = input[path]; + + // Check if the value is a string + if (typeof value === 'string') { + for (const replacement of replacements) { + const original = replacement.original; + const recommended = replacement.recommended; + + // Use a regular expression to match 'original' as a standalone word + const regex = new RegExp(`\\b${original}\\b`, 'g'); + + // Check if 'original' exists in the value + if (regex.test(value)) { + errors.push(replacement); + suggestions.push(` Telco-specific terminology found in input: Consider replacing '${original}' with '${recommended}'.`); + } + } + } + } + + // Check if any word from 'replacements' is in the suggestions + if (errors.length > 0) { + console.log(`Hint camara-language-avoid-telco ` + suggestions.join(', ')); + } +}; diff --git a/artifacts/linting_rules/lint_function/camara-reserved-words.js b/artifacts/linting_rules/lint_function/camara-reserved-words.js new file mode 100644 index 00000000..c28e63ab --- /dev/null +++ b/artifacts/linting_rules/lint_function/camara-reserved-words.js @@ -0,0 +1,98 @@ +// CAMARA Project - support function for Spectral linter +// 31.01.2024 - initial version + +const reservedWords = [ + 'abstract', + 'apiclient', + 'apiexception', + 'apiresponse', + 'assert', + 'boolean', + 'break', + 'byte', + 'case', + 'catch', + 'char', + 'class', + 'configuration', + 'const', + 'continue', + 'do', + 'double', + 'else', + 'extends', + 'file', + 'final', + 'finally', + 'float', + 'for', + 'goto', + 'if', + 'implements', + 'import', + 'instanceof', + 'int', + 'interface', + 'list', + 'localdate', + 'localreturntype', + 'localtime', + 'localvaraccept', + 'localvaraccepts', + 'localvarauthnames', + 'localvarcollectionqueryparams', + 'localvarcontenttype', + 'localvarcontenttypes', + 'localvarcookieparams', + 'localvarformparams', + 'localvarheaderparams', + 'localvarpath', + 'localvarpostbody', + 'localvarqueryparams', + 'long', + 'native', + 'new', + 'null', + 'object', + 'offsetdatetime', + 'package', + 'private', + 'protected', + 'public', + 'return', + 'short', + 'static', + 'strictfp', + 'stringutil', + 'super', + 'switch', + 'synchronized', + 'this', + 'throw', + 'throws', + 'transient', + 'try', + 'void', + 'volatile', + 'while' +]; +// Reserved word 'enum' and 'default' are removed from above reserved word array as they are common in openAPI keyword +export default async function lintReservedWords(input) { + // Iterate over properties of the input object + for (const path in input) { + if (typeof path === 'string') { + + for (const word of reservedWords) { + const regex = new RegExp(`\\b${word}\\b`, 'g'); // Use a regular expression to match 'word' as a standalone word + + if (regex.test(path)) { + const warningRuleName = 'camara-reserved-words'; + const description = `Reserved words found in input: Consider avoiding the use of reserved word '${word}'`; + // const location = `${path}`; + + console.log(`warning ${warningRuleName} ${description} ${path}`); + } + } + } + } +} diff --git a/artifacts/linting_rules/lint_function/camara-security-no-secrets-in-path-or-query-parameters.js b/artifacts/linting_rules/lint_function/camara-security-no-secrets-in-path-or-query-parameters.js new file mode 100644 index 00000000..ebbff2a4 --- /dev/null +++ b/artifacts/linting_rules/lint_function/camara-security-no-secrets-in-path-or-query-parameters.js @@ -0,0 +1,26 @@ +// CAMARA Project - support function for Spectral linter +// 31.01.2024 - initial version + +const sensitiveData = ['MSISDN','IMSI','phoneNumber']; + +export default async function (input) { + + // Iterate over properties of the input object + for (const path in input) { + + if (typeof path === 'string') { + for (const word of sensitiveData ) { + const regex = new RegExp(`\\b${word}\\b`, 'g'); // Use a regular expression to match 'word' as a standalone word + + if (regex.test(path)) { + + const warningRuleName = 'camara-security-no-secrets-in-path-or-query-parameters'; + const description = `sensitiveData Data found in path: Consider avoiding the use of sensitiveData data '${word}'`; + const location = `paths.${path}`; + console.log(`warning ${warningRuleName} ${description} ${location}`); + + } + } + } + } +} diff --git a/documentation/API-design-guidelines.md b/documentation/API-design-guidelines.md index cbc2eaf2..c1e1109a 100644 --- a/documentation/API-design-guidelines.md +++ b/documentation/API-design-guidelines.md @@ -685,6 +685,7 @@ It should be noted that this point is open to continuous evolution over time thr ## 8. Pagination, Sorting and Filtering Exposing a resources collection through a single URI can cause applications to fetch large amounts of data when only a subset of the information is required. For example, suppose a client application needs to find all orders with a cost greater than a specific value. You could retrieve all orders from the /orders URI and then filter these orders on the client side. Clearly, this process is highly inefficient. It wastes network bandwidth and processing power on the server hosting the web API. +To alleviate the above-mentioned issues and concerns, Pagination, Sorting and Filtering may optionally be supported by the API provider. Following subsections apply when such functionality is supported. ### 8.1 Pagination Services can answer with a resource or article collections. Sometimes these collections may be a partial set due to performance or security reasons. Elements must be identified and arranged consistently on all pages. Paging can be enabled by default on the server side to mitigate denial of service or similar. @@ -716,6 +717,7 @@ Petitions examples: ### 8.2 Sorting Sorting the result of a query on a resources collection requires two main parameters: +Note: Services must accept and use these parameters when sorting is supported. If a parameter is not supported, service should return an error message. - `orderBy`: it contains the names of the attributes on which the sort is performed, with comma separated if there is more than one criteria. - `order`: by default, sorting is done in descending order. @@ -729,18 +731,28 @@ https://api.mycompany.com/v1/orders?orderBy=rating,reviews,name&order=desc ### 8.3 Filtering -Filtering consists of restricting the number of resources queried by specifying some attributes and their expected values. It is possible to filter a collection on multiple attributes at the same time and allow multiple values for a filtered attribute. +Filtering consists of restricting the number of resources queried by specifying some attributes and their expected values. When filtering is supported, it is possible to filter a collection on multiple attributes at the same time and allow multiple values for a filtered attribute. -Next, it is specified how it should be used according to the filtering based on the type of data being searched for: a text, a number or a date and the type of operation. +Next, it is specified how it should be used according to the filtering based on the type of data being searched for: a number or a date and the type of operation. + +Note: Services may not support all attributes for filtering. In case a query includes an attribute for which filtering is not supported, it may be ignored by the service. -| **Operation** | Text | Numbers | Dates | -| ----- | ----- | ----- | ----- | -| Equal | `GET .../?name=Juan` | `GET .../?amount=807.24` | `GET .../?executionDate=2018-30-05` | -| Greater or equal | N/A | `GET .../?amount.gte=807.24` | `GET.../?executionDate.gte=2018-30-05` | -| Strictly greater | N/A | `GET .../?amount.gt=807.24` | `GET.../?executionDate.gt=2018-30-05` | -| smaller or equal | N/A | `GET .../?amount.lte=807.24` | `GET.../?executionDate.lte=2018-30-05` | -| Strictly smaller | N/A | `GET .../?amount.lt=807.24` | `GET.../?executionDate.lt=2018-30-05` | -|Contains | `GET .../?name=~Juan` |N/A | N/A | +| **Operation** | Numbers | Dates | +| ----- | ----- | ----- | +| Equal | `GET .../?amount=807.24` | `GET .../?executionDate=2024-02-05T09:38:24Z` | +| Greater or equal | `GET .../?amount.gte=807.24` | `GET.../?executionDate.gte=2018-05-30` | +| Strictly greater | `GET .../?amount.gt=807.24` | `GET.../?executionDate.gt=2018-05-30` | +| smaller or equal | `GET .../?amount.lte=807.24` | `GET.../?executionDate.lte=2018-05-30` | +| Strictly smaller | `GET .../?amount.lt=807.24` | `GET.../?executionDate.lt=2018-05-30` | + + +And according to the filtering based on string and enums data being searched for: + +| **Operation** | **Strings/enums** | +| ----- | ----- | +| equal | `GET .../?name=Juan` | +| non equal | `GET .../?name!=Jonh` | +| Contains | `GET .../?name=~Rafa` | **Additional rules**: @@ -816,8 +828,7 @@ With the aim of standardizing the request observability and traceability process | Name | Description | Type | Pattern | Longitude | Location | Required by API Consumer | Required in OAS Definition | Example | |---|---|---|---|---|---|---|---|---| -| `X-Version` | Service version description to help observability process | String| N/A | | Request | No | No | | -| `X-Correlator`| Service correlator to make E2E observability | String | UUID (8-4-4-4-12) | Max 36 | Request/Response | No | No | b4333c46-49c0-4f62-80d7-f0ef930f1c46 | +| `X-Correlator`| Service correlator to make E2E observability | String | UUID (8-4-4-4-12) | Max 36 | Request/Response | No | Yes | b4333c46-49c0-4f62-80d7-f0ef930f1c46 | When the API Consumer includes the "X-Correlator" header in the request, the API provider must include it in the response with the same UUID as was used in the request. Otherwise, it is optional to include the "X-Correlator" header in the response with any valid (preferably randomly chosen) UUID. @@ -923,28 +934,14 @@ The scopes allow defining the permission scopes that a system or a user has on a Scopes should be represented as: - API Name: address-management, numbering-information... - Protected Resource: orders, billings… -- Grant-level: read, write… +- Grant-level, action on resource: read, write… To correctly define the scopes, when creating them, the following recommendations should be taken: - **Appropriate granularity**. Scopes should be granular enough to match the types of resources and permissions you want to grant. - **Use a common nomenclature for all resources**. Scopes must be descriptive names and that there is no conflict between the different resources. - **Use the kebab-case nomenclature** to define API names, resources, and scope permissions. -- **Recommended Format**: `--` -Next, we illustrate an example on how to define a series of scopes for a OpenAPI file. - -The first step is to create the security definitions according to the nomenclature that we have defined. -

- -

- -Then, each operation is assigned the necessary scope: -

- -

-

- -

+See section [11.6 Security Definition](#116-security-definition) for detailed guidelines on how to define scopes and other security-related properties in a OpenAPI file. Data security @@ -994,7 +991,6 @@ The API must validate the signature of the JWT in the payload following next req - Making sure that the JWT payload has the same structure and values as the decrypted part of "`JWT_Signature`". - ## 11. Definition in OpenAPI API documentation helps customers integrate with the API by explaining what it is and how to use it. All APIs must include documentation for the developer who will consume your API. @@ -1222,17 +1218,48 @@ When IpAddr is used in a payload, the property objectType MUST be present to ind } ``` -### 11.6 OAuth Definition +### 11.6 Security definition + +The [CAMARA API Specification - Authorization and authentication common guidelines](https://github.com/camaraproject/IdentityAndConsentManagement/blob/main/documentation/CAMARA-API-access-and-user-consent.md#camara-api-specification---authorization-and-authentication-common-guidelines) are discussed and maintained by the [Identity and Consent Management Working Group](https://github.com/camaraproject/IdentityAndConsentManagement). In particular, the following aspects are detailed: + +- Use of openIdConnect as protocol in `securitySchemes`. +- How to fill the `security` property per operation. +- How to fill the "Authorization and authentication" section in `info.description`. + +#### 11.6.1 Scope naming + +Regarding scope naming, the guidelines are: + +* Define a scope per API operation with the structure: + +`api-name:[resource:]action` + +where + +* `api-name` is the API name specified as the base path, prior to the API version, in the `servers[*].url` property. For example, from `/location-verification/v0`, it would be `location-verification`. + +* `resource` is optional. For APIs with several `paths`, it may include the resource in the path. For example, from `/qod/v0/sessions/{sessionId}`, it would be `sessions`. + +* `action`: There are two cases: + - For POST operations with a verb, it will be the verb. For example, from `POST /location-verification/v0/verify`, it would be `verify`. + - For endpoints designed as POST but with underlying logic retrieving information, a CRUD action `read` may be added, but if it is a path with single operation and it is not expected to have more operations on it, the CRUD action is not necessary. + - For CRUD operations on a resource in paths, it will be one of: + - `create`: For operations creating the resource, typically `POST`. + - `read`: For operations accessing to details of the resource, typically `GET`. + - `update`: For operations modifying the resource, typically `PUT` or `PATCH`. + - `delete`: For operations removing the resource, typically `DELETE`. + - `write` : For operations creating or modifying the resource, when differentiation between `create` and `update` is not needed. + +* Eventually we may need to add an additional level to the scope, such as `api-name:[resource:]action[:detail]`, to deal with cases when only a set of parameters/information has to be allowed to be returned. Guidelines should be enhanced when those cases happen. + +##### Examples -Finally, this part describes the OAuth security applied to the API. This spec is for client testing purposes only, but -there should be as similar as possible to the OAuth flows in your production environment. This definition has the -following aspects: +| API | path | method | scope | +| --- | --- | --- | --- | +| location-verification | /verify | POST | `location-verification:verify` | +| qod | /sessions | POST | `qod:sessions:create`, or
`qod:sessions:write` | +| qod | /qos-profiles | GET | `qod:qos-profiles:read` | -- Security Type: oauth2, oauth… -- Security Flow (Depends on security type): implicit, password… -- Security Flow description applied (String) -- Endpoint token URL -- URL to endpoint authorization ( If flow is based on "`authorizationCode`"). ## 12. Subscription, Notification & Event diff --git a/documentation/API-linting-Implementation-Guideline.md b/documentation/API-linting-Implementation-Guideline.md new file mode 100644 index 00000000..f26256b0 --- /dev/null +++ b/documentation/API-linting-Implementation-Guideline.md @@ -0,0 +1,81 @@ +# CAMARA OpenAPI Linting Rules Implementaion Guideline [ How to integrate the rules into CAMARA repository ] + +## Introduction + +This guide provides instructions how to implement linting rules for the CAMARA APIs using two methods: **[GitHub Actions](API-linting-Implementation-Guideline.md#github-actions-integration)** and **[local deployment](API-linting-Implementation-Guideline.md#github-actions-integration)**, both methods use [Spectral tool](https://docs.stoplight.io/docs/spectral/674b27b261c3c-overview). +All needed files are stored in [artifacts subfolder](https://github.com/camaraproject/Commonalities/tree/API-linting-Implementation-Guideline/artifacts/linting_rules). + +The target method is linting rules integration with CAMARA API subproject repositories using GitHub Actions. + + +## Spectral Configuration + +The Spectral configuration consists of .spectral.yml file, which contains all the rules defined for CAMARA OpenAPI specification as described in [Linting-rules.md](Linting-rules.md) + +This file consolidates all rules: + +1. Spectral Core OpenAPI specification linting ruleset: + + `Ruleset extension: extends: "spectral:oas"` + +2. Spectral rules with built-in functions +3. Spectral rules with custom JavaScript functions + + +## GitHub Actions Integration + +1. Add **[.spectral.yml](https://github.com/camaraproject/Commonalities/blob/main/artifacts/linting_rules/.spectral.yml)** (rules) file to -> root location of repository + +2. Create **lint-function** folder + + Make a folder named `lint_function` at root location and add custom [JavaScript function files](https://github.com/camaraproject/Commonalities/tree/API-linting-Implementation-Guideline/artifacts/linting_rules/lint_function) that are imported in .spectral.yml (some rules require custom JavaScript functions to execute). + +3. Add **[spectral_oas_lint.yml](https://github.com/camaraproject/Commonalities/blob/main/artifacts/linting_rules/.github/workflows/spectral_oas_lint.yml)** to GitHub action workflows in `.github/workflows` folder + which includes the configuration of Spectral workflow for GitHub actions. + +4. Add [megalinter.yml](https://github.com/camaraproject/Commonalities/blob/main/artifacts/linting_rules/.github/workflows/megalinter.yml) to GitHub action workflows in `.github/workflows` folder + which includes the configuration of Megalinter and Spectral for GitHub actions. + +### Manually running linting workflow + +**spectral_oas_lint.yml** includes configuration of the OAS linting workflow to be run manually as described in [GitHub Actions documentation](https://docs.github.com/en/actions/using-workflows/manually-running-a-workflow). + +The rules will be applied to all files with *.yaml extension in '/code/API_definitions/' folder of the repository. +Write access to the repository is required to perform these steps. + +The output from Spectral can be seen by expanding the step **Run Spectral Linting** of given worflow run Actions section of GitHub repository. + + +### Megalinter integration + +[Megalinter](https://megalinter.io/latest/) is an Open-Source tool for CI/CD workflows that analyzes the consistency of code, configurations and scripts in repository sources. Megalinter supports Spectral linting. +The Megalinter job will be automatically activated once you submit a pull request on the [main/master] branch of the CAMARA repository, as configured in megalinter.yml. + +The Megalinter configuration consists of the megalinter.yml file containing the necessary settings to run Megalinter and Spectral jobs on GitHub actions. + +Additionally, Megalinter also supports linting of YAML files. To enable this, users need to add the following ruleset files to the root location. + +- YAML Linting: .yamllint.yaml + + + + +## API Linting configuration steps for local deployment + +1. Install Spectral locally: + + npm install -g @stoplight/spectral + +2. Install Spectral functions locally: + + npm install --save @stoplight/spectral-functions + +3. Save files locally: + + Save ".spectral.yml" file (contains Linting rules) and lint_function folder (contains JavaScript customized functions) at the root location. + +4. Apply spectral rules on API specification loacally: + + spectral lint openapi.yaml --verbose --ruleset .spectral.yml + + *Replace **'openapi.yaml'** with the path to your OpenAPI specification file.* diff --git a/documentation/Linting-rules.md b/documentation/Linting-rules.md new file mode 100644 index 00000000..625a9694 --- /dev/null +++ b/documentation/Linting-rules.md @@ -0,0 +1,409 @@ +# CAMARA API Linting Rules Documentation + +## Introduction + + +## 1. Spectral core ruleset + +Spectral has a built-in OpenAPI Specification ruleset that can be used to validate OpenAPI files. + +With `extends: "spectral:oas"` ("oas" being shorthand for OpenAPI Specification) in the ruleset file to rules for OpenAPI v2 and v3.x, depending on the appropriate OpenAPI version being used (this is automatically detected through formats) are added to the final ruleset. + +All of the rules in this ruleset are described in [OpenAPI Rules](https://docs.stoplight.io/docs/spectral/4dec24461f3af-open-api-rules). + + +`extends: [[spectral:oas, off]]` - this avoids running any rules from the extended ruleset as they are disabled. Each rule then can be [enabled individually](https://docs.stoplight.io/docs/spectral/0a73453054745-recommended-or-all#enabling-rules). + +### Recommended rules +Spectral's built-in OpenAPI ruleset is a two-tier system: with subset of rules marked as [recommended](https://docs.stoplight.io/docs/spectral/0a73453054745-recommended-or-all#recommended-or-all) to be used by default, and addtional rules marked with `recommended: false`. +Recommended rules cover more basic requirements. + +### Rule severity + +The `severity` keyword is optional in rule definition and can be `error`, `warn`, `info`, `hint`, or `off`. +The default value is `warn`. + +### OpenAPI v2 & v3 + +Rules applying to both OpenAPI v2.0, v3.0, and most likely v3.1 - details are described in [Spectral Documentation](https://docs.stoplight.io/docs/spectral/4dec24461f3af-open-api-rules#openapi-v2--v3). + +|Name| Desc| Recom mended|CAMARA use|Spectral severity | CAMARA severity | +|---|---|---|--|---|--| +|contact-properties| contact object is full of the most useful properties: `name`, `url`, and `email`| No | No | Warning | Warning | +|duplicated-entry-in-enum| Each value of an `enum` must be different from one another | Yes | Yes | Warning | Warning | +|info-contact |Info object should contain `contact` object |Yes | Yes | Warning | Warning | +|info-description |Info object should contain `description` object | Yes | Yes| Warning | Warning | +|info-license |Info object should contain `license` object |Yes | Yes | Warning | Warning | +|license-url | link to the full text of licence | Yes | Yes| Warning | Warning | +|no-$ref-siblings| Before OpenAPI v3.1, keywords next to $ref were ignored | Yes | Yes| Error | Error | +|no-eval-in-markdown | injecting `eval()` JavaScript statements could lead to an XSS attack | Yes | Yes | Warning | Warning | +|no-script-tags-in-markdown | injecting `