[filebeat][httpjson] - Separation of global transform contexts and introduction of parent transform context within chains

[ShourieG]

Type of change

• Enhancement

What does this PR do?

This PR introduces the following enhancements to httpjson -

1. It introduces separation of transform contexts from parent (root) requests and chain requests.
2. It introduces a parent transform context object parentTrCtx as a new attribute of trCtx (main transform context
   object) which is a self referential pointer and this will point to the transform context of the root request when accessed
   from within chain steps.
3. It introduces the .parent_last_response.* clause, which helps access the last_response of the parent request
   during chaining.

Why is it important?

1. This change helps improve clarity between chain steps & non-chain steps by clearly maintaining separate instances
   of their respective transform contexts, hence reducing the chances of unexpected errors due to overlapping transform
   contexts.
2. By introducing the .parent_last_response.* clause & the parentTrCtx pointer, we open up various new use-
   cases to be implemented. For example a request with chaining, where the root request on every page response
   receives an unique id in the body, which is then passed to the respective chain requests to extract the required
   information. The test included simulates this scenario.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
  -[ ] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc.

Author's Checklist

• Code structure , syntax & comments
• Code optimization & quality

Related issues

• Closes HttpJson new feature - Introduce a global transform context, accessible from within chain calls #33147

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @ShourieG? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-11-08T05:54:30.392+0000

• Duration: 73 min 38 sec

Test stats 🧪

Test	Results
Failed	0
Passed	2315
Skipped	167
Total	2482

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b httpjson/trCtx upstream/httpjson/trCtx
git merge upstream/main
git push upstream httpjson/trCtx

[efd6]

Minor comment, otherwise LGTM.

[ShourieG]

@efd6 have made the necessary improvements with updated docs and test cases. Now dot's in expressions and hard coded values will be properly processed.

[ShourieG]

@efd6 updated the docs and the processExpression method. In the end decided to keep the default behaviour as it made more sense. The docs have been updated accordingly and test cases have been add to support the same.

[ShourieG]

@efd6 updated PR with the processException tests.

[efd6]

Thanks