Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aioredis instrumentation fix with extra context and statement #1406

Conversation

ajaygupta2790
Copy link
Contributor

@ajaygupta2790 ajaygupta2790 commented Nov 16, 2021

What does this pull request do?

aioredis instrumentation is not available in elastic apm

Just like other instrumentations like asyncpg where we show what statement is executed, this PR adds the statement that is executed using the aioredis instrumentation

An example for the same is as follows:
GET service:key
Screenshot 2021-11-17 at 12 14 46 AM

Adds Redis under dependencies as well

@github-actions github-actions bot added agent-python community Issues opened by the community triage Issues awaiting triage labels Nov 16, 2021
@apmmachine
Copy link
Contributor

apmmachine commented Nov 16, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-08-30T18:14:40.799+0000

  • Duration: 44 min 3 sec

Test stats 🧪

Test Results
Failed 0
Passed 13067
Skipped 14889
Total 27956

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /test linters : Run the Python linters only.

  • /test full : Run the full matrix of tests.

  • /test benchmark : Run the APM Agent Python benchmarks tests.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@ajaygupta2790
Copy link
Contributor Author

/test

1 similar comment
@basepi
Copy link
Contributor

basepi commented Nov 17, 2021

/test

@basepi
Copy link
Contributor

basepi commented Nov 17, 2021

Good catch! I'm not sure how this was missed in the original instrumentation.

@basepi basepi self-assigned this Nov 17, 2021
@basepi
Copy link
Contributor

basepi commented Nov 18, 2021

@ajaygupta2790 A few failing tests. Keep me posted if you need help resolving them.

@ajaygupta2790
Copy link
Contributor Author

Yes I will need some help here, could you help me understand the test a little bit?

@basepi basepi assigned beniwohli and unassigned basepi Nov 29, 2021
@beniwohli
Copy link
Contributor

Hey @ajaygupta2790

The reason that the test is failing is that you're adding a span where we didn't create spans before, namely in RedisConnectionInstrumentation. That instrumentation was added to add destination data to an already existing aioredis span, because the needed information isn't available where that span is created. It's a somewhat ugly workaround, but it works (or, used to work until we renamed the subtype in the main instrumentation to redis and didn't also make that change in the if condition in RedisConnectionInstrumentation 🤦 looks like we have a testing gap there).

More generally speaking, so far we do not capture redis queries in any of our agents. This has been a deliberate choice, as redis keys quite often contain sensitive information like user names, ids, hashed passwords and similar. The risk of leaking such data from your production system to your monitoring system (in this case, Elasticsearch), is quite high. For SQL queries, we can do some sanitization, but this isn't possible for redis keys, which are essentially free-form.

I'll take this topic to our agents team to see if we want to change this and capture redis queries going forward.

@basepi
Copy link
Contributor

basepi commented Aug 26, 2022

/test

@apmmachine
Copy link
Contributor

apmmachine commented Aug 26, 2022

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (67/67) 💚
Files 100.0% (226/226) 💚
Classes 100.0% (226/226) 💚
Lines 90.565% (17700/19544) 👍 0.724
Conditionals 77.584% (3243/4180) 👍 0.329

@basepi
Copy link
Contributor

basepi commented Aug 26, 2022

I simplified and updated your solution to not add an additional span. Though the more I look at this the more I think it probably belongs in the instrumentation classes which actually create the spans.... I'll think on it.

@basepi
Copy link
Contributor

basepi commented Aug 26, 2022

/test

@basepi
Copy link
Contributor

basepi commented Aug 26, 2022

/test linters

@basepi
Copy link
Contributor

basepi commented Aug 30, 2022

/test full

@basepi basepi enabled auto-merge (squash) August 30, 2022 18:14
@basepi basepi merged commit a4413c1 into elastic:main Aug 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
agent-python community Issues opened by the community triage Issues awaiting triage
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants