Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support grpc Server & Client (WIP) #862

Closed
wants to merge 7 commits into from

Conversation

RyanKung
Copy link
Contributor

What does this pull request do?

Emm, Support grpc server.

Related issues

#645

def request_started(self, handler_call_details):
meta_data = handler_call_details.invocation_metadata[0]._asdict()
trace_parent = TraceParent.from_headers(meta_data)
method = handler_call_details.method

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

local variable 'method' is assigned to but never used

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

should be a callable object that follow:
Callable[str] -> bool
"""
def __init__(self, client=None, client_cls=Client, logging=False, config={}, **defaults):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not use mutable data structures for argument defaults. They are created during function definition time. All calls to the function reuse this one instance of that data structure, persisting changes between them.

from elasticapm.contrib.flask.utils import get_data_from_request, get_data_from_response
from elasticapm.handlers.logging import LoggingHandler
from elasticapm.traces import execution_context
from elasticapm.utils import build_name_with_http_method_prefix

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'elasticapm.utils.build_name_with_http_method_prefix' imported but unused

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

import elasticapm.instrumentation.control
from elasticapm.base import Client
from elasticapm.conf import constants, setup_logging
from elasticapm.contrib.flask.utils import get_data_from_request, get_data_from_response

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'elasticapm.contrib.flask.utils.get_data_from_request' imported but unused
'elasticapm.contrib.flask.utils.get_data_from_response' imported but unused

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

import elasticapm
import elasticapm.instrumentation.control
from elasticapm.base import Client
from elasticapm.conf import constants, setup_logging

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'elasticapm.conf.constants' imported but unused

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@apmmachine
Copy link
Contributor

apmmachine commented Jun 16, 2020

❕ Build Aborted

The PR is not allowed to run in the CI yet

Pipeline View Test View Changes Artifacts

Expand to view the summary

Build stats

  • Build Cause: [Branch indexing]

  • Reason: The PR is not allowed to run in the CI yet

  • Start Time: 2020-08-25T16:27:11.952+0000

  • Duration: 3 min 42 sec

  • Commit: 436eaf7

Steps errors

Expand to view the steps failures

  • Name: Error signal
    • Description: githubPrCheckApproved: The PR is not allowed to run in the CI yet. (Only users with write permission

    • Duration: 0 min 0 sec

    • Start Time: 2020-08-25T16:29:51.452+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-08-25T16:29:28.426Z]  > git config core.sparsecheckout # timeout=10
[2020-08-25T16:29:28.436Z]  > git checkout -f 436eaf719168d18bb80c1d3a5ab64183b3880cd0 # timeout=15
[2020-08-25T16:29:28.499Z]  > git remote # timeout=10
[2020-08-25T16:29:28.518Z]  > git config --get remote.origin.url # timeout=10
[2020-08-25T16:29:28.537Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-08-25T16:29:28.542Z]  > git merge 34681949dc9ab3fff5b58b2faef46eddb1ec5d2c # timeout=10
[2020-08-25T16:29:28.620Z]  > git rev-parse HEAD^{commit} # timeout=10
[2020-08-25T16:29:28.637Z]  > git config core.sparsecheckout # timeout=10
[2020-08-25T16:29:28.641Z]  > git checkout -f 04d7c21116a0fae1927fc0292fbbdf4e2933b4cc # timeout=15
[2020-08-25T16:29:32.218Z] Commit message: "Merge commit '34681949dc9ab3fff5b58b2faef46eddb1ec5d2c' into HEAD"
[2020-08-25T16:29:32.249Z] First time build. Skipping changelog.
[2020-08-25T16:29:32.249Z] Cleaning workspace
[2020-08-25T16:29:32.994Z] Masking supported pattern matches of $JOB_GCS_BUCKET or $NOTIFY_TO
[2020-08-25T16:29:33.033Z] The timestamps step is unnecessary when timestamps are enabled for all Pipeline builds.
[2020-08-25T16:29:33.093Z] Timeout set to expire in 1 hr 0 min
[2020-08-25T16:29:33.225Z] [INFO] Number of builds to be searched 10
[2020-08-25T16:29:33.385Z] [INFO] 'shallow' is forced to be disabled when running on PullRequests
[2020-08-25T16:29:33.398Z] Running in /var/lib/jenkins/workspace/thon_apm-agent-python-mbp_PR-862/src/github.com/elastic/apm-agent-python
[2020-08-25T16:29:33.413Z] [INFO] gitCheckout: Checkout SCM PR-862 with default customisation from the Item.
[2020-08-25T16:29:33.438Z] [INFO] Override default checkout
[2020-08-25T16:29:33.469Z] Sleeping for 10 sec
[2020-08-25T16:29:32.221Z]  > git rev-list --no-walk 19f80fafdee446c7d1a6462bd49c36883a9a4d13 # timeout=10
[2020-08-25T16:29:32.252Z]  > git rev-parse --verify HEAD # timeout=10
[2020-08-25T16:29:32.261Z] Resetting working tree
[2020-08-25T16:29:32.261Z]  > git reset --hard # timeout=10
[2020-08-25T16:29:32.301Z]  > git clean -fdx # timeout=10
[2020-08-25T16:29:43.608Z] using credential f6c7695a-671e-4f4f-a331-acdce44ff9ba
[2020-08-25T16:29:43.629Z] Wiping out workspace first.
[2020-08-25T16:29:43.639Z] Cloning the remote Git repository
[2020-08-25T16:29:43.639Z] Using shallow clone with depth 3
[2020-08-25T16:29:43.639Z] Avoid fetching tags
[2020-08-25T16:29:43.654Z] Cloning repository git@github.com:elastic/apm-agent-python.git
[2020-08-25T16:29:43.680Z]  > git init /var/lib/jenkins/workspace/thon_apm-agent-python-mbp_PR-862/src/github.com/elastic/apm-agent-python # timeout=10
[2020-08-25T16:29:43.688Z] Fetching upstream changes from git@github.com:elastic/apm-agent-python.git
[2020-08-25T16:29:43.688Z]  > git --version # timeout=10
[2020-08-25T16:29:43.698Z]  > git --version # 'git version 2.17.1'
[2020-08-25T16:29:43.699Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-08-25T16:29:43.703Z]  > git fetch --no-tags --progress -- git@github.com:elastic/apm-agent-python.git +refs/heads/*:refs/remotes/origin/* # timeout=15
[2020-08-25T16:29:44.866Z] Cleaning workspace
[2020-08-25T16:29:44.886Z] Using shallow fetch with depth 3
[2020-08-25T16:29:44.886Z] Pruning obsolete local branches
[2020-08-25T16:29:44.844Z]  > git config remote.origin.url git@github.com:elastic/apm-agent-python.git # timeout=10
[2020-08-25T16:29:44.848Z]  > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
[2020-08-25T16:29:44.857Z]  > git config remote.origin.url git@github.com:elastic/apm-agent-python.git # timeout=10
[2020-08-25T16:29:44.869Z]  > git rev-parse --verify HEAD # timeout=10
[2020-08-25T16:29:44.876Z] No valid HEAD. Skipping the resetting
[2020-08-25T16:29:44.876Z]  > git clean -fdx # timeout=10
[2020-08-25T16:29:44.890Z] Fetching upstream changes from git@github.com:elastic/apm-agent-python.git
[2020-08-25T16:29:44.890Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-08-25T16:29:44.897Z]  > git fetch --no-tags --progress --prune -- git@github.com:elastic/apm-agent-python.git +refs/pull/862/head:refs/remotes/origin/PR-862 +refs/heads/master:refs/remotes/origin/master # timeout=15
[2020-08-25T16:29:45.569Z] Merging remotes/origin/master commit 34681949dc9ab3fff5b58b2faef46eddb1ec5d2c into PR head commit 436eaf719168d18bb80c1d3a5ab64183b3880cd0
[2020-08-25T16:29:45.709Z] Merge succeeded, producing d694e67613f199736f03ee3b46e718ae431c5d7e
[2020-08-25T16:29:45.710Z] Checking out Revision d694e67613f199736f03ee3b46e718ae431c5d7e (PR-862)
[2020-08-25T16:29:45.756Z] Commit message: "Merge commit '34681949dc9ab3fff5b58b2faef46eddb1ec5d2c' into HEAD"
[2020-08-25T16:29:45.763Z] First time build. Skipping changelog.
[2020-08-25T16:29:45.763Z] Cleaning workspace
[2020-08-25T16:29:46.644Z] Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD
[2020-08-25T16:29:45.572Z]  > git config core.sparsecheckout # timeout=10
[2020-08-25T16:29:45.576Z]  > git checkout -f 436eaf719168d18bb80c1d3a5ab64183b3880cd0 # timeout=15
[2020-08-25T16:29:45.624Z]  > git remote # timeout=10
[2020-08-25T16:29:45.628Z]  > git config --get remote.origin.url # timeout=10
[2020-08-25T16:29:45.635Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-08-25T16:29:45.639Z]  > git merge 34681949dc9ab3fff5b58b2faef46eddb1ec5d2c # timeout=10
[2020-08-25T16:29:45.703Z]  > git rev-parse HEAD^{commit} # timeout=10
[2020-08-25T16:29:45.713Z]  > git config core.sparsecheckout # timeout=10
[2020-08-25T16:29:45.716Z]  > git checkout -f d694e67613f199736f03ee3b46e718ae431c5d7e # timeout=15
[2020-08-25T16:29:45.759Z]  > git rev-list --no-walk 19f80fafdee446c7d1a6462bd49c36883a9a4d13 # timeout=10
[2020-08-25T16:29:45.766Z]  > git rev-parse --verify HEAD # timeout=10
[2020-08-25T16:29:45.770Z] Resetting working tree
[2020-08-25T16:29:45.770Z]  > git reset --hard # timeout=10
[2020-08-25T16:29:45.802Z]  > git clean -fdx # timeout=10
[2020-08-25T16:29:47.267Z] + git fetch https://****:****@github.com/elastic/apm-agent-python.git +refs/pull/*/head:refs/remotes/origin/pr/*
[2020-08-25T16:29:48.693Z] Archiving artifacts
[2020-08-25T16:29:49.306Z] + git rev-parse HEAD
[2020-08-25T16:29:49.668Z] + git rev-parse HEAD
[2020-08-25T16:29:49.968Z] + git rev-parse origin/pr/862
[2020-08-25T16:29:50.014Z] [INFO] githubEnv: Found Git Build Cause: pr
[2020-08-25T16:29:50.281Z] Masking supported pattern matches of $GITHUB_TOKEN
[2020-08-25T16:29:51.183Z] [INFO] githubPrCheckApproved: Title: Support grpc Server & Client (WIP) - User: RyanKung - Author Association: CONTRIBUTOR
[2020-08-25T16:29:51.454Z] ERROR: githubPrCheckApproved: The PR is not allowed to run in the CI yet
[2020-08-25T16:29:51.454Z] ERROR: githubPrCheckApproved: The PR is not allowed to run in the CI yet. (Only users with write permissions can do so.)
[2020-08-25T16:29:51.498Z] [INFO] Let's stop build #27. The PR is not allowed to run in the CI yet
[2020-08-25T16:29:51.516Z] Sleeping for 5 sec
[2020-08-25T16:29:52.534Z] Stage "Sanity checks" skipped due to earlier failure(s)
[2020-08-25T16:29:52.586Z] Stage "Test" skipped due to earlier failure(s)
[2020-08-25T16:29:52.632Z] Stage "Building packages" skipped due to earlier failure(s)
[2020-08-25T16:29:52.676Z] Stage "Integration Tests" skipped due to earlier failure(s)
[2020-08-25T16:29:52.722Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2020-08-25T16:29:52.821Z] Stage "Prepare Release" skipped due to earlier failure(s)
[2020-08-25T16:29:52.844Z] Stage "Prepare Release" skipped due to earlier failure(s)
[2020-08-25T16:29:52.892Z] Stage "Prepare Release" skipped due to earlier failure(s)
[2020-08-25T16:29:52.937Z] Stage "Prepare Release" skipped due to earlier failure(s)
[2020-08-25T16:29:53.241Z] Running on worker-854309 in /var/lib/jenkins/workspace/thon_apm-agent-python-mbp_PR-862
[2020-08-25T16:29:53.360Z] [INFO] getVaultSecret: Getting secrets
[2020-08-25T16:29:53.461Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-08-25T16:29:56.016Z] + chmod 755 generate-build-data.sh
[2020-08-25T16:29:56.016Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-862/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-862/runs/27 ABORTED 162158
[2020-08-25T16:29:56.017Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-862/runs/27/steps/?limit=10000 -o steps-info.json
[2020-08-25T16:29:58.355Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-862/runs/27/tests/?status=FAILED -o tests-errors.json
[2020-08-25T16:29:58.355Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-python/apm-agent-python-mbp/PR-862/runs/27/log/ -o pipeline-log.txt

@RyanKung RyanKung changed the title Support grpc server Support grpc server(WIP) Jun 16, 2020
try:
return self.with_transaction(handler_call_details, continuation)
except Exception:
logger.error("Exception during handle transaction", exc_info)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

undefined name 'exc_info'

logger.error("Exception durinng handle request", exc_info=True)


def request_finished(self, result):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too many blank lines (2)

]

def call(self, module, method, wrapped, instance, args, kwargs):
channel = instance

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

local variable 'channel' is assigned to but never used

@RyanKung RyanKung changed the title Support grpc server(WIP) Support grpc Server & Client (WIP) Jun 27, 2020
@beniwohli beniwohli self-assigned this Nov 3, 2020
@Zioyi
Copy link

Zioyi commented Dec 12, 2020

Hi, I am care about this PR, how is going?[excited]

@skywalkerisnull
Copy link

I am also very interested in this PR. Is there anything we can do to get the ball rolling again @RyanKung @beniwohli

@yswtrue
Copy link

yswtrue commented May 21, 2021

Hi, when will this RP merge?

@upcFrost
Copy link

upcFrost commented Jul 4, 2022

Tried the code from this PR. My findings so far:

  • won't work with grpc.aio,
  • won't work with .future
  • it assumes that the traceparent is the first header in handler_call_details.invocation_metadata[0]._asdict(), which is not always the case. Also it doesn't seem to take the tracestate into account.

Asyncio and headers are easy to fix, but im not sure about the future.

@RyanKung RyanKung closed this Jul 9, 2022
@RyanKung
Copy link
Contributor Author

RyanKung commented Jul 9, 2022

Sorry guys, I don't have enough time to finish this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants