Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using AWS resource detector with auto-instrumentation #2370

Closed
abstractOwl opened this issue Mar 26, 2024 · 1 comment · Fixed by #2382
Closed

Using AWS resource detector with auto-instrumentation #2370

abstractOwl opened this issue Mar 26, 2024 · 1 comment · Fixed by #2382
Labels
bug Something isn't working

Comments

@abstractOwl
Copy link
Contributor

abstractOwl commented Mar 26, 2024

Describe your environment Describe any aspect of your environment relevant to the problem, including your Python version, platform, version numbers of installed dependencies, information about your cloud hosting provider, etc. If you're reporting a problem with a specific version of a library in this repo, please check whether the problem has been fixed on main.

Running a basic Flask application instrumented using opentelemetry-instrument in Docker python:3.12.2-slim. Set environment variable OTEL_EXPERIMENTAL_RESOURCE_DETECTORS=aws_ecs.

Dependency versions:

  • Flask==3.0.2
  • opentelemetry-distro[otlp]==0.41b0
  • opentelemetry-sdk-extension-aws==2.0.1

Steps to reproduce
Describe exactly how to reproduce the error. Include a code sample if applicable.

Run docker-compose up -d with entrypoint opentelemetry-instrument flask run --host=0.0.0.0 --port=5000

What is the expected behavior?
What did you expect to see?
Start up using ECS resource detector

What is the actual behavior?
What did you see instead?

web_1             | Distribution distro configuration failed
web_1             | Traceback (most recent call last):
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/instrumentation/auto_instrumentation/_load.py",
line 40, in _load_distro
web_1             |     distro = entry_point.load()()
web_1             |              ^^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 2471, in load
web_1             |     return self.resolve()
web_1             | ^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 2477, in resolve
web_1             |     module = __import__(self.module_name,
fromlist=['__name__'], level=0)
web_1             |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/distro/__init__.py",
line 22, in <module>
web_1             |     from opentelemetry.sdk._configuration import
_OTelSDKConfigurator
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_configuration/__init__.py",
line 36, in <module>
web_1             |     from opentelemetry.sdk._logs import
LoggerProvider, LoggingHandler
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/__init__.py",
line 16, in <module>
web_1             |     from opentelemetry.sdk._logs._internal import (
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/_internal/__init__.py",
line 579, in <module>
web_1             |     class LoggerProvider(APILoggerProvider):
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/_internal/__init__.py",
line 582, in LoggerProvider
web_1             |     resource: Resource = Resource.create(),
web_1             |                          ^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/resources/__init__.py",
line 185, in create
web_1             |     next(
web_1             | StopIteration
web_1             | Failed to auto initialize opentelemetry
web_1             | Traceback (most recent call last):
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py",
line 36, in initialize
web_1             |     distro = _load_distro()
web_1             |              ^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/instrumentation/auto_instrumentation/_load.py",
line 55, in _load_distro
web_1             |     raise exc
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/instrumentation/auto_instrumentation/_load.py",
line 40, in _load_distro
web_1             |     distro = entry_point.load()()
web_1             |              ^^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 2471, in load
web_1             |     return self.resolve()
web_1             |            ^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 2477, in resolve
web_1             |     module = __import__(self.module_name,
fromlist=['__name__'], level=0)
web_1             |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/distro/__init__.py",
line 22, in <module>
web_1             |     from opentelemetry.sdk._configuration import
_OTelSDKConfigurator
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_configuration/__init__.py",
line 36, in <module>
web_1             |     from opentelemetry.sdk._logs import
LoggerProvider, LoggingHandler
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/__init__.py",
line 16, in <module>
web_1             |     from opentelemetry.sdk._logs._internal import (
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/_internal/__init__.py",
line 579, in <module>
web_1             |     class LoggerProvider(APILoggerProvider):
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/_internal/__init__.py",
line 582, in LoggerProvider
web_1             |     resource: Resource = Resource.create(),
web_1             |                          ^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/resources/__init__.py",
line 185, in create
web_1             |     next(
web_1             | StopIteration
web_1             |  * Debug mode: on
web_1             | WARNING: This is a development server. Do not use it
in a production deployment. Use a production WSGI server instead.
web_1             |  * Running on all addresses (0.0.0.0)
web_1             |  * Running on http://127.0.0.1:4000/
web_1             |  * Running on http://192.168.48.3:4000/
web_1             | Press CTRL+C to quit
web_1             |  * Restarting with stat
web_1             |  * Debugger is active!
web_1             |  * Debugger PIN: 102-228-721
web_1             | 192.168.48.1 - - [26/Mar/2024 07:50:59] "GET /
HTTP/1.1" 200 -

Additional context
Add any other context about the problem here.

@abstractOwl
Copy link
Contributor Author

abstractOwl commented Mar 29, 2024

I did a little digging and I think I might see what's going on:

Looking at the code, the auto-instrumentation looks for resource detectors registered under the "opentelemetry_resource_detector" namespace:
https://github.com/open-telemetry/opentelemetry-python/blob/721beb8b530e7a830c1e27b70c2fb9af6465baf1/opentelemetry-sdk/src/opentelemetry/sdk/resources/__init__.py#L191-L194

I see that the Azure resource detectors have been registered using that entry point here:

[project.entry-points.opentelemetry_resource_detector]
azure_app_service = "opentelemetry.resource.detector.azure.app_service:AzureAppServiceResourceDetector"
azure_vm = "opentelemetry.resource.detector.azure.vm:AzureVMResourceDetector"

So a corresponding change would need to be made to the sdk-aws pyproject.toml file here:

[project.entry-points.opentelemetry_id_generator]
xray = "opentelemetry.sdk.extension.aws.trace.aws_xray_id_generator:AwsXRayIdGenerator"

I was able to confirm this by adding the following line to my Dockerfile, which writes the expected section to the package entrypoints:

RUN echo "\n\n[opentelemetry_resource_detector]\naws_elastic_beanstalk =
opentelemetry.sdk.extension.aws.resource.beanstalk:AwsBeanstalkResourceDetector"
 >>
/usr/local/lib/python3.12/site-packages/opentelemetry_sdk_extension_aws-2.0.1.dist-info/entry_points.txt

Using a fake /var/elasticbeanstalk/xray/environment.conf file, I was able to get this output in my Docker logs:

web_1             | {
web_1             |     "resource_metrics": [
web_1             |         {
web_1             |             "resource": {
web_1             |                 "attributes": {
web_1             |                     "telemetry.sdk.language": "python",
web_1             |                     "telemetry.sdk.name":
"opentelemetry",
web_1             |                     "telemetry.sdk.version": "1.20.0",
web_1             |                     "cloud.provider": "aws",
web_1             |                     "cloud.platform":
"aws_elastic_beanstalk",
web_1             |                     "service.name": "otel_test",
web_1             |                     "service.instance.id": "abc123",
web_1             |                     "service.namespace": "12345",
web_1             |                     "service.version": "0.0.1",
web_1             |                     "telemetry.auto.version": "0.41b0"
web_1             |                 },
web_1             |                 "schema_url": ""
web_1             |             },
web_1             |             "scope_metrics": [],
web_1             |             "schema_url": ""
web_1             |         }
web_1             |     ]
web_1             | }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant