symbolicator request failed, "Failed to deserialize the JSON body into the target type: unknown variant portablepdb"
#1860
Comments
|
PortablePDB is a format for .NET... so that's really odd. Could you try upgrading to the 8.x cocoa SDK and see if that stops the errors from happening? |
|
Since upgrading from 22.10 to 22.11 we encounter this exact same issue. My guess would then be this is not related to the SDK being used |
|
Looking at the latest Changing that version to |
|
Ok, that's great to hear! This should get fixed in the release tomorrow then, we will be bumping symbolicator in the 22.12 release. Kind of unfortunate the error message is so unclear :/ |
|
I believe this has nothing to do with the cocoa SDK, but rather the This source configuration is being sent to (and decoded by) symbolicator just in case you wanted to fetch such files from the microsoft symbol server. It is a bit surprising though that the published github release is being used by selfhosted deployments, rather than an uptodate docker image. We have no real release schedule for tagged github releases of symbolicator, and the releases that we do, do not match any self-hosted sentry releases. I believe that is the main problem here. |
As part of the self-hosted release infrastructure, which runs once a month when we do a self-hosted release, we push the latest gcr docker image to dockerhub which is what we use in self-hosted.
Ah I see, so it seems that Sentry <-> Symbolicator were incompatible? I suppose it would be good to add a test for a symbol-related workload to the self-hosted CI (we're looking to expand our tests this quarter). Thank you for pointing that out! |
yes exactly.
That seems like a good idea, yes. To cover this particular usecase, you would need a test that uses all the builtin symbol servers. A Windows minidump perhaps. We should have one already in the main Sentry tests, though those tests are always testing the very latest symbolicator master. |
|
This is fixed in 22.12.0. Please open a new issue if you still encounter problems going forward! |
Self-Hosted Version
22.11.0
CPU Architecture
x86_64
Docker Version
20.10.21
Docker Compose Version
2.12.2
Steps to Reproduce
Upgraded from 22.5.0 to 22.11.0. Uploaded new events from iOS sentry-cocoa 7.9.0.
Expected Result
No errors on the instance's
internalproject?Actual Result
I see an issue similar to #1818 (comment), many many times for "portablepdb" symbols. This was not happening prior to upgrading.
Raw event json:
{ "event_id": "fde7bf53567549d5821cf0df5463bfc8", "project": 1, "release": "backend@22.11.0+60999b0f47b3ffe7f2fbbd5e86d9bb9711e5a2c0", "dist": null, "platform": "python", "message": "Symbolicator request failed", "datetime": "2022-12-11T16:47:58.226467Z", "tags": [ [ "celery_task_id", "a2d6b1fc-8f7b-4790-93a6-a4336fb2171e" ], [ "environment", "production" ], [ "get_default_cache", "true" ], [ "level", "info" ], [ "processing_event_for_project", "13" ], [ "project", "13" ], [ "runtime", "CPython 3.8.13" ], [ "runtime.name", "CPython" ], [ "release", "backend@22.11.0+60999b0f47b3ffe7f2fbbd5e86d9bb9711e5a2c0" ], [ "server_name", "d258f2cdcf50" ], [ "task_name", "sentry.tasks.store.symbolicate_event" ], [ "transaction", "sentry.tasks.store.symbolicate_event" ], [ "used_default_cache", "true" ] ], "_meta": { "extra": { "celery-job": { "kwargs": { "data": { "": { "len": 25 }, "logentry": { "formatted": { "": { "rem": [ [ "!limit", "x" ] ] } } } } } } } }, "_metrics": { "bytes.ingested.event": 5663, "bytes.stored.event": 6489 }, "breadcrumbs": { "values": [ { "timestamp": 1670777278.213607, "type": "redis", "category": "redis", "level": "info", "message": "GET 'c:1:symbolicator:038d40db14de4631aa79c45dc8aa2b21:13'", "data": { "redis.command": "GET", "redis.is_cluster": false, "redis.key": "c:1:symbolicator:038d40db14de4631aa79c45dc8aa2b21:13" } }, { "timestamp": 1670777278.225896, "type": "http", "category": "httplib", "level": "info", "data": { "method": "POST", "reason": "Unprocessable Entity", "status_code": 422, "url": "http://symbolicator:3021/symbolicate?timeout=10&scope=13" } } ] }, "contexts": { "runtime": { "name": "CPython", "version": "3.8.13", "build": "3.8.13 (default, Aug 23 2022, 09:42:19) \n[GCC 10.2.1 20210110]", "type": "runtime" }, "trace": { "trace_id": "cb374a69af9d46ffbefa947103097e9a", "span_id": "a53dbb5856cf63b7", "parent_span_id": "bafc55fef829852a", "op": "tasks.store.symbolicate_event.process_payload", "status": "unknown", "type": "trace" } }, "culprit": "sentry.tasks.store.symbolicate_event", "environment": "production", "extra": { "celery-job": { "args": [], "kwargs": { "cache_key": "e:038d40db14de4631aa79c45dc8aa2b21:13", "data": { "event_id": "038d40db14de4631aa79c45dc8aa2b21", "level": "error", "logentry": { "formatted": "'downloading of changes failed to execute'" }, "logger": "", "platform": "cocoa", "received": 1670777277.013964, "timestamp": 1670777276.559669, "transaction": "MyViewController", "type": "default", "version": "7" }, "event_id": "038d40db14de4631aa79c45dc8aa2b21", "has_attachments": false, "queue_switches": 0, "start_time": 1670777277 }, "task_name": "sentry.tasks.store.symbolicate_event" }, "symbolicator_response": "Failed to deserialize the JSON body into the target type: unknown variant `portablepdb`, expected one of `pe`, `pdb`, `mach_debug`, `mach_code`, `elf_debug`, `elf_code`, `wasm_debug`, `wasm_code`, `breakpad`, `sourcebundle`, `uuidmap`, `bcsymbolmap`, `usym` at line 1 column 425", "sys.argv": [ "[/usr/local/bin/sentry](https://sentry.digart.dev/usr/local/bin/sentry)", "run", "worker" ] }, "fingerprint": [ "{{ default }}" ], "grouping_config": { "enhancements": "eJybzDRxY3J-bm5-npWRgaGlroGxrpHxBABcYgcZ", "id": "newstyle:2019-10-29" }, "hashes": [ "bb803b1415f623e2964910b19afd21f3" ], "key_id": "1", "level": "info", "logentry": { "formatted": "Symbolicator request failed" }, "logger": "", "metadata": { "title": "Symbolicator request failed" }, "modules": { "amqp": "2.6.1", "async-generator": "1.10", "attrs": "19.2.0", "avalara": "20.9.0", "beautifulsoup4": "4.7.1", "billiard": "3.6.4.0", "boto3": "1.22.12", "botocore": "1.25.12", "brotli": "1.0.9", "cachetools": "4.2.4", "celery": "4.4.7", "certifi": "2022.5.18.1", "cffi": "1.15.1", "chardet": "4.0.0", "click": "8.0.4", "confluent-kafka": "1.9.2", "croniter": "0.3.37", "cryptography": "38.0.3", "cssselect": "1.0.3", "cssutils": "2.4.0", "datadog": "0.29.3", "decorator": "5.1.1", "django": "2.2.28", "django-crispy-forms": "1.14.0", "django-pg-zero-downtime-migrations": "0.11", "djangorestframework": "3.12.4", "drf-spectacular": "0.22.1", "email-reply-parser": "0.5.12", "fido2": "0.9.2", "google-api-core": "2.10.1", "google-auth": "1.35.0", "google-cloud-bigtable": "2.11.3", "google-cloud-core": "2.3.2", "google-cloud-functions": "1.8.1", "google-cloud-pubsub": "2.13.6", "google-cloud-spanner": "3.20.0", "google-cloud-storage": "2.5.0", "google-crc32c": "1.3.0", "google-resumable-media": "2.3.3", "googleapis-common-protos": "1.56.4", "grpc-google-iam-v1": "0.12.4", "grpcio": "1.48.1", "grpcio-status": "1.48.1", "h11": "0.13.0", "hiredis": "0.3.1", "idna": "2.10", "inflection": "0.5.1", "isodate": "0.6.1", "jmespath": "0.10.0", "jsonschema": "3.2.0", "kombu": "4.6.11", "lxml": "4.6.5", "maxminddb": "2.0.3", "milksnake": "0.1.5", "mistune": "2.0.4", "mmh3": "3.0.0", "msgpack": "1.0.4", "natsort": "8.1.0", "oauthlib": "3.1.0", "outcome": "1.2.0", "packaging": "21.3", "parsimonious": "0.8.0", "petname": "2.6", "phabricator": "0.7.0", "phonenumberslite": "8.12.0", "pillow": "9.2.0", "pip": "22.0.4", "planout": "0.6.0", "progressbar2": "3.41.0", "proto-plus": "1.22.1", "protobuf": "4.21.6", "psycopg2-binary": "2.8.6", "pyasn1": "0.4.5", "pyasn1-modules": "0.2.4", "pycountry": "17.5.14", "pycparser": "2.21", "pyjwt": "2.4.0", "pyopenssl": "22.0.0", "pyparsing": "3.0.9", "pyrsistent": "0.18.1", "pysocks": "1.7.1", "python-dateutil": "2.8.1", "python-memcached": "1.59", "python-rapidjson": "1.4", "python-u2flib-server": "5.0.0", "python-utils": "3.3.3", "python3-saml": "1.14.0", "pytz": "2018.9", "pyuwsgi": "2.0.20", "pyvat": "1.3.15", "pyyaml": "5.4", "rb": "1.9.0", "redis": "3.4.1", "redis-py-cluster": "2.1.0", "reportlab": "3.6.1", "requests": "2.25.1", "requests-oauthlib": "1.2.0", "rfc3339-validator": "0.1.2", "rfc3986-validator": "0.1.1", "rsa": "4.8", "s3transfer": "0.5.2", "selenium": "4.3.0", "sentry": "22.11.0", "sentry-arroyo": "1.1.0", "sentry-relay": "0.8.15", "sentry-sdk": "1.11.0", "setuptools": "56.0.0", "simplejson": "3.17.6", "six": "1.16.0", "sniffio": "1.2.0", "snuba-sdk": "1.0.1", "sortedcontainers": "2.4.0", "soupsieve": "2.3.2.post1", "sqlparse": "0.3.0", "statsd": "3.3.0", "stripe": "2.61.0", "structlog": "21.1.0", "symbolic": "10.1.1", "toronado": "0.1.0", "trio": "0.21.0", "trio-websocket": "0.9.2", "typing-extensions": "3.10.0.2", "ua-parser": "0.10.0", "unidiff": "0.7.4", "uritemplate": "4.1.1", "urllib3": "1.26.11", "vine": "1.3.0", "wheel": "0.37.1", "wsproto": "1.1.0", "xmlsec": "1.3.12", "zstandard": "0.18.0" }, "nodestore_insert": 1670777279.312781, "received": 1670777278.242471, "sdk": { "name": "sentry.python.django", "version": "1.11.0", "integrations": [ "argv", "atexit", "celery", "dedupe", "django", "django_atomic", "excepthook", "logging", "modules", "redis", "rust_info", "stdlib", "threading" ], "packages": [ { "name": "pypi:sentry-sdk", "version": "1.11.0" } ] }, "timestamp": 1670777278.226467, "title": "Symbolicator request failed", "transaction": "sentry.tasks.store.symbolicate_event", "transaction_info": {}, "type": "default", "version": "7", "location": null }Event ID
No response
The text was updated successfully, but these errors were encountered: