Consider adding short keys for JSON encoding #412
Labels
wontfix
This will not be worked on
Comments
9 tasks
|
Here is size comparison using one or two letter JSON keys: The uncompressed JSON improvement with short keys is around 1.3-1.5 times compared to the regular keys, which use full field names. |
tigrannajaryan
added a commit
to tigrannajaryan/opentelemetry-proto
that referenced
this issue
Jul 5, 2022
Resolves open-telemetry#412 This change sets short one or two letter keys for all fields when JSON encoding is used. This results in about 1.3-1.5 times smaller uncompressed payload. Unfortunately this is a breaking change for default configuration of Protobuf/JSON marshaller, which marshals field names in lowerCamelCase. This is not a breaking change for marshals which use the "OrigName=true" JSON marshaling option. Nothing changes in the output when "OrigName=true" is used. I do not see an easy way to make this change gracefully. It will require duplicating the entire proto, give the duplicate messages different names, then handle the duplicates in the receivers. It is quite a lot of work that can be also error-prone. I think in this particular case we should not attempt to handle it gracefully and simply still to our formal stability guarantees, which for JSON are "Alpha" and allow any changes any time. ### Example Outputs Current JSON output, using default lowerCamelCase marshaller: ```json { "resource_spans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "int_value": "12345678" } }, { "key": "Pid", "value": { "int_value": "1234" } }, { "key": "HostName", "value": { "string_value": "fakehost" } }, { "key": "ServiceName", "value": { "string_value": "generator" } } ] }, "scope_spans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "trace_id": "AQAAAAAAAADw3rwKeFY0Eg==", "span_id": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "start_time_unix_nano": "1572516672000000013", "end_time_unix_nano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "string_value": "!##$" } } ], "events": [ { "time_unix_nano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "int_value": "1" } } ] } ] } ] } ] } ] } ``` Current JSON output, using OrigName=true marshaller. ```json { "resource_spans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "int_value": "12345678" } }, { "key": "Pid", "value": { "int_value": "1234" } }, { "key": "HostName", "value": { "string_value": "fakehost" } }, { "key": "ServiceName", "value": { "string_value": "generator" } } ] }, "scope_spans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "trace_id": "AQAAAAAAAADw3rwKeFY0Eg==", "span_id": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "start_time_unix_nano": "1572516672000000013", "end_time_unix_nano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "string_value": "!##$" } } ], "events": [ { "time_unix_nano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "int_value": "1" } } ] } ] } ] } ] } ] } ``` New format, using short keys proposed in this change: ```json { "s": [ { "r": { "a": [ { "k": "StartTimeUnixnano", "v": { "i": "12345678" } }, { "k": "Pid", "v": { "i": "1234" } }, { "k": "HostName", "v": { "s": "fakehost" } }, { "k": "ServiceName", "v": { "s": "generator" } } ] }, "s": [ { "i": { "n": "io.opentelemetry" }, "s": [ { "ti": "AQAAAAAAAADw3rwKeFY0Eg==", "si": "AQAAAAAAAAA=", "n": "load-generator-span", "k": "SPAN_KIND_CLIENT", "s": "1572516672000000013", "e": "1572516672000000013", "a": [ { "k": "db.mongodb.collection", "v": { "s": "!##$" } } ], "ev": [ { "t": "1572516672000000013", "a": [ { "k": "te", "v": { "i": "1" } } ] } ] } ] } ] } ] } ```
tigrannajaryan
added a commit
to tigrannajaryan/opentelemetry-proto
that referenced
this issue
Jul 5, 2022
Resolves open-telemetry#412 This change sets short one or two letter keys for all fields when JSON encoding is used. This results in about 1.3-1.5 times smaller uncompressed payload. Unfortunately this is a breaking change for default configuration of Protobuf/JSON marshaler, which marshals field names in lowerCamelCase. This is not a breaking change for marshalers which use the "OrigName=true" JSON marshaling option. Nothing changes in the output when "OrigName=true" is used. I do not see an easy way to make this change gracefully. It will require duplicating the entire proto, give the duplicate messages diffent names, then handle the duplicates in the receivers. It is quite a lot of work that can be also error prone. I think in this pasrticular case we should not attenmpt to handle it gracefully and simply still to our formal stability guarantees, which for JSON are "Alpha" and allow any changes any time. ### Examlple Outputs Current JSON output before this change, using default lowerCamelCase marshaler: ```json { "resourceSpans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "intValue": "12345678" } }, { "key": "Pid", "value": { "intValue": "1234" } }, { "key": "HostName", "value": { "stringValue": "fakehost" } }, { "key": "ServiceName", "value": { "stringValue": "generator" } } ] }, "scopeSpans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "traceId": "AQAAAAAAAADw3rwKeFY0Eg==", "spanId": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "startTimeUnixNano": "1572516672000000013", "endTimeUnixNano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "stringValue": "!##$" } } ], "events": [ { "timeUnixNano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "intValue": "1" } } ] } ] } ] } ] } ] } ``` JSON output before and after this change, using OrigName=true marshaler. ```json { "resource_spans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "int_value": "12345678" } }, { "key": "Pid", "value": { "int_value": "1234" } }, { "key": "HostName", "value": { "string_value": "fakehost" } }, { "key": "ServiceName", "value": { "string_value": "generator" } } ] }, "scope_spans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "trace_id": "AQAAAAAAAADw3rwKeFY0Eg==", "span_id": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "start_time_unix_nano": "1572516672000000013", "end_time_unix_nano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "string_value": "!##$" } } ], "events": [ { "time_unix_nano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "int_value": "1" } } ] } ] } ] } ] } ] } ``` JSON output after this change, using proposed short keys and default lowerCamelCase marshaler: ```json { "s": [ { "r": { "a": [ { "k": "StartTimeUnixnano", "v": { "i": "12345678" } }, { "k": "Pid", "v": { "i": "1234" } }, { "k": "HostName", "v": { "s": "fakehost" } }, { "k": "ServiceName", "v": { "s": "generator" } } ] }, "s": [ { "i": { "n": "io.opentelemetry" }, "s": [ { "ti": "AQAAAAAAAADw3rwKeFY0Eg==", "si": "AQAAAAAAAAA=", "n": "load-generator-span", "k": "SPAN_KIND_CLIENT", "s": "1572516672000000013", "e": "1572516672000000013", "a": [ { "k": "db.mongodb.collection", "v": { "s": "!##$" } } ], "ev": [ { "t": "1572516672000000013", "a": [ { "k": "te", "v": { "i": "1" } } ] } ] } ] } ] } ] } ```
tigrannajaryan
added a commit
to tigrannajaryan/opentelemetry-proto
that referenced
this issue
Jul 5, 2022
Resolves open-telemetry#412 This change sets short one or two letter keys for all fields when JSON encoding is used. This results in about 1.3-1.5 times smaller uncompressed payload. Unfortunately this is a breaking change for default configuration of Protobuf/JSON marshaler, which marshals field names in lowerCamelCase. This is not a breaking change for marshalers which use the "OrigName=true" JSON marshaling option. Nothing changes in the output when "OrigName=true" is used. I do not see an easy way to make this change gracefully. It will require duplicating the entire proto, give the duplicate messages diffent names, then handle the duplicates in the receivers. It is quite a lot of work that can be also error prone. I think in this pasrticular case we should not attenmpt to handle it gracefully and simply still to our formal stability guarantees, which for JSON are "Alpha" and allow any changes any time. ### Examlple Outputs Current JSON output before this change, using default lowerCamelCase marshaler: ```json { "resourceSpans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "intValue": "12345678" } }, { "key": "Pid", "value": { "intValue": "1234" } }, { "key": "HostName", "value": { "stringValue": "fakehost" } }, { "key": "ServiceName", "value": { "stringValue": "generator" } } ] }, "scopeSpans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "traceId": "AQAAAAAAAADw3rwKeFY0Eg==", "spanId": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "startTimeUnixNano": "1572516672000000013", "endTimeUnixNano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "stringValue": "!##$" } } ], "events": [ { "timeUnixNano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "intValue": "1" } } ] } ] } ] } ] } ] } ``` JSON output before and after this change, using OrigName=true marshaler. ```json { "resource_spans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "int_value": "12345678" } }, { "key": "Pid", "value": { "int_value": "1234" } }, { "key": "HostName", "value": { "string_value": "fakehost" } }, { "key": "ServiceName", "value": { "string_value": "generator" } } ] }, "scope_spans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "trace_id": "AQAAAAAAAADw3rwKeFY0Eg==", "span_id": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "start_time_unix_nano": "1572516672000000013", "end_time_unix_nano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "string_value": "!##$" } } ], "events": [ { "time_unix_nano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "int_value": "1" } } ] } ] } ] } ] } ] } ``` JSON output after this change, using proposed short keys and default lowerCamelCase marshaler: ```json { "s": [ { "r": { "a": [ { "k": "StartTimeUnixnano", "v": { "i": "12345678" } }, { "k": "Pid", "v": { "i": "1234" } }, { "k": "HostName", "v": { "s": "fakehost" } }, { "k": "ServiceName", "v": { "s": "generator" } } ] }, "s": [ { "i": { "n": "io.opentelemetry" }, "s": [ { "ti": "AQAAAAAAAADw3rwKeFY0Eg==", "si": "AQAAAAAAAAA=", "n": "load-generator-span", "k": "SPAN_KIND_CLIENT", "s": "1572516672000000013", "e": "1572516672000000013", "a": [ { "k": "db.mongodb.collection", "v": { "s": "!##$" } } ], "ev": [ { "t": "1572516672000000013", "a": [ { "k": "te", "v": { "i": "1" } } ] } ] } ] } ] } ] } ```
tigrannajaryan
added a commit
to tigrannajaryan/opentelemetry-proto
that referenced
this issue
Jul 5, 2022
Resolves open-telemetry#412 This change sets short one or two letter keys for all fields when JSON encoding is used. This results in about 1.3-1.5 times smaller uncompressed payload. Unfortunately this is a breaking change for default configuration of Protobuf/JSON marshaller, which marshals field names in lowerCamelCase. This is not a breaking change for marshallers which use the "OrigName=true" JSON marshaling option. Nothing changes in the output when "OrigName=true" is used. I do not see an easy way to make this change gracefully. It will require duplicating the entire proto, give the duplicate messages different names, then handle the duplicates in the receivers. It is quite a lot of work that can be also error-prone. I think in this particular case we should not attempt to handle it gracefully and simply still to our formal stability guarantees, which for JSON are "Alpha" and allow any changes any time. ### Example Outputs Current JSON output before this change, using default lowerCamelCase marshaller: ```json { "resourceSpans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "intValue": "12345678" } }, { "key": "Pid", "value": { "intValue": "1234" } }, { "key": "HostName", "value": { "stringValue": "fakehost" } }, { "key": "ServiceName", "value": { "stringValue": "generator" } } ] }, "scopeSpans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "traceId": "AQAAAAAAAADw3rwKeFY0Eg==", "spanId": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "startTimeUnixNano": "1572516672000000013", "endTimeUnixNano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "stringValue": "!##$" } } ], "events": [ { "timeUnixNano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "intValue": "1" } } ] } ] } ] } ] } ] } ``` JSON output before and after this change, using OrigName=true marshaller. ```json { "resource_spans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "int_value": "12345678" } }, { "key": "Pid", "value": { "int_value": "1234" } }, { "key": "HostName", "value": { "string_value": "fakehost" } }, { "key": "ServiceName", "value": { "string_value": "generator" } } ] }, "scope_spans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "trace_id": "AQAAAAAAAADw3rwKeFY0Eg==", "span_id": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "start_time_unix_nano": "1572516672000000013", "end_time_unix_nano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "string_value": "!##$" } } ], "events": [ { "time_unix_nano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "int_value": "1" } } ] } ] } ] } ] } ] } ``` JSON output after this change, using proposed short keys and default lowerCamelCase marshaller: ```json { "s": [ { "r": { "a": [ { "k": "StartTimeUnixnano", "v": { "i": "12345678" } }, { "k": "Pid", "v": { "i": "1234" } }, { "k": "HostName", "v": { "s": "fakehost" } }, { "k": "ServiceName", "v": { "s": "generator" } } ] }, "s": [ { "i": { "n": "io.opentelemetry" }, "s": [ { "ti": "AQAAAAAAAADw3rwKeFY0Eg==", "si": "AQAAAAAAAAA=", "n": "load-generator-span", "k": "SPAN_KIND_CLIENT", "s": "1572516672000000013", "e": "1572516672000000013", "a": [ { "k": "db.mongodb.collection", "v": { "s": "!##$" } } ], "ev": [ { "t": "1572516672000000013", "a": [ { "k": "te", "v": { "i": "1" } } ] } ] } ] } ] } ] } ```
tigrannajaryan
added a commit
to tigrannajaryan/opentelemetry-proto
that referenced
this issue
Jul 5, 2022
Resolves open-telemetry#412 This change sets short one or two letter keys for all fields when JSON encoding is used. This results in about 1.3-1.5 times smaller uncompressed payload. Here is size comparison using some sample data from exp-otelproto test bench: ``` ===== Encoded sizes Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Logs 52577 by [1.000] 4601 by [1.000] ShortKeys/Logs 39668 by [1.325] 4344 by [1.059] Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Trace/Attribs 41704 by [1.000] 3189 by [1.000] ShortKeys/Trace/Attribs 31998 by [1.303] 3353 by [0.951] Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Trace/Events 49302 by [1.000] 1917 by [1.000] ShortKeys/Trace/Events 34396 by [1.433] 2430 by [0.789] Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Metric/Histogram 42376 by [1.000] 1067 by [1.000] ShortKeys/Metric/Histogram 27071 by [1.565] 839 by [1.272] Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Metric/MixOne 184836 by [1.000] 2778 by [1.000] ShortKeys/Metric/MixOne 119031 by [1.553] 2143 by [1.296] Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Metric/MixSeries 707615 by [1.000] 11482 by [1.000] ShortKeys/Metric/MixSeries 457010 by [1.548] 9829 by [1.168] ``` Unfortunately this is a breaking change for default configuration of Protobuf/JSON marshaler, which marshals field names in lowerCamelCase. This is not a breaking change for marshalers which use the "OrigName=true" JSON marshaling option. Nothing changes in the output when "OrigName=true" is used. I do not see an easy way to make this change gracefully. It will require duplicating the entire proto, give the duplicate messages diffent names, then handle the duplicates in the receivers. It is quite a lot of work that can be also error prone. I think in this pasrticular case we should not attenmpt to handle it gracefully and simply still to our formal stability guarantees, which for JSON are "Alpha" and allow any changes any time. ### Examlple Outputs Current JSON output before this change, using default lowerCamelCase marshaler: ```json { "resourceSpans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "intValue": "12345678" } }, { "key": "Pid", "value": { "intValue": "1234" } }, { "key": "HostName", "value": { "stringValue": "fakehost" } }, { "key": "ServiceName", "value": { "stringValue": "generator" } } ] }, "scopeSpans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "traceId": "AQAAAAAAAADw3rwKeFY0Eg==", "spanId": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "startTimeUnixNano": "1572516672000000013", "endTimeUnixNano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "stringValue": "!##$" } } ], "events": [ { "timeUnixNano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "intValue": "1" } } ] } ] } ] } ] } ] } ``` JSON output before and after this change, using OrigName=true marshaler. ```json { "resource_spans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "int_value": "12345678" } }, { "key": "Pid", "value": { "int_value": "1234" } }, { "key": "HostName", "value": { "string_value": "fakehost" } }, { "key": "ServiceName", "value": { "string_value": "generator" } } ] }, "scope_spans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "trace_id": "AQAAAAAAAADw3rwKeFY0Eg==", "span_id": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "start_time_unix_nano": "1572516672000000013", "end_time_unix_nano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "string_value": "!##$" } } ], "events": [ { "time_unix_nano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "int_value": "1" } } ] } ] } ] } ] } ] } ``` JSON output after this change, using proposed short keys and default lowerCamelCase marshaler: ```json { "s": [ { "r": { "a": [ { "k": "StartTimeUnixnano", "v": { "i": "12345678" } }, { "k": "Pid", "v": { "i": "1234" } }, { "k": "HostName", "v": { "s": "fakehost" } }, { "k": "ServiceName", "v": { "s": "generator" } } ] }, "s": [ { "i": { "n": "io.opentelemetry" }, "s": [ { "ti": "AQAAAAAAAADw3rwKeFY0Eg==", "si": "AQAAAAAAAAA=", "n": "load-generator-span", "k": "SPAN_KIND_CLIENT", "s": "1572516672000000013", "e": "1572516672000000013", "a": [ { "k": "db.mongodb.collection", "v": { "s": "!##$" } } ], "ev": [ { "t": "1572516672000000013", "a": [ { "k": "te", "v": { "i": "1" } } ] } ] } ] } ] } ] } ```
tigrannajaryan
added a commit
to tigrannajaryan/opentelemetry-proto
that referenced
this issue
Jul 5, 2022
Resolves open-telemetry#412 This change sets short one or two letter keys for all fields when JSON encoding is used. This results in about 1.3-1.5 times smaller uncompressed payload. Here is size comparison using some sample data from exp-otelproto test bench: ``` ===== Encoded sizes Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Logs 52577 by [1.000] 4601 by [1.000] ShortKeys/Logs 39668 by [1.325] 4344 by [1.059] Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Trace/Attribs 41704 by [1.000] 3189 by [1.000] ShortKeys/Trace/Attribs 31998 by [1.303] 3060 by [1.042] Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Trace/Events 49302 by [1.000] 1917 by [1.000] ShortKeys/Trace/Events 34396 by [1.433] 1806 by [1.061] Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Metric/Histogram 42376 by [1.000] 1067 by [1.000] ShortKeys/Metric/Histogram 27071 by [1.565] 839 by [1.272] Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Metric/MixOne 184836 by [1.000] 2778 by [1.000] ShortKeys/Metric/MixOne 119031 by [1.553] 2143 by [1.296] Encoding Uncomp/json[Improved] zlib/json[Improved] OTLP 0.18/Metric/MixSeries 707615 by [1.000] 11482 by [1.000] ShortKeys/Metric/MixSeries 457010 by [1.548] 9829 by [1.168] ``` Unfortunately **this is a breaking change** for default configuration of Protobuf/JSON marshaler, which marshals field names in lowerCamelCase. This is not a breaking change for marshalers which use the "OrigName=true" JSON marshalling option. Nothing changes in the output when "OrigName=true" is used. I do not see an easy way to make this change gracefully. It will require duplicating the entire proto, give the duplicate messages different names, then handle the duplicates in the receivers. It is quite a lot of work that can be also error prone. I think in this particular case we should not attempt to handle it gracefully and simply still to our formal stability guarantees, which for JSON are "Alpha" and allow any changes any time. ### Example Outputs Current JSON output before this change, using default lowerCamelCase marshaler: ```json { "resourceSpans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "intValue": "12345678" } }, { "key": "Pid", "value": { "intValue": "1234" } }, { "key": "HostName", "value": { "stringValue": "fakehost" } }, { "key": "ServiceName", "value": { "stringValue": "generator" } } ] }, "scopeSpans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "traceId": "AQAAAAAAAADw3rwKeFY0Eg==", "spanId": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "startTimeUnixNano": "1572516672000000013", "endTimeUnixNano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "stringValue": "!##$" } } ], "events": [ { "timeUnixNano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "intValue": "1" } } ] } ] } ] } ] } ] } ``` JSON output before and after this change, using OrigName=true marshaler. ```json { "resource_spans": [ { "resource": { "attributes": [ { "key": "StartTimeUnixnano", "value": { "int_value": "12345678" } }, { "key": "Pid", "value": { "int_value": "1234" } }, { "key": "HostName", "value": { "string_value": "fakehost" } }, { "key": "ServiceName", "value": { "string_value": "generator" } } ] }, "scope_spans": [ { "scope": { "name": "io.opentelemetry" }, "spans": [ { "trace_id": "AQAAAAAAAADw3rwKeFY0Eg==", "span_id": "AQAAAAAAAAA=", "name": "load-generator-span", "kind": "SPAN_KIND_CLIENT", "start_time_unix_nano": "1572516672000000013", "end_time_unix_nano": "1572516672000000013", "attributes": [ { "key": "db.mongodb.collection", "value": { "string_value": "!##$" } } ], "events": [ { "time_unix_nano": "1572516672000000013", "attributes": [ { "key": "te", "value": { "int_value": "1" } } ] } ] } ] } ] } ] } ``` JSON output after this change, using proposed short keys and default lowerCamelCase marshaler: ```json { "s": [ { "r": { "a": [ { "k": "StartTimeUnixnano", "v": { "i": "12345678" } }, { "k": "Pid", "v": { "i": "1234" } }, { "k": "HostName", "v": { "s": "fakehost" } }, { "k": "ServiceName", "v": { "s": "generator" } } ] }, "s": [ { "i": { "n": "io.opentelemetry" }, "s": [ { "ti": "AQAAAAAAAADw3rwKeFY0Eg==", "si": "AQAAAAAAAAA=", "n": "load-generator-span", "k": "SPAN_KIND_CLIENT", "s": "1572516672000000013", "e": "1572516672000000013", "a": [ { "k": "db.mongodb.collection", "v": { "s": "!##$" } } ], "ev": [ { "t": "1572516672000000013", "a": [ { "k": "te", "v": { "i": "1" } } ] } ] } ] } ] } ] } ```
|
We could even use the numeric tag values from the binary protocol (stringized) as JSON keys (decimal or hex or base64). Then we would also get more stable / renaming-resistant keys. |
|
To summarize the findings in the PR:
We also discussed this in the Spec meeting yesterday and no-one in the call believed that we need the JSON with short keys given that other options are available. JSON with short keys loses its readability, while not gaining as much in the performance as the other options we have. Given the above, I am marking this issue as "won't do". If there are new arguments the issue can be re-opened (but it must happen before we declare JSON format Stable). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
json_nameoption can be used to shorten keys for JSON encoding. This can result in significant size savings for uncompressed JSON, which appears to be important for e.g. open-telemetry/oteps#208 (comment)The text was updated successfully, but these errors were encountered: