Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix endianness of Jaeger IDs for transmission #832

Merged
merged 4 commits into from
Jun 10, 2021

Conversation

ThomsonTan
Copy link
Contributor

Fixes #831

Changes

Detect the host endianness and convert all IDs to big endian if needed.

For significant contributions please make sure you have completed the following items:

  • CHANGELOG.md updated for non-trivial changes
  • Unit tests have been added
  • Changes in public API reviewed

@ThomsonTan ThomsonTan requested a review from a team June 7, 2021 19:25
@@ -39,12 +39,25 @@ void Recordable::PopulateAttribute(nostd::string_view key, const common::Attribu
void Recordable::SetIdentity(const trace::SpanContext &span_context,
trace::SpanId parent_span_id) noexcept
{
// IDs should be converted to big endian before transmission.
// https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk_exporters/jaeger.md#ids
#if JAEGER_IS_LITTLE_ENDIAN == 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, to be more precise, something like this?

#define htonll(x) ((1==htonl(1)) ? (x) : ((uint64_t)htonl((x) & 0xFFFFFFFF) << 32) | htonl((x) >> 32))
#define ntohll(x) ((1==ntohl(1)) ? (x) : ((uint64_t)ntohl((x) & 0xFFFFFFFF) << 32) | ntohl((x) >> 32))

Copy link
Member

@lalitb lalitb Jun 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nicely done. this will check the endianness too before performing conversion

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I considered htonl as the first option, but I'd like to avoid the runtime check on endianness and preferred the compiler intrinsic to perform byte order swap for simplicity and slight perf win.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could change the function name to to_big_endian and define it as as direct return for big endian host. In this way, we don't need check the endianness at callsite of this function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Second thought, leave the JAEGER_IS_LITTLE_ENDIAN in the callsite could save us to create bswap_128.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering if it's something you can move into a separate static inline utility function, then it can be reused elsewhere? I think it might be common requirement, to transmit the buffer in big endian, not unique to Jaeger?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far I only saw the requirement in Jaeger. I'd be happen to move this to common utility if it is used in other places.

#elif defined(_WIN32)
# define JAEGER_IS_LITTLE_ENDIAN 1
#else
# error "Endian detection needs to be set up for your compiler"
Copy link
Member

@lalitb lalitb Jun 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about using run-time determination if it's not set by compiler, something like:

    const int value { 0x01 };
    bool isLittleEndian = ( *(static_cast<const unsigned char *>(static_cast<const void *>(&value))) == 0x01);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to avoid the runtime check as it is a static property known at compile time. I think our current definition should cover most real cases if not all.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a reference implementation see https://github.com/abseil/abseil-cpp/blob/master/absl/base/internal/endian.h I think you can use that code (not directly if avoiding dependency is needed) but is a good implementation.

Copy link
Member

@lalitb lalitb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@codecov
Copy link

codecov bot commented Jun 8, 2021

Codecov Report

Merging #832 (dfe421b) into main (5acf4cf) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #832   +/-   ##
=======================================
  Coverage   95.50%   95.50%           
=======================================
  Files         156      156           
  Lines        6614     6614           
=======================================
  Hits         6316     6316           
  Misses        298      298           

Copy link
Contributor

@maxgolov maxgolov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm Okay with the essence of this change. But I think it would have been nicer to refactor the implementation into separate utility header. This refactor won't anyhow degrade the perf, if you make the function static inline. By the looks of it what you are doing is htonllll , which should really be standard utility function. I believe we'd need this in other exporters.

@lalitb lalitb merged commit c2c7976 into open-telemetry:main Jun 10, 2021
@ThomsonTan ThomsonTan deleted the FixJaegerIds branch June 15, 2021 02:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Jaeger should transmit all IDs in big endian
4 participants