feat(o11y): Attach tracing context to ClientActor work items #7773
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Use a faster way of converting `shard_id` to a string. Crate `itoa` provides the fastest way of converting ints to strings. Snapshot of metrics from a mainnet node. Note that some metrics are not incremented by 1 (`inc()`), but rather by a larger number using `inc_by()`. ``` 1948843343 near_chunk_cache_hits # Not hot 399446179 near_chunk_cache_misses # Not hot 376782133 near_shard_cache_hits # Not hot 276622570 near_peer_manager_messages_time_count 239378614 near_applied_trie_deletions # Optimized 206898552 near_database_op_latency_by_op_and_column_count 184492507 near_peer_message_sent_by_type_total 179740890 near_peer_message_received_total 179344494 near_peer_message_received_by_type_total 168098152 near_applied_trie_insertions # Optimized 160684011 near_shard_cache_pop_hits # Not hot 100536077 near_client_triggers_time_count 100064991 near_requests_count_by_type_total 98868135 near_client_messages_processing_time_count 98868135 near_client_messages_count 97408226 near_peer_client_message_received_by_type_total 94021301 near_network_routed_msg_latency_count 80880283 partial_encoded_chunk_request_delay_count 78394032 near_shard_cache_gc_pop_misses 51277767 partial_encoded_chunk_response_delay_count 43248133 near_partial_encoded_chunk_request_processing_time_count 22664046 near_shard_cache_misses # Not hot ```
…work items. Change `trace_span` to `debug_span` for messages handled by PeerActor. The reason is that this lets us compute the delay between a work item being passed between different actors.
…work items. Change `trace_span` to `debug_span` for messages handled by PeerActor. The reason is that this lets us compute the delay between a work item being passed between different actors.
Refactor attaching span context
mina86
approved these changes
Oct 6, 2022
nagisa
reviewed
Oct 10, 2022
| /// This lets us trace execution across several actix Actors. | ||
| #[derive(actix::Message, Debug)] | ||
| #[rtype(result = "<T as actix::Message>::Result")] | ||
| pub struct WithSpanContext<T: actix::Message> { |
There was a problem hiding this comment.
If we ever want to make this more generally applicable in the future, we could implement the actix::Message trait manually, thus removing the T: actix::Message bound here. It is fine as-is for now, though. Maybe we will never get to the point where we need this sort of functionality elsehwere.
nikurt
added a commit
that referenced
this pull request
Nov 9, 2022
Also refactor attaching span context: Use a extension trait to easily wrap actix messages into a struct `WithSpanContext` that is initialized with the current context. cc: @nagisa
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Also refactor attaching span context: Use a extension trait to easily
wrap actix messages into a struct
WithSpanContextthat isinitialized with the current context.
cc: @nagisa