Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New component: DNS lookup processor #34398

Open
3 tasks
kaisecheng opened this issue Aug 2, 2024 · 2 comments
Open
3 tasks

New component: DNS lookup processor #34398

kaisecheng opened this issue Aug 2, 2024 · 2 comments
Labels
Sponsor Needed New component seeking sponsor

Comments

@kaisecheng
Copy link
Contributor

kaisecheng commented Aug 2, 2024

The purpose and use-cases of the new component

The DNS lookup Processor is for resolving hostnames to IP addresses and vice versa. It is particularly useful when the GeoIP processor receives a hostname and needs to convert it to an IP address in order to resolve the geo location.
The processor performs A record/CNAME record lookups to resolve hostnames to IP addresses, and PTR record lookups for reverse DNS resolution (IP to hostname). The resolved value replaces the original value.

The processor utilizes two LRU (Least Recently Used) caches with TTL (Time-to-Live): one for storing successful resolutions (hit cache) and another for storing failed lookups.

The processor can resolve hostnames/IPs using the system's default resolver, a custom hostfile, or a custom DNS server.

Example configuration for the component

procoessors:
  dnslookup:
    # A list of attributes to be resolved from hostname to IP
    resolve_attributes: []
    # A list of attributes to be reverse-looked up from IP to hostname
    reverse_attributes: []
    # Maximum number of failed resolutions to cache. Default value is `1000`. Set to `0` to disable caching
    failed_cache_size: 1000 
    # Time-to-live (in seconds) for failed resolution cache entries. Default value is `5` seconds
    failed_cache_ttl: 5
    # Maximum number of successful resolutions to cache. Default value is `1000`. Set to `0` to disable caching 
    hit_cache_size: 1000 
    # Time-to-live (in seconds) for successful resolution cache entries. Default value is `60` seconds
    hit_cache_ttl: 60 
    # Maximum number of retry attempts for DNS lookups. Default value is `2`
    max_retries: 2 
    # Timeout (in seconds) for DNS lookups. Default value is `0.5`
    timeout: 0.5
    # Path to a custom hostfile
    hostfile: "/path/to/hosts"
    # Address of a custom DNS server
    nameserver: 8.8.8.8
    # Enable system's default resolver. Default value is `false`
    enable_system_resolver: false 
    # Defines how the processor handles errors. Valid values are:
    # - `ignore`: Processor logs errors and continues to the next processor (default).
    # - `silent`: Processor ignores errors without logging and continues to the next processor.
    # - `propagate`: Processor returns errors, resulting in the payload being dropped from the collector.
    error_mode`: ignore

Resolution Order

The resolution order depends on the specified configuration and the enable_system_resolver setting:
With enable_system_resolver set to true:

  1. When both hostfile and nameserver are specified:
    Custom hostfile -> Custom DNS server -> System's default resolver

  2. When only nameserver is specified:
    Custom DNS server -> System's default resolver

  3. When only hostfile is specified:
    Custom hostfile -> System's default resolver

  4. When neither hostfile nor nameserver are specified:
    System's default resolver

With enable_system_resolver set to false, this affects all scenarios above by removing System's default resolver from the resolution order.
The processor will fail to start and throw an error If neither hostfile nor nameserver are specified and enable_system_resolver is set to false.

Considerations

  • Performance impact of DNS resolutions: If each DNS lookup takes 2 milliseconds, the maximum throughput a single processor can achieve is 500 events per second.

Telemetry data types supported

Logs, metrics, traces.

Is this a vendor-specific component?

  • This is a vendor-specific component
  • If this is a vendor-specific component, I am a member of the OpenTelemetry organization.
  • If this is a vendor-specific component, I am proposing to contribute and support it as a representative of the vendor.

Code Owner(s)

No response

Sponsor (optional)

No response

Additional context

No response

@kaisecheng kaisecheng added needs triage New item requiring triage Sponsor Needed New component seeking sponsor labels Aug 2, 2024
@kaisecheng kaisecheng changed the title New component: DNS resolve processor New component: DNS lookup processor Aug 2, 2024
@strawgate
Copy link
Contributor

Could we have this support multiple nameservers right away, or is the intention to add that in a later revision? Any thoughts on TCP transport?

@kaisecheng
Copy link
Contributor Author

Could we have this support multiple nameservers

Yes, it makes sense to support multiple.

Any thoughts on TCP transport?

I think it is a nice to have feature and could be in a later version

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Sponsor Needed New component seeking sponsor
Projects
None yet
Development

No branches or pull requests

3 participants