Tips - Workaround incorrect DNS resolutions on MacOS

[tisonkun]

I'd like to share my experience of workaround incorrect DNS resolutions on MacOS for Pulsar standalone.

When you run Pulsar standalone by bin/pulsar standalone on MacOS, you should always see the following warnings:

2022-08-11T20:24:10,581+0800 [main] WARN  io.netty.resolver.dns.DnsServerAddressStreamProviders - Can not find io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider in the classpath, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS.

This is because we don't bundle the related netty resolver for DNS on MacOS. It may or may not cause an issue. I meet one issue today:

2022-08-11T14:32:22,296+0800 [main] WARN  io.netty.resolver.dns.DnsServerAddressStreamProviders - Can not find io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider in the classpath, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS.
2022-08-11T14:32:24,381+0800 [main] ERROR org.apache.pulsar.functions.worker.PulsarWorkerService - Error Starting up in worker
java.lang.ExceptionInInitializerError: null
	at io.netty.resolver.dns.UnixResolverDnsServerAddressStreamProvider.parseSilently(UnixResolverDnsServerAddressStreamProvider.java:80) ~[netty-resolver-dns-4.1.77.Final.jar:4.1.77.Final]

This error causes the standalone process to fail and exit.

To workaround the issue, download the necessary libs and add them to the classpath.

1. Download netty-resolver-dns-classes-macos-4.1.77.Final.jar and netty-resolver-dns-native-macos-4.1.77.Final-osx-aarch_64.jar (version and arch can be varied on your env).
2. Place these two libs under a chosen folder, said /path/to/my/libs.
3. Run Pulsar standalone with env var PULSAR_EXTRA_CLASSPATH which allows adding extra libs:

PULSAR_EXTRA_CLASSPATH=/path/to/my/libs/* ./bin/pulsar standalone

The issue should be resolved atm :D

[tisonkun]

BTW, this can be a troubleshooting doc for running Pulsar on MacOS.

For the document part, cc @Anonymitaet @momo-jun
Do we have a place to attach such notes?

For the technical part, cc @lhotari @codelipenghui
Is it possible we bundle these libs on condition? Although I can regard it as a platform-specific issue and should be resolved by related users.

[Anonymitaet]

How about adding a troubleshooting or QA chapter to https://pulsar.apache.org/docs/next/getting-started-standalone at the end?

The Install JDK on M1 chapter can also be in that chapter since we want to navigate users to the core steps of getting Pulsar started in the quickest way? Thoughts?

[michaeljmarshall]

@tisonkun - thanks for your comment about DNS resolution. I've also found that an issue can come from how the /etc/hosts file is configured for your machine's hostname. I documented my findings here and solution: #15401 (comment).

[Somnium99]

@tisonkun
Hi! It's happening for me when running on IntelliJ + Ventura MacOS.
Do you happen to know to fix it using intellij ?
I tried adding this depdendency :
runtimeOnly("io.netty:netty-resolver-dns-native-macos:4.1.85.Final:osx-x86_64")
But to no avail

[os-architect]

same here. did you have any luck getting this to work?

[michaeljmarshall]

What error are you seeing?

[ShubhamMaddhashiya-bidgely]

Hi @Somnium99 were you able to resolve the issue, I'm also getting similar error while running pulsar client.

Jun 20, 2023 2:12:30 PM org.apache.pulsar.shade.io.netty.resolver.dns.DnsServerAddressStreamProviders <clinit>
WARNING: Can not find org.apache.pulsar.shade.io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider in the classpath, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS.

My OS: Ventura MacOS (intel)
IDE: IntelliJ

POM file:

 <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <pulsar.version>2.10.3</pulsar.version>
  </properties>


  <dependencies>
    <dependency>
      <groupId>org.apache.pulsar</groupId>
      <artifactId>pulsar-client</artifactId>
      <version>${pulsar.version}</version>
    </dependency>
    <dependency>
      <groupId>io.netty</groupId>
      <artifactId>netty-resolver-dns-native-macos</artifactId>
      <scope>runtime</scope>
      <classifier>osx-x86_64</classifier>
      <version>4.1.72.Final</version>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

Main method:

import org.apache.pulsar.client.api.Producer;
import org.apache.pulsar.client.api.PulsarClient;
import org.apache.pulsar.client.api.PulsarClientException;
import org.apache.pulsar.client.api.Schema;

/**
 * Hello world!
 *
 */
public class App {
    public static void main( String[] args )  {
        try {
            PulsarClient client = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
            Producer<String> stringProducer = client.newProducer(Schema.STRING)
                    .topic("my-topic")
                    .create();
            stringProducer.send("My message");
            stringProducer.close();
        }
        catch (PulsarClientException e) {
            e.printStackTrace();
        }
    }
}