Sniffing

A client uses a sniffing process to find all nodes of your cluster by default, automatically. The official clients for e.g. Ruby do the same. If you don't need this, you can disable it with the option elastic.SetSniff(false).

The sniffing process starts in the background when you create a new client with elastic.NewClient(...). It periodically invokes the Cluster Nodes Info API on all provided URLs. If you did not specify a URL to connect to in NewClient(...), the default URL of http://127.0.0.1:9200 is used.

Here's what the Cluster Nodes Info API of Elasticsearch returns when successful (newer versions of Elasticsearch look slightly different):

$ curl -XGET 'http://127.0.0.1:9200/_nodes/http?pretty=1'
{
  "cluster_name" : "elasticsearch",
  "nodes" : {
    "msEWB6vOQ6qVzC2y8fTOhw" : {
      "name" : "Blackheath",
      "transport_address" : "inet[/127.0.0.1:9300]",
      "host" : "aero",
      "ip" : "192.168.10.10",
      "version" : "1.4.4",
      "build" : "c88f77f",
      "http_address" : "inet[/127.0.0.1:9200]",
      "http" : {
        "bound_address" : "inet[/127.0.0.1:9200]",
        "publish_address" : "inet[/127.0.0.1:9200]"
      }
    }
  }
}

You see that it returns a map with the node ID as its key, and the HTTP (or HTTPS) address to connect to the given node. Elastic will now happily use the results of this call to update its internal list of nodes/connections.

This sniffing process happens periodically (every 15 minutes by default). This ensures that Elastic always has an up-to-date list of nodes in your cluster. When a node in your cluster goes down, or you add a new node to your cluster, Elastic happily picks it up and uses it. You can control the sniffing interval with SetSnifferInterval(...) (see client configuration settings.

When sniffing the cluster fails as you create the new client, an error is returned: Elastic might have a problem to find any suitable node in your cluster. In subsequent sniffing processes, the sniffer might you fail silently (it logs failures of course).

If you don't want sniffing, you can disable it when creating a client:

client, err := elastic.NewClient(elastic.SetSniff(false))

Sniffing callback

You can use the elastic.SetSniffCallback(...) option to get a hook into the sniffing process. The callback will be invoked whenever a new node will be found. You, as the implementor, can then decide whether the newly found node will get requests from Elastic. One use case of this feature is to only send requests to powerful nodes. Here's an example of using a sniffing callback (see also elastic.NodesInfoNode:

callback := func(node *elastic.NodesInfoNode) bool {
    // You can e.g. check node.Attributes to decide if the node will get requests
    if _, found := node.Attributes["ignore-in-elastic"]; found {
        return false
    }
    return true // will get requests
}
_, err := elastic.NewClient(elastic.SetSnifferCallback(callback))
if err == nil {
    panic(err)
}

Notice that the sniffing process has a companion health check process.