net: add UnwrapErr field and Unwrap method to *DNSError

[mateusz834]

Edit: current proposal: #63116 (comment)

---

This is a proposal to fix the #63109 issue.

I propose adding a causeErr field to DNSError and an Unwrap method that returns the causeErr.

// DNSError represents a DNS lookup error.
type DNSError struct {
    causeErr error // returned by the Unwrap method.
    // (....)     
}

func (e *DNSError) Unwrap() error { return e.causeErr }

This way we keep backwards compatibility for the Error method, we always construct the error from
the Err string field (we keep the current implementation). In the net pacakge we will then make Err field to be the result of the first call to causeErr.Error().

[bcmills]

• Should we do the same for AddrError? (It also has an Err string field.)
• All fields of DNSError and AddrError are currently exported, so it is possible for packages outside of net to construct one of these errors directly. Should the new field be exported too?

[mateusz834]

Should we do the same for AddrError? (It also has an Err string field.)

AFAIK the AddrError seems to be a standalone error (i.e. it doesn't seem to wrap anything).

All fields of DNSError and AddrError are currently exported, so it is possible for packages outside of net to construct one of these errors directly. Should the new field be exported too?

I don't know whether we want to allow this (we would have to enforce somehow that DNSError.causeErr.Error() == DNSError.Err). Maybe we should leave this for future consideration? We can export it afterwards.

There is still an issue (with the current proposal), that someone might change the Err field without updating the causeErr. (with a *DNSError returned from Lookup functions).

I am unsure about this (in either way).

[ianlancetaylor]

Since all the existing fields of DNSError are exported, I think the new one should also be exported. I suggested Underlying.

[ianlancetaylor]

Here is a test for #63109 that I wrote last night.

func TestCancelBeforeDial(t *testing.T) {
	ln := newLocalListener(t, "tcp")
	defer ln.Close()
	_, port, err := SplitHostPort(ln.Addr().String())
	if err != nil {
		t.Fatal(err)
	}

	ctx, cancel := context.WithCancel(context.Background())
	cancel()
	var d Dialer
	_, err = d.DialContext(ctx, "tcp", JoinHostPort("localhost", port))
	if err == nil {
		t.Error("DialContext with canceled context did not fail")
	} else if !errors.Is(err, context.Canceled) {
		t.Errorf("error.Is(%v, context.Canceled) failed", err)
	}
}

[mateusz834]

Fine, let's make the error field exported.

The proposal looks like this now:

// DNSError represents a DNS lookup error.
type DNSError struct {
	// Underlying is an error that caused this DNSError.
	// It is returned by the Unwrap method.
	//
	// When updating this field make sure to update
	// the Err field accordingly, so that the Error
	// and Unwrap methods don't diverge.
	Underlying error

	// Err is a description of the error, for backwards compatibility
	// the net package sets this error to the value
	// returned by Underlying.Error(), it used by
	// the Error method to produce the error string.
	Err string

	// (...)
}

// Unwrap returns e.Underlying.
func (e *DNSError) Unwrap() error { return e.Underlying }

[mjl-]

Having an exported error may also come in handy with possible future support for "extended dns errors", see https://datatracker.ietf.org/doc/html/rfc8914. Could be good to take that into account. It is useful to learn about specifics of dns failures, especially with dnssec errors, and for getting a more descriptive message than "servfail".

[mateusz834]

@mjl- Great idea! I made #63192 for this. This should allow for detailed error messages from the pure go resolver. Not sure how the exported error helps with this. I assume you mean an exported (new) error type just for extended dns errors, I don't think we will ever do this it would be impossible to support this on windows and on unix cgo resolver.

[mjl-]

I assume you mean an exported (new) error type just for extended dns errors, I don't think we will ever do this it would be impossible to support this on windows and on unix cgo resolver.

Yes, a new error type for extended dns errors is what I had in mind. I would think windows and unix c would at some point make these extended dns errors accessible, perhaps with a new API. They'll want it sooner or later. We could make these detailed errors optional. DNS servers (recursive resolvers) don't have to support EDE, and not all do, so there is already no guarantee you'll get them. Anyway, if underlying errors can be unwrapped, and I can set those errors, I could make a new package that can return a net.DNSError with specific EDE errors of my own package. It would make it easier to provide a drop-in replacement resolver package.

[mateusz834]

I implemented this change in CL 532217, it also simplifies the code a bit with an internal constructor for the *DNSError, so that we don't have to set the Err, IsNotFound, IsTimeout, IsTemporary fields manually each time we construct the error, these fields will be now determined based on the Underlying error type.

Edit: this also implies that the net package will always construct *DNSError with Underlying != nil even for unexported internal errors from the net package, this way we don't overcomplicate this change.

[gopherbot]

Change https://go.dev/cl/532217 mentions this issue: net: add Unwrap to *DNSError

[mateusz834]

@rsc Any chance of getting this through proposal process? This is a bug-fix that needs a proposal. It would be nice to fix this for go 1.23. Thanks

[rsc]

I thought when we added Unwrap we explicitly looked at all types with an Err field and decided which to do and not do. For net.AddrError, we explicitly decided against it.

For net.DNSError, there is no Err field currently, so we could add one. Except there is an 'Err string' field.

There are lots of times when errors should not be wrapped. I don't see a discussion here of which errors to expose with Unwrap and why that is correct.

[mateusz834]

There are lots of times when errors should not be wrapped. I don't see a discussion here of which errors to expose with Unwrap and why that is correct.

What kind of errors shouldn't be wrapped? I think that we should wrap all errors and just for backwards-compatibility set the Err field to Underlying.Error(), as in https://go.dev/cl/532217. I can think of one case that we don't strictly speaking have to wrap errNoSuchHost, because we have the IsNotFound field, but considering that errNoSuchHost is not exported i don't see a reason not to wrap it.

[neild]

Surveying existing *DNSError errors, the Err field is:

• A constant string, such as "unrecognized address". In this case there is no error to wrap.
• The string value of an unexported error value, such as errNoSuchHost. Wrapping the unexported error value seems harmless.
• The string value of a context error (cancelation or timeout). Wrapping seems like a strict improvement.
• The string value of an error returned by a syscall. This is the the most ambiguous case, but I think wrapping is somewhere between useful and harmless here.

So I support adding DNSError.Underlying and wrapping everything.

(It's been a while, but I believe the only reason we didn't add an Unwrap method to DNSError back when we added Unwrap is because it has an Err string field, and we were limiting ourselves to adding Unwrap methods, not adding new error fields.)

[rsc]

I disagree about at least the syscall return. I think wrapping is harmful, because it exposes the underlying implementation in ways that are easier to depend on. Most syscall errors are also highly non-specific. I can see wanting Unwrap to work for the context error. I think that if we proceed, we should default to Underlying==nil and Err != "" and only set Underlying for the specific cases that we want Unwrap to work for (like context).

I'm also not convinced about Underlying; perhaps it should be UnwrapErr.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[mateusz834]

I'm also not convinced about Underlying; perhaps it should be UnwrapErr.

Underlying sounds better to me

[rsc]

Underlying is a concept in go/types; it is never used for errors in API names.
UnwrapErr is clunky but clear: it is the error that Unwrap returns, which may or may not be nil.

[rsc]

Have all remaining concerns about this proposal been addressed?

The proposal is to add UnwrapErr error to DNSError, and an Unwrap method that returns UnwrapErr. By default, DNSErrors will have UnwrapErr set to nil, but we can set it explicitly for specific errors. To start, the context errors seem to be the only ones. If there are others that programs need to distinguish, please let us know.

[rsc]

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

The proposal is to add UnwrapErr error to DNSError, and an Unwrap method that returns UnwrapErr. By default, DNSErrors will have UnwrapErr set to nil, but we can set it explicitly for specific errors. To start, the context errors seem to be the only ones. If there are others that programs need to distinguish, please let us know.

[mateusz834]

FYI, currently the net package does not return directly errors from the context package, but it maps them with mapErr.

go/src/net/cgo_unix.go

Line 81 in c9ed561

Err: mapErr(ctx.Err()).Error(),

go/src/net/net.go

Lines 427 to 438 in c9ed561

	// mapErr maps from the context errors to the historical internal net
	// error values.
	func mapErr(err error) error {
	switch err {
	case context.Canceled:
	return errCanceled
	case context.DeadlineExceeded:
	return errTimeout
	default:
	return err
	}
	}

Do we want to keep it this way?

[ianlancetaylor]

We should keep it that for now. Changing it seems likely to break existing code (which is we do the mapping in the first place). If somebody wants to investigate changing this behavior, that can be a separate proposal.

[mateusz834]

How do we want to handle errors from Resolver.Dial and from the returned net.Conn?

Should we map the context errors, with mapErr? I think it is harmful, because we will not be able to map wrapped errors.
Should we only return errors that match: errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded), without mapping?

go/src/net/dnsclient_unix.go

Lines 176 to 179 in 68d3a9e

	c, err := r.dial(ctx, network, server)
	if err != nil {
	return dnsmessage.Parser{}, dnsmessage.Header{}, err
	}

We might also do something like:

if errors.Is(err, context.Canceled) {
    return &DnsError{
        Err: err.Error(),
        UnwrapErr: errCanceled, // same error as returned by mapErr
    }
}

I think that if we do not expose all errors, then the third approach is the best.

[ianlancetaylor]

Sorry, I'm not sure exactly what you are asking.

If we returned a mapped error today, we should continue to do that.

Note that the mapped errors already return true for things like errors.Is(err, context.Canceled) or errors.Is(err, context.DeadlineExceeded.

[mateusz834]

We currently do not map errors returned by Resolver.Dial:

go/src/net/dnsclient_unix.go

Lines 176 to 179 in 68d3a9e

	c, err := r.dial(ctx, network, server)
	if err != nil {
	return dnsmessage.Parser{}, dnsmessage.Header{}, err
	}

How we should handle errors returned from the Dial function?

Do we want to return the error as is.

c, err := r.dial(ctx, network, server) 
if err != nil { 
    return &DnsError{
        Err: err.Error(),
        UnwrapErr: err,
    } 
} 

Return the error, but only when it matches context.Canceled or context.DeadlineExceeded

c, err := r.dial(ctx, network, server) 
if err != nil { 
    if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
        return &DnsError{
            Err: err.Error(),
            UnwrapErr: err,
        }  
    }
    return &DnsError{
        Err: err.Error(),
    }  
} 

Or return only errCanceled, errTimeout.

c, err := r.dial(ctx, network, server) 
if err != nil { 
    if errors.Is(err, context.Canceled)  {
        return &DnsError{
            Err: err.Error(),
            UnwrapErr: errCanceled,
        }  
    }
    if errors.Is(err, context.DeadlineExceeded)  {
        return &DnsError{
            Err: err.Error(),
            UnwrapErr: errTimeout,
        }  
    }
    return &DnsError{
        Err: err.Error(),
    }  
} 

[ianlancetaylor]

Looking at Resolver.dial, it already calls mapErr. The code that you mention is in Resolver.exchange; the only caller of Resolver.exchange already wraps the error up in a DNSError. So I think the only question is what Resolver.tryOneName should put in the new UnwrapErr field of the returned DNSError. I think it should set UnwrapErr to the error returned by Resolver.exchange. Perhaps I am missing something.

[mateusz834]

Looking at Resolver.dial, it already calls mapErr.

Oh, right I somehow missed that, but the mapping is obviously not great, it does not go through Unwrap errors, so it might lead to some confusing cases:

r := net.Resolver{
	Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
		return nil, context.Canceled
	},
}
_, err := r.LookupMX(context.Background(), "test")
fmt.Println(err) // (*DnsError).UnwrapErr = errCancaled

r := net.Resolver{
	Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
	    return nil, &net.OpError{Err: context.Canceled} // wrapped error
	},
}
_, err := r.LookupMX(context.Background(), "test")
fmt.Println(err) // (*DnsError).UnwrapErr = &net.OpError{Err: context.Canceled}

I think it should set UnwrapErr to the error returned by Resolver.exchange. Perhaps I am missing something.

The current status of this proposal is to only expose context errors, so we should not expose all errors from Resolver.Dial. I think that we should only return the errCanceled, errTimeout when: errors.Is(err, context.Canceled) or errors.Is(err, context.DeadlineExceeded).

So:

go/src/net/dnsclient_unix.go

Lines 305 to 311 in 68d3a9e

	p, h, err := r.exchange(ctx, server, q, cfg.timeout, cfg.useTCP, cfg.trustAD)
	if err != nil {
	dnsErr := &DNSError{
	Err: err.Error(),
	Name: name,
	Server: server,
	}

becomes:

p, h, err := r.exchange(ctx, server, q, cfg.timeout, cfg.useTCP, cfg.trustAD)
if err != nil {
     dnsErr := &DNSError{
	Err:    err.Error(),
	Name:   name,
	Server: server,
    }
    if errors.Is(err, context.Canceled) {
        dnsErr.UnwrapErr = errCalceled
    }
    if errors.Is(err, context.DeadlineExceeded) {
        dnsErr.UnwrapErr = errTimeout
    }
    // (...)
}

This way we do not expose implementation errors and allow the *DnsError to match context.DeadlineExceeded and context.Canceled through errors.Is.

[ianlancetaylor]

That is OK with me. I guess the only question is how we should handle an error returned by a system call: should we wrap it or not? Since we haven't wrapped it in the past, I'm fine with continuing to not wrap it, at least until we have a clear reason to do so. Thanks.

[rsc]

It sounds like the only use of Unwrap will be for Is, in which case we could decide that instead of adding an Unwrap method we could implement an Is method on DNSError. Is there a reason to prefer one or the other? Probably Unwrap is more general for dealing with future issues?

[mateusz834]

Just as you mentioned, Unwrap is better for errors that we might want to unwrap in future, but at the current state of the proposal there is no difference between adding a Is or Unwrap method. But i guess that for Is we would also have to add a UnwrapErr field, so in that case Unwrap feels better to me.

[neild]

Unwrap is simpler and clearer. If you've got a choice between Unwrap and Is, I'd always recommend picking Unwrap.

Is is more general, you can do more with it, but it's more mechanism. I don't see a reason to use it here.

[rsc]

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

The proposal is to add UnwrapErr error to DNSError, and an Unwrap method that returns UnwrapErr. By default, DNSErrors will have UnwrapErr set to nil, but we can set it explicitly for specific errors. To start, the context errors seem to be the only ones. If there are others that programs need to distinguish, please let us know.