os/signal: add NotifyContext function

[henvic]

As previously discussed on issues such as #21521 and #16472, or on several other places handling POSIX signals through context seems to be somewhat useful (and a little bit hard to do correctly?), and I would like to propose a way to handle it by adding a new signal.WithContext function.

There's also an idea to improve the current approach to handling signals (see #21521 (comment)). I didn't give it a try yet, unfortunately. I only found the earlier proposals here after trying to write some code, so I decided to create this new issue anyway to share my idea.

People using some sort of it or talking about the subject:

• https://github.com/henvic/ctxsignal (package I wrote a while ago)
• https://twitter.com/dmitshur/status/1227777318162030592 (thread)
• https://twitter.com/matryer/status/869096368039710720 (thread)
• Make Ctrl+C cancel the context.Context (blog post)
• oklog/run@9c53bcd
• https://github.com/FiloSottile/mostly-harmless/blob/22c6a9e08ad95b602b470652ad2d401d8750e264/covfefe/covfefe.go#L118-L139
• https://github.com/shurcooL/home/blob/d0f5a32e4901b05ae9979b1a9e7b67caba549778/main.go#L61-L73
• https://github.com/search?l=Go&p=1&q=context+signal&type=Repositories lists 18 other packages for the same thing.

[gopherbot]

Change https://golang.org/cl/219640 mentions this issue: os/signal: add WithContext to control single usage signals easily.

[henvic]

Also please notice that this change would, unfortunately, require introducing packages time and context as dependencies for package os/signal.

[ianlancetaylor]

Could you describe the new function in this issue, so that we don't have to figure it out from the CL? Thanks.

[henvic]

The new function receives a parent context and a variadic number of signals that can be used to cancel this context. When a parent context is canceled or when one of the signals is handled, the context is canceled.

Example of handling cancelation of a slow process through SIGTERM and SIGINT:

func main() {
	ctx, cancel := signal.WithContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
	defer cancel()

	// ...
	slow.Run(ctx)
	cancel()

	// to verify if the slow function was terminated by a signal, we can use:
	var sigErr signal.ContextError
	if errors.As(ctx.Err(), &sigErr) {
		fmt.Printf("slow function terminated by signal %q\n")
	}
}

On another side, we have the example shown for https://golang.org/pkg/net/http/#Server.Shutdown that, in my opinion, would not be simplified by this.

[robpike]

Perhaps this should be a function of the context package instead, since it creates a context and it's pretty much all about contexts.

ctx, cancel := context.Signal(context.Background(), syscall.SIGTERM, syscall.SIGINT)

Not sure it's the right idea at all, just throwing it out there.

[mattn]

If several packages handle same signal, and runtime does not handle those contexts equally, context.Signal won't work correctly, I think. If runtime does not handle them, context.Signals is not best way . The os/signal package has the potential feeling for us that it is only used in applications, but the context package is used by everyone. So I'm thinking signal.WithContext is better since it make users known that we should do it in only one place. (But I'm also not sure this is right idea)

[rsc]

To be clear, the runtime does already broadcast signals to anyone who has signed up to hear about them, using signal.Notify. So this could be done as a separate package without worrying about other possible uses of os/signal.

context.WithCancelSignal seems like the clearest name, and it would appear right next to context.WithCancel in the docs.

/cc @Sajmani for thoughts

[rsc]

It sounds like maybe we have converged on context.WithCancelSignal. I've retitled.
Based on the discussion above, this seems like a likely accept.

[henvic]

Great! Thanks for the feedback. I'll try to send a new CL proposing an implementation for a context.WithCancelSignal command Monday.

[undeconstructed]

Is there a plan for how this would be documented? Signals are process scoped, so any library using this feature could be problematic. And in a server I would normally catch signals at main level and propagate down, which can be easily/lazily done already by cancelling a single parent context. So using signals for local task cancellation seems only suited for interactive apps. If that's not too controversial, could it be made clear in the docs?

[henvic]

I pushed a new implementation to https://go-review.googlesource.com/c/go/+/219640/1

[ianlancetaylor]

I note that currently according to go/build/deps_test.go context depends only on

{"errors", "internal/reflectlite", "sync", "sync/atomic", "time"}

This is going to add a dependency of context on os and os/signal, which pulls in a lot more. I don't know that it matters much, but it seems worth mentioning. Is anybody concerned about the additional dependencies?

[robpike]

I was worried too, but if you look at the full lists it doesn't add much transitively. In the current tree:

% deps io os os/signal | sort -u
errors
internal/bytealg
internal/cpu
internal/oserror
internal/poll
internal/race
internal/reflectlite
internal/syscall/execenv
internal/syscall/unix
internal/testlog
io
os
runtime
runtime/internal/atomic
runtime/internal/math
runtime/internal/sys
sync
sync/atomic
syscall
time
unsafe
% deps context
errors
internal/bytealg
internal/cpu
internal/oserror
internal/race
internal/reflectlite
runtime
runtime/internal/atomic
runtime/internal/math
runtime/internal/sys
sync
sync/atomic
syscall
time
unsafe
%

The only addition would be io and os themselves, and three internals:internal/syscall/{unix,execenv} and internal/testlog.

I'm not saying I'm happy about this, but it's arguably OK.

[mattn]

If receiving the signal in goroutine and context.WithCancelSignal, I wonder how this work.

package main

import (
	"context"
	"os"
	"os/signal"
	"time"
)

func doSomething(ctx context.Context) {
	// ...
}

func main() {
	s := make(chan os.Signal, 1)
	signal.Notify(s, os.Interrupt)

	ctx := context.WithCancelSignal(s)
	doSomething(ctx)

	go func() {
		<-s
		println("interrupted")
	}()

	var b [1]byte
	os.Stdin.Read(b[:])
	time.Sleep(time.Second)
	println("exit")
}

[ianlancetaylor]

One possibility to avoid adding a dependency on os and os/signal is to have the context package reach into the runtime package as the os/signal package already does. One consideration about depending on the os/signal package is that the os/signal package has an init function that slightly changes how the runtime package handles signals. But we might be able to get rid of that anyhow.

[ianlancetaylor]

@mattn The proposed context.WithCancelSignal function takes a signal as an argument, not a channel.

But assuming you meant to pass os.Interrupt to context.WithCancelSignal, then on receipt of a signal both 1) the context would be canceled; 2) the signal would be sent on the channel s.

[mattn]

Oh, I see.

[henvic]

I'm worried about the number of changes in the signal package too. I'll take a look into the runtime package Friday to see if I understand @ianlancetaylor's idea and to see if I can come up with something.

[rsc]

FWIW, I think we can easily hook this into the runtime package directly (with an internal package if needed) and bypass os and os/signal, to keep the deps lite. The general trend is to avoid os where possible, and all this code only needs the runtime.

No change in consensus here, so accepted.

[ianlancetaylor]

I'm not necessarily opposed to context.SignalError, but I think that if you want to make decisions based on which signal was received then you shouldn't be using signal.NotifyContext. You should be using context.WithCancel and signal.Notify, and canceling the context when receiving a value from the signal.Notify channel. We already have a mechanism for handling signals; the point of this change is to provide a convenient way to cancel a context on receipt of a signal. If you have to do different things based on different signals, we already have mechanisms for that.

[ianlancetaylor]

Ouch, I hadn't thought about changes in signal handling behavior. I agree that the only possible choice here is that the signal handling behavior only changes when or if the cancel function is explicitly called. Otherwise the wrong thing would happen if the program received two SIGHUP signals in a row.

[pierrre]

Personally I will probably not use this new function.
I'm in favor of not adding this to the stdlib.
It's very simple to write your own implementation in your app and the need may be different in each app.

I've already built a package that I'm using at my company, and I don't think it's compatible with your proposal.
Here what it is doing:

• the first time it receives a signal, it cancels the context (without unregistering the signal watcher)
• the second time it receives the signal, it exits the app
• the caller must explicitly unregister the signal watcher

Here is the code if someone is interested:

// Package ctxsignal provides a bridge for context and POSIX signals.
package ctxsignal

import (
	"context"
	"log"
	"os"
	"os/signal"
	"sync"
	"syscall"

	"github.com/xxx/yyy/goroutine"
)

// RegisterCancel registers a listener for the SIGINT/SIGTERM signals.
// The first time that the signal is received, the context is canceled.
// The second time, the program exits with status 0.
func RegisterCancel(parent context.Context) (ctx context.Context, unregister func()) {
	ctx, cancel := context.WithCancel(parent)
	c := make(chan os.Signal, 1)
	signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
	waitWatchSignal := goroutine.Go(func() {
		watchSignal(c, cancel)
	})
	unregister = newUnregisterFunc(c, waitWatchSignal)
	return ctx, unregister
}

func watchSignal(c <-chan os.Signal, cancel func()) {
	sig, ok := <-c
	if !ok {
		return
	}
	log.Printf("%q signal received, context canceled. Send it again to exit the program immediately.", sig)
	cancel()
	_, ok = <-c
	if !ok {
		return
	}
	log.Println("Signal received twice, exit now.")
	os.Exit(0)
}

func newUnregisterFunc(c chan os.Signal, waitWatchSignal func()) func() {
	var once sync.Once
	return func() {
		once.Do(func() {
			signal.Stop(c)
			close(c)
		})
		waitWatchSignal()
	}
}

(waitWatchSignal is a function that blocks until the goroutine is finished).

PS: I use it at the very beginning of my applications, in order to support graceful stop.

[henvic]

Not an expert in signal handling by any means, but...

Why would it be bad to change the signal handling behavior when the parent context is canceled? Wouldn't also be similarly bad if it doesn't (especially if there is a way to check what signal caused the context to end)? It's not clear to me what an example would be. Instead, shouldn't NotifyContext be used first when this happens?

Another thing to worry: if you want to use this for something like handling configuration reload through SIGHUP, it might be fine if you lose a signal (if you replace the context immediately to keep this going on - I'm not sure if this would be good btw). However, if you're listening to, say, SIGTERM, if a second SIGTERM arrives before you called signal.Notify again, your process might terminate before it should.

[bcmills]

The idea is to make it possible to identify what signal canceled it.

There is already a pretty strong precedent that ctx.Err() is exactly either context.Canceled or context.DeadlineExceeded. Rather than deviating from that precedent, it might be less intrusive to encode the signal in the Value method and provide a top-level accessor for it:

package signal

// ContextSignal reports which (if any) signal caused ctx to be canceled.
//
// If ctx was not canceled due to a signal, ContextSignal returns (nil, false).
func ContextSignal(ctx context.Context) (os.Signal, bool) {
	sig, ok := ctx.Value(signalContextKey{}).(os.Signal)
	return sig, ok
}

[henvic]

@bcmills, I like it.

[narqo]

@bcmills, If somewhere within a call stack, I had an operation, that accepted the context and reacted to its cancellation, I don't think I would check something related to the signal in this operation, but rather returned ctx.Err(). After this error bubbled up to someone who "cares" about the signals, it would

1. check if the error wasn't nil
2. check if the context was cancelled
3. check if there is a signal in the context's value

Is that how ContextSignal expected to be used?

[rsc]

To fix the double-signal problem, it sounds like we have to deviate from the usual context.WithCancel to require calling the cancellation function if you want to stop the captured signal behavior. That is, the docs would change to:

package signal
func NotifyContext(parent context.Context, signals ...os.Signal) (ctx context.Context, stop context.CancelFunc)

NotifyContext returns a copy of the parent context that is marked done
(it's Done channel is closed) when one of the listed signals arrives,
when the returned stop function is called, or when the parent context's
Done channel is closed, whichever happens first.

The stop function unregisters the signal behavior, which, like signal.Reset,
may restore the default behavior for a given signal. For example, the default
behavior of a Go program receiving os.Interrupt is to exit. Calling
NotifyContext(parent, os.Interrupt) will change the behavior to cancel
the returned context. Future interrupts received will not trigger the default
(exit) behavior until the returned stop function is called.

The stop function releases resources associated with it, so code should
call stop as soon as the operations running in this Context complete and
signals no longer need to be diverted to the context.

---

It remains unclear that we really need the signal details. For that, it's easy enough to write the simple code that handles the signal, records it, and cancels a context. That is, NotifyContext is meant to be more a convenience than a complete Swiss army knife. We can always add it later if we need to, but it probably makes sense to start without it.

Thoughts?

[henvic]

It makes sense to me. I liked the idea of calling the cancel function stop to differentiate things.

[rsc]

Based on the discussion above, it seems like we've reconverged on signal.NotifyContext, documented in #37255 (comment).

This seems like a likely accept (take two).

[rsc]

No change in consensus (yet?) so accepted (again).

[danp]

@henvic Were you planning to adapt your CL (or a new one) based on the change to os/signal.NotifyContext? I'd be happy to help however I can, would be great to get this in for 1.16.

[henvic]

@danp, thank you for the help!

Hello @ianlancetaylor, might you please take a second look at the updated CL? I have sent a new patchset. Thank you!

https://go-review.googlesource.com/c/go/+/219640