what about implement a buffer writer syncer?

[sysulq]

Support sync through a buffer writer, zap should be faster...

Just like golang/glog does.
https://github.com/golang/glog/blob/master/glog.go#L841

Glad to send a PR if you guys are interested :-)

Simple Demo:

type bufferWriterSync struct {
	buf *bufio.Writer
}

func Buffer(ws WriteSyncer, size int) WriteSyncer {
        bw := &bufferWriterSync{
		buf: bufio.NewWriterSize(ws, size),
	}

        ws = zapcore.Lock(bw)
        return ws
}

// Sync syncs data to file
func (w bufferWriterSync) Sync() error {
	return w.buf.Flush()
}

// Write writes data to buffer
func (w bufferWriterSync) Write(p []byte) (int, error) {
	return w.buf.Write(p)
}

[akshayjshah]

We added Sync to the zapcore.WriteSyncer API for exactly this use case, but haven't actually shipped a buffering sink yet. Any buffer we ship will need to flush periodically, though; otherwise low-output loggers may retain logs indefinitely without actually writing them to disk.

[sysulq]

Yes, and this can be done by a timer loop goroutine.

Like this:

	bw := &bufferWriterSync{
		buf: bufio.NewWriterSize(ws, size),
	}

	// bufio is not goroutine safe, so add lock writer here
	ws = zapcore.Lock(bw)

	// flush buffer every 5s
	go func() {
		for range time.NewTicker(5 * time.Second).C {
			ws.Sync()
		}
	}()

	return ws

[akshayjshah]

Exactly! I'd prefer to automatically start the goroutine in whatever constructor we expose (rather than making the user do it), and we'll also need to expose the ability to cleanly shut down the goroutine to avoid resource leaks.

[pavelnemirovsky]

@akshayjshah when u plan to release buffer sink capability ? Would be glad to test it. Thx for great project.

[BGraco]

I am also very interested in this, mostly to reduce ops/s as that drives cost on cloud storage instances.

[zplzpl]

anyone currently doing this?

[sysulq]

@zplzpl
Maybe I could send a pull request for this feature, unfortunately I just get stuck in Wuhan...
which means I got a lot of time for coding...

[zplzpl]

@hnlq715 bro , take care yoursellf,bless for you

[michelaquino]

Hi, any update about that?

[sysulq]

FYI~

This issue has been fixed by #782 and #961 :-)

[YevSent]

Is there any documentation on how to use BufferedWriteSyncer? The source code comments don't contain many details and from the tests, it's also difficult to understand how exactly BufferedWriteCyncer should be initialized.

[abhinav]

Is there any documentation on how to use BufferedWriteSyncer? The source code comments don't contain many details and from the tests, it's also difficult to understand how exactly BufferedWriteCyncer should be initialized.

Oops, you're right, there's significant room for improvement here.
I've tried to add more documentation for BufferedWriteSyncer in #1139.
Hope this helps!