archive/tar: rounding timestamps to the nearest second may result in archives from the future

[milahu]

when a tar archive, generated by archive/tar/writer.go, is extracted immediately,
tar shows warnings that some timestamps are in the future

downstream issue: ipfs/kubo#8406
downstream workaround: ipfs/go-ipfs-files#41

probably here, Floor should be used

go/src/archive/tar/writer.go

Lines 83 to 92 in b86e8dd

	// Round ModTime and ignore AccessTime and ChangeTime unless
	// the format is explicitly chosen.
	// This ensures nominal usage of WriteHeader (without specifying the format)
	// does not always result in the PAX format being chosen, which
	// causes a 1KiB increase to every header.
	if tw.hdr.Format == FormatUnknown {
	tw.hdr.ModTime = tw.hdr.ModTime.Round(time.Second)
	tw.hdr.AccessTime = time.Time{}
	tw.hdr.ChangeTime = time.Time{}
	}

[dsnet]

I don't think there's a right answer here. The current logic is entirely reasonable as it optimizes for accuracy at the cost of occasionally being in the future for 500ms. The suggested logic is 500ms less accurate, but avoids being in the future. Both are reasonable and I don't think it's a compelling reason to change, since this will probably break many use-cases that have come to depend on archive/tar for deterministic outputs (see Hyrum's Law).

Have you considered using FormatPAX? That supports high-precision timestamps and may entirely avoid this problem. Also, whatever is flagging archives from the future should probably tolerate an error on the same order as the smallest representable time unit (1s in this case).

[milahu]

this will probably break many use-cases that have come to depend on archive/tar for deterministic outputs (see Hyrum's Law).

yepp, api consumers hate surprises

its just unexpected behavior, cos usually float-to-int casts mean rounding down

Have you considered using FormatPAX? That supports high-precision timestamps and may entirely avoid this problem.

workaround in ipfs: pass integer time to archive/tar

 func writeFileHeader(w *tar.Writer, fpath string, size uint64) error {
 	return w.WriteHeader(&tar.Header{
 		Name:     fpath,
 		Size:     int64(size),
 		Typeflag: tar.TypeReg,
 		Mode:     0644,
-		ModTime:  time.Now(),
+		ModTime:  time.Now().Truncate(time.Second),
 	})
 }

whatever is flagging archives from the future should probably tolerate an error on the same order as the smallest representable time unit (1s in this case).

yes. in this case nixos unpackFile → _defaultUnpack → tar xf "$fn"
this should be tar xf "$fn" --warning=no-timestamp

thanks! : )

closing

[dsnet]

That seems like an entirely reasonable and easy workaround.