Skip to content

Commit

Permalink
yaml_test: adapt around undefined behavior in float64->int64 casting
Browse files Browse the repository at this point in the history
In TestJSONObjectToYAMLObject the "uint64 big" case accepts a float64
of size 2^63. The value is passed to jsonToYAMLValue() and the
float64 branch of the switch is entered. For values that do not
fit int64 the first cast to int64() is undefined behavior
in most languages and apparently in Golang:
http://c0x.coding-guidelines.com/6.3.1.4.pdf

This means the value of int64(float64(2^63)) can end up as either
-9223372036854775808 or 9223372036854775807.
From experimentation, it appears compiler optimization
determines the result.

The value is then casted back to float64 and matched against
the original float64, which may or may not pass. Depending of FPU
rounding mode for IEEE754-Doubles any input above 9223372036854775296
may get casted to 9223372036854775808.0.

Adapt the unit test for "uint64 big" to not feed a big float64 and enter
the problem cases. Instead pass the nearest rounded value of 2^63
as a uint64 and expect to receive the same result:
  bigUint64 = ((uint64(1) << 63) + 500) / 1000 * 1000
  • Loading branch information
neolit123 committed Nov 12, 2020
1 parent 8aabd9a commit 7e97a1a
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions yaml_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -428,6 +428,7 @@ foo: baz
}

func TestJSONObjectToYAMLObject(t *testing.T) {
const bigUint64 = ((uint64(1) << 63) + 500) / 1000 * 1000
intOrInt64 := func(i64 int64) interface{} {
if i := int(i64); i64 == int64(i) {
return i
Expand Down Expand Up @@ -459,7 +460,7 @@ func TestJSONObjectToYAMLObject(t *testing.T) {
"map": map[string]interface{}{"foo": "bar"},
"slice": []interface{}{"foo", "bar"},
"string": string("foo"),
"uint64 big": float64(math.Pow(2, 63)),
"uint64 big": bigUint64,
},
expected: yaml.MapSlice{
{Key: "nil slice"},
Expand All @@ -476,7 +477,7 @@ func TestJSONObjectToYAMLObject(t *testing.T) {
{Key: "map", Value: yaml.MapSlice{{Key: "foo", Value: "bar"}}},
{Key: "slice", Value: []interface{}{"foo", "bar"}},
{Key: "string", Value: string("foo")},
{Key: "uint64 big", Value: uint64(1) << 63},
{Key: "uint64 big", Value: bigUint64},
},
},
}
Expand Down

0 comments on commit 7e97a1a

Please sign in to comment.