Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
yaml_test: adapt around undefined behavior in float64->int64 casting
In TestJSONObjectToYAMLObject the "uint64 big" case accepts a float64 of size 2^63. The value is passed to jsonToYAMLValue() and the float64 branch of the switch is entered. For values that do not fit int64 the first cast to int64() is undefined behavior in most languages and apparently in Golang: http://c0x.coding-guidelines.com/6.3.1.4.pdf This means the value of int64(float64(2^63)) can end up as either -9223372036854775808 or 9223372036854775807. From experimentation, it appears compiler optimization determines the result. The value is then casted back to float64 and matched against the original float64, which may or may not pass. Depending of FPU rounding mode for IEEE754-Doubles any input above 9223372036854775296 may get casted to 9223372036854775808.0. Adapt the unit test for "uint64 big" to not feed a big float64 and enter the problem cases. Instead pass the nearest rounded value of 2^63 as a uint64 and expect to receive the same result: bigUint64 = ((uint64(1) << 63) + 500) / 1000 * 1000
- Loading branch information