[Question] Current place in code to change floating point resolution

[DeannaGelbart]

I'd like to reduce the number of floating point digits produced by dump().

Where in json.hpp do I do that?

This old issue #768 says to replace std::numeric_limits<number_float_t>::digits10 at line 6701 but digits10 is no longer at that line. I tried replacing this symbol in dump_float() but it didn't do the trick and when I tried replacing every occurrence of this symbol in json.hpp I got assertion failures.

Thanks!

[DeannaGelbart]

I think I found a workaround, if I switch from float to double and then do this based on https://stackoverflow.com/a/11209187 before calling dump()

double ToTwoDecimalPlaces(double d) {

  int i;

  if (d >= 0)

    i = static_cast<int>(d * 100 + 0.5);

  else

    i = static_cast<int>(d * 100 - 0.5);


  return (i / 100.0);

}

[RokerHRO]

doubleliterals don't have any suffix in C nor C++. Is that suffix "d" an extension of your compiler?
https://en.cppreference.com/w/cpp/language/floating_literal

[DeannaGelbart]

I've removed the d suffix from the floating point literals in the code snippet since as you point out, it's meaningless.

[DeannaGelbart]

Unfortunately, while this workaround has been working for me, it's not working reliably for my colleague. Presumably due to some numbers not being representable exactly enough as a double.

[DeannaGelbart]

It turns out my colleague just made a typo.

Yesterday I tried about 50,000 floating point numbers with my code and they all were formatted with the correct number of decimal places

However these numbers are all in the range from -60 to 60 so I don't know for sure that my code works reliably for every number.

[DeannaGelbart]

It's possible that once numbers get large enough that there is considerably less floating precision left to represent the part after the decimal point, my code starts failing. This could be tested with a loop that covers the representation range of the double type.