Skip to content
This repository has been archived by the owner on Mar 1, 2024. It is now read-only.

Unexpected output #5

Closed
bkkrueger opened this issue Apr 6, 2015 · 1 comment · Fixed by #6
Closed

Unexpected output #5

bkkrueger opened this issue Apr 6, 2015 · 1 comment · Fixed by #6

Comments

@bkkrueger
Copy link

I'm trying Julia out (and have been using a lot of Python lately), so I like the idea of having Python-like formatting in Julia. But I found a but in your Formatting.jl package. Try

using Formatting
fe = FormatExpr("{1:8.2e}\n")
x::Float64 = 0.0
for i::Integer in 1:20
    x += 0.05
    printfmt(fe, x)
end

You should see
1.00e-01
2.00e-01
3.00e-01
4.00e-01
5.00e-01
6.00e-01
7.00e-01
8.00e-01
9.00e-01
1.00e-00

Instead the output is
1.00e-01
2.00e-01
3.00e-01
4.00e-01
5.00e-01
5.100e-01
6.100e-01
7.100e-01
8.100e-01
9.100e-01

I'm not sure if the error has to do with rounding or something else entirely. But I thought I'd let you know something was up.

@odow
Copy link
Contributor

odow commented Apr 28, 2015

I thought I was going crazy and couldn't workout how 1 + 0.999 = 1.1000

Looks like a bug when rounding the last digit

fmt(".2f", 0.989) -> "0.99"
fmt(".2f",0.994) -> "0.99"
fmt(".2f",0.995) -> "0.100"
fmt(".2f",0.996) -> "0.100"
fmt(".2f",0.997) -> "0.100"
fmt(".2f",0.998) -> "0.100"
fmt(".2f",0.999) -> "0.100"

fmt(".2f", 0.9989) -> "0.999"
fmt(".3f", 0.9994) -> 0.999"
fmt(".3f", 0.9995) -> 0.1000"

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants