Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FLOOR may silently give incorrect results #24

Open
ajcz opened this issue Mar 11, 2018 · 9 comments
Open

FLOOR may silently give incorrect results #24

ajcz opened this issue Mar 11, 2018 · 9 comments

Comments

@ajcz
Copy link

ajcz commented Mar 11, 2018

FLOOR seems to give incorrect results for larger numbers. In my environment, FLOOR(1.0D10) is −2147483648 on NT386 and FLOOR(1.0D20) is −9223372036854775808 on AMD64_LINUX.

@mikanystrom
Copy link
Contributor

mikanystrom commented Mar 11, 2018 via email

@ajcz
Copy link
Author

ajcz commented Mar 11, 2018

Gee, I didn’t even realize FLOOR returns INTEGER. (I’ll change the issue title.)

Actually I was trying to calculate the fractional part of Time.T values, which turned out to be too large. The comment in Math.i3 suggests the reader to use the built-in rounding functions instead; I guess that could be a bit misleading.

Getting checked runtime errors does seem a lot better than getting incorrect results. (Or could something go wrong?)

@ajcz ajcz changed the title FLOOR sometimes gives incorrect results FLOOR may silently give incorrect results Mar 11, 2018
@mikanystrom
Copy link
Contributor

mikanystrom commented Mar 11, 2018 via email

@ajcz
Copy link
Author

ajcz commented Mar 11, 2018

There’s no problem with the Date interface. I just need a Date.T with a finer resolution, hence the need for the fractional part of Time.T.

@mikanystrom
Copy link
Contributor

mikanystrom commented Mar 11, 2018 via email

@ajcz
Copy link
Author

ajcz commented Mar 12, 2018

How far into the past or future do you need to go?

Not too far from the present thankfully. I think the Posix implementation already uses 64-bit times internally where possible. On Windows Time uses WinBase.FILETIME, which has a resolution of 0.1 ms and is good for about 30,000 years around January 1, 1601.

If you need to convert later dates or with a different epoch you can do a stepwise thing. Divide the time by a nice round number, say 2048, do the floor, subtract it off, multiply by 2048...

Do you mean that if a Time.T is too large for Date.FromTime I can calculate suitable a remainder first, convert it to a Date.T, and then adjust that Date.T accordingly? But then how do I know when a Time.T is too large? Date.FromTime doesn’t raise any exceptions.

@mikanystrom
Copy link
Contributor

mikanystrom commented Mar 12, 2018 via email

@ajcz
Copy link
Author

ajcz commented Mar 12, 2018

Umm... I didn’t have any problem with Date.FromTime.

Since all the epochs that I know about start from an integral second […]

That’s my assumption as well as well as what is said in the Time interface’s comment. I didn’t get the correct fractional part of VAR now := Time.Now() because now is too large for INTEGER and I was calculating now - FLOAT(FLOOR(now), Time.T) instead of now - Math.floor(now) or maybe just Math.fmod(now, 1.0D0).

@mikanystrom
Copy link
Contributor

mikanystrom commented Mar 12, 2018 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants