Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extending Kotlin API for BigInteger and BigDecimal #49

Closed
ilya-g opened this issue Sep 16, 2016 · 63 comments
Closed

Extending Kotlin API for BigInteger and BigDecimal #49

ilya-g opened this issue Sep 16, 2016 · 63 comments
Assignees
Labels

Comments

@ilya-g
Copy link
Member

ilya-g commented Sep 16, 2016

Discussions about the Extending Kotlin API for BigInteger and BigDecimal proposal will be held here.

@ilya-g
Copy link
Member Author

ilya-g commented Sep 16, 2016

We can head to implementing a prototype/tests now. I think it could be in a form of a pull request to https://github.com/jetbrains/kotlin

@ilya-g ilya-g added the stdlib label Sep 16, 2016
@voddan
Copy link
Contributor

voddan commented Oct 8, 2016

What is the decision on the BigInteger#mod problem?

@ilya-g
Copy link
Member Author

ilya-g commented Nov 7, 2016

We consider % operator for BigInteger calculating mod instead of remainder a bug, and are going to fix it. However the fix isn't straightforward and requires additional discussion. One of related issues is KT-14650

@ilya-g
Copy link
Member Author

ilya-g commented Nov 25, 2016

mod operator convention will be deprecated, and instead we'll introduce binary operator rem, which then can be provided as extension for BigInteger

@voddan
Copy link
Contributor

voddan commented Nov 25, 2016

WAT? You mean in the whole language mod will be replaced by rem? Why?

I would thought that since BigInteger % was used extremely rare, it would be possible to brake binary compatibility here. Is there any other reason why the whole mod is being deprecated?

@elizarov
Copy link
Contributor

elizarov commented Nov 25, 2016

mod is not going to be removed anytime soon. It is going to be derecated, but will continue to work as before. We are not breaking compatibility here.

The reason is very simple. mod is, unfortunately, a wrong and confusing name. The corresponding operation is called remainder in all modern languages, including Java (see "15.17.3. Remainder Operator %") and JVM bytecode ("irem"). It is also called this way in math, while mod is a different mathematical operation.

@cypressious
Copy link
Contributor

cypressious commented Nov 25, 2016

Are you going to change the operator rules so that % resolves to operator fun rem? If so, that's breaking source compatibility.

@elizarov
Copy link
Contributor

There cannot be operator fun rem in existing Kotlin code. It will not compile.

@cypressious
Copy link
Contributor

cypressious commented Nov 25, 2016

I'm talking about code like

BigInteger("2") % BigInteger("2")

that currently resolves to java.math.BigInteger#mod and my question is whether it will still resolve to the same method after the change.

@elizarov
Copy link
Contributor

That's a bug that we'll fix by this change. BigInteger("2") % BigInteger("2") will resolve to java.math.BigInteger#remainder.

@cypressious
Copy link
Contributor

cypressious commented Nov 25, 2016

You may be right in that it's a bug, but it's still breaking source compatibility. I just wanted to make that clear. I'm neither for nor against it.

@elizarov
Copy link
Contributor

elizarov commented Nov 25, 2016

That's a bug in stdlib, actually. '%' was accidentally bound to the wrong function in BigInteger. Our users have found it and had filed a bug. It had prompted us to reconsider the language operator, too, but it is still a separate library bug. We are trying not maintain bugs in our libraries, unless there are very good reasons to.

@elizarov
Copy link
Contributor

Yes, it is breaking change, albeit minor, so it will not go into "1.0.x".

@voddan
Copy link
Contributor

voddan commented Nov 25, 2016

@elizarov Actually, in Math it is the modulo operator. That's why Pascal called it mod, and LaTeX use the \mod notation.

IMHO you should leave operator mod as is.

@elizarov
Copy link
Contributor

You are right, that in Pascal -1 mod 3 = -1. I don't know why it was called this way in Pascal in defiance of established mathematical notation, because in Math -1 mod 3 == 2. That is what LaTeX math \mod stands for, too.

Java and all other modern languages don't copy Pascal, but follow the Math. They call '%' a remainder operation, because -1 % 3 == -1. Java BigInteger also follows well-established mathematical tradition. You can check it with the following code:

    println(BigInteger("-1").mod(BigInteger("3")) == BigInteger("2")) // true
    println(BigInteger("-1").remainder(BigInteger("3")) == BigInteger("-1")) // true

@voddan
Copy link
Contributor

voddan commented Nov 25, 2016

I see your point.

Then, it we want the name of the operator to originate from "reminder", could that be a more clear naming than rem? "REM" as a word is very non-intuitive.

See that in Java. They call it remainder, not rem. Since there is no pressure in Kotlin (unlike in ASM) to have short operator names, can we name it operator remainder?

@elizarov
Copy link
Contributor

elizarov commented Nov 25, 2016

There is a Kotlin tradition to give short names to mathematical operators. Operator for / is called div, not division, so % shall be called rem, not remainder.

@voddan
Copy link
Contributor

voddan commented Nov 25, 2016

On the other hand, we have minus, not sub. I think the right balance here is to use a well recognised name. mod was such a name, but it had a different meaning. IMO there is nothing "to long" in remainder, since it is only used in declarations.

I see no harm in using reminder, while rem will confuse people with a background similar to mine.

@elizarov
Copy link
Contributor

elizarov commented Nov 25, 2016

Can you, please, clarify what's confusing with remfor you? Does it hold some negative connotations or bears similarity to some unrelated concept?

@voddan
Copy link
Contributor

voddan commented Nov 25, 2016

It is that at first rem did not associated in my mind with anything. I have worked with Pascal, C/C++, ASM, Python, Java. In this mix nothing has rem as a term.

I have worked with Haskell some time ago, but I totally forgot it had mod and rem.

So "confusing" in this context is when one looks at an unfamiliar Kotlin code (operator fun rem() and has to go to the language docs to understand it.

@drothmaler
Copy link

drothmaler commented Nov 25, 2016

The only thing, that rem reminds me of, is REM in windows batch files (where it stands for REMARK [comments]). My next thought would be, that it might be the short version of remove.
So I think, it would not be very intuitive to use it for remainder.
On the other hand: how often do you need to declare a remainder operator function? So typing 6 more letters really shouldn't hurt much.

@elizarov
Copy link
Contributor

elizarov commented Nov 25, 2016

Let's do a roundup:

@voddan
Copy link
Contributor

voddan commented Nov 25, 2016

Options suggested so far:

  • mod (incorrect)
  • rem - ala Haskell
  • remainder - human readable, ala Java

@elizarov
Copy link
Contributor

elizarov commented Nov 25, 2016

Naming things is famously the most complex problem in computer science. On one hand, there is indeed a similarity of rem to REMARK, while on the other hand a shortened division to div establishes a tradition to follow the same shortening for remainder to rem.

P.S. Calling a remainder operator mod is the most confusing and weird option. Moreover, it utterly harms interop with BigInteger class that gives quite correct names for its methods.

@elizarov
Copy link
Contributor

I've just noticed that BASIC uses REM for single-line comment, while MATLAB uses % for single-line comment. Coincidence? I don't think so :)

@voddan
Copy link
Contributor

voddan commented Mar 29, 2017

I've updated the proposal to reflect on the changes introduced in Kotlin 1.1
Please merge the PR: #65

@ilya-g
Copy link
Member Author

ilya-g commented May 4, 2017

I mean regular overloads, similar to String.toInt(radix) from KEEP #19

ilya-g added a commit that referenced this issue Jun 15, 2017
@ezioamf
Copy link

ezioamf commented Oct 19, 2017

I would love to see support for BigDecimal constants on Kotlin. You could use a dollar ($) prefix for the BigDecimal constants.

This way you can write:

var balance = $1_000.00 // var balance = BigDecimal("1000.00")
balance = balance - $100.00 //balance = balance - BigDecimal("100.00")

@elizarov
Copy link
Contributor

elizarov commented Oct 19, 2017

You can define the following extension val:

val Double.bd: BigDecimal get() = toBigDecimal()

and then write:

var balance = 1_000.00.bd
balance = balance - 100.00.bd

Seems to be much clearer.

@ezioamf
Copy link

ezioamf commented Oct 19, 2017

@elizarov Double cannot accurately represent base 10 so a BigDecimal constructor using double is unpredictable. You could try to do the same thing using a String instead of Double but "1000.00".bd is not that clear.

@elizarov
Copy link
Contributor

@ezioamf I'm not suggesting to use BigDecimal constructor using double. I'm suggesting to use Kotlin's toBigDecimal() extension function that is available in Kotlin 1.2 and which internally uses a string constructor. In Kotlin 1.1 you can define it like this:

val Double.bd: BigDecimal get() = BigDecimal(toString())

It is very predictable, in the sense that println(0.1.bd) prints precisely "0.1". You can try it here.

@ezioamf
Copy link

ezioamf commented Oct 19, 2017 via email

@elizarov
Copy link
Contributor

@ezioamf Double.toString() does convert the double to string. You can read the specification of this conversion here. The most important part of that spec if here:

How many digits must be printed for the fractional part of m or a? There must be at least one digit to represent the fractional part, and beyond that as many, but only as many, more digits as are needed to uniquely distinguish the argument value from adjacent values of type double.

That is why 0.1 gets converted to string "0.1", but not to "0.1000000000000000055511151231257827021181583404541015625" (which is the actual decimal value of the 0.1 double literal).

@ezioamf
Copy link

ezioamf commented Oct 19, 2017 via email

@elizarov
Copy link
Contributor

elizarov commented Oct 19, 2017

I don't need to run this code. I can count digits to see that the literal 9_999_999_999.000001 is on the edge of the precision of double. Kotlin IDE should actually highlight it to avoid mistakes (non related to big-decimals at all, will bite you in any kind of numeric code). Good catch. We really need such kind of an inspection: KT-20837

If you are into writing such long decimals, you can also declare a string extension for them:

val String.bd: BigDecimal get() = BigDecimal(this.replace("_", ""))

Not as neat, but the you can do "9_999_999_999.000001".bd to get a correct BigDecimal value. Combined with KT-20837 for safe usage of short literals that should be quite usable.

@ezioamf
Copy link

ezioamf commented Oct 19, 2017 via email

@elizarov
Copy link
Contributor

Support for custom literals in Kotlin is in the future far-far away. If you look at the road that C++ took to support custom literals, you'll see that the support for constexpr is the prerequisite for it. And Kotlin is still far from even having that: KT-14652. There are more pressing problems at Kotlin's plate at the moment.

@ezioamf
Copy link

ezioamf commented Oct 19, 2017 via email

@elizarov
Copy link
Contributor

Kotlin does not have a built-in BigDecimal type. Moreover, BigDecimal is currently not even a part of Kotlin stdlib (it is available only on JVM as java.math.BigDecimal). While it might become a part of Kotlin stdlib in the future, it is still too weak of a connection to support "big decimal literals" in compiler. It has to built-in to have literals. But big decimals seem to be too niche of a feature (for a general-purpose language like Kotlin) to support them as a built-in type.

@ezioamf
Copy link

ezioamf commented Oct 19, 2017 via email

@elizarov
Copy link
Contributor

elizarov commented Oct 19, 2017

It is niche. I've been doing all three in various proportions for 15+ years and I never had to use BigDecimal. I was only using it for esoteric (not from real life) sports programming problems. All enterprise finance I was doing had all its monetary amounts nicely fitting into double (never needed more than 15 digits of precision) and all the science I was doing was using IEEE floating-point as its bread and butter.

@ezioamf
Copy link

ezioamf commented Oct 19, 2017 via email

@elizarov
Copy link
Contributor

elizarov commented Oct 19, 2017

@ezioamf I will use it again as long as I don't need more than 15 decimal digits.

Unfortunately, few universities teach future programmers on how to use floating point properly (how it works, how to avoid loss of precision, how to keep decimals decimal, etc), so most people use BigDecimal whether they actually need it or not. It is a dream land for consultants. You come to a customer who complains of poor performance in their "scientific" code and without having to figure out what the code actually does you can usually speed it up multiple-fold by getting rid of big decimals ;)

The fact is that you only need BigDecimal if you need more than 15 decimal digits of precision. It is quite, quite rare in practice.

@ezioamf
Copy link

ezioamf commented Oct 19, 2017 via email

@elizarov
Copy link
Contributor

elizarov commented Oct 19, 2017

That SO thread is quite old. I've been stumbling upon it from time to time and there is nothing I can add or correct there.

It is funny, that the answer faithfully describes how IEEE floating point works, but gives no clue on how it was, is, will, and should be used to work with monetary amounts up to 15 decimal digits. Many people will be surprised at the number and scope of financial and scientific systems around the world, that they likely interact daily with, and that work perfectly and meet all their business requirements using only double arithmetics.

Let me quote that SO question here:

I've always been told never to represent money with double or float types

I am just coming from a totally different approach to education, education for engineers, where future engineers are taught the basic of how the tools at their disposal work in different situations, what are the tradeoff between different tools, so that in the future they know how to pick the best tool for the job at hand given the constraints and requirements they have.

Let me repeat. There is nothing wrong with representing money with double type as long as you are sure that you don't need more that 15 significant digits. Actually, the credit here goes not to IEEE committee, but the Java/JVM authors, who had specified floating point operations in such a painstaking details, to make it actually reliable and usable. You have to be careful when you are doing arithmetics on those numbers to avoid the infamous 0.1 + 0.2 != 0.3 problem, but having the need to be careful is not the reason to avoid it altogether.

@ezioamf
Copy link

ezioamf commented Oct 19, 2017 via email

@elizarov
Copy link
Contributor

It is just a tradeoff like everything else in programming. You trade simple-to-reason semantics of BigDecimal versus performance of double. Which one to choose for your financial application? The right answer is -- it depends. It don't like it, when people give some other answer.

@voddan
Copy link
Contributor

voddan commented Oct 19, 2017

If I may interrupt this discussion, I would like to note that the initial @ezioamf's proposal was about BigDecimal literals, not long arithmetics in general. And they are not the same.

This KEEP issue was about convenient long arithmetics calculations, and the current implementation is apparently enough for all major use cases we considered.

For now stdlib contains a full operators set for BigDecimal, but does not contain any special way to initialize constants (a.k.a literals). This allows you to write complete formulas with BigDecimal variables, assuming that the actual values are derived from the input programatically. According to several DM/ML/banking experts we have consulted during the implementation of the proposal (@elizarov being one of them ), this covers the absolute majority of BigDecimal uses except for (a) tests and (b) small example programs, both of with were considered low priority.

For details please refer to the text of the proposal or to its earlier drafts, some of which contained more functionality than the current one.

@ezioamf
Copy link

ezioamf commented Oct 19, 2017 via email

@voddan
Copy link
Contributor

voddan commented Oct 19, 2017

@ezioamf although your previous comment is correct, it is irrelevant in a discussion about a language since it is up to a programmer to choose the second sample or modify the first to a working condition. As a language, Kotlin provides you an easy way to transition from double to BigDecimal and back, shell you deem it necessary. For contrast, Java lacks this level of support, and you would have to change not only the initialization of the values, but also the algorithm.

@ezioamf
Copy link

ezioamf commented Oct 19, 2017

@voddan I don't see this as irrelevant.

Languages have been created and improved to help developers to create clean, concise, elegant, and correct code.

Most languages have arbitrary precision types as secondary class citizens then you see buggy implementations using float, double, long, two longs... Even very experienced developers can make this kind of mistakes.

Java with a good implementation of BigDecimal make it a little better but using BigDecimal to do Math creates very verbose code. See more about it here:

http://www.gavaghan.org/blog/2007/11/06/c-decimal-and-java-bigdecimal-solve-roundoff-problems/

Kotlin by having operator overload led to a clean code when dealing with BigDecimal. I just wanted something simple that is BigDecimal literals so I could have even cleaner code. I could write the same code as:

fun main(args: Array) {
var amount = $0.1
var balance = $0.3

balance -= amount
balance -= amount

if(balance < amount) print ("not enough balance")
}

Better than just literals Kotlin show have a native decimal type backed by java.lang.BigDecimal. You may say that there is no correspondent type on CPU but this is not always the case. Generic languages such C and C++ gave support for floating point types before mathematical coprocessors or floating point instructions and for 64bit longs on 32bit CPUs to help developers.

Microsoft created the decimal type on C# for the same reason. Decimal on C# do not have the problems of Double by been implemented on a base 10 floating.

https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/decimal

So you can write code in C# as:

decimal amount = 0.1m;
decimal balance = 0.3m;

if(amount < balance) ...

A good and concise decimal implementation would further help the adoption of Kotlin by financial and enterprise software.

@elizarov
Copy link
Contributor

elizarov commented Oct 23, 2017

@voddan Thanks for bringing the discussion back on-topic. Let's move a discussion on BigDecimal literals into a separate venue.

For a start, here is an issue on bringing BigDecimal into Kotlin proper (into stdlib), which should precede any discussion on BigDecimal literals: KT-20912. I've also created an issue for BigDecimal literals themselves: KT-20913, so that we don't mix the too.

I also wrote a short post on using floating-point for decimal arithmetics, so that we can have a separate place to discuss that, too: https://medium.com/@elizarov/floating-point-for-decimals-fc2861898455

@ezioamf
Copy link

ezioamf commented Oct 23, 2017

I will continue the discussion on the medium post. See my comment on it.

@dzharkov
Copy link
Contributor

The declarations have been released in Kotlin 1.2, thus I'm closing the KEEP. Do not hesitate to open a YouTrack issue for any additional suggestions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

8 participants