Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated doc for decimal support #4750

Merged
merged 6 commits into from
Feb 13, 2022
Merged

Updated doc for decimal support #4750

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
88ecc84
Updated doc for decimal
viadea Feb 11, 2022
ebb6d3d
Update docs/supported_ops.md
viadea Feb 11, 2022
f186e94
Update docs/supported_ops.md
viadea Feb 11, 2022
7b9e0e5
Update docs/supported_ops.md
viadea Feb 11, 2022
6355d7a
Modify TypeChecks.scala as well
viadea Feb 11, 2022
63c5999
fix 100 char limit
viadea Feb 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 6 additions & 8 deletions docs/supported_ops.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,12 @@ apply to other versions of Spark, but there may be slight changes.

# General limitations
## `Decimal`
The `Decimal` type in Spark supports a precision
up to 38 digits (128-bits). The RAPIDS Accelerator in most cases stores values up to
64-bits and will support 128-bit in the future. As such the accelerator currently only
supports a precision up to 18 digits. Note that
decimals are disabled by default in the plugin, because it is supported by a relatively
small number of operations presently. This can result in a lot of data movement to and
from the GPU, slowing down processing in some cases.
Result `Decimal` precision and scale follow the same rule as CPU mode in Apache Spark:
The `Decimal` type in Spark supports a precision up to 38 digits (128-bits).
The RAPIDS Accelerator supports 128-bit starting from version 21.12 and decimals are
enabled by default.
Please check [Decimal Support](compatibility.md#decimal-support) for more details.

`Decimal` precision and scale follow the same rule as CPU mode in Apache Spark:

```
* In particular, if we have expressions e1 and e2 with precision/scale p1/s1 and p2/s2
Expand Down
14 changes: 6 additions & 8 deletions sql-plugin/src/main/scala/com/nvidia/spark/rapids/TypeChecks.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1706,14 +1706,12 @@ object SupportedOpsDocs {
println()
println("# General limitations")
println("## `Decimal`")
println("The `Decimal` type in Spark supports a precision")
println("up to 38 digits (128-bits). The RAPIDS Accelerator in most cases stores values up to")
println("64-bits and will support 128-bit in the future. As such the accelerator currently only")
println(s"supports a precision up to ${DType.DECIMAL64_MAX_PRECISION} digits. Note that")
println("decimals are disabled by default in the plugin, because it is supported by a relatively")
println("small number of operations presently. This can result in a lot of data movement to and")
println("from the GPU, slowing down processing in some cases.")
println("Result `Decimal` precision and scale follow the same rule as CPU mode in Apache Spark:")
println("The `Decimal` type in Spark supports a precision up to 38 digits (128-bits). ")
println("The RAPIDS Accelerator supports 128-bit starting from version 21.12 and decimals are ")
println("enabled by default.")
println("Please check [Decimal Support](compatibility.md#decimal-support) for more details.")
println()
println("`Decimal` precision and scale follow the same rule as CPU mode in Apache Spark:")
println()
println("```")
println(" * In particular, if we have expressions e1 and e2 with precision/scale p1/s1 and p2/s2")
Expand Down