From b1a74f1c49e9783ce1e56ac33fa172a9edf2ab62 Mon Sep 17 00:00:00 2001 From: Raza Jafri Date: Wed, 15 Jun 2022 16:58:35 -0700 Subject: [PATCH] updated docs (#5840) Signed-off-by: Raza Jafri Co-authored-by: Raza Jafri --- docs/configs.md | 2 +- .../src/main/scala/com/nvidia/spark/rapids/RapidsConf.scala | 3 +-- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/configs.md b/docs/configs.md index e3a743970ad..971b788b999 100644 --- a/docs/configs.md +++ b/docs/configs.md @@ -100,7 +100,7 @@ Name | Description | Default Value spark.rapids.sql.hasExtendedYearValues|Spark 3.2.0+ extended parsing of years in dates and timestamps to support the full range of possible values. Prior to this it was limited to a positive 4 digit year. The Accelerator does not support the extended range yet. This config indicates if your data includes this extended range or not, or if you don't care about getting the correct values on values with the extended range.|true spark.rapids.sql.hasNans|Config to indicate if your data has NaN's. Cudf doesn't currently support NaN's properly so you can get corrupt data if you have NaN's in your data and it runs on the GPU.|true spark.rapids.sql.hashOptimizeSort.enabled|Whether sorts should be inserted after some hashed operations to improve output ordering. This can improve output file sizes when saving to columnar formats.|false -spark.rapids.sql.improvedFloatOps.enabled|For some floating point operations spark uses one way to compute the value and the underlying cudf implementation can use an improved algorithm. In some cases this can result in cudf producing an answer when spark overflows. Because this is not as compatible with spark, we have it disabled by default.|true +spark.rapids.sql.improvedFloatOps.enabled|For some floating point operations spark uses one way to compute the value and the underlying cudf implementation can use an improved algorithm. In some cases this can result in cudf producing an answer when spark overflows.|true spark.rapids.sql.improvedTimeOps.enabled|When set to true, some operators will avoid overflowing by converting epoch days directly to seconds without first converting to microseconds|false spark.rapids.sql.incompatibleDateFormats.enabled|When parsing strings as dates and timestamps in functions like unix_timestamp, some formats are fully supported on the GPU and some are unsupported and will fall back to the CPU. Some formats behave differently on the GPU than the CPU. Spark on the CPU interprets date formats with unsupported trailing characters as nulls, while Spark on the GPU will parse the date with invalid trailing characters. More detail can be found at [parsing strings as dates or timestamps](compatibility.md#parsing-strings-as-dates-or-timestamps).|false spark.rapids.sql.incompatibleOps.enabled|For operations that work, but are not 100% compatible with the Spark equivalent set if they should be enabled by default or disabled by default.|true diff --git a/sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsConf.scala b/sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsConf.scala index e26cd60a820..be400a32d22 100644 --- a/sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsConf.scala +++ b/sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsConf.scala @@ -575,8 +575,7 @@ object RapidsConf { val IMPROVED_FLOAT_OPS = conf("spark.rapids.sql.improvedFloatOps.enabled") .doc("For some floating point operations spark uses one way to compute the value " + "and the underlying cudf implementation can use an improved algorithm. " + - "In some cases this can result in cudf producing an answer when spark overflows. " + - "Because this is not as compatible with spark, we have it disabled by default.") + "In some cases this can result in cudf producing an answer when spark overflows.") .booleanConf .createWithDefault(true)