Skip to content

Commit

Permalink
update download and databricks doc for 21.06.2 (#3214)
Browse files Browse the repository at this point in the history
Signed-off-by: Peixin Li <pxli@nyu.edu>
  • Loading branch information
pxLi authored Aug 12, 2021
1 parent fe4346c commit 20f4020
Show file tree
Hide file tree
Showing 2 changed files with 44 additions and 2 deletions.
44 changes: 43 additions & 1 deletion docs/download.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,48 @@ New functionality and performance improvements for this release include:
For a detailed list of changes, please refer to the
[CHANGELOG](https://github.com/NVIDIA/spark-rapids/blob/main/CHANGELOG.md).

## Release v21.06.2
This is a patch release to address an issue with the plugin in the Databricks 8.2 ML runtime.

Hardware Requirements:

GPU Architecture: NVIDIA V100, T4 or A10/A30/A100 GPUs

Software Requirements:

OS: Ubuntu 18.04, Ubuntu 20.04 or CentOS 7, CentOS 8

CUDA & Nvidia Drivers*: 11.0 or 11.2 & v450.80.02+

Apache Spark 3.0.1, 3.0.2, 3.1.1, 3.1.2, Cloudera CDP 7.1.7, Databricks 7.3 ML LTS or 8.2 ML Runtime, and GCP Dataproc 2.0

Apache Hadoop 2.10+ or 3.1.1+ (3.1.1 for nvidia-docker version 2)

Python 3.6+, Scala 2.12, Java 8

*Some hardware may have a minimum driver version greater than v450.80.02+. Check the GPU spec sheet
for your hardware's minimum driver version.

### Download v21.06.2
* Download the [RAPIDS
Accelerator for Apache Spark 21.06.2 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/21.06.2/rapids-4-spark_2.12-21.06.2.jar)
* Download the [RAPIDS cuDF 21.06.1 jar](https://repo1.maven.org/maven2/ai/rapids/cudf/21.06.1/cudf-21.06.1-cuda11.jar)

This package is built against CUDA 11.2 and has [CUDA forward
compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/index.html) enabled. It is tested
on Tesla datacenter GPUs with CUDA 11.0 and 11.2. For those using other types of GPUs which
do not have CUDA forward compatibility (for example, GeForce), CUDA 11.2 is required. Users will
need to ensure the minimum driver (450.80.02) and CUDA toolkit are installed on each Spark node.

### Release Notes
This release patches the plugin to address a backwards incompatible change to Parquet filters made
by Databricks in the Databricks 8.2 ML runtime. More information is in [issue
3191](https://github.com/NVIDIA/spark-rapids/issues/3191) in the RAPIDS Spark repository. See the
[Release v21.06.0](#release-v21060) release notes for more detail about new features in 21.06.

For a detailed list of changes, please refer to the
[CHANGELOG](https://github.com/NVIDIA/spark-rapids/blob/main/CHANGELOG.md).

## Release v21.06.1
This is a patch release to address an issue with the plugin in the Databricks 7.3 ML LTS runtime.

Expand All @@ -86,7 +128,7 @@ Software Requirements:

CUDA & Nvidia Drivers*: 11.0 or 11.2 & v450.80.02+

Apache Spark 3.0.1, 3.0.2, 3.1.1, 3.1.2, Cloudera CDP 7.1.7, Databricks 7.3 ML LTS or 8.2 ML Runtime, and GCP Dataproc 2.0
Apache Spark 3.0.1, 3.0.2, 3.1.1, 3.1.2, Cloudera CDP 7.1.7, and GCP Dataproc 2.0

Apache Hadoop 2.10+ or 3.1.1+ (3.1.1 for nvidia-docker version 2)

Expand Down
2 changes: 1 addition & 1 deletion docs/get-started/getting-started-databricks.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ CUDA 11.0 toolkit on the cluster. This can be done with the [generate-init-scri
Spark plugin and the CUDA 11 toolkit.
- [Databricks 8.2
ML](https://docs.databricks.com/release-notes/runtime/8.2ml.html#system-environment) has CUDA 11
installed. In this case use
installed. Users will need to use 21.06.2 or later on Databricks 8.2 ML. In this case use
[generate-init-script.ipynb](../demo/Databricks/generate-init-script.ipynb) which will install
the RAPIDS Spark plugin.
2. Once you are in the notebook, click the “Run All” button.
Expand Down

0 comments on commit 20f4020

Please sign in to comment.