Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update download and databricks doc for 21.06.2 [skip ci] #3214

Merged
merged 1 commit into from
Aug 12, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 43 additions & 1 deletion docs/download.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,48 @@ New functionality and performance improvements for this release include:
For a detailed list of changes, please refer to the
[CHANGELOG](https://github.com/NVIDIA/spark-rapids/blob/main/CHANGELOG.md).

## Release v21.06.2
This is a patch release to address an issue with the plugin in the Databricks 8.2 ML runtime.

Hardware Requirements:

GPU Architecture: NVIDIA V100, T4 or A10/A30/A100 GPUs

Software Requirements:

OS: Ubuntu 18.04, Ubuntu 20.04 or CentOS 7, CentOS 8

CUDA & Nvidia Drivers*: 11.0 or 11.2 & v450.80.02+

Apache Spark 3.0.1, 3.0.2, 3.1.1, 3.1.2, Cloudera CDP 7.1.7, Databricks 7.3 ML LTS or 8.2 ML Runtime, and GCP Dataproc 2.0

Apache Hadoop 2.10+ or 3.1.1+ (3.1.1 for nvidia-docker version 2)

Python 3.6+, Scala 2.12, Java 8

*Some hardware may have a minimum driver version greater than v450.80.02+. Check the GPU spec sheet
for your hardware's minimum driver version.

### Download v21.06.2
* Download the [RAPIDS
Accelerator for Apache Spark 21.06.2 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/21.06.2/rapids-4-spark_2.12-21.06.2.jar)
* Download the [RAPIDS cuDF 21.06.1 jar](https://repo1.maven.org/maven2/ai/rapids/cudf/21.06.1/cudf-21.06.1-cuda11.jar)

This package is built against CUDA 11.2 and has [CUDA forward
compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/index.html) enabled. It is tested
on Tesla datacenter GPUs with CUDA 11.0 and 11.2. For those using other types of GPUs which
do not have CUDA forward compatibility (for example, GeForce), CUDA 11.2 is required. Users will
need to ensure the minimum driver (450.80.02) and CUDA toolkit are installed on each Spark node.

### Release Notes
This release patches the plugin to address a backwards incompatible change to Parquet filters made
by Databricks in the Databricks 8.2 ML runtime. More information is in [issue
3191](https://github.com/NVIDIA/spark-rapids/issues/3191) in the RAPIDS Spark repository. See the
[Release v21.06.0](#release-v21060) release notes for more detail about new features in 21.06.

For a detailed list of changes, please refer to the
[CHANGELOG](https://github.com/NVIDIA/spark-rapids/blob/main/CHANGELOG.md).

## Release v21.06.1
This is a patch release to address an issue with the plugin in the Databricks 7.3 ML LTS runtime.

Expand All @@ -86,7 +128,7 @@ Software Requirements:

CUDA & Nvidia Drivers*: 11.0 or 11.2 & v450.80.02+

Apache Spark 3.0.1, 3.0.2, 3.1.1, 3.1.2, Cloudera CDP 7.1.7, Databricks 7.3 ML LTS or 8.2 ML Runtime, and GCP Dataproc 2.0
Apache Spark 3.0.1, 3.0.2, 3.1.1, 3.1.2, Cloudera CDP 7.1.7, and GCP Dataproc 2.0

Apache Hadoop 2.10+ or 3.1.1+ (3.1.1 for nvidia-docker version 2)

Expand Down
2 changes: 1 addition & 1 deletion docs/get-started/getting-started-databricks.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ CUDA 11.0 toolkit on the cluster. This can be done with the [generate-init-scri
Spark plugin and the CUDA 11 toolkit.
- [Databricks 8.2
ML](https://docs.databricks.com/release-notes/runtime/8.2ml.html#system-environment) has CUDA 11
installed. In this case use
installed. Users will need to use 21.06.2 or later on Databricks 8.2 ML. In this case use
[generate-init-script.ipynb](../demo/Databricks/generate-init-script.ipynb) which will install
the RAPIDS Spark plugin.
2. Once you are in the notebook, click the “Run All” button.
Expand Down