Compress large test event logs

.github/CODEOWNERS

@@ -0,0 +1,14 @@
+#  Copyright (c) 2020, NVIDIA CORPORATION.
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+
+/jenkins/	@jlowe @revans2 @tgravescs @GaryShen2008 @NvTimLiu
+pom.xml		@jlowe @revans2 @tgravescs @GaryShen2008 @NvTimLiu

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,25 @@
+---
+name: Bug report
+about: Create a bug report to help us improve RAPIDS Accelerator for Apache Spark
+title: "[BUG]"
+labels: "? - Needs Triage, bug"
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**Steps/Code to reproduce bug**
+Please provide a list of steps or a code sample to reproduce the issue.
+Avoid posting private or sensitive data.
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Environment details (please complete the following information)**
+ - Environment location: [Standalone, YARN, Kubernetes, Cloud(specify cloud provider)]
+ - Spark configuration settings related to the issue
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/documentation-request.md

@@ -0,0 +1,35 @@
+---
+name: Documentation request
+about: Report incorrect or needed documentation
+title: "[DOC]"
+labels: "? - Needs Triage, documentation"
+assignees: ''
+
+---
+
+## Report incorrect documentation
+
+**Location of incorrect documentation**
+Provide links and line numbers if applicable.
+
+**Describe the problems or issues found in the documentation**
+A clear and concise description of what you found to be incorrect.
+
+**Steps taken to verify documentation is incorrect**
+List any steps you have taken:
+
+**Suggested fix for documentation**
+Detail proposed changes to fix the documentation if you have any.
+
+---
+
+## Report needed documentation
+
+**Report needed documentation**
+A clear and concise description of what documentation you believe it is needed and why.
+
+**Describe the documentation you'd like**
+A clear and concise description of what you want to happen.
+
+**Steps taken to search for needed documentation**
+List any steps you have taken:

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for RAPIDS Accelerator for Apache Spark
+title: "[FEA]"
+labels: "? - Needs Triage, feature request"
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I wish the RAPIDS Accelerator for Apache Spark would [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context, code examples, or references to existing implementations about the feature request here.

.github/ISSUE_TEMPLATE/submit-question.md

@@ -0,0 +1,10 @@
+---
+name: Submit question
+about: Ask a general question about RAPIDS Accelerator for Apache Spark
+title: "[QST]"
+labels: "? - Needs Triage, question"
+assignees: ''
+
+---
+
+**What is your question?**

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,33 @@
+<!--
+
+Thank you for contributing to RAPIDS Accelerator for Apache Spark!
+
+Here are some guidelines to help the review process go smoothly.
+
+1. Please write a description in this text box of the changes that are being
+   made.
+
+2. Please ensure that you have written units tests for the changes made/features
+   added.
+
+3. If you are closing an issue please use one of the automatic closing words as
+   noted here: https://help.github.com/articles/closing-issues-using-keywords/
+
+4. If your pull request is not ready for review but you want to make use of the
+   continuous integration testing facilities please label it with `[WIP]`.
+
+5. If your pull request is ready to be reviewed without requiring additional
+   work on top of it, then remove the `[WIP]` label (if present).
+
+6. Once all work has been done and review has taken place please do not add
+   features or make changes out of the scope of those requested by the reviewer
+   (doing this just add delays as already reviewed code ends up having to be
+   re-reviewed/it is hard to tell what is new etc!). Further, please avoid
+   rebasing your branch during the review process, as this causes the context
+   of any comments made by reviewers to be lost. If conflicts occur during
+   review then they should be resolved by merging into the branch used for
+   making the pull request.
+
+Many thanks in advance for your cooperation!
+
+-->

.github/codecov.yaml

@@ -0,0 +1,16 @@
+# Copyright (c) 2021, NVIDIA CORPORATION.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+codecov:
+  max_report_age: off

.github/workflows/auto-merge.yml

@@ -0,0 +1,41 @@
+# Copyright (c) 2020-2021, NVIDIA CORPORATION.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# A workflow to keep BASE branch up-to-date from HEAD branch
+name: auto-merge HEAD to BASE
+
+on:
+  pull_request_target:
+    branches:
+    - branch-21.06
+    types: [closed]
+
+jobs:
+  auto-merge:
+    if: github.event.pull_request.merged == true
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          ref: branch-21.06 # force to fetch from latest upstream instead of PR ref
+
+      - name: auto-merge job
+        uses: ./.github/workflows/auto-merge
+        env:
+          OWNER: NVIDIA
+          REPO_NAME: spark-rapids
+          HEAD: branch-21.06
+          BASE: branch-21.08
+          AUTOMERGE_TOKEN: ${{ secrets.AUTOMERGE_TOKEN }} # use to merge PR

.github/workflows/auto-merge/Dockerfile

@@ -0,0 +1,22 @@
+# Copyright (c) 2020, NVIDIA CORPORATION.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM python:alpine
+
+WORKDIR /
+COPY automerge .
+RUN pip install requests && chmod +x /automerge
+
+# require envs: OWNER,REPO_NAME,HEAD,BASE,GITHUB_TOKEN
+ENTRYPOINT ["/automerge"]

.github/workflows/auto-merge/action.yml

@@ -0,0 +1,20 @@
+# Copyright (c) 2020, NVIDIA CORPORATION.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: 'auto-merge action'
+description: 'auto-merge HEAD to BASE'
+runs:
+  using: 'docker'
+  image: 'Dockerfile'
+

.github/workflows/auto-merge/automerge

@@ -0,0 +1,122 @@
+#!/usr/bin/env python
+
+# Copyright (c) 2020, NVIDIA CORPORATION.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""A auto-merge tool
+
+Create a PR to merge HEAD to BASE branch.
+NOTE:
+    The generated PR should be automatically merged if no conflict. Otherwise, manual operation will be required.
+"""
+
+import os
+import sys
+import time
+
+import requests
+
+# ENV
+OWNER = os.environ.get('OWNER')
+assert OWNER, 'env OWNER should not be empty'
+REPO_NAME = os.environ.get('REPO_NAME')
+assert REPO_NAME, 'env REPO_NAME should not be empty'
+HEAD = os.environ.get('HEAD')
+assert HEAD, 'env HEAD should not be empty'
+BASE = os.environ.get('BASE')
+assert BASE, 'env BASE should not be empty'
+AUTOMERGE_TOKEN = os.environ.get('AUTOMERGE_TOKEN')
+assert AUTOMERGE_TOKEN, 'env AUTOMERGE_TOKEN should not be empty'
+# static
+API_URL = 'https://api.github.com'
+AUTH_HEADERS = {
+    'Authorization': 'token ' + AUTOMERGE_TOKEN
+}
+
+
+def create():
+    url = f'{API_URL}/repos/{OWNER}/{REPO_NAME}/pulls'
+    params = {
+        'title': f'[auto-merge] {HEAD} to {BASE} [skip ci] [bot]',
+        'head': HEAD,
+        'base': BASE,
+        'body': f'auto-merge triggered by github actions on `{HEAD}` to create a PR keeping `{BASE}` up-to-date. If '
+                'this PR is unable to be merged due to conflicts, it will remain open until manually fix.',
+        'maintainer_can_modify': True
+    }
+    r = requests.post(url, headers=AUTH_HEADERS, json=params)
+    if r.status_code == 201:
+        print('SUCCESS - create PR')
+        pull = r.json()
+        number = str(pull['number'])
+        sha = str(pull['head']['sha'])
+        return number, sha, False
+    if r.status_code == 422:  # early-terminate if no commits between HEAD and BASE
+        print('SUCCESS - No commits')
+        print(r.json())
+        return '', '', True
+    # FAILURE
+    print('FAILURE - create PR')
+    print(f'status code: {r.status_code}')
+    print(r.json())
+    sys.exit(1)
+
+
+def auto_merge(number, sha):
+    url = f'{API_URL}/repos/{OWNER}/{REPO_NAME}/pulls/{number}/merge'
+    params = {
+        'sha': sha,
+        'merge_method': 'merge'
+    }
+    r = requests.put(url, headers=AUTH_HEADERS, json=params)
+    if r.status_code == 200:
+        comment(number, '**SUCCESS** - auto-merge')
+        print('SUCCESS - auto-merge')
+        sys.exit(0)
+    else:
+        print('FAILURE - auto-merge')
+        comment(number=number, content=f"""**FAILURE** - Unable to auto-merge. Manual operation is required.
+```
+{r.json()}
+```
+""")
+        print(f'status code: {r.status_code}')
+        print(r.json())
+        sys.exit(1)
+
+
+def comment(number, content):
+    url = f'{API_URL}/repos/{OWNER}/{REPO_NAME}/issues/{number}/comments'
+    params = {
+        'body': content
+    }
+    r = requests.post(url, headers=AUTH_HEADERS, json=params)
+    if r.status_code == 201:
+        print('SUCCESS - create comment')
+    else:
+        print('FAILURE - create comment')
+        print(f'status code: {r.status_code}')
+        print(r.json())
+
+
+def main():
+    number, sha, term = create()
+    if term:
+        sys.exit(0)
+
+    auto_merge(number, sha)
+
+
+if __name__ == '__main__':
+    main()