*: merge BR into TiDB (pingcap#26655)

.codecov.yml

@@ -0,0 +1,22 @@
+codecov:
+  require_ci_to_pass: yes
+
+comment:
+  layout: "reach, diff, flags, files"
+  behavior: default
+  require_changes: false  # if true: only post the comment if coverage changes
+  require_base: no        # [yes :: must have a base report to post]
+  require_head: yes       # [yes :: must have a head report to post]
+  branches:               # branch names that can post comment
+    - "master"
+
+coverage:
+  status:
+    project:
+      default:
+        # Allow the coverage to drop by 3%
+        target: 85%
+        threshold: 3%
+        branches:
+          - master
+    patch: off

.dockerignore

@@ -0,0 +1 @@
+.gitignore

.editorconfig

@@ -0,0 +1,10 @@
+[*]
+end_of_line = lf
+insert_final_newline = true
+charset = utf-8
+
+# tab_size = 4 spaces
+[*.go]
+indent_style = tab
+indent_size = 4
+trim_trailing_whitespace = true

.gitattributes

@@ -0,0 +1 @@
+*_generated.go linguist-generated=true

.github/ISSUE_TEMPLATE/bug-report.md

@@ -0,0 +1,45 @@
+---
+name: "🐛 Bug Report"
+about: Something isn't working as expected
+title: ''
+labels: 'type/bug '
+---
+
+Please answer these questions before submitting your issue. Thanks!
+
+1. What did you do?
+If possible, provide a recipe for reproducing the error.
+
+
+2. What did you expect to see?
+
+
+
+3. What did you see instead?
+
+
+
+4. What version of BR and TiDB/TiKV/PD are you using?
+
+<!--
+br -V
+tidb-lightning -V
+tidb-server -V
+tikv-server -V
+pd-server -V
+-->
+
+5. Operation logs
+   - Please upload `br.log` for BR if possible
+   - Please upload `tidb-lightning.log` for TiDB-Lightning if possible
+   - Please upload `tikv-importer.log` from TiKV-Importer if possible
+   - Other interesting logs
+
+
+6. Configuration of the cluster and the task
+   - `tidb-lightning.toml` for TiDB-Lightning if possible
+   - `tikv-importer.toml` for TiKV-Importer if possible
+   - `topology.yml` if deployed by TiUP
+
+
+7. Screenshot/exported-PDF of Grafana dashboard or metrics' graph in Prometheus if possible

.github/ISSUE_TEMPLATE/feature-request.md

@@ -0,0 +1,19 @@
+---
+name: "🚀 Feature Request"
+about: I have a suggestion
+labels: 'type/feature-request'
+---
+
+## Feature Request
+
+### Describe your feature request related problem:
+<!-- A description of what the problem is. -->
+
+### Describe the feature you'd like:
+<!-- A description of what you want to happen. -->
+
+### Describe alternatives you've considered:
+<!-- A description of any alternative solutions or features you've considered. -->
+
+### Teachability, Documentation, Adoption, Migration Strategy:
+<!-- If you can, explain some scenarios how users might use this, or situations in which it would be helpful. Any API designs, mockups, or diagrams are also helpful. -->

.github/ISSUE_TEMPLATE/question.md

@@ -0,0 +1,24 @@
+---
+name: "\U0001F914 Question"
+labels: "type/question"
+about: Usage question that isn't answered in docs or discussion
+
+---
+
+## Question
+
+
+<!--
+
+Before asking a question, make sure you have:
+
+- Searched existing Stack Overflow questions.
+- Googled your question.
+- Searched open and closed [GitHub issues](https://github.com/pingcap/br/issues?q=is%3Aissue)
+- Read the documentation:
+    * [BR documentation](https://docs.pingcap.com/tidb/stable/backup-and-restore-tool)
+    * [Lightning documentation](https://docs.pingcap.com/tidb/stable/tidb-lightning-overview)
+    * [BR 中文文档](https://docs.pingcap.com/zh/tidb/stable/backup-and-restore-tool)
+    * [Lightning 中文文档](https://docs.pingcap.com/zh/tidb/stable/tidb-lightning-overview)
+
+-->

.github/challenge-bot.yml

@@ -0,0 +1 @@
+defaultSigLabel: sig/migrate

.github/pull_request_template.md

@@ -0,0 +1,42 @@
+<!--
+Thank you for working on BR! Please read BR's [CONTRIBUTING](https://github.com/pingcap/br/blob/master/CONTRIBUTING.md) document **BEFORE** filing this PR.
+-->
+
+### What problem does this PR solve? <!--add issue link with summary if exists-->
+
+
+### What is changed and how it works?
+
+
+### Check List <!--REMOVE the items that are not applicable-->
+
+Tests <!-- At least one of them must be included. -->
+
+ - Unit test
+ - Integration test
+ - Manual test (add detailed scripts or steps below)
+ - No code
+
+Code changes
+
+ - Has exported function/method change
+ - Has exported variable/fields change
+ - Has interface methods change
+ - Has persistent data change
+
+Side effects
+
+ - Possible performance regression
+ - Increased code complexity
+ - Breaking backward compatibility
+
+Related changes
+
+ - Need to cherry-pick to the release branch
+ - Need to update the documentation
+
+### Release note
+
+ -
+
+<!-- fill in the release note, or just write "No release note" -->

.github/workflows/build.yml

@@ -0,0 +1,68 @@
+name: build
+on:
+  push:
+    branches:
+      - master
+      - 'release-[0-9].[0-9]*'
+    paths-ignore:
+      - '**.html'
+      - '**.md'
+      - 'CNAME'
+      - 'LICENSE'
+      - 'docs/**'
+      - 'tests/**'
+      - 'docker/**'
+      - '.github/workflows/**.yml'
+  pull_request:
+    branches:
+      - master
+      - 'release-[0-9].[0-9]*'
+    paths-ignore:
+      - '**.html'
+      - '**.md'
+      - 'CNAME'
+      - 'LICENSE'
+      - 'docs/**'
+      - 'tests/**'
+      - 'docker/**'
+      - '.github/workflows/**.yml'
+
+jobs:
+  compile:
+    name: ${{ matrix.os }} / ${{ matrix.target}}
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        include:
+          - os: macos-latest
+            target: x86_64-apple-darwin
+
+          - os: ubuntu-latest
+            target: aarch64-unknown-linux-gnu
+
+          - os: windows-latest
+            target: x86_64-pc-windows-msvc
+    steps:
+      - uses: actions/checkout@v2.1.0
+
+      - name: Set up Go
+        uses: actions/setup-go@v2
+        with:
+          go-version: 1.16
+
+      - name: Run build
+        run: make build
+
+  compile-freebsd:
+    name: Compile for FreeBSD job
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2.1.0
+
+      - name: Set up Go
+        uses: actions/setup-go@v2
+        with:
+          go-version: 1.16
+
+      - name: Compile for FreeBSD
+        run: GOOS=freebsd make build

.github/workflows/compatible_test.yml

@@ -0,0 +1,71 @@
+name: compatibility-test
+
+on:
+  push:
+    branches:
+      - master
+      - 'release-[0-9].[0-9]*'
+    paths-ignore:
+      - '**.html'
+      - '**.md'
+      - 'CNAME'
+      - 'LICENSE'
+      - 'docs/**'
+      - 'tests/**'
+      - 'docker/**'
+      - '.github/workflows/**.yml'
+  pull_request:
+    branches:
+      - master
+      - 'release-[0-9].[0-9]*'
+    paths-ignore:
+      - '**.html'
+      - '**.md'
+      - 'CNAME'
+      - 'LICENSE'
+      - 'docs/**'
+      - 'tests/**'
+      - 'docker/**'
+      - '.github/workflows/**.yml'
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    timeout-minutes: 25
+    steps:
+
+    - name: Free disk space
+      run: |
+        sudo rm -rf /usr/local/lib/android
+        sudo rm -rf /usr/share/dotnet
+        docker volume prune -f
+        docker image prune -f
+
+    - uses: actions/checkout@v2
+
+    - name: Set up Go
+      uses: actions/setup-go@v2
+      with:
+        go-version: 1.16
+
+    - name: Generate compatibility test backup data
+      timeout-minutes: 15
+      run: sh compatibility/prepare_backup.sh
+
+    - name: Start server
+      run: |
+        TAG=nightly PORT_SUFFIX=1 docker-compose -f compatibility/backup_cluster.yaml rm -s -v
+        TAG=nightly PORT_SUFFIX=1 docker-compose -f compatibility/backup_cluster.yaml build
+        TAG=nightly PORT_SUFFIX=1 docker-compose -f compatibility/backup_cluster.yaml up --remove-orphans -d
+        TAG=nightly PORT_SUFFIX=1 docker-compose -f compatibility/backup_cluster.yaml exec -T control make compatibility_test
+
+    - name: Collect component log
+      if: ${{ failure() }}
+      run: |
+        tar czvf ${{ github.workspace }}/logs.tar.gz /tmp/br/docker/backup_logs/*
+
+    - uses: actions/upload-artifact@v2
+      if: ${{ failure() }}
+      with:
+        name: logs
+        path: ${{ github.workspace }}/logs.tar.gz

.gitignore

@@ -0,0 +1,14 @@
+/br
+/bin
+/.idea
+/docker/data/
+/docker/logs/
+*.swp
+.DS_Store
+/go.mod
+/go.sum
+
+# for the web interface
+web/node_modules/
+web/dist/
+.vscode/

.golangci.yml

@@ -0,0 +1,12 @@
+linters-settings:
+  gocyclo:
+    min-complexity: 40
+
+issues:
+  # Excluding configuration per-path, per-linter, per-text and per-source
+  exclude-rules:
+    # TODO Remove it.
+    - path: split_client.go
+      text: "SA1019:"
+      linters:
+        - staticcheck

COMPATIBILITY_TEST.md

@@ -0,0 +1,42 @@
+# Compatibility test
+
+## Background
+
+We had some incompatibility issues in the past, which made BR cannot restore backed up data in some situations.
+So we need a test workflow to check the compatiblity.
+
+## Goal
+
+- Ensure backward compatibility for restoring data from the previous 3 minor versions
+
+## Workflow
+
+### Data Preparation
+
+This workflow needs previous backup data. To get this data. we perform the following steps
+
+- Run a TiDB cluster with previous version.
+- Run backup jobs with corresponding BR version, with different storages (s3, gcs).
+
+Given we test for the previous 3 versions, and there are 2 different storage systems, we will produce 6 backup archives for 6 separate compatibility tests.
+
+### Test Content
+
+- Start TiDB cluster with nightly version.
+- Build BR binary with current directory.
+- Use BR to restore different version backup data one by one.
+- Make sure restore data is expected.
+
+### Running tests
+
+Start a cluster with docker-compose and Build br with latest version.
+
+```sh
+docker-compose -f docker-compose.yaml rm -s -v && \
+docker-compose -f docker-compose.yaml build && \
+docker-compose -f docker-compose.yaml up --remove-orphans
+```
+
+```sh
+docker-compose -f docker-compose.yaml control make compatibility_test
+```