Skip to content

Commit

Permalink
[Improve][Core] Add the spotless plugin for unified code style. (apac…
Browse files Browse the repository at this point in the history
  • Loading branch information
FlechazoW authored Feb 10, 2023
1 parent f165b18 commit 290c4a4
Show file tree
Hide file tree
Showing 13 changed files with 137 additions and 692 deletions.
17 changes: 8 additions & 9 deletions .github/workflows/backend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ jobs:
with:
submodules: true
- name: Check code style
run: ./mvnw --batch-mode --quiet --no-snapshot-updates clean checkstyle:check
run: ./mvnw --batch-mode --quiet --no-snapshot-updates clean spotless:check

dead-link:
if: github.repository == 'apache/incubator-seatunnel'
Expand Down Expand Up @@ -277,7 +277,6 @@ jobs:
./mvnw -B -q install -DskipTests
-D"maven.test.skip"=true
-D"maven.javadoc.skip"=true
-D"checkstyle.skip"=true
-D"license.skipAddThirdParty"
- name: Check Dependencies Licenses
run: tools/dependencies/checkLicense.sh
Expand All @@ -302,14 +301,14 @@ jobs:
- name: run all modules unit test
if: needs.changes.outputs.api == 'true'
run: |
./mvnw -B -T 1C clean verify -D"maven.test.skip"=false -D"checkstyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates
./mvnw -B -T 1C clean verify -D"maven.test.skip"=false -D"license.skipAddThirdParty"=true --no-snapshot-updates
env:
MAVEN_OPTS: -Xmx4096m

- name: run updated modules unit test
if: needs.changes.outputs.api == 'false' && needs.changes.outputs.ut-modules != ''
run: |
./mvnw -B -T 1C clean verify -D"maven.test.skip"=false -D"checkstyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl ${{needs.changes.outputs.ut-modules}} -am -Pci
./mvnw -B -T 1C clean verify -D"maven.test.skip"=false -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl ${{needs.changes.outputs.ut-modules}} -am -Pci
env:
MAVEN_OPTS: -Xmx4096m

Expand All @@ -333,7 +332,7 @@ jobs:
- name: run updated modules integration test
if: needs.changes.outputs.api == 'false' && needs.changes.outputs.it-modules != ''
run: |
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"checkstyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl ${{needs.changes.outputs.it-modules}} -am -Pci
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl ${{needs.changes.outputs.it-modules}} -am -Pci
env:
MAVEN_OPTS: -Xmx2048m

Expand All @@ -357,7 +356,7 @@ jobs:
- name: run some modules integration test
if: needs.changes.outputs.api == 'true'
run: |
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"checkstyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl :connector-seatunnel-e2e-base -am -Pci
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl :connector-seatunnel-e2e-base -am -Pci
env:
MAVEN_OPTS: -Xmx4096m

Expand Down Expand Up @@ -410,7 +409,7 @@ jobs:
./mvnw help:evaluate -Dexpression=project.modules -q -DforceStdout -pl :seatunnel-connector-v2-e2e >> /tmp/sub_module.txt
sub_modules=`python tools/update_modules_check/update_modules_check.py sub /tmp/sub_module.txt`
run_it_modules=`python tools/update_modules_check/update_modules_check.py sub_it_module $sub_modules 5 0`
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"checkstyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl $run_it_modules -am -Pci
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl $run_it_modules -am -Pci
env:
MAVEN_OPTS: -Xmx4096m

Expand Down Expand Up @@ -439,7 +438,7 @@ jobs:
./mvnw help:evaluate -Dexpression=project.modules -q -DforceStdout -pl :seatunnel-connector-v2-e2e >> /tmp/sub_module.txt
sub_modules=`python tools/update_modules_check/update_modules_check.py sub /tmp/sub_module.txt`
run_it_modules=`python tools/update_modules_check/update_modules_check.py sub_it_module $sub_modules 5 1`
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"checkstyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl $run_it_modules -am -Pci
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl $run_it_modules -am -Pci
env:
MAVEN_OPTS: -Xmx4096m

Expand Down Expand Up @@ -468,7 +467,7 @@ jobs:
./mvnw help:evaluate -Dexpression=project.modules -q -DforceStdout -pl :seatunnel-connector-v2-e2e >> /tmp/sub_module.txt
sub_modules=`python tools/update_modules_check/update_modules_check.py sub /tmp/sub_module.txt`
run_it_modules=`python tools/update_modules_check/update_modules_check.py sub_it_module $sub_modules 5 2`
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"checkstyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl $run_it_modules -am -Pci
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl $run_it_modules -am -Pci
env:
MAVEN_OPTS: -Xmx4096m

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/codeql.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 120
env:
JAVA_TOOL_OPTIONS: -Xmx2G -Xms2G -Dhttp.keepAlive=false -Dmaven.test.skip=true -Dcheckstyle.skip=true -Dlicense.skipAddThirdParty=true -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.http.retryHandler.count=3 -Dmaven.wagon.httpconnectionManager.ttlSeconds=120
JAVA_TOOL_OPTIONS: -Xmx2G -Xms2G -Dhttp.keepAlive=false -Dmaven.test.skip=true -Dlicense.skipAddThirdParty=true -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.http.retryHandler.count=3 -Dmaven.wagon.httpconnectionManager.ttlSeconds=120

strategy:
fail-fast: false
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ jobs:
run: |
./mvnw -B package \
-Dmaven.test.skip=true \
-Dcheckstyle.skip=true \
-Dlicense.skipAddThirdParty=true \
-Dhttp.keepAlive=false \
-Dmaven.wagon.http.pool=false \
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/license.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ jobs:
${{ runner.os }}-maven-
- name: Generate THIRD-PARTY
run: |
./mvnw clean license:aggregate-add-third-party -DskipTests -Dcheckstyle.skip -U
./mvnw clean license:aggregate-add-third-party -DskipTests -U
- name: Check LICENSE file
run: |
python3 tools/dependencies/license.py seatunnel-dist/target/THIRD-PARTY.txt seatunnel-dist/release-docs/LICENSE true
1 change: 0 additions & 1 deletion .github/workflows/publish-docker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,6 @@ jobs:
./mvnw -B clean deploy \
-Dmaven.test.skip \
-Dmaven.javadoc.skip \
-Dmaven.checkstyle.skip \
-Dlicense.skipAddThirdParty=true \
-Dmaven.deploy.skip \
-Ddocker.tag=${{ github.sha }} \
Expand Down
7 changes: 3 additions & 4 deletions .github/workflows/schedule_backend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
with:
submodules: true
- name: Check code style
run: ./mvnw --batch-mode --quiet --no-snapshot-updates clean checkstyle:check
run: ./mvnw --batch-mode --quiet --no-snapshot-updates clean spotless:check

dead-link:
if: github.repository == 'apache/incubator-seatunnel'
Expand Down Expand Up @@ -95,7 +95,6 @@ jobs:
-D"maven.test.skip"=true
-D"maven.javadoc.skip"=true
-D"scalastyle.skip"=true
-D"checkstyle.skip"=true
-D"license.skipAddThirdParty"
- name: Check Dependencies Licenses
run: tools/dependencies/checkLicense.sh
Expand All @@ -118,7 +117,7 @@ jobs:
cache: 'maven'
- name: run all modules unit test
run: |
./mvnw -B -T 1C clean verify -D"maven.test.skip"=false -D"checkstyle.skip"=true -D"scalastyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates
./mvnw -B -T 1C clean verify -D"maven.test.skip"=false -D"scalastyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates
env:
MAVEN_OPTS: -Xmx2048m

Expand All @@ -140,7 +139,7 @@ jobs:
cache: 'maven'
- name: run all modules integration test
run: |
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"checkstyle.skip"=true -D"scalastyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"scalastyle.skip"=true -D"license.skipAddThirdParty"=true --no-snapshot-updates
env:
MAVEN_OPTS: -Xmx2048m

45 changes: 15 additions & 30 deletions docs/en/contribution/coding-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,24 +43,9 @@ This guide documents an overview of the current Apache SeaTunnel modules and bes

Tips:**For more details, you can refer to [issue guide](https://seatunnel.apache.org/community/contribution_guide/contribute#issue) and [pull request guide](https://seatunnel.apache.org/community/contribution_guide/contribute#pull-request)**

4. Apache SeaTunnel's code style has the following specifications (it is recommended to use the auto-formatted code feature of idea).
4. Code segments are never repeated. If a code segment is used multiple times, define it multiple times is not a good option, make it a public segment for other modules to use is a best practice.

> https://github.com/apache/incubator-seatunnel/pull/2641
>
> #2641 will introduce the spotless plugin, contributors need to use the plugin to format the code before submitting pr
1. Indent to 4 spaces
2. Keyword (try if else catch...) there should be 1 space between keyword and `(`
3. There should be 1 space between `)` and `{`
4. There should be 1 space between parameter splitting
5. If there are more than 3 chain calls, there should be a separate line for each call
6. If-else should be wrapped in `{}` even if the method body is only one line
7. etc....

Tips: **For more details, you can refer to `checkStyle.xml` that at tools/checkstyle**

5. Code segments are never repeated. If a code segment is used multiple times, define it multiple times is not a good option, make it a public segment for other modules to use is a best practice.

6. When throwing an exception, throw the exception along with a hint message and the exception should be smaller in scope.Throwing overly broad exceptions promotes complex error handling code that is more likely to contain security vulnerabilities.For example, if your connector encounters an `IOException` while reading data, a reasonable approach would be to the following:
5. When throwing an exception, throw the exception along with a hint message and the exception should be smaller in scope.Throwing overly broad exceptions promotes complex error handling code that is more likely to contain security vulnerabilities.For example, if your connector encounters an `IOException` while reading data, a reasonable approach would be to the following:

```java
try {
Expand All @@ -70,7 +55,7 @@ This guide documents an overview of the current Apache SeaTunnel modules and bes
}
```

7. The Apache project has very strict licensing requirements, so every file in an Apache project should contain a license statement. Check that each new file you add contains the `Apache License Header` before submitting pull request:
6. The Apache project has very strict licensing requirements, so every file in an Apache project should contain a license statement. Check that each new file you add contains the `Apache License Header` before submitting pull request:

```java
/*
Expand All @@ -91,13 +76,13 @@ This guide documents an overview of the current Apache SeaTunnel modules and bes
*/
```

8. Apache SeaTunnel has uniform specifications and requirements for code formatting. Check the format of your code before submitting pull request will better help you pass CI:
7. Apache SeaTunnel uses `Spotless` for code style and formatting checks. You could run the following command and `Spotless` will automatically fix the code style and formatting errors for you:

```shell
./mvnw checkstyle:check
./mvnw spotless:apply
```

9. Before you submit your pull request, make sure the project will compile properly after adding your code, you can use the following commands to package the whole project:
8. Before you submit your pull request, make sure the project will compile properly after adding your code, you can use the following commands to package the whole project:

```shell
# multi threads compile
Expand All @@ -109,23 +94,23 @@ This guide documents an overview of the current Apache SeaTunnel modules and bes
./mvnw clean package
```

10. Before submitting pull request, do a full unit test and integration test locally can better verify the functionality of your code, best practice is to use the `seatunnel-examples` module's ability to self-test to ensure that the multi-engine is running properly and the results are correct.
9. Before submitting pull request, do a full unit test and integration test locally can better verify the functionality of your code, best practice is to use the `seatunnel-examples` module's ability to self-test to ensure that the multi-engine is running properly and the results are correct.
11. If you submit a pull request with a feature that requires updated documentation, always remember to update the documentation.
10. If you submit a pull request with a feature that requires updated documentation, always remember to update the documentation.
12. Submit the pull request of connector type can write e2e test to ensure the robustness and robustness of the code, e2e test should include the full data type, and e2e test as little as possible to initialize the docker image, write the test cases of sink and source together to reduce the loss of resources, while using asynchronous features to ensure the stability of the test. A good example can be found at: [MongodbIT.java](https://github.com/apache/incubator-seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-mongodb-e2e/src/test/java/org/apache/seatunnel/e2e/connector/v2/mongodb/MongodbIT.java)
13. The priority of property permission in the class is set to `private`, and mutability is set to `final`, which can be changed reasonably if special circumstances are encountered.
12. The priority of property permission in the class is set to `private`, and mutability is set to `final`, which can be changed reasonably if special circumstances are encountered.
14. The properties in the class and method parameters prefer to use the base type(int boolean double float...), not recommended to use the wrapper type(Integer Boolean Double Float...), if encounter special circumstances reasonable change.
13. The properties in the class and method parameters prefer to use the base type(int boolean double float...), not recommended to use the wrapper type(Integer Boolean Double Float...), if encounter special circumstances reasonable change.
15. When developing a sink connector you need to be aware that the sink will be serialized, and if some properties cannot be serialized, encapsulate the properties into classes and use the singleton pattern.
14. When developing a sink connector you need to be aware that the sink will be serialized, and if some properties cannot be serialized, encapsulate the properties into classes and use the singleton pattern.
16. If there are multiple `if` process judgments in the code flow, try to simplify the flow to multiple ifs instead of if-else-if.
15. If there are multiple `if` process judgments in the code flow, try to simplify the flow to multiple ifs instead of if-else-if.
17. Pull request has the characteristic of single responsibility, not allowed to include irrelevant code of the feature in pull request, once this situation deal with their own branch before submitting pull request, otherwise the Apache SeaTunnel community will actively close pull request
16. Pull request has the characteristic of single responsibility, not allowed to include irrelevant code of the feature in pull request, once this situation deal with their own branch before submitting pull request, otherwise the Apache SeaTunnel community will actively close pull request
18. Contributors should be responsible for their own pull request. If your pull request contains new features or modifies old features, add test cases or e2e tests to prove the reasonableness and functional integrity of your pull request is a good practice.
17. Contributors should be responsible for their own pull request. If your pull request contains new features or modifies old features, add test cases or e2e tests to prove the reasonableness and functional integrity of your pull request is a good practice.
19. If you think which part of the community's current code is unreasonable (especially the core `core` module and the `api` module), the function needs to be updated or modified, the first thing to do is to propose a `discuss issue` or `email` with the community to discuss the need to modify this part of the function, if the community agrees to submit pull request again, do not submit the issue and pull request directly without discussion, so the community will directly consider this pull request is useless, and will be closed down.
18. If you think which part of the community's current code is unreasonable (especially the core `core` module and the `api` module), the function needs to be updated or modified, the first thing to do is to propose a `discuss issue` or `email` with the community to discuss the need to modify this part of the function, if the community agrees to submit pull request again, do not submit the issue and pull request directly without discussion, so the community will directly consider this pull request is useless, and will be closed down.

18 changes: 10 additions & 8 deletions docs/en/contribution/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,13 +64,15 @@ See [install plugins for IDEA](https://www.jetbrains.com/help/idea/managing-plug
Before running the following example, you should also install JetBrains IntelliJ IDEA's [Lombok plugin](https://plugins.jetbrains.com/plugin/6317-lombok).
See [install plugins for IDEA](https://www.jetbrains.com/help/idea/managing-plugins.html#install-plugins) if you want to.

### Install JetBrains IDEA CheckStyle-IDEA Plugin
### Code Style

Before coding, you should also install JetBrains IntelliJ IDEA's [CheckStyle-IDEA plugin](https://plugins.jetbrains.com/plugin/1065-checkstyle-idea).
See [install plugins for IDEA](https://www.jetbrains.com/help/idea/managing-plugins.html#install-plugins) if you want to.
Next, you should go to `Preferences -> Editor -> Code style -> Scheme -> Import Scheme -> CheckStyle Configration` and import `tools/checkstyle/checkStyle.xml`
![checkstyle.png](../images/checkstyle.png)
If you want to change to automatically formatting, these configurations are also required.
Apache SeaTunnel uses `Spotless` for code style and formatting checks. You could run the following command and `Spotless` will automatically fix the code style and formatting errors for you:

```shell
./mvnw spotless:apply
```

You could copy the `pre-commit hook` file `/tools/spotless_check/pre-commit.sh` to your `.git/hooks/` directory so that every time you commit your code with `git commit`, `Spotless` will automatically fix things for you.

## Run Simple Example

Expand All @@ -81,7 +83,6 @@ it in IDEA](https://www.jetbrains.com/help/idea/run-debug-configuration.html) as
Here we use `seatunnel-examples/seatunnel-flink-connector-v2-example/src/main/java/org/apache/seatunnel/example/flink/v2/SeaTunnelApiExample.java`
as an example, when you run it successfully you could see the output as below:


```log
+I[Ricky Huo, 71]
+I[Gary, 12]
Expand All @@ -95,7 +96,7 @@ as an example, when you run it successfully you could see the output as below:

All our examples use simple source and sink to make it less dependent and easy to run. You can change the example configuration
in `resources/examples`. You could change your configuration as below, if you want to use PostgreSQL as the source and
sink to console.
sink to console.

```conf
env {
Expand All @@ -115,3 +116,4 @@ sink {
ConsoleSink {}
}
```

Loading

0 comments on commit 290c4a4

Please sign in to comment.