Skip to content

Commit

Permalink
Merge branch 'dev' into dev
Browse files Browse the repository at this point in the history
  • Loading branch information
@hailin0
hailin0 authored Jul 17, 2023
2 parents a394642 + c37b85b commit 7686541
Show file tree
Hide file tree
Showing 105 changed files with 2,719 additions and 324 deletions.
75 changes: 66 additions & 9 deletions .github/workflows/backend.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@
name: Backend
on:
push:
pull_request:
branches:
- dev
pull_request:
paths-ignore:
- 'docs/**'
- '**/*.md'
Expand All @@ -32,7 +32,7 @@ concurrency:

jobs:
license-header:
if: github.repository == 'apache/seatunnel'
if: github.repository == '${{github.actor}}/seatunnel'
name: License header
runs-on: ubuntu-latest
timeout-minutes: 10
Expand All @@ -44,7 +44,7 @@ jobs:
uses: apache/skywalking-eyes@985866ce7e324454f61e22eb2db2e998db09d6f3

code-style:
if: github.repository == 'apache/seatunnel'
if: github.repository == '${{github.actor}}/seatunnel'
name: Code style
runs-on: ubuntu-latest
timeout-minutes: 10
Expand All @@ -56,7 +56,7 @@ jobs:
run: ./mvnw --batch-mode --quiet --no-snapshot-updates clean spotless:check

dead-link:
if: github.repository == 'apache/seatunnel'
if: github.repository == '${{github.actor}}/seatunnel'
name: Dead links
runs-on: ubuntu-latest
timeout-minutes: 30
Expand All @@ -69,7 +69,7 @@ jobs:
done
sanity-check:
if: github.repository == 'apache/seatunnel'
if: github.repository == '${{github.actor}}/seatunnel'
name: Sanity check results
needs: [ license-header, code-style, dead-link ]
runs-on: ubuntu-latest
Expand All @@ -83,8 +83,7 @@ jobs:
changes:
runs-on: ubuntu-latest
# To prevent error when there's no base branch
if: github.repository == 'apache/seatunnel'
if: github.repository == '${{github.actor}}/seatunnel'
timeout-minutes: 10
outputs:
api: ${{ steps.filter.outputs.api }}
Expand Down Expand Up @@ -312,7 +311,7 @@ jobs:
- name: run updated modules integration test (part-1)
if: needs.changes.outputs.api == 'false' && needs.changes.outputs.it-modules != ''
run: |
sub_modules=`python tools/update_modules_check/update_modules_check.py sub_update_it_module ${{needs.changes.outputs.it-modules}} 2 0`
sub_modules=`python tools/update_modules_check/update_modules_check.py sub_update_it_module ${{needs.changes.outputs.it-modules}} 4 0`
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl $sub_modules -am -Pci
env:
MAVEN_OPTS: -Xmx2048m
Expand All @@ -337,7 +336,65 @@ jobs:
- name: run updated modules integration test (part-2)
if: needs.changes.outputs.api == 'false' && needs.changes.outputs.it-modules != ''
run: |
sub_modules=`python tools/update_modules_check/update_modules_check.py sub_update_it_module ${{needs.changes.outputs.it-modules}} 2 1`
sub_modules=`python tools/update_modules_check/update_modules_check.py sub_update_it_module ${{needs.changes.outputs.it-modules}} 4 1`
if [ ! -z $sub_modules ]; then
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl $sub_modules -am -Pci
else
echo "sub modules is empty, skipping"
fi
env:
MAVEN_OPTS: -Xmx2048m

updated-modules-integration-test-part-3:
needs: [ changes, sanity-check ]
if: needs.changes.outputs.api == 'false' && needs.changes.outputs.it-modules != ''
runs-on: ${{ matrix.os }}
strategy:
matrix:
java: [ '8', '11' ]
os: [ 'ubuntu-latest' ]
timeout-minutes: 90
steps:
- uses: actions/checkout@v2
- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v3
with:
java-version: ${{ matrix.java }}
distribution: 'temurin'
cache: 'maven'
- name: run updated modules integration test (part-3)
if: needs.changes.outputs.api == 'false' && needs.changes.outputs.it-modules != ''
run: |
sub_modules=`python tools/update_modules_check/update_modules_check.py sub_update_it_module ${{needs.changes.outputs.it-modules}} 4 2`
if [ ! -z $sub_modules ]; then
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl $sub_modules -am -Pci
else
echo "sub modules is empty, skipping"
fi
env:
MAVEN_OPTS: -Xmx2048m

updated-modules-integration-test-part-4:
needs: [ changes, sanity-check ]
if: needs.changes.outputs.api == 'false' && needs.changes.outputs.it-modules != ''
runs-on: ${{ matrix.os }}
strategy:
matrix:
java: [ '8', '11' ]
os: [ 'ubuntu-latest' ]
timeout-minutes: 90
steps:
- uses: actions/checkout@v2
- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v3
with:
java-version: ${{ matrix.java }}
distribution: 'temurin'
cache: 'maven'
- name: run updated modules integration test (part-4)
if: needs.changes.outputs.api == 'false' && needs.changes.outputs.it-modules != ''
run: |
sub_modules=`python tools/update_modules_check/update_modules_check.py sub_update_it_module ${{needs.changes.outputs.it-modules}} 4 3`
if [ ! -z $sub_modules ]; then
./mvnw -T 1C -B verify -DskipUT=true -DskipIT=false -D"license.skipAddThirdParty"=true --no-snapshot-updates -pl $sub_modules -am -Pci
else
Expand Down
38 changes: 19 additions & 19 deletions docs/en/concept/schema-feature.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,25 @@ Some NoSQL databases or message queue are not strongly limited schema, so the sc

## What type supported at now

| Data type | Description |
|:----------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| string | string |
| boolean | boolean |
| tinyint | -128 to 127 regular. 0 to 255 unsigned*. Specify the maximum number of digits in parentheses. |
| smallint | -32768 to 32767 General. 0 to 65535 unsigned*. Specify the maximum number of digits in parentheses. |
| int | All numbers from -2,147,483,648 to 2,147,483,647 are allowed. |
| bigint | All numbers between -9,223,372,036,854,775,808 and 9,223,372,036,854,775,807 are allowed. |
| float | Float-precision numeric data from -1.79E+308 to 1.79E+308. |
| double | Double precision floating point. Handle most decimals. |
| decimal | DOUBLE type stored as a string, allowing a fixed decimal point. |
| null | null |
| bytes | bytes. |
| date | Only the date is stored. From January 1, 0001 to December 31, 9999. |
| time | Only store time. Accuracy is 100 nanoseconds. |
| timestamp | Stores a unique number that is updated whenever a row is created or modified. timestamp is based on the internal clock and does not correspond to real time. There can only be one timestamp variable per table. |
| row | Row type,can be nested. |
| map | A Map is an object that maps keys to values. The key type includes `int` `string` `boolean` `tinyint` `smallint` `bigint` `float` `double` `decimal` `date` `time` `timestamp` `null` , and the value type includes `int` `string` `boolean` `tinyint` `smallint` `bigint` `float` `double` `decimal` `date` `time` `timestamp` `null` `array` `map`. |
| array | A array is a data type that represents a collection of elements. The element type includes `int` `string` `boolean` `tinyint` `smallint` `bigint` `float` `double` `array` `map`. |
| Data type | Value type in Java | Description |
|:----------|:---------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| string | `java.lang.String` | string |
| boolean | `java.lang.Boolean` | boolean |
| tinyint | `java.lang.Byte` | -128 to 127 regular. 0 to 255 unsigned*. Specify the maximum number of digits in parentheses. |
| smallint | `java.lang.Short` | -32768 to 32767 General. 0 to 65535 unsigned*. Specify the maximum number of digits in parentheses. |
| int | `java.lang.Integer` | All numbers from -2,147,483,648 to 2,147,483,647 are allowed. |
| bigint | `java.lang.Long` | All numbers between -9,223,372,036,854,775,808 and 9,223,372,036,854,775,807 are allowed. |
| float | `java.lang.Float` | Float-precision numeric data from -1.79E+308 to 1.79E+308. |
| double | `java.lang.Double` | Double precision floating point. Handle most decimals. |
| decimal | `java.math.BigDecimal` | DOUBLE type stored as a string, allowing a fixed decimal point. |
| null | `java.lang.Void` | null |
| bytes | `byte[]` | bytes. |
| date | `java.time.LocalDate` | Only the date is stored. From January 1, 0001 to December 31, 9999. |
| time | `java.time.LocalTime` | Only store time. Accuracy is 100 nanoseconds. |
| timestamp | `java.time.LocalDateTime` | Stores a unique number that is updated whenever a row is created or modified. timestamp is based on the internal clock and does not correspond to real time. There can only be one timestamp variable per table. |
| row | `org.apache.seatunnel.api.table.type.SeaTunnelRow` | Row type,can be nested. |
| map | `java.util.Map` | A Map is an object that maps keys to values. The key type includes `int` `string` `boolean` `tinyint` `smallint` `bigint` `float` `double` `decimal` `date` `time` `timestamp` `null` , and the value type includes `int` `string` `boolean` `tinyint` `smallint` `bigint` `float` `double` `decimal` `date` `time` `timestamp` `null` `array` `map`. |
| array | `ValueType[]` | A array is a data type that represents a collection of elements. The element type includes `int` `string` `boolean` `tinyint` `smallint` `bigint` `float` `double` `array` `map`. |

## How to use schema

Expand Down
Loading

0 comments on commit 7686541

Please sign in to comment.