use prettier check in CI (#453)

* use prettier check in CI * format two more files * fix docs/specification/invariants.md
apache · Jun 3, 2021 · a1b8305 · a1b8305
1 parent 33ff660
commit a1b8305
Show file tree

Hide file tree

Showing 4 changed files with 88 additions and 85 deletions.
diff --git a/.github/workflows/dev.yml b/.github/workflows/dev.yml
@@ -23,7 +23,6 @@ on:
   pull_request:
 
 jobs:
-
   lint:
     name: Lint C++, Python, R, Rust, Docker, RAT
     runs-on: ubuntu-latest
@@ -37,3 +36,17 @@ jobs:
         run: pip install -e dev/archery[docker]
       - name: Lint
         run: archery lint --rat
+
+  prettier:
+    name: Use prettier to check formatting of documents
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-node@v2
+        with:
+          node-version: "14"
+      - name: Prettier check
+        run: |
+          # if you encounter error, try rerun the command below with --write instead of --check
+          # and commit the changes
+          npx prettier@2.3.0 --check {ballista,datafusion,datafusion-examples,dev,docs,python}/**/*.md README.md DEVELOPERS.md
diff --git a/DEVELOPERS.md b/DEVELOPERS.md
@@ -105,8 +105,8 @@ new specifications as you see fit.
 
 Here is the list current active specifications:
 
-* [Output field name semantic](docs/specification/output-field-name-semantic.md)
-* [Invariants](docs/specification/invariants.md)
+- [Output field name semantic](docs/specification/output-field-name-semantic.md)
+- [Invariants](docs/specification/invariants.md)
 
 ## How to format `.md` document
 

diff --git a/docs/specification/invariants.md b/docs/specification/invariants.md
@@ -42,11 +42,11 @@ know what they need to enforce at the coding level.
 
 ## Notation
 
-* Field or physical field: the tuple name, `arrow::DataType` and nullability flag (a bool whether values can be null), represented in this document by `PF(name, type, nullable)`
-* Logical field: Field with a relation name. Represented in this document by `LF(relation, name, type, nullable)`
-* Projected plan: plan with projection as the root node.
-* Logical schema: a vector of logical fields, used by logical plan.
-* Physical schema: a vector of physical fields, used by both physical plan and Arrow record batch.
+- Field or physical field: the tuple name, `arrow::DataType` and nullability flag (a bool whether values can be null), represented in this document by `PF(name, type, nullable)`
+- Logical field: Field with a relation name. Represented in this document by `LF(relation, name, type, nullable)`
+- Projected plan: plan with projection as the root node.
+- Logical schema: a vector of logical fields, used by logical plan.
+- Physical schema: a vector of physical fields, used by both physical plan and Arrow record batch.
 
 ### Logical
 
@@ -62,8 +62,8 @@ logical_field(lf1: LF, lf2: LF, ...) -> LF
 
 Examples:
 
-* `plus(a,b) -> LF(None, "{a} Plus {b}", d(a.type,b.type), a.nullable | b.nullable)` where d is the function mapping input types to output type (`get_supertype` in our current implementation).
-* `length(a) -> LF(None, "length({a})", u32, a.nullable)`
+- `plus(a,b) -> LF(None, "{a} Plus {b}", d(a.type,b.type), a.nullable | b.nullable)` where d is the function mapping input types to output type (`get_supertype` in our current implementation).
+- `length(a) -> LF(None, "length({a})", u32, a.nullable)`
 
 #### Plan
 
@@ -91,8 +91,8 @@ physical_field(PF1, PF2, ...) -> PF
 
 Examples:
 
-* `plus(a,b) -> PF("{a} Plus {b}", d(a.type,b.type), a.nullable | b.nullable)` where d is a complex function (`get_supertype` in our current implementation) whose computation is for each element in the columns, sum the two entries together and return it in the same type as the smallest type of both columns.
-* `length(&str) -> PF("length({a})", u32, a.nullable)` whose computation is "count number of bytes in the string".
+- `plus(a,b) -> PF("{a} Plus {b}", d(a.type,b.type), a.nullable | b.nullable)` where d is a complex function (`get_supertype` in our current implementation) whose computation is for each element in the columns, sum the two entries together and return it in the same type as the smallest type of both columns.
+- `length(&str) -> PF("length({a})", u32, a.nullable)` whose computation is "count number of bytes in the string".
 
 #### Plan
 
@@ -216,8 +216,8 @@ guarantee this invariant.
 
 In particular:
 
-* The derived DataType matches the code it uses to build the array for every branch of valid input type combinations.
-* The nullability flag matches how the values are built.
+- The derived DataType matches the code it uses to build the array for every branch of valid input type combinations.
+- The nullability flag matches how the values are built.
 
 #### Validation
 

diff --git a/docs/specification/output-field-name-semantic.md b/docs/specification/output-field-name-semantic.md
@@ -23,20 +23,20 @@ This specification documents how field names in output record batches should be
 generated based on given user queries. The filed name rules apply to
 Datafusion queries planned from both SQL queries and Dataframe APIs.
 
-##  Field name rules
-
-* All field names MUST not contain relation/table qualifier.
-  * Both `SELECT t1.id`, `SELECT id` and `df.select_columns(&["id"])` SHOULD result in field name: `id`
-* Function names MUST be converted to lowercase.
-  * `SELECT AVG(c1)` SHOULD result in field name: `avg(c1)`
-* Literal string MUST not be wrapped with quotes or double quotes.
-  * `SELECT 'foo'` SHOULD result in field name: `foo`
-* Operator expressions MUST be wrapped with parentheses.
-  * `SELECT -2` SHOULD result in field name: `(- 2)`
-* Operator and operand MUST be separated by spaces.
-  * `SELECT 1+2` SHOULD result in field name: `(1 + 2)`
-* Function arguments MUST be separated by a comma `,` and a space.
-  * `SELECT f(c1,c2)` and `df.select(vec![f.udf("f")?.call(vec![col("c1"), col("c2")])])`  SHOULD result in field name: `f(c1, c2)`
+## Field name rules
+
+- All field names MUST not contain relation/table qualifier.
+  - Both `SELECT t1.id`, `SELECT id` and `df.select_columns(&["id"])` SHOULD result in field name: `id`
+- Function names MUST be converted to lowercase.
+  - `SELECT AVG(c1)` SHOULD result in field name: `avg(c1)`
+- Literal string MUST not be wrapped with quotes or double quotes.
+  - `SELECT 'foo'` SHOULD result in field name: `foo`
+- Operator expressions MUST be wrapped with parentheses.
+  - `SELECT -2` SHOULD result in field name: `(- 2)`
+- Operator and operand MUST be separated by spaces.
+  - `SELECT 1+2` SHOULD result in field name: `(1 + 2)`
+- Function arguments MUST be separated by a comma `,` and a space.
+  - `SELECT f(c1,c2)` and `df.select(vec![f.udf("f")?.call(vec![col("c1"), col("c2")])])` SHOULD result in field name: `f(c1, c2)`
 
 ## Appendices
 
@@ -66,26 +66,24 @@ JOIN t2 ON t1.id = t2.id
 
 Datafusion Arrow record batches output:
 
-| id | a   | id | b     |
-|----|-----|----|-------|
-| 1  | foo | 1  | hello |
-| 2  | bar | 2  | world |
-
+| id  | a   | id  | b     |
+| --- | --- | --- | ----- |
+| 1   | foo | 1   | hello |
+| 2   | bar | 2   | world |
 
 Spark, MySQL 8 and PostgreSQL 13 output:
 
-| id | a   | id | b     |
-|----|-----|----|-------|
-| 1  | foo | 1  | hello |
-| 2  | bar | 2  | world |
+| id  | a   | id  | b     |
+| --- | --- | --- | ----- |
+| 1   | foo | 1   | hello |
+| 2   | bar | 2   | world |
 
 SQLite 3 output:
 
-| id | a   | b     |
-|----|-----|-------|
-| 1  | foo | hello |
-| 2  | bar | world |
-
+| id  | a   | b     |
+| --- | --- | ----- |
+| 1   | foo | hello |
+| 2   | bar | world |
 
 #### Function transformed columns
 
@@ -98,41 +96,38 @@ SELECT ABS(t1.id), abs(-id) FROM t1;
 Datafusion Arrow record batches output:
 
 | abs(id) | abs((- id)) |
-|---------|-------------|
+| ------- | ----------- |
 | 1       | 1           |
 | 2       | 2           |
 
-
 Spark output:
 
 | abs(id) | abs((- id)) |
-|---------|-------------|
+| ------- | ----------- |
 | 1       | 1           |
 | 2       | 2           |
 
-
 MySQL 8 output:
 
 | ABS(t1.id) | abs(-id) |
-|------------|----------|
+| ---------- | -------- |
 | 1          | 1        |
 | 2          | 2        |
 
 PostgreSQL 13 output:
 
 | abs | abs |
-|-----|-----|
+| --- | --- |
 | 1   | 1   |
 | 2   | 2   |
 
 SQlite 3 output:
 
 | ABS(t1.id) | abs(-id) |
-|------------|----------|
+| ---------- | -------- |
 | 1          | 1        |
 | 2          | 2        |
 
-
 #### Function with operators
 
 Query:
@@ -143,40 +138,38 @@ SELECT t1.id + ABS(id), ABS(id * t1.id) FROM t1;
 
 Datafusion Arrow record batches output:
 
-| id + abs(id) | abs(id * id) |
-|--------------|--------------|
-| 2            | 1            |
-| 4            | 4            |
-
+| id + abs(id) | abs(id \* id) |
+| ------------ | ------------- |
+| 2            | 1             |
+| 4            | 4             |
 
 Spark output:
 
-| id + abs(id) | abs(id * id) |
-|--------------|--------------|
-| 2            | 1            |
-| 4            | 4            |
+| id + abs(id) | abs(id \* id) |
+| ------------ | ------------- |
+| 2            | 1             |
+| 4            | 4             |
 
 MySQL 8 output:
 
-| t1.id + ABS(id) | ABS(id * t1.id) |
-|-----------------|-----------------|
-| 2               | 1               |
-| 4               | 4               |
+| t1.id + ABS(id) | ABS(id \* t1.id) |
+| --------------- | ---------------- |
+| 2               | 1                |
+| 4               | 4                |
 
 PostgreSQL output:
 
 | ?column? | abs |
-|----------|-----|
+| -------- | --- |
 | 2        | 1   |
 | 4        | 4   |
 
 SQLite output:
 
-| t1.id + ABS(id) | ABS(id * t1.id) |
-|-----------------|-----------------|
-| 2               | 1               |
-| 4               | 4               |
-
+| t1.id + ABS(id) | ABS(id \* t1.id) |
+| --------------- | ---------------- |
+| 2               | 1                |
+| 4               | 4                |
 
 #### Project literals
 
@@ -188,33 +181,30 @@ SELECT 1, 2+5, 'foo_bar';
 
 Datafusion Arrow record batches output:
 
-| 1 | (2 + 5) | foo_bar |
-|---|---------|---------|
-| 1 | 7       | foo_bar |
-
+| 1   | (2 + 5) | foo_bar |
+| --- | ------- | ------- |
+| 1   | 7       | foo_bar |
 
 Spark output:
 
-| 1 | (2 + 5) | foo_bar |
-|---|---------|---------|
-| 1 | 7       | foo_bar |
+| 1   | (2 + 5) | foo_bar |
+| --- | ------- | ------- |
+| 1   | 7       | foo_bar |
 
 MySQL output:
 
-| 1 | 2+5 | foo_bar |
-|---|-----|---------|
-| 1 | 7   | foo_bar |
-
+| 1   | 2+5 | foo_bar |
+| --- | --- | ------- |
+| 1   | 7   | foo_bar |
 
 PostgreSQL output:
 
 | ?column? | ?column? | ?column? |
-|----------|----------|----------|
+| -------- | -------- | -------- |
 | 1        | 7        | foo_bar  |
 
-
 SQLite 3 output:
 
-| 1 | 2+5 | 'foo_bar' |
-|---|-----|-----------|
-| 1 | 7   | foo_bar   |
+| 1   | 2+5 | 'foo_bar' |
+| --- | --- | --------- |
+| 1   | 7   | foo_bar   |