Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(sql-lab): remove redundant onChange schema property #24422

Merged
merged 2 commits into from
Jun 16, 2023

Conversation

villebro
Copy link
Member

@villebro villebro commented Jun 16, 2023

SUMMARY

While creating a Java Client from the the Superset OpenAPI spec with the OpenAPI codegen, a collision between the changedOn and changed_on properties in the SQL Lab QueryRestultSchema caused an error. This happened because it automatically converts the snake_case property to a camelCase one. This type of overlap doesn't exist in other schemas. In addition, when checking other similar schemas, all the other ones are of type fields.DateTime, except SliceSchema and ChartEntityResponseSchema, which were fields.String. For this reason, the redundant changedOn property was removed from QueryResultSchema, and the type of changed_on was updated to DateTime for QueryResultSchema, SliceSchema and ChartEntityResponseSchema.

Furthermore, when looking at the SQL Lab code, I noticed inconsistencies in the test fixtures and the reducer:

  • The SouthPane test didn't have changed_on in the fixture, only changedOn. Furthermore, the type was wrong - the fixture had an epoch, while the value is in fact an ISO8601 string (see screenshot). These were updated.
  • The SqlLab fixture also showed the changedOn property as an epoch. However, the changed_on property there was in correct format, i.e. ISO-8601 string. To comply with the updated schema, changedOn was removed.
  • the SQL Lab reducer was comparing the ISO-8601 string value of changedOn with an epoch-valued queriesLastUpdate. I'm not sure what kind of results that resulted in, but to fix this, we now refer to changed_on, and convert it to epoch format first before doing the comparison.

This inconsistency in types has been verified on both a SQLite and Postgres metastore backend, so it shouldn't be affected by the metastore's temporal column logic.

SCREENSHOTS

Currently on master, the values of changedOn and changed_on are identical in the query response:
image

Notice how queriesLastUpdate is an epoch, but changedOn which it's being compared to in the reducer is an ISO-8601 string?

image
image

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@codecov
Copy link

codecov bot commented Jun 16, 2023

Codecov Report

Merging #24422 (c990622) into master (d2b0b8e) will decrease coverage by 0.06%.
The diff coverage is 81.39%.

❗ Current head c990622 differs from pull request most recent head 308c5c6. Consider uploading reports for the commit 308c5c6 to get more accurate results

@@            Coverage Diff             @@
##           master   #24422      +/-   ##
==========================================
- Coverage   68.96%   68.91%   -0.06%     
==========================================
  Files        1904     1904              
  Lines       74088    73920     -168     
  Branches     8120     8119       -1     
==========================================
- Hits        51096    50943     -153     
+ Misses      20880    20866      -14     
+ Partials     2112     2111       -1     
Flag Coverage Δ
hive 53.92% <73.68%> (+0.05%) ⬆️
javascript 55.65% <66.66%> (+<0.01%) ⬆️
mysql 79.20% <100.00%> (-0.06%) ⬇️
postgres 79.29% <100.00%> (-0.05%) ⬇️
presto 53.84% <73.68%> (+0.05%) ⬆️
python 83.28% <100.00%> (-0.04%) ⬇️
sqlite 77.79% <100.00%> (-0.05%) ⬇️
unit 54.62% <73.68%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ackages/superset-ui-core/src/utils/featureFlags.ts 100.00% <ø> (ø)
superset-frontend/src/SqlLab/fixtures.ts 100.00% <ø> (ø)
superset-frontend/src/SqlLab/reducers/sqlLab.js 40.93% <0.00%> (-0.25%) ⬇️
...end/src/dashboard/components/SliceHeader/index.tsx 90.56% <ø> (ø)
superset-frontend/src/pages/DatasetList/index.tsx 57.76% <ø> (ø)
...rset-frontend/src/profile/components/Favorites.tsx 80.00% <ø> (ø)
superset/charts/api.py 87.07% <ø> (ø)
superset/config.py 91.97% <ø> (ø)
superset/connectors/sqla/models.py 90.37% <ø> (-0.07%) ⬇️
superset/dashboards/api.py 92.58% <ø> (ø)
... and 20 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@villebro villebro force-pushed the villebro/sql-result-changed-on branch from d89fa08 to 308c5c6 Compare June 16, 2023 10:47
@villebro villebro merged commit 20e0036 into apache:master Jun 16, 2023
@villebro villebro deleted the villebro/sql-result-changed-on branch June 16, 2023 16:07
jinghua-qa added a commit to preset-io/superset that referenced this pull request Jun 22, 2023
jinghua-qa added a commit to preset-io/superset that referenced this pull request Jun 22, 2023
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 3.0.0 labels Mar 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/S 🚢 3.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants