BigQuery, MySQL, SQLite and SQL Server support #27
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
wendy-aw
commented
May 28, 2024
wendy-aw
commented
- Added a new script that translates the Postgres .sql files into the different dialects. Translation uses both sqlglot as well as regex operations.
- The script performs translation, and also creation of databases in the different db systems. It further checks that values have been inserted properly.
- Some modifications have been made to the original .sql files to strictly adhere to the data types set out in the DDL statements.
rishsriv
reviewed
May 29, 2024
| This will create one new SQL file per database per dialect. | ||
| For SQLite, the `.db` files will be saved in the folder `sqlite_dbs`. | ||
| Note that BigQuery, MySQL and SQLite do not support schemas and hence the SQL files will be modified to skip schema creation. | ||
|
|
There was a problem hiding this comment.
Very helpful! Thanks for laying this down so clearly :D
rishsriv
reviewed
May 29, 2024
| (2, 'Advanced Calculus', 'Mathematics', 'MATH201', 4, 'CS101', NULL, 'This course covers advanced topics in calculus.', 1, 3, false, false, true, true, 5, 4, 2, 3), | ||
| (3, 'Introduction to Physics', 'Physics', 'PHYS101', 3, NULL, 'MATH201', 'This course provides an introduction to physics principles.', 2, 1, true, true, true, true, 8, 4, 3, 5), | ||
| (4, 'Distributed Databases', 'Computer Science', 'CS302', 3, NULL, 'CS101', 'This course provides an introduction to distributed databases.', 2, 2, true, true, false, true, 4, 2, 1, 5) | ||
| (1, 'Introduction to Computer Science', 'Computer Science', 'CS101', '3', NULL, NULL, 'This course introduces the basics of computer science.', 2, 2, true, false, true, false, 10, 5, 3, 4), |
There was a problem hiding this comment.
Thanks for the fix here! TIL that Postgres likely automatically casted this to a string earlier
rishsriv
reviewed
May 29, 2024
There was a problem hiding this comment.
THANK YOU FOR THIS! Wow this would've taken forever to get right. Much appreciated!
rishsriv
approved these changes
May 29, 2024
wongjingping
approved these changes
May 29, 2024
| @@ -228,6 +228,19 @@ def load_embeddings(emb_path: str) -> tuple[dict, dict]: | |||
| "journal.journalname,text,Name or title of the journal", | |||
| ], | |||
| }, | |||
| "broker": { | |||
There was a problem hiding this comment.
Thanks for adding these placeholders!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.