Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supports new error messages. #520

Merged
merged 5 commits into from
Nov 29, 2022
Merged

Supports new error messages. #520

merged 5 commits into from
Nov 29, 2022

Conversation

ueshin
Copy link
Contributor

@ueshin ueshin commented Nov 16, 2022

resolves #515

Description

Supports new error messages.

In SparkAdapter.get_columns_in_relation, it checks the error message when the specified table or view doesn't exist:

if "Table or view not found" in errmsg or "NoSuchTableException" in errmsg:

but, Spark will change the error message in the future release (apache/spark#37887), which causes the function to raise the dbt.exceptions.RuntimeException instead of returning an empty list.

The function should also check whether the error message contains [TABLE_OR_VIEW_NOT_FOUND] or not.

Checklist

@cla-bot cla-bot bot added the cla:yes label Nov 16, 2022
@jtcohen6 jtcohen6 added the ready_for_review Externally contributed PR has functional approval, ready for code review from Core engineering label Nov 17, 2022
@McKnight-42 McKnight-42 self-requested a review November 28, 2022 17:52
@McKnight-42 McKnight-42 reopened this Nov 28, 2022
Copy link
Contributor

@McKnight-42 McKnight-42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all seems like a reasonable change to me

@McKnight-42 McKnight-42 self-requested a review November 28, 2022 21:09
@McKnight-42 McKnight-42 self-requested a review November 28, 2022 21:21
@McKnight-42
Copy link
Contributor

@ueshin seems the change you made for the tuple may of introduced something black doesn't like if you could run pre-commit run --all-files locally there might be a spacing, or extra line issue its throwing an error against.

Copy link
Contributor

@McKnight-42 McKnight-42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work @ueshin this looks like a good early change.

@McKnight-42 McKnight-42 merged commit becc101 into dbt-labs:main Nov 29, 2022
McKnight-42 pushed a commit that referenced this pull request Nov 29, 2022
* Supports new error messages.

* changie

* tuple

* fix.

* suggestion.
@ueshin ueshin deleted the errmsg branch November 30, 2022 19:37
McKnight-42 added a commit that referenced this pull request Dec 2, 2022
* Supports new error messages.

* changie

* tuple

* fix.

* suggestion.

Co-authored-by: Takuya UESHIN <ueshin@databricks.com>
McKnight-42 added a commit that referenced this pull request Dec 2, 2022
* manual backport/modifcation of #520 to mergable to 1.2.latest

* modification to handle older style of code

Co-authored-by: Takuya UESHIN <ueshin@databricks.com>
McKnight-42 added a commit that referenced this pull request Dec 2, 2022
* manual backport/modifcation of #520 to mergable to 1.1.latest

* returning to older style call in logic

Co-authored-by: Takuya UESHIN <ueshin@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla:yes ready_for_review Externally contributed PR has functional approval, ready for code review from Core engineering
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-1503] [Feature] Support new error messages in the future Spark.
4 participants