Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQuerySource does not validate whether the table exists or not #1596

Closed
achals opened this issue May 28, 2021 · 2 comments · Fixed by #1627
Closed

BigQuerySource does not validate whether the table exists or not #1596

achals opened this issue May 28, 2021 · 2 comments · Fixed by #1627
Assignees
Labels

Comments

@achals
Copy link
Member

achals commented May 28, 2021

Expected Behavior

When specifying an invalid table, feast should surface an error that explicitly states this.

Current Behavior

We currently get an error that implies that the schema cannot be parsed.

$ feast apply
Traceback (most recent call last):
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/bin/feast", line 33, in <module>
    sys.exit(load_entry_point('feast', 'console_scripts', 'feast')())
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/achal/tecton/feast/.direnv/python-3.7.10/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/achal/tecton/feast/sdk/python/feast/cli.py", line 192, in apply_total_command
    apply_total(repo_config, repo)
  File "/Users/achal/tecton/feast/sdk/python/feast/telemetry.py", line 151, in exception_logging_wrapper
    result = func(*args, **kwargs)
  File "/Users/achal/tecton/feast/sdk/python/feast/repo_operations.py", line 131, in apply_total
    repo = parse_repo(repo_path)
  File "/Users/achal/tecton/feast/sdk/python/feast/repo_operations.py", line 97, in parse_repo
    module = importlib.import_module(module_path)
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/achal/tecton/feast/splendid_lark/example.py", line 46, in <module>
    table_ref="feast-oss.demo_data.driver_activity"
  File "/Users/achal/tecton/feast/sdk/python/feast/data_source.py", line 677, in __init__
    or self._infer_event_timestamp_column("TIMESTAMP|DATETIME"),
  File "/Users/achal/tecton/feast/sdk/python/feast/data_source.py", line 547, in _infer_event_timestamp_column
    """
TypeError:
                    Unable to infer DataSource event_timestamp_column due to an absence of columns that satisfy the criteria.
                     Please specify event_timestamp_column explicitly.

Steps to reproduce

  • Create a feature view with an invalid BigQuery source.

driver_stats_fv = FeatureView(
    name="driver_activity",
    entities=["driver_id"],
    input=BigQuerySource(
        table_ref="feast-oss.demo_data.driver_activity"
    )
)

and then run feast apply

Specifications

  • Version:
  • Platform:
  • Subsystem:

Possible Solution

Run client.get_table as validation before any other operations.

@achals achals added good first issue Good for newcomers priority/p1 labels May 28, 2021
@codyjlin
Copy link
Contributor

codyjlin commented Jun 3, 2021

I'll be working on this, supervised by David Liu @mavysavydav from Twitter

@woop
Copy link
Member

woop commented Jun 4, 2021

Sounds good!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants