Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI feast materialize does not work with custom providers #1589

Closed
MattDelac opened this issue May 27, 2021 · 1 comment · Fixed by #1594
Closed

CLI feast materialize does not work with custom providers #1589

MattDelac opened this issue May 27, 2021 · 1 comment · Fixed by #1594

Comments

@MattDelac
Copy link
Collaborator

MattDelac commented May 27, 2021

Expected Behavior

This should work as it works with feast apply

Current Behavior

You get a ModuleNotFoundError
image

Steps to reproduce

I have my own custom provider

# feature_store.yaml
project: local
registry: tmp/registry.db
provider: pano.provider.LocalProvider
offline_store:
    type: bigquery
online_store:
    type: sqlite
    path: tmp/database.db
  • feast apply works as expected
  • feast materialize-incremental does not because it cannot find my "pano" module

Specifications

  • Version: feast 0.10.5
  • Platform: Mac
  • Subsystem:

Possible Solution

They both use get_provider() from infra/provider.py

The apply method in the CLI will do

os.chdir(repo_path)
sys.path.append("")
...
infra_provider = get_provider(repo_config, repo_path)

The materialize-incremental method of the CLI will do

provider = self._get_provider()
# That will return `return get_provider(self.config, self.repo_path)`

So the difference is that the FeatureStore does not append the current directory in the sys path

@woop
Copy link
Member

woop commented May 27, 2021

The solution seems reasonable, if only for consistency sake. Although I am curious how you are structuring your repository now, and whether you think our current approach of importing the current working directory is appropriate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants