Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve robustness of get_task HF pipeline invocations #4848

Closed
vblagoje opened this issue May 9, 2023 · 0 comments · Fixed by #5284
Closed

Improve robustness of get_task HF pipeline invocations #4848

vblagoje opened this issue May 9, 2023 · 0 comments · Fixed by #5284
Assignees
Labels
P1 High priority, add to the next sprint topic:promptnode

Comments

@vblagoje
Copy link
Member

vblagoje commented May 9, 2023

Is your feature request related to a problem? Please describe.
As a Haystack user, I noticed that when HF infra is temp down I can't use a newly added Cohere model. The issue has been traced to the PromptModelInvocationLayer.supports invocation order. Because HFLocalInvocationLayer is listed in haystack/nodes/prompt/invocation_layer/__init__.py before CohereInvocationLayer, HFLocalInvocationLayer gets registered via __init_subclass__ mechanism before CohereInvocationLayer. Then when we invoke HFLocalInvocationLayer.supports, the get_task call blocks indefinitely, thus preventing the use of the Cohere-based PromptNode.

Describe the solution you'd like
I would like Haystack PromptNode initialization to be as robust as possible and not dependent on HF infra outage, especially when I am not using one of the HF invocation layers for PromptNode.

Describe alternatives you've considered
No resolution is possible atm aside from waiting for the outage to stop

Additional context
Add any other context or screenshots about the feature request here.

@julian-risch julian-risch added P2 Medium priority, add to the next sprint if no P1 available topic:promptnode labels May 25, 2023
@julian-risch julian-risch added P1 High priority, add to the next sprint and removed P2 Medium priority, add to the next sprint if no P1 available labels Jul 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 High priority, add to the next sprint topic:promptnode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants