Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caching in the facade and after pre-processing in Argo stores ? #204

Open
gmaze opened this issue Apr 6, 2022 · 4 comments
Open

Caching in the facade and after pre-processing in Argo stores ? #204

gmaze opened this issue Apr 6, 2022 · 4 comments
Labels
internals Internal machinery performance stale No activity over the last 90 days

Comments

@gmaze
Copy link
Member

gmaze commented Apr 6, 2022

At this point, caching is used at the Argo store lowest level, where data are directly fetched from any sources using fsspec methods.

But they are 2 other levels of the argopy data workflow that should be implementing cache because operations may be long:

  1. at the Argo store level, during the preprocessing step
  2. at the facade level, once data sources have post-processed data returned by Argo stores

This would significantly increase the cache size, and may be expiration time should be different (#196), but the performance gain could be worth it !

@gmaze gmaze added performance internals Internal machinery labels Apr 6, 2022
@github-actions
Copy link

github-actions bot commented Jul 5, 2022

This issue was marked as staled automatically because it has not seen any activity in 90 days

@github-actions github-actions bot added the stale No activity over the last 90 days label Jul 5, 2022
@gmaze gmaze removed the stale No activity over the last 90 days label Sep 23, 2022
@github-actions
Copy link

This issue was marked as staled automatically because it has not seen any activity in 90 days

@github-actions github-actions bot added the stale No activity over the last 90 days label Dec 22, 2022
Copy link

This issue was closed automatically because it has not seen any activity in 365 days

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Dec 23, 2023
@gmaze gmaze reopened this Jan 8, 2024
@github-actions github-actions bot removed closed-as-stale stale No activity over the last 90 days labels Jan 8, 2024
Copy link

This issue was marked as staled automatically because it has not seen any activity in 90 days

@github-actions github-actions bot added the stale No activity over the last 90 days label Apr 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internals Internal machinery performance stale No activity over the last 90 days
Projects
None yet
Development

No branches or pull requests

1 participant