Add a system to refresh all cached products for Nutri-Score v2

[teolemon]

What

• Work is going on to implement the new formula of the Nutri-Score, which will become the official one later this year.
• feat: New Nutri-Score formula (v2) for food (2022) and beverages (2023) - ready for review openfoodfacts-server#8832
• There will be two fields with the old and new formula.

Tasks

Beta Give feedback

1. We need to make it possible for the app to switch to the new value
2. Update of all the local products from the server #5099
   data refresh local cache +2
3. We should then display an onboarding about the change, in phase with the official Nutri-Score messaging (not yet available)

Part of

• 🎯 What can I work on ? #525

[monsieurtanuki]

About the refresh of local products:

1. that could be a specific but reusable background task ("refresh all local products")
2. it would be triggered at app init time, with a flag in the preferences so that we call it only once
3. in a best world scenario the server would provide a URL that returns the current version of the score algorithms, which we would compare to a locally stored value
4. in a best world scenario the server would also provide some html text, to be displayed on the specific "onboarding" page
5. with all that in place we could manage additional nutriscore changes without any impact on the Smoothie code.

Beyond that is the problem of using 2 nutriscores for the same product.
If the "new" nutriscore has the same tag as now, that's ok, and there could be an additional nutriscore_previous field for whoever wants to see the evolution.

[monsieurtanuki]

The good thing is that we already have a background task that refreshes all the local products: BackgroundTaskFullRefresh.
Currently we use it only in dev mode, when the user specifically asks us to refresh all the local products.

@teolemon @stephanegigandet Would the server be ready to provide us with a URL like off.org/latest-algorithm.pl?cc=fr&lc=fr, returning

• a unique tag about the current score computation algorithm (e.g. '2017-nutri')
• a localized explanation of the current score evolutions (blablabla olive oil...)

[stephanegigandet]

We should probably weight how much complexity we add versus the benefits we get. Having an old Nutri-Score in the local cache may not be a big problem, especially if we refresh those items when they actually get displayed to the user.

We could just consider that the local cache is just a cache, that it can be outdated (it's already the case for any other edit to the product page), and that it's ok.

Under what conditions / actions do we use the local cache? If I have network connectivity, do we just always get fresh content anyway?

[monsieurtanuki]

@stephanegigandet Actually we use the local cache (local SQLite database) as the main source. Instant results.
We don't refresh automatically.
The user needs to explicitly refresh (e.g. swipe down gesture), or after a successful local edit the product will be automatically refreshed with the latest server data.

What we could do do implement a lazy refresh, is to async'ly download the product each time we go to the product page - while still displaying the local cache version. After the product is successfully re-downloaded, the displayed data will be refreshed.
How often should we do it? In which pages? Those are just details.

[teolemon]

I think whenever we reopen the product page, we should attempt to refresh, and flash the content whenever that happens (possibly with an animated transition)

[monsieurtanuki]

That wouldn't be a problem: we already have a BackgroundTaskRefreshLater that downloads the product 10 minutes later (cf. Robotoff). We can reuse it and set a 0 second delay instead of 10 minutes.

I'm a bit worried about how to prevent cluttering, as we use a single queue for our background tasks. And about low connectivity context.

Perhaps we could add this in dev mode only first, in order to monitor it.

[M123-dev]

Keep in mind I am not up to date with the current background manager implementation, but we could add a kind of priority system, so that things like this are always executed first.

+ Other use: when a query fails (image too small) we could downgrade its importance so that other working queries are not blocked forever

[monsieurtanuki]

Keep in mind I am not up to date with the current background manager implementation, but we could add a kind of priority system, so that things like this are always executed first.

Given how the background manager works and the pain in the neck it would be to amend it (assuming that the concept of priority would be helpful), a moderately easy way to implement a priority could be to use 2 managers, one for fast queries (like product refresh) and one for slower queries (like image upload).
Or to just play it lazy with a simple async refresh called without await in initState.
But anyway, with slow connection we're doomed.

Other use: when a query fails (image too small) we could downgrade its importance so that other working queries are not blocked forever

• the other working queries are never blocked
• the "image too small" will fail forever even with a low priority, and that causes discrepancies between the local data and the server data (cf. related issues)
  => I would be more than happy to hear suggestions about how to deal UX-wise with obvious failed queries (image too small or wrong password) and the way to warn the user.

[monsieurtanuki]

cf. #5016 (comment)

For refresh, the target is refresh everything to Nutriscore v2, with the ability to test it for privileged users before going public.

We can easily have all the products refreshed: we would just need to clear all local products "download language" field, and start a fake "app language change background task".
@teolemon Would you like me to add a dev mode action about it?

Regarding "normal" users, the trick would be to check a one-shot preference value at init time (e.g. "if 0 then set to 1 and refresh all products"). Possibly with a date check "only if date is after April 1st 2024".

[M123-dev]

It has been a while, I guess the nutriscore v2 was rolled out already. Is this still a concern @teolemon