Skip to content
This repository has been archived by the owner on Apr 6, 2023. It is now read-only.

feat: initialize suggestion providers at start up #193

Merged
merged 9 commits into from
Nov 9, 2021

Conversation

jrconlin
Copy link
Member

@jrconlin jrconlin commented Nov 1, 2021

Moves initialization of suggestion providers to startup and shares the
initialized data between threads.

Closes #127

Moves initialization of suggestion providers to startup and shares the
initialized data between threads.

Closes #127
@jrconlin
Copy link
Member Author

jrconlin commented Nov 1, 2021

TL:DR; moved the provider initialization to the async startup, outside of the app. Two tests are failing because the app isn't reading the remote settings URL because the data is already cached(?).

@mythmon
Copy link
Contributor

mythmon commented Nov 1, 2021

Two tests are failing because the app isn't reading the remote settings URL because the data is already cached(?).

I think this is spot on. We'll need a way to resync the data from the server. Maybe we could add a resync method to SuggestionProvider that the tests could call? It may be nice to expose that in a /debug endpoint later too, so it isn't entirely wasted on tests.

@jrconlin
Copy link
Member Author

jrconlin commented Nov 1, 2021

Yeah, mostly working out how the TestingTools are actually failing, but that might be a rabbit hole.

@mythmon mythmon changed the title Feat/1247 remote settings feat: initialize suggestion providers at start up Nov 2, 2021
@jrconlin jrconlin marked this pull request as ready for review November 3, 2021 14:39
@jrconlin jrconlin requested a review from a team as a code owner November 3, 2021 14:39

#![allow(unused_imports)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What import is unused, and if so why do we need it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a number, including:

crate::utils::test_tools::TestReqwestClient,
crate::TestingTools
anyhow::Result
merino_settings::providers::*
request::StatusCode
std::collections::*

As noted in the comment above, setting #![cfg(test)] causes the compile to fail because merino-integration-tests::utils::test_tools::merino_test cannot find crate::suggest, presumably because crate::suggest is not compiled as part of the non test build. So, instead, I enabled suggest, with the assumption that the remaining, non-test function setup_remote_settings_collection(...) is fine being part of the release build.

// note `#[cfg(test)]` will cause a compile error as the
// `merino_test` proc_macro will not be able to find this
// field.
pub test_changes: Option<Vec<String>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd think this should be nested under the Remote Settings configuration object, instead of at the top level, or at least mention that it is for Remote Settings. Also some documentation about what it is and how to do it would be warranted.

@@ -39,6 +39,7 @@ impl Proportion {
}
}

/// Define helper impls for float values
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heads up: I think this might conflict with some other PR, but I forget which one.

merino-integration-tests/src/suggest.rs Outdated Show resolved Hide resolved
merino-integration-tests/src/suggest.rs Outdated Show resolved Hide resolved
merino-web/src/endpoints/suggest.rs Outdated Show resolved Hide resolved
merino-web/src/endpoints/providers.rs Outdated Show resolved Hide resolved
Copy link
Member

@pjenvey pjenvey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missed that other clone on the SuggestionProviderRef that could be removed

@pjenvey pjenvey merged commit e1c2561 into main Nov 9, 2021
@pjenvey pjenvey deleted the feat/1247-remote_settings branch November 9, 2021 20:54
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Set up the remote settings suggestion provider at server start
3 participants