Add Brave Search beta to the list of available search engines

[bsclifton]

Test plan

(for iOS, see brave/brave-ios#3745)

@brave/legacy_qa please also see brave/brave-core#8966 (comment) re: test cases for Android/Desktop.

Onboarding experience (Desktop)

1. Fresh profile
2. Upon launching Brave, welcome screen will show
3. Navigate to fourth screen (Set default search engine)
   a. (US and Canada) Brave should show as an option in the dropdown
   b. (not US or Canada) Brave should not be in the dropdown

Onboarding experience (Android)

1. Fresh profile
2. Upon launching Brave, onboarding will show
3. Confirm that:
   a. (US and Canada) Brave Search beta should show as the 2nd search option
   b. (not US or Canada) Brave should not show as an option won't be shown in this list

Search engines screen (all platforms)

1. Fresh profile
2. Launch Brave and go to settings
3. Under settings, pick Search engines
4. In settings pick Manage search engines
5. Brave should show under Default search engines area. Go ahead and choose it as default
6. Do a search in omnibox and verify search is going to search.brave.com SERP

Autocomplete suggestions test (all platforms)

1. Fresh profile
2. Launch Brave and go to settings
3. Under settings, pick Privacy and security
4. Set the toggle for Autocomplete searches and URLs to true
5. Perform steps above (Search engines screen (all platforms)) to set Brave search as default
6. Type in the omnibox. Try something like how do i and verify autocomplete results are being shown

Description

We recently announced having a search (https://brave.com/brave-search/) and this issue is intended to capture adding this to the browser as an alternate search. It would be in the Default search engines list in brave://settings/searchEngines

Because this search is currently limited (https://brave.com/search/ is currently waitlist'ing folks; team is in Beta phase), we should put this behind a flag in brave://flags. When disabled (default setting) it wouldn't show search in Default search engines list. If true (user needs to set manually) then it will.

When we're ready to expand search to a larger audience we can change the value of this using Griffin or just remove the flag

[kjozwiak]

@brave/legacy_qa I've also added a bunch of more test cases that we ran via iOS that should also be checked via Android & Desktop. Please find them via brave/brave-core#8966 (comment). Combination of what @bsclifton provided via #15663 (comment) and what was checked via iOS.

Also important note, re: Default Callback API, we can now use search.brave.com rather than search-dev.brave.com.

[stephendonner]

Verified PASSED using

Brave	1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
Revision	4bb19460e8d88c3446b360b0df8fd991fee49c0b-refs/branch-heads/4472@{#1496}
OS	macOS Version 11.4 (Build 20F71)

Onboarding/Setting Test Cases

• ensured that Brave Search beta is only displayed under brave://welcome onboarding for Canada & USA
  • ensured that Japan, German & France weren't displaying Brave Search beta via search onboarding
• ensured that Brave Search beta appears via brave://settings/search & brave://settings/searchEngines for all regions
• ensured that the query URL appears as https://search.brave.com/search?q=%s&source=desktop under brave://settings/searchEngines

Japan onboarding	Germany onboarding	France onboarding	USA & Canada onboarding

Japan SE settings	Germany SE settings	France SE settings	USA & Canada settings

URL Query Test Cases

• ensured that the correct query URL is being used when Brave Search beta is the default search engine:

  • omnibox --> https://search.brave.com/search?q=brave&source=desktop
  • https://search.brave.com --> https://search.brave.com/search?q=basic+attention+token

Default Callback Test Cases

Test Case 1: (Setting as default via page)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• click on the Set default button via the default modal within https://search-dev.brave.com
• you should see the button turn from Set default --> Done and disappear
• check and ensure https://search-dev.brave.com is set as the default via brave://settings/search

example	example	example

Test Case 2: (Private Browsing)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• launch open * login into https://search-dev.brave.com via Private Browsing
• search for Basic Attention Token via the text field
• shouldn't see any Brave Search Default modals

Test Case 3: (Default modal retries per session)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• the default modal should appear
• refresh the page two times (should see the default modal appear
• refresh again and you shouldn't see the default modal appear for this session
• open a new tab, open https://search-dev.brave.com and try another search (shouldn't see any more modals for this session)

As per the above, this checks to make sure that the Default modal only appears three times per session.

Test Case 4: (Maximum 10 retries)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• the default modal should appear
• refresh the page two times (should see the default modal appear
• refresh again and you shouldn't see the default modal appear for this session
• open a new tab, open https://search-dev.brave.com and try another search (shouldn't see any more modals for this session)
• close Brave and move the time forward by 24hrs

Repeat the above. The basic logic for the above is that on desktop/android, we should only see 3 modals per session per day to a maximum of 10 in a span of four days.

Test Case 5: (Setting Brave Search as default via onboarding)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• Set Brave Search beta as the default SE via brave://welcome
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• ensure that the default modal isn't being displayed

Test Case 6: (Setting Brave Search as default via brave://settings/search)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• Set Brave Search beta as the default SE via brave://settings/search
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• ensure that the default modal isn't being displayed

Fallback Test Cases

• ensured that the Enable Google Fallback mixing appears when searching for terms like brave 123 456 789
• ensured that tapping on Dismiss correctly closes the Enable Google Fallback mixing card
  • ensured that it re-appears when another term that needs fallback is used
• ensured that Learn more. opens https://search.brave.com/help/google-fallback
• ensured that https://www.google.com isn't being contact when fallback is disabled
• ensured that https://www.google.com is being contact once fallback has been enabled

example	example	example

---

Verified PASSED using

Brave	1.26.65 Chromium: 91.0.4472.114 (Official Build) (64-bit)
Revision	4bb19460e8d88c3446b360b0df8fd991fee49c0b-refs/branch-heads/4472@{#1496}
OS	Windows 10 OS Version 2009 (Build 21390.2025)

Onboarding/Setting Test Cases

• ensured that Brave Search beta is only displayed under brave://welcome onboarding for Canada & USA
  • ensured that Japan, German & France weren't displaying Brave Search beta via search onboarding
• ensured that Brave Search beta appears via brave://settings/search & brave://settings/searchEngines for all regions
• ensured that the query URL appears as https://search.brave.com/search?q=%s&source=desktop under brave://settings/searchEngines

Japan onboarding	Germany onboarding	France onboarding	USA & Canada onboarding

Japan SE settings	Germany SE settings	France SE settings	USA & Canada settings

URL Query Test Cases

• ensured that the correct query URL is being used when Brave Search beta is the default search engine:

  • omnibox --> https://search.brave.com/search?q=brave&source=desktop
  • https://search.brave.com --> https://search.brave.com/search?q=basic+attention+token
    

Default Callback Test Cases

Test Case 1: (Setting as default via page)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• click on the Set default button via the default modal within https://search-dev.brave.com
• you should see the button turn from Set default --> Done and disappear
• check and ensure https://search-dev.brave.com is set as the default via brave://settings/search

example	example	example

Test Case 2: (Private Browsing)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• launch open * login into https://search-dev.brave.com via Private Browsing
• search for Basic Attention Token via the text field
• shouldn't see any Brave Search Default modals

Test Case 3: (Default modal retries per session)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• the default modal should appear
• refresh the page two times (should see the default modal appear
• refresh again and you shouldn't see the default modal appear for this session
• open a new tab, open https://search-dev.brave.com and try another search (shouldn't see any more modals for this session)

As per the above, this checks to make sure that the Default modal only appears three times per session.

Test Case 4: (Maximum 10 retries)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• the default modal should appear
• refresh the page two times (should see the default modal appear
• refresh again and you shouldn't see the default modal appear for this session
• open a new tab, open https://search-dev.brave.com and try another search (shouldn't see any more modals for this session)
• close Brave and move the time forward by 24hrs

Repeat the above. The basic logic for the above is that on desktop/android, we should only see 3 modals per session per day to a maximum of 10 in a span of four days.

Test Case 5: (Setting Brave Search as default via onboarding)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• Set Brave Search beta as the default SE via brave://welcome
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• ensure that the default modal isn't being displayed

Test Case 6: (Setting Brave Search as default via brave://settings/search)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• Set Brave Search beta as the default SE via brave://settings/search
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• ensure that the default modal isn't being displayed

Fallback Test Cases

• ensured that the Enable Google Fallback mixing appears when searching for terms like brave 123 456 789
• ensured that tapping on Dismiss correctly closes the Enable Google Fallback mixing card
  • ensured that it re-appears when another term that needs fallback is used
• ensured that Learn more. opens https://search.brave.com/help/google-fallback
• ensured that https://www.google.com isn't being contact when fallback is disabled
• ensured that https://www.google.com is being contact once fallback has been enabled

example	example

---

Verification PASSED

Brave | 1.26.67 Chromium: 91.0.4472.114 (Official Build) (64-bit)
--- | ---
Revision | 4bb19460e8d88c3446b360b0df8fd991fee49c0b-refs/branch-heads/4472@{#1496}
OS | Linux

• Verified the test plan from Add Brave Search as an alternative search brave-core#8966 (comment)

Onboarding/Setting Test Cases

• ensured that Brave Search beta is only displayed under brave://welcome onboarding for Canada & USA
  • ensured that Japan, German & France weren't displaying Brave Search beta via search onboarding
• ensured that Brave Search beta appears via brave://settings/search & brave://settings/searchEngines for all regions
• ensured that the query URL appears as https://search.brave.com/search?q=%s&source=desktop under brave://settings/searchEngines

Japan onboarding	Germany onboarding	France onboarding	USA & Canada onboarding

Japan SE settings	Germany SE settings	France SE settings	USA & Canada settings

URL Query Test Cases

• ensured that the correct query URL is being used when Brave Search beta is the default search engine:

  • omnibox --> https://search.brave.com/search?q=brave&source=desktop
  • https://search.brave.com --> https://search.brave.com/search?q=basic+attention+token
    

Default Callback Test Cases

Test Case 1: (Setting as default via page)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• click on the Set default button via the default modal within https://search-dev.brave.com
• you should see the button turn from Set default --> Done and disappear
• check and ensure https://search-dev.brave.com is set as the default via brave://settings/search

example	example	example

Test Case 2: (Private Browsing)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• launch open * login into https://search-dev.brave.com via Private Browsing
• search for Brave browser via the text field
• shouldn't see any Brave Search Default modals

Test Case 3: (Default modal retries per session)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• the default modal should appear
• refresh the page two times (should see the default modal appear
• refresh again and you shouldn't see the default modal appear for this session
• open a new tab, open https://search-dev.brave.com and try another search (shouldn't see any more modals for this session)

As per the above, this checks to make sure that the Default modal only appears three times per session.

Test Case 4: (Maximum 10 retries)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• don't run through the search onboarding and leave Google as the default browser
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• the default modal should appear
• refresh the page two times (should see the default modal appear
• refresh again and you shouldn't see the default modal appear for this session
• open a new tab, open https://search-dev.brave.com and try another search (shouldn't see any more modals for this session)
• close Brave and move the time forward by 24hrs

Repeat the above. The basic logic for the above is that on desktop/android, we should only see 3 modals per session per day to a maximum of 10 in a span of four days.

Test Case 5: (Setting Brave Search as default via onboarding)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• Set Brave Search beta as the default SE via brave://welcome
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• ensure that the default modal isn't being displayed
  

Test Case 6: (Setting Brave Search as default via brave://settings/search)

• install 1.26.65 Chromium: 91.0.4472.114 (Official Build) (x86_64)
• Set Brave Search beta as the default SE via brave://settings/search
• login into https://search-dev.brave.com
• search for Brave Browser via the text field
• ensure that the default modal isn't being displayed

Fallback Test Cases

• ensured that the Enable Google Fallback mixing appears when searching for terms like brave 123 456 789
• ensured that tapping on Dismiss correctly closes the Enable Google Fallback mixing card
  • ensured that it re-appears when another term that needs fallback is used
• ensured that Learn more. opens https://search.brave.com/help/google-fallback
• ensured that https://www.google.com isn't being contact when fallback is disabled
• ensured that https://www.google.com is being contact once fallback has been enabled

example	example	example

[srirambv]

Verification passed on OnePlus 6T with Android 10 running 1.26.65 x64 build & 1.26.67 x64 build

On-boarding

• Verified Brave Search beta is on shown in SE onboarding screen only for US and Canada
• Verified Brave Search beta is shown second on the list during onboarding
• Verified Brave Search beta is not listed for other locales for SE onboarding
• Verified selecting Brave Search beta during onboarding sets Brave search as default for both normal and private tabs

Search Engine Setting

• Verified Brave Search beta is shown second on the list in search engine selection screen
• Verified selecting Brave Search beta from the SE list sets it as default
• Verified able to set Brave Search beta as default for both normal and private tabs from search engine selection screen
• Verified upgrading from 1.25.x to 1.26.65 adds Brave Search beta to the list at the bottom

URL Tests

• Verified setting Brave Search beta as default and doing a search from omnibox adds &source=android to the query URL
  Ex: https://search.brave.com/?q=test&source=android
• Verified performing a search on the search page doesn't append any &source=android to the query URL

Default Callback

Case 1: via Search page

• Install 1.26.65
• Select URL bar to trigger SE onboarding
• Select continue so Google is set as default for both normal and private tab
• Visit https://search.brave.com and perform a search
• Click on Set as defaultand wait for the modal to change to Done and then auto-dismiss
• Verified Brave Search beta is set as default for normal mode only and not for private mode

Case 2: via Search Page on Private tab

• Install 1.26.65
• Select URL bar to trigger SE onboarding
• Select continue so Google is set as default for both normal and private tab
• Visit https://search.brave.com and perform a search in private tab
• Verified no Set as default modal is shown on SERP
• Verified search engines settings remain same as before in settings

Case 3: Default modal retries

• Install 1.26.65
• Select URL bar to trigger SE onboarding
• Select continue so Google is set as default for both normal and private tab
• Visit https://search.brave.com and perform a search
• Set as default modal should be shown
• Reload the page two times, modal should still be shown
• Reload the page a third time, modal should be removed
• Open a new tab and perform a search on search.brave.com, no default modal is shown
• Clear browsing data and try again, still no default modal is shown

Case 4: Default modal retries (max 10)

• Install 1.26.65
• Select URL bar to trigger SE onboarding
• Select continue so Google is set as default for both normal and private tab
• Visit https://search.brave.com and perform a search
• Set as default modal should be shown
• Reload the page two times, modal should still be shown
• Reload the page a third time, modal should be removed
• Open a new tab and perform a search on search.brave.com, no default modal is shown
• Close browser and move device time by 24hr
• Search again on search.brave.com, default modal should be shown

Fallback Test Cases

• Verified Enable Google Fallback mixing shows up when searching for term like brave 123 456
• Verified tapping on Dismiss hides the fallback modal
• Verified modal reappears every time a weird search term is used like above
• Verified Learn more. opens https://search.brave.com/help/google-fallback
• Verified https://www.google.com isn't being contact when fallback is disabled
• Verified https://www.google.com is being contact once fallback has been enabled

image

---

Verification passed on OnePlus 6T with Android 10 running 1.26.65 x64 build & 1.26.67 x64 build

On-boarding

• Verified Brave Search beta is on shown in SE onboarding screen only for US and Canada
• Verified Brave Search beta is shown second on the list during onboarding
• Verified Brave Search beta is not listed for other locales for SE onboarding
• Verified selecting Brave Search beta during onboarding sets Brave search as default for both normal and private tabs

Search Engine Setting

• Verified Brave Search beta is shown second on the list in search engine selection screen
• Verified selecting Brave Search beta from the SE list sets it as default
• Verified able to set Brave Search beta as default for both normal and private tabs from search engine selection screen
• Verified upgrading from 1.25.x to 1.26.65 adds Brave Search beta to the list at the bottom

URL Tests

• Verified setting Brave Search beta as default and doing a search from omnibox adds &source=android to the query URL
  Ex: https://search.brave.com/?q=test&source=android
• Verified performing a search on the search page doesn't append any &source=android to the query URL

Default Callback

Case 1: via Search page

• Install 1.26.65
• Select URL bar to trigger SE onboarding
• Select continue so Google is set as default for both normal and private tab
• Visit https://search.brave.com and perform a search
• Click on Set as defaultand wait for the modal to change to Done and then auto-dismiss
• Verified Brave Search beta is set as default for normal mode only and not for private mode

Case 2: via Search Page on Private tab

• Install 1.26.65
• Select URL bar to trigger SE onboarding
• Select continue so Google is set as default for both normal and private tab
• Visit https://search.brave.com and perform a search in private tab
• Verified no Set as default modal is shown on SERP
• Verified search engines settings remain same as before in settings

Case 3: Default modal retries

• Install 1.26.65
• Select URL bar to trigger SE onboarding
• Select continue so Google is set as default for both normal and private tab
• Visit https://search.brave.com and perform a search
• Set as default modal should be shown
• Reload the page two times, modal should still be shown
• Reload the page a third time, modal should be removed
• Open a new tab and perform a search on search.brave.com, no default modal is shown
• Clear browsing data and try again, still no default modal is shown

Case 4: Default modal retries (max 10)

• Install 1.26.65
• Select URL bar to trigger SE onboarding
• Select continue so Google is set as default for both normal and private tab
• Visit https://search.brave.com and perform a search
• Set as default modal should be shown
• Reload the page two times, modal should still be shown
• Reload the page a third time, modal should be removed
• Open a new tab and perform a search on search.brave.com, no default modal is shown
• Close browser and move device time by 24hr
• Search again on search.brave.com, default modal should be shown

Fallback Test Cases

• Verified Enable Google Fallback mixing shows up when searching for term like brave 123 456
• Verified tapping on Dismiss hides the fallback modal
• Verified modal reappears every time a weird search term is used like above
• Verified Learn more. opens https://search.brave.com/help/google-fallback
• Verified https://www.google.com isn't being contact when fallback is disabled
• Verified https://www.google.com is being contact once fallback has been enabled