Incorporate "debouncing" lists as part of top-level domain blocking #15090

pes10k · 2021-04-02T16:30:00Z

Follow up / related to top level domain blocking: #14134

For some of the bounce tracking cases top level domain blocking is trying to guard against, we can solve them without bothering users by automatically following the bounce to URLs for them. For example, common pattern:

User clicks link, intending to go to destination.org
They are instead taken to an intermediate bounce-tracking URL like track.org?id=<click_id>&dest=destination.org
track.org records the click event and then forwards to destination.org

@fmarier and @diracdeltas have found several lists that identify the above cases and allow for them to be automatically skipped.

It would be neat to incorporate these lists into (or along side) the top-level domain blocking feature, so that we could warp users past the well-know trackers, while having domain blocking still in place to protect against the more sophisticated or less well known cases.

fmarier · 2021-04-07T01:29:12Z

Here are the three lists I mentioned on https://github.com/brave/security/issues/316#issuecomment-812278055:

Dryader · 2021-04-07T19:05:56Z

Hi there. Just out of curiosity, I've seen this extension being recommended, would that incorporate the same features you guys are planning to implement into brave? (Not extremely knowledgeable on the topic just curious.) https://github.com/ClearURLs/Addon

fmarier · 2021-04-07T20:59:02Z

Thanks for that @Dryader . There are indeed some more usable rules in that extension: https://github.com/ClearURLs/Rules/blob/master/data.json (see redirections)

e.g. Reddit, Facebook and Disqus.

fmarier · 2021-05-05T17:02:53Z

@antonok-edm found another one: https://github.com/Sainan/Universal-Bypass/blob/master/rules.json

kjozwiak · 2022-01-20T02:37:03Z

@brave/legacy_qa the above changes are already live as we enabled debouncing via Griffin. I went through verifications via brave/brave-variations#195 (comment) on Nightly but it's worth going through a quick check to make sure it's working on the release channel.

stephendonner · 2022-01-20T22:21:03Z

Verified PASSED using

Brave	1.34.81 Chromium: 97.0.4692.99 (Official Build) (x86_64)
Revision	d740da257583289dbebd2eb37e8668928fac5ead-refs/branch-heads/4692@{#1461}
OS	macOS Version 11.6.1 (Build 20G224)

Followed the verification steps from brave/brave-variations#195 (comment).

Enabled

Launched Brave using --enable-logging=stderr --v=1 --variations-server-url=https://variations.bravesoftware.com/seed and verified the following:

ensured that Brave Local Data Updater > Brave Local Data Updater - Version: 1.0.45 which was the version that wasn't working when initially running through the above. Most recent version appears to be Brave Local Data Updater - Version: 1.0.52 which has the latest debounce list
ensured that BraveDebounceStudy:Enabled under brave://version (after restarting twice due to a Griffin limitation)

example	example

Went through the cases under https://dev-pages.brave.software/navigation-tracking/debouncing.html and ensured the following:

YT

ensured that clicking on Run returned to the test page and Observed Referrer --> dev-pages.brave.software

[7296:259:0120/140556.291001:VERBOSE1:debounce_throttle.cc(51)] SiteForCookies: {site=https://youtube.com; schemefully_same=true}
[7296:259:0120/140556.291095:VERBOSE1:debounce_throttle.cc(60)] Debouncing rule applied: https://www.youtube.com/redirect?q=https%3A%2F%2Fdev-pages.brave.software%2Fnavigation-tracking%2Fdebouncing.html%3Fcomplete -> https://dev-pages.brave.software/navigation-tracking/debouncing.html?complete
[7296:259:0120/140556.291165:VERBOSE1:debounce_throttle.cc(76)] SiteForCookies: {site=https://brave.software; schemefully_same=true}

Reddit

ensured that clicking on Run returned to the test page and Observed Referrer --> dev-pages.brave.software

[7296:259:0120/140620.161242:VERBOSE1:debounce_throttle.cc(51)] SiteForCookies: {site=https://reddit.com; schemefully_same=true}
[7296:259:0120/140620.161318:VERBOSE1:debounce_throttle.cc(60)] Debouncing rule applied: https://out.reddit.com/test?url=https%3A%2F%2Fdev-pages.brave.software%2Fnavigation-tracking%2Fdebouncing.html%3Fcomplete -> https://dev-pages.brave.software/navigation-tracking/debouncing.html?complete
[7296:259:0120/140620.161365:VERBOSE1:debounce_throttle.cc(76)] SiteForCookies: {site=https://brave.software; schemefully_same=true}

Disabled

Launched Brave using --enable-logging=stderr --v=1 --variations-server-url=https://variations.bravesoftware.com/seed and ensured the following:

YT

ensured that clicking on Run didn't return to the test page and Observed Referrer --> https://www.youtube.com when clicking on GO TO SITE
went through the logs and ensured that Debouncing rule applied wasn't visible as no rules are being applied

Reddit

ensured that clicking on Run didn't return to the test page (no debouncing occurring)
went through the logs and ensured that Debouncing rule applied wasn't visible as no rules are being applied

kjozwiak · 2022-01-20T22:56:37Z

Labelled as QA/Blocked till we get brave/brave-variations#195 pushed into production via https://github.com/brave/brave-variations/blob/production/seed/seed.json.

kjozwiak · 2022-01-21T03:57:48Z

Removing QA/Blocked as brave/brave-variations#212 was uplifted into production.

stephendonner · 2022-01-21T06:19:40Z

Verified PASSED using

Brave	1.34.81 Chromium: 97.0.4692.99 (Official Build) (x86_64)
Revision	d740da257583289dbebd2eb37e8668928fac5ead-refs/branch-heads/4692@{#1461}
OS	macOS Version 11.6.1 (Build 20G224)

Followed the verification steps from brave/brave-variations#195 (comment).

Enabled - the default now for PRODUCTION

Launched Brave using only --enable-logging=stderr --v=1 and ensured the following::

ensured that Brave Local Data Updater > Brave Local Data Updater - Version: 1.0.45 which was the version that wasn't working when initially running through the above. Most recent version appears to be Brave Local Data Updater - Version: 1.0.53 which has the latest debounce list
ensured that BraveDebounceStudy:Enabled under brave://version (after restarting twice due to a Griffin limitation)

`brave://components`	`brave://version

Went through the cases under https://dev-pages.brave.software/navigation-tracking/debouncing.html and ensured the following:

YT

ensured that clicking on Run returned to the test page and Observed Referrer --> dev-pages.brave.software

[7265:259:0120/221416.456714:VERBOSE1:debounce_throttle.cc(51)] SiteForCookies: {site=https://youtube.com; schemefully_same=true}
[7265:259:0120/221416.456771:VERBOSE1:debounce_throttle.cc(60)] Debouncing rule applied: https://www.youtube.com/redirect?q=https%3A%2F%2Fdev-pages.brave.software%2Fnavigation-tracking%2Fdebouncing.html%3Fcomplete -> https://dev-pages.brave.software/navigation-tracking/debouncing.html?complete
[7265:259:0120/221416.456810:VERBOSE1:debounce_throttle.cc(76)] SiteForCookies: {site=https://brave.software; schemefully_same=true}

Reddit

ensured that clicking on Run returned to the test page and Observed Referrer --> dev-pages.brave.software

[7265:259:0120/221524.353302:VERBOSE1:debounce_throttle.cc(51)] SiteForCookies: {site=https://reddit.com; schemefully_same=true}
[7265:259:0120/221524.353357:VERBOSE1:debounce_throttle.cc(60)] Debouncing rule applied: https://out.reddit.com/test?url=https%3A%2F%2Fdev-pages.brave.software%2Fnavigation-tracking%2Fdebouncing.html%3Fcomplete -> https://dev-pages.brave.software/navigation-tracking/debouncing.html?complete
[7265:259:0120/221524.353395:VERBOSE1:debounce_throttle.cc(76)] SiteForCookies: {site=https://brave.software; schemefully_same=true}

Negative-Test

launched with --v=1 --variations-server-url=https://no-thanks.invalid --enable-logging=stderr and confirmed I didn't get the Griffin seed for enabling debouncing 👍

Verification passed on

Brave	1.34.81 Chromium: 97.0.4692.99 (Official Build) (64-bit)
Revision	adefa7837d02a07a604c1e6eff0b3a09422ab88d-refs/branch-heads/4692@{#1247}
OS	Ubuntu 18.04 LTS

Followed the verification steps from brave/brave-variations#195 (comment).

Enabled - the default now for PRODUCTION

Launched Brave using only --enable-logging=stderr --v=1 and ensured the following::

ensured that Brave Local Data Updater > Brave Local Data Updater - Version: 1.0.45 which was the version that wasn't working when initially running through the above. Most recent version appears to be Brave Local Data Updater - Version: 1.0.53 which has the latest debounce list
ensured that BraveDebounceStudy:Enabled under brave://version (after restarting twice due to a Griffin limitation)

`brave://components`	`brave://version

Went through the cases under https://dev-pages.brave.software/navigation-tracking/debouncing.html and ensured the following:

YT

ensured that clicking on Run returned to the test page and Observed Referrer --> dev-pages.brave.software

[19570:19570:0121/163258.964315:VERBOSE1:debounce_throttle.cc(51)] SiteForCookies: {site=https://youtube.com; schemefully_same=true}
[19570:19570:0121/163258.964412:VERBOSE1:debounce_throttle.cc(60)] Debouncing rule applied: https://www.youtube.com/redirect?q=https%3A%2F%2Fdev-pages.brave.software%2Fnavigation-tracking%2Fdebouncing.html%3Fcomplete -> https://dev-pages.brave.software/navigation-tracking/debouncing.html?complete
[19570:19570:0121/163258.964473:VERBOSE1:debounce_throttle.cc(76)] SiteForCookies: {site=https://brave.software; schemefully_same=true}

Reddit

ensured that clicking on Run returned to the test page and Observed Referrer --> dev-pages.brave.software

[19570:19570:0121/163505.341093:VERBOSE1:debounce_throttle.cc(51)] SiteForCookies: {site=https://reddit.com; schemefully_same=true}
[19570:19570:0121/163505.341324:VERBOSE1:debounce_throttle.cc(60)] Debouncing rule applied: https://out.reddit.com/test?url=https%3A%2F%2Fdev-pages.brave.software%2Fnavigation-tracking%2Fdebouncing.html%3Fcomplete -> https://dev-pages.brave.software/navigation-tracking/debouncing.html?complete
[19570:19570:0121/163505.341504:VERBOSE1:debounce_throttle.cc(76)] SiteForCookies: {site=https://brave.software; schemefully_same=true}

Negative-Test

launched with --v=1 --variations-server-url=https://no-thanks.invalid --enable-logging=stderr and confirmed I didn't get the Griffin seed for enabling debouncing

Verification PASSED on


Brave | 1.34.81 Chromium: 97.0.4692.99 (Official Build) (64-bit)
-- | --
Revision | d740da257583289dbebd2eb37e8668928fac5ead-refs/branch-heads/4692@{#1461}
OS | Windows 10 Version 21H2 (Build 19044.1466)

ensured that Brave Local Data Updater > Brave Local Data Updater - Version: 1.0.45 which was the version that wasn't working when initially running through the above. Most recent version appears to be Brave Local Data Updater - Version: 1.0.53 which has the latest debounce list
ensured that BraveDebounceStudy:Enabled under brave://version (after restarting twice due to a Griffin limitation)

`brave://components`	`brave://version

Went through the cases under https://dev-pages.brave.software/navigation-tracking/debouncing.html and ensured the following:

YT

ensured that clicking on Run returned to the test page and Observed Referrer --> dev-pages.brave.software

[20064:20092:0121/212529.341:VERBOSE1:debounce_throttle.cc(51)] SiteForCookies: {site=https://youtube.com; schemefully_same=true}
[20064:20092:0121/212529.341:VERBOSE1:debounce_throttle.cc(60)] Debouncing rule applied: https://www.youtube.com/redirect?q=https%3A%2F%2Fdev-pages.brave.software%2Fnavigation-tracking%2Fdebouncing.html%3Fcomplete -> https://dev-pages.brave.software/navigation-tracking/debouncing.html?complete
[20064:20092:0121/212529.341:VERBOSE1:debounce_throttle.cc(76)] SiteForCookies: {site=https://brave.software; schemefully_same=true}

Reddit

ensured that clicking on Run returned to the test page and Observed Referrer --> dev-pages.brave.software

[15720:14316:0121/211843.873:VERBOSE1:debounce_throttle.cc(51)] SiteForCookies: {site=https://reddit.com; schemefully_same=true}
[15720:14316:0121/211843.874:VERBOSE1:debounce_throttle.cc(60)] Debouncing rule applied: https://out.reddit.com/test?url=https%3A%2F%2Fdev-pages.brave.software%2Fnavigation-tracking%2Fdebouncing.html%3Fcomplete -> https://dev-pages.brave.software/navigation-tracking/debouncing.html?complete
[15720:14316:0121/211843.874:VERBOSE1:debounce_throttle.cc(76)] SiteForCookies: {site=https://brave.software; schemefully_same=true}

pes10k added privacy privacy/feature User-facing privacy- & security-focused feature work. OS/Android Fixes related to Android browser functionality privacy-pod Feature work for the Privacy & Web Compatibility pod OS/Desktop labels Apr 2, 2021

pes10k assigned antonok-edm and pilgrim-brave Apr 2, 2021

pes10k mentioned this issue May 18, 2021

First-party domain blocking #14134

Closed

pilgrim-brave mentioned this issue Jun 9, 2021

Implement debouncing brave/brave-core#9065

Merged

24 tasks

fmarier mentioned this issue Aug 17, 2021

Disintermediate tracking widgets #817

Closed

pilgrim-brave closed this as completed in brave/brave-core#9065 Sep 10, 2021

pilgrim-brave mentioned this issue Sep 23, 2021

Enable debounce at 5 percent on nightly brave/brave-variations#125

Merged

goodov mentioned this issue Jan 11, 2022

Redirect AMP pages to their canonical links brave/brave-core#11750

Merged

37 tasks

kjozwiak added QA/Yes release-notes/exclude labels Jan 20, 2022

kjozwiak added this to the 1.34.x - Release #2 milestone Jan 20, 2022

kjozwiak added the QA/Test-All-Platforms label Jan 20, 2022

stephendonner added the QA Pass-macOS label Jan 20, 2022

kjozwiak added QA/Blocked and removed QA Pass-macOS labels Jan 20, 2022

kjozwiak removed the QA/Blocked label Jan 21, 2022

stephendonner added the QA Pass-macOS label Jan 21, 2022

btlechowski added the QA Pass-Linux label Jan 21, 2022

GeetaSarvadnya added the QA Pass-Win64 label Jan 21, 2022

fmarier mentioned this issue May 28, 2022

Audit all debouncing rules brave/adblock-lists#871

Closed

ShivanKaul mentioned this issue Jun 17, 2022

Add ability to debounce URLs based on a regex applied to the path #23121

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Incorporate "debouncing" lists as part of top-level domain blocking #15090

Incorporate "debouncing" lists as part of top-level domain blocking #15090

pes10k commented Apr 2, 2021 •

edited

Loading

fmarier commented Apr 7, 2021

Dryader commented Apr 7, 2021 •

edited by fmarier

Loading

fmarier commented Apr 7, 2021 •

edited

Loading

fmarier commented May 5, 2021

kjozwiak commented Jan 20, 2022

stephendonner commented Jan 20, 2022

kjozwiak commented Jan 20, 2022

kjozwiak commented Jan 21, 2022

stephendonner commented Jan 21, 2022 •

edited by GeetaSarvadnya

Loading

Incorporate "debouncing" lists as part of top-level domain blocking #15090

Incorporate "debouncing" lists as part of top-level domain blocking #15090

Comments

pes10k commented Apr 2, 2021 • edited Loading

fmarier commented Apr 7, 2021

Dryader commented Apr 7, 2021 • edited by fmarier Loading

fmarier commented Apr 7, 2021 • edited Loading

fmarier commented May 5, 2021

kjozwiak commented Jan 20, 2022

stephendonner commented Jan 20, 2022

kjozwiak commented Jan 20, 2022

kjozwiak commented Jan 21, 2022

stephendonner commented Jan 21, 2022 • edited by GeetaSarvadnya Loading

pes10k commented Apr 2, 2021 •

edited

Loading

Dryader commented Apr 7, 2021 •

edited by fmarier

Loading

fmarier commented Apr 7, 2021 •

edited

Loading

stephendonner commented Jan 21, 2022 •

edited by GeetaSarvadnya

Loading