Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Prebid 7: refactoring of refererInfo, pageUrl, publisherDomain #8450

Merged
merged 18 commits into from
Jun 7, 2022
Merged

Prebid 7: refactoring of refererInfo, pageUrl, publisherDomain #8450

merged 18 commits into from
Jun 7, 2022

Conversation

dgirardi
Copy link
Collaborator

@dgirardi dgirardi commented May 23, 2022 •
edited
Loading

Type of change

  • Refactoring (no functional changes, no api changes)

Description of change

This refactors refererDetection (seen by adapters as bidderRequest.refererInfo) to:

  • rename referer to topmostLocation, keeping it for backwards compatibility.
  • add location - the value of window.top.location, or null when that is unavailable (blocked by cross-origin protections)
  • add ref - the value of window.top.document.referrer, or null when that is unavailable
  • add page - canonicalUrl falling back to location
  • add domain - the domain portion of page

Additionally, modules that use refererDetection (which is nearly every one) have been refactored with approximately this logic:

  • if they were just looking at refererInfo.referer, they are now using refererInfo.page instead
  • if they were clearly expecting refererInfo.referer to be the referrer (who would blame them), they are now using refererInfo.ref instead. Unfortunately, the original intent is rarely clear - this mostly applies to ortb2 payloads that were filling out both page and ref.
  • if they were doing "advanced" stuff, such as looking at refererInfo.stack, reachedTop etc, then refererInfo.referer was replaced with refererInfo.topmostLocation to keep the same behavior.

Because of: the ambiguity of the original name choices; the sheer number of affected modules, and the impressive variation in how they deal with this task, I am sure to have broken some adapters. Please check how your module is affected; I have done my best to mark ambiguities with TODO comments.

Other information

Resolves #5637
Documentation PR: prebid/prebid.github.io#3799

@dgirardi
Squashed commit of the following:
1fd8431 
commit ef322e9964924bcdbe9548e56df69b00b000f642
Merge: 9c1b79086 a8c4a80
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon May 23 12:08:14 2022 -0700

    Merge branch 'prebid-7' into referer

commit 9c1b7908659aaf904abd2bc4fcf53cd42a5b81cc
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon May 23 12:03:29 2022 -0700

    fix lint

commit 4d21818db17e08c038153eb2c77a1c7ebace13be
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon May 23 11:55:18 2022 -0700

    convert bidders: rtbhouse

commit 056c3172dcbae31ef1b7c63b5d033a5df713f0be
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon May 23 11:49:20 2022 -0700

    convert bidders: richaudience

commit 3b95d559a8075345da16ab59bca40def2572ee1c
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon May 23 11:12:43 2022 -0700

    convert bidders: revcontent

commit 9c3acfc9c86d9dc7c1928f21855288110ab87a8e
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon May 23 11:01:40 2022 -0700

    convert bidders: relaid

commit 65dbe122a7156ea1b380db37c083a90a0e7aa11a
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon May 23 10:47:48 2022 -0700

    convert bidders: quantcast

commit dc8fbb0c147c3229c8ceb4528b049a053b03b276
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon May 23 10:33:06 2022 -0700

    convert bidders: pulsepoint

commit 775ca874aaac052dae51498db10910cf25f51fbe
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon May 23 10:26:51 2022 -0700

    convert bidders: pubmatic

commit 60df0651fc27524100e944edca457a3ef95e1f4c
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon May 23 10:07:58 2022 -0700

    convert bidders: otm

commit 5ec60e4
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 12:44:58 2022 -0700

    convert bidders: operaads

commit e6bd48c
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 12:21:01 2022 -0700

    convert bidders: missena

commit d0703d4
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 12:00:38 2022 -0700

    convert bidders: microad

commit 8da461f
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 11:47:18 2022 -0700

    convert bidders: mediakeys

commit 15ee1b4
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 11:35:24 2022 -0700

    convert bidders: luponmedia

commit ab4e778
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 11:32:52 2022 -0700

    convert bidders: loglyft

commit f6c3be0
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 11:25:01 2022 -0700

    convert bidders: livewrapped

commit e2260b6
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 11:20:34 2022 -0700

    convert bidders: kubient

commit 50f0610
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 11:18:34 2022 -0700

    convert bidders: kobler

commit 8ba55cd
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 11:11:44 2022 -0700

    convert bidders: ix

commit 4486b4f
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:55:59 2022 -0700

    convert bidders: iqm

commit f95f031
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:41:11 2022 -0700

    convert bidders: iprom

commit cead162
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:38:11 2022 -0700

    convert bidders: insticator

commit 729a870
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:36:08 2022 -0700

    convert bidders: inskin

commit 93d6a47
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:35:07 2022 -0700

    convert bidders: inmar

commit 1105c2a
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:34:03 2022 -0700

    convert bidders: improveddigital

commit 535ff28
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:30:13 2022 -0700

    convert bidders: hybrid

commit 561b10b
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:27:31 2022 -0700

    convert bidders: gumgum

commit 1fcf289
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:24:35 2022 -0700

    convert bidders: grid

commit 46bf009
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:16:13 2022 -0700

    convert bidders: gmoss

commit 37c735a
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:13:33 2022 -0700

    convert bidders: glomex

commit 9feb6f5
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:07:43 2022 -0700

    convert bidders: glimpse

commit 90a3134
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:04:49 2022 -0700

    do not remove www in parseDomain

commit 6aeecac
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 10:01:51 2022 -0700

    convert bidders: gamoshi

commit 8b66ebb
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 09:54:23 2022 -0700

    convert bidders: feedad

commit effdac6
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 09:53:16 2022 -0700

    convert bidders: fabrick

commit c9f3a95
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 09:51:15 2022 -0700

    convert bidders: eplanning

commit 2a93dab
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 09:38:55 2022 -0700

    convert bidders: emx

commit 7f5c15a
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 09:31:43 2022 -0700

    convert bidders: deltaprojects

commit ce45157
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 09:18:27 2022 -0700

    convert bidders: criteoId

commit 985dcbc
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 09:14:29 2022 -0700

    convert bidders: criteo

commit 5cf6955
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 09:05:44 2022 -0700

    convert bidders: conversant

commit edf8b41
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 09:02:25 2022 -0700

    convert bidders: connecta

commit e5c71dd
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 08:59:02 2022 -0700

    convert bidders: concert

commit 29cd262
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 08:54:51 2022 -0700

    convert bidders: codefuel

commit f5c6357
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 08:54:05 2022 -0700

    convert bidders: cleanmedia

commit abd70f9
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 08:47:54 2022 -0700

    convert bidders: brightcom

commit a04ad82
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 08:42:14 2022 -0700

    convert bidders: bridgewell

commit a237df5
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 08:39:44 2022 -0700

    convert bidders: brave

commit 94337ef
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 08:26:21 2022 -0700

    convert bidders: bliink

commit b29945e
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri May 20 08:21:50 2022 -0700

    convert bidders: beachfront

commit 97ea0df
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu May 19 09:21:54 2022 -0700

    convert bidders: axonix

commit 0f5d97b
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu May 19 09:01:18 2022 -0700

    convert bidders: appnexus

commit 2d57f41
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu May 19 08:53:30 2022 -0700

    convert bidders: apacdex

commit 2ef8eaf
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu May 19 08:46:01 2022 -0700

    convert bidders: amx

commit fafb541
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu May 19 08:36:35 2022 -0700

    convert bidders: alkimi

commit 113e717
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 18 13:03:57 2022 -0700

    convert bidders: adyoulike

commit b9a4d56
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 18 12:50:19 2022 -0700

    convert bidders: adxcg

commit 06b0c9d
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 18 12:35:36 2022 -0700

    convert bidders: aduptech

commit 039444b
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 18 12:23:16 2022 -0700

    convert bidders: adrino

commit 70305f4
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 18 12:07:09 2022 -0700

    convert bidders: adnuntius

commit c614eca
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 18 12:03:30 2022 -0700

    convert bidders: adloox

commit 43327a9
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 18 11:55:07 2022 -0700

    convert bidders: adkernel

commit 0170539
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 18 11:20:28 2022 -0700

    convert bidders: adgeneration

commit f849022
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 18 11:09:30 2022 -0700

    convert adapters: adagio

commit e861922
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 18 10:50:25 2022 -0700

    rename referer to topmostLocation, make location all-or-nothing

commit b31e7cd
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue May 17 10:42:45 2022 -0700

    Updated refererInfo logic
@dgirardi dgirardi mentioned this pull request May 25, 2022
Merged
9 tasks
@dgirardi
Squashed commit of the following:
4e3915a 
commit d6d54cab25563e0ebece456d75733c9b9d78175e
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 25 07:40:48 2022 -0700

    fix lint

commit 43898cdf487b67061e48d65b7c469884df05918a
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 25 07:37:46 2022 -0700

    Do not provide a default value for publisherDomain

commit 5db64028f42935d2b62f58052562b9ae908c58f8
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 25 07:33:45 2022 -0700

    convert bidders: pass 2

commit a916edbb527f3342abfc2f55485918b08f8c2d30
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 25 07:09:05 2022 -0700

    convert bidders: done!

commit 1922bfbf57058b333540f9fef157bb177288dd51
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 25 06:53:44 2022 -0700

    convert bidders: visx

commit 31bb9c05044ca911ed683c7f86147ff9475964a1
Merge: dd22a3ddf e524628
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 25 06:40:54 2022 -0700

    Merge branch 'prebid-7' into referer-merge

commit e524628
Merge: c885816 14ae8d5
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Wed May 25 06:35:12 2022 -0700

    Merge branch 'master' into prebid-7

commit 14ae8d5
Author: Patrick McCann <patmmccann@gmail.com>
Date:   Tue May 24 15:58:21 2022 -0400

    Update fintezaAnalyticsAdapter_spec.js (prebid#8467)

commit b01ec40
Author: Prebid.js automated release <prebidjs-release@prebid.org>
Date:   Tue May 24 18:40:10 2022 +0000

    Increment version to 6.27.0-pre

commit e728f54
Author: Prebid.js automated release <prebidjs-release@prebid.org>
Date:   Tue May 24 18:40:10 2022 +0000

    Prebid 6.26.0 release

commit 170f6fd
Author: Patrick McCann <patmmccann@gmail.com>
Date:   Tue May 24 14:23:40 2022 -0400

    Revert "Akamai RTD: fixed bugs on rtd module and added the entropy values (prebid#8284)" (prebid#8464)

    This reverts commit c434889.

commit a6e3c44
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue May 24 11:21:26 2022 -0700

    Remove medianetRtdProvider tests (prebid#8463)

commit dd22a3ddf7dd6b01a9adfacada728f9c435cd0b0
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue May 24 10:31:14 2022 -0700

    convert bidders: unicorn

commit de3801d8709e362d0507ee75dd92807734824bbf
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue May 24 09:17:20 2022 -0700

    convert bidders: sovrn

commit 063dfb337e67fb8fff98a738e86459ab3851b13e
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue May 24 08:54:47 2022 -0700

    convert bidders: smarthub

commit 1a4d03b0fb3c362f3d73a2ac4ee652aa2dba3768
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue May 24 08:33:45 2022 -0700

    convert bidders: rubicon

commit c885816
Author: Nic G <nic.gallardo@freestar.io>
Date:   Tue May 24 11:09:27 2022 -0400

    Sortable Adapter Removal (prebid#8447)

commit f11bcda
Author: asurovenko-zeta <80847074+asurovenko-zeta@users.noreply.github.com>
Date:   Tue May 24 20:51:05 2022 +0700

    Zeta global ssp bid adapter: add shortname param (prebid#8454)

    * zeta_global_sspBidAdapter shortname was added

    * remove the trash

    Co-authored-by: Surovenko Alexey <surovenko.alexey@gmail.com>

commit c434889
Author: Vikas Srivastava <30315503+visrivastava@users.noreply.github.com>
Date:   Tue May 24 19:17:57 2022 +0530

    Akamai RTD: fixed bugs on rtd module and added the entropy values (prebid#8284)

    * Fixed bugs on rtd module and added the entropy values required by Akamai DAP

    * Fixed the timeout issue in build browserstack tests

    * Fixing review comments

    * Fixing review comments - using storage manager for managing localStorage

    * Fixing review comments - using loadExternalScript method to load the script

    * Fixed unit test case

    * Fixing review comments - Added consent handling

commit 06ecc38
Author: haruka-yamashita2 <39541428+haruka-yamashita2@users.noreply.github.com>
Date:   Tue May 24 22:24:00 2022 +0900

    Rename id to aoneId from dacId (prebid#8453)
@dgirardi dgirardi marked this pull request as ready for review May 25, 2022 14:45
@lgtm-com
Copy link

lgtm-com bot commented May 25, 2022

This pull request introduces 1 alert when merging 4e3915a into e524628 - view on LGTM.com

new alerts:

  • 1 for Useless conditional

@dgirardi dgirardi mentioned this pull request May 25, 2022
Merged
@patmmccann patmmccann linked an issue May 25, 2022 that may be closed by this pull request
Overriding referrer: publisherDomain vs pageUrl #5637
Closed
@patmmccann
Copy link
Collaborator

@robertrmartinez as this resolves #8294 as well, seeking your review

leonardlabat
leonardlabat reviewed Jun 2, 2022
View reviewed changes
modules/criteoBidAdapter.js
}
const queryString = parseUrl(referrer).search;
const queryString = parseUrl(bidderRequest?.refererInfo?.topmostLocation).search;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be referrerInfo.page here too
What made you think it was better to use topmostlocation ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish I wrote that down at the time, but I believe my reasoning was - queryString is only used for this:

    debug: queryString['pbt_debug'] === '1',
    noLog: queryString['pbt_nolog'] === '1',

those seem unlikely to be present in either setConfig(pageUrl) or <link rel="canonical" /> which have precedence for determining page.

I can make the change if you think it's still the better choice though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nop indeed it makes sense, let's keep topmostlocation.

Thanks !

@patmmccann patmmccann mentioned this pull request Jun 2, 2022
Closed
robertrmartinez
robertrmartinez reviewed Jun 6, 2022
View reviewed changes
modules/rubiconBidAdapter.js
@@ -831,11 +831,12 @@ function _getScreenResolution() {
* @returns {string}
*/
function _getPageUrl(bidRequest, bidderRequest) {
let pageUrl = config.getConfig('pageUrl');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is config.getConfig('pageUrl') removed? Is it deprecated?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it overrides canonicalUrl (and therefore also page). The idea is to make adapters respect it by default.

(Also see discussion with @jsnellbaker above - up until this PR, the override would happen only some of the time).

robertrmartinez
robertrmartinez approved these changes Jun 6, 2022
View reviewed changes
Copy link
Collaborator

@robertrmartinez robertrmartinez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good from Rubicon adapter standpoint, thanks!

@patmmccann patmmccann merged commit 6e3f8dd into prebid:prebid-7 Jun 7, 2022
@dgirardi dgirardi mentioned this pull request Jun 10, 2022
Closed
2 tasks
@ChrisHuie ChrisHuie mentioned this pull request Jul 26, 2022
Merged
10 tasks
@osazos osazos mentioned this pull request Aug 11, 2022
Merged
2 tasks
@dgirardi dgirardi mentioned this pull request Oct 25, 2022
Merged
1 task
@dgirardi dgirardi mentioned this pull request Nov 9, 2022
Closed
@dgirardi dgirardi mentioned this pull request Jan 3, 2023
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-baptiste-haudegand github-baptiste-haudegand github-baptiste-haudegand left review comments

@nllerandi3lift nllerandi3lift nllerandi3lift left review comments

@lasloche lasloche lasloche left review comments

@patmmccann patmmccann patmmccann left review comments

@leonardlabat leonardlabat leonardlabat left review comments

@jsnellbaker jsnellbaker jsnellbaker approved these changes

@robertrmartinez robertrmartinez robertrmartinez approved these changes

Assignees

@robertrmartinez robertrmartinez

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Overriding referrer: publisherDomain vs pageUrl
8 participants
@dgirardi @patmmccann @leonardlabat @robertrmartinez @jsnellbaker @lasloche @nllerandi3lift @github-baptiste-haudegand