Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core & Multiple modules: activity controls #9802

Merged
merged 43 commits into from
Jun 1, 2023

Conversation

dgirardi
Copy link
Collaborator

@dgirardi dgirardi commented Apr 13, 2023

Type of change

  • Feature

Description of change

This is an implementation of the activity controls infrastructure, and a refactoring of many of the current control mechanisms:

  • storage access control (bidderSettings.*.storageAllowed, deviceAccess) are now routed through the accessDevice activity
  • user sync controls (filterSettings) go through syncUser
  • all the checks done by the GDPR enforcement module now use activities (in additions to the above, they also affect fetchBids and enrichUids)
  • there are completely new checks for the transmit* and enrich* activites, for both bid adapters and RTD modules

Other information

Closes #9546
Documentations coming soon!

Copy link
Collaborator

@robertrmartinez robertrmartinez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just couple minor comments.

Beast of a PR but very very cool!

I tested it very thoroughly using mock CMP's, all the different Activities, FPD transmit and updates and it all seems to work as expected.

One minor comment which we spoke in slack about is that for storage for bidders the default is NOT ALLOW.

And there is no easy way with Activity Set Config to flip this

This does not work for example:

                allowActivities: {
                    accessDevice: {
                        default: true,
                    }
                }

Because there is special bidder handling, bidder settings still denies bidders from storage.

So a workaround is like so:

                    accessDevice: {
                        rules: [
                            {
                                condition: ({componentType, componentName}) => {
                                    return true
                                },
                                allow: true
                            }
                        ]
                    }

To allow all access to storage (or of course using standard bidderSettings.)

Which I think is fine.

Another thing we spoke about offline is moving the cfg.js logic (handling of activity setConfig stuff) to an optional module.

Lastly, When using Activity controls, many console logs come up and clutter the console. Especially when adapters use storage. We spoke a bit on some ways to suppress them but not sure there is a nice clean way besides a debounce which may not be worth doing. Users can just filter out Activity in their consoles if needed really. -Activity will do this in chrome.

Anyways, this is really really well done. Thanks @dgirardi . Great stuff


config.getConfig(CFG_NAME, (cfg) => {
clearAllRules();
Object.entries(cfg[CFG_NAME]).forEach(([activity, cfg]) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cfg is used twice here, inside the getConfig callback as the entire config, and inside the activities for each callback as the specific activity. Is there a reason to keep them same name or could we rename the inner one to be more clear?

res = rule(params);
} catch (e) {
logger.logError(`Exception in rule ${name} for '${activity}'`, e);
res = {allow: false, reason: e};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if exception, we default to not allow?

Which is different than the default of allow when no rule is set right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct, my reasoning is when it throws the pub clearly meant to do something else, so failing loudly seems the best strategy.

];
}

export const [registerActivityControl, isActivityAllowed] = ruleRegistry();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reasoning behind ruleRegistry returning array vs object for these exports. I have not come across it much and curious to learn more!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a desperate attempt to save half a byte :) object property names are not mangled by the minifier so returning {someName, someOtherName} will keep the full names in the final bundle, but as an array it will become something like [a, b].

It's not a great technique because if the objects get just a little more complex they are unreadable as arrays, but I haven't found yet a way to tell uglify "please mangle this class" for example. I'm also open to learning more!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh yes makes sense. Awesome thanks and makes sense. Every byte counts when people are always complaining haha!

@dgirardi
Copy link
Collaborator Author

condition can be omitted if the rule is always active (referring to the workaround above), e.g.

setConfig({
   allowActivities: {
     accessDevice: {
        rules: [
            {
                allow: true
            }
        ]
      }
   }
})

I do plan to update this very soon with configuration logic extracted into a module and cleaner logging (yes it can be filtered out, but it'd bother me if I had to).

function canUseStorage(submodule) {
switch (submodule.config?.storage?.type) {
case LOCAL_STORAGE:
if (submodule.storageMgr.cookiesAreEnabled()) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noting this for myself - this should be localStorageIsEnabled

Copy link
Contributor

@pm-harshad-mane pm-harshad-mane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @robertrmartinez for a very detailed review.
I did a high level review of this GRAND PR and it looks great to me 👍
Thanks a lot @dgirardi for working this huge and highly important feature.

@pm-harshad-mane
Copy link
Contributor

@dgirardi there seems an issue with E2E tests, can you please take a look?

@dgirardi dgirardi merged commit 0a7ca9f into prebid:master Jun 1, 2023
mscottnelson added a commit to 33Across/Prebid.js that referenced this pull request Jun 15, 2023
* master:
  Colossus Bid Adapter: gpp support (prebid#10096)
  Increment version to 8.1.0-pre
  Prebid 8.0.0 release
  Prebid 8: initial release (prebid#10071)
  Increment version to 7.54.1-pre
  Prebid 7.54.0 release
  Update topicsFpdModule.js (prebid#9990)
  Added GPP support (prebid#10094)
  Teads Bid Adapter:Remove auctionId from since it is not used anymore  (prebid#10098)
  HypeLab Bid Adapter: Initial Release (prebid#10003)
  Oxxion Rtd Module: listen onBidResponseEvent instead of onAuctionEndEvent (prebid#10083)
  PubMatic Analytics Adapter : Added extra fields in tracker for analytics (prebid#10090)
  AdHash Bidder Adapter: update for brand safety (prebid#10087)
  videojsVideoProvider: use contrib-ads event names (prebid#10081)
  Increment version to 7.54.0-pre
  Prebid 7.53.0 release
  Add bidgency alias (prebid#10077)
  Weborama RTD Module : start Bidder specific handling removal (prebid#10005)
  ZetaGlobalSspBidAdapter: change logic around mediaType (prebid#10074)
  add tmax to sovrnBidAdapter (prebid#10059)
  Mediasquare Bid Adapter: Avoid to run bidwon event on video impressions (prebid#10068)
  FreePass User ID Module : initial release (prebid#9814)
  Conceptx Bid Adapter : initial release (prebid#9835)
  beOp BidAdapter: FirstPartyData management and ingest Permutive segments (prebid#9947)
  StroeerCore Bid Adapter: add price floor support (prebid#9962)
  YieldlabBidAdapter updated native asset mapping (prebid#9989)
  Mediasquare Bid Adapter: handle context for video bids (prebid#10055)
  Add new alias - Apester (prebid#10057)
  Criteo Bid Adapter: Pass outstream video renderer method (prebid#10054)
  Add: Viously GVL ID (prebid#10061)
  1plus x RTD Adapter: remove bidder specific handling enforcement (DC 3634) (prebid#10001)
  Sirdata RTD Module : bidder specific handling removal (prebid#9970)
  feat: pass video.plcmt [PB-1736] (prebid#10058)
  Update adkernelBidAdapter.js (prebid#10056)
  Eskimi Bid Adapter: Added support for video (prebid#9985)
  Richaudience Bid Adapter : add compability with GPID (prebid#9928)
  Add GVL ID for pairId userId submodule. (prebid#10053)
  Update nexx360BidAdapter.js (prebid#10042)
  Logicad Bid Adapter: Add topics and uach support (prebid#10045)
  change session_id fallback to bidderRequestId (prebid#10047)
  ZetaGlobalSsp Bid Adapter : process array of sizes (prebid#10039)
  AdagioBidAdapter: change outstream renderer (prebid#10035)
  update greenbids analytics endpoint (prebid#10037)
  ID5 User Id module - get whole ext object from server response (prebid#10036)
  Criteo Bid Adapter : Bump fastbid version to 136 (prebid#9973)
  UID2 & EUID Modules: Add support for EUID and prefer localStorage for both modules. (prebid#9968)
  Schain module: do not share schain objects across different bid requests (prebid#10021)
  airgridRtdProvider: use provided tag insertion method `loadExternalScript` (prebid#9901)
  Increment version to 7.53.0-pre
  Prebid 7.52.0 release
  Yahoo ConnectId - fixed tests. (prebid#10033)
  Core & Multiple modules: activity controls (prebid#9802)
  Yahoo ConnectId - gpp consent module usage. (prebid#10022)
  Yahoo bid adapter:  User sync pixels, consent signals update (prebid#10028)
  Core: fix `markWinningBidAsUsed` to set bid as winning and not just rendered (prebid#10014)
  Adriver Id System: add cfa param to url (prebid#10024)
  Undertone Bid Adapter : added GPP and video placements (prebid#10016)
  Oxxion Analytics Adapter : initial adapter release (prebid#9682)
  Connect id : storage duration updates and no longer require puid or he to be set (prebid#9965)
  Update ad generation adapter 1.6.0: update userSync (prebid#9984)
  fix module type (prebid#10019)
  Stv Bid Adapter: add schain support (prebid#10010)
  GrowthCode RTD  : initial release (prebid#9852)
  ShareThrough Bid Adapter : fix playerSize (prebid#10011)
  chore: update default video placement value [PB-1560] (prebid#9948)
  Smartadserver Bid Adapter: support GPID (prebid#10004)
  Criteo Adapter: Add support of bcat/badv/bapp (prebid#10013)
  Zeta Global SSP Bid Adapter: Added support for video.plcmt (prebid#10009)
  Pair Id System: less noisy errors (prebid#9998)
  Increment version to 7.52.0-pre
  Prebid 7.51.0 release
  Appnexus Bid Adapter: added uol as an alias (prebid#10002)
  Adquery Bid Adapter : added bid request: version and bidPageUrl (prebid#9946)
  MinuteMedia Bid Adapter: support sua and plcmt params (prebid#9997)
  Adkernel Bid Adapter: add didnadisplay alias (prebid#10000)
  Adrino Bid Adapter: pass adUnitCode to adserver (prebid#9993)
musikele pushed a commit to rubicon-project/Prebid.js that referenced this pull request Aug 28, 2023
* Core: allow restriction of cookies / localStorage through `bidderSettings.*.storageAllowed`

* Add test cases

* Remove gvlid param from storage manager logic

* Refactor every invocation of `getStorageManager`

* GVL ID registry

* Refactor gdprEnforcement gvlid lookup

* fix lint

* Remove empty file

* Undo prebid#9728 for realVu

* Fix typo

* Activity control rules

* Rule un-registration

* fetchBids enforcement

* fetchBids rule for gdpr

* enableAnalytics check

* reportAnalytics TCF2 rule

* Update logging condition for multiple GVL IDs

* Change core to prebid

* Refactor userID to use non-core storage manager when storing for submodules

* enrichEids check

* gdpr enforcement for enrichEids

* syncUser activity check

* gdpr enforcement for syncUser

* refactor  gdprEnforcement

* storageManager activity checks

* gdpr enforcement for accessDevice

* move alias resolution logic to adapterManager

* Refactor file structure to get around circular deps

* transmit(Eids/Ufpd/PreciseGeo) enforcement for bid adapters

* Object transformers and guards

* transmit* and enrich* enforcement for RTD modules

* allowActivities configuration

* improve comments

* do not pass private activity params to pub-defined rules

* fix objectGuard edge case: null values

* move config logic into a module

* dedupe log messages
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GPP Phase 3 - Activity control infrastructure
4 participants