Skip to content

Latest commit

 

History

History
2228 lines (1720 loc) · 110 KB

bidding_auction_services_api.md

File metadata and controls

2228 lines (1720 loc) · 110 KB
Raw

Authors:
Priyanka Chatterjee, Google Privacy Sandbox
Itay Sharfi, Google Privacy Sandbox

Bidding and Auction services

The Privacy Sandbox aims to develop technologies that enable more private advertising on the web and mobile devices. Today, real-time bidding and ad auctions are executed on servers that may not provide technical guarantees of security. Some users have concerns about how their data is handled to generate relevant ads and in how that data is shared.

Protected Auctions API provides ways to preserve privacy and limit third-party data sharing by serving personalized ads based on previous mobile app or web engagement. Protected Auctions include:

The Bidding and Auction Services (B&A) proposal outlines a way to allow Protected Auction computation to take place on cloud servers in a trusted execution environment, rather than running locally on a user's device. Running workloads in a Trusted Execution Environment (TEE) in cloud have the following benefits:

  • Scalable ad auctions.

    • A scalable ad auction may include several buyers and sellers and that can demand more compute resources and network bandwidth.

  • Lower latency of ad auctions.

    • Server to server communication on the cloud is faster than multiple device to server calls.
    • Adtech code can execute faster on servers with higher computing power.

  • Higher utility of ad auctions.

    • Servers have better processing power, therefore adtechs can run compute intensive workloads on a server for better utility.
    • Lower latency of ad auctions also positively impact utility.

  • Security protection

  • System health of the user's device.

    • Ensure better system health of user's device by freeing up computational cycles and network bandwidth.

This document focuses on timeline and roadmap, high level design, API for Bidding and Auction services.

Useful information

Client integration

Clients imply web browsers and Android platforms. In this context:

  • Web browsers imply browsers on desktop and Android devices.
  • Android implies Android apps.

Chrome and Android announced to integrate with Bidding and Auction services. See blog.

Browser

Refer to browser API for Bidding and Auction services.

In case of browser, B&A request and response payload are Concise Binary Object Representation (CBOR) encoded. Following are the web schemas used by browsers.

Near drop-in replacement

Bidding and Auction services integrate into Protected Audience API for browsers and can scale as a near drop-in replacement for adtechs who already adopted Protected Audience API for on-device execution and are interested in exploring a server side solution.

  • Interest Groups (Custom Audience) creation and management can stay the same.
  • User defined functions (UDF) developed by adtechs:
    • The function signatures of UDFs for bid generation, scoring and reporting can stay exactly the same.
    • Buyer's code for generateBid() would mostly work. However, certain updates will be required for payload optimization, see here for more details.
  • Key-value services:
    • Seller key-value service do not require additional updates.
    • Buyer key-value service require some additional updates. The trusted bidding signals may need to include more information to support payload optimization, see here for more details.
  • Seller integration:
    • Seller's code on client would require additional changes to call an API (provided by client) to fetch encrypted B&A request payload (B&A request ciphertext) and include that in ad request to seller's ad service.
    • Seller's ad service would require additional changes to call B&A.
    • Seller's ad service would require additional changes to handle encrypted B&A response (B&A response ciphertext) and send that back to the client.
    • Seller's ad service would require additional changes to call an API (provided by client) to decrypt B&A response ciphertext.

Android app

In case of Android app, the B&A request and response payload are based on protobufs.

Privacy Preserving Ads

Ad Selection API is a proposal by Microsoft Edge browser and similar to Protected Audience. The proposal supports server-side bidding and auction in trusted execution environment.

Related documents

Following explainers are recommended to adtechs onboarding to B&A. You may see also see the full list of documents related to Protected Auctions and service trust model.

Github discussion

Please file an issue in Github/WICG/protected-auction-services-discussion for feedback and discussion.

Code repository

Bidding and Auction services code is available in Github and a new code version is released every few weeks.

  • Refer to releases page for the available code versions, prod and non-prod image hashes of the services corresponding to each released version.
  • The prod images are approved by the Coordinators and must be used in production environment.
  • The non-prod images do not require Coordinator approval and must be used in B&A services that enables TEST_MODE and disables attestation.

Adtechs will have to deploy the binaries and configurations to a supported public cloud platform.

Supported public cloud platforms

Bidding and Auction services will be available within the Trusted Execution Environment(TEE) on AWS and GCP 2023 onwards.

More cloud platforms may be supported eventually. See the Public Cloud TEE requirements explainer for more details.

Note: SSP and DSP can operate Bidding and Auction services on different cloud platforms that are supported. For multi seller auctions, SSPs can operate on different cloud platforms.

Bidding and Auction services will not support a “Bring Your Own Server” model, similar to what is made available to Protected Audience’s Key/Value server. Bidding and Auction services can only be deployed within approved TEE cloud environments. This is valid for Alpha, Beta, Scale testing programs and beyond.

AWS support

Bidding and Auction services runs in Nitro Enclaves on AWS. Refer here for more details.

GCP support

Bidding and Auction services runs in Confidential Space (Confidential Computing) on GCP. Refer here for more details.

Types of ad auctions

Bidding and Auction services supports single-seller and Multi seller auctions for web and app traffic.

Timeline and roadmap

The following sections include the timelines for ad techs interested in testing Bidding and Auction services for Protected Auctions. Protected Auctions refer to Protected Audiences and Protected App Signals ad targeting products.

Testing phases

The supported testing phases for Bidding and Auction services are Alpha, Beta, and Scale. These phases are rolling windows and specific timelines may vary for every ad tech.

Alpha testing

During this phase, ad techs onboard, set up, and run services on non-production user opt-in traffic.

The goal of this phase is to complete end-to-end functional testing from client to server. This would require a seller integrating with clients (web browsers, Android) and one or more partner buyers; and a buyer integrating with at least one partner seller.

For alpha testing, check the following:

  • Coordinator integration is not required.
  • Debug build and production build would be available for testing. Ad techs can run Bidding and Auction services locally (debug mode) or in TEE using test mode.
  • As an interim milestone, ad techs can start testing their Bidding and Auction setup, even before fully integrating with clients or partner ad techs.
    • Buyers or DSPs can independently test their setup, before fully integrating with partner sellers.
    • Sellers can independently test their setup with a real partner buyer or fake buyer, before fully integrating with the seller's ad server or clients.
  • Clients do not necessarily need to enable a traffic experiment. Testing can be done with test or developer-owned devices.
  • Ad tech testing or experimentation can be enabled on user opt-in traffic (that is, on a small set of users).

Beta testing

During this phase, clients (web, Android) would enable Bidding and Auction APIs on limited stable (end user) devices. Ad techs can run services on limited stable production traffic.

For beta testing, check the following:

  • Enrollment with coordinators is required.
  • Ad techs must run Bidding and Auction services in TEE in production mode.
    • Sellers must integrate with clients and at least one partner buyer for conducting protected auctions.
    • The buyer must integrate with at least one seller.
  • Clients would enable and ramp experiments to enable Bidding and Auction APIs on a small percentage of real end user devices.
    • Note: Clients would determine the ramp percentage for an experiment.
  • Ad techs can enable and ramp experiments on a limited stable production or live traffic for initial performance analysis.
    • Note: Seller or SSP would determine the ramp percentage for an experiment and align with partner buyers.

Scale testing

During this phase, ad techs can run services on full stable, production scale testing.

For scale testing, check the following:

  • Enrollment with Coordinators is required.
  • Adtechs must run Bidding and Auction services in TEE, in production mode.
    • Sellers must integrate with clients and at least one partner buyer for conducting protected auctions.
    • The buyer must integrate with at least one seller.
  • Clients would ramp experiments to enable Bidding and Auction APIs on a larger percentage of real end user devices.
  • Ad techs can enable and ramp experiments on full stable, production or live traffic. At this point, ad techs can do advanced performance, utility, and cost analysis.

Timelines

The following timelines show the availability of Bidding and Auction services.

API Alpha testing Beta testing Scale testing
Protected Audience

(web browser on desktop and Android)

July 2023 onwards

Available

  • Beta 1: Nov 2023 onwards
  • Beta 2: April 2024 onwards Production experiment ramp:
    • Chrome browser enabled B&A APIs for 1% stable user traffic.
Jan 2025

Production experiment ramp:

  • Chrome browser will enable B&A APIs for 100% stable user traffic.
Protected Audience (Android) Available April 2024 onwards Jan 2025
Protected App Signals (Android) Available September 2024 onwards Jan 2025

Roadmap

Note: The common server side features are common to all types of Protected Auctions on web and Android apps.

Timeline Common server side features Protected Audience for web browser on desktop and Android Protected Audience and Protected App Signals (PAS) on Android Apps
July 2023
November 2023
January 2024
  • Protected App Signals
    • Client and Bidding and Auction integration to support Protected App Signals.
    • Combined PAS and PA support.
    • Reporting: Event-level win reporting.
    • Cloud support: GCP support.
March 2024
April 2024
  • Reporting:
    • BuyerReportingId
  • Reporting and debugging:
    • forDebugOnly (debug reporting) for single seller auctions.
July 2024
  • Security protections:
    • OHTTP Encapsulation Media Type
  • Multi cloud support on Android
    • Android devices can fetch public / encryption keys for more than one cloud platform to encrypt the ProtectedAuctionInput for B&A.
  • Multiseller auction:
    • Waterfall mediation
  • Debugging:
  • Protected App Signals:
    • Cloud support: AWS support.
    • B&A and Inference integration
August 2024
September 2024
October 2024
November 2024
  • Adtech UDF execution support
    • Multiple versions of adtech code blobs
  • Data version header
Jan 2025
March 2025
April 2025
June 2025
  • Adtech UDF execution support:
    • Publisher / subscriber message queue - B&A integration: This will support emergency rollouts of adtech code stored in cloud storage.
July 2025 and beyond
  • Adtech UDF execution support:
    • Code blob signing and verification
  • Parallelization of contextual and Bidding and Auction auctions

Further details to be added in future updates.
  • Optimizations related to multi slot ads
  • Support for ad size
  • Reporting and debugging:
    • forDebugOnly (debug reporting) for server orchestrated component auctions
  • Waterfall Mediation optimization with server side truncation
  • Protected App Signals:
    • Noising for event level egress
    • Protected App Signals and Protected Audience isolation support in Bidding service on supported cloud platforms.

Onboarding and alpha testing guide

Following is the guide for onboarding to Bidding and Auction services and participating in Alpha testing.

Guidance to sellers / SSPs:

  • Refer to Spec for SSP section.

  • Develop ScoreAd() for Protected Audience auction.

  • Develop ReportResult() for event level reporting.

  • Setup Seller's Key/Value service.

  • Add support such that seller's code on publisher web page calls browser API to fetch encrypted ProtectedAudienceInput. Then includes encrypted ProtectedAudienceInput in the request to seller's ad server.

  • Add support in Seller's ad server to send SelectAd request to Bidding and Auction services for Protected Audience auctions.

  • Add support for forwarding client metadata in the request to Bidding and Auction services (SellerFrontEnd).

  • Review Logging section.

  • Bidding and Auction services code and configurations is open sourced to Github repo.

  • Deploy SellerFrontEnd and Auction server instances to your preferred cloud platform that is supported.

  • Set up experiments for ad auctions and target user opt-in traffic. Include one or more partner buyers in the same experiment.

  • Chrome browser supports a flag FledgeBiddingAndAuctionKeyURL. Users' browsers that enable the flag can be targeted.

  • Enroll with coordinators and / or run servers in TEST_MODE.

    During onboarding, it is recommended to start with Option 1 and then switchover to Option 2. However, getting started with Option 2 is also feasible.

    • Option 1: Run Bidding and Auction services in TEST_MODE.

      • TEST_MODE supports cryptographic protection with hardcoded public-private key pairs, while disabling TEE server attestation. During initial phases of onboarding, this would allow adtechs test Bidding and Auction server workloads even before integration with Coordinators.
      • Set TEST_MODE flag to false in seller's Bidding and Auction server configurations (AWS, GCP).
      • The following options are available for testing the auction flow with Bidding and Auction services.

        • Option A: Using secure invoke tool.

          • The tool uses hardcoded public keys to encrypt payloads and then sends requests to TEE based Bidding and Auction services. The corresponding private keys of the same version are hardcoded / configured in Bidding and Auction services such that the encrypted payloads can be correctly decrypted.
          • The tool can generate SelectAdRequest payload for communication with TEE based SellerFrontEnd if a plaintext request payload is supplied. The payload will include ProtectedAudienceInput ciphertext that is encrypted with hardcoded public keys.
          • The tool also has the capability to decrypt the response received from Bidding and Auction services and print out the out human readable plaintext response.
          • Refer to the README for more information about the tool.

        • Option B: End to end flow from Chrome browser.

          • With this option, the FledgeBiddingAndAuctionKeyURL flag should be set to the following endpoint. The endpoint would serve a public key such that corresponding private keys of the same version are known to the Bidding and Auction services.

            FledgeBiddingAndAuctionKeyURL/http%3A%2F%2Flocalhost%3A8000%2Fkey

            The above endpoint can be configured to serve a public key with similar to the following.

            $ mkdir -p /tmp/bna && cd /tmp/bna && echo '{ "keys": [{ "id": "40", "key": "87ey8XZPXAd+/+ytKv2GFUWW5j9zdepSJ2G4gebDwyM="}]}' > key && python3 -m http.server 8000

    • Option 2: Enroll with coordinators.

      • During Alpha, Google Privacy Sandbox Engineers will act as Coordinators and operate the key management systems. Reach out to your Privacy Sandbox partner to get enrolled with Alpha Coordinators.
      • Integration of Bidding and Auction server workloads with Alpha Coordinators would enable TEE server attestation and allow fetching live encryption / decryption keys from public or private key service endpoints in Bidding and Auction services.
      • The flag supported by Chrome browser FledgeBiddingAndAuctionKeyURL should point to the public key service endpoint of key management systems run by Alpha Coordinators. This would allow the browser to fetch live public keys for encryption of ProtectedAudienceInput payload.

Guidance to buyers / DSPs:

  • Refer to Spec for DSP section.

  • Develop GenerateBid() for bidding.

  • Develop ReportWin() for event level reporting.

  • Setup Buyer's Key/Value service.

  • Optimise payload.

  • Review Logging section.

  • Bidding and Auction services code and configurations is open sourced to Github repo.

  • Deploy BuyerFrontEnd and Bidding server instances to your preferred cloud platform that is supported.

  • Enroll with coordinators or run servers in TEST_MODE.

    During onboarding, it is recommended to start with Option 1 and then switchover to Option 2. However, getting started with Option 2 is also feasible.

    • Option 1: Run Bidding and Auction services in TEST_MODE.

      • TEST_MODE supports cryptographic protection with hardcoded public-private key pairs, while disabling TEE server attestation. During initial phases of onboarding, this would allow adtechs test Bidding and Auction server workloads even before integration with Coordinators.
      • Set TEST_MODE flag to false in buyer's Bidding and Auction server configurations (AWS, GCP).
      • The following tool can facilitate testing buyer's Bidding and Auction services (BuyerFrontEnd, Bidding) and bid generation flow independently.
        • Secure invoke tool.
          • The tool uses hardcoded public keys to encrypt payloads and then sends requests to TEE based Bidding and Auction services. The corresponding private keys of the same version are hardcoded / configured in Bidding and Auction services such that the encrypted payloads can be correctly decrypted.
          • The tool can generate GetBidsRequest payload for communication with TEE based BuyerFrontEnd if a plaintext request payload is supplied.
          • The tool also has the capability to decrypt the response received from TEE based BuyerFrontEnd and print out the out human readable plaintext response.
          • Refer to the README for more information about the tool.

    • Option 2: Enroll with coordinators.

      • During Alpha, Google Privacy Sandbox Engineers will act as Coordinators and operate the key management systems. Reach out to your Privacy Sandbox partner to get enrolled with Alpha Coordinators.
      • Integration of Bidding and Auction server workloads with Alpha Coordinators would enable TEE server attestation and allow fetching live encryption / decryption keys from public or private key service endpoints in Bidding and Auction services.

  • Reach out to partner SSPs to include in experiments for Protected Audience auctions.

    • Note: Buyers can also independently start integrating and testing the bidding flow before they are included in a seller supported ad auction experiments.

Enroll with coordinators

Adtechs would have to enroll with two Coordinators running key management systems that provision keys to Bidding and Auction services after server attestion.

Adtechs should only enroll with the Coordinators for the specific cloud platform where they plan to run Bidding and Auction services.

Enrollment with AWS coordinators

An adtech should provide their AWS Account Id to both the Coordinators.

The Coordinators would create IAM roles. After adtechs provide the AWS account Id, they would attach that information to the IAM roles and include in an allowlist. Then the Coordinators would let adtechs know about the IAM roles and that should be included in the B&A server Terraform configs that fetch cryptographic keys from key management systems.

Following config parameters in buyer or seller server configs would include the IAM roles information provided by the Coordinators.

  • PRIMARY_COORDINATOR_ACCOUNT_IDENTITY
  • SECONDARY_COORDINATOR_ACCOUNT_IDENTITY

Enrollment with GCP coordinators

An adtech should provide IAM service account email to both the Coordinators.

The Coordinators would create IAM roles. After adtechs provide their service account email, the Coordinators would attach that information to the IAM roles and include in an allowlist. Then the Coordinators would let adtechs know about the IAM roles and that should be included in the B&A server Terraform configs that fetch cryptographic keys from key management systems.

Following config parameters in buyer or seller server configs would include the IAM roles information provided by the Coordinators.

  • PRIMARY_COORDINATOR_ACCOUNT_IDENTITY
  • SECONDARY_COORDINATOR_ACCOUNT_IDENTITY

Specifications for adtechs

Near drop-in replacement

Bidding and Auction services integrate into Protected Audience API for browsers and can scale as a near drop-in replacement for adtechs who already adopted Protected Audience API and are interested in exploring a server side solution.

  • Interest Groups (Custom Audience) creation and management can stay the same.

  • Key-value services can stay nearly the same.

    • Seller's key-value service can stay the same.
    • Buyer's key-value service instances can stay the same; however to support payload optimization, trusted_bidding_signals may need to include additional data.

  • Code developed by adtechs following the guidance in Protected Audience API for browsers will mostly work with Bidding and Auction services. The function signatures can stay exactly the same. * Seller's code for ScoreAd() can stay the same. * Seller's code for ReportResult() can stay the same. * Buyer's code for GenerateBid() would mostly work. However, certain updates will be required for payload optimization. * Buyer's code for ReportWin() can stay the same.

Spec for SSP

scoreAd()

The Auction service exposes an API endpoint ScoreAds. The SellerFrontEnd service sends a ScoreAdsRequest to the Auction service for running an auction. ScoreAdsRequest includes bids from each buyer and other required signals. The code for auction, i.e. ScoreAd() is prefetched from Cloud Storage, cached and precompiled in Auction service. After all ads are scored, the Auction service picks the highest scored ad candidate and returns the score and other related data for the winning ad in ScoreAdsResponse.

Note: If an SSP develops scoreAd() following web platform's Protected Audience API explainer, that should also work as-is for execution in the Auction service.

Adtech's scoreAd function signature is as follows.

scoreAd(adMetadata, bid, auctionConfig, trustedScoringSignals, bid_metadata) {
  ...
  return {desirability: desirabilityScoreForThisAd,
              allowComponentAuction: true_or_false};
}
Arguments
  • adMetadata: Arbitrary metadata provided by the buyer.
  • bid: A numerical bid value.
  • auctionConfig: This would include sellerSignals (auctionConfig.sellerSignals) and auctionSignals (auctionConfig.auctionSignals).
  • trustedScoringSignals: trustedScoringSignals fetched from seller's Key/Value service
    • Note: Only the signals required for scoring the ad / bid is passed to scoreAd().
  • bid_metadata: This refers to an object created in the Auction service based on the render_url of the bid and other information known to the Auction service.
{ 'topWindowHostname': 'www.example-publisher.com',
  'interestGroupOwner': 'https://www.example-dsp.com',
  'renderUrl': 'https://cdn.com/render_url_of_bid',
  'adComponents': ['https://cdn.com/ad_component_of_bid',
                   'https://cdn.com/next_ad_component_of_bid',
                   ...],
 'dataVersion': 1, /* Data-Version value from the trusted scoring signals server's response */
}

Seller BYOS Key/Value service

The SellerFrontEnd service looks up trustedScoringSignals from seller's Key/Value service. The base url (domain) for Key/Value service is configured in SellerFrontEnd service so that the connection can be prewarmed. All render_urls corresponding to all bids from buyers participating in an auction are encoded, then batched and looked up in a single request. The lookup url and response are in the same format as described in Chrome Protected Audience explainer.

The lookup url is in the following format:

<base_url>?renderUrls=<url_1>..<url_n>&adComponentRenderUrls=<url1>,..<urln>

Where <base_url>, <url1>... are substituted

The response is in the following format:

{ 'renderUrls': {
      'https://cdn.com/render_url_of_some_bid': arbitrary_json,
      'https://cdn.com/render_url_of_some_other_bid': arbitrary_json,
      ...},
  'adComponentRenderUrls': {
      'https://cdn.com/ad_component_of_a_bid': arbitrary_json,
      'https://cdn.com/another_ad_component_of_a_bid': arbitrary_json,
      ...}
}

The trustedScoringSignals passed to scoreAd() is as follows:

{ 
  'renderUrl': {'https://cdn.com/render_url_of_bidder': arbitrary_value_from_signals},
  'adComponentRenderUrls': {
      'https://cdn.com/ad_component_of_a_bid': arbitrary_value_from_signals,
      'https://cdn.com/another_ad_component_of_a_bid': arbitrary_value_from_signals,
      ...}
}

reportResult()

Event level win reporting would work with Bidding and Auction services and function signatures can be the same as described in web platform's Protected Audience API explainer.

The reporting urls for the seller and registered ad beacons (for Fenced Frame reporting) would be generated in Auction service and returned to the client in encrypted AuctionResult. The client will ping the seller's reporting endpoint using the reporting url.

Note: Refer to the event level reporting explainer for the detailed design.

reportResult(auctionConfig, reporting_metadata) {
  ...
  registerAdBeacon({"click", clickUrl,"view", viewUrl});
  sendReportTo(reportResultUrl);
  return signalsForWinner;
}
Arguments
  • auctionConfig: This would include sellerSignals (auctionConfig.sellerSignals) and auctionSignals (auctionConfig.auctionSignals).
  • reporting_metadata: This refers to an object created in the Auction service based on the information known to the Auction service.

Seller service configurations

Server configurations are based on Terraform and is open sourced to Github repo for cloud deployment.

The configurations include environment variables and parameters that may vary per seller. These can be set by the seller in the configuration before deployment. The configurations also include urls that can be ingested when the service starts up for prewarming the connections.

Refer to the README for deployment on AWS or README for deployment on GCP. Refer to example config on AWS or example config on GCP for the Terraform config required for deployment to the cloud. The config requires update of some parameter values (that vary per adtech) before deployment to cloud.

Following are some examples of data configured in service configurations.

SellerFrontEnd service configurations

  • Seller Key/Value service endpoint (scoring_signals_url): This endpoint is configured in SellerFrontEnd service configuration and ingested at service startup to prewarm connections to seller's Key/Value service.

  • BuyerFrontEnd endpoint: The domain address of BuyerFrontEnd services operated by Buyers that this Seller has partnered with. This is ingested at service startup to prewarm connection.

  • Auction service endpoint: The domain address of Auction service. This is ingested at service startup to prewarm connection.

  • Seller origin: The origin of the seller.

    • Note: The seller origin information is also passed by the seller's ad Service in SelectAd request and SellerFrontEnd validates that with the origin information configured.

  • Map of {InterestGroupOwner, BuyerFrontEnd endpoint}: Map of InterestGroupOwner (buyer origin) to BuyerFrontEnd domain address.

  • Global timeout for Buyer: This information can be used to set a timeout on each buyer; however, will be overridden by the buyer_timeout_ms passed by the seller's ad service to SellerFrontEnd in SelectAd request.

  • Private Key Hosting service and Public Key Hosting service endpoints in key management systems.

Auction service configurations

  • Cloud Storage endpoint: The endpoint of Cloud Storage from where seller's code is hot reloaded by the Auction service.

  • Private Key Hosting service and Public Key Hosting service endpoints in key management systems.

Spec for DSP

generateBid()

The Bidding service exposes an API endpoint GenerateBids. The BuyerFrontEnd service sends GenerateBidsRequest to the Bidding service, that includes required input for bidding. The code for bidding, i.e. generateBid() is prefetched from Cloud Storage, cached and precompiled in Bidding service. After processing the request, the Bidding service returns the GenerateBidsResponse which includes bids that correspond to each ad, i.e. AdWithBid.

The function can be implemented in Javascript (or WASM driven by Javascript) or compiled into a standalone binary. The specifcation for both is described in detail below.

generateBid() Javascript/WASM spec

Note: If a DSP develops generateBid() following web platform's Protected Audience API explainer, that should also execute in Bidding service. However, certain updates will be required for payload optimization.

generateBid(interestGroup, auctionSignals, perBuyerSignals, trustedBiddingSignals,  deviceSignals) {
  ...
  return {'ad': adObject,
          'bid': bidValue,
          'render': renderUrl,
          'adComponents': ["adComponentRenderUrlOne", "adComponentRenderUrlTwo"],
          'allowComponentAuction': false};
 }
Arguments

  • interestGroup: The InterestGroup (Custom Audience) object. Refer InterestGroup data structure to understand what is sent in this object from the client.

    • The InterestGroup is serialized and passed to generateBid() exactly as-sent, except for the following divergences: ** DeviceSignals are serialized and passed separately, see below. ** component_ads are serialized in a field named adComponentRenderIds ** bidding_signals_keys are serialized in a field named trustedBiddingSignalsKeys to align with the On-Device interestGroup spec.
    • Note: To reduce payload over the network and further optimize latency, our goal is to minimize the information sent in this object. We will work with Adtechs for the long term to reduce the amount of information sent in this object and try to find a solution to fetch those on the server side.

  • auctionSignals: Contextual signal that is passed from seller's ad service to SellerFrontEnd in SelectAd request.

  • perBuyerSignals: Contextual signal generated by the buyer during Real Time Bidding that is passed from seller's ad service to SellerFrontEnd in SelectAd request.

  • trustedBiddingSignals: Real time signals fetched by BuyerFrontEnd service from Buyer's Key/Value service.

    • Note: Only the trustedBiddingSignals required for generating bid(s) for the interestGroup are passed to generateBid().

  • deviceSignals: This refers to browserSignals or androidSignals, built by the client (browser, Android). This includes Frequency Cap (statistics related to previous win of ads) for the user's device.

generateBid() Binary spec

The signature for the GenerateBid binary is specified as proto objects. That means, the input to the GenerateBid binary will be a proto object and the output will be a proto object. The function signature looks like this -

GenerateProtectedAudienceBidResponse generateBid(GenerateProtectedAudienceBidRequest);

The definition for these high level protos along with nested types is specified in the API code. This is different from the Javascript signature - the parameters and return values are encapsulated in high level proto objects. These differences are discussed as follows.

Arguments
  • GenerateProtectedAudienceBidRequest: This is the request object that encapsulates all the arguments for the generateBid() UDF (similar to the parameters in the JS spec for generateBid like interestGroup, deviceSignals, etc.).
message GenerateProtectedAudienceBidRequest {
  ProtectedAudienceInterestGroup interest_group = 1 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {description:
      'This will be prepared by the Bidding service based on the data received'
      ' in the BuyerInput from the device.'
}];

  string auction_signals = 2 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {description:
      'Auction signals are sent by the seller in the Auction Config. This can'
      ' be encoded any way by the seller and will be passed as-is to the'
      ' generateBid() UDF.'
}];

  string per_buyer_signals = 3 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {description:
      'Per buyer signals are sent by the seller in the Auction Config. This can'
      ' be encoded any way by the seller and will be passed as-is to the'
      ' generateBid() UDF.'
}];

  string trusted_bidding_signals = 4 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {description:
      'This will be passed as the JSON response received from the buyer\'s'
      ' key/value server.'
}];

  oneof ProtectedAudienceDeviceSignals {
    ProtectedAudienceAndroidSignals android_signals = 5 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {description:
        'This will be prepared by the Bidding server based on information'
        ' passed by the Android app.'
}];

    ProtectedAudienceBrowserSignals browser_signals = 6 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {description:
        'This will be prepared by the Bidding server based on information'
        ' passed by the browser on desktop or Android.'
}];

    ServerMetadata server_metadata = 7 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {description:
        'This will be prepared by the Bidding server and will contain config'
        ' information about the current execution environment.'
}];
  }
}
  • ServerMetadata: The server passes additional config information for the current execution in the ServerMetadata message. This will inform the binary if logging or debug reporting functionality is available for the current execution.
message ServerMetadata {
  option (privacysandbox.apis.roma.app_api.v1.roma_mesg_annotation) = {description:
      'Config information about the current execution environment for a'
      ' GenerateBidRequest.'
};

  bool debug_reporting_enabled = 1 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {description:
      'A boolean value which indicates if event level debug reporting is'
      ' enabled or disabled for the request. Adtechs should only return debug'
      ' URLs if this is set to true, otherwise the URLs will be ignored and'
      ' creating these will be wasted compute.'
}];

  bool logging_enabled = 2 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {description:
      'A boolean value which indicates if logging is enabled or disabled for'
      ' the request. If this is false, the logs returned from the RPC in the'
      ' response will be ignored. Otherwise, these will be outputted to the'
      ' standard logs or included in the response.'
}];
}
  • GenerateProtectedAudienceBidResponse: This is the response field expected from the generateBid UDF. It contains the bid(s) for ad candidate(s) corresponding to a single Custom Audience (a.k.a Interest Group) (similar to the return values from the JS spec for generateBid).
message GenerateProtectedAudienceBidResponse {
  repeated ProtectedAudienceBid bids = 1 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {description:
      'The generateBid() UDF can return a list of bids instead of a single bid.'
      ' This is added for supporting the K-anonymity feature. The maximum'
      ' number of bids allowed to be returned is specified by the seller. When'
      ' K-anonymity is disabled or not implemented, only the first candidate'
      ' bid will be considered.'
}];

  LogMessages log_messages = 2 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {description:
      'Adtechs can add logs to the response if logging was enabled in the'
      ' request. Logs will be printed out to the console in case of non-prod'
      ' builds and added to the server response in case of debug consented'
      ' requests.'
}];
}
  • DebugReportUrls: URLs to support debug reporting, when auction is won and auction is lost. There is no forDebuggingOnly method/API and the debug URLs for a bid have to be directly included by the binary in the proto response. These can be added in the DebugReportUrls field in the ProtectedAudienceBid proto. The format for the URLs stays the same as the browser definition and they will be pinged in the exact same way.
message DebugReportUrls {
   string auction_debug_win_url = 1 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = 
   {description:'URL to be triggered if the Interest Group wins the auction. If undefined'
      ' or malformed, it will be ignored.'
  }];

    string auction_debug_loss_url = 2 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = {
      description:'URL to be triggered if the Interest Group loses the auction. If'
        ' undefined or malformed, it will be ignored.'
  }];
}
  • LogMessages: The standard logs from the binary are not exported for now (This will be added later on in 2025). For now, any logs from the binary will be discarded. As a workaround, the GenerateProtectedAudienceBidResponse proto includes the log_messages field for logs and error messages.
message LogMessages {
  option (privacysandbox.apis.roma.app_api.v1.roma_mesg_annotation) = 
  {description: 'Logs, errors, and warnings populated by the generateBid() UDF.'};

  repeated string logs = 1 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = 
  {description: 'Optional list of logs.'}];

  repeated string errors = 2 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = 
  {description: 'Optional list of errors.'}];

  repeated string warnings = 3 [(privacysandbox.apis.roma.app_api.v1.roma_field_annotation) = 
  {description: 'Optional list of warnings.'}];
}

The logs, errors and warnings in this proto will be printed to the cloud logs in non_prod builds, and included in the server response in case of consented debug requests.

reportWin()

Event level win reporting would work with Bidding and Auction services and function signatures can be the same as described in web platform's Protected Audience API explainer.

ReportWin() will be executed in Auction service, reporting url for the buyer and registered ad beacons (for Fenced Frame reporting) would be generated in Auction service and returned to the client in encrypted AuctionResult. The client will ping the buyer's reporting endpoint using the reporting url.

Note: Refer to the event level reporting explainer for the detailed design.

reportWin(auctionSignals, perBuyerSignals, signalsForWinner, reporting_metadata) {
  ...
  registerAdBeacon({"click", clickUrl,"view", viewUrl});
  sendReportTo(reportWinUrl);
  return;
}
Arguments
  • auctionSignals: Contextual signal generated by the seller.
  • perBuyerSignals: Contextual signal generated by the buyer.
  • signalsForWinner: Object returned by seller's ReportResult().
  • reporting_metadata: This refers to an object created in the Auction service based on the information known to the Auction service.

Buyer BYOS Key/Value service

The BuyerFrontEnd service looks up biddingSignals from Buyer's BYOS Key/Value service. The base url (domain) for Key/Value service is configured in BuyerFrontEnd service so that the connection can be prewarmed. All lookup keys are batched together in a single lookup request. The lookup url and response are in the same format as described in Chrome Protected Audience explainer.

The lookup url is the following format:

<base_url>/getvalues?hostname=<publisher.com>&experimentGroupId=<kv_exp_id>&keys=<key_1>,..
<key_n>

Where <base_url>, <publisher.com>, <kv_exp_id>, key_1>,...<key_n> are substituted

Note: If keys are the same as InterestGroups names, then those are not looked up more than once.

The response is in the following format:

{ 'keys': {
      'key1': arbitrary_json,
      'key2': arbitrary_json,
      ...},
  'perInterestGroupData': {
      'name1': {
      },
      ...
  }
}

Note: The trustedBiddingSignals passed to generateBid() for an Interest Group (Custom Audience) is the value corresponding to each lookup key in the Interest Group but not the entire response. Following is an example, if key1 is the lookup key in an interest group, then the following is passed to generateBid() in trustedBiddingSignals.

  'key1': arbitrary_json
Filtering in buyer's Key/Value service

Filtering interest groups in buyer's Key/Value service can help reduce number of interest groups for bidding; and therefore optimize latency and reduce cost of Bidding service.

To support filtering of interest groups in buyer's BYOS Key/Value service, metadata received from the client will be forwarded in the HTTP request headers of the trustedBiddingSignals lookup request.

Refer to metadata forwarding for more details.

Buyer service configurations

Server configurations are based on Terraform and is open sourced to Github repo for cloud deployment.

The configurations will include environment variables and parameters that may vary per buyer. These can be set by the buyer in the configuration before deployment. The configurations also include urls that can be ingested when the service starts up for prewarming the connections.

Refer to the README for deployment on AWS or README for deployment on GCP. Refer to example config on AWS or example config on GCP for the Terraform config required for deployment to the cloud. The config requires update of some parameter values (that vary per adtech) before deployment to cloud.

Following are some examples of data configured in service configurations.

BuyerFrontEnd service configurations

  • Buyer's Key/Value service endpoint (bidding_signals_url): This endpoint is configured in BuyerFrontEnd service configuration and ingested at service startup to prewarm connections to buyer's Key/Value service.

  • Bidding service endpoint: The domain address of Bidding service. This is ingested at service startup to prewarm connection.

  • Private Key Hosting service and Public Key Hosting service endpoints in key management systems.

Bidding service configurations

  • Cloud Storage endpoint: The endpoint of Cloud Storage from where buyer's code is hot reloaded by the Bidding service.

  • Private Key Hosting service and Public Key Hosting service endpoints in key management systems.

Metadata forwarding

Metadata added by client

Browser will forward the following metadata in the request headers of unified request.

  • Accept-Language
  • User-Agent
  • IP / geo information

Metadata forwarded by seller's ad service

Seller's ad service will forward the metadata in the following non-standard HTTP headers in the request to SellerFrontEnd service.

  • X-Accept-Language
  • X-User-Agent
  • X-BnA-Client-IP

Note: If the seller's ad service uses a gRPC client to send request to SellerFrontEnd, the standard User-Agent header may be altered by gRPC. Due to that reason, non-standard HTTP headers are required so that it cannot be overridden or altered by the gRPC client.

Seller's ad service can do one of the following to forward the metadata:

  • If the seller's ad service sends SelectAd as HTTPS to SFE, the header can be forwarded.
  • If the seller's ad service sends SelectAd as gRPC, metadata needs to be created and added.

Metadata forwarded by SellerFrontEnd service

SellerFrontEnd will add metadata to gRPC request sent to BuyerFrontEnd service.

  • X-Accept-Language
  • X-User-Agent
  • X-BnA-Client-IP

SellerFrontEnd will forward the metadata in the request headers to seller's Key/Value service. Seller's Key/Value service may ingest this information to generate scoring signals. Seller's Key/Value service may use X-BnA-Client-IP header to monitor requests from TEE based SellerFrontEnd service.

  • Accept-Language
  • User-Agent
  • X-BnA-Client-IP

Metadata forwarded by BuyerFrontEnd service

BuyerFrontEnd will forward the metadata in the request headers to buyer's Key/Value service.

This may help with filtering of interest groups (custom audiences) in buyer's Key/Value service. Buyer's Key/Value service may use X-BnA-Client-IP header to monitor requests from TEE based BuyerFrontEnd service.

  • Accept-Language
  • User-Agent
  • X-BnA-Client-IP

High level design

Architecture diagram.

Note:

  • All arrows in the diagram are bidirectional, implying request and response.
  • Based on feedback from Adtechs, we may incorporate further optimization in the flow while ensuring it is privacy safe.

Seller's code (in the Publisher web page or app) sends one unified request for contextual and Protected Audience auctions to seller's ad service.

Then the seller’s ad service makes two sequential requests.

  • [Existing flow] The seller may send real-time bidding (RTB) requests to a select partner buyers for contextual bids, then conducts the contextual auction.
  • [Server side Protected Audience flow] The seller sends a SelectAd request to SellerFrontEnd service to start the Protected Audience auction if seller determines there is incremental value in conducting the auction. The request payload includes encrypted ProtectedAudienceInput, AuctionConfig and other required information. The encrypted ProtectedAudienceInput can only be decrypted by an attested service running in trusted execution environment, in this case the SellerFrontEnd service.

Unified Contextual and Protected Audience Auctions Flow is important to optimize e2e auction latency.

Unified request

A seller’s single request for the contextual and server-side Protected Audience auction, from their code on a publisher site or app. The request includes contextual payload and encrypted ProtectedAudienceInput. The seller's code on the publisher site or app calls the client API to get the encrypted ProtectedAudienceInput before sending the unified request.

Components

Browser API for Bidding and Auction services

Refer to browser API and integration design.

Sell-side platform (SSP) system

The following are the Protected Audience services that will be operated by an SSP, also referred to as a Seller.

Seller's ad service

With the Unified Contextual and Protected Audience Auction flow, the seller's ad service will receive one request from the client. The request would include contextual request payload and encrypted ProtectedAudienceInput from the client.

The encrypted ProtectedAudienceInput includes Interest Group (Custom Audience) information on the user's device. The size of ProtectedAudienceInput is required to be small; and there may be a per-buyer size limit set by the client or the seller. Refer to payload optimization explainer for the guidance around optimizing ProtectedAudienceInput payload size.

Refer to more details here.

SellerFrontEnd service

The front-end service of the system that runs in the trusted execution environment on a supported cloud platform. The service receives requests from seller's ad service to initiate Protected Audience auction flow. Then the service orchestrates requests (in parallel) to Buyers / DSPs participating in the auction for bidding.

This service also fetches real-time scoring signals required for the auction and calls Auction service for Protected Audience auction.

Refer to more details here.

Auction service

The Auction service runs in the trusted execution environment on a supported cloud platform. This service responds to requests from the SellerFrontEnd service and doesn't have access to arbitrary untrusted endpoints.

The Auction service prefetches code blobs owned by seller from Cloud Storage (or an endpoint provided by the seller). The code is prefetched at service startup, periodically thereafter and cached. More than one code version can be supported to facilitate experimentation by adtechs.

SSP's code for scoring ads can be written in JavaScript and / or WebAssembly (WASM). The code runs in a custom V8 sandbox within the TEE that has tighter security restrictions; that can not log information or has no disk or network access in production mode. For a ScoreAds request from SellerFrontEnd, SSP's scoring code is executed per ad within a separate V8 worker thread but all execution can happen in parallel. Between two executions in the service, there is no state saved.

Note: The hosting environment protects the confidentiality of the Seller's code, if the execution happens only in the cloud.

Refer to more details here.

Seller's Key/Value service

A seller's Key/Value service is a critical dependency for the auction system. The Key/Value service receives requests from the SellerFrontEnd service. The service returns real-time seller data required for auction that corresponds to lookup keys available in buyers' bids (such as ad_render_urls or ad_component_render_urls).

Note: The Seller Key/Value system may be BYOS Key/Value Service or trusted Key/Value Service depending on the timeline.

Demand-side platform (DSP) system

This section describes Protected Audience services that will be operated by a DSP, also called a buyer.

BuyerFrontEnd service

The front-end service of the system that runs in the trusted execution environment on a supported cloud platform. This service receives requests to generate bids from a SellerFrontEnd service. This service fetches real-time bidding signals that are required for bidding and calls Bidding service.

Refer to more details here.

Bidding service

The Bidding service runs in the trusted execution environment on a supported cloud platform. This service responds to requests from BuyerFrontEnd service and doesn't have access to arbitrary untrusted endpoints.

The Bidding service prefetches code blobs owned by the buyer from Cloud Storage (or an endpoint provided by the seller). The code is prefetched at service startup, periodically thereafter and cached. More than one code version can be supported to facilitate experimentation by adtechs.

Buyer's code for generating bids can be written in JavaScript and / or WebAssembly (WASM). The code runs in a custom V8 sandbox within the TEE that has tighter security restrictions; that can not log information or has no disk or network access in production. For a GenerateBids request from BuyerFrontEnd, buyer's code is executed per Interest Group (Custom Audience) within a separate V8 worker thread but all execution can happen in parallel. Between two executions in the service, there is no state saved.

Note: This environment protects the confidentiality of a buyer's code, if the execution happens only in the cloud.

Refer to more details here.

Buyer’s Key/Value service

A buyer's Key/Value service is a critical dependency for the bidding system. The Key/Value service receives requests from the BuyerFrontEnd service. The service returns real-time buyer data required for bidding, corresponding to lookup keys.

Note: The buyer’s Key/Value system may be BYOS Key/Value Service or trusted Key/Value service depending on timeline.

Flow

  • Clients (browser, Android) builds encrypted ProtectedAudienceInput.

    • Client prefetch a set of public keys from the key management systems in the non request path every 7 days. The public keys are used for encrypting ProtectedAudienceInput.

    • This encrypted data includes Interest Group (Custom Audience) information for different buyers, i.e. BuyerInput. On the client, the BuyerInput(s) are compressed, then ProtectedAudienceInput is encrypted and padded. The padding may be done such that the size of padded ProtectedAudienceInput ciphertext falls in one of the 7 different size buckets.

      • From privacy perspective, 7 size buckets is chosen because this would leak less than 3 bits of information in the worst case.

  • Seller's code in publisher page on the browser sends to seller's ad service.

    • Web: Seller's code in publisher webpage on the browser sends HTTP request to (untrusted) seller's ad service.

      • [Existing request, but excluding 3P Cookie] Contextual payload.
      • Seller's code in publisher webpage asks browser for encrypted ProtectedAudienceInput to be included in request.
      • Client adds metadata to HTTP request headers.

    • Android: Seller's code in publisher SDK in Android sends HTTP request to (untrusted) seller's ad service.

      • [Existing request] Contextual payload.
      • Seller's code in publisher SDK asks Android for encrypted ProtectedAudienceInput to be included in request.

  • Seller's ad service makes two requests.

    • [Existing flow] May send Real Time Bidding (RTB) requests to partner buyers for contextual bids and then conduct a contextual auction to select a contextual ad winner.

    • Sends SelectAd request to SellerFrontEnd service if there is incremental value in conducting the Protected Audience auction. The request payload includes encrypted ProtectedAudienceInput, AuctionConfig and other required information.

      • Encrypted ProtectedAudienceInput should be Base64 encoded string of bytes if SelectAd request is sent as HTTPS request.
      • AuctionConfig includes contextual signals like seller_signals, auction_signals; per buyer signals / configuration and other data. Contextual ad winner may be part of seller_signals.
        • If the SelectAd request is sent after contextual auction concludes, the seller_signals may include information about the contextual auction winner that can filter Protected Audience bids during Protected Audience auction.
      • Forwards client metadata in non-standard HTTP headers in the request to SellerFrontEnd service.

    Note: It is upto the seller / SSP to decide whether SelectAd request should be sent after contextual / RTB auction concludes or in parallel with contextual auction. The seller's ad server may incorporate traffic shaping and determine incremental value / demand for calling SellerFrontEnd for an ad request. The seller's ad server may call SellerFrontEnd while the contextual auction is running and this can optimize overall auction latency even further; however in this case contextual ad winner can not take part in Protected Audience auction to filter Protected Audience bids. If contextual signals (buyer_signals, seller_signals and auction_signals) are required for bidding, auction and reporting, that should be sent in SelectAdRequest.

  • Protected Audience auction kicks off in Bidding and Auction Services.

    • The SellerFrontEnd service decrypts encrypted ProtectedAudienceInput using decryption keys prefetched from key management systems.

    • The SellerFrontEnd service orchestrates GetBids requests to participating buyers’ BuyerFrontEnd services in parallel.

    • Within each buyer system:

      • The BuyerFrontEnd service decrypts GetBidsRequest using decryption keys prefetched from key management systems.
      • The BuyerFrontEnd service fetches real-time data (trustedBiddingSignals) from the buyer’s Key/Value service required for generating bids.
      • The BuyerFrontEnd service sends a GenerateBids request to the Bidding service.
      • The Bidding service returns ad candidates with bid(s) for each InterestGroup.
        • The Bidding service deserializes and splits trustedBiddingSignals such that generateBid() execution for an InterestGroup can only ingest trustedBiddingSignals for the interest group / bidding signal key.
      • The BuyerFrontEnd returns all bid(s) (AdWithBid) to SellerFrontEnd.

    • Once SellerFrontEnd has received bids from all buyers, it requests real-time data (trustedScoringSignals) for all ad_render_urls and ad_component_render_urls (corresponding to ad with bids) from the seller’s Key/Value service. These signals are required to score the ads during Protected Audience auction.

    • SellerFrontEnd sends a ScoreAdsRequest to the Auction service to score ads and selects a winner.

      • Auction service deserializes and splits scoring signal such that scoreAd() execution for an ad can only ingest trustedScoringSignals for the ad.

    • The Auction service selects the winning ad, generates reporting urls and returns ScoreAdsResponse to SellerFrontEnd service.

    • SellerFrontEnd returns winning ad, other metadata and reporting urls as an encrypted AuctionResult back to seller's ad service.

    • There are a few cases when a fake, padded, encrypted AuctionResult will be returned to the client. For such cases, contextual ad winner should be rendered on the client.

      • The is_chaff field in AuctionResult will be set and that would indicate to the client that this is a fake Protected Audience auction result. Following are the possible scenarios when this would be set.

        1. Protected Audience auction returns no winner; this may be a possible scenario when contextual ad winner (part of seller_signals) participate in Protected Audience auction to filter all bids. Therefore, the contextual ad wins and should be rendered on the client.
        2. When none of the participating buyers in the auction return any bid.

      • The error field in AuctionResult is for the client to ingest and handle. Following are the possible scenarios when this would be set.

        1. The required fields in ProtectedAudienceInput doesn't pass validation.
        2. There is failure in de-compression of BuyerInput.
        3. The private keys (decryption keys) are compromised and revoked. In this
          scenario, ProtectedAudienceInput ciphertext can be decrypted with a compromised key but the request won't be processed further. The error field would be set in AuctionResult to indicate to the client this corresponding public key with the same version shouldn't be used again for encryption. The AuctionResult will be encrypted and returned in the response.

    • SellerFrontEnd will propagate downstream service errors back to seller's ad service and will also return gRPC / HTTP errors when required fields in SelectAdRequest.AuctionConfig are missing.

  • Seller's ad service returns the encrypted AuctionResult back to the client.

    • Contextual ad and / or encrypted ProtectedAudienceInput will be sent back to the client. In case the contextual ad wins, a padded (chaff) but encrypted AuctionResult will be sent in response.

  • Seller's code in publisher web page or app receives the response from seller's ad service and passes the encrypted AuctionResult (auction_result_ciphertext) to the client.

  • Only the client (browser, Android) would be able to decrypt AuctionResult ciphertext.

  • Ad is rendered on the device.

Client <> server and server <> server communication

Client <> seller ad service communication

Client would send a HTTPS request (unified request) to seller's ad service.

The ProtectedAudienceInput included in the unified request will be encrypted on the client using a protocol called Oblivious HTTP that is based on bidirectional Hybrid Public Key Encryption(HPKE). The Protected Audience response, i.e. AuctionResult will also be encrypted in SellerFrontEnd using Oblivious HTTP.

The seller's ad service will not be able to decrypt or have access to ProtectedAudienceInput or AuctionResult in plaintext.

Data format
Compression, encryption, padding
  • For both web and Android, the BuyerInput(s) in ProtectedAudienceInput will be compressed. Then ProtectedAudienceInput will be encrypted and padded. An exponential padding scheme will be used.

Seller ad service <> SellerFrontEnd communication

Seller's ad service can send gRPC or HTTPS request to SellerFrontEnd service. There would be an Envoy Proxy service instance hosted with SellerFrontEnd for HTTPS to gRPC translation.

  • If seller's ad service sends HTTPS request to SellerFrontEnd, ProtectedAudienceInput ciphertext should be Base64 encoded; similarly the response to seller's ad service would be Base64 encoded. This encoding is not required if seller's ad service and SellerFrontEnd communication is gRPC.

The communication between seller's ad service and SellerFrontEnd service would be protected by TLS / SSL that provide communications security by encrypting data sent over the untrusted network to an authenticated peer.

Communication between Bidding and Auction Services

All communication between services running in trusted execution environment is protected by TLS / SSL and the request and response payloads are encrypted using bidirectional Hybrid Public Key Encryption(HPKE).

The TLS / SSL session terminates at the load balancer or the first hop in-front of a service, therefore the data over the wire from the load balancer to service needs to be protected; hence the request-response is end-to-end encrypted using bidirectional HPKE.

For client metadata forwarded in the requests, refer here.

Payload compression

Most request/response payload sent over the wire should be compressed.

  • The BuyerInput(s) in ProtectedAudienceInput will be compressed on the client using gzip. The payload needs to be compressed first and then encrypted. In SellerFrontEnd, ProtectedAudienceInput will be decrypted first and then decompressed.

  • Seller's ad service can compress SelectAd request payload when calling SellerFrontEnd service to save network bandwidth cost and reduce latency; gzip is accepted by SellerFrontEnd service. The AuctionResult will be compressed using gzip, then encrypted and returned in SelectAdResponse.

  • Request to BuyerFrontEnd service from SellerFrontEnd will be compressed first using gzip and then encrypted. In BuyerFrontEnd service, the request will have to be decrypted first and then decompressed.

    • Note: This would be similar for communication between BuyerFrontEnd <> Bidding services and SellerFrontEnd <> Auction services.

  • For Key/Value Server HTTP lookup, the Accept-Encoding HTTP request header would be set to specify the correct compression algorithm, i.e. gzip. The Key/Value server may set the Content-Encoding Representation Header to specify the content encoding (i.e. gzip) used to compress the response payload.

    • Note:
      • It is recommended to compress Key/Value server response to optimize Key/Value lookup latency and reduce network bandwidth cost for adtechs.
      • The request payload to Key/Value service need not be compressed given the size is expected to be small.
      • The request-response payload between SellerFrontEnd <> seller's Key/Value services and BuyerFrontEnd <> buyer's Key/Value services do not require additional encryption using HPKE. However, the communication between these services is over TLS that provide communications security by encrypting data sent over the untrusted network to an authenticated peer.

Payload optimization

The size of compressed ProtectedAudienceInput should be small. Refer to payload optimization explainer for more details.

Service code and framework

Bidding and Auction services are developed in C++. The service configurations required for cloud deployment are based on Terraform.

The service framework is based on gRPC. gRPC is an open source, high performance RPC framework built on top of HTTP2 that is used to build scalable and fast APIs. gRPC uses protocol buffers as the interface description language and underlying message interchange format.

Bidding and Auction services code and configurations are open sourced to Github repo.

Adtech code

Adtech code for generateBid(), scoreAd(), reportResult(), reportWin() can follow the same signature as described in the Protected Audience API for the browser.

Note:

  • Code can be Javascript only or WASM only or WASM instantiated with Javascript.
  • If the code is in Javascript, then Javascript context is initialized before every execution.
  • No limit on code blob size.
  • More than one version of code can be supported to facilitate adtech experimentation.
  • Adtech can upload their code to Cloud Storage supported by the Cloud Platform.
  • Code is prefetched by Bidding / Auction services running in trusted execution environment from the Cloud Storage bucket owned by adtech.

Refer to more details here.

Cloud deployment

Bidding and Auction services are deployed by adtechs to a public cloud platform so that they are co-located within a cloud region.

Servers can be replicated in multiple cloud regions and the availability Service-Level-Objective (SLO) will be decided by adtechs.

SSP system

There will be a Global Load balancer for managing / routing public traffic to SellerFrontEnd service. Traffic between SellerFrontEnd and Auction service would be over private VPC network. To save cost, SellerFrontEnd and Auction server instances will be configured in a service mesh.

DSP system

There will be a Global Load balancer for managing / routing public traffic to BuyerFrontEnd services. Traffic between BuyerFrontEnd and Bidding service would be over private VPC network. To save cost, BuyerFrontEnd and Bidding server instances will be configured in a service mesh.

Logging

Debug / non-prod build

Bidding and Auction server logs will be available with debug (non-prod) build / mode. The debug binaries can be built with higher level of verbose logging. For GCP, these logs will be exported to Cloud Logging.

The context logger in Bidding and Auction servers supports logging generation_id passed by the client in encrypted ProtectedAudienceInput and optional (per) buyer_debug_id and seller_debug_id passed in SelectAdRequest.AuctionConfig for an ad request. The adtech_debug_id (buyer_debug_id or seller_debug_id) can be an internal log / query id used in an adtech's non TEE based systems and if available can help the adtech trace the ad request log in Bidding and Auction servers and map with the logs in their non TEE based systems.

Logs from adtech's code can be made available in debug mode.

Production build

Bidding and Auction servers will support safe logging in production mode and the logs will be exported to Cloud Logging / Cloud Watch. Refer Debugging Protected Audience API services for more details.

Logs from adtech's code can be made available if adtech / user consented debugging is enabled.

Dependencies

Through techniques such as prefetching and caching, the following dependencies are in the non-critical path of ad serving.

Key management systems

The key management systems are required for Protected Audience service attestation and cryptographic key generation. Learn more in the Overview of Protected Audience Services Explainer. The key management systems will be deployed to all supported public clouds. Services in the key management systems will be replicated in multiple cloud regions.

All services running in TEE prefetch encryption and decryption keys from key management systems at service startup and periodically in the non critical path. All communication between a service in TEE and another service in TEE is end-to-end encrypted using Hybrid Public Key Encryption and TLS. Refer here for more details.

Service APIs

Refer to Bidding and Auction services APIs in open source repo.

Client <> server data

Following section describes the data that flows from client (e.g. browser, Android) to Bidding and Auction Services through Seller Ad service and the data received by client from Bidding and Auction Services.

ProtectedAudienceInput

ProtectedAudienceInput is built and encrypted by client (browser, Android). Then sent to Seller Ad service in the unified request. This includes per BuyerInput and other required data.

Refer to ProtectedAudienceInput message.

BuyerInput

BuyerInput is part of ProtectedAudienceInput. This includes data for each buyer / DSP.

Refer to BuyerInput message.

BrowerSignals

Information about an Interest Group known to the browser. These are required to generate bid.

Refer to BrowserSignals message.

AndroidSignals

Information passed by Android for Protected Audience auctions. This will be updated later.

Refer to AndroidSignals message.

AuctionResult

Protected Audience auction result returned from SellerFrontEnd service to the client through the Seller Ad service. The data is encrypted by SellerFrontEnd service and decrypted by the client. The Seller Ad service will not be able to decrypt the data.

In case the contextual ad wins, an AuctionResult will still be returned that includes fake data and has is_chaff field set to true. Clients should ignore AuctionResult after decryption if is_chaff is set to true.

Refer to AuctionResult message.

Public APIs

SellerFrontEnd service and API endpoints

The SellerFrontEnd service exposes an API endpoint (SelectAd). The Seller Ad service would send a SelectAd RPC or HTTPS request to SellerFrontEnd service. After processing the request, SellerFrontEnd would return a SelectAdResponse that includes an encrypted AuctionResult.

The AuctionResult will be encrypted in SellerFrontEnd using Oblivious HTTP that is based on bidirectional HPKE.

Refer to the API.

LogContext

Context for logging requests in Bidding and Auction servers. This includes generation_id passed by the client in encrypted ProtectedAudienceInput and optional (per) buyer_debug_id and seller_debug_id passed in SelectAdRequest.AuctionConfig. The adtech_debug_id (buyer_debug_id or seller_debug_id) can be an internal log / query id used in an adtech's non TEE based systems and if available can help the adtech trace the ad request log in Bidding and Auction servers and map with the logs in their non TEE based systems.

Refer to the LogContext message.

BuyerFrontEnd service and API endpoints

The BuyerFrontEnd service exposes an API endpoint GetBids. The SellerFrontEnd service sends encrypted GetBidsRequest to the BuyerFrontEnd service that includes BuyerInput and other data. After processing the request, BuyerFrontEnd returns GetBidsResponse, which includes bid(s) for each Interest Group. Refer to AdWithBid for more information.

The communication between the BuyerFrontEnd service and the SellerFrontEnd service is TEE to TEE communication and is end-to-end encrypted using HPKE and TLS/SSL. The communication will happen over public network and that can also be cross cloud networks.

Refer to the API.

AdWithBid

The AdWithBid for an ad candidate, includes ad (i.e. ad metadata), bid, render (i.e. ad render url), allow_component_auction and interest_group_name. This is returned in GetBidsResponse by BuyerFrontEnd to SellerFrontEnd.

Refer to the AdWithBid message.

Internal API

Internal APIs refer to the interface for communication between Protected Audience services within a SSP system or DSP system.

Bidding service and API endpoints

The Bidding service exposes an API endpoint GenerateBids. The BuyerFrontEnd service sends GenerateBidsRequest to the Bidding service, that includes required input for bidding. The code for bidding is prefetched from Cloud Storage and cached in Bidding service. After processing the request, i.e. generating bids, the Bidding service returns the GenerateBidsResponse to BuyerFrontEnd service.

The communication between the BuyerFrontEnd service and Bidding service occurs between each service’s TEE and request-response is end-to-end encrypted using HPKE and TLS/SSL. The communication also happens over a private VPC network.

Refer to the API.

Auction service and API endpoints

The Auction service exposes an API endpoint ScoreAds. The SellerFrontEnd service sends a ScoreAdsRequest to the Auction service for running auction; ScoreAdsRequest includes bids from each buyer and other required signals. The code for auction is prefetched from Cloud Storage and cached in Auction service. After all ads are scored, the Auction service picks the highest scored ad candidate and returns the score and other related data for the winning ad in ScoreAdsResponse.

The communication between the SellerFrontEnd service and Auction service occurs within each service’s TEE and request-response is end-to-end encrypted using HPKE and TLS/SSL. The communication also happens over a private VPC network.

Refer to the API.

WinReporting Urls

Refer to WinReportingUrls message.

DebugReporting Urls

Refer to DebugReportingUrls message.