Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for com.apple.fps.3_0 #2630

Closed
wants to merge 6 commits into from

Conversation

valotvince
Copy link
Contributor

@valotvince valotvince commented Apr 7, 2020

This PR will...

Add support for Fairplay w/ MSE

Why is this Pull Request needed?

Based on a feature request @BedrockStreaming needed to capture EXT-X tags in the manifest so we could do Client-side Ad Insertion on Mac / Safari

Are there any points in the code the reviewer needs to double check?

  • EME Controller handles both Widevine & Fairplay correctly

Resolves issues:

Checklist

  • changes have been done against master branch, and PR does not conflict
  • new unit / functional tests have been added (whenever applicable)
  • API or design changes are documented in API.md

@@ -344,6 +412,10 @@ class EMEController extends EventHandler {
logger.log(`Generating key-session request for "${initDataType}" init data type`);
keysListItem.mediaKeysSessionInitialized = true;

if (initDataType === 'sinf') {
keysListItem.mediaKeyId = this._findKeyInSinf(initData);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ Don't know if we should pass the whole tenc box here so any integrator could retrieve all the info (IV & encrypted info + ...) + could do their own transformation on the encrypted info from binary data

In my case that's the content keyId, so I guess I'll need to change the name because it depends on the integrator ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a generic thing? getting the mediaKeyId from the tenc in order to send it in the XHR headers?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it depends on each DRM Provider because some would support sinf initDataType while others don't... In my use case, I need it but others might not need it at all.

Copy link
Collaborator

@itsjamie itsjamie Apr 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, I'm getting to the point with a DRM solution for a video-engine layer like hls.js, if the best solution would be to get out of everyone's way, and simply offer a callback that a user can return a Promise from which we can use when it resolves to setMediaKeys on the video element. You can build on top of this incredibly simple design for each different DRM provider, but the core logic doesn't need to concern itself with this information.

We detect when the browser has signalled through the encrypted event, or the manifest has signalled through tags that it is encrypted, and provide the raw information to the user. They become responsible for providing the completed handshake which we can setMediaKeys with. We could still do the navigator key system check and provide this metadata, or not. I'm not convinced if we should or not, given that the priority of which key system gets used can change based on business arrangements.

I feel like we can offer built-in pure functions for the simpler DRM cases, but then there is nothing preventing a vendor from getting their custom need met.

I think this is inline with what Oren is suggesting about abstracting, but we could go even further and at the most "core" hls logic, completely abstract how a media key is resolved, and kick that out as a user-provided interface. This will then allow DRM support across the board, without blocking anyone, and we can continue to iterate on support for individual DRM vendors / systems to reduce duplication of effort, and complexity.

@robwalch @OrenMe @valotvince Thoughts on this as an approach?

const xhr = new XMLHttpRequest();
const licenseXhrSetup = this._licenseXhrSetup;
const xhrSetupData : LicenseXHRAdditionalData = { keyId: keysListItem.mediaKeyId };
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should allow integrators to populate xhr headers / post info with correct data depending of their DRM providers

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is keyId in header a generic FPS 3.0 requirement? If it is not then this is specific per a DRM service and should be abstracted.
Also the type LicenseXHRAdditionalData is for Fairplay but this is a method that handles all DRM systems so not sure this is fitting.
In general having a getter in every function that needs to know about the type of DRM system might get dirty as more implementation details will surface.
Might be better to rethink the whole stack - for instance:

  1. have a generic interface to handle DRM and then write adapters for each DRM system, which should be quite easy as they all adhere to CDM interface and differ in internal non spec-ed per DRM system configs.
  2. Use the network stack already built - xhr-loader - this will give access to same retry config and logic and performance metrics and is better suited to future work for adding fetch support.
    3.maybe now it is a good time to expose a some sort of mechanism to add middleware like to request/response instead of todays xhrSetup or loaders config. Shaka are doing it and it is quite nice. Of course it will require to use same loader as mentioned in handle xhr timeout and i/o errors #2

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my answer above, i think it's more of a provider constraint than a requirement for FPS 3.0

this._attemptKeySystemAccess(KeySystems.WIDEVINE, audioCodecs, videoCodecs);
console.log(data.levels);

this._attemptKeySystemAccess(KeySystems.FAIRPLAY, audioCodecs, videoCodecs);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆘 From there, I don't really know how to select the good keySystem, based on what gives the manifest, any ideas on this ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think based on manifest metadata is best as you will have the key system type there.
You will have to add parser support for that.
Do you have a sample manifest to use?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have one sample that works but I should rethink of #2606 so the encrypt information is still flagged but not loaded as done today

@@ -4,6 +4,7 @@
export enum KeySystems {
WIDEVINE = 'com.widevine.alpha',
PLAYREADY = 'com.microsoft.playready',
FAIRPLAY = 'com.apple.fps.3_0'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ I don't really know the differences between 2_0 et 3_0 (only that from 2_0 we can use MSE)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you have a sample that uses com.apple.fps.3_0 and that is actually supported on Safari for any given domain?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a working sample with our DRM provider (DRMToday), but it only works with assets having one encryption key. Assets with multiple encryption keys doesn't work yet but it seems to be a provider limitation rather than a Safari one.

@valotvince
Copy link
Contributor Author

Hi @robwalch :)
I have a few questions before going on with the rest of the implem and the tests, do you mind taking a look at them whenever you have the time to ? Thanks !

src/controller/eme-controller.ts Show resolved Hide resolved
const xhr = new XMLHttpRequest();
const licenseXhrSetup = this._licenseXhrSetup;
const xhrSetupData : LicenseXHRAdditionalData = { keyId: keysListItem.mediaKeyId };
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is keyId in header a generic FPS 3.0 requirement? If it is not then this is specific per a DRM service and should be abstracted.
Also the type LicenseXHRAdditionalData is for Fairplay but this is a method that handles all DRM systems so not sure this is fitting.
In general having a getter in every function that needs to know about the type of DRM system might get dirty as more implementation details will surface.
Might be better to rethink the whole stack - for instance:

  1. have a generic interface to handle DRM and then write adapters for each DRM system, which should be quite easy as they all adhere to CDM interface and differ in internal non spec-ed per DRM system configs.
  2. Use the network stack already built - xhr-loader - this will give access to same retry config and logic and performance metrics and is better suited to future work for adding fetch support.
    3.maybe now it is a good time to expose a some sort of mechanism to add middleware like to request/response instead of todays xhrSetup or loaders config. Shaka are doing it and it is quite nice. Of course it will require to use same loader as mentioned in handle xhr timeout and i/o errors #2

this._attemptKeySystemAccess(KeySystems.WIDEVINE, audioCodecs, videoCodecs);
console.log(data.levels);

this._attemptKeySystemAccess(KeySystems.FAIRPLAY, audioCodecs, videoCodecs);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think based on manifest metadata is best as you will have the key system type there.
You will have to add parser support for that.
Do you have a sample manifest to use?

@@ -4,6 +4,7 @@
export enum KeySystems {
WIDEVINE = 'com.widevine.alpha',
PLAYREADY = 'com.microsoft.playready',
FAIRPLAY = 'com.apple.fps.3_0'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you have a sample that uses com.apple.fps.3_0 and that is actually supported on Safari for any given domain?

@singhcool
Copy link

@valotvince Hla fair play support is completed? Can you give some demo source code?

@valotvince
Copy link
Contributor Author

@singhcool Hi :)

This was complete, I was able to make it work with a single-key but not with a multi-key encrypted video... It should only be a limitation from my DRM provider.

    this.hls = new Hls({
      emeEnabled: true,
      licenseXhrSetup: async (xhr, url, { mediaKeySystemDomain }) => {
        const {
          drm: {
            token: { handler },
          },
        } = this.options;

        if (mediaKeySystemDomain === 'com.apple.fps.3_0') {
          const token = await handler();

          xhr.setRequestHeader('x-dt-auth-token', token.value);
          xhr.setRequestHeader('content-type', 'application/x-www-form-urlencoded');
        }

        return xhr;
      },
      emeGenerateLicenseChallengeFunc: ({ mediaKeySystemDomain, mediaKeyInitData }, message) => {
        if (mediaKeySystemDomain === 'com.apple.fps.3_0') {
          const spc = btoa(String.fromCharCode.apply(null, new Uint8Array(message)))
            .replace(/\+/g, '-')
            .replace(/\//g, '_');

          const keyId = binaryToHex(contentIdFromInitData(new Uint8Array(mediaKeyInitData)));

          return `spc=${spc}&keyId=${keyId}`;
        }

        return message;
      },
      fairplayCertificateUrl: '...',
      fairplayLicenseUrl: '...',
    });	

Where contentIdFromInitData is a function retrieving the keyId from the schi > tenc mp4 box, to give it directly to the DRM provider (useful in my case, not necessarily in yours).

Hope that helps !

Best,

@singhcool
Copy link

singhcool commented Aug 31, 2020

@valotvince Thank you. If i need any support during integration, please help me. Advance Thanks ! :)

@robwalch robwalch added the DRM label Apr 6, 2021
@valotvince valotvince closed this Dec 28, 2021
@valotvince valotvince deleted the feat-fps-3_0 branch December 28, 2021 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Development

Successfully merging this pull request may close these issues.

5 participants