Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for com.apple.fps keySystem #4474

Merged

Conversation

valotvince
Copy link
Contributor

@valotvince valotvince commented Dec 28, 2021

This PR will...

Add support for com.apple.streamingkeydelivery KEYFORMAT with MSE (some copy/paste from #2630)
It brings several changes that would need be separated in other PRs (but to have a MVP, I chose to make it inside one PR)

  • licenseXhrSetup can return a Promise (backward-compatible with before)
  • Pass current keySystem to licenseXhrSetup & licenseResponseCallback to allow developers to do different treatment to xhr and data depending on the used keySystem
  • drmSystems configuration for any supported keySytem (backward-compatible with widevineLicenseUrl
  • fetch and set serverCertificate when the URL is specified inside drmSystems
  • Specific error when session.update rejects

To make it work with my own DRMProvider, I had to write that configuration on my app, hope this helps developers when integrating fps keySystem (maybe add it somewhere it the configuration ?)

{
  licenseResponseCallback: (xhr, url, keySystem) => {
    if (keySytem === 'com.apple.fps') {
      const utf8decoder = new TextDecoder();
      const base64 = utf8decoder.decode(xhr.response);
      const decoded = window.atob(base64);
  
      const result = new Uint8Array(decoded.length);
      for (let i = 0; i < decoded.length; i++) {
        result[i] = decoded.charCodeAt(i);
      }
  
      return result;
    }
  
    return xhr.response;
  },
  licenseXhrSetup: async (xhr, url) => {
    const { token: { handler: fetchToken } = {} } = this.options.drm;
  
    const token = await fetchToken();
  
    xhr.open('POST', url, true);
    xhr.setRequestHeader('x-dt-auth-token', token.value);
  },
}

⚠️ com.apple.fps keySystem correctly works with MSE when using fMP4 (or CMAF). It won't work properly when using mp2t with MSE, directly or when using mux.js (it might be different with the hls.js transmuxer, but I didn't took the opportunity to test it). See shaka-project/shaka-player#3776 for more information.

Still to be done and I might need tips to go faster 😄 :

Why is this Pull Request needed?

Because it allows to quit Safari's streaming black-box and use MSE/EME to play content

Are there any points in the code the reviewer needs to double check?

  • While this PR is in draft mode please don't take care of the files listed below, I had to add them to test it on my app, for unknown reasons yarn link doesn't work properly with hls.js
    • dist/*
    • .gitignore
    • package.json
    • demo/index.html
  • Widevine DRM decryption should still work as expected

Resolves issues:

Checklist

  • changes have been done against master branch, and PR does not conflict
  • new unit / functional tests have been added (whenever applicable)
  • API or design changes are documented in API.md

@robwalch
Copy link
Collaborator

robwalch commented Jul 16, 2022

Hi @valotvince,

Would you be able to resolve the conflicts in the PR? I know it's been a while. I'll be putting together a working branch for DRM work soon, and think these changes (minus the working files (dist/*)) provide a really nice and baseline for a future release with multi-DRM support.

@valotvince
Copy link
Contributor Author

Hi @robwalch
Sure will do next week :)

@stale stale bot removed the Stale label Jul 16, 2022
@@ -702,7 +832,8 @@ class EMEController implements ComponentAPI {
(videoCodec: string | undefined): videoCodec is string => !!videoCodec
);

this._attemptKeySystemAccess(KeySystems.WIDEVINE, audioCodecs, videoCodecs);
// TBD We should try a keySystem based on manifest information
this._attemptKeySystemAccess(KeySystems.FAIRPLAY, audioCodecs, videoCodecs);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For that part, what would be the best way you want it handled ?

Try all the keySystems and take the first one that works ?

Copy link
Collaborator

@robwalch robwalch Aug 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add a config option called keySystemPreference to specify which key-system is preferred (see below).

If not set there, we could map each key in _drmSystems (+_widevineLicenseUrl) to a key-system. If there is only one, we can assume that's the one we need to access. If more than one, then we could check them all, at this point there should only be one that works and that would be the one we want.

I think it would make more sense to not set this.mediaKeysPromise until onMediaEncrypted. At that point the playlist and media are loaded, and from that we could reduce how many calls to requestMediaKeySystemAccess we need to make, especially if the playlist keys tell us what decrypt method and keysystems can be used.

Let me know your thoughts. I'm making some assumptions looking at the code, but have a limited set of sample streams and license servers to work with at the moment.

### `keySystemPreference`

(default: `void 0`)

The value used to select the right key system. The playlist may signal multiple key systems, selection of the key system is done based on the preference specified here.
Possible values,

 - `fairplaystreaming`:  com.apple.streamingkeydelivery
 - `playready`:  com.microsoft.playready
 - `widevine`:  urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed
 - `clearkey`:  org.w3.clearkey
 - `undefined`: Select the key system from the first compatible EXT-X-KEY tag

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that for backwards compatibility I cannot merge this change as-is, as it breaks WIDEVINE support. If you can address this, perhaps checking access of both types but using the first that passes then I can take it from there.

Copy link
Collaborator

@robwalch robwalch Sep 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @valotvince,

The test branch I posted solves key-system access by selecting the key-system based on key formats found in the playlist or the systems present for the configured license URLs. Access is no longer attempted on ATTACH_MEDIA. It happens when the first key(s) for a segment are loaded. This is when available key formats are known which is mapped to key-systems to request access to. This happens later in the process of loading an HLS stream, so there could be a desire to request access earlier even if it means querying systems the stream does not have keys for. Session keys will be the first option if present in the manifest (#4927). The queries could be done on ATTACH_MEDIA based on licenses in the config, but I prefer there to avoid requesting access to key-systems for HLS streams that are not encrypted, even when emeEnabled is true and license urls are defined.

@@ -367,8 +367,10 @@ export default class M3U8Parser {
const decryptkeyformat =
keyAttrs.enumeratedString('KEYFORMAT') ?? 'identity';

// TBD we might need to check if we try to MSE playback mp2t content
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to throw an handled error if someone tries to read MP2T Fairplay-encrypted content ? Or do we only rely on documentation ?

If there's a need for an error to be thrown, I might need some guidance to see where I should do it :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need. We can rely on documented supported features in the README.

I want us to avoid any kind of file type detection as that has tended to be problematic.

@valotvince
Copy link
Contributor Author

👋 @robwalch The branch is rebased, still two things left to do but I'll wait for your input before going further

@@ -367,8 +367,10 @@ export default class M3U8Parser {
const decryptkeyformat =
keyAttrs.enumeratedString('KEYFORMAT') ?? 'identity';

// TBD we might need to check if we try to MSE playback mp2t content
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need. We can rely on documented supported features in the README.

I want us to avoid any kind of file type detection as that has tended to be problematic.

Comment on lines +198 to +199
if (keySystem === KeySystems.WIDEVINE && this._widevineLicenseUrl) {
return this._widevineLicenseUrl;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_widevineLicenseUrl can be replaced with _config.widevineLicenseUrl.
(remove private _widevineLicenseUrl?: string)

@@ -702,7 +832,8 @@ class EMEController implements ComponentAPI {
(videoCodec: string | undefined): videoCodec is string => !!videoCodec
);

this._attemptKeySystemAccess(KeySystems.WIDEVINE, audioCodecs, videoCodecs);
// TBD We should try a keySystem based on manifest information
this._attemptKeySystemAccess(KeySystems.FAIRPLAY, audioCodecs, videoCodecs);
Copy link
Collaborator

@robwalch robwalch Aug 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add a config option called keySystemPreference to specify which key-system is preferred (see below).

If not set there, we could map each key in _drmSystems (+_widevineLicenseUrl) to a key-system. If there is only one, we can assume that's the one we need to access. If more than one, then we could check them all, at this point there should only be one that works and that would be the one we want.

I think it would make more sense to not set this.mediaKeysPromise until onMediaEncrypted. At that point the playlist and media are loaded, and from that we could reduce how many calls to requestMediaKeySystemAccess we need to make, especially if the playlist keys tell us what decrypt method and keysystems can be used.

Let me know your thoughts. I'm making some assumptions looking at the code, but have a limited set of sample streams and license servers to work with at the moment.

### `keySystemPreference`

(default: `void 0`)

The value used to select the right key system. The playlist may signal multiple key systems, selection of the key system is done based on the preference specified here.
Possible values,

 - `fairplaystreaming`:  com.apple.streamingkeydelivery
 - `playready`:  com.microsoft.playready
 - `widevine`:  urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed
 - `clearkey`:  org.w3.clearkey
 - `undefined`: Select the key system from the first compatible EXT-X-KEY tag

@@ -702,7 +832,8 @@ class EMEController implements ComponentAPI {
(videoCodec: string | undefined): videoCodec is string => !!videoCodec
);

this._attemptKeySystemAccess(KeySystems.WIDEVINE, audioCodecs, videoCodecs);
// TBD We should try a keySystem based on manifest information
this._attemptKeySystemAccess(KeySystems.FAIRPLAY, audioCodecs, videoCodecs);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that for backwards compatibility I cannot merge this change as-is, as it breaks WIDEVINE support. If you can address this, perhaps checking access of both types but using the first that passes then I can take it from there.

@robwalch robwalch added this to the 1.3.0 milestone Aug 27, 2022
@robwalch robwalch changed the base branch from master to feature/drm-key-systems-with-fps August 31, 2022 16:07
@robwalch robwalch marked this pull request as ready for review September 2, 2022 21:46
@robwalch robwalch merged commit 6b484a8 into video-dev:feature/drm-key-systems-with-fps Sep 2, 2022
@valotvince valotvince deleted the feat-apple-fps branch September 5, 2022 08:16
@robwalch robwalch mentioned this pull request Sep 28, 2022
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

2 participants