feat: EME Logger Manifest V3 Migration

[beback4u]

EME Logger Manifest V3 Migration

Summary

The Chrome browser will no longer run Manifest V2 extensions after January 2023 as per The transition of Chrome extensions to Manifest V3. Hence, EME Logger's Manifest version needs to be updated from V2 to V3:

• log-window.js:
  • The background page is replaced with a service worker since the background page cannot be used with manifest v3.
  • chrome.extension.getURL is replaced with chrome.runtime.getURL.
  • Removed this line window.EmeLogWindow = EmeLogWindow; because window is not defined in the log_window.js since the script is running in a service worker (a separate thread). Now the singleton instance of EmeLogWindow (EmeLogWindow.instance) lives in the service worker.
  • Replaced window.open with chrome.windows.create to create a new window
  • chrome.browerAction is replaced with chrome.action.
  • Moved the logic handling DOM from the service worker (log-window.js) to the log window (log.js) and communicated between two scripts via chrome message handling APIs since a service worker cannot access the document or DOM object. As a result, EmeLogWindow.appendLog() and clear() code dealing with the document object has been moved to the log window script (log.js)
  • Used chrome.downloads.download APIwith adding "downloads" to the "permission" section in the manifest file since URL.createObjectURL is not available in a service worker.
• manifest.json:
  • web_accessible_resources is now an array of objects instead of a single object.
  • browser_action is replaced with action.
  • Added "<all_urls>" into matches in the web_accessible_resources section because no log is getting collected due to the permission issue.
• log.js:
  • Used chrome.runtime.onMessage.addListener and chrome.runtime.sendMessage APIs to communicate because the pop-up window (log.js) cannot access the EmeLogWindow instance in the service worker (log-window.js).
  • Used chrome.windows.onRemoved.addListener to observe the termination of window because there is no .closed property in the window created by chrome.windows.create.
• content-script.js:
  • chrome.extension.getURL is replaced with chrome.runtime.getURL.
• spec/log-window-tests.js:
  • Added the mock & fake properties/methods for chrome.runtime and chrome.windows APIs because Jasmine-browser and Selenium-webdriver are not working with chrome APIs and we need test code changes to test async calls.
  • Updated the required dependency packages: npm run build & npm test complains about the out-dated versions of selenium-webdriver and other dependencies
  • Used Promise/async-wait patterns for EmeLogWindow.instance.open() API tests. Otherwise, EmeLogWindow.instance.isOpen()'s return value is not guaranteed since .open() is an async operation.
  • Added new test cases for EmeLogWindow.instance.appendLog() and EmeLogWindow.instance.clear() APIs
• spec/support/jasmine-browser.json:
  • Added "log.js" to the srcFiles section because we cannot directly test the appendLog() logic without the chrome messaging APIs that are not available on the Jasmine framework. As a workaround, log-window-tests.js calls the appendLog() method defined in the log.js first and passes it down EmeLogWindow.appendLog() in the tests.
• package-lock.json:
  • Updated selenium-webdriver to the latest version (4.3.1) to fix the Invalid URL issue by selenium-webdriver when running npm test. Other dependencies are updated together.

Test Plan

All unit tests with 44 specs were passed by npm test. Integration tests were manually done on the MacOSX.

Unit Tests

cd ~/eme_logger
npm run build
npm test

Test results:

> test
> jasmine-browser-runner runSpecs
Jasmine server is running here: http://localhost:51432
Jasmine tests are here:         /Users/sangbaekpark/github/eme_logger/spec
Source files are here:          /Users/sangbaekpark/github/eme_logger
Running tests in the browser...
Started
............................................
44 specs, 0 failures
Finished in 4.903 seconds

Integration Tests

Prerequisites:

• Clone the eme_logger repository with the change
• Build and run test
• Make sure all dependencies are installed (use npm list command)

npm install

npm list
    eme_logger@3.3.0 /Users/sangbaekpark/github/eme_logger
    ├── chromedriver@95.0.0
    ├── gulp-rename@2.0.0
    ├── gulp-zip@5.1.0
    ├── gulp@4.0.2
    ├── jasmine-browser-runner@0.9.0
    ├── jasmine-core@3.10.1
    ├── json5@2.2.0
    └── trace-anything@1.0.1

npm run build
npm test

• You will have the unpacked chrome extension binary.

Test steps:

1. Reload the chrome extension binary (via either "Load unpacked" button or Refresh icon) -> Expected: The chrome extension is loaded without any error.
2. Click "Errors" button and clear all errors to isolate between tests -> Expected: All errors are cleared.
3. Open the log window by clicking the Eme Logger chrome extension icon -> Expected: A log window should be popped up and activated (focused).
4. Activate another window other than the log window -> Expected: The previously created log window should be deactivated (not focused).
5. Reopen the log window -> Expected: The previously created log window should be re-activated (focused).
6. Close the log window -> Expected: The log window should be closed.
7. Reopen the log window -> Expected: A log window should be popped up and activated (focused).
8. Reload the video player page. Test video link (Choose "Widevine DRM") -> Expected: New logs should be shown in the log window
9. Click "Clear" button -> Expected: The logs captured should be cleared.
10. Click "Download" button -> Expected: Text formatted log should be downloaded.
11. Leave the log window with prior logs for 1hr without having any additional logs (to be idle) -> Expected: No additional logs are captured unless you trigger any EME related operations.
12. Reload the video player page or move around the current frame -> Expected: The new logs should be appended to the log window after the 1hr idle time. Rationale: This is to make sure the service worker's life cycle still runs. Not something like it gets terminated after 5 min idle.

Test results:

• All test steps passed.

Migration Issues

1. Error: The "background.scripts" key cannot be used with manifest_version 3. Use the "background.service_worker" key instead.

Failed to load extension: Could not load manifest.

File: ~/github/eme_logger

https://stackoverflow.com/questions/66055882/chrome-extensions-use-the-background-service-worker-key-instead-manifest-vers

https://developer.chrome.com/docs/extensions/mv3/intro/mv3-migration/#man-sw

https://developer.chrome.com/docs/extensions/mv3/migrating_to_service_workers/

--> Replace "scripts" with "service_worker" for the "background" configuration

2. Error: Error at key 'web_accessible_resources'. Parsing array failed at index 0: expected dictionary, got string

Failed to load extension: Could not load manifest.

File: ~/github/eme_logger

https://developer.chrome.com/docs/extensions/mv3/intro/mv3-migration/#web-accessible-resources

--> Provide an array of objects instead of a single object

3. 'browser_action' requires manifest version of 2 or lower.

  "browser_action": {
    "default_title": "EME Logger"
  },

https://developer.chrome.com/docs/extensions/mv3/intro/mv3-migration/#action-api-unification

--> Use action API instead

4. Uncaught TypeError: chrome.extension.getURL is not a function

content-script.js:22 (anonymous function)

const absoluteUrl = chrome.extension.getURL(url);

https://developer.chrome.com/docs/extensions/mv3/mv3-migration-checklist/#api_checklist

--> Use chrome.runtime.getURL() to build resource URLs at runtime.

5. Service worker registration failed

  "background": {
    "service_worker": "log-window.js"
  },

Uncaught ReferenceError: window is not defined

window.EmeLogWindow = EmeLogWindow;

https://stackoverflow.com/questions/68194103/error-in-event-handler-referenceerror-window-is-not-defined-chrome-extension-w

ManifestV3 extension uses a service worker so it doesn't have DOM or window.

https://developer.chrome.com/docs/extensions/mv3/intro/mv3-migration/#background-service-workers

https://developer.chrome.com/docs/workbox/service-worker-overview/

https://developer.chrome.com/docs/extensions/mv3/migrating_to_service_workers/

https://stackoverflow.com/questions/28799892/how-to-launch-a-new-window-in-google-chrome-extension

--> Use chrome.windows.getCurrent to get the size/position of the current browser window (only for window info).

--> Use chrome.windows.create to create a new window.

--> Remove this line window.EmeLogWindow = EmeLogWindow; since the singleton instance of EmeLogWindow (EmeLogWindow.instance) lives in the service worker.

6. Uncaught TypeError: Cannot read properties of undefined (reading 'onClicked')

chrome.browserAction.onClicked.addListener((tab) => {

https://developer.chrome.com/docs/extensions/mv3/migrating_to_service_workers/#event_listeners

--> Manifest V3 consolidates chrome.browserAction and chrome.pageAction into a single chrome.action API.

7. Cannot access an instance of EmeLogWindow from a pop-up window (log.js) to the service worker (log-window.js).

https://developer.chrome.com/docs/extensions/mv3/messaging/

https://www.freecodecamp.org/news/chrome-extension-message-passing-essentials/

--> Use chrome.runtime.onMessage.addListener and chrome.runtime.sendMessage APIs instead

8. Uncaught TypeError: Cannot read properties of undefined (reading 'closed')

this.logWindow_.closed in the EmeLogWindow class.

--> Use chrome.windows.onRemoved.addListener to observe the termination of window

9. Cannot access the document or DOM object from a service worker.

https://stackoverflow.com/questions/4532236/how-to-access-the-webpage-dom-rather-than-the-extension-page-dom

Issue 1191971: Chrome extension: chrome.scripting.executeScript not working in my manifest v3 Chrome Extension: https://bugs.chromium.org/p/chromium/issues/detail?id=1191971

Chrome extension injecting script get error: https://stackoverflow.com/questions/36762389/chrome-extension-injecting-script-get-error

Resolved: Extension manifest must request permission to access the respective host or a different Error: https://dailydevsblog.com/troubleshoot/resolved-extension-manifest-must-request-permission-to-access-the-respective-host-or-a-different-error-53911/

https://stackoverflow.com/questions/53024819/chrome-extension-sendresponse-not-waiting-for-async-function

How to Send Data Between Chrome Extension Scripts

--> If we use chrome.scripting.executeScript instead, it didn't work with an error "log-window.js:76 Uncaught (in promise) Error: Cannot access contents of the page. Extension manifest must request permission to access the respective host." error

--> Move the logic handling DOM from log-window.js (service worker) to log.js (log window) and communicate between two scripts via chrome.runtime.sendMessage and chrome.runtime.onMessage.addListener APIs

10. Error in event handler: TypeError: URL.createObjectURL is not a function at EmeLogWindow.getTextLogUri

Issue 1224027: Creating a download from generated content in a ManifestV3 service worker

TypeError: Error URL.createObjectURL() not a function

https://stackoverflow.com/questions/4771758/how-to-generate-a-file-for-download-in-a-google-chrome-extension

--> The download won't get triggered because the messaging is an async operation if we use string.replace() simply to do return 'data:text/plain,' + this.textLogs_.replace("#", "%23");

--> Use chrome.downloads.download API with adding "downloads" to the "permission" section in the manifest file.

11. No log is getting collected

--> Add "<all_urls>" into "matches" in the "web_accessible_resources" section in the manifest file.

"web_accessible_resources": [
  {
    "matches": [
      "<all_urls>"
    ],
    ...
]

12. Error after clicking "Download log" button: "Unchecked runtime.lastError: A listener indicated an asynchronous response by returning true, but the message channel closed before a response was received"

--> Turned out we should not do return true; to keep the messaging channel open in the message handler of chrome.runtime.onMessage.addListener

13. 23 test failures when running npm test with Jasmine-browser and Selenium-webdriver

Unhandled promise rejection: TypeError: Cannot read properties of undefined (reading 'create')
Uncaught TypeError: Cannot read properties of undefined (reading 'onRemoved')
TypeError: Cannot read properties of null (reading 'textContent')

https://stackoverflow.com/questions/19394459/unable-to-access-chrome-message-passing-api-from-selenium-execute-script

https://stackoverflow.com/questions/45637489/how-to-test-a-function-which-calls-async-function-using-jasmine

https://howtodoinjava.com/javascript/jasmine-unit-testing-tutorial/

Unit Testing Async Calls and Promises with Jasmine

--> Set up the mock/fake properties/methods for chrome.runtime and chrome.windows APIs: Jasmine-browser and Selenium-webdriver are not working with chrome.xxx APIs and we need test code changes to test async calls.

--> Updated the required dependency packages: npm run build & npm test complains about the out-dated versions of selenium-webdriver and other dependencies

--> Used Promise/async-wait patterns for EmeLogWindow.instance.open() API tests

--> Added new test cases for EmeLogWindow.instance.appendLog() and EmeLogWindow.instance.clear() APIs

14. Invalid URL by selenium-webdriver when running npm test

Error: Invalid URL: http:/
    at getRequestOptions (/Users/xxx/github/eme_logger/node_modules/selenium-webdriver/http/index.js:51:11)

--> Updated selenium-webdriver to the latest version (4.3.1)

References

• The transition of Chrome extensions to Manifest V3
• Manifest V3 now available on M88 Beta
• Manifest v3 Migration Checklist
• How To Build A Chrome Extension Manifest Version 3
• Chrome Extension Tutorial: Migrating to Manifest V3 from V2
• Migrating from background pages to service workers
• Chrome Extension Tutorial: How to Pass Messages from a Page's Context
• How to access the webpage DOM rather than the extension page DOM?
• Using promises
• How to launch a new window in Google Chrome Extension
• How to Send Data Between Chrome Extension Scripts
• Unit Testing Async Calls and Promises with Jasmine

[joeyparrish]

Looks great overall! Thanks for doing this!