Import of v0.28.0 (#1803)

CHANGES.md

@@ -1,12 +1,59 @@
 Change log
 ==========
 
-All notable changes to this program is documented in this file.
+All notable changes to this program are documented in this file.
 
+0.28.0  (2020-11-03, `c00d2b6acd3f`)
+--------------------
+
+### Known problems
+
+- _macOS 10.15 (Catalina):_
+
+  Due to the requirement from Apple that all programs must be
+  notarized, geckodriver will not work on Catalina if you manually
+  download it through another notarized program, such as Firefox.
+
+  Whilst we are working on a repackaging fix for this problem, you can
+  find more details on how to work around this issue in the [macOS
+  notarization] section of the documentation.
+
+### Added
+
+- The command line flag `--android-storage` has been added, to allow geckodriver
+to also control Firefox on root-less Android devices. See the [documentation][Flags]
+for available values.
+
+### Fixed
 
-0.27.0  (2020-07-27, `90ec81285ff6`)
+- Firefox can be started again via a shell script that is located outside of the
+  Firefox directory on Linux.
+
+- If Firefox cannot be started by geckodriver the real underlying error message is
+  now being reported.
+
+- Version numbers for minor and extended support releases of Firefox are now parsed correctly.
+
+### Removed
+
+- Since Firefox 72 extension commands for finding an element’s anonymous children
+  and querying its attributes are no longer needed, and have been removed.
+
+0.27.0  (2020-07-27, `7b8c4f32cdde`)
 --------------------
 
+### Security Fixes
+
+- CVE-2020-15660
+
+  - Added additional checks on the `Content-Type` header for `POST`
+    requests to disallow `application/x-www-form-urlencoded`,
+    `multipart/form-data` and `text/plain`.
+
+  - Added checking of the `Origin` header for `POST` requests.
+
+  - The version number of Firefox is now checked when establishing a session.
+
 ### Known problems
 
 - _macOS 10.15 (Catalina):_
@@ -15,9 +62,9 @@ All notable changes to this program is documented in this file.
   notarized, geckodriver will not work on Catalina if you manually
   download it through another notarized program, such as Firefox.
 
-  Whilst we are working on a repackaging fix for this problem, you
-  can find more details on how to work around this issue in the
-  [macOS notarization] section of the documentation.
+  Whilst we are working on a repackaging fix for this problem, you can
+  find more details on how to work around this issue in the [macOS
+  notarization] section of the documentation.
 
 ### Added
 
@@ -42,8 +89,7 @@ All notable changes to this program is documented in this file.
 
 - _Android:_
 
-  * Firefox running on Android devices can now be controlled from a
-    Windows host.
+  * Firefox running on Android devices can now be controlled from a Windows host.
 
   * Setups with multiple connected Android devices are now supported.
 
@@ -1327,6 +1373,7 @@ and greater.
 [Firefox Preview]: https://play.google.com/store/apps/details?id=org.mozilla.fenix
 [Firefox Reality]: https://play.google.com/store/apps/details?id=org.mozilla.vrbrowser
 [Capabilities]: https://firefox-source-docs.mozilla.org/testing/geckodriver/Capabilities.html
+[Flags]: https://firefox-source-docs.mozilla.org/testing/geckodriver/Flags.html
 [enable remote debugging on the Android device]: https://developers.google.com/web/tools/chrome-devtools/remote-debugging
 [macOS notarization]: https://firefox-source-docs.mozilla.org/testing/geckodriver/Notarization.html
 

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "geckodriver"
-version = "0.27.0"
+version = "0.28.0"
 description = "Proxy for using WebDriver clients to interact with Gecko-based browsers."
 keywords = ["webdriver", "w3c", "httpd", "mozilla", "firefox"]
 repository = "https://hg.mozilla.org/mozilla-central/file/tip/testing/geckodriver"
@@ -17,17 +17,17 @@ hyper = "0.13"
 lazy_static = "1.0"
 log = { version = "0.4", features = ["std"] }
 marionette = { path = "./marionette" }
-mozdevice = "0.2.0"
+mozdevice = "0.3.0"
 mozprofile = "0.7.0"
-mozrunner = "0.11"
-mozversion = "0.3"
+mozrunner = "0.12.0"
+mozversion = "0.4.0"
 regex = { version="1.0", default-features = false, features = ["perf", "std"] }
 serde = "1.0"
 serde_derive = "1.0"
 serde_json = "1.0"
 serde_yaml = "0.8"
 uuid = { version = "0.8", features = ["v4"] }
-webdriver = "0.41"
+webdriver = "0.42.0"
 zip = { version = "0.4", default-features = false, features = ["deflate"] }
 
 [[bin]]

doc/Flags.md

@@ -1,7 +1,53 @@
 Flags
 =====
 
-#### <code>-b <var>BINARY</var></code>/<code>--binary <var>BINARY</var></code>
+#### <code>--android-storage <var>ANDROID_STORAGE</var></code>
+
+Selects the test data location on the Android device, eg. the Firefox profile.
+By default `auto` is used.
+
+<style type="text/css">
+  table { width: 100%; margin-bottom: 2em; }
+  table, th, td { border: solid gray 1px; }
+  td, th { padding: 10px; text-align: left; vertical-align: middle; }
+  td:nth-child(1), th:nth-child(1) { width: 10em; text-align: center; }
+</style>
+
+<table>
+ <thead>
+  <tr>
+    <th>Value
+    <th>Description
+  </tr>
+ </thead>
+
+ <tr>
+  <td>auto
+  <td>Best suitable location based on whether the device is rooted.<br/>
+    If the device is rooted <code>internal</code> is used, otherwise <code>app</code>.
+ <tr>
+  <td>app
+  <td><p>Location: <code>/data/data/%androidPackage%/test_root</code></p>
+    Based on the <code>androidPackage</code> capability that is passed as part of
+    <code>moz:firefoxOptions</code> when creating a new session. Commands that
+    change data in the app's directory are executed using run-as. This requires
+    that the installed app is debuggable.
+ <tr>
+  <td>internal
+  <td><p>Location: <code>/data/local/tmp/test_root</code></p>
+    The device must be rooted since when the app runs, files that are created
+    in the profile, which is owned by the app user, cannot be changed by the
+    shell user. Commands will be executed via <code>su</code>.
+ <tr>
+  <td>sdcard
+  <td><p>Location: <code>/mnt/sdcard/test_root</code></p>
+    This location is not supported on Android 11+ due to the
+    <a href="https://developer.android.com/about/versions/11/privacy/storage">
+    changes related to scoped storage</a>.
+</table>
+
+
+#### <code>-b <var>BINARY</var></code> / <code>--binary <var>BINARY</var></code>
 
 Path to the Firefox binary to use.  By default geckodriver tries to
 find and use the system installation of Firefox, but that behaviour
@@ -28,7 +74,7 @@ scanning the Windows registry.
 [whereis(1)]: http://www.manpagez.com/man/1/whereis/
 
 
-#### `--connect-existing`
+#### <code>--connect-existing</code>
 
 Connect geckodriver to an existing Firefox instance.  This means
 geckodriver will abstain from the default of starting a new Firefox
@@ -55,6 +101,12 @@ Set the Gecko and geckodriver log level.  Possible values are `fatal`,
 `error`, `warn`, `info`, `config`, `debug`, and `trace`.
 
 
+#### <code>--marionette-host <var>HOST</var></code>
+
+Selects the host for geckodriver’s connection to the [Marionette]
+remote protocol. Defaults to 127.0.0.1.
+
+
 #### <code>--marionette-port <var>PORT</var></code>
 
 Selects the port for geckodriver’s connection to the [Marionette]
@@ -70,7 +122,7 @@ under geckodriver’s control, it will simply connect to <var>PORT</var>.
 [`--connect-existing`]: #connect-existing
 
 
-#### <code>-p <var>PORT</var></code>/<code>--port <var>PORT</var></code>
+#### <code>-p <var>PORT</var></code> / <code>--port <var>PORT</var></code>
 
 Port to use for the WebDriver server.  Defaults to 4444.
 

doc/Releasing.md

@@ -124,10 +124,10 @@ repository, the changeset id for the release has to be added to the
 change log. Therefore add a final place-holder commit to the patch
 series, to already get review for.
 
-Once all previous revisions of the patch series have been reviewed and
-landed, it's known which commit id the version bump commit has, finalize the
-change log, and land that remaining revision.
-
+Once all previous revisions of the patch series have been landed, and got merged
+to `mozilla-central`, the changeset id from the merge commit has to picked for
+finalizing the change log. This specific id is needed because Taskcluster creates
+the final signed builds based on that merge.
 
 Release new in-tree dependency crates
 -------------------------------------

doc/Support.md

@@ -23,6 +23,16 @@ and required versions of Selenium and Firefox:
   </tr>
  </thead>
 
+ <tr>
+  <td>0.28.0
+  <td>≥ 3.11 (3.14 Python)
+  <td>60
+  <td>n/a
+ <tr>
+  <td>0.27.0
+  <td>≥ 3.11 (3.14 Python)
+  <td>60
+  <td>n/a
  <tr>
   <td>0.26.0
   <td>≥ 3.11 (3.14 Python)

doc/Usage.md

@@ -68,9 +68,9 @@ Using [curl(1)]:
 	% geckodriver &
 	[1] 16010
 	% 1491834109194   geckodriver     INFO    Listening on 127.0.0.1:4444
-	% curl -d '{"capabilities": {"alwaysMatch": {"acceptInsecureCerts": true}}}' http://localhost:4444/session
-	{"sessionId":"d4605710-5a4e-4d64-a52a-778bb0c31e00","value":{"XULappId":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","acceptSslCerts":false,"appBuildId":"20160913030425","browserName":"firefox","browserVersion":"51.0a1","command_id":1,"platform":"LINUX","platformName":"linux","platformVersion":"4.9.0-1-amd64","processId":17474,"proxy":{},"raisesAccessibilityExceptions":false,"rotatable":false,"specificationLevel":0,"takesElementScreenshot":true,"takesScreenshot":true,"version":"51.0a1"}}
-	% curl -d '{"url": "https://mozilla.org"}' http://localhost:4444/session/d4605710-5a4e-4d64-a52a-778bb0c31e00/url
+	% curl -H 'Content-Type: application/json' -d '{"capabilities": {"alwaysMatch": {"acceptInsecureCerts": true}}}' http://localhost:4444/session
+	{"value":{"sessionId":"d4605710-5a4e-4d64-a52a-778bb0c31e00","capabilities":{"acceptInsecureCerts":true,[...]}}}
+	% curl -H 'Content-Type: application/json' -d '{"url": "https://mozilla.org"}' http://localhost:4444/session/d4605710-5a4e-4d64-a52a-778bb0c31e00/url
 	{}
 	% curl http://localhost:4444/session/d4605710-5a4e-4d64-a52a-778bb0c31e00/url
 	{"value":"https://www.mozilla.org/en-US/"