WebExtensions timeline/checklist

[Hainish]

1. Wait until a few weeks after Tor Browser 7.0 stable has been released, so that users have had the chance to upgrade to a Tor Browser based on Firefox 52 ESR.
2. Change https://github.com/EFForg/https-everywhere-docker-base/blob/master/Dockerfile to use ESR 52.
3. Merge the webextensions branch into master.
4. Make a new release, which will follow the WebExtensions date-based versioning. In AMO, ensure the versioning is correct, so that users are upgraded only if they are on Firefox >= 52 or Fennec >= 53.
5. Generate a signature for https://www.eff.org/files/https-everywhere-eff-update-2048.rdf which ensures upgrading to the WebExtension version if users are on Firefox >= 52 or Fennec >= 53.
6. Create https://www.eff.org/files/https-everywhere-updates.json and ensure it is signed, so that the WebExtensions version can update itself.
7. Create the 2017.X.X tag, and push it to both tor and github remotes

[Hainish]

cc @J0WI @jeremyn @gloomy-ghost @jsha

[jeremyn]

It would be really nice if you could release another version of HTTPS Everywhere, call it "HTTPS Everywhere (WebExtensions Beta)" and make that available alongside the live version for one or two Firefox release cycles. You could even just release it on the EFF site, with a hidden link, and not AMO. I feel like we'll have a much lower chance of catastrophic failure if even a few of us could run with it for a month or two before it goes out to a million people.

[Hainish]

@jeremyn I think that's a fine idea. I'm currently waiting on Mozilla to document how in WebExtensions you can provide a signature and embed the extension public key. This is supported in XPCOM, but maybe not WebExtensions (yet?) This provides a mechanism above and beyond TLS keys to ensure that extension upgrades are securely deployed.

Regardless, providing a temporary WebExtensions beta to interested users wouldn't necessitate that level of secure signing, and can probably be done pretty easily.

[jeremyn]

Is that process different than the lower level review that Mozilla does for add-ons that aren't distributed through AMO?

[jeremyn]

I mean the "unlisted" process described here.

[Hainish]

Yes, it's a separate thing that allows app developers to embed their own public key into an extension, and then Firefox will only upgrade when the upgrade manifest is signed with this public key. This is only available for self-hosted, not for AMO.

[jeremyn]

Oh okay. Can you point to some discussion on that somewhere? I'd like to know more about it.

[jeremyn]

I guess you mean something like https://addons-server.readthedocs.io/en/latest/topics/api/signing.html .

[Hainish]

@jeremyn you can see this by downloading the self-hosted XPI, then checking the install.rdf file in it (it's also in src/ in the source tree), which lists the public key. The extension then checks for new files and attempts to verify the signature in https://www.eff.org/files/https-everywhere-eff-update-2048.rdf and if it verifies, an upgrade can take place.

The documentation for updateKey is here.

There is more comprehensive documentation of the entire process here

[jeremyn]

Thanks!

[Hainish]

I've created a new tag to note issues with the WebExtensions version for Firefox, just called webextensions-firefox. I've already noted one issue: #9965

@jeremyn: here's a link to the testing versions of HTTPS Everywhere: https://www.eff.org/files/https-everywhere-test/index.html

I have several files listed there. The first few (older) files are to test the upgrade path. Feel free to install and test it yourself, and if you encounter any issues just tag it with the above-mentioned label.

[Bisaloo]

Was the observatory removed in the process? I can't find anything about it on the new UI.

[ivysrono]

https://bugzilla.mozilla.org/show_bug.cgi?id=1345893
So, Please set WebExtension's HTTPSew:
Firefox ESR 52+
or
Firefox Release 54+

[jeremyn]

Thanks @Hainish , I installed https-everywhere-5.2.16.1-eff.xpi and after forcing "Check for updates" twice in about:addons -> "gear", it's updated to the latest https-everywhere-2017.5.19.1-eff.xpi , so that looks fine.

The upgrade from 2017.5.19-eff.xpi to 2017.5.19.1-eff.xpi didn't prompt me to restart my browser for updates to take effect, is that normal?

Also, like @Bisaloo , the Observatory screen is gone, as is the "Preferences" button in about:addons.

[Hainish]

@Bisaloo @jeremyn wrt the observatory, there is no way to do certificate introspection within the WebExtensions API currently. We're working with the Chrome team to make this possible in the future. Right now, this is an intentional omission - hopefully we'll be able to add it back in the future.

@jeremyn this is expected - WebExtensions are restartless - no need any more to restart the browser after addon upgrades!

@ivysrono Interesting, I'll look into this.

[AlexBeLi]

I have some problem in private browsing mode with webext version on latest Nightlly (2017-05-22). Somebody have too?

[jeremyn]

I get the problem @AlexBeLi is getting in private browsing mode on Firefox 53.0.2.

[Hainish]

@Bisaloo, @AlexBeLi please open separate issues for these, and link to this issue. I (or one of the other collaborators) can then add the webextensions-firefox tag to it.

[J0WI]

See also some missing features in the chrome/WebExtension version:
#1291
#1292
#1293
#1681
#1779

The user rule feature as we have it in Firefox is quite important for testing.

[J0WI]

@Hainish for the observatory you should get in touch with Mozilla https://bugzilla.mozilla.org/show_bug.cgi?id=1215059

[Hainish]

@J0WI we're in touch with Mozilla, but as far as WebExtension API development, all of the momentum moving forward with API specs has been within Chrome. We're working closely with them to make this possible, but it likely won't make MVP by the time we deploy WebExtensions for Firefox.

[davidhedlund]

Excellent answer @gloomy-ghost!

[Hainish]

Development version 2017.9.12.1337 has been released: https://www.eff.org/https-everywhere#developers_note

[J0WI]

Next week, Firefox 57 will ship to beta.

[ghost]

@Hainish Are we ready for WebExtensions?

[Hainish]

We've been ready for quite some time. We are allowing the maximum window for people who have upgraded to the Embedded WebExtension from the XPCOM extension to restart their browsers. Full WebExtensions will launch in late October or early November, as I've already explained here. Our timeline has not changed.

[Hainish]

For reference, the Firefox release schedule is available here: https://wiki.mozilla.org/RapidRelease/Calendar.

Firefox Stable will make the transition to 57 on November 14th. Transitioning to full WebExtensions in late October should give us enough padding to fix any issues if they arise as well as allow time for AMO review.

[J0WI]

What about the open ToDos for Webextensions?

[jeremyn]

Seconding @J0WI 's question about the remaining open issues. For reference, they are https://github.com/EFForg/https-everywhere/issues?q=is%3Aopen+is%3Aissue+label%3Awebextensions-firefox and many of them are serious.

[Diapolo]

Firefox Quantum 57.0b3 (64-Bit) is now on my machine, sadly HTTPSEverywhere is not yet ready :-/.

[strigona-worksight]

@Diapolo You can install the developer version which is WebExtension here: https://www.eff.org/https-everywhere

[Diapolo]

@strigona-worksight That is working fine, thank you. 2 more questions, will I now stay on the developer version or will that be upgraded to release version once finished? I had 2 versions listed in parallel, the old non-WebExtension one (which I removed manually) and the new developer version, why?

[uniquePWD]

I'm not sure if it's been noted, but if you upload the WebExtension to AMO and set the minimum version to 57, users that don't have 57 won't be upgraded until their browser updates to 57.

[Hainish]

@pwd-github we plan to do this regardless, but doing so would shorten the time that the embedded WebExtension has the opportunity to migrate users settings.

[FranklinYu]

Not sure whether it helps, but AMO provides beta channel.

[mgol]

I'm not sure if it's been noted, but if you upload the WebExtension to AMO and set the minimum version to 57, users that don't have 57 won't be upgraded until their browser updates to 57.

@pwd-github we plan to do this regardless, but doing so would shorten the time that the embedded WebExtension has the opportunity to migrate users settings.

@Hainish How so? Users of Firefox <57 should get an update to the latest supported version, shouldn't they? Only Firefox 57+ users (currently beta & nightly) would get the update to a WebExtension version and those are versions where the embedded WebExtension wouldn't work anyway.

[J0WI]

The first full WebExtensions release is just around the corner: #13420.

[Hainish]

Full WebExtensions release has been made. Thank you for all your hard work in fleshing out the bugs missing features. There's still some work to be done to replace some of the features that we lost when moving from XPCOM. This can be tracked here: https://github.com/EFForg/https-everywhere/projects/1