Preventing activity-stream.topstories.json from getting downloaded

[Gitoffthelawn]

Even with:

user_pref("browser.newtabpage.activity-stream.enabled", false);
user_pref("browser.library.activity-stream.enabled", false); // (FF57+)

Firefox 60.0b8 is downloading data to create activity-stream.topstories.json within the current profile.

UPDATE: Also activity-stream.tippytop.json

How do we make it stop?

[Atavic]

The "refreshed new tab page" has a repo

mozilla/activity-stream@c260e0a
mozilla/activity-stream@0376728

Maybe blocking getpocket.com and getpocket.cdn.mozilla.net may help.

https://bugzilla.mozilla.org/show_bug.cgi?id=1410075 RESOLVED WONTFIX

[grauenwolfe]

From DXR, "testing" setup: "Use an empty list of sites to avoid fetching"

// Use an empty list of sites to avoid fetching
user_pref("browser.newtabpage.activity-stream.default.sites", "");
user_pref("browser.newtabpage.activity-stream.telemetry", false);
user_pref("browser.newtabpage.activity-stream.tippyTop.service.endpoint", "");
user_pref("browser.newtabpage.activity-stream.feeds.section.topstories", false);
user_pref("browser.newtabpage.activity-stream.feeds.snippets", false);

[Gitoffthelawn]

@Thorin-Oakenpants You got me thinking... I never even see FF's newpage tab. So I forced FF to use it, and then clicked on the cog, and turned everything off. So far, no spurious files being added to the profile.

I had done that in the past, but FF must have turned everything back on when they came out with a new version of newtab.

[2glops]

Nightly 61.0a1
about:preferences#home UI shows :

Homepage and new windows : blank page, Firefox Home (default), custom URLs
New tabs : blank page, Firefox Home (default)

I set blank page for both.

Firefox Home Content : Choose what content you want on your Firefox Home screen (web search, top sites, recommended by Pocket, highlights, snippets could be un-ticked)

[2glops]

are you telling me that a new tab can't use a custom URL

Yes.
At least on the UI, I don't search about:config to find an option to do that.

[Gitoffthelawn]

I'm now getting activity-stream.tippytop.json in the profile on 60.0b8. WTF.

Will try the following (NOTE: SEMICOLON MISSING IN THORIN's POST ABOVE):
// hide activity stream sections (may help stop connections)
user_pref("browser.newtabpage.activity-stream.showSearch", false);
user_pref("browser.newtabpage.activity-stream.feeds.section.highlights", false);
user_pref("browser.newtabpage.activity-stream.feeds.snippets", false);
user_pref("browser.newtabpage.activity-stream.showTopSites", false);
// block activity stream connections
user_pref("browser.newtabpage.activity-stream.feeds.section.topstories.options", ""); // pocket <--- NOTE SEMICOLON ADDED AFTER PREF HERE
user_pref("browser.newtabpage.activity-stream.telemetry.ping.endpoint", ""); // tiles service
user_pref("browser.newtabpage.activity-stream.feeds.topsites", false);
user_pref("browser.newtabpage.activity-stream.feeds.telemetry", false);
user_pref("browser.newtabpage.activity-stream.telemetry", false);

[Atavic]

@Thorin-Oakenpants asks: browser.newtabpage.activity-stream.telemetry.ping.endpoint points to "https://tiles.services.mozilla.com/v4/links/activity-stream"

blank it to stop?

Not enough, according to: https://github.com/mozilla/onyx/blob/master/onyx/default_settings.py#L18

Look, there are 4 CDN links -> https://tiles.cdn.mozilla.net/

activity-stream is full-off telemetry:

// These fields are sent from the client
"client_id": "374dc4d8-0cb2-4ac5-a3cf-c5a9bc3c602e",
"session_id": "{12dasd-213asda-213dkakj}",
"profile_creation_date": 14786,

// These fields are generated on the server
"ip": "10.192.171.13",
"receive_at": 1457396660000

Source: https://github.com/mozilla/activity-stream/blob/b69291f6f37d1625134247c537d8796aae2c0695/docs/v2-system-addon/data_dictionary.md

The server is onyx

[grauenwolfe]

@Gitoffthelawn You may know this already and sorry if you mentioned this and I missed it but you can just delete the activity-stream system extension from your application folder. I do this after every update or fresh install along with the other system extnesions. My about:support shows nothing under Firefox Features.

[Gitoffthelawn]

@grauenwolfe No harm in mentioning it! :)

[earthlng]

just tell people to delete the xpi and change their homepage etc

👍

The telemetry uses ping-centre and should be covered by 0506 and 0333.

If you wanna add the prefs I'd just disable everything:

user_pref("browser.newtabpage.activity-stream.disableSnippets", true);
user_pref("browser.newtabpage.activity-stream.feeds.aboutpreferences", false); // FF61+
user_pref("browser.newtabpage.activity-stream.feeds.favicon", false);
user_pref("browser.newtabpage.activity-stream.feeds.messagecenterfeed", false); // FF61+
user_pref("browser.newtabpage.activity-stream.feeds.migration", false);
user_pref("browser.newtabpage.activity-stream.feeds.newtabinit", false);
user_pref("browser.newtabpage.activity-stream.feeds.places", false);
user_pref("browser.newtabpage.activity-stream.feeds.prefs", false);
user_pref("browser.newtabpage.activity-stream.feeds.section.highlights", false);
user_pref("browser.newtabpage.activity-stream.feeds.section.topstories", false);
user_pref("browser.newtabpage.activity-stream.feeds.section.topstories.options", "");
user_pref("browser.newtabpage.activity-stream.feeds.sections", false);
user_pref("browser.newtabpage.activity-stream.feeds.snippets", false);
user_pref("browser.newtabpage.activity-stream.feeds.systemtick", false);
user_pref("browser.newtabpage.activity-stream.feeds.telemetry", false);
user_pref("browser.newtabpage.activity-stream.feeds.theme", false); // FF61+
user_pref("browser.newtabpage.activity-stream.feeds.topsites", false);
user_pref("browser.newtabpage.activity-stream.messageCenterExperimentEnabled", false); // FF61+
user_pref("browser.newtabpage.activity-stream.migrationExpired", true);
user_pref("browser.newtabpage.activity-stream.prerender", false);
user_pref("browser.newtabpage.activity-stream.section.highlights.includePocket", false);
user_pref("browser.newtabpage.activity-stream.showSearch", false);
user_pref("browser.newtabpage.activity-stream.showSponsored", false);
user_pref("browser.newtabpage.activity-stream.showTopSites", false);
user_pref("browser.newtabpage.activity-stream.telemetry", false);
user_pref("browser.newtabpage.activity-stream.telemetry.ping.endpoint", "");
user_pref("browser.newtabpage.activity-stream.telemetry.ut.events", false);
user_pref("browser.newtabpage.activity-stream.tippyTop.service.endpoint", "");

[Gitoffthelawn]

Which is what in this case? Deleting the XPI is nice and clean, except for the fact that it gets reinstalled at (almost) every release (which is weekly is you are on beta and daily if you are on nightly).

[overdodactyl]

How hard would it be to get essentially a "blank" add-on submitted to AMO with the same ID as activity stream?

Available locations, starting from the highest priority include:

    1. temporary install (about:debugging)
    2. normal user install into profile (about:addons or AMO/TestPilot/etc.)
    3. system add-on updates
    4. built-in system add-ons

This makes it possible for a developer or user to override a system add-on by installing an add-on with the same ID from AMO or TestPilot or as a temporary add-on.

https://firefox-source-docs.mozilla.org/toolkit/mozapps/extensions/addon-manager/SystemAddons.html

Was just trying to think of a way to override the .xpi file on a per-profile basis and not have to worry about the file being re-installed upon updates.

[overdodactyl]

I would have assumed the same (I'm also not very familiar with webextension development), but the documentation seems to imply it's possible, specifically:

This makes it possible for a ... user to override a system add-on by installing an add-on with the same ID from AMO

Maybe @earthlng would have a little more insight here?

[claustromaniac]

A blank xpi would be an easy and elegant set-and-forget solution for everyone but, personally, this is the kind of task I use scripts for. I don't give a shit if the xpi's keep coming back.

A likely working alternative would be to make a blank xpi and tamper with access rights to prevent it from being overwritten by Firefox, but I don't know if the latter would be OK with that.

[overdodactyl]

I was just about to look into the blank addon a bit more. Simply using the following for manifest.json works to override the activity-stream@mozilla.org.xpi when loaded as a temporary add-on:

{

  "description": "A blank extension used to override the Activity Stream system add-on. See https://firefox-source-docs.mozilla.org/toolkit/mozapps/extensions/addon-manager/SystemAddons.html",
  "manifest_version": 2,
  "name": "Activity Stream Override",
  "version": "1.0",
  
  "applications": {
  "gecko": {
    "id": "activity-stream@mozilla.org"
  }
}

}

I can try submitting it to AMO and see if it's rejected or not.

A script would certainly be an effective method, but personally I would like a per-profile solution (I need access to the activity stream for another repo I maintain).

[overdodactyl]

No luck submitting to AMO. I get the following from the validation report:

You cannot submit an add-on with a guid ending "@mozilla.org" or "@shield.mozilla.org" or "@pioneer.mozilla.org"

I can try using the signing API, but wouldn't be surprised if it's the same issue.

Edit: Same problem...no go via the signing API

[overdodactyl]

I just tried to upload a blank one using the id for uBO and it failed with the following error:

Your add-on failed validation with 1 error.

    * Duplicate add-on ID found.

So it doesn't look like that would be possible.

[Gitoffthelawn]

For Windows, I have written a script that runs Firefox using Shell.Run. (It does a lot more, but we can discuss that separately.)

I'm thinking that before the Shell.Run command, to delete any unwanted Firefox Feature .xpi files.

The challenge to doing this is that the script must have admin privileges in order to delete those .xpi files.

But it seems like a real bad idea to run Firefox from an admin elevated script.

Is it? And if so, is there a way to accomplish this goal programmatically (that is, to delete the unwanted .xpi's using admin privileges, then run Firefox without any privilege escalation)?

[claustromaniac]

The challenge to doing this is that the script must have admin privileges in order to delete those .xpi files.

Except on Firefox portable or installations outside of Program Files, I presume.

But it seems like a real bad idea to run Firefox from an admin elevated script.

Is it? And if so, is there a way to accomplish this goal programmatically (that is, to delete the unwanted .xpi's using admin privileges, then run Firefox without any privilege escalation)?

Nowadays Firefox runs sandboxed and there are now plenty of limitations on add-ons (since webextensions came to be), so I think it should be safe enough to run elevated. Still, you know how this has always been. You can't do much against zero-day attacks before they are discovered, and running Firefox with low privileges is one of the few things you can do.

As for your second question, I searched a bit and, apparently, it can be done using runas.exe, by passing trustlevel:0x20000 to it. Still haven't tried it out.

[Gitoffthelawn]

@claustromaniac Thanks. How do you tell if Firefox is actually running with elevated (admin) privileges?

[claustromaniac]

@Gitoffthelawn You mean... to see if the script is working?

You can see it in the processes tab in the task manager. Under Virtualization, it should say Not Allowed if the process is running with admin privileges, otherwise it means it's not. You may need to make that column visible first from the menu (View > Select Columns).

@Thorin-Oakenpants the prefsCleaner doesn't care about admin privs, though. I'm sure there must be ways to check for that in VBS and PowerShell scripts, but I don't know if that's what @Gitoffthelawn wants anyway.

[Gitoffthelawn]

@claustromaniac I just wanted to be able to know a way to determine whether or not FF is running with admin privs. Your tip regarding task manager will likely do the trick. Thanks. 👍

BTW, the column is entitled UAC Virtualization in Win7.

[claustromaniac]

BTW, the column is entitled UAC Virtualization in Win7.

True. Sorry about that.

You know, I'm a bit curious about what you said before that the script does more than just delete those xpi files. Is it related or more personal-preference stuff? If there are other good reasons for using this sort of launcher script, maybe others can benefit from something like it too.

BTW, I was now thinking that, for people that want to remove those files on a per-profile basis, like @overdodactyl, a script like this could simply rename/move the files instead of deleting them. I know it's far from ideal but it might be worth it in the long run. I know it wouldn't take too long to write a script like that.

[overdodactyl]

for people that want to remove those files on a per-profile basis, like @overdodactyl, a script like this could simply rename/move the files instead of deleting them

Definitely a possibility! Unfortunately that wouldn't allow you to run both profiles simultaneously, but for many use cases it would good way to go :)

[earthlng]

Couldn't you remove the system addons from the features folder and install them as normal addons in 1 profile?

[overdodactyl]

I just gave that a try and interestingly get the error, "This addon could not be installed because it has not been verified"

[earthlng]

Lol, mozilla and their double-standards ;)

maybe it works if you just put it in your profile's extensions folder instead of trying to install it from file (?)

[overdodactyl]

It is kind of ironic their own extension isn't signed haha.

No luck there either unfortunately...no error message but it just doesn't go into effect.

[earthlng]

^^ just tried that in my nightly and apparently activity-stream is not only unsigned but also a "legacy extension" - LO-fucking-L, mozilla and their fucking double-standards!

[earthlng]

but it has to work somehow - how else would updates to system addons be able to work? maybe it needs to go into a special profile subfolder? has anyone ever received an update for one of the system addons and would know where these files are placed?

[earthlng]

one workaround is to set extensions.legacy.enabled to true and xpinstall.signatures.required to false but that's only allowed in ESR and Nightly.

[overdodactyl]

Ya I wish you were allowed to change that setting in stable as well.

[overdodactyl]

Looking at the system addons, I'm curious how many of those are truly necessary:

• aushelper@mozilla.org.xpi
• firefox@getpocket.com.xpi
• followonsearch@mozilla.com.xpi
• formautofill@mozilla.org.xpi
• onboarding@mozilla.org.xpi
• screenshots@mozilla.org.xpi
• shield-recipe-client@mozilla.org.xpi
• webcompat@mozilla.org.xpi
• activity-stream@mozilla.org.xpi

[overdodactyl]

I'm not seeing any references to activity-stream within a specific profile

[overdodactyl]

I guess yet another workaround would be to simply use a different install of FF - one were the xpi file is deleted, one where it's not.

[earthlng]

https://firefox-source-docs.mozilla.org/toolkit/mozapps/extensions/addon-manager/SystemAddons.html

System add-on updates
These are installed into the users profile under the features directory.

[overdodactyl]

And even if an install like that was around, its more work than it is to just delete the xpi files manually

True, but I was trying to address the issue of wanting to be able to access activity stream in one profile, while simultaneously having it removed from another without user.js.

I agree though, the best solution unfortunately seems to be to add those prefs.

These are installed into the users profile under the features directory.

The system addons (for me at least) are located in the features directory of my Firefox app (/Applications/Firefox.app/Contents/Resources/browser/features), but the features directory of an individual profile is empty

[earthlng]

ok I figured it out, the trick is to customize the pref extensions.systemAddonSet and the xpi needs to go in profile/features/{UUID}/ where UUID is a valid, well, UUID :)

for example this works:

set extensions.systemAddonSet to {"schema":1,"directory":"{9c223810-fd9c-4c4d-809b-838623587fe7}","addons":{"activity-stream@mozilla.org":{"version":"2018.04.13.1371-d6edded2"}}}

To get the version of your activity stream xpi place it in your extensions folder and check about:support.
If necessary update the version in the extensions.systemAddonSet pref. Then move the xpi to profile/features/{9c223810-fd9c-4c4d-809b-838623587fe7}/

[overdodactyl]

Thanks @earthlng!! That seemed to do the trick :)

[Atavic]

@Thorin-Oakenpants
https://sql.telemetry.mozilla.org/dashboard/activity-stream-metrics-summary - requires login

[earthlng]

I created a patch (branch only, now deleted) with changes a week ago

I didn't realize it was in a different branch, I thought you already committed it to the master.

As long as AS remains a system addon, the short one without the new prefs is fine IMO.