From the department of forgetting to renew certificates
-
This time it's Mozilla. They apparently forgot to regenerate some certificates used in signing add-ons and they all stopped working.
https://www.reddit.com/r/firefox/comments/bkfte9/if_you_have_issues_with_your_addons_being_marked/
-
Certificates are, like, hard man.
-
With special support from the department of requiring third party authorization to run code on your own damn computer.
-
Would love to be a fly @Mozilla-HQ right now.
/'cos setting up an alert/check in Nagios|Monitoring-tool-du-jour is that hard
//getting approval to replace said cert, now that is a different beast
///actually finding the servers where aforementioned cert needs to be replaced might actually be unpossible 'cos of
anyways: Piehole running here, didn't notice any difference except the message from FF
-
They're rolling out an emergency fix through the Studies mechanism...
[FIXED] Certificate issue causing add-ons to be disabled or fail to install
![[FIXED] Certificate issue causing add-ons to be disabled or fail to install](https://discourse-prod-uploads-81679984178418.s3.dualstack.us-west-2.amazonaws.com/original/3X/1/e/1e9bf330a7eb757e98e400cdc695a6037f2ce9a0.png)
09:00 a.m. UTC/ 02:00 a.m. PDT: The team is still working on fixing and testing.
...the one they taught us all to turn off when they used it to deliver a Mr. Robot ad at the end of 2017.
https://what.thedailywtf.com/topic/23791/how-will-you-deal-with-the-coming-firefox-apocalypse/242
-
Relevant Discord screenshot:
-
"macOS Sierra Dark Theme" could not be verified for use with Firefox and has been deactivated.
Well, at least you didn't make that shit the default so the one part of the browser that remains functioning is correct colors.
EDIT: Guess I'll stay off the internet in general until ABP and GreaseMonkey are working again.
-
@Parody Don't they already have an efficient update mechanism? It shouldn't have taken them 10 hours to publish a fix.
-
@Parody The real fix, i.e. the same thing without the studies nonsense, is linked in the blog comments / on hackernews:
Samuel Vuorela said on blog.mozilla.org:
Why not just post a link to the fix that can be installed WITHOUT enabling Studies? This sounds like a clever plan to get more people to share their data via Studies…
The fix in question can be installed by clicking this link [1]. It’s signed by Mozilla.
Thanks to user gpm at Hacker News, who posted this tip [2].[1] https://storage.googleapis.com/moz-fx-normandy-prod-addons/extensions/hotfix-update-xpi-intermediate%40mozilla.com-1.0.2-signed.xpi
[2] https://news.ycombinator.com/item?id=19826903Just tried it and it worked for me.
-
The bug doesn't occur here, and I have "studies" disabled so I doubt I got the emergency fix. Dunno why, but I'm not complaining.
-
@iKnowItsLame said in From the department of forgetting to renew certificates:
Would love to be a fly @Mozilla-HQ right now.
Mozilla doesn't give two shits. That's why stuff like this happens.
-
It didn't occur for me for most of the day, but it did right now. Thankfully, the fix @topspin posted worked perfectly, and didn't even require restart! Thanks man.
-
Same here. I spoke too soon a few posts above.
-
Oh is that why Firefox disabled my addons earlier? I just filed it as a "next week problem" as it was on my work laptop and I don't need to worry about that until Tuesday.
-
@anonymous234 said in From the department of forgetting to renew certificates:
@Parody Don't they already have an efficient update mechanism? It shouldn't have taken them 10 hours to publish a fix.
They'll get there...eventually. (The last Twitter post I'd seen was that it's in testing now.)
@topspin said in From the department of forgetting to renew certificates:
@Parody The real fix, i.e. the same thing without the studies nonsense, is linked in the blog comments / on hackernews:
It didn't come up when I was poking around at it. More importantly, it's still the same workaround as the Shield Study: set the last time it checked everything back to when the certificate was still valid. Not the best Band-Aid®, but what can you do?
-
@Parody said in From the department of forgetting to renew certificates:
It didn't come up when I was poking around at it.I only posted that as a reply to you because I found it thanks to your link.
-
@topspin said in From the department of forgetting to renew certificates:
@Parody said in From the department of forgetting to renew certificates:
It didn't come up when I was poking around at it.I only posted that as a reply to you because I found it thanks to your link.
There wasn't a blog post when I posted it, IIRC.
-
@Parody It's cool, I was just trying to be helpful.
-
@topspin said in From the department of forgetting to renew certificates:
@Parody It's cool, I was just trying to be helpful.
Ah. Sorry. :)
-
@topspin said in From the department of forgetting to renew certificates:
@Parody It's cool, I was just trying to be helpful.
He didn't ask for help </blakey>
-
@topspin said in From the department of forgetting to renew certificates:
Works. Thanks.
I don't use Firefox a lot but I occasionally do some stuff on my old laptop and there are some websites that seem to work better with Firefox rather than Palemoon.
I did discover another work-around for this problem: install the latest ESR release of Firefox (60.6) which still allows you to disable Mozilla's retarded fuckery.
-
@Parody said in From the department of forgetting to renew certificates:
Not the best Band-Aid®, but what can you do?
Ideally: hard-code the checker to trust the expired certificate. Second-best band-aid: disable certificate checking for extensions entirely. It's not even a security risk, it's just executive wankery.
Then you reverse it when you renew your damn certificate.
-
@topspin said in From the department of forgetting to renew certificates:
@Parody The real fix, i.e. the same thing without the studies nonsense, is linked in the blog comments / on hackernews:
Samuel Vuorela said on blog.mozilla.org:
Why not just post a link to the fix that can be installed WITHOUT enabling Studies? This sounds like a clever plan to get more people to share their data via Studies…
The fix in question can be installed by clicking this link [1]. It’s signed by Mozilla.
Thanks to user gpm at Hacker News, who posted this tip [2].[1] https://storage.googleapis.com/moz-fx-normandy-prod-addons/extensions/hotfix-update-xpi-intermediate%40mozilla.com-1.0.2-signed.xpi
[2] https://news.ycombinator.com/item?id=19826903Just tried it and it worked for me.
This fix even works on Android! Although it does require restart here.
-
@levicki said in From the department of forgetting to renew certificates:
@anonymous234 said in From the department of forgetting to renew certificates:
Then you reverse it when you renew your damn certificate.
That's two builds to make, test and release on CDNs for little benefit except to placate a few angry users of a free, unsupported, product.
It doesn't help that they broke the builds:
From the bug description:
Linux builds are failing on trees where signatures are verified (mozilla-release, mozilla-esr60, mozilla-beta) and it blocks the release of the builds with the fix for the expired intermediate certificate for the add-ons.
-
@topspin said in From the department of forgetting to renew certificates:
@Parody The real fix, i.e. the same thing without the studies nonsense, is linked in the blog comments / on hackernews:
Samuel Vuorela said on blog.mozilla.org:
Why not just post a link to the fix that can be installed WITHOUT enabling Studies? This sounds like a clever plan to get more people to share their data via Studies…
The fix in question can be installed by clicking this link [1]. It’s signed by Mozilla.
Thanks to user gpm at Hacker News, who posted this tip [2].[1] https://storage.googleapis.com/moz-fx-normandy-prod-addons/extensions/hotfix-update-xpi-intermediate%40mozilla.com-1.0.2-signed.xpi
[2] https://news.ycombinator.com/item?id=19826903Just tried it and it worked for me.
I had an error message that Firefox could not download the xpi file due to a connection error. So I had to download it using Edge and then right-click the file and "Open with Firefox". That fixed it. Thanks
-
@levicki said in From the department of forgetting to renew certificates:
Never subvert proper checking
Proper checking was already broken here.
it is a security risk to not have signed addons because a) malware can change unsigned addons and you won't know it and b) sites can trick you into installing (or other malware can drop it in your Firefox install) addons which can steal your private data
Malware that's already running in your computer. You're already compromised, it's just a secondary measure to make it slightly harder to mess with the user's home page.
-
: What Firefox troubles?
-
@Parody Sounds like someone over there is having a really busy weekend.
-
@Parody said in From the department of forgetting to renew certificates:
...the one they taught us all to turn off when they used it to deliver a Mr. Robot ad at the end of 2017.
I missed that, but somebody apparently didn't:
-
Mozilla, has really fucked up this time, and according to /r/Firefox, the only reason people aren't jumping ship is because Chromium browsers are the only feasible alternative. I also find it poetically ironic that Mozilla, being the biggest supporter of Let's Encrypt, has fucked their certificates. Even worse, they set it to expire on a weekend, when absolutely nobody was in office.
Curb Your Enthusiasm Theme – 01:08
— Theme Guy
-
@Sumireko said in From the department of forgetting to renew certificates:
the only reason people aren't jumping ship is because Chromium browsers are the only feasible alternative
It's still better than all the alternatives.
-
@topspin said in From the department of forgetting to renew certificates:
@Sumireko said in From the department of forgetting to renew certificates:
the only reason people aren't jumping ship is because Chromium browsers are the only feasible alternative
It's still better than all the alternatives.
Which is a shame, really. As for Mozilla, they are about to learn the lesson of if they want to act like Chrome by "helping" the user regardless of their want, then people are just going to give up and go to Chrome anyway.
-
@levicki said in From the department of forgetting to renew certificates:
Finally there is fully automated LetsEncrypt.
I don't think that's so useful for code signing or intermediate CAs, which is what was going on here. SSL (and HTTPS) is a totally different class of use case (as it doesn't involve deploying signed data/code widely, unlike with code signing). No, they needed to take action on this a minimum of a few weeks ago, preferably a month or more, so that the switch over could have been done smoothly. Someone really fucked up.
-
@Sumireko said in From the department of forgetting to renew certificates:
they set it to expire on a weekend, when absolutely nobody was in office.
I doubt anyone's even thought about what day of the week the expiry was because expiry would have just been start + X rather than a specially chosen day and no-one would have imagined it expiring before being renewed anyway.
Someone done fucked up.
-
@levicki said in From the department of forgetting to renew certificates:
malware can change unsigned addons and you won't know it
Or, malware could change the Firefox executable to have malware built in. Or install eFast change all shortcuts to Firefox to eFast instead. Or change the Firefox executable to not check signatures, making the whole point moot wankery in the first place.
-
The cherry on the cake:
https://twitter.com/dreamcat4/status/1124630192486481922The URL it's pointing to? discourse.mozilla.org.
-
@levicki said in From the department of forgetting to renew certificates:
Finally there is fully automated LetsEncrypt. There is absolutely no vaild excuse for this mistake -- we don't even need to know how and why they fucked up, but the heads should roll.
Well we know they don't check their LE mechanisms – some server or other went through several renewals which simply were never deployed to production last autumn (IIRC). End result: you simply didn't get addon updates, and the only way to find out was a singular line in the error console.
-
I have not noticed this with Firefox on Windows or macOS, got no warnings or deactivated addons the latest few days. But upon coming to work I got a nice yellow banner in Firefox informing me of deactivated addons. But my addons are still active and working. So, uh, I dunno...
One or more installed addons not can be verified and have been deactivated.
(I preserved the bonus WTF in the error message word order.)
-
@Zerosquare said in From the department of forgetting to renew certificates:
The URL it's pointing to? discourse.mozilla.org.
502 OK.
-
@Atazhaia said in From the department of forgetting to renew certificates:
I have not noticed this with Firefox on Windows or macOS, got no warnings or deactivated addons the latest few days. But upon coming to work I got a nice yellow banner in Firefox informing me of deactivated addons. But my addons are still active and working. So, uh, I dunno...
One or more installed addons not can be verified and have been deactivated.
(I preserved the bonus WTF in the error message word order.)That's about what happened to me with Firefox on Ubuntu 18.04 (1), except that when I clicked the "learn more" button, it showed me the Add-ons Manager, where it told me (in non-selectable text
) that some add-ons could not be verified and had been disabled. Which ones? Er. None at all. (1) Firefox on Ubuntu is several different levels of
all its own, since it's Canonical's build, not Mozilla's, and automatic installation of updates cannot be turned off at all, not even into "automatically find and download them and tell me about it so that I can choose when to have my work interrupted by a browser that wants to restart." (It's a $JOB PC.)It wants to restart. It probably even tries to restart. But of course it fails miserably at doing more than just exiting. (Well, it probably finishes the installation, but it doesn't restart like it promised it would.)
Mozilla's forums provided the answer on how to resolve this: uninstall Canonical's build and install Mozilla's in its place, although I'm a little worried about what it will manage to break if I try.
EDIT: it occurs to me that I'm contemplating removing Canonical's build in favour of a canonical build...
-
@Steve_The_Cynic said in From the department of forgetting to renew certificates:
(1) Firefox on Ubuntu is several different levels of all its own, since it's Canonical's build, not Mozilla's, and automatic installation of updates cannot be turned off at all
Compared to Firefox on Linux Mint, which also uses a custom build, but as it is updated through the update manager I as the user has full control on when updates are installed. (Unless I explicitly turn on automatic updates, but that would be
, especially on a work PC.)
-
@Atazhaia said in From the department of forgetting to renew certificates:
But my addons are still active and working. So, uh, I dunno...
I wish I had this luck, but my addons have been deactivated.
No way I'll be using my browser without Tree Style Tab and uBlock Origin, so I'm going to apply post-haste the fix someone posted previously. It's already worked in another computer where this happened.
-
To me it looks like they didn't merely forget to renew a certificate, but also forgot to have their signed extensions timestamped, which exists precisely to avoid this problem (having a signature timestamped by a timestamping authority makes it stay valid after the certificate expires).
-
@topspin said in From the department of forgetting to renew certificates:
@Parody The real fix, i.e. the same thing without the studies nonsense, is linked in the blog comments / on hackernews:
Samuel Vuorela said on blog.mozilla.org:
Why not just post a link to the fix that can be installed WITHOUT enabling Studies? This sounds like a clever plan to get more people to share their data via Studies…
The fix in question can be installed by clicking this link [1]. It’s signed by Mozilla.
Thanks to user gpm at Hacker News, who posted this tip [2].[1] https://storage.googleapis.com/moz-fx-normandy-prod-addons/extensions/hotfix-update-xpi-intermediate%40mozilla.com-1.0.2-signed.xpi
[2] https://news.ycombinator.com/item?id=19826903Just tried it and it worked for me.
@topspin , I can't thank you enough.
-
You shouldn't need the temporary fix any longer, Mozilla finally released a new version of Firefox (66.0.4) which has the bug fixed.
-
@levicki said in From the department of forgetting to renew certificates:
Except it is a security risk to not have signed addons
Signed addons is a great idea. Making them mandatory is perfectly fine and sensible.
Until you fuck it up.
And then all your users are fucked because it's impossible to install any extensions.
And this isn't even the first time. Mozilla did this same thing almost exactly 3 years ago (May 2, 2016).
https://i.imgur.com/DHrOyTN.gif
-
@levicki What is Normandy?
-
@Medinoc said in From the department of forgetting to renew certificates:
@levicki What is Normandy?
It's the internal name for the mechanism Firefox uses to install Shield Studies and slowly roll out new preferences. In the Options it's under "Privacy & Security"/"Firefox Data Collection and Use"/"Allow Firefox to install and run studies".
-
@El_Heffe said in From the department of forgetting to renew certificates:
And this isn't even the first time. Mozilla did this same thing almost exactly 3 years ago (May 2, 2016).
First time is a warning, second time, someone better be getting shitcanned
-
@Medinoc said in From the department of forgetting to renew certificates:
@levicki What is Normandy?
A region along the French coast across the channel from Britain.
1548973 - (armagadd-on-2.0) [meta] All extensions disabled due to expiration of intermediate signing cert
