Boss's luck has ran out

cartman82

3 months ago

Me: Hey boss, the entire VPS host has been compromised. They have a backdoor into each VPS. We need to purge everything, rebuild all VPS-s, reinstall everything.

Boss: Ugh... we need to deliver all these features. And these are all old projects in maintenance mode. Nothing's happening anyway. Let's do it later.

2 months ago

Me: Just a quick reminder, the VPS host is compromised. We need to rebuild it.

Boss: I know, I know. Anyway, about that new feature...

1 month ago

Me: Boss, about the VPS host...

Boss: Yes, yes. Anyway, that latest project...

Today

Client: Your server is down

Me: (RDP in)

0_1494861127384_i201^cimgpsh_orig.png

************************************************************
SYSTEM DAMAGED! FILES WILL BE DELETED!
URGENT ATTENTION!
************************************************************
To restore your files and access them, you need to pay 5 bitcoins
Bitcoins have to be sent to this address: *********
After payment contact us to receive your password key.
Contact Email : *****@keemail.me
With subject (Personal ID) : *********
In order to purchase Bitcions you can use :
www.coinbase.com
www.localbitcoin.com
============================================================
IMPORTANT! IF YOU DON'T PAY IN MAXIM 24 HOURS ALL YOUR FILES
WILL BE PERMANENTLY DELETED!!!
============================================================

Me: Hmm, what about backups?

0_1494861275842_2017-05-15 17_14_22-Clipboard.png

Well, fuck.

pydsigner

@cartman82 Relax, bitcoins aren't real money. 5 of them? That'll only be like 10k USD.

cartman82

@pydsigner said in Boss's luck has ran out:

@cartman82 Relax, bitcoins aren't real money. 5 of them? That'll only be like 10k USD.

Hahahahaha. They are not getting that money from us.

Weng

Note that after you've fixed it, this will be all your fault.

pydsigner

@Weng said in Boss's luck has ran out:

Note that after you've fixed it, this will be all your fault.

I am a little surprised that @cartman82 didn't take some backups 3 months ago when his boss refused a full fix just so there'd be some semblance of failguard.

The_Quiet_One

Was it just a coincidence this happened during the big WannaCry attacks, which to my knowledge could be fixed by simply keeping your Windows up to date? Sounds like this was due to a different problem, but just wanted to make sure.

accalia

@cartman82 said in Boss's luck has ran out:

Me: Boss, about the VPS host...

@cartman82 said in Boss's luck has ran out:

Me: Boss, about the VPS host...

please please please tell me you have that documented......

cartman82

@Weng said in Boss's luck has ran out:

Note that after you've fixed it, this will be all your fault.

Nah. Boss has his faults, but he's not the scapegoatey type.

@pydsigner said in Boss's luck has ran out:

I am a little surprised that @cartman82 didn't take some backups 3 months ago when his boss refused a full fix just so there'd be some semblance of failguard.

I didn't mentioned it in OP, but in the meantime, Boss has hired a part time admin. So I was officially off the server jockeying duty. It was the admin's duty to do the backups.

Unfortunately, this guy's attitude is kind of "tell me what you want done, and I'll get it done". He didn't quite take on the responsibility for proactively taking care of things.

But whatever. Not my problem anymore, so...

cartman82

@The_Quiet_One said in Boss's luck has ran out:

Was it just a coincidence this happened during the big WannaCry attacks, which to my knowledge could be fixed by simply keeping your Windows up to date? Sounds like this was due to a different problem, but just wanted to make sure.

This was like Windows 2008 that hasn't been touched in months, so yeah, could be.

We are kind of off Windows platform. We just had the last few holdout projects on this last god forsaken server.

I guess we are 100% off Windows now :)

The_Quiet_One

@cartman82 said in Boss's luck has ran out:

Unfortunately, this guy's attitude is kind of "tell me what you want done, and I'll get it done". He didn't quite take on the responsibility for proactively taking care of things.

Uuuh, isn't autonomously managing servers without being told what to do kinda one of the fundamental responsibilities of an admin?

cartman82

I've been waiting an hour for Boss to take the initiative here. After swearing that he doesn't negotiate with terrorists, and telling me not to do anything, nothing.

Me: So... should I tell the admin to isolate the server from network, so it doesn't spread to other VPS-s?
Boss: Yes! Yes! And tell him to fix this.
Me: (Fix.... how? .... nevermind)

So he really doesn't give a shit about this server and the client.

Whatever. Off my plate this goes.

cartman82

Client's "admin" is trying to help.

0_1494869007863_c55a1744-b4ed-4afe-bc8f-f2dedc0648ea-image.png

wharrgarbl

@cartman82 did you try (first google result tip)?

e4tmyl33t

@cartman82 Toby Faire, Malwarebytes totally does work great for removing malware on a PC.

You can probably even use it as a realtime scanner if you buy a subscription to it, rather than just running one-off scans periodically. Whether you'd want to do that on a server/VPS is something else.

As is the fact that while it may remove the malware, it sure as hell ain't decrypting anything it's got its mitts on...

Dreikin

This thread reminds me I really need to update. I tried a few days ago but it wouldn't do anything beyond saying "Downloading 0%".

dcon

@Dreikin said in Boss's luck has ran out:

This thread reminds me I really need to update. I tried a few days ago but it wouldn't do anything beyond saying "Downloading 0%".

I had that issue on one machine. Took running the troubleshooter on it to finally clean that up.

xaade

@cartman82 said in Boss's luck has ran out:

Me: Hmm, what about backups?

How did they get to your backups?

izzion

@xaade
Backup volume is left attached to the Windows PC at all times (and usually enumerated as a drive letter, though modern cryptoware will search all attached volumes whether they have a drive letter or not).

xaade

@izzion Right, they didn't make any external backups.

pydsigner

@xaade said in Boss's luck has ran out:

@cartman82 said in Boss's luck has ran out:

Me: Hmm, what about backups?

How did they get to your backups?

They didn't.

@xaade said in Boss's luck has ran out:

@izzion Right, they didn't make any external backups.

They did. You'll notice that there is a backup, and that it's an image of the entire VM (this is a VPS, after all). The problem is the timestamp on the backup. Good luck making your client happy with a 15-months-dead restore.

xaade

@pydsigner said in Boss's luck has ran out:

The problem is the timestamp on the backup. Good luck making your client happy with a 15-months-dead restore.

Ah, I overlooked the date.

wharrgarbl

@cartman82 said in Boss's luck has ran out:

@pydsigner said in Boss's luck has ran out:

@cartman82 Relax, bitcoins aren't real money. 5 of them? That'll only be like 10k USD.

Hahahahaha. They are not getting that money from us.

If it's this new wannacry thing, paying wouldn't help

David Glance / Oct 21, 2019

WannaCry hackers had no intention of giving users their files back even if they paid

As the effects of the WannaCry global ransomware attack continue to be felt, there is evidence emerging that the hackers are not decrypting the files of victims who decide to pay the US $300 ransom. A…

dangeRuss

@wharrgarbl said in Boss's luck has ran out:

@cartman82 said in Boss's luck has ran out:

@pydsigner said in Boss's luck has ran out:

@cartman82 Relax, bitcoins aren't real money. 5 of them? That'll only be like 10k USD.

Hahahahaha. They are not getting that money from us.

If it's this new wannacry thing, paying wouldn't help

David Glance / Oct 21, 2019

WannaCry hackers had no intention of giving users their files back even if they paid

As the effects of the WannaCry global ransomware attack continue to be felt, there is evidence emerging that the hackers are not decrypting the files of victims who decide to pay the US $300 ransom. A…

Sounds like MS released it on purpose to get people to upgrade....

e4tmyl33t

@Dreikin Yeah, I just updated & ran the copy on my main PC on Saturday or so as my monthly "just-in-case".

coldandtired

Have you tried password or hunter2?

masonwheeler

@dcon You actually got the Windows Troubleshooter to successfully fix a problem ??? !!! ???

dcon

@masonwheeler said in Boss's luck has ran out:

@dcon You actually got the Windows Troubleshooter to successfully fix a problem ??? !!! ???

I did !!!

masonwheeler

@dcon I think that's the first time in the history of ever that I've ever heard of Troubleshooter successfully doing anything useful.

dcon

@masonwheeler said in Boss's luck has ran out:

@dcon I think that's the first time in the history of ever that I've ever heard of Troubleshooter successfully doing anything useful.

Specifically, I came into it by clicking 'Windows Update':
0_1494883161950_4753ea6f-e1e2-4b15-96cf-6b6665316b71-image.png

bb36e

@masonwheeler said in Boss's luck has ran out:

@dcon You actually got the Windows Troubleshooter to successfully fix a problem ??? !!! ???

I use it as a shortcut for resetting my network connection

RaceProUK

@bb36e said in Boss's luck has ran out:

@masonwheeler said in Boss's luck has ran out:

@dcon You actually got the Windows Troubleshooter to successfully fix a problem ??? !!! ???

I use it as a shortcut for resetting my network connection

That's the only problem it can actually fix ;)

Dreikin

@dcon said in Boss's luck has ran out:

@Dreikin said in Boss's luck has ran out:

This thread reminds me I really need to update. I tried a few days ago but it wouldn't do anything beyond saying "Downloading 0%".

I had that issue on one machine. Took running the troubleshooter on it to finally clean that up.

My desktop acts weird across sleeps. Today it's working (at 14% now).

Dreikin

@masonwheeler said in Boss's luck has ran out:

@dcon I think that's the first time in the history of ever that I've ever heard of Troubleshooter successfully doing anything useful.

On Windows 7 it used to help with one of the problems from the aforementioned sleep weirdness. Specifically, I sometimes have to disable and then re-enable the wireless adapter to get connected again. The troubleshooter used to do that for me, but it stopped in Windows 10 so I have to use device manager now.

Dreikin

@bb36e said in Boss's luck has ran out:

@masonwheeler said in Boss's luck has ran out:

@dcon You actually got the Windows Troubleshooter to successfully fix a problem ??? !!! ???

I use it as a shortcut for resetting my network connection

@RaceProUK said in Boss's luck has ran out:

@bb36e said in Boss's luck has ran out:

@masonwheeler said in Boss's luck has ran out:

@dcon You actually got the Windows Troubleshooter to successfully fix a problem ??? !!! ???

I use it as a shortcut for resetting my network connection

That's the only problem it can actually fix ;)