How to fool customers
-
So, aside from interviews being bizarre I also had a bizarre case yesterday.
For context, we only provide support for a small number of PCs on a hospital network - and ours usually hide behind a NAT/PAT (Linux Zentyal box) that only lets in traffic on a select bunch of ports only.
So this site managed to get a hospital wide virus infection. They usually don't spread to us, but this time they did. My policy, since we do control software for radiation treatment, is to reinstall any servers and any machines involved in real time treatment delivery again to make sure they are clinically safe. Workstations where doctors letters are written or where treatment plans are reviewed I am less strict about.
This attitude, as usual, clashed with customer IT who had disinfected the server PCs and were assuring me they were fine. In the fifteen minutes I was one of those things the error log had ballooned to the size of the New York phone book and I had to reset my teamviewer (yes, I hate it but they wanted to see what I'm doing) connection because of a bsodding driver issue - but sure, everything is fine. What could possibly happen if the server bsods during a treatment. Naturally, IT's driver reinstallation did not make this go away.
I was arguing with IT when I remembered these guys don't know anything about MS SQL. So while team viewer was running I fired up the VPN, opened my trusty old friend SQL server management studio, set the database to emergency mode and then proceeded to show them in the other session.
"Have you seen the database is red? We have data corruption!!!1111!!!oneone1! Oh my god, what do we do?"
"Don't worry. If you set up a new server for me, I can restore from a backup."
"Yes, give us an hour."
Problem solved.
-
IT
They do not get to use that term to describe themselves if they're honestly OK with that. That is (or should be) criminally incompetent - The only way to "disinfect" a computer is to reinstall it, full stop.
Props on being able to scare them into reinstalling though - you sound like those PC tech support scammers though :)
-
Yeah it's a shitty thing to do, but I feel more obliged to the customer's patients than their IT - so I do what is needed whether I like it or not.
-
Absolutely.
If you could get those "IT" idiots fired, you'd probably be doing the world a favor too :)
-
It's always scary to see what clowns get jobs in IT at hospitals.
I have one customer who insists its impossible to schedule a monthly reboot on a Windows server 2008 r2 PC because the OS won't allow an auto login.
One of those days where I needed lavender tea to keep calm.
-
-
The only way to "disinfect" a computer is to reinstall it, full stop
That hasn't been strictly true for as long as motherboards have had flashable BIOSes, and now that UEFI is ubiquitous it won't be long before malware is routinely hiding spores in there. Nuke it from orbit, it's the only way to be sure.
-
its impossible to schedule a monthly reboot on a Windows server 2008 r2 PC because the OS won't allow an auto login
Leaving aside the fact that login on a server is nothing more than a mere convenience, Sysinternals Autologon works just fine on 2008r2.
-
preaching to the converted here XD
-
The only way to "disinfect" a computer is to reinstall it
Unless the virus modified some firmware. Then you may be fucked.
It's always scary to see what clowns get jobs in IT at hospitals.
This. I've seen hospital "IT" people who didn't know what SQL was and couldn't understand why I went into panic mode when I found a Windows 95 PC with internet access.
-
BTW: It's funny what kind of antiquated hardware and software you sometimes find in hospitals. One of the hospitals I've been in had an X-ray machine running SunOS 2 or 3, in 2010.
-
couldn't understand why I went into panic mode when I found a Windows 95 PC with internet access
It's well known in certain quarters that Windows 95 is actually more secure than any modern Windows because none of the extant malware is compatible with it any more.
-
I have one customer who insists its impossible to schedule a monthly reboot on a Windows server 2008 r2 PC because the OS won't allow an auto login.
Wow that's wrong in like 3 or 4 different ways.
- If you need to schedule a reboot on a server, fix it instead. It's broken
- You can use auto-login on Server 2008, I don't think the option even changed location in the control panels
- Sometimes my toilet seat's really cold in the morning despite the rest of the house being like 70, I don't get why this happens
- Why would it matter if you're logged-on to the server or not? If your server requires a logged-on user to function, fix it instead. It's broken
-
I have one customer who insists its impossible to schedule a monthly reboot on a Windows server
Upgrade them to 2012 and it should automatically force a reboot for updates
-
Unfortunately, they insist on using a retarded workflow that was never intended by development and would be a non-trivial change to implement properly for a prospective customer base of one.
Restarting the application/ service would actually suffice but according to this customer that is "not possible" so they need the auto-login restart which is also not possible.
Oh woe is them.
-
In other news because it is fun.... The same site that sparked this topic called me again today because they managed to break he network connection to their treatment hardware.
Since yesterday they closed the vpn, all outgoing traffic and made the site a "virtual prison" as they call it. Securitate to prevent another virus infection.
So patients are not treated for 6 hours today because I needed to explain to an idiot how to edit a batch file to actually address the correct network card on the server. Their IT decided it would be fun to just reinstall all the drivers and change the indices.
Apparently, typing wmic nic is really, really hard.
-
@royal_poet said:
I have one customer who insists its impossible to schedule a monthly reboot on a Windows server
Upgrade them to 2012 R2 and it should automatically force a reboot for updates
I think. Honestly I don't even know if the Windows 10 server equivalent does that. I know 2012 doesn't though, I've got a couple Server 2012 machines and they only reboot when I tell them too (which unfortunately usually is not often enough)
-
If the application's running as a service, they shouldn't need to auto-login. The service should start back up automatically, regardless of whether anyone logs in.
-
If you need to schedule a reboot on a server, fix it instead. It's broken
But, you said it's faster then restarting a service, and it's the only way to apply updates.
-
Sometimes my toilet seat's really cold in the morning despite the rest of the house being like 70, I don't get why this happens
You need to set up watercooling on your PC, but get rid of the radiator, and run the cooling pipes to the toilet seat.
-
If your server requires a logged-on user to function, fix it instead
Yeah, I don't write that particular software that requires this. It's relegated to the last remaining Windows 2003 machine on the network.
-
But, you said it's faster then restarting a service,
Quote me.
and it's the only way to apply updates.
Are these monthly reboots specifically for updates, or just because the buggy-ass software fills up RAM with its memory leaks? You seem to be making assumptions not in the original post.
-
It's actually a memory fragmentation compounded by severely under spec hardware issue. The module they have does some image processing on CTs. It wants 8GB of memory, prefers 16GB but actually has 2GB. It also likes to have a processor core of its own which its not getting with these guys.
When a new CT comes in, the software tries to allocate a continuous block of memory for it.
On most systems this doesn't cause issues because CTs are a decent size. These guys do a head to toe CT of every patient which is a huge datafile. Eventually the system (that also has some other software on it that is not from us) just chokes the application to death.
It doesn't fail as such - it just gets stuck in paging hell.
Of course the hospital can't afford a new server - and of course Dev refuses to write a patch for them because it's ridiculous.
So failure eventually comes every two weeks to a month when the memory looks like a Swiss cheese.
-
Quote me.
No, because Discosearch.Beside, when I prove you wrong, you just stop answering.
Are these monthly reboots specifically for updates, or just because the buggy-ass software fills up RAM with its memory leaks? You seem to be making assumptions not in the original post.
Doesn't change the outcome : reboot = update applied && memory leak fixed.
-
Of course the hospital can't afford a new server
And they won't pay for a couple of DIMMs? That could be paid for with the padding one one CT.
-
Customers are s
-
And they won't pay for a couple of DIMMs? That could be paid for with the padding one one CT.
We have an industrial CT that had similar issues that we acquired by picking the corpse of one of our competitors. Their transplanted scientists (we didn't actually get them when we bought their equipment, they just happened to see which way the wind was blowing) swore up and down that you couldn't get RAM that would work in it anywhere. (It was ECC registered - nothing says like running WindowsXP on server hardware)...
The thing sure does run better now that it can stitch the images together without having to page.
-
-
-
-
A mythical beast most IT departments are forever searching for.
See Holy Grail.
-
-
-
-
-
We have one new resident of Lojbanistan and that means two people here speak it so they talk to each other.
-
-
?won nabjoL si hsilgnE sdrawkcaB
-
If you need to schedule a reboot on a server, fix it instead. It's broken
So...install Linux?
-
I think. Honestly I don't even know if the Windows 10 server equivalent does that. I know 2012 doesn't though, I've got a couple Server 2012 machines and they only reboot when I tell them too (which unfortunately usually is not often enough)
2012 and 2012R2 default behavior is to install updates and then reboot whenever the hell it feels like it. You have changed that behavior.
-
2012 and 2012R2 default behavior is to install updates and then reboot whenever the hell it feels like it. You have changed that behavior.
Are you sure? The last couple 2012R2 VMs I built defaulted to "download only" for updates. Or maybe I'm just …
-
?os dneterp ot ynnuf ti si dna ,retcarahc LTR na ereht si ,sdneped tahT
-
Are you sure?
Positive. From clean ISOs, the behavior we see is to install updates and then reboot. That caught us up twice before we started adding it to the checklist. ( on our part)
The last couple 2012R2 VMs I built defaulted to "download only" for updates. Or maybe I'm just …
Are you sure those ISOs are not modified in any way?
I am curious now though, I am going to spin up a fresh VM and check.
-
Are you sure those ISOs are not modified in any way?
Pretty sure I downloaded it using my MSDN account (it's been a few months).
-
Creating VM right now.
-
I quite like 'Nabjol' though, I'm saving that one for later...
-
I quite like 'Nabjol' though, I'm saving that one for later...
It is sort of a palindrome, it is jibberish L-R and R-L.
-
It seems the default is "Not Configured".
But, when you go to install updates the first time it nags you to turn on automatic updates. I wish I had looked more carefully, but may have clicked the wrong button. I think as soon as you check for updates it enables it.
Automatically installing updates triggers an automatic reboot.
If anyone is interested, I could blow out that VM to get back to a freshly installed state and see for certain. I should have taken another screenshot.
-
It seems the default is "Not Configured".
Huh. Well, I guess that's better than defaulting to "I will reboot for updates when I damn well please."
-
Agreed. I am about 90% sure that the first time you update it forces you to turn on automatic updating, that you can then turn back off if you wish.
But I am only about 30% sure of that. ;)