But I want it online!
-
I recently went through our production database and deleted a little more than 100 gigbytes of stale reference data. Granted that's a lot of data, but in a 70+ terabyte database, it's a drop in the bucket. Still, it's a small step in the right direction.
That data is no longer used as it's all been superceded. It's all backed up numerous times on keep-forever tapes, so it's available for auditing. It was approved by the development managers, database managers and DBAs.
I hit the button, and in a short couple of hours, it was gone. The space was reclaimed. Joy.
The next day, someone noticed it in the status report and complained that the data might be needed for a production run.
Um, no; the newer data would be used. We are (by regulatory edict) prohibited from using the older data in the manner you suggest!
But what if regulations change?
It's backed up on tape; we can always get it back.
But I want it on line.
You also want the application to perform faster than a crippled slug stuck in molasses on a cold winter day!
...
Back and forth it went. Finally, I told the guy to take it up with the folks who actually pay the bills for faster servers and more disk space.
When they called me to ask, I pointed out the co$t savings they would realize by ignoring this guy.
They told him to F.O.
There is hope!
-
@snoofle said:
But I want it on line.
.. was gonna say: "provide me with a business case as to how your daily productivity is impacted by this 100G of stale data and we will look at funding its operational support"...
@snoofle said:
When they called me to ask, I pointed out the co$t savings they would realize by ignoring this guy.
.. but someone beat me to it!
@snoofle said:
They told him to F.O.
His data's held at the Foriegn Office? Good place for it, I guess.
I take it this is related to the other post about writing a report to clean stuffs down?
-
@Cassidy said:
@snoofle said:
But I want it on line.
.. was gonna say: "provide me with a business case as to how your daily productivity is impacted by this 100G of stale data and we will look at funding its operational support"...
Yeah, people hate having stuff taken away, even if they never use it. It's why we wind up with boxes in our basements that haven't been opened in ten years.
Sometimes you can get away with just deleting the data or feature or whatever, and NOT announcing you've done so...and waiting to see if anyone notices.
-
Yes, but what if we need to rerun a job from, say, August 1992? We can't afford to wait while you fetch that stuff back from archive tape ...
-
@FrostCat said:
Sometimes you can get away with just deleting the data or feature or whatever, and NOT announcing you've done so...and waiting to see if anyone notices.
I prefer announcing that I've deleted it and see who complains first. If there's no reaction, I can proceed.
I also knew of a DBA that would regularly (every 6-10 weeks) email around notifications that he'd made some tweaks and hopefully it would improve performance. He'd get a number of replies, some claiming things were better, others blaming lag on his change - even though he'd made no alterations whatsoever.
-
@Fjp said:
Yes, but what if we need to rerun a job from, say, August 1992? We can't afford to wait while you fetch that stuff back from archive tape ...
I heard from someone (NPR???) that it is no longer cost-effective to delete old emails -- that the cost of someone sitting and clicking the "delete" key is more than the cost (including ongoing costs) of keeping that data on a disk somewhere. We're living in a world where data just isn't going to be deleted, because it actually costs more to delete it than to keep it. Maybe you can justify deleting data by examining the cost to other systems for keeping that data online, but surely you could make a case for moving the old data to a backup server somewhere? Then you get the best of all worlds -- production speeds up; and no one can ding you for deleting potentially valuable data.
Just a thought.
-
@DrPepper said:
that the cost of someone sitting and clicking the "delete" key is more than the cost (including ongoing costs) of keeping that data on a disk somewhere.
Birdgit69.. that you?
Either way: I can't see how the cost of clicking "delete" can possily be lower than the cost of storing that data plus associated overheads of wading through trash to locate important stuff.
-
@DrPepper said:
@Fjp said:
Disk is cheap. Backups are expensive, but if you aren't going to back it up...Yes, but what if we need to rerun a job from, say, August 1992? We can't afford to wait while you fetch that stuff back from archive tape ...
I heard from someone (NPR???) that it is no longer cost-effective to delete old emails -- that the cost of someone sitting and clicking the "delete" key is more than the cost (including ongoing costs) of keeping that data on a disk somewhere. We're living in a world where data just isn't going to be deleted, because it actually costs more to delete it than to keep it. Maybe you can justify deleting data by examining the cost to other systems for keeping that data online, but surely you could make a case for moving the old data to a backup server somewhere? Then you get the best of all worlds -- production speeds up; and no one can ding you for deleting potentially valuable data.
Just a thought.
-
@DrPepper said:
I heard from someone (NPR???) that it is no longer cost-effective to delete old emails --
I've been saying that for years.
-
@Cassidy said:
@DrPepper said:
that the cost of someone sitting and clicking the "delete" key is more than the cost (including ongoing costs) of keeping that data on a disk somewhere.
Birdgit69.. that you?
Either way: I can't see how the cost of clicking "delete" can possily be lower than the cost of storing that data plus associated overheads of wading through trash to locate important stuff.
It's the wading through that's the problem. And the important email that gets lost in the shuffle and forgotten because of the 30 newsletters and 20 cat-photo chains and 50 quick back-and-forths and 200 automated notifications that came in since it was sent this morning.
That and email clients that search painfully slowly or just plain crash / get corrupted if you have too much data.
-
@blakeyrat said:
@DrPepper said:
You just delete new emails now?I heard from someone (NPR???) that it is no longer cost-effective to delete old emails --
I've been saying that for years.
-
@dkf said:
@blakeyrat said:
@DrPepper said:
You just delete new emails now?I heard from someone (NPR???) that it is no longer cost-effective to delete old emails --
I've been saying that for years.Without ever reading them.
Makes my day oh-so-much more productive!
-
@Rhywden said:
@dkf said:
I have my system set up to automatically reject emails that contain requests with a cryptic error message.@blakeyrat said:
@DrPepper said:
You just delete new emails now?I heard from someone (NPR???) that it is no longer cost-effective to delete old emails --
I've been saying that for years.Without ever reading them.
Makes my day oh-so-much more productive!
-
@Rick said:
Disk is cheap.
Purchasing disks is cheap. Managing the data stored on them can be costly. Purchasing more storage isn't a solution.@Rick said:
Backups are expensive
that, yup.@sprained said:
That and email clients that search painfully slowly or just plain crash / get corrupted if you have too much data.
Depends upon the storage type. For business use, most mail clients are front-ends to a managed service. For home use, many rely on maintaining their own DBs and they probably don't scale as well as server-based systems.
Either way, that's not a reason to limit storage, that's a reason to find a client that deals with your volumes of data.
-
@FrostCat said:
Yeah, people hate having stuff taken away, even if they never use it.
Digital hoarders. We all know folks who keep every file they ever downloaded, even though that WinZip 2.0 installation won't work on anything but Windows 95.
-
@Rick said:
Disk is cheap. Backups are expensive
Actually, backups are cheap too, it's the time and effort spent restoring that's expensive. Still, keeping stale data around only compounds the difficulty of your restoration strategy.
-
@Soviut said:
@FrostCat said:
Yeah, people hate having stuff taken away, even if they never use it.
Digital hoarders. We all know folks who keep every file they ever downloaded, even though that WinZip 2.0 installation won't work on anything but Windows 95.
That's why I kept my Win95 install floppies.
-
@Soviut said:
Actually, backups are cheap too, it's the time and effort spent restoring that's expensive
Storage has an associated cost, but much of that is probably measured in ongoing human effort to catalogue and safeguard libraries rather than purchase costs of storage space.
-
@Cassidy said:
Storage has an associated cost, but much of that is probably measured in ongoing human effort to catalogue and safeguard libraries rather than purchase costs of storage space.
The cost of detecting failing storage are appreciable (particularly as that's when the screw-ups happen). I've heard it argued that solid-state memory is a better solution than disks purely on the basis that you can keep it plugged in on a network and don't have the problem of someone dropping it and busting the hardware, and I shudder to think of the kind of disaster that caused that decision to be taken.
-
Sorry, I meant backup storage costs: maintaining tape libraries, paper records, security policies in place for check-in/out, audits to check for unauthorised retrieval.
I'm thinking more outside IT and places like financial houses, evidence lockups and big shadowy aircraft hangers repurposed for underhand government usage ala X-Files.
-
"But what if regulations change?"
"Get back to me after they do and we'll talk. Until then, please don't hesitate to GTFO."
-
@Cassidy said:
@FrostCat said:
Sometimes you can get away with just deleting the data or feature or whatever, and NOT announcing you've done so...and waiting to see if anyone notices.
I prefer announcing that I've deleted it and see who complains first. If there's no reaction, I can proceed.
I also knew of a DBA that would regularly (every 6-10 weeks) email around notifications that he'd made some tweaks and hopefully it would improve performance. He'd get a number of replies, some claiming things were better, others blaming lag on his change - even though he'd made no alterations whatsoever.
"I prefer announcing that I've deleted it and see who complains first."
In my experience, you can guarantee that someone will object, which is why I advocate doing it the other way. (In fact, I actually WAS the guy who complained, once...working for a company that gave me a dumb terminal instead of a PC in the 90s. One day they shut off the terminal network, and I was one of the few people who was still using it. At least THAT got me a PC.)
-
@FrostCat said:
"I prefer announcing that I've deleted it and see who complains first."
In my experience, you can guarantee that someone will object, which is why I advocate doing it the other way.
Firstly, I don't believe in keeping stakeholders in the dark. If there's some unrelated service loss and I'm asked "did you change anything?" then my reply is going to paint a target upon me - not only will I be blamed for the incident, but the real cause will still remain unknown whilst everyone's succumbing to target-blindness.
Secondly... I meant the approach of only announcing it but not implementing any change then see who seems to feel it's affecting their productivity, or who has been impacted by this change. It helps to draw out Walter Mitty in advance.
Thirdly - someone will always object. That doesn't mean the change cannot proceed, nor that their objections are unfounded. By drawing them out and addressing them, you keep people onside. Doing underhand things can only foster a culture of secrecy and distrust.
