SSD question-- compress drive?
-
Simple question: is there any reason to not use the NTFS option "compress this drive to save disk space" on an SSD?
-
@blakeyrat To provide a discussion point maybe, here's an article that says: "no, there's no reason to leave it off":
-
Gut reaction: No, there's not reason to leave it off, unless you were intending to use the drive outside Windows.
-
@Tsaukpaetra Right; the article on Toms says it increases writes which could (in 50 years) lower the drive's lifetime, but that's only while it's compressing the files. Once they're compressed, it should REDUCE reads and writes.
-
@blakeyrat said in SSD question-- compress drive?:
Once they're compressed, it should REDUCE reads and writes.
Exactly. And if the FS starts out compressed, the first portion doesn't even happen to begin with.
IIRC Windows 10 is supposed to be smarter with this and applying compression where it thinks the system would benefit, but I can't site any sources at the moment.
-
@Tsaukpaetra It's creeping me out because it gives me this:
Uh, wha? Why are you trying to do disk compression on the swapfile? Is there another UI I should be using to turn this on?
-
@blakeyrat Since that error is creeping me out, I'm going to specifically just check the "compress" box on Program Files and Program Files x86, which is 95% of my drive anyway.
-
@blakeyrat said in SSD question-- compress drive?:
Uh, wha? Why are you trying to do disk compression on the swapfile? Is there another UI I should be using to turn this on?
Short answer: Because the dialog isn't very smart. ;)
No, that's probably the easiest way to apply it to particular files/folders, there's probably a command-line applet to do it but you probably don't need it.
-
@Tsaukpaetra Man when I used it back in XP I seem to recall it being smart enough to skip system files without asking. But oh well.
-
@blakeyrat said in SSD question-- compress drive?:
Man when I used it back in XP I seem to recall it being smart enough to skip system files without asking.
Or it failed silently. Don't ever attribute intelligence to that which might be accidental. ;)
Silently ignoring errors is one of my peeves.
At least the message you got included "Ignore all" so you could know that, despite telling Windows "Compress All the Things!", some of them might not get compressed for whatever reason and to expect that.
-
@blakeyrat said in SSD question-- compress drive?:
is there any reason to not use the NTFS option "compress this drive to save disk space" on an SSD?
The data volume is reduced by at some cost in read/write delay and CPU usage.
They do not say what the delay is, and probably it is not trivial to measure, because of caches.
Maybe the trade is worth it.However, if one is willing to sacrifice some speed of access to increase the available disk size, switching to magnetic HDD would yield significantly more space.
The crucial question is how does the speed of SSD+compression compare to HDD.
-
@blakeyrat said in SSD question-- compress drive?:
Man when I used it back in XP I seem to recall it being smart enough to skip system files without asking.
Seems to be the expected behaviour
-
@Jaloopa Right but the thing is, I thought it just skipped those, not give me a "no permissions" warning.
-
Anyway it took a few hours but my SSD is all compressed now. I had to do some close-Explorer juju to get 7-Zip's Explorer plugin DLL to compress. So it took me like 20 minutes to save like 30k, that's a good trade-off.
No noticeable difference in performance, not that I'd expect any with an i7 and a SSD. But about 40 free GB I didn't have before.
-
@blakeyrat said in SSD question-- compress drive?:
I had to do some close-Explorer juju to get 7-Zip's Explorer plugin DLL to compress
Why not just put all your files inside 7z archives? Sure it'll take longer to access them, but you can store more files on your disk that way!
-
@blakeyrat said in SSD question-- compress drive?:
20 minutes to save like 30k
30kb? That's damn terrible!
-
@blakeyrat said in SSD question-- compress drive?:
Right but the thing is, I thought it just skipped those, not give me a "no permissions" warning.
Well yeah. Like the article says, boot files won't give an error so something done fucked up
-
@blakeyrat totally was't worth it. Especially when you consider the hidden costs of added complexity and brittleness.
-
@Tsaukpaetra said in SSD question-- compress drive?:
if the FS starts out compressed, the first portion doesn't even happen to begin with.
You don't even need to compress the whole FS. Creating an empty folder, then applying the compression attribute to that, will cause any files you put inside it afterwards to be stored compressed.
-
@blakeyrat said in SSD question-- compress drive?:
@blakeyrat Since that error is creeping me out, I'm going to specifically just check the "compress" box on Program Files and Program Files x86, which is 95% of my drive anyway.
No reason to be creeped out. If you just click Ignore on files that Windows is trying to compress but can't, nothing bad will happen. And if their containing folders have Compressed turned on, then the files concerned will get compressed next time they're rewritten if it's safe to do that.
-
This post is deleted!
-
@blakeyrat said in SSD question-- compress drive?:
Once they're compressed, it should REDUCE reads and writes.
Shouldn't reads be not much of a problem with SSDs anyway? I don't think they cause a state change of the limited-lifetime components…
Reducing writes OTOH, might be worth it and the basic compression algorithms with direct hardware support are really fast.
-
I've being doing to my os disks for a while and haven't noticed anything awful going on besides more games on disk. Still ran out of space because I'm so going to replay all those games I've finished. Then I discovered that maven had eaten a third of my harddrive uncovering why our build machines need regular cleaning. >_<
-
OH NO SIDE EFFECTS
Now explorer shows that little two arrows kissing each other icon on every single icon on my computer, which is mildly annoying!!!
But I'm too lazy to figure out how to turn that icon overlay off.
-
@blakeyrat How lucky for you, then, that somebody else has done it for you.
-
@Tsaukpaetra said in SSD question-- compress drive?:
Gut reaction: No, there's not reason to leave it off, unless you were intending to use the drive outside Windows.
A very valid point!
There are now three reasons why I will never use drive compression. Well, there have always been three, but this third and "new" reason has always been latent and thank you @Tsaukpaetra for articulating it in a way that that caused a "light bulb moment".
-
System performance: Drive / file compression has been around longer than systems have had the performance capability to make it un-noticeable. So I guess it has become habit not to.
-
Loss: Every time the file is re-compressed you re-introduce the chance that it may get corrupted. Witness JPEG.
-
Longevity: I have digital files going back 20+ years, who is to say (or guarantee) that a file compressed using the tech available today will still be accessible in 20 years times. It's bad enough that I have to cope with the current version of Word being unable to natively read files saved by the previous version of it (an over dramatization but still true).
So. If I need more disk space I either archive off with redundant copies or I get a bigger disk.
INB4: Apologies if this point or similar has been made down thread, but this thread has now become irrelevant to me and other that to derive amusement from subsequent "discussion", I have no reason to read further.
-
-
@loose said in SSD question-- compress drive?:
Every time the file is re-compressed you re-introduce the chance that it may get corrupted. Witness JPEG.
This point is completely inapplicable to lossless compression schemes like that used by NTFS for file compression.
-
@flabdablet Oh dear! My bad! Wrong example and wrong use of an example. In order to compress something, something has to be "lost". What is lost has to be "recreated". The higher the compression (file type and content not withstanding) the more has to be discarded or re-encoded in a "different way". JPEG is a worst case example of this.
In a related way: there is the consideration of just how much reduction you are getting for the effort involved.
-
@loose said in SSD question-- compress drive?:
In order to compress something, something has to be "lost".
Think of compression as an algorithm for creating instructions on how to recreate the original input data. If the data was something trivial like
000000000000
… then it would be easy to compress (the instructions would be “give me this many0
characters”; classic run-length encoding) but the more complex the data, the more difficult it is to compress. There are a few theoretical limits involved (see Shannon limit, Kolmogorov complexity) but often quite a lot can be done with no information loss.Lossy compression schemes are a bit different, in that they work to a theory that some information can be lost because nobody would have noticed it when viewing the image (/video/audio/etc.) The algorithms concerned are based on theories of how people actually perceive media, and the best of them work really quite well most of the time. They lose true fidelity, but get much better compression. Whether this matters depends on what you're doing.
Finally, compressed data doesn't really compress any further. Even poor compression schemes tend to leave the actual byte sequence in a state where there's very little that can be done further to it (other than uncompressing it), or at least not unless further structured space-wasteage is added in (such as base64-encoding).
-
I'll just leave this here:
I saw this yesterday. It may have subconsciously influenced my decision to interact with this thread, but it has not influenced my views on the matter except to act as some form of affirmation or justification of them.
-
@loose The mouseover text for that one amused me a lot.
-
@loose said in SSD question-- compress drive?:
In order to compress something, something has to be "lost". What is lost has to be "recreated".
With a lossless compression scheme like the one NTFS uses (or like zip, gzip, 7zip, rar or bz2) the only thing that's lost is redundancy. When decompressing during read, all of the original data bits are restored. Repeated recompression and decompression causes no information loss.
Lossless compression is not like jpeg or mp3 or photocopies of photocopies of screenshots.
The chance of something going wrong during the compression or decompression steps is markedly lower than the chance of something going wrong with the disk hardware.
The only reliability advantage that uncompressed files have over NTFS-compressed ones is that running NTFS-unaware data recovery tools against a crashed NTFS disk will most likely not recover much of the compressed stuff, due to well-known file formats looking different when compressed. But if you're actually expecting to rely on that kind of data recovery tool, your backup strategy is bad enough that worrying about the reliability of NTFS compression amounts to rearranging deckchairs on the Titanic.
-
@dkf said in SSD question-- compress drive?:
The mouseover text for that one amused me a lot.
The text or the intentional encoding fail?
-
@dkf said in SSD question-- compress drive?:
The algorithms concerned are based on theories of how people actually perceive media, and the best of them work really quite well most of the time. They lose true fidelity, but get much better compression. Whether this matters depends on what you're doing.
Quite so.
-
@asdf why not both?
-
@loose said in SSD question-- compress drive?:
In order to compress something, something has to be "lost".
But it's not actually lost. Redundancy is eliminated. "aaaaaaaaaaaaaaaaaaaaaaa" -> "23a".
ETA:
-
What is "lost" and subsequently recreated is, to use the example provide by @FrostCat, the existence of a consecutive series of 23 discrete entities with an arbitrary value of "a". You have altered the original data, you rely on the integrity of the process involved in the transformation and subsequent storage of that data not being subject to corruption. Admittedly the chances of "23a" being corrupted are less than that of "aaaaaaaaaaaaaaaaaaaaaaa". However, the chances of a single entity being corrupted are and always will be the same. In simplistic terms, the difference is that the effect of the corruption "23a" will be greater than that of "aaaaaaaaaaaaaaaaaaaaaaa"
-
@asdf said in SSD question-- compress drive?:
intentional encoding fail
Since it was intentional, it wasn't an encoding fail.
-
@loose said in SSD question-- compress drive?:
In simplistic terms, the difference is that the effect of the corruption "23a" will be greater than that of "aaaaaaaaaaaaaaaaaaaaaaa"
Perhaps, but if the likelihood of corruption is low enough, who cares?! Indeed, the kinds of corruption most likely would hit both equally, and the real big problem with long-term storage is keeping the metadata so that you can find the data again and do something useful with it.
-
@loose NTFS compression mitigates against that effect to some extent by compressing each 4KiB cluster independently. It's not like a Zip file where one bad byte somewhere in the guts of it can ruin everything that follows.
And again, if you're worried about the effects of corruption on any given copy of your data, your backup strategy is inadequate. Digital data, compressed or otherwise, doesn't really exist until you can put your hands on at least two separate storage devices containing identical copies.
-
@flabdablet said in SSD question-- compress drive?:
It's not like a Zip file where one bad byte somewhere in the guts of it can ruin everything that follows.
A zip file compresses per member, so it's only that one member that's screwed. Compressed tarballs, on the other hand...
-
@loose said in SSD question-- compress drive?:
System performance: Drive / file compression has been around longer than systems have had the performance capability to make it un-noticeable. So I guess it has become habit not to.
That changed like literally a decade ago.
@loose said in SSD question-- compress drive?:
Loss: Every time the file is re-compressed you re-introduce the chance that it may get corrupted. Witness JPEG.
That might make sense if NTFS used lossy compression; it does not.
@loose said in SSD question-- compress drive?:
Longevity: I have digital files going back 20+ years, who is to say (or guarantee) that a file compressed using the tech available today will still be accessible in 20 years times. It's bad enough that I have to cope with the current version of Word being unable to natively read files saved by the previous version of it (an over dramatization but still true).
Compression is part of NTFS, so anything that supports NTFS ought to support compression to. That said, I'm sure a lot of shitty open source products claim to support NTFS but fucked it up and choke on compressed files, because open source developers are idiots.
@loose said in SSD question-- compress drive?:
So. If I need more disk space I either archive off with redundant copies or I get a bigger disk.
SSDs are expensive.
I get the distinct sense, reading the stuff he's defecated in my thread, that loose has absolutely no idea what he's talking about.
-
@blakeyrat Yeah, and fuck you too sweetie :)
I just stated why I don't use disk compression. I'm not telling you not to, I didn't start the shit and crap about an interpretation of "loss".
In hindsight, I should have noticed that it was your thread and not engaged. Not because it's yours but because your threads tend to be baited.
I shouldn't be engaging with you now, but I'm have a bad day.
-
@PleegWat said in SSD question-- compress drive?:
Compressed tarballs, on the other hand
Depends how they're made. Gzip has an --rsyncable option that makes corruption propagate less far than it otherwise would.
But really, it shouldn't matter because there should always be a clean copy available from elsewhere.
-
@blakeyrat said in SSD question-- compress drive?:
I'm sure a lot of shitty open source products claim to support NTFS but fucked it up and choke on compressed files
That's just your prejudice talking. The ntfs-3g suite, which is what every open source project that has anything to do with NTFS almost certainly relies on, has had read-only support for compression for at least ten years, and full read/write/append support (including sparse compressed files) for six.
-
@flabdablet I'm not the one in this thread who brought up "don't use compression if you're not using windows derp derp derp derp derp!!!"
Yell at the person who did. Who I don't remember who it is.
-
Compression won't save you as much diskspace as just running defrag every night at 3:00am.
-
@Lorne-Kates Popping open the drive casing and giving the flash chips a bit of a shine up with Windex is worth considering as well.
-
@flabdablet Too bad there aren't spinning heads on the SSD. You can wax those to get some really good friction-reduction.
-
@Lorne-Kates If you put a slice of ham into a DVD player, it will show a short film about pigs.