DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one
-
Article rightly points out several ways in which Linux sucks.
-
@blakeyrat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Article rightly points out several ways in which Linux sucks.
From the article
Finally, it never ceases to amaze me that Linux has yet to take hold on the desktop market. Given the disaster that has been Windows over the last few years, and that Apple hardware is often out of the price range of the average user, Linux should be primed to take over the market.
You're right
-
@blakeyrat Okay, I get it that Dropbox is dealing with files in a very specific way, so my immediate response of "why do you care about FS at all?" doesn't apply to it quite as strongly as to other apps. Still, you're way to close to the system level if you care that the filesystem is encrypted.
But really, "multiple init systems"? What the actual fuck?
I repeat: You shouldn't care about that at all!The only thing I see as relevant is "multiple distributions", as they have different package managers and different versions of system libraries to link to.
-
I remember seeing something about this somewhere around here recently. Anywho...
@blakeyrat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Article rightly points out several ways in which Linux sucks.
Such as?
Also, this seems weird:
Let's consider Adobe (and Photoshop). If Adobe wanted to port their industry-leading product to Linux, how do they do that? Do they spend the time developing support for ext4, btrfs, Ubuntu, Fedora, GNOME, Mate, KDE, systemd? You see how that might look from the eyes of any given company?
Why would Adobe have to worry about file systems? Why would Dropbox? The only possible explanation from TFA is this ass extraction:
However, I think that "unencrypted" part cannot be ignored. Chances are this is a top-down decision that has to do with the powers that be way above most of our pay grades (think NSA and other security-minded organizations). And, before you make any assumption on that front, I am not in the know—that is purely speculation on my part.
-
@topspin said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
The only thing I see as relevant is "multiple distributions", as they have different package managers and different versions of system libraries to link to.
And some companies managed that nicely, like Chrome is available to any Debian-based distro with a simple entry added to your APT source list, or an entry added to Yum for a RedHat-based distro
-
The article is rather light on details. As others have mentioned, why does it care about FS stuff?
On Windows, does it need to understand NTFS, FAT32, and exFAT/FAT64? Does it need to understand NTFS encryption?
-
@boomzilla said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Also, this seems weird:
Let's consider Adobe (and Photoshop). If Adobe wanted to port their industry-leading product to Linux, how do they do that? Do they spend the time developing support for ext4, btrfs, Ubuntu, Fedora, GNOME, Mate, KDE, systemd? You see how that might look from the eyes of any given company?
I'm on KDE and can run any Gnome-based applications. You can also do the opposite.
Also, most apps don't care what display manager you're running, and they shouldn't.
And why the fuck should your init system be of concern to an app
-
@TimeBandit yeah, although if you want to run Gnome stuff you presumably at least have to have it installed. Which if you're running a particular file system, you must have the file system stuff installed. Which should be handled transparently from the perspective of an application, I'd think. Not that I've done a lot of low level filesystem stuff.
-
Seems strange, considering that even for an application that deals with files, at most it needs to what, diff the file on disk against the one on the server? Is it trying to look directly at the representaron of the file on the disk? I can't imagine any performance gain that gives compared to syncing over the net.
-
@boomzilla said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
if you want to run Gnome stuff you presumably at least have to have it installed
Well, you don't need Gnome installed, you just need a bunch of Gnome libraries
-
@mikehurley said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
The article is rather light on details. As others have mentioned, why does it care about FS stuff?
From skimming the Dropbox support thread, it appears to use filesystem features to keep track of extra information about the files.
@mikehurley said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
On Windows, does it need to understand NTFS, FAT32, and exFAT/FAT64? Does it need to understand NTFS encryption?
The support page on Dropbox's website says it only supports NTFS. (This may be a similarly recent change, I don't know.)
-
@TimeBandit said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
@boomzilla said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
if you want to run Gnome stuff you presumably at least have to have it installed
Well, you don't need Gnome installed, you just need a bunch of Gnome libraries
I fail to see the difference. Note, I didn't say that you needed to be running it as your desktop environment.
Filed Under: No doubt blakey loves this subthread
-
@Kian said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Seems strange, considering that even for an application that deals with files, at most it needs to what, diff the file on disk against the one on the server? Is it trying to look directly at the representaron of the file on the disk? I can't imagine any performance gain that gives compared to syncing over the net.
I could see them having a case if they had a feature like OneDrive has where you only have a representation of the file unless you explicity set the file (or folder it's in) to "Always Download".
But DropBox? It's only a glorified rsync.
-
@Parody said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
@mikehurley said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
The article is rather light on details. As others have mentioned, why does it care about FS stuff?
From skimming the Dropbox support thread, it appears to use filesystem features to keep track of extra information about the files.
@mikehurley said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
On Windows, does it need to understand NTFS, FAT32, and exFAT/FAT64? Does it need to understand NTFS encryption?
The support page on Dropbox's website says it only supports NTFS. (This may be a similarly recent change, I don't know.)
Interesting. I can't think of any FS specific metadata/features that would be needed for basic functionality. I'm surprised they just don't support all of these now-unsupported configurations with a simple, dumb, basic sync mode. Seems odd to not support it at all.
-
@Rhywden said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
@Kian said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Seems strange, considering that even for an application that deals with files, at most it needs to what, diff the file on disk against the one on the server? Is it trying to look directly at the representaron of the file on the disk? I can't imagine any performance gain that gives compared to syncing over the net.
I could see them having a case if they had a feature like OneDrive has where you only have a representation of the file unless you explicity set the file (or folder it's in) to "Always Download".
In that case they still wouldn't rely on a filesystem. They'd be a filesystem.
-
@TimeBandit said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
And why the fuck should your init system be of concern to an app
AFAIR, dropbox starts some stuff automatically at start-up, so it probably uses a service for that, hence the init system... Although I think it's more like when logging in, or when starting the desktop, so this might not actually be a service. But maybe they've got to start a system-wide service so that the rest can work afterwards.
Also, as much as Dropbox decision does initially seem a
to me, if they do indeed have some stuff that depends on the actual file system type, I can understand their decision to not spend resources on anything but one (hopefully the most used by their users, but if they really do FS-specific stuff, they probably have that stat available to them!).I did get an email from them about this a few days ago and it specifically pin-pointed one of my machines (that apparently still uses ext3... I guess that never changed since the last time I reinstalled it, years ago...), so they do know what FS I'm using, so they probably know how many users are really targeted. I don't know if they've talked about it, but maybe if they said "oh hey this change is actually only affecting 1% of our users" this would put matters in perspective?
-
@topspin said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
But really, "multiple init systems"? What the actual fuck?
I repeat: You shouldn't care about that at all!You should if you install a Service, which DropBox likely does. (It does on Windows.)
-
@Kian said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Seems strange, considering that even for an application that deals with files, at most it needs to what, diff the file on disk against the one on the server? Is it trying to look directly at the representaron of the file on the disk? I can't imagine any performance gain that gives compared to syncing over the net.
It needs to have non-polling updates on filesystem activity, I don't know if all Linux filesystems provide that equally well.
-
If Dropbox are relying on filesystem specifics for basic functionality they’re very much
as I cannot see any need for anything like that. Why can’t they just handle the files with the normal system calls and have the OS handle whatever filesystem the files happens to reside on?Another thing I cannot understand is this whole mentality of “there’s so much effort porting applications to Linux because you have support every permutation of software combination”. No, you don’t. Just port it to the major dists to cover most of the userbase. Like, just limit it to Debian/Ubuntu and RHEL/Fedora and you got the majority right there. The effort is only as large as you make it. If Theodore Thuckerwut gets upset over Adobe not supporting his special snowflake dist running SysV init, a 2.6 kernel and EFL desktop that’s his problem. But the people running those esoteric dists also tend to be the most fanatic FOSS supporters and wouldn’t want commercial applications anyway.
-
@Atazhaia said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
If Dropbox are relying on filesystem specifics for basic functionality they’re very much as I cannot see any need for anything like that. Why can’t they just handle the files with the normal system calls and have the OS handle whatever filesystem the files happens to reside on?
DropBox needs to subscribe to non-polling filesystem updates. Do all filesystems support that? Do all filesystems support it equally well?
@Atazhaia said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Another thing I cannot understand is this whole mentality of “there’s so much effort porting applications to Linux because you have support every permutation of software combination”. No, you don’t.
Yes you do.
@Atazhaia said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Just port it to the major dists to cover most of the userbase. Like, just limit it to Debian/Ubuntu and RHEL/Fedora and you got the majority right there.
That's still almost twice the effort of making a Windows build, and with worse less-productive tools. (What's the Linux equivalent of a Windows checked build, for just one example?)
@Atazhaia said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
If Theodore Thuckerwut gets upset over Adobe not supporting his special snowflake dist running SysV init, a 2.6 kernel and EFL desktop that’s his problem. But the people running those esoteric dists also tend to be the most fanatic FOSS supporters and wouldn’t want commercial applications anyway.
The problem is he can do all that and still have to report itself as "Debian".
-
@blakeyrat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
@Kian said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Seems strange, considering that even for an application that deals with files, at most it needs to what, diff the file on disk against the one on the server? Is it trying to look directly at the representaron of the file on the disk? I can't imagine any performance gain that gives compared to syncing over the net.
It needs to have non-polling updates on filesystem activity, I don't know if all Linux filesystems provide that equally well.
AFAIK, there's no filesystem specific APIs in userspace. And them doing things in kernel space seems unlikely somehow.
Filesystem notification via inotify is supported for most non-networked file systems, but has the downside that you can only monitor files and directories, not entire directory trees or file systems. So you'd have to track every directory in the tree separately, and for this purpose you would definitely want to watch an entire tree.
Inotify may well just work via the kernel tracking open files, and not require filesystem support at all.
-
@PleegWat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
AFAIK, there's no filesystem specific APIs in userspace. And them doing things in kernel space seems unlikely somehow.
Ok well DropBox needs them, whoever provides them.
In Windows, it's just a simple OS feature.
@PleegWat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Filesystem notification via inotify is supported for most non-networked file systems, but has the downside that you can only monitor files and directories, not entire directory trees or file systems. So you'd have to track every directory in the tree separately, and for this purpose you would definitely want to watch an entire tree.
So there's a shitty not-as-good version of the thing Windows has. Great.
-
@blakeyrat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
@PleegWat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
AFAIK, there's no filesystem specific APIs in userspace. And them doing things in kernel space seems unlikely somehow.
Ok well DropBox needs them, whoever provides them.
In Windows, it's just a simple OS feature.
@PleegWat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Filesystem notification via inotify is supported for most non-networked file systems, but has the downside that you can only monitor files and directories, not entire directory trees or file systems. So you'd have to track every directory in the tree separately, and for this purpose you would definitely want to watch an entire tree.
So there's a shitty not-as-good version of the thing Windows has. Great.
Looking at the inotify manpage (The inotify name made me think maybe @PleegWat accidentally mentioned a Windows API), looks like some degree of child events are received. So you'd initially create events on all existing objects and add/remove listeners as children change. Kind of annoying but not too hard.
-
@blakeyrat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
@topspin said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
But really, "multiple init systems"? What the actual fuck?
I repeat: You shouldn't care about that at all!You should if you install a Service, which DropBox likely does. (It does on Windows.)
Hmm....I found this:
...which seems to put it's configuration in
~/.config/autostart/dropbox.desktopI have no idea how that is accomplished. I don't seem to have kept the .deb file from installation.
-
@boomzilla said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Hmm....I found this:
...which seems to put it's configuration in
~/.config/autostart/dropbox.desktopI have no idea how that is accomplished. I don't seem to have kept the .deb file from installation.
Which confirms what I was thinking: Dropbox doesn't use a service on Linux, just an auto-started program.
-
@TimeBandit Well ok but I still answered the question asked. ("Why would an app have to know about the init system?")
-
@blakeyrat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
@Atazhaia said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Another thing I cannot understand is this whole mentality of “there’s so much effort porting applications to Linux because you have support every permutation of software combination”. No, you don’t.
Yes you do.
Why? Give me a single good reason to achieve compatibility with every single actively supported dist, even the ones with very few users.
@Atazhaia said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Just port it to the major dists to cover most of the userbase. Like, just limit it to Debian/Ubuntu and RHEL/Fedora and you got the majority right there.
That's still almost twice the effort of making a Windows build, and with worse less-productive tools. (What's the Linux equivalent of a Windows checked build, for just one example?)
Could be as simple as just needing to supply configuration files for two different package managers. Also, why would I need a checked build of the OS for normal application development?
@Atazhaia said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
If Theodore Thuckerwut gets upset over Adobe not supporting his special snowflake dist running SysV init, a 2.6 kernel and EFL desktop that’s his problem. But the people running those esoteric dists also tend to be the most fanatic FOSS supporters and wouldn’t want commercial applications anyway.
The problem is he can do all that and still have to report itself as "Debian".
So I can throw together an OS and call it "Windows" and then require everyone who releases software for Windows to also include my Windows?
-
@Atazhaia said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Also, why would I need a checked build of the OS for normal application development?
... why wouldn't they?
But the point I was making is: Linux has nothing equivalent to that. Linux tooling sucks.
@Atazhaia said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
So I can throw together an OS and call it "Windows" and then require everyone who releases software for Windows to also include my Windows?
If you ran my app on your hacked-together "Windows" and it crashed, you'd think my app was shit. When in actuality it was your hacked-together shit that was shit.
-
@blakeyrat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
@TimeBandit Well ok but I still answered the question asked. ("Why would an app have to know about the init system?")
No.
A service on the other hand...
-
@TimeBandit Many apps require services. If your backup app had no service for example, it'd just fail to do anything if it ran without a user logged in. It'd be a really shitty backup app.
-
The solution seems to be straight-forward: trick the dropbox client into thinking its operating on a unencrypted ext4 (while it actually is not) and because the filesystems should be transparent and work the same, it will continue working as usual.
-
As somebody pointed out, I too think that file system notifications are handled at a higher level than the actual filesystem.
The limitation to NTFS on Windows and ext4 on linux sounds much more like they want to use the alternate data streams (NTFS)/extended attributes on ext4 for storing metadata. The latter exist in some form or another in a bunch of different filesystems, but I don't know if there's a common interface or anything.
-
@cvi Could they not use another solution to accomplish the same thing? Like storing said metadata in a separate file or something?
-
@Atazhaia Sure. I guess that's what you'd normally do.
From what I know, I think the ADS/ext attr have some nice properties, like, you're attaching meta data directly to the file. There are a bunch of problems with them too. For example- what happens if you copy the file to a filesystem that doesn't support them, or pass the file through some application that doesn't know about those (which I think is most of them)?
And, just to clarify, dropbox using them is just speculation. I don't know if that's the reason - I don't have the dropbox client installed or anything. If somebody does, it should be easy to verify. Just check if the files managed by the dropbox client have non-standard ADSs or extended attributes on them.
-
@cvi said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Just check if the files managed by the dropbox client have non-standard ADSs or extended attributes on them.
How would I check this on ext4?
-
@Adynathos said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
The solution seems to be straight-forward: trick the dropbox client into thinking its operating on a unencrypted ext4 (while it actually is not) and because the filesystems should be transparent and work the same, it will continue working as usual.
how about creating a large file inside your encrypted file system, formating it as ext4 and mounting it?
-
@boomzilla said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
How would I check this on ext4?
getfattr -d -m - $somefilewhere
-dis for dump and-m -matches all possible names (according to the man-page). There's also '-R' to recurse through a directory tree. I actually found that something setuser.xdg.origin.urlanduser.xdg.referrer.urlon a couple of my files (I don't run ext4 either, so either this tool also understands btrfs extended attrs, or there is a semi-standard way of dealing with them).
-
@blakeyrat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
If you ran my app on your hacked-together "Windows" and it crashed, you'd think my app was shit. When in actuality it was your hacked-together shit that was shit.
And how's that any different from Theodore Thuckerwut's hacked-together shit being shit?
-
@cvi said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
I don't run ext4 either, so either this tool also understands btrfs extended attrs, or there is a semi-standard way of dealing with them
attr(5) under filesystem differences mentions ext2, ext3, ext4, XFS, reiserfs, and JFS.
-
@PleegWat Nice. Mine actually also mentions btrfs in addition to the ones you listed.
Actually makes me wonder about my speculation above. Seems like the extended attributes are more readily available that what I thought before.
-
@cvi I'm not sure either what it could be. Particularly banning encrypted filesystems is weird - in most cases linux does not use encrypted filesystems. It uses encrypted partitions, which can then contain any filesystem you like. This may work differently with certain modern filesystems which also have snapshotting and multi-disk functionality though.
-
@cvi said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
@boomzilla said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
How would I check this on ext4?
getfattr -d -m - $somefilewhere
-dis for dump and-m -matches all possible names (according to the man-page). There's also '-R' to recurse through a directory tree. I actually found that something setuser.xdg.origin.urlanduser.xdg.referrer.urlon a couple of my files (I don't run ext4 either, so either this tool also understands btrfs extended attrs, or there is a semi-standard way of dealing with them).I get
user.com.dropbox.atrtributes=a-bunch-of-random-looking-stuff
-
@PleegWat Yeah. I'm guessing that an ext4 on a encrypted partition would still work, because they really shouldn't be able to see any difference.
I can sort-of understand why they wouldn't want to support stuff like encfs (basically a userspace overlay that encrypts files individually; the encrypted files are places in a folder, and you can tell encfs to provide a decrypted view in a different folder). Putting the encrypted files onto dropbox would probably put a serious dent into their deduplication and compression capabilities.
I wonder if/how they deal with sparse files.
-
@boomzilla So maybe that's a factor after all? Hmm.
-
@cvi I think Linux users are fat enough.
-
@blakeyrat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
@cvi I think Linux users are fat enough.
-
@blakeyrat But has there ever been an edgi
er and hipsterier way of getting fat? It's got all the cool branding and stuff.
-
@blakeyrat said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
DropBox needs to subscribe to non-polling filesystem updates.
"Needs to" or "wants to, because that's how they decided to do it and now they're stuck with their shitty design"?
Caring about the filesystem that deeply really smells like a case of
-
@El_Heffe Well, you could go through the entire directory every half-second and you'd get about the same result. But I have it on good authority that Dropbox wants to be even slightly efficient and battery conscious.
-
@El_Heffe said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
"Needs to" or "wants to, because that's how they decided to do it and now they're stuck with their shitty design"?
Need to, assuming they don't want to cut the device's battery life by like 50% and also constantly spin-up hard drives for no reason.
@El_Heffe said in DropBox dropping (GET IT? GET IT? GET IT?) all Linux filesystems and file encryption except one:
Caring about the filesystem that deeply really smells like a case of
If you think polling is ok for an app like DropBox, you really need to up your standards.
Open source | TechRepublic
