(Linux) Why can't I delete these files?

cartman82

A user, let's call him joe, has the following setup on a server:

apache is serving files from /var/www/site. It's running as the standard user www-data``/www-data
user joe is a member of www-data as well. He owns the content of /var/www/site.
all the files are with permissions 775/664 (so group rights are elevated)
therefore, both apache and joe should be able to have full access to all the files

I have ssh-ed into this server, did a sudo su joe and am now trying to delete some caches.

Observe:

$ ls -lad assets
drwxrwxr-x+ 2 www-data www-data 4096 Dec  7 06:53 assets

$ ls -la assets
total 728
drwxrwxr-x+ 2 www-data www-data   4096 Dec  7 06:53 .
drwxrwxr-x+ 4 joe     www-data   4096 Dec  8 17:17 ..
-rw-rw-r--+ 1 www-data www-data  14280 Dec  7 06:53 02eba54e7cdc9219510e011cbc144441
...(snip, a bunch more files)

$ id
uid=1001(joe) gid=33(www-data) groups=33(www-data),27(sudo)

$ groups
www-data sudo

Clear enough? Deep inside this hiearchy, there's a directory named assets with all this crap I need to delete. All permissions and groups seem OK.

But...

$ rm assets/02eba54e7cdc9219510e011cbc144441 -f
rm: cannot remove ‘assets/02eba54e7cdc9219510e011cbc144441’: Permission denied

???

What's going on here? Why can't I delete this file?

Note that

I already know this is a bad setup, I'm not looking for lectures
I know I can just delete everything as root, that's not the point. I want to understand what's going on.

Nocha

What are the permissions on assets? And what about it's contents?

Tsaukpaetra

@Nocha said:

permissions on assets

^^ this.

cartman82

Look at the first two ls commands.

Am I missing something?

@Nocha too.

asdf

Maybe ACL permissions?

getfacl assets/02eba54e7cdc9219510e011cbc144441

The + in the ls output indicates that ACL permissions are set, I think.

Tsaukpaetra

@cartman82 said:

Look at the first two ls commands.

I derped, not sure why.

PleegWat

@asdf said:

Maybe ACL permissions?
getfacl assets/02eba54e7cdc9219510e011cbc144441 
The + in the ls output indicates that ACL permissions are set, I think.

This. There are ACLs in effect.

cartman82

@asdf said:

The + in the ls output indicates that ACL permissions are set, I think.

$ getfacl assets/02eba54e7cdc9219510e011cbc144441
# file: assets/02eba54e7cdc9219510e011cbc144441
# owner: www-data
# group: www-data
user::rw-
user:root:rwx			#effective:rw-
user:www-data:rwx		#effective:rw-
group::r-x			#effective:r--
mask::rw-
other::r--

ACL permissions? First time I hear about those.

Googling...

skotl

Jeez. I used UNIX since 1986 and Linux for a loooong time and I had no idea that this existed.
Good spot - I saw the "+" but didn't register it. First rule of computers; if something seems off then there's a reason for it, so go figure out what that reason is!

asdf

Basically, in addition to the usual permissions, most Linux filesystems support ACLs as well.

Nocha

@Tsaukpaetra said:

I derped, not sure why.

Me too. I was thinking about needing write access to a directory to remove files within. But ACLs appears to be the answer.

loopback0

@skotl said:

I saw the "+" but didn't register it.

Same here.

cartman82

Good job, @asdf

That was exactly it.

I read a bit into this ACL thing, my conclusions so far:

Seems powerful, perhaps kind of like NTFS permissions
Poorly explained, no obvious easy tutorial in the first few links
I don't need it
I'm too tired to deal with this

Thus, my solution:

# nano /etc/fstab

...
UUID=56d............4ff9b /               ext4    noacl,errors=remount-ro 0       1
...

# mount -o remount /

Thanks guys, that was fast help!

skotl

I didn't think of this as a possible cause (and it isn't in cartman's situation) but I just remembered a similar situation where we were getting "Permission denied" and it was because the volume entry in /etc/fstab had a "readonly" tag.

Took a while to find that one...

asdf

@cartman82 said:

Seems powerful, perhaps kind of like NTFS permissions

It's pretty much the same, yeah. The funny thing is: It's been there for many years and 50% of the people who use Linux don't know about it. Maybe 5% ever used it. Seems like user,group,world is good enough™ for most people.

@cartman82 said:

Thus, my solution:
[...]

A simple setfacl would have done the trick as well, but nuking ACLs from orbit solves the problem as well, of course. ;)

skotl

@asdf said:

A simple setfacl would have done the trick as well, but nuking ACLs from orbit solves the problem as well, of course.

WE ARE ROOT! WE HAVE THE MISSILEZ! WE WILL sorry? oh yeah - problem solved. Piece of piss, actually - it's to do with your facls...