"Shell Shock", the bash complement to heartbleed (AS IF)



  • Just when we got over the heartbleed fiasco, another cornerstone of the *nix world is found to be vulnerable.

    The "moneyshot" test code you can paste into your terminal:

    env X="() { :;} ; echo busted" /bin/sh -c "echo completed"
    env X="() { :;} ; echo busted" `which bash` -c "echo completed"
    

    I have no idea how this works, but I'll be damned if it doesn't.

    Not sure how dangerous this really is in the real world context. I mean, how do you realistically inject into a bash environmental variable on a server?

    It is amusing, however, that these security conscious nix people failed to figure this out for 20+ years.



  • Thanks for the heads up. I'm not sure how dangerous it is from that initial article, but I'll do a bit more reading.



  • @cartman82 said:

    I mean, how do you realistically inject into a bash environmental variable on a server?

    One examepl CGI scripts with bash as interpreter will receive variables from the webserver. Read it as BASH-injection.



  • @cartman82 said:

    Not sure how dangerous this really is in the real world context. I mean, how do you realistically inject into a bash environmental variable on a server?

    From what I understand, the problem is with (crappy?) CGIs and the likes, and systems where /bin/sh is a link to /bin/bash. Calls to system() (and apparently popen()) actually run /bin/sh before the actual command they were supposed to execute.

    Now, if you have a setup that copies stuff from the HTTP header or from the query string (or whatever) into the environment before executing the CGI/whatever, you're screwed. (And there's nothing you can do about it either, since the problem is before your program is exectued.)

    Similarly, it might be a problem if you use system()/popen() or whatever in your CGI/PHP/... without sanitizing the environment first.

    (Major distros apparently have already updated their bash packages, and there seem to be backported patches for bash releases for the last decade or so. Also, if your /bin/sh isn't /bin/bash, and you're not running bash-shell scripts anywhere in your webstuff, you might not be in trouble either way.)



  • Still, that's a lot of hoops to jump through:

    • Run CGI scripts instead of something from the 21st century
    • Alias bash to /bin/sh
    • Copy http headers into environment willy-nilly

    And even if you manage to inject something, all you get is probably an unprivileged account access.

    Doesn't seem quite the heartbleed level of fuckery.



  • @cartman82 said:

    Doesn't seem quite the heartbleed level of fuckery.

    No, not even close.


  • area_deu

    Symlinking/bin/sh to be /bin/bash is common in Linux distros, though.



  • On my debian

    lrwxrwxrwx 1 root root 4 Mar  1  2012 /bin/sh -> dash
    

    I had no idea what dash was or that I was even using it, until today.

    TIL some nerd at some point grew bored and wrote their own "lightweight" bash replacement for debian.
    The speed gain was minuscule and no one cared.
    Then the "shell shock" bursts out and all debians are instantly safe. No need to upgrade.

    Err.. thank you, bored nerd?



  • @cvi said:

    From what I understand, the problem is with (crappy?) CGIs and the likes, and systems where /bin/sh is a link to /bin/bash. Calls to system() (and apparently popen()) actually run /bin/sh before the actual command they were supposed to execute.

    That's the problem, that if you invoke system calls from e.g. Perl the shell is invoked. You could perhaps make /bin/sh point to another shell, but I'm sure you will find out then how many utilities are not shell agnostic and assume /bin/sh to have bash behaviour.



  • Well, I don't do webstuff typically, so I don't really know the extents of the problem. But:

    <?php
    echo "foo\n";
    $o = shell_exec( "date" );
    echo "$o";
    ?>
    

    Run as $ env X="() { :;} ; echo busted" php prints

    foo
    busted
    Thu Sep 25 12:07:32 CEST 2014
    

    on an unpatched system for me.

    Dunno how easy it's to control the environment variables from a HTTP request, but if that's possible, this would be a problem. (Also assuming people do stuff like shell_exec() and the likes on a regular basis - again IDK.)

    Filed under: First PHP code I've written copy-pasted in 4+ years.


  • kills Dumbledore

    Can someone explain this exploit in terms understandable to a non Linux user? From what I can gather, you can set functions as Bash environment variables, and these are then executed whenever Bash is loaded?



  • @cartman82 said:

    - Run CGI scripts instead of something from the 21st century

    What do you think happens when you call Discourse with Javascript off?

    Ruby on Rails is a framework, nothing more.



  • I am not a linux person myself, so I could be wrong, but I think the problem isn't that the functions are executed, but that code AFTER the function declaration is IMMEDIATELY executed, potentially with elevated privileges as compared to those required to write to the env variable (someone confirm?)



  • @faoileag said:

    What do you think happens when you call Discourse with Javascript off?

    Ruby on Rails is a framework, nothing more.

    Err... rails handles the request as usual? You just don't get the response through ember, but as raw html.

    Also, I'm pretty sure there's no CGI involved in the mechanism by which ruby servers pass requests along to ruby apps. They are certainly not spinning off a new process for each request. That's a crazy architecture. No one's doing that anymore.

    @algorythmics said:

    I am not a linux person myself, so I could be wrong, but I think the problem isn't that the functions are executed, but that code AFTER the function declaration is IMMEDIATELY executed, potentially with elevated privileges as compared to those required to write to the env variable (someone confirm?)

    It doesn't seem to involve elevated privileges. You are just able to pass along and force execute code through a mechanism that was intended to handle data. Pretty much exactly like XSS in browsers.



  • @jaloopa said:

    Can someone explain this exploit in terms understandable to a non Linux user? From what I can gather, you can set functions as Bash environment variables, and these are then executed whenever Bash is loaded?

    You can define functions in bash with the following syntax

    hello () { echo "Hello, world"; }
    

    You can export these functions so they are available to a child bash process by calling export -f hello. When you do this, Bash sets the environment variable hello to the value () { echo "Hello, world"; } and exports it.
    When the child Bash process is started, it scans its environment variables. It finds the environment variable hello=() { echo "Hello, world"; } which was exported by its parent. Since the value starts with (), it concatenates the variable name with its value and executes it.
    So in that case, it will execute

    hello () { echo "Hello, world"; }
    

    and the function will be correctly redefined.

    The problem is that it blindly trusts the environment variables to have the correct syntax. If it finds an environment variable hello=() { echo "Hello, world"; }; malicious_command, it will execute

    hello () { echo "Hello, world"; }; malicious_command
    

    resulting in the immediate execution of malicious_command as soon as Bash starts.

    This is a serious problem for two reasons:

    • Environment variables are inherited recursively by child processes
    • The /bin/sh shell, which is often bash, is used by most shell scripts; starting a shell script with a malicious environment variable is then sufficient to trigger the attack. /bin/sh is also used when a program wants to execute a shell command (see system(3)), so programs running shell commands are also vulnerable.

    Besides CGI scripts, there may be ways to do a privilege escalation on a vulnerable system if a privileged program ends up running shell commands, or shell scripts.



  • THANK YOU +100

    Fantastic explanation. Now I understand how this works.



  • @cartman82 said:

    The "moneyshot" test code you can paste into your terminal:

    env X="() { :;} ; echo busted" /bin/sh -c "echo completed"
    env X="() { :;} ; echo busted" which bash -c "echo completed"

    I have no idea how this works, but I'll be damned if it doesn't.

    That triggered the problem initially, but an Ubuntu update seems to have fixed it.



  • I've been doing web development in Linux (PHP, Perl, Java, Python) for 15 years and I've never, ever have used any system call directly (not sure if a 3rd party lib did it on my behalf though) in my code.

    But, seeing @cvi PHP's example and the amount of WTF PHP developers produce, I would be worried.


  • SockDev

    There are three use-cases I am aware of for system calls from PHP with levels of legitimate.

    Firstly: getting a hostname from an IP address used to be a complete ballache. Still is. Which is why host is a thing.

    Secondly: leveraging ImageMagick for image processing. Yes, there are bindings in the form of Imagick and MagickWand but not all systems have those set up when they might have ImageMagick itself available.

    Thirdly: ffmpeg. If you are, say, handling media uploads, you might want to do some processing on them and there's no bindings for ffmpeg from PHP as far as I know.



  • Sure, but any way, you have to be pretty stupid to not sanitize whatever you're receiving, specially, when doing system calls. I mean, it's SQL injection stupidity all over again.



  • @Eldelshell said:

    Sure, but any way, you have to be pretty stupid to not sanitize whatever you're receiving, specially, when doing system calls. I mean, it's SQL injection stupidity all over again.

    Most programs do not receive input from environment variables. They just ignore them. I’m all for validating input before acting on it, but worrying about non-input data seems overkill for me. Should every program ever written carefully clean up their environment on startup? Should system(3) wipe up the environment before forking? Maybe. But if your program tries to check every aspect of the system before deciding to run, it will not get any work done.


  • SockDev

    @Eldelshell said:

    Sure, but any way, you have to be pretty stupid to not sanitize whatever you're receiving, specially, when doing system calls. I mean, it's SQL injection stupidity all over again.

    Of course, you'd always be sanitising! In the three use cases in question, there is a suitable level of sanity going on before that.



  • You don't seem to be understanding the issue here. The problem is not the environment, the problem is allowing any HTTP header, cookie or URL variable to be directly executed on system()



  • Read this which basically overwrites nginx with a malwared one and there's no need for root since the process running forking to bash and nginx are the same.



  • @Eldelshell said:

    You don't seem to be understanding the issue here. The problem is not the environment, the problem is allowing any HTTP header, cookie or URL variable to be directly executed on system()

    Because that stuff gets put into the environment by the shell, which is mishandling stuff.



  • @boomzilla said:

    Because that stuff gets put into the environment by the shell, which is mishandling stuff.

    Yes, but how you get to the point of Bash executing the exploit. Try any server and send the exploit code. Nothing will happen unless you're doing some insane stuff with the request like:

    <?php
    foreach (apache_request_headers() as $header => $value) {
        shell_exec($value);
    }
    ?>


  • @Eldelshell said:

    Try any server and send the exploit code.

    Or, as has already been pointed out: a server running CGI.



  • @boomzilla said:

    Or, as has already been pointed out: a server running CGI.

    There aren't many servers using CGI these days (routers, on the other hand). And all the CGI I've seen is made with Perl or Python, not Bash. But yes, if and only if, you're using a bash CGI and running the web server as root, then yes, you're pretty much affected.



  • Here's a good analysis of the flaw.

    Here's one of the worst UIs I've seen in quite awhile:



  • http://milankragujevic.com/projects/shellshock/ <- There's a nice and simple site tester here, also. None of my sites are vulnerable, which means either DreamHost is super on-the-ball, or that I was just lucky they're configured to not use CGI. Too lazy to check which.



  • @blakeyrat said:

    Here's one of the worst UIs I've seen in quite awhile:

    Yeah, Wireshark is pretty awful in that matter. Then again, it's one of those tools you need to spend 10 years in seclusion with Buddhist monks to begin using, so one more year for the UI doesn't really matter much.



  • It's reasonable for protocol analysis, which is the only thing I use it for. The fact it crashes every 2 minutes on my work machine is more problematic.



  • I used to use NetMon 3, IIRC, when I was doing network testing for Xbox Live games, it wasn't... great... but compared to the screenshot I'm seeing illustrated up there, I'm guessing it was brilliant.

    Great tool, in any case.



  • @blakeyrat said:

    I used to use NetMon 3, ... it wasn't... great...
    ...
    Great tool, in any case.

    I'm confused. Either "Great tool, in any case" refers to something other than NetMon 3, in which case it is entirely unclear what it refers to, because it was the only tool mentioned in your post, or NetMon 3 is a "Great tool" that "wasn't... great..." Huh?



  • @PleegWat said:

    It's reasonable for protocol analysis, which is the only thing I use it for. The fact it crashes every 2 minutes on my work machine is more problematic.

    Yeah, I had to use it today for uni. Unfortunately, there were so many packets coming in so quickly that I couldn't stop the capturing, and eventually I had to kill it.

    It looks almost as though it's coded in Java...



  • I guess I had a mini-stroke in-between paragraphs there.



  • Oh, I don't use it for capture. I typically capture with tcpdump. So my typical case is I've got a small on-disk dump I want to look at, and I can just reload if it crashes. Still annoying when it crashes so fast...



  • Good explanation.

    I tested sudo scripts, of which I have lots, but they seem OK, sudo sanitizes the environment.



  • Network Monitor's killer feature IMO is that it groups traffic by processes. I spent what felt like hours fiddling in wireshark and couldn't get that to work. Run NetMon for the first time and BOOM, it's there. Never looked back.



  • It's really easy to set up filters, too. Which you need, because otherwise your traffic gets insane VERY quickly. On our test bench, the test PC had a dedicated ethernet port for the Xbox 360, so it was trivial to see only packets going to/from the Xbox. And we had further filters for voice data, game update data, Xbox OS/store data, etc.



  • Is it just CGI, or is fastCGI impacted too?


  • Discourse touched me in a no-no place

    Since it isn't using environment variables to pass values, fastCGI should be fine.



  • Doesn't that extremely reduce the attack surface of success likelyhood then? I don't think there's any current big-name web server still using cgi instead of fastcgi. (I think new installs also default to fastcgi)

    Which pretty much leaves crufty legacy applications (likely to also be using exec nonsense) and newbie lost copy pasta coders (also likely to be using exec nonsense)


  • Discourse touched me in a no-no place

    There's still quite a bit of CGI about, since it's one of the easier ways to do user privilege separation in a web server. (Most of the alternatives work better with dedicated servers, which is a different use case.)


    Huh. One of my servers is so old that the bash on it doesn't have the bug…



  • I think @VinDuv has written the most concise and correct explanation I've seen so far, which is quite a feat given how much poor/misinformation I've seen. I think most explainers assume RHEL or similar where /bin/sh is bash. But for my own understanding, I'm trying to summarise the issue here.

    1. Bash executes code it receives through environment variables crafted in a certain way, upon initialisation
    2. Environment variables are inherited by child processes (and their children, and their grandchildren...)
    • The CGI protocol passes through things such as cookies and the query string through environment variables
    • If the CGI program happens to be run by bash, you're pretty much hosed
    • If the CGI program happens to call bash (See point 5), you're pretty much hosed
      • Unless you clean the environment first. Except nobody does that.
    1. ssh logins might be dangerous, but only if the user is authorised
    • ssh scrubs the environment but lets some variables through by default.
    • Some server programs such as rsync and version-control systems use ssh to encrypt the data stream, which might allow for shell access if the ssh login calls bash
      • I have no idea whether they do that. Those accounts are supposed to be set to nologin anyway. No bash that way.
    1. This issue is made 1000% worse if /bin/sh is bash (like it is in many if not most linux distros, but not e.g. Debian any more.)
      • Not in BSDs either. Probably not in any of the commercial Unices either.
      • But OS X's sh is bash.
    2. The system() family of functions in most programming languages calls /bin/sh -c "your code here"
    • Which might call bash. See point 4.
    • But many programming languages have a version of system() that doesn't call the shell at all.
      • Such as Perl's system(@list), as opposed to system($string)
        • Try it yourself: system('/bin/echo', '$SHELL'); system('/bin/echo $SHELL');
      • But nine out of ten people don't use system() this way and probably aren't even aware of its existence. Despite it being both faster and safer.

    Am I missing any vectors?



  • Wow, @VinDuv has shot past 20 likes for that answer! Is he approaching @Lorne_Kates's goodbye speech?

    Your summary sounds correct to me. Theoretically shellshock is a big deal, but attack vectors are so specific and unlikely, it's not a matter of great urgency.

    IMO



  • This is why we all should be using formal grammars fed to parser generators whenever we want to handle input (aside from GUI events, ofc).

    Filed under: TRWTF is ad-hoc parsing code



  • Fun fact: bash got a second update on Ubuntu 14.04. What's weird about it is that they claim Ubuntu 14.04 didn't get updated by the first update... and yet I tested it after the first patch and yes, it did.

    I guess Ubuntu's maintainers are their own WTF.



  • @powerlord said:

    What's weird about it is that they claim Ubuntu 14.04 didn't get updated by the first update

    What I saw was them saying the first update was an imcomplete fix. What did you see?



  • A lot of systems had a first patch that half-sealed the exploit, but left open a slightly-different way of exploiting it. That's probably what happened to Ubuntu.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.