:headdesk: - popen() in C


  • Discourse touched me in a no-no place

    I have only myself to blame for this one, having spent 2 minutes writing it and 2 days on it figuring out :wtf: was causing the symtoms I was seeing, namely calling /usr/local/sbin/in-house-program (from the inner clause) returned data when called from the command line but was returning nothing from the code presented here:

        if( !(fp = popen("/usr/sbin/in-house-program","r")) ){ /* try /usr/sbin first */
            if( !(fp = popen("/usr/local/sbin/in-house-program","r")) ){ /* else try /usr/local/sbin */
                    /* log fact that both failed */
                    return;
            }
        }
    

    Spoilers:
    [spoiler]
    The first popen() always succeeds regardless of whether the program in quotes exists, since what popen() would fail on is calling the sh that tries to call the program in quotes.
    ­
    Thus the second, inner, clause never gets called under normal circumstances, and if it got that far it too would probably fail anyway for the same reason the first would - i.e. before it even considered what was in the quotes.
    ­
    Anyway - that was my main headdesk moment.... Others:
    WTF#2: the fact that the program could be in one of two places to begin with which I was trying to mitigate
    WTF#3: output from the program succeeding but having nothing to report, is the same as the program not running at all.
    [/spoiler]

    :headdesk:


  • ♿ (Parody)

    @PJH said:

    if( !(fp = popen("/usr/sbin/in-house-program","r")) ){ // try /usr/bin first

    I found TRWTF: /usr/sbin != /usr/bin!


  • Discourse touched me in a no-no place

    Bugger. Got fscked in translation - the comment was wrong.

    OP Fixed.


  • ♿ (Parody)

    @PJH said:

    the comment was wrong.

    I figured, but wanted to prove that I was paying attention.


  • kills Dumbledore

    That's a horrible bracing/commenting style to my eyes. Not helped by the parser not picking the comments out as such


  • ♿ (Parody)

    @Jaloopa said:

    That's a horrible bracing/commenting style to my eyes.

    Agreed. Most compilers will throw a fit about unmatched braces. 🚎


  • Discourse touched me in a no-no place

    @boomzilla said:

    unmatched braces🚎

    Any more distractions, before the idiocy intended to be highlighted in the code shines through?

    :rolleyes:


  • FoxDev

    @PJH said:

    Any more distractions

    i hadn't planned on any, but i'm sure i could come up with some.


  • ♿ (Parody)

    Your spoilered link doesn't work with middle click?

    Presumably, sbin isn't on the expected user's path. Is there a reason it's in there instead of plain old bin and then just let the $PATH take care of stuff?

    I guess you changed to stat the file or something instead?



  • @boomzilla said:

    Your spoilered link doesn't work with middle click?

    No Repro.


  • Discourse touched me in a no-no place

    @boomzilla said:

    Is there a reason it's in there instead of plain old bin and then just let the $PATH take care of stuff?

    Raisins. And legacy stuff I don't want to get distracted by, the least of which is that $PATH might be empty, since the code is running in a bare-bones chroot.

    @boomzilla said:

    I guess you changed to stat the file or something instead?

    The intended route, yes.


  • Java Dev

    Nasty. Probably not a popen() bug though, as that, like system(), works via a shell invocation that will succeed.

    You'll probably have to write your own pipe/fork/exec.


  • Discourse touched me in a no-no place

    @PleegWat said:

    You'll probably have to write your own pipe/fork/exec.

    access("...", X_OK) before popen().


  • Discourse touched me in a no-no place

    @PleegWat said:

    You'll probably have to write your own pipe/fork/exec.

    That only really becomes worthwhile once you start wanting to pass arbitrary strings through, since going to the effort of doing it by hand is easier than figuring out quoting correctly.


  • Fake News

    @PJH said:

    The first popen() always succeeds regardless of whether the program in quotes exists, since what popen() would fail on is calling the sh that tries to call the program in quotes.

    While not immediately obvious, the man page you link to does have some nugget which should have told you that some funky stuff is going on:

    Failure to execute the shell is indistinguishable from the shell's failure to execute command, or an immediate exit of the command. The only hint is an exit status of 127.

    In other words, you need to call pclose() on the pipe to see if it actually worked at all...


    Filed under: Hindsight is 20/20, yada yada



  • ... why would Unix C library implementations spawn a shell just to forward stdin/stdout/stderr from that shell to the program?

    Also related: >having to clone your process address space before being able to start a new process, and doing that by unloading your cloned process - why? (and don't bring up 'spawn'; there's no such syscall on Linux and glibc implements that using fork/exec anyway - unlike NtCreateProcess in NT and some similar call to have fork compatibility...)



  • Because there are cases where you'd want to fork() without an exec(), and there may even be cases where you'd want to exec() without a fork()?

    (NtCreateProcess() is quite annoying in that it's both operations in one shot, which is nice for the 'run a program in a subprocess' case, but makes the 'create a subprocess of myself to do some task` case all sorts of ugly.)


  • Discourse touched me in a no-no place

    @tarunik said:

    fork()

    :giggity:()


  • Java Dev

    I believe fork() uses copy-on-write pages.

    I expect system() and popen() exist because they're convenient to use. The alternative, without shell inbetween, requires one of these:

    execlp( "rm", "rm", "-rf", "foo/bar", (char *)NULL );
    

    or

    char * args[4];
    args[0] = "rm";
    args[1] = "-rf";
    args[2] = "foo/bar";
    args[3] = NULL;
    execvp( "rm", args );
    

  • Java Dev

    exec() can be useful by itself in certain circumstances.

    fork() even more so. Most specifically, a program can background itself from the terminal by calling fork(), setsid(), fork() in succession and letting the parents exit.

    In a fork/exec sequence, there can be other functions inbetween. Most importantly, this affects which file descriptors are inherited by the child process. The following does approximately the same as popen("zcat foo.gz"), without the error handling:

    int fds[2];
    
    pipe( fds );
    
    if( 0 == fork() )
    {
        dup2( fds[1], 1 );
        close( 0 );
        close( fds[0] );
        /* Close all other file descriptors */
        execvp( "zcat", "zcat", "foo.gz", (char *)NULL );
    }
    
    close( fds[1] );
    
    return fdopen( fds[0] );
    


  • @PleegWat said:

    fork() even more so. Most specifically, a program can background itself from the terminal by calling fork(), setsid(), fork() in succession and letting the parents exit.

    in NT a program can background itself by using a convenience API to talk to the CSR (i.e. FreeConsole), or by setting a tag in the executable header so the CSR does not even attempt to do so.

    @PleegWat said:

    In a fork/exec sequence, there can be other functions inbetween. Most importantly, this affects which file descriptors are inherited by the child process. The following does approximately the same as popen("zcat foo.gz"), without the error handling:

    what the fuck, do even fork/exec cases inherit by default? NT's a lot more granular with this, at least from basedll API...

    @tarunik said:

    (NtCreateProcess() is quite annoying in that it's both operations in one shot, which is nice for the 'run a program in a subprocess' case, but makes the 'create a subprocess of myself to do some task` case all sorts of ugly.)

    NtCreateProcess was internally used by the former native POSIX subsystem to have fork() compatibility, so it can do that as well - just that the Win32 subsystem (win32k especially, basesrv is somewhat reliable) doesn't like that usage at times.

    @PleegWat said:

    I believe fork() uses copy-on-write pages.

    still, you're cloning your page table and handles for no reason

    also, I think I'm being a bad imitation of the usual NT-lovers here again... just that POSIX makes no sense to me being NT-headed :)



  • @PJH said:

    WTF#3: output from the program succeeding but having nothing to report, is the same as the program not running at all.

    Almost...

    You can read from the fd until you get an EOF condition, then run pclose() on the fd to get the exit code from the process. The shell will return a nonzero exit code if the executable doesn't exist.



  • @NTAuthority said:

    still, you're cloning your page table and handles for no reason

    ...fork creates an exact copy of your process, so why is this a surprise?


  • Java Dev

    As I understand it, overall, the two APIs are different in the underlying ideas. POSIX prefers multiple simpler system calls, while NT prefers a single more complicated one, and neither are fundamentally superior.

    POSIX does also define a vfork() system call specifically to reduce overhead when the next call will be exec() anyway, but many implementations alias it to fork() because it's a big limitation for small benefit. Particularly after a vfork() it is impossible to handle a failed exec().



  • @NTAuthority said:

    NtCreateProcess was internally used by the former native POSIX subsystem to have fork() compatibility, so it can do that as well - just that the Win32 subsystem (win32k especially, basesrv is somewhat reliable) doesn't like that usage at times.

    Yeah -- why is that, if I may ask? It seems like a reasonable usecase to support...

    (Also, did MS kill SFU for good? sigh)

    @NTAuthority said:

    what the fuck, do even fork/exec cases inherit by default? NT's a lot more granular with this, at least from basedll API...

    They inherit descriptors by default, yes -- this is because things like pipes would break if fork smashed the handles on you.


  • Discourse touched me in a no-no place

    @NTAuthority said:

    what the fuck, do even fork/exec cases inherit by default?

    It's configurable on a per-FID basis with the FD_CLOEXEC flag. It's useful to be able to open and close files and pipes after the fork() in any case; that's how you construct the more complex pipelines anyway. It's also when you can do other things like changing the working directory before the exec() launches, and other more obscure inherited stuff.


  • Java Dev

    In extension of that, you could also let a privileged parent process start an unprivileged child, but pass some resources in that require privileges to obtain (EG a low-numbered TCP listening socket).



  • Also @discoursebot is not blurring the links.

    Also also WhyTF does typing '@discoursebot' and pressing tab change it to @accalia?



  • @‍gravejunker - Last Day Without A Discourse Bug: null


  • FoxDev

    @gravejunker said:

    Also also WhyTF does typing '@discoursebot' and pressing tab change it to @accalia?

    Z because the auto complete mention thing is far too zealous.

    It will complete the mention, even if you finished it yourself and are a whole paragraph away before it finished loading.



  • @accalia said:

    before it [auto complete] finished losing.

    It has always been losing.


  • FoxDev

    s/losing/loading/

    although the swypo is accurate too.


Log in to reply