Question about console writes and redirection



  • Simple question hopefully. Take a Linux library that is extremely verbose and constantly spits out tons of stuff to stdout and stderr. Would the application be faster if I had it redirect all output to /dev/null instead of the screen? This is happening on a headless Raspberry Pi.

    I could go through the effort of benchmarking but figured someone here might already know. The module in question is raspicam for Nodejs. We're talking hundreds of lines of console text per second. :wtf:

    [poll]

    • Yes
    • No
    • Unsure
      [/poll]

  • Notification Spam Recipient

    Honestly, it depends.

    See some stuff:

    I would recommend trying to figure out ways to reduce the program's output, as cutting output to /dev/null is only half the problem (remember, it's generating that output still, just not getting sent anywhere).



  • @Tsaukpaetra said:

    I would recommend trying to figure out ways to reduce the program's output, as cutting output to /dev/null is only half the problem (remember, it's generating that output still, just not getting sent anywhere).

    I don't feel like editing other people's libraries though. It's probably a fairly simple change, or if I really wanted to I could probably replace console.log with a no-op.


  • FoxDev

    @mott555 said:

    I don't feel like editing other people's libraries though.

    Is the verbosity not configurable?


  • Notification Spam Recipient

    @RaceProUK said:

    Is the verbosity not configurable?

    Depends on how he's using it I guess. Apparently the command line has a -v but not a --quiet.



  • @RaceProUK said:

    Is the verbosity not configurable?

    Not that I can find. https://www.npmjs.com/package/raspicam



  • The answer is yes, but unless it's outputting in a tight loop thousands of lines you probably wouldn't notice the difference. If it's dumping tons for long periods of times, it can absolutely improve noticeably.


  • Notification Spam Recipient

    @mott555 said:

    I can find.

    Flags
    v, verbose : Output verbose information during run Link
    This might be true by default for some reason?



  • @Tsaukpaetra said:

    Flagsv, verbose : Output verbose information during run LinkThis might be true by default for some reason?

    Must be true by default for some insane reason, because I'm not setting it at all.

    It's hundreds of lines of "raspicam::watcher::event change" spewing on the console each second, and an occasional "stderr: mmal: Skipping frame {num1} to restart at frame {num2}". I'm taking fake video (periodic JPEG images) at 10 fps.



  • Adding "verbose": false to the configuration parameter made it more verbose. :wtf:


  • Notification Spam Recipient

    @mott555 said:

    Adding "verbose": false to the configuration parameter made it more verbose. :wtf:

    I am truly sorry for your loss...

    Maybe you should set it to zero instead of false?

    Edit: Pull Requests accepted? :trollface:



  • Hmm. The thing about outputting to /dev/null is it still has to store the data on a buffer, do a system call and switch to kernel code, then the kernel code can finally call the /dev/null driver and pass it the pointer to the data which it immediately discards.

    It's been a while since I read on Linux streams, but if we were talking about an actual (compiled) library, you could maybe redirect the stream to the same process, and let your custom handler ignore the data, saving some time.

    But in a javascript library, this almost certainly doesn't apply.

    I'd still redirect it just in case.


  • Notification Spam Recipient

    @Tsaukpaetra said:

    Pull Requests accepted?

    Um... :wtf:

    Apparently if it's true it SKIPS adding the argument?!

    It might be a good idea to call the command manually, just to see if the program is arbitrarily spewing things for no reason.

    And it looks like someone already logged an issue:



  • From two years ago. :facepalm: And those unhandled pull requests deal with at least one other bug I've encountered.

    Maybe I should write my own library.


  • Notification Spam Recipient

    Fun thing I saw in the pull requests:

    This seems to have a "silent" option ready to go!



  • Pure conjecture on my part but maybe the log messages are generated by motion detection code. The messages sound like that no? Maybe the threshold is too low. I would assume that motion detection takes more CPU on its own than the log messages it generates. So it might make sense to disable that. Or just devnull the fucker and don't look back.

    Of course the real WTF is the parameter convention where -v increases verbosity! It should be +v!



  • Oh I see the discussion has moved on. Maybe I should read before post?!



  • No motion detection, it's taking pictures all the time.

    I'm not really having any trouble, but the "skipping frame messages" seem to indicate it occasionally gets backlogged and skips generating a frame. I was wondering if the console spam was enough to cause this, since Nodejs is single-threaded and I'm running on a battery-operated ARM processor. The issue isn't really noticeable.

    What is a problem though is all the console spam when I'm trying to debug something. But the camera code is fairly stable, most of the code is actually running on a camera-less server and a small website.



  • @mott555 said:

    What is a problem though is all the console spam when I'm trying to debug something.

    Ah it pisses its debug output right into tty1? Manual workaround is ALT+F2 :-)



  • To the best of my recollection, output to the / a / any TTY / STDOUT / STDERR (which is what a console on a machine is) is a slow, archaic and painful process irrespective of processing power, be it 1 character or several billion. Thank god for control-s and control-q.



  • That depends a lot of how much is being spit into stderr and stdout.

    I think it's a lot slower on a graphical terminal emulator, but I never checked it out. Did anyone test it?



  • @fbmac said:

    That depends a lot of how much is being spit into stderr and stdout.

    I think it's a lot slower on a graphical terminal emulator, but I never checked it out. Did anyone test it?

    I have noticed that printing text to the screen is quite a bit slower than redirecting to a file, but I haven't done any extensive testing.



  • @mott555 said:

    Take a Linux library that is extremely verbose and constantly spits out tons of stuff to stdout and stderr.

    Why is a library writing to the console?

    Sorry it's General Help, but... seriously WTF!



  • It's just a wrapper around Raspistill/Raspivid, and it's the only wrapper out there. And the author dropped off the face of the planet in 2014 or is purposefully ignoring bug fixes. I think I'm doing to ditch it and write my own wrapper, it can't be more than a couple dozen lines once I strip it down to the parts I actually need.

    The Raspberry Pi is a pretty cool device, but the community is full of terrible programmers.



  • You mean programmers for a $25 machine don't yield quality shit for Linux?

    I'm shocked.



  • Read this:

    @mott555 said:

    The Raspberry Pi is a pretty cool device, but the community is full of terrible programmers.

    And immediately thought of this guy:
    http://wiki.lspace.org/mediawiki/Newton_Pulsifer (:disco: 🐎 really tried, but failed to OneBox it)

    Newton Pulsifer, AKA Newt was born in Dorking, Surrey. As a boy, he was always rather proud of his sloppily assembled model airplanes and his electronic experiments. His mother was convinced of his genius despite his consistent failings. On one memorable occasion he assembled a joke circuit which had diodes the wrong way round, transistors upside-down, and a flat battery, but when he was through it picked up radio Moscow. He'd written the magazine from which he got it a letter of complaint that was never returned.

    This brief character decryption does not do his enthusiasm justice - you really need to read the book to get the full flavour.


  • BINNED

    The good books thread is :arrows:.


Log in to reply