What is the Truck-Factor of Popular GitHub Applications?



  • http://mtov.github.io/Truck-Factor/

    The Truck Factor designates the minimal number of developers that have to be hit by a truck (or quit) before a project is incapacitated

    We calculate the Truck Factor for 133 popular GitHub applications, in six languages: JavaScript (22 systems), Python (22 systems), Ruby (33 systems) , C/C++ (18 systems), Java (21 systems), and PHP (17 systems). We also used github/linguist to discard non-source code files (e.g., documentation) and also third-party files.

    Interesting idea. Click through to see the full list.

    Some of the projects with a surprisingly low factor (that I recognize):

    • clojure/clojure
      (popular JVM language)
    • less/less.js, sass/sass
      (99% of frontend devs use these to write CSS)
    • mbostock/d3
      (basis for many graphing libraries)
    • mitsuhiko/flask
      (2nd most popular python web framework)
    • strongloop/express
      (90% of node apps use this for http api)
    • drupal/drupal
      (popular PHP CMS)

    For some of the corporate sponsored projects not mentioned here, I suspect they have a "github guy" who does all the exporting from their private repos.

    On the other hand, torvalds/linux has a comfy factor of 90. Meaning, despite the fears, there's plenty of linux goodness to come once Linus goes away.


  • I survived the hour long Uno hand

    I've always heard this called "bus factor". I think SockDrawer's is one: @accalia


  • sockdevs

    yeah, but i'm working on that!



  • Yeah, I called it bus factor too. But since they linked to wikipedia, I guess it's "truck factor" now....



  • @cartman82 said:

    But since they linked to wikipedia, I guess it's "truck factor" now....

    The Wikipedia article they linked was:

    How does that make it truck?



  • Ha! You caught me, I didn't click through. Now I'll have to burn this account and start anew under a different nickname.



  • @cartman82 said:

    start anew under a different nickname.

    @cartman83 is available



  • @Luhmann said:

    @cartman83 is available

    To confuse us, he'll probably create the account @stan82 or @kennymccormick82.



  • I have an issue with the way this number was calculated. A lot of times, it's easier if there's just one person actively working on a project. That doesn't mean there wouldn't be plenty of people ready to step in if the worst were to happen. I mean, kudos for gathering the data, but I expect that the things that actually make a project resilient are harder to measure than just the number of active contributors. I'm sure you can imagine that there might be some projects that would actually be better off if the person in charge was hit by a bus.


  • area_deu

    https://github.com/discourse/discourse?

    EDIT: Note the lack of discourse/discourse in TFA. I guess it's not so popular after all~!





  • @Buddy said:

    I have an issue with the way this number was calculated. A lot of times, it's easier if there's just one person actively working on a project. That doesn't mean there wouldn't be plenty of people ready to step in if the worst were to happen.

    Not sure about that. The bus factor on my main project is one. If I went away, I'm pretty sure my successor would need months before they are fully caught up.

    For OSS there's an additional problem that everyone wants to do greenfield on their own passion projects, no one likes to read someone else's code.



  • Ok, but what about the people who already have read the code? Did they account for those? The people who are already familiar with the project, because they've been using it for years. Those are the people who have probably only needed to submit one or two commits over the years, but if the original creator went awol they would pretty much need to keep the project alive, unless they wanted their own shit to go down the pipe after it.


  • I survived the hour long Uno hand

    I'm disappointed they didn't put the source code for their calculations on github. What's the point of the empty repo?


  • I survived the hour long Uno hand

    @Buddy said:

    The people who are already familiar with the project, because they've been using it for years

    Dig up some case studies, but I suspect that in reality, they don't step up and maintain the project. If it's truly mission-critical, they fork, and if you're lucky, they leave the fork open-source, but...

    Well, for an ironic case study, look at Phil Booth's Complexity Measures: https://github.com/philbooth/complexity-report/commits/master

    It's too soon to tell for sure, but my money's on that going away rather than being picked up by someone else.



  • @cartman82 said:

    mbostock/d3(basis for many graphing libraries)

    D3 is an unmaintainable mess of shit. With no error-checking. It surprises me not one bit that only a single moron is in charge of it, because if there were a second person he'd be going, "hey, you're writing an unmaintainable mess of shit with no error-checking, stop it you moron."



  • @blakeyrat said:

    D3 is an unmaintainable mess of shit. With no error-checking. It surprises me not one bit that only a single moron is in charge of it, because if there were a second person he'd be going, "hey, you're writing an unmaintainable mess of shit with no error-checking, stop it you moron."

    Hmm, I never got into D3, but it's on the schedule for the graphs in my main app. It seems lots of graphing libraries are based on it.

    You have an alternative?


  • I survived the hour long Uno hand

    D3 is very powerful, but the learning curve... fwoo. Not something I enjoyed working with at all.

    I wanted to use chart.js, but D3 was already on the approved libraries list, so I gave that a shot instead. It didn't help that I wasn't intimately familiar with the SVG format, which D3 is tightly intertwined with; if you know the desired output in SVG format, you're halfway there with D3, you just have to figure out how to trick it into doing what you want. If you just know what the picture should be, you're more screwed. In my case I was making a pie chart.



  • The last time I looked into chart.js, I ended up here:

    There was no way to dynamically update the charts, so that was that.

    But now...

    Update: Looks like charjs has been updated (see comment below). There are some examples up that look very nice:

    Here's an example of updating a line chart using new data: http://jsbin.com/yitep/5/edit
    Here's how we can update existing data on a line chart: http://jsbin.com/yitep/4/edit

    It's back on agenda. Won't be on the schedule for a few more months, but looks nice.



  • We're generating svg from php, and use an external rasterizer (rsvg) to generate pngs. svg isn't too bad if you can think in vectors, though I never worked too closely with that code. rsvg though, at least in the version included with redhat 5, has a number of issues particularly in text alignment. Other rasterizers have other issues - I know our svgs don't render correctly in firefox or chrome. Some elements are not rendered.



  • @cartman82 said:

    You have an alternative?

    I don't do that anymore, thank God.

    @Yamikuronue said:

    D3 is very powerful, but the learning curve... fwoo.

    It has ZERO error checking. Any error you manage to produce (that is, about 500 a day) has a stack trace like this:

    'null' is null or not an object in Anonymous Function:23
    Anonymous Function:9
    Anonymous Function:5
    Anonymous Function:3
    Anonymous Function:9
    Anonymous Function:5
    Anonymous Function:3
    Anonymous Function:9
    Anonymous Function:5
    Anonymous Function:3
    Anonymous Function:9
    Anonymous Function:5
    Anonymous Function:3
    Anonymous Function:9
    Anonymous Function:5
    Anonymous Function:3
    Anonymous Function:9
    Anonymous Function:5
    Anonymous Function:3
    Anonymous Function:9
    Anonymous Function:5
    Anonymous Function:3
    Anonymous Function:9
    Anonymous Function:5
    Anonymous Function:3
    Anonymous Function:9
    Anonymous Function:5
    Anonymous Function:3
    YourSourceFile.js:42

    How do you solve that error? WELCOME TO GUESS-AND-CHECK THE BEST KIND OF SOFTWARE DEVELOPMENT!

    It doesn't check errors, it has no logging, and it can't be debugged. IT TAKES ACTUAL WORK TO FUCK UP THAT HARD IN JAVASCRIPT!



  • @blakeyrat said:

    How do you solve that error? WELCOME TO GUESS-AND-CHECK THE BEST KIND OF SOFTWARE DEVELOPMENT!

    It doesn't check errors, it has no logging, and it can't be debugged. IT TAKES ACTUAL WORK TO FUCK UP THAT HARD IN JAVASCRIPT!

    Meh that's just javascript. My soul is already numb to such horrors.



  • @cartman82 said:

    Meh that's just javascript.

    Just JavaScript written in that horrible functional style. There's nothing wrong with normal JavaScript, it's actually a joy to debug normally. Of course you use all those open source bullshit libraries from morons, so you've probably never seen normal JavaScript before.

    Yes, JavaScript can be used to write functional code. Great. Whee. But no, it's a fucking terrible idea, stop, don't do it, STOP DOING IT PLEASE GOD!


  • I survived the hour long Uno hand

    @blakeyrat said:

    Yes, JavaScript can be used to write functional code. Great. Whee. But no, it's a fucking terrible idea, stop, don't do it, STOP DOING IT PLEASE GOD!

    A like is not enough. I want this embossed on my hand so it gets engraved on people's faces when I pimp-slap them.


  • sockdevs

    @cartman82 said:

    mbostock/d3

    i've looked at D3..... i am not surprised.


  • I survived the hour long Uno hand

    So I got nerd-sniped hardcore...



  • My current project:

    Authors:
    ========================
    cartman(2920) [80%]
    ---------Bus has killed project----------
    (564) [15%]
    Intern(79) [2%]
    Other guy(37) [1%]
    Third guy(27) [1%]
    third guy(21) [1%]
    
    
    
    Bus factor: 1
    Total files: 3648
    

    Very interesting. Nice work @Yamikuronue!


  • sockdevs

    hmm..... i should take a look at what's causing those 564 files to have no owner.....

    because they should have an owner.


  • I survived the hour long Uno hand

    Yeah. When it dogfooded its own repo, the only files showing up with no owner were License and Readme, which I let Github auto-generate, so they really did have no owner.

    On Sockbot, though, there's a bunch of doc files. Maaauybe I didn't have git configured right when they were generated and my name field was blank, but there might be a bigger issue here.



  • The readme is sadly less informative than I require to figure this out. How does one run this?


  • I survived the hour long Uno hand

    Oh, it has built-in help text, but to run it use "node ./index.js" (after doing an npm install)

    Options:
      -r, --repo  Repository to scan                             [string] [required]
    
      -t, --type  Type of repository. Accepts "git", "svn", or "auto"
                                                          [string] [default: "auto"]
    

  • sockdevs

    node index.js


  • I survived the hour long Uno hand

    No you don't, pull changes again :)


  • sockdevs

    well you had to this morning! how's a fox to keep up?


  • I survived the hour long Uno hand

    Yeah, the last thing I did before announcing it here was hook up yargs so it takes command-line arguments and prints the nice help text when required arguments are missing. I love yargs, but it's so easy to parameterize that I often leave it until last XD



  • @Yamikuronue said:

    Yeah. When it dogfooded its own repo, the only files showing up with no owner were License and Readme, which I let Github auto-generate, so they really did have no owner.

    On Sockbot, though, there's a bunch of doc files. Maaauybe I didn't have git configured right when they were generated and my name field was blank, but there might be a bigger issue here.

    Nope, I checked a few that were blank. I was the author. Nothing special about them.


  • I survived the hour long Uno hand

    Can you get me the output from a git blame on one of them? My next task is to hook up a testing apparatus so I can feed it known input :)



  • @Yamikuronue said:

    Oh, it has built-in help text, but to run it use "node ./index.js" (after doing an npm install)

    Ah, yes, figured that out....now I get:

    $ nodejs index.js -t svn -r svn://svn-server/project/trunk
    
    fs.js:647
      return binding.mkdir(pathModule._makeLong(path),
                     ^
    Error: ENOENT, no such file or directory 'tmp/wc4'
        at Object.fs.mkdirSync (fs.js:647:18)
        at /home/boomzilla/bz-project/BusFactor/lib/svn.js:13:7
        at next (/home/boomzilla/bz-project/BusFactor/node_modules/rimraf/rimraf.js:72:7)
        at CB (/home/boomzilla/bz-project/BusFactor/node_modules/rimraf/rimraf.js:108:9)
        at /home/boomzilla/bz-project/BusFactor/node_modules/rimraf/rimraf.js:134:14
        at Object.oncomplete (fs.js:107:15)
    


  • @Yamikuronue said:

    Can you get me the output from a git blame on one of them? My next task is to hook up a testing apparatus so I can feed it known input

    Slightly anonymized.

    As I said, nothing special here.

    $ git blame scripts/inspector.sh
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100  1) #!/bin/bash
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100  2) 
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100  3) TARGET=$1
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100  4) if [ -z "$TARGET" ]; then
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100  5)        TARGET="manager"
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100  6) fi
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100  7) 
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100  8) usage() {
    7a117966 scripts/inspector.sh (cartmans.name 2014-12-04 18:30:27 +0100  9)        echo "Usage: $0 -h/-k/APP_NAME"
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 10)        echo
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 11)        echo "APP_NAME can be 'manager', 'webui' etc..."
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 12)        echo "Use with -k to just kill the existing one"
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 13)        exit 0
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 14) }
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 15) 
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 16) kill_inspector() {
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 17)        print_output=$1
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 18)        local pid=`ps -o pid=,args= -w -C node | grep node-inspector | awk '{ print $1 }'`
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 19)        if [ -z "$pid" ]; then
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 20)                if [ "$print_output" = true ]; then
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 21)                        echo "No node-inspector instance was found"
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 22)                fi
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 23)        else
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 24)                echo "Killing node-inspector at $pid..."
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 25)                kill $pid
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 26)        fi
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 27) }
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 28) 
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 29) start_inspector() {
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 30)        local pid="`ps -o pid=,args= -w -C node | grep postmaster | awk '{ print $1 }'`"
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 31)        if [ -z "$pid" ]; then
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 32)                echo "Couldn't find pid for $TARGET" >&2
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 33)                exit 1
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 34)        fi
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 35) 
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 36)        echo "Sending USR1 to $TARGET at pid $pid..."
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 37)        kill -s USR1 $pid
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 38)        echo "Starting node-inspector..."
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 39)        node-inspector --web-port 46000 > /dev/null &
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 40)        if [ "$?" != "0" ]; then
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 41)                echo "Failed to start node-inspector!" >&2
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 42)                exit 1
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 43)        fi
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 44) 
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 45)        echo "Debugger link: http://1.2.3.4:46000/debug?port=5858"
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 46) }
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 47) 
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 48) case $TARGET in
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 49)        "-k" | "--kill" )
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 50)                kill_inspector true
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 51)                ;;
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 52)        "-h" | "--help" )
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 53)                usage
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 54)                ;;
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 55)        *)
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 56)                kill_inspector false
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 57)                start_inspector
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 58)                ;;
    add7a76f scripts/inspect.sh   (cartmans.name 2014-02-26 13:17:57 +0100 59) esac
    
    

  • I survived the hour long Uno hand

    Yup, I see it. Thanks. I've pushed a fix for that :)

    (You'll need to run npm install again after pulling, I added a new dependency)


  • I survived the hour long Uno hand

    Bookmarked to investigate after work :) Thanks



  • OK, next error: :smile:

    $ nodejs index.js -t svn -r svn://svn-server/project/trunk
    
    child_process.js:935
        throw errnoException(process._errno, 'spawn');
              ^
    Error: spawn EMFILE
        at errnoException (child_process.js:988:11)
        at ChildProcess.spawn (child_process.js:935:11)
        at exports.spawn (child_process.js:723:9)
        at Object.module.exports.getOwner (/home/boomzilla/bz-project/BusFactor/lib/svn.js:53:11)
        at async.each.numfiles (/home/boomzilla/bz-project/BusFactor/index.js:51:16)
        at /home/boomzilla/bz-project/BusFactor/node_modules/async/lib/async.js:159:20
        at /home/boomzilla/bz-project/BusFactor/node_modules/async/lib/async.js:227:13
        at _arrayEach (/home/boomzilla/bz-project/BusFactor/node_modules/async/lib/async.js:78:13)
        at _each (/home/boomzilla/bz-project/BusFactor/node_modules/async/lib/async.js:69:13)
        at Object.async.forEachOf.async.eachOf (/home/boomzilla/bz-project/BusFactor/node_modules/async/lib/async.js:226:9)
    

  • I survived the hour long Uno hand

    Do you have SVN installed?

    .....And/or are you on OSX?



  • I have svn 1.8.8 installed. I'm on Linux.


  • I survived the hour long Uno hand

    Huh. Google says that error means "maximum file handle limit exceeded". How big is the repo you asked it to scan? Maybe I need to put in some kind of throttle so it doesn't try to do every file at once?



  • Ah...yeah...8869 files.

    EDIT: tried a smaller repo that only had 722 files. Same error.


  • I survived the hour long Uno hand


  • I survived the hour long Uno hand

    @boomzilla said:

    EDIT: tried a smaller repo that only had 722 files. Same error.

    Damn. Back to the drawing board then.



  • Tried on some subdirectories...crapping out somewhere between 364 and 700 files. By default, my system allows 1024 open file handles. I assume some of those are being used by node.



  • So, I changed async.each(lsFiles to use async.eachSeries and it worked (though it took a while). I only looked at java code (no xhtml files):

    Authors:
    ========================
    boomzilla(735) [55%]
    ---------Bus has killed project----------
    Competent Contractor(266) [20%]
    Incompetent "Senior" Developer(112) [8%]
    Solid Dev Team Lead(103) [8%]
    System Architect, no longer on the project(85) [6%]
    Arrogant Contractor, long gone(22) [2%]
    Annoying Contractor, long gone(10) [1%]
    Previous Jr Dev, long gone(9) [1%]
    Guy who helped out with some technical issues, not really on project(1) [0%]
    
    Bus factor: 1
    Total files: 1343
    

    This surprised me not at all.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.