I am TRWTF (JavaScript that never should have been written?)


  • sockdevs

    I wrote a Promised based, limited concurrency task queue for javascript....

    why? because i had no idea what i was getting myself in for.....

    function _thenable(fn, args) {
        try {
            var value = fn.apply(null, args || []),
                thenable = value;
            if (!value || typeof value.then !== 'function') {
                thenable = Promise.resolve(value);
            }
            return thenable;
        } catch (err) {
            return Promise.reject(err);
        }
    }
    
    function queue(worker, concurrency) {
        var queueTasks = [],
            workers = 0,
            q;
    
        function noop() {}
    
        function notify(fn, err, value) {
            if (typeof fn === 'function') {
                return fn(err, value);
            }
        }
    
        function taskify(tasks, progress) {
            var result = {
                tasks: [],
                promises: []
            };
            tasks.forEach(function (task) {
                var fn, promise = new Promise(function (resolve, reject) {
                    fn = function () {
                        return _thenable(worker, [task]).then(function (v) {
                            setImmediate(function () {
                                resolve(v);
                            });
                            return Promise.resolve(v);
                        }, function (e) {
                            setImmediate(function () {
                                reject(e);
                            });
                            return Promise.reject(e);
                        });
                    };
                    fn.progress = progress;
                });
                result.tasks.push(fn);
                result.promises.push(promise);
            });
            return result;
        }
    
        function next() {
            function after() {
                workers -= 1;
                if (queueTasks.length === 0 && workers === 0) {
                    notify(q.drain);
                }
                next();
            }
    
            function success(task) {
                return function (value) {
                    notify(task.progress, null, value);
                    after();
                };
            }
    
            function failure(task) {
                return function (value) {
                    notify(task.progress, value);
                    after();
                };
            }
            while (!q.paused && workers < q.concurrency && queueTasks.length > 0) {
                var task = queueTasks.shift();
                workers += 1;
                task().then(success(task), failure(task));
                if (queueTasks.length === 0) {
                    notify(q.empty);
                }
                q.started = true;
            }
        }
    
        function addTasks(adder, tasks, progress) {
            if (!Array.isArray(tasks)) {
                tasks = [tasks];
            }
            var t = taskify(tasks, progress);
            adder.apply(queueTasks, t.tasks);
            if (queueTasks.length + workers >= q.concurrency) {
                notify(q.saturated);
            }
            next();
            return Promise.all(t.promises);
        }
        concurrency = Math.floor(concurrency);
        if (Number.isNaN(concurrency) || concurrency <= 0) {
            concurrency = 1;
        }
        q = {
            concurrency: concurrency,
            started: false,
            length: function length() {
                return queueTasks.length;
            },
            running: function running() {
                return workers;
            },
            idle: function idle() {
                return queueTasks.length === 0 && workers === 0;
            },
            push: function push(tasks, progress) {
                return addTasks([].push, tasks, progress);
            },
            unshift: function unshift(tasks, progress) {
                return addTasks([].unshift, tasks, progress);
            },
            paused: false,
            saturated: noop,
            empty: noop,
            drain: noop,
            pause: function () {
                q.paused = true;
            },
            resume: function () {
                q.paused = false;
                next();
            },
            kill: function () {
                q.paused = true;
                queueTasks = [];
            }
        };
        return q;
    }
    


  • I am TRTWTF (JavaScript that should never have been written?)

    FTFY.



  • The madness that possesses people. Really I know you were bored but why would you subject us to this. We're all forced to dabble but somethings are just never meant to be more than that.

    On an unrelated matter could you recommend a javascript book that doesn't involve jcuntery and have you licensed that code?



  • @DogsB said:

    jcuntery

    :laughing:


  • Winner of the 2016 Presidential Election



  • I really wish there was a seal emoji, so we could say ":seal: of approval".





  • No, I meant something more along these lines:


  • Winner of the 2016 Presidential Election

    MOD ABUSE! I got no royalties for that!



  • Spoiler that will you ...



  • @Onyx said:

    MOD ABUSE! I got no royalties for that!

    :wambulance:


  • sockdevs

    @DogsB said:

    could you recommend a javascript book that doesn't involve jcuntery

    https://books.google.com/books/about/JavaScript.html?id=F9ybAgAAQBAJ&hl=en

    @DogsB said:

    have you licensed that code?
    MIT license, if you want a different license have your perople talk to my people.
    https://github.com/AccaliaDeElementia/aplus


  • mod

    @Onyx said:

    MOD ABUSE! I got no royalties for that!

    USER ABUSE! You can't claim mod abuse from a non-mod!


  • sockdevs

    @abarker said:

    @Onyx said:
    MOD ABUSE! I got no royalties for that!

    USER ABUSE! You can't claim mod abuse from a non-mod!

    when did you get promoted to admin?

    :tada:

    belated congratulations!

    <I jest of course



  • @accalia said:

    when did you get promoted to admin?

    He didn't? :laughing:


  • sockdevs

    @loopback0 said:

    @accalia said:
    when did you get promoted to admin?

    He didn't? :laughing:


    :ahem:
    <view raw maybe?


  • Winner of the 2016 Presidential Election

    Damned people being mods on some instances and not on other ones mumble grumble


  • mod

    @accalia said:

    when did you get promoted to admin?

    :tada:

    belated congratulations!

    :question:

    @loopback0 said:

    He didn't? :laughing:

    This. ^

    @Onyx said:

    Damned people being mods on some instances and not on other ones mumble grumble

    Ok, I am an admin on the NodeBB and Flarum instances, for exploratory purposes.


  • sockdevs

    @abarker said:

    @accalia said:
    when did you get promoted to admin?

    :tada:

    belated congratulations!

    :question:

    .... does NO ONE read the raw?



  • @accalia said:

    @abarker said:
    @accalia said:
    when did you get promoted to admin?

    :tada:

    belated congratulations!

    :question:

    .... does NO ONE read the raw?

    Do I check for hidden text on every post, just in case?
    No. No I do not.


  • mod

    When there is no indication that there is any raw to read?



  • @accalia said:

    I wrote a Promised based, limited concurrency task queue for javascript....

    why? because i had no idea what i was getting myself in for.....

    End of the line please:

    Last week I wrote a RequireJS loader plugin that spawns a web worker thread, injects RequireJS (and its map and path configuration) into it, loads the specified module into it, hooks up task completion (bridged via cross-thread message posting) as a promise and am currently figuring out how to integrate cancellation tokens.

    It's actually pretty neat to be able to do

    define([ "task!../tasks/long-running" ], function( Task ) {
      // ...
      Task
        .run( a, b, c )
        .then( function( result ) {
          // ...
        });
    })
    

    and just have it work...

    but in building it I think I lost a few pieces of my sanity.



  • @Ragnax said:

    in building it I think I lostupgraded a few pieces of my sanity.

    eat and be eaten... }}}()



  • That's some convoluted mess you've got there.
    I'm afraid to ask... does it work?


  • sockdevs

    actually.... YES!

    tested on node.js 0.10, 0.12, 4.0, and io.js 1.7, 1.8, 2.0, and 3.0



  • @accalia said:

    actually.... YES!

    tested on node.js 0.10, 0.12, 4.0, and io.js 1.7, 1.8, 2.0, and 3.0

    Heh, I guess you've just reached the point where you're sick of the limitations of async.js and started doing your own concurrency stuff. I already have several of these freak children buried in my back yard.

    My favourite part so far:

                var fn, promise = new Promise(function (resolve, reject) {
                    fn = function () {
                       // ...
                    };
                    fn.progress = progress;
                });
                result.tasks.push(fn);
    

    Better hope that function is always called synchronously. And that nothing leaks.


  • Winner of the 2016 Presidential Election

    @cartman82 said:

    I already have several of these freak children buried in my back yard.

    I thought the usual practice is to keep them in the attic and feed them fish heads, so that one day you can parade them on TDWTF.



  • @Onyx said:

    I thought the usual practice is to keep them in the attic and feed them fish heads, so that one day you can parade them on TDWTF.

    No no no... no. They are to remain safely buried away, never to be seen by innocent eyes.


  • Winner of the 2016 Presidential Election

    @Onyx said:

    TDWTF

    @cartman82 said:

    innocent

    I found the error in your reasoning.


  • sockdevs

    @cartman82 said:

    Heh, I guess you've just reached the point where you're sick of the limitations of async.js and started doing your own concurrency stuff.

    something like that..... if you look at the full project you'll notice it looks suspiciously liek a clone of the async.js api, but with promises not callbacks.

    mostly it was because i wanted to see if i could do it.

    so far i only have one runtime dependency: promise-polyfill and it's only there to add Promise to node 0.10.

    @cartman82 said:

    My favourite part so far:

    yeah that was an interesting bit.

    @cartman82 said:

    Better hope that function is always called synchronously.
    it should be. that's a closure covered function.

    @cartman82 said:

    And that nothing leaks.
    so far i havent found a leak, at least assuming you provide promises that do in fact eventually resolve or reject.

    havent extensively tested the queue yet though, because it's still pretty new.



  • @accalia said:

    .... does NO ONE read the raw?

    Only when I see comments like this. Or when something seems off. Or when the post is blank. So, yeah, not much.



  • @accalia said:

    .... does NO ONE read the raw?

    No.

    Not even if the post specifically says to.

    Because fuck that.

    I also don't read the really tiny text.

    And every time I see both, I hate the poster just a little bit more.



  • @blakeyrat said:

    I also don't read the really tiny text.

    Is this text large enough to meet your standards?


  • area_deu

    @accalia said:

    workers += 1;

    I see your problem ...


  • Winner of the 2016 Presidential Election

    It should be ++workers;?

    :trolleybus:



  • @accalia said:

    .... does NO ONE read the raw?

    No, that is why there are whisper-posts!


  • area_deu

    Actually, yes.



  • Why are you doing this:

    return Promise.resolve(v);

    When you already have resolve in the arguments?


  • sockdevs

    @Eldelshell said:

    When you already have resolve in the arguments?

    Because i have two separate promises to resolve/reject.

    i should figure out how to chain them properly but i don't think it's possible because they are created at different times. still there might be a better way.

    This bit of code here:

    https://github.com/AccaliaDeElementia/aplus/blob/master/lib/aplus.js#L557-L572

    is creating a promise to return our of the queue, as well as a function (stored in fn) to be called when the task's turn is up in the queue, this inner function returns a different promise that's used for maintaining the queue internals. both promises need to resolve with the same value, the first one because that's the one that gets passed out of the queue to the caller, and the second one because that's where I notify the progress callback of a completed task.



  • Yes.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.