Is MongoDB for me? (No, it's not, this is now @mott555's Random Node.js Question Thread)

FrostCat

What are these promise things all y'all keep talking about? Is it node-related or something more general?

accalia

What are these promise things all y'all keep talking about?

a design pattern for async things. the implementations and spec (Promises/A+) we're talking about are node specific but promises are not necessarily JS specific.

Promises/A+

mott555

Basically a better way of chaining callbacks for async results in Node.js. It's very similar to the old way, but there are far fewer lines of code required and it's quite a bit easier to read and follow (once you understand it).

accalia

@mott555 said:

(once you understand it).

indeed, although i would say it';s no harder to comprehend than CPS

mott555

@accalia said:

CPS

Child Protective Services?

It took me a few hours to understand promises. Before then, the code I had written while trying to understand it was CodeSOD-worthy. I had promises nested within promises nested within promises, it looked worse than callback hell and I was questioning why promises were a good thing. Then it all clicked.

accalia

@mott555 said:

Child Protective Services?

Continuation Passing Style.

it's the fancypants name for "callback hell"

Sentenryu

You could write that as:

    var commandChain = commands[0].Try();
    for(var i = 1; i< commands.length; i++){
        commandChain = commandChain.then(function(){
            return commands[i].Try();
        });
    }

Not sure if the assignment is even needed, but included it just in case.

accalia

@Sentenryu said:

You could write that as:
    var commandChain = commands[0].Try();
    for(var i = 1; i< commands.length; i++){
        commandChain = commandChain.then(function(){
            return commands[i].Try();
        });
    }
Not sure if the assignment is even needed, but included it just in case.

reading

parsing...

processing...

processing....

....

processing..............

bing!

complete.

that... that would actually work. a bit wasteful in memory if you have a very large amount of commands (the chain gets created at the very beginning instead of as needed) but otherwise, yeah that should work just fine.

Sentenryu

If memory is really an issue, then i would probably use the recursion method and hope for tail call optimization to work.

Alternatively, if i didn't need the array of commands for other things, i would get rid of it altogether and have the chain be the only data structure representing the commands.

Also, is there really a need for each command to be asynchronous? looks like they always process synchronous relative to each other, so having an async method that iterates and executes every command synchronously should be enough.

mott555

@Sentenryu said:

If memory is really an issue, then i would probably use the recursion method and hope for tail call optimization to work.

Memory isn't really an issue. I ended up using recursion. I only have a few commands reimplemented right now but I doubt there will ever be more than a few dozen, including admin/moderator stuff.

@Sentenryu said:

Also, is there really a need for each command to be asynchronous? looks like they always process synchronous relative to each other, so having an async method that iterates and executes every command synchronously should be enough.

The major reason is that some commands may require database calls, which are always async anyway.

mott555

I'm having some crazy problems with the bluebird promise library suddenly. I did some minor refactoring (adding setUp() and tearDown() to deal with all the boilerplate of creating a new world and closing the database) to my unit tests and now bluebird is exploding.

Here's the offending code:

module.exports.setUp = function(callback) {
    if (fs.existsSync(DB_FILENAME))
        fs.unlinkSync(DB_FILENAME);
    database = new Database(DB_FILENAME);
    database.Open()
    .then(function() {
        universe = new Universe(database);
        return universe.Initialize();
    }).then(function() {
        admin = universe.PlayerCharacters["admin"];
        cmd = new CommandProcessor(admin, universe);
        callback();
    });
};

module.exports.tearDown = function(callback) {
    database.Close()
    .then(function() {
        fs.unlinkSync(DB_FILENAME);
        callback();
    });
};

exports.testCmdAdminSay = function(test) {
    
    test.ok(admin);
    test.ok(cmd);
    test.equal(admin.Entity.IsAdmin, 1);
    
    cmd.Process("admin Hello, admins!")
    .then(function(r) {
        
        test.equal(r.result, "success");
        test.equal(r.cmd.Name, "CmdAdminSay");
        
        // Revoke admin rights and try again.
        admin.Entity.IsAdmin = 0;
        return cmd.Process("admin Hello, admins!");
    }).then(function(r) {
        test.equal(r.result, "notFound");
        test.ifError(r.cmd);
    }).finally(function() {
        test.done();
    });
};

The error message, occurring in testCmdAdminSay, is:

Unhandled rejection TypeError: cmd.Process(...).then(...).then(...).finally is not a function

I have this pattern all through my unit test suites and this is the only one that's having trouble. I don't see any obvious differences, and Googling this error has taken me down several unrelated rabbit holes that have nothing to do with my issue.

If I remove the .finally() handler it compiles, though that leaves me no reliable way of calling test.done() at the end.

Any ideas what I should be looking for?

accalia

@mott555 said:

Any ideas what I should be looking for?

your second then() call does not return a promise or a value. i suspect it is therefore being treated as a teminal call (not being chained off of) return something like return Promise.resolve(); and i think it will start working again.

mott555

@accalia said:

your second then() call does not return a promise or a value.

I'll mess with that and see, but I'm doing this kind of thing all over the place without other problems. Maybe I shouldn't be?

Well, no change, it still errors.

exports.testCmdAdminSay = function(test) {
    
    test.ok(admin);
    test.ok(cmd);
    test.equal(admin.Entity.IsAdmin, 1);
    
    cmd.Process("admin Hello, admins!")
    .then(function(r) {
        
        test.equal(r.result, "success");
        test.equal(r.cmd.Name, "CmdAdminSay");
        
        // Revoke admin rights and try again.
        admin.Entity.IsAdmin = 0;
        return cmd.Process("admin Hello, admins!");
    }).then(function(r) {
        test.equal(r.result, "notFound");
        test.ifError(r.cmd);
        return Promise.resolve();
    }).catch(function(err) {
        test.ifError(err);
    }).finally(function() {
        test.done();
    });
};

mott555

Meanwhile, this in a totally different test suite works fine.

module.exports.setUp = function(callback) {
    if (fs.existsSync(DB_FILENAME))
        fs.unlinkSync(DB_FILENAME);
    database = new Database(DB_FILENAME);
    database.Open()
    .then(function() {
        callback();
    });
};

module.exports.tearDown = function(callback) {
    database.Close()
    .then(function() {
        fs.unlinkSync(DB_FILENAME);
        callback();
    });
};

module.exports.testRealmSave = function(test) {
    Realm.GetRealm(0, database)
    .then(function(realm) {
        test.notEqual(realm.Entity.Description, "New realm name");
        test.notEqual(realm.Entity.R, 1);
        realm.Entity.Description = "New realm name";
        realm.Entity.R = 1;
        return realm.Save();
    }).then(function() {
        // Confirm Realm 0 is gone.
        return Realm.GetRealm(0, database);
    }).then(function(realm) {
        test.ifError(realm);
        
        // Confirm Realm 1 exists.
        return Realm.GetRealm(1, database);
    }).then(function(realm) {
        test.ok(realm);
        test.equals(realm.Entity.R, 1);
        test.equals(realm.Entity.Description, "New realm name");
    }).catch(function(err) {
        test.ifError(err);
    }).finally(function() {
        test.done();
    });
};

mott555

Looks like I was using exports.myTest = function(test) and module.exports.myTest = function(test) interchangeably. I made them consistent, no difference though.

accalia

@mott555 said:

Well, no change, it still errors.

huh.... well there goes my idea....

mott555

It has something to do with my setUp function. If I strip that down to only open the database, and not initialize the universe, it works. Initializing a universe can take a while so I wonder if this is a weird timing-dependent thing.

anotherusername

@mott555 said:

Here's the offending code:

module.exports.setUp = function(callback) {

if (fs.existsSync(DB_FILENAME))

fs.unlinkSync(DB_FILENAME);

database = new Database(DB_FILENAME);

database.Open()

.then(function() {

universe = new Universe(database);

return universe.Initialize();

}).then(function() {

admin = universe.PlayerCharacters["admin"];

cmd = new CommandProcessor(admin, universe);

callback();

});

};

@mott555 said:

It has something to do with my setUp function. If I strip that down to only open the database, and not initialize the universe, it works

Is universe still defined when you try to access universe.PlayerCharacters["admin"]? It's defined in a function, wouldn't its scope be limited to only that function?

mott555

Last I knew, yes. It's defined at the top of the file, outside any functions, so it's in scope everywhere. I just didn't show that part.

I worked around the problem by making my own setUp() function and calling it manually at the start of each test. I have to have hit some weird edge case when combining promises with nodeunit because it works if I do it myself.

anotherusername

Ah... okay, what's the pattern of sometimes then(function (parameter) {...}) and sometimes just then(function () {...})? Where does the parameter come from?

mott555

The parameter is returned by the previous promise if that promisified function needs to return a value.

A void promise:

function myFunc() {
    return new Promise(function(fulfill, reject) {
        // Do something interesting.
        fulfill();
    });
}

myFunc().then(function() {
    // No parameter.
});

One that returns something:

function myFunc() {
    return new Promise(function(fulfill, reject) {
        var retVal;
        // Do something interesting.
        fulfill(retVal);
    });
}

myFunc().then(function(retVal) {
    // Parameter exists
});

anotherusername

The other difference I see between the error one and the working one is cmd.Process vs. Realm.GetRealm at the beginning of the chain.

Is it possible that cmd.Process is a hack job that somehow only lets you call one then on its return value?

mott555

Maybe. cmd.Process is doing some really strange things with recursion. Although once I have the working setUp() code I've had no problem tacking a ton of thens onto cmd.Process. That's basically how I'm testing my command processor, feed it stuff, check which command (if any) reported it did some work, then check a bunch of world state to make sure it did what it's supposed to.

I actually narrowed down the problem to finally. If I didn't have finally in there the error I was seeing went away. But that wasn't useful because I needed a guaranteed place to do some stuff at the end of testing.