Broken Poll: What database engine should I use?


  • SockDev

    Working on @sockbot and i'm thinking i need some sort of persistent storage. What should i use for that?

    postgres is already a dependency for the backup and statsporn modules so there's that.

    but then there's mongodb which would let me be much more flexible in what kinds of things i can store

    Of course there's always sqlite or even just serializing JSON or YAML to disk (because i don't anticipate needing more than a couple of megabytes of persistent storage

    [poll]

    • Postgres
    • MongoDB
    • Sqlite
    • MySQL
    • MsSQL
    • DB2
    • YAML
    • JSON
    • DATABASE_NOT_FOUND
    • TDEMSYR
    • I LIKE CAPITAL LETTERS
    • Oracle, because the bots need to die in a fire
    • OTHER_DATABASE_NOT_FOUND
      [/poll]


  • @accalia has summoned me, and so I appear.

    <!-- Posted by SockBot 0.15.1 "Zany Zoe" on Wed, 04 Mar 2015 13:32:04 GMT-->

  • SockDev

    I would vote sqlite, but voting is broken

    <!-- Emoji'd by MobileEmoji 0.2.0-->


  • Me, too.



    • MS Access
    • LibreOffice Base/another obscure MS Access clone
    • Some cool new hip database engine written purely in Javascript/Go/Swift/Dart/Brainfuck/jQuery/Discourse
    • Your own engine written in pure assembly that stores values on raw disk sectors (because partitions and filesystems just add unnecessary overhead)


  • @anonymous234 said:

    Your own engine written in pure assembly that stores values on raw disk sectors

    So, APDB?


  • SockDev

    @RaceProUK said:

    I would vote sqlite, but voting is broken

    oh...... right.

    forgot about that.

    :frowning:



  • @RaceProUK said:

    I would vote sqlite, but voting is broken

    Same here, the poll thinks I'm not logged in or something.



  • WELL, I LIKE CAPITAL LETTERS.

    (Poll is broken, yeah.)



  • I LIKE CAPITAL LETTERS

    @anonymous234 said:

    Discourse

    now that's an idea. make a bot to store the data in pm's and retrieve it with discosearch.



  • SQLITE


    Filed under: "body sucks try to be more descriptive" is a barrier.



  • I would vote "use an ORM that abstracts different databases under a common API (with SQLite under that)"


  • SockDev

    got one in mind? (keeping in mind that Sockbot is written in nodejs?)



  • @accalia said:

    but then there's mongodb which would let me be much more flexible in what kinds of things i can store

    Doing aggregate queries can be a pain though, in my limited experience.


  • SockDev

    true, but then just about any NoSQL DB has that issue.


  • Winner of the 2016 Presidential Election

    Why is Discourse not an option? Clearly for a Discoursebot to store data the only solution :wtf: enough is to (ab)use Discourse.

    Filed Under: each new user gets a topic in a local Discourse etc.
    Also Filed Under: mad ideas go into some topic...


  • SockDev

    huh... so editing the topic title to close the poll generates a notification to me... but clicking the close poll button doesn't?

    Discoursistency!



  • To be honest, I don't have much experience with NodeJS in general and ORMs in particular, other than a failed project which used Mongoose (and therefore Mongo).

    Which leads me to say this: stay away from that MongoDB if you want your project to scale in complexity.

    Mongoose+MongoDB is very nice for fast prototyping, but when you start needing more complex queries it becomes very painful very fast. It might be alright for sockbot, but go in that direction if you need a simple structured object store more than a database.

    If you do end up wanting an honest-to-Codd database, Knex.js looks good and I've been meaning to give it a try if I ever touch NodeJS again. It will need driver libraries underneath, but that's explained in its homepage.

    But.. I don't have any actual experience with it whatsoever, so take this recommendation with a huge grain of salt!

    Oh, and it's not really an ORM per se, more of a query builder, I guess :blush:. But that's a good thing in my book.


  • SockDev

    good to know.

    well that strikes NOSQL databases off the table i guess. ;-)



  • @Kuro said:

    Why is Discourse not an option? Clearly for a Discoursebot to store data the only solution :wtf: enough is to (ab)use Discourse.

    Filed Under: each new user gets a topic in a local Discourse etc.
    Also Filed Under: mad ideas go into some topic...

    The obvious solution is to use an in-memory SQLite database, and after each write dump the whole database into the bot’s Gender profile field. When the bot starts up, just read the data back and there you go!


    Filed under: This way you can use the bot on embedded systems without a filesystem


  • WTF I'm getting a login dialog whenever I try to vote.

    Also, the correct answer is MS Access.


  • SockDev

    @JazzyJosh said:

    WTF I'm getting a login dialog whenever I try to vote.

    Known issue :stuck_out_tongue:

    <!-- Emoji'd by MobileEmoji 0.2.0-->

  • SockDev

    any news on a fix?


  • area_deu



  • I assume @RaceProUK has been hitting it with a hammer the entire time and it's doing nothing.

    Related:

    THIS IS HOW WE FIX PROBLEMS ON RUSSIAN SPACE STATION!!!!!! – 00:42
    — ZioVicProductions


  • SockDev

    @accalia said:

    any news on a fix?

    Do I look like a Discodev? :stuck_out_tongue:
    @JazzyJosh said:
    I assume @RaceProUK has been hitting it with a hammer the entire time and it's doing nothing.

    Only one problem: I use my hammer to break things :stuck_out_tongue:

    <!-- Emoji'd by MobileEmoji 0.2.0-->

  • BINNED



  • @Kuro said:

    Why is Discourse not an option? Clearly for a Discoursebot to store data the only solution enough is to (ab)use Discourse.

    @Jarry said:

    now that's an idea. make a bot to store the data in pm's and retrieve it with discosearch.

    :eyes:



  • Just wanted to add: using the sqlite3 module on its own might not be a bad idea.

    It already gives you parameter binding and results as objects:

    var db = new sqlite3.Database(':memory:');
    
    db.serialize(function() {
      db.run("CREATE TABLE lorem (info TEXT)");
    
      var stmt = db.prepare("INSERT INTO lorem VALUES (?)");
      for (var i = 0; i < 10; i++) {
          stmt.run("Ipsum " + i);
      }
      stmt.finalize();
    
      db.each("SELECT rowid AS id, info FROM lorem", function(err, row) {
          console.log(row.id + ": " + row.info);
      });
    });
    
    db.close();
    

    When you select over multiple tables you'll need to split the data into separate objects yourself, and you'll need unique columns names in the returned set, but it's not that bad.


  • SockDev

    well then i think i'll be doing that.

    but i'll use an on disk database. some isntances run in very memory constrained setups.



  • @Jarry said:

    now that's an idea. make a bot to store the data in pm's and retrieve it with discosearch.

    Wasn't there some guy who abused gmail like that?



  • http://gmail-disk.com/ ??
    this was before dropbox IIRC



  • Could be. I can't remember enough of the details now to confirm or deny.


  • SockDev

    :wtf: we have website filtering here?

    also why is this marked illegal or unethical?


  • SockDev

    This is why I like working for small companies. They never bother with all that blocking shit; they have better things to spend their money on. Like winning new clients.

    <!-- Emoji'd by MobileEmoji 0.2.0-->


  • If you already have postgres all set up, you don't need anything else. With json support in newer versions, postgres can do pretty much everything mongo can. The reverse is not the case.

    Use mongo only if you want to add some cred to your CV, which is totally legit concern for a hobby project.


  • SockDev

    hmm i could upgrade postgres from an optional dependency to a required.


  • SockDev

    Guess I'll be installing postgres on my EC2 then…

    <!-- Emoji'd by MobileEmoji 0.2.0-->

  • SockDev

    unless I decide to stick with sqlite3

    i'm still feeling this out.

    EDIT: besides didn't you already want to install that so you could test the queries you keep writing?


  • SockDev

    @accalia said:

    besides didn't you already want to install that so you could test the queries you keep writing?

    Oh yeah…

    <!-- Emoji'd by MobileEmoji 0.2.0-->

  • Grade A Premium Asshole

    @accalia said:

    unless I decide to stick with sqlite3

    My vote is for sqlite, FWIW. This is essentially what it was made for.

    Well, it was made to be an embedded database for the Navy I think? But the point still stands, you are weaponizing annoying bots. ;)


  • SockDev

    @Polygeekery said:

    you are weaponizing annoying bots

    So that's what the Peacekeeper module is for… :laughing:

    <!-- Emoji'd by MobileEmoji 0.2.0-->

  • SockDev

    shhhh.... dont tell them that or they'll find the LowOrbitIonCannon module!



  • @RaceProUK said:

    I would vote sqlite, but voting is broken

    I would vote sqlite, but voting is broken



  • @Zecc said:

    using the sqlite3 module on its own might not be a bad idea.

    I think I just used whichever one came out of npm, but it did everything databasey that I needed it to do...



  • Furthermore, SQLite is great and you should use it at every opportunity.


  • SockDev

    https://github.com/AccaliaDeElementia/SockBot/blob/master/persist.js

    i went with SQLITE3

    and yeah.... that's gonna need rework soon, but for now....



  • @tar said:

    Furthermore, SQLite is great and you should use it at every opportunity.

    QFT -- it's a fantastic solution for all those times you wish you had a database, but your budget can't handle something the size of Ellison's yacht.

    (It's also capable of running on a large-ish Cortex-M -- try that one with your favorite DBMS sometime ;)



  • @tarunik said:

    but your budget can't handle something the size of Ellison's yacht.

    MS SQL is like $700.

    (Although, given, it takes like 4 PTOs a month to figure out how the fuck to license it.)


  • SockDev

    @accalia said:

    i went with SQLITE3

    well... @ben_lubar pointed out that i had basically implemented NoSQL in SQL so.... yeah that persistence layer is getting thrown out and either a sane SQL structure built or get replaces with an actual NoSQL solution.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.