Sqlite



  • I can't post any source code snippets because I'm writing this in XML on an embedded system with no filesystem, but if you select a line from sqlite's source code at random, you'll either get a comment that was written on a telegram with extremely high cost/word or a line of code with a security hole, a memory leak, or a mutex lock that somehow causes a segfault.

    Also the entire thing is one 100000 line C file.



  • This proves my favorite expression which is that "no application is too small to be poorly written".



  • You are not looking at the raw source code for SQLite but rather the amalgamation.



  • You see, as SQLite is mostly used in web apps they joined and minified sources to optimize user experience!



  • @sinan said:

    You are not looking at the raw source code for SQLite but rather the amalgamation.

    OK, so we can scratch the bit about the 100000-line source file.


    What about the rest?



  • @Ben L. said:

    if you select a line from sqlite's source code at random, you'll either get a comment that was written on a telegram with extremely high cost/word
    Apparently you have really bad luck when selecting things at random?  @sinan said:
    the amalgamation.
    @The Amalgamation said:
    The amalgamation is a single C code file, named "sqlite3.c", that contains all C code for the core SQLite library and the FTS3 and RTREE extensions. This file contains about 110K lines of code (65K if you omit blank lines and comments).
    So out of 110k lines of code, 45k are comments.  That seems reasonably verbose to me.



  • @Hatshepsut said:

    What about the rest?

    Where do you see all these memory leaks? SQLite is used in a lot of places these days, so if there were any major memory leaks I'd have expected people to have noticed them by now. If anything it seems to really go out of its way when it comes to memory management.



  • @qbolec said:

    You see, as SQLite is mostly used in web apps they joined and minified sources to optimize user experience!
     

    It makes the code run faster!



  • @Hatshepsut said:

    OK, so we can scratch the bit about the 100000-line source file.


    What about the rest?


    I found [url=http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2007-1888]one security hole listed in the National Vulnerability Database[/url] - against an obsolete version. Other than that I got nuthin'. If there are any others, no-one's found them or no-one's talking.



  • @Watson said:

    @qbolec said:

    You see, as SQLite is mostly used in web apps they joined and minified sources to optimize user experience!
     

    It makes the code run faster!

     

    Indeed, it does. It competes with plain text files (and XML), it's much faster than them.

     



  • @Watson said:

    @qbolec said:

    You see, as SQLite is mostly used in web apps they joined and minified sources to optimize user experience!
     

    It makes the code run faster!

     

    Memory leaks also make it run faster! None of that stupid memory management. Just allocate. It all adds up



  • @TGV said:

    Memory leaks also make it run faster! None of that stupid memory management. Just allocate. It all adds up

     

    Maybe they should put some in, then.

     



  • It's difficult to tell sometimes whether Sqlite is difficult to deal with because it's linked into one's own (C++) program, or if it has inherent problems of its own. However, I have definitely documented cases where it does not work as advertised. It's probably not a coincidence that "Sqlite" and "NoSQL" entered the IT lexicon at roughly the same time. People who try and defend Sqlite based on the fact that it's widespread are just bandwagon - jumpers incapable of forming their own opinions.



  • @bridget99 said:

    However, I have definitely documented cases where it does not work as advertised
     

    Could you link to any of these then? So we could form our own opinions?

    The only issue I had with SQLite not working properly is two PHP apps accessing the same DB and one fetching stale data (a forum module mining version data from SVN/Trac). I've not used it a great deal since, but for many lightweight web-apps I've been advised to use SQLite over MySQL.



  • @Cassidy said:

    @bridget99 said:

    However, I have definitely documented cases where it does not work as advertised
     

    Could you link to any of these then? So we could form our own opinions?

    The only issue I had with SQLite not working properly is two PHP apps accessing the same DB and one fetching stale data (a forum module mining version data from SVN/Trac). I've not used it a great deal since, but for many lightweight web-apps I've been advised to use SQLite over MySQL.

    I can link to URLs, not issues. Issues exist independently of whether I provide a URL. Besides that, it seems like you know of an issue. I'll add one: I was never able to make Sqlite work properly from multiple threads of the same process. I wasn't even trying to access the same database. Rather, I was trying to have one thread work on one database and another thread work on a second database. Everyone's attitude was "oh, that SHOULD work" but it did not, at least not for me. I wrestled with this problem for a while, and then just decided to spawn a second process. I suspect that there is a lot of this sort of thing going on; if you really think all of the bugs have been worked out of Sqlite, you're deluding yourself.



  • @bridget99 said:

    I can link to URLs

    I can click on URLs


     


    but you didn't provide any.



  • @bridget99 said:

    @Cassidy said:

    @bridget99 said:

    However, I have definitely documented cases where it does not work as advertised
     

    Could you link to any of these then? So we could form our own opinions?

    The only issue I had with SQLite not working properly is two PHP apps accessing the same DB and one fetching stale data (a forum module mining version data from SVN/Trac). I've not used it a great deal since, but for many lightweight web-apps I've been advised to use SQLite over MySQL.

    I can link to URLs, not issues. Issues exist independently of whether I provide a URL. Besides that, it seems like you know of an issue. I'll add one: I was never able to make Sqlite work properly from multiple threads of the same process. I wasn't even trying to access the same database. Rather, I was trying to have one thread work on one database and another thread work on a second database. Everyone's attitude was "oh, that SHOULD work" but it did not, at least not for me. I wrestled with this problem for a while, and then just decided to spawn a second process. I suspect that there is a lot of this sort of thing going on; if you really think all of the bugs have been worked out of Sqlite, you're deluding yourself.

    There are specific flags you compile with to enable multi-threaded support. Read the documentation.



  • @Cassidy said:

    @bridget99 said:

    it does not work as advertised

    two PHP apps accessing the same DB

    Isn't SQLite specifically contraindicated for this use case?

    @JoeCool said:

    @bridget99 said:

    thread

    There are specific flags you compile with to enable multi-threaded support. Read the documentation.

    So, you're saying that it does, in fact, work exactly as advertised?

    @bridget99 said:

    if you really think all of the bugs have been worked out of foo, you're deluding yourself.

    Brillant insight.



  • @Kittemon said:

    @JoeCool said:
    @bridget99 said:

    thread

    There are specific flags you compile with to enable multi-threaded support. Read the documentation.

    So, you're saying that it does, in fact, work exactly as advertised?

    I've used both sqlite2 and sqlite3 and the only time I had multi-threading issues was when compiling for a specific *nix platform, I forgot to set the multi-thread flag.



  • @Kittemon said:

    Isn't SQLite specifically contraindicated for this use case?
     

    Not sure what you're trying to say, but if the layman's term is "it don't work that way, fool!" then.. yes.

    I suppose I should have clarified my original tale with "I didn't use it in the way in which it was intended, so I encountered unexpected results." I fully admit that's no fault of SQLite, so it's a non-issue really.

     



  • @JoeCool said:

    @bridget99 said:

    @Cassidy said:

    @bridget99 said:

    However, I have definitely documented cases where it does not work as advertised
     

    Could you link to any of these then? So we could form our own opinions?

    The only issue I had with SQLite not working properly is two PHP apps accessing the same DB and one fetching stale data (a forum module mining version data from SVN/Trac). I've not used it a great deal since, but for many lightweight web-apps I've been advised to use SQLite over MySQL.

    I can link to URLs, not issues. Issues exist independently of whether I provide a URL. Besides that, it seems like you know of an issue. I'll add one: I was never able to make Sqlite work properly from multiple threads of the same process. I wasn't even trying to access the same database. Rather, I was trying to have one thread work on one database and another thread work on a second database. Everyone's attitude was "oh, that SHOULD work" but it did not, at least not for me. I wrestled with this problem for a while, and then just decided to spawn a second process. I suspect that there is a lot of this sort of thing going on; if you really think all of the bugs have been worked out of Sqlite, you're deluding yourself.

    There are specific flags you compile with to enable multi-threaded support. Read the documentation.

    Did it ever occur to you that I already knew that? I did. I actually worked very diligently to figure out what was going on, but eventually my desire to move on to actually getting things done (vs. sorting through "D. Richard Hipp's" messes) led me to the idea of using multiple processes instead of multiple threads. And just for future reference, when you post a sarcastic response that assumes extreme stupidity on the part of someone else, particularly someone doing real programming on embedded devices, you just look like a dipshit.



  • @bridget99 said:

    However, I have definitely documented cases where it does not work as advertised.

    Has this been demonstrated yet..?



  • @bridget99 said:

    Did it ever occur to you that I already knew that? I did. I actually worked very diligently to figure out what was going on, but eventually my desire to move on to actually getting things done (vs. sorting through "D. Richard Hipp's" messes) led me to the idea of using multiple processes instead of multiple threads. And just for future reference, when you post a sarcastic response that assumes extreme stupidity on the part of someone else, particularly someone doing real programming on embedded devices, you just look like a dipshit.

    I think you look more like a dipshit when you can't get something to work that is well documented, other developers are having success with, and you are blaming the tools.



  • @JoeCool said:

    @bridget99 said:
    Did it ever occur to you that I already knew that? I did. I actually worked very diligently to figure out what was going on, but eventually my desire to move on to actually getting things done (vs. sorting through "D. Richard Hipp's" messes) led me to the idea of using multiple processes instead of multiple threads. And just for future reference, when you post a sarcastic response that assumes extreme stupidity on the part of someone else, particularly someone doing real programming on embedded devices, you just look like a dipshit.

    I think you look more like a dipshit when you can't get something to work that is well documented, other developers are having success with, and you are blaming the tools.

    My experience with code that has parallelism-related issues is that the resulting bugs are very subtle. There are so many variables at play. What works on one system will fail on another, because of things like how instructions get interleaved, how and where memory gets allocated, and so on. The fact that something seems to work for thousands of other people doesn't mean a whole lot.



    Now, I don't know if the version of Sqlite I was using had a bug related to threading or not, but for you to just assume I didn't make the most basic of configuration calls is asinine. To draw an analogy, it's as if you came to me and said "Excel won't open on my computer any more" and I assumed that your computer was unplugged. I realize that this is not a tech support forum, but if your attitude toward people who come to you for real help is anything like that, then I feel sorry for the poor, misguided people who sign your paycheck. I run into people with that sort of dismissive attitude a lot in "IT" and I just want to grab their bosses by the lapels and shout "you can do so much better!" It seriously reminds me of Battered Wife Syndrome, or Stockholm Syndrome. It doesn't work on me, though... bro.



  • @bridget99 said:

    @JoeCool said:
    @bridget99 said:
    Did it ever occur to you that I already knew that? I did. I actually worked very diligently to figure out what was going on, but eventually my desire to move on to actually getting things done (vs. sorting through "D. Richard Hipp's" messes) led me to the idea of using multiple processes instead of multiple threads. And just for future reference, when you post a sarcastic response that assumes extreme stupidity on the part of someone else, particularly someone doing real programming on embedded devices, you just look like a dipshit.

    I think you look more like a dipshit when you can't get something to work that is well documented, other developers are having success with, and you are blaming the tools.

    My experience with code that has parallelism-related issues is that the resulting bugs are very subtle. There are so many variables at play. What works on one system will fail on another, because of things like how instructions get interleaved, how and where memory gets allocated, and so on. The fact that something seems to work for thousands of other people doesn't mean a whole lot.



    Now, I don't know if the version of Sqlite I was using had a bug related to threading or not, but for you to just assume I didn't make the most basic of configuration calls is asinine. To draw an analogy, it's as if you came to me and said "Excel won't open on my computer any more" and I assumed that your computer was unplugged. I realize that this is not a tech support forum, but if your attitude toward people who come to you for real help is anything like that, then I feel sorry for the poor, misguided people who sign your paycheck. I run into people with that sort of dismissive attitude a lot in "IT" and I just want to grab their bosses by the lapels and shout "you can do so much better!" It seriously reminds me of Battered Wife Syndrome, or Stockholm Syndrome. It doesn't work on me, though... bro.

    Multithreading issues: look at multithreading configuration
    Excel issue: look at computer unplugged

    Yeah, I can see why you had issues.



  • @JoeCool said:

    @bridget99 said:
    @JoeCool said:
    @bridget99 said:
    Did it ever occur to you that I already knew that? I did. I actually worked very diligently to figure out what was going on, but eventually my desire to move on to actually getting things done (vs. sorting through "D. Richard Hipp's" messes) led me to the idea of using multiple processes instead of multiple threads. And just for future reference, when you post a sarcastic response that assumes extreme stupidity on the part of someone else, particularly someone doing real programming on embedded devices, you just look like a dipshit.

    I think you look more like a dipshit when you can't get something to work that is well documented, other developers are having success with, and you are blaming the tools.

    My experience with code that has parallelism-related issues is that the resulting bugs are very subtle. There are so many variables at play. What works on one system will fail on another, because of things like how instructions get interleaved, how and where memory gets allocated, and so on. The fact that something seems to work for thousands of other people doesn't mean a whole lot.



    Now, I don't know if the version of Sqlite I was using had a bug related to threading or not, but for you to just assume I didn't make the most basic of configuration calls is asinine. To draw an analogy, it's as if you came to me and said "Excel won't open on my computer any more" and I assumed that your computer was unplugged. I realize that this is not a tech support forum, but if your attitude toward people who come to you for real help is anything like that, then I feel sorry for the poor, misguided people who sign your paycheck. I run into people with that sort of dismissive attitude a lot in "IT" and I just want to grab their bosses by the lapels and shout "you can do so much better!" It seriously reminds me of Battered Wife Syndrome, or Stockholm Syndrome. It doesn't work on me, though... bro.

    Multithreading issues: look at multithreading configuration
    Excel issue: look at computer unplugged

    Yeah, I can see why you had issues.

    He didn't ask if I knew about the setting. He assumed I did not: "There are specific flags you compile with to enable multithreaded support. Read the documentation." Then, one of his circle-jerk partners jumped in with: "so it does work as advertised." Two completely unjustified logical leaps were made there, out of arrogance. What I see here are a bunch of sad little men who need to make peace with the size of their own genitalia and their general irrelevance to human culture, i.e. typical IT neckbeards.



  • @bridget99 said:

    @JoeCool said:
    @bridget99 said:
    @JoeCool said:
    @bridget99 said:
    Did it ever occur to you that I already knew that? I did. I actually worked very diligently to figure out what was going on, but eventually my desire to move on to actually getting things done (vs. sorting through "D. Richard Hipp's" messes) led me to the idea of using multiple processes instead of multiple threads. And just for future reference, when you post a sarcastic response that assumes extreme stupidity on the part of someone else, particularly someone doing real programming on embedded devices, you just look like a dipshit.

    I think you look more like a dipshit when you can't get something to work that is well documented, other developers are having success with, and you are blaming the tools.

    My experience with code that has parallelism-related issues is that the resulting bugs are very subtle. There are so many variables at play. What works on one system will fail on another, because of things like how instructions get interleaved, how and where memory gets allocated, and so on. The fact that something seems to work for thousands of other people doesn't mean a whole lot.



    Now, I don't know if the version of Sqlite I was using had a bug related to threading or not, but for you to just assume I didn't make the most basic of configuration calls is asinine. To draw an analogy, it's as if you came to me and said "Excel won't open on my computer any more" and I assumed that your computer was unplugged. I realize that this is not a tech support forum, but if your attitude toward people who come to you for real help is anything like that, then I feel sorry for the poor, misguided people who sign your paycheck. I run into people with that sort of dismissive attitude a lot in "IT" and I just want to grab their bosses by the lapels and shout "you can do so much better!" It seriously reminds me of Battered Wife Syndrome, or Stockholm Syndrome. It doesn't work on me, though... bro.

    Multithreading issues: look at multithreading configuration
    Excel issue: look at computer unplugged

    Yeah, I can see why you had issues.

    He didn't ask if I knew about the setting. He assumed I did not: "There are specific flags you compile with to enable multithreaded support. Read the documentation." Then, one of his circle-jerk partners jumped in with: "so it does work as advertised." Two completely unjustified logical leaps were made there, out of arrogance. What I see here are a bunch of sad little men who need to make peace with the size of their own genitalia and their general irrelevance to human culture, i.e. typical IT neckbeards.

    Why are you responding to my post, talking about me as if I am a different person?



  • So... Can we have the documented cases yet..?



  • @Xyro said:

    So... Can we have the documented cases yet..?
    NO.


  • Trolleybus Mechanic

    @Xyro said:

    So... Can we have the documented cases yet..?
     

    Please, post them! When the forum posting email arrives, I want to see if my Lotus Notes recognized the address and with a single click on the address, did the search, opened that side-bar panel and displayed the BugTraq results.


Log in to reply