The Autograph System



  • G'day folks, I've been lurking for a while now with a view to posting something about the system that I work on, so here goes:

     The tl;dr version:

    -TCL being used to write the entire system
    - 300,000 lines of TCL code in the codebase
    - Custom database written from scratch in TCL
    - Database with only one two-column table for all of the data about everything
    -OpenBSD 3.9 still being used on all of the servers

    At first glance the system, let's call it "Autograph", is very innovative. I have no doubts that had it been done properly the first time, it would have really taken off. The system is described as an "integrated living solution" - an IT infrastructure that is built into expensive apartment complexes, incorporating IP security camera surveillance, access control, power, water, gas and internet metering and touch-screen computers embedded into a wall in every apartment acting as both an information terminal and a VoIP intercom. Best of all, everything was done over the network so the building was only wired up with CAT-5e, no telephone lines, no analogue CCTV cables and PoE devices meant that we often didn't even need power cables.
    The problem, as usual, was the result of consultants with crazy "reinvent the wheel" type ideas and management not giving the developers enough time to implement them properly.

    1.    The Language
    Everything is written in TCL, from the services running on our servers to the software on the information terminals (with a GUI written in TK). There are almost 300,000 lines of TCL code in the codebase.
    For those of you who are unfamiliar with TCL, it is a scripting language designed for rapid prototyping and while it is a very powerful language, it is not designed for writing large programs with wide-scale use in a production system.
    We have fairly large performance and stability issues resulting from using an interpreted language on this scale. Access control suffered a big hit, at times it could take three seconds or more for a door to open after swiping an access card, and everything would crash if we left the servers for more than a week without a reboot.
    Certainly, though, the biggest language-related problem was code maintenance. TCL doesn't have any IDEs in the same calibre as Eclipse/Netbeans or even Visual Studio, so the developers all used vi (or vim if we were lucky enough to have it) and grep to find bugs and edit the code.

    2.  The server OS
    All of our servers still run OpenBSD 3.9, I think originally the idea was to have an extremely secure system, but that kind of goes out the door when you are six or seven versions behind the current release (4.6) because it is "too much trouble to upgrade". Our most recent product release(late 2008) was deployed on OpenBSD 4.0.

    3.  The "Database"
    By far the biggest problem of this system was the database. During the design process, one of the consultants decided that regular SQL-type databases were too slow and unwieldy so he created his own, from scratch, in TCL. It runs like a dog.
    The table structure in this database is woeful. Firstly there is only one table with two columns - key and value - for all of the data that we store. Oh, and we store all of our data for everything in that table, which means resident details are stored right next to keycard IDs, PINs and MAC addresses for network devices.

    Secondly, half of the data looks like this:
    Key    Value
    Keyname(key index)    Value

    And the other half of the data looks like this:
    Key    Value
    Keyname(value)    Key index

    That means that to change a single piece of data, we must update two rows in the table that mutually reference each other.

    Thirdly, database updates are not atomic, so there have been many times where only partial updates occur, and only the first value in a pair of keys is modified - so we are left with keyname(key index)  = new value and keyname(old value) = key index. Hilarity, and overtime work ensues.

    Fourthly, loading new data into the database is painfully slow: a rate of about one record per second. We only have around 5000 rows in this database and that takes almost two hours to load in, so rebuilding the database after a major failure is a painful procedure.

    Fifthly, there is no way easy to delete a row from the database short of taking a text dump of the entire database, manually erasing a row, deleting the in-memory copy and then reloading the whole thing from the text dump. Instead of this, we set the value to -1 and leave the dead record to take up space in the database.

    4.  The system architecture
    Everything in this system is interdependent and combined together in strange ways. For instance, the power, water and gas metering are controlled by the same program that runs the access control system and the software that runs on the information terminals and this program requires the database to be working. For some reason, nothing within our internal network will run without the firewall between our internal network and the internet running. Basically, if any server or network link in the system fails, the entire system is brought down.



  • TCL doesn't have any IDEs in the same calibre as Eclipse/Netbeans or even Visual Studio

    TRWTF is that you think Eclipse and Netbeans are better than Visual Studio.

    We only have around 5000 rows in this database and that takes almost two hours to load in, so rebuilding the database after a major failure is a painful procedure.

    I was going to suggest just setting up a MySQL or PostgreSQL server without asking, but then I realized that chances are the database access code is scattered all over those 300,000 lines of TCL, so you'd have to hunt down every single data access...



  • We have actually successfully put a MySQL backend on our database so that all of the programs can still interface in the same way, but get the added speed and reliability of a normal database, however it is extremely difficult to push out updates to our production servers, so I doubt that this will ever see the light of day.



  • @Heron said:

    TCL doesn't have any IDEs in the same calibre as Eclipse/Netbeans or even Visual Studio

    TRWTF is that you think Eclipse and Netbeans are better than Visual Studio.


    No, TRWTF is both of you think that any of those three are better than vim.@Heron said:

    We only have around 5000 rows in this database and that takes almost
    two hours to load in, so rebuilding the database after a major failure
    is a painful procedure.

    I was going to suggest just setting up a MySQL or PostgreSQL server without asking, but then I realized that chances are the database access code is scattered all over those 300,000 lines of TCL, so you'd have to hunt down every single data access...

    There's also BDB, which AFAIK actually stores records as mere K/V pairs.



  • @rararargh said:

    Certainly, though, the biggest language-related problem was code maintenance. TCL doesn't have any IDEs in the same calibre as Eclipse/Netbeans or even Visual Studio
     

    There is an Eclipse plugin for TCL: http://www.eclipse.org/dltk/



  • @Lingerance said:

    @Heron said:

    TCL doesn't have any IDEs in the same calibre as Eclipse/Netbeans or even Visual Studio

    TRWTF is that you think Eclipse and Netbeans are better than Visual Studio.

    No, TRWTF is both of you think that any of those three are better than vim.
     

    That's fair enough, but I'm unfamiliar with vim beyond :q! and I don't really have any intention of learning.  (I don't use emacs either, so don't start a vim-emacs flamewar with me.)



  • @Heron said:

    TCL doesn't have any IDEs in the same calibre as Eclipse/Netbeans or even Visual Studio

    TRWTF is that you think Eclipse and Netbeans are better than Visual Studio

    Having worked in both Eclipse and Visual Studio I have to agree with OP that Eclipse is better.  Move lines up and down without copying and pasting, Open Type (ctrl-shift-T), Open Resource (ctrl-shift-R), far superoir refactoring, incremental builds, Organize Imports,  free plug-ins (I know VS has them but the second they get usable the author normally goes to a paid version). 

    You can program for hours in Eclipse without taking your hands of the keyboard or taking a trip through the Solution Explorer.

    Best of all its free, most of its plug-ins are free, and its got an open community that you can join to make it better. 



  • @AJAXdrivenBuzzwords said:

    Having worked in both Eclipse and Visual Studio I have to agree with OP that Eclipse is better.  Move lines up and down without copying and pasting, Open Type (ctrl-shift-T), Open Resource (ctrl-shift-R), far superoir refactoring, incremental builds, Organize Imports,  free plug-ins (I know VS has them but the second they get usable the author normally goes to a paid version). 

    You can program for hours in Eclipse without taking your hands of the keyboard or taking a trip through the Solution Explorer.

    Best of all its free, most of its plug-ins are free, and its got an open community that you can join to make it better. 

     

    Don't forget: being able to modify code while in debug mode.



  • Instead of going off on a pet peeve of my own (subtle hint!), I will say that nobody here probably understands your pain. I learnt TCL/Tk once, for fun, because of its platform independent GUI capabilities, but the language is an atrocity. Everything is a string (including statements in a program). I abandonded it on a small-sized project, but 300k lines TCL ... that's like telling someone your teacher made you count to infinity. And back. The mind cannot grasp such abominations.

    And the database... I suddenly like Oracle a lot better.

    Whatever the others say, yours is a Real WTF.

    PS You should try emacs (muhahahaha).



  • I dunno. My coworker in a cube facing mine came from Mentor where he wrote Tcl/Tk pretty much full time. From what I understand, the GUIs of most of Mentor's EDA products are almost entirely in Tcl/Tk, and they aren't small pieces of code.

     Of course, we're talking about Mentor, the company that tried to switch their entire codebase to C++ back before any of it was standardized and almost sank beneath the waves for it.



  • @Heron said:

    (I don't use emacs either, so don't start a vim-emacs flamewar with me.)

    No one needs that fancy-schmancy vim, or even ed. Echo and cat are all you ever need to edit text files. Or office files. Hell, I even do image processing this way, and get far better results than those idiot kids with PhotoShop.



  • @smxlong said:

    the GUIs of most of Mentor's EDA products are almost entirely in Tcl/Tk, and they aren't small pieces of code
     

    That may be true, but I think the way that Tk has been implemented where I work is probably a bit different to the way that most others do it. One fine example is the information terminals that we provide: the entire GUI is pre-rendered when the terminal first boots and then stored as a stack of pages so that the program can flip to the relevant page when it is requested by the user. The entire pre-rendering process can take up to ten minutes, so testing changes to the UI can be a very slow process. Furthermore, the particular framework that we are building our GUI on doesn't allow much more than buttons, text and images, so you can forget about ever trying to stay up to date with the latest trends like animation and other special effects.


  • :belt_onion:

    @amischiefr said:

    @AJAXdrivenBuzzwords said:

    Having worked in both Eclipse and Visual Studio I have to agree with OP that Eclipse is better.  Move lines up and down without copying and pasting, Open Type (ctrl-shift-T), Open Resource (ctrl-shift-R), far superoir refactoring, incremental builds, Organize Imports,  free plug-ins (I know VS has them but the second they get usable the author normally goes to a paid version). 

    You can program for hours in Eclipse without taking your hands of the keyboard or taking a trip through the Solution Explorer.

    Best of all its free, most of its plug-ins are free, and its got an open community that you can join to make it better. 

     

    Don't forget: being able to modify code while in debug mode.

    Did you mean: "as opposed to the "Edit & Continue" support that has been part of Visual Studio since VB6"?



  • @rararargh said:

    -TCL being used to write the entire system

    @rararargh said:
    3.  The "Database"
    By far the biggest problem of this system was the database. During the design process, one of the consultants decided that regular SQL-type databases were too slow and unwieldy so he created his own, from scratch, in TCL.

    Last I knew, TCL was strictly interpreted.  I seem to recall the authors of TCL were among the die-hards of the "strict interpretation" crowd, so I suspect that has not changed.  As such, these two quotes seem wildly incongruous to me: All regular SQL-type databases are implemented in multiple languages, and many of them are low-level compiled.  Some of them can return really simple queries faster than TCL can parse and execute a single moderately complicated line.  I cannot fathom anyone actually thinking they can write a database in TCL that will run faster than the standard available databases.  Unless, of course, they made a poor choice in their standard available database.

    In any event, it does not sound like the goal was achieved, and it also sounds like the system has major privacy concerns as well - concerns great enough that, depending upon the country this resides in, it's possible it may result in jail time for the people responsible - and that includes management.  (Of course, different countries are different, and IANAL.)



  • @tgape said:

    I cannot fathom anyone actually thinking they can write a database in TCL that will run faster than the standard available databases.

    And neither could any of the other developers, but the way things were decided in the design process was based on who could yell the loudest and be the most obstinate about getting their own way.

    @tgape said:

    depending upon the country this resides in

    @rararargh said:
    G'day folks

    And I'll give you three guesses as to which country that might be :P



  • @rararargh said:

    @tgape said:
    depending upon the country this resides in

    @rararargh said:
    G'day folks

    And I'll give you three guesses as to which country that might be :P

    Yes, but I know nothing of the laws of that country, except for the fact that they're probably mostly written in English.



  • @tgape said:

    Yes, but I know nothing of the laws of that country, except for the fact that they're probably mostly written in English.

    (para-quoting something I read once here)

    "It's not English. It's legalese, a completely different language which looks confusingly similiar."



  • @derula said:

    No one needs that fancy-schmancy vim, or even ed. Echo and cat are [b]is[/b] all you ever need to edit text files. Or office files. Hell, I even do image processing this way, and get far better results than those idiot kids with PhotoShop.

    FTFY


Log in to reply