In-house Scripting language - WTF



  • I didn't want to hijack the other (already long) thread, but this needed a response because I can pile WTF on top of the basic ground-level WTF.  In the other thread, Blakeyrat asks this:

    "What the... where does this come from? Who uses an in-house scripting language in 2012?"

    One scenario are those who developed the script in the 1980's, sold it as part of the software package to hundreds of customers, and then had to maintain it, and as technology improved they had to make it work with the newer stuff.  Let me elaborate:

    The base product was a mixture of Cobol and assembler that ran on DEC Vaxes, then later Alphas, and ultimately Itanium.  That sets the stage.  The scripting language was developed a few years after the personal computer became popular in the business world, about the time when businesses wanted to ditch the stupid old VT terminals and use Windows client/server apps.  We (I wasn't actually working there yet, but I'll just use team language to make it easy to describe) developed the scripting language first as a way to provide a Windows front end app that communicated Via DECNET to the Vax.  It emulated the character cell UI of dumb terminals and was horrible and barely worked.  When Windows 95 became a success, we improved the script to support a GUI and talk to the Vaxen over TCPIP.  From that point on, we generally improved the networking performance and UI.  It's still in use to the best of my knowledge.

    Here's the second layer of WTF:  As best I can figure (with all records lost to time) they decided to go that route instead of using something like Visual Basic because the customer base didn't want to learn a programming language or hire programmers.   So at first, they tried to make the scripting language easy for non-techies to work with.  That fell by the wayside quicker than you can say "we need more functionality".  The few customers who tried to work with it scared the rest (there was a national user group), so we ended up always having to support it for everybody.  At a price of course, which worked well for us.  There was no standardized upgrade and release path, enhancements were completed only for customers who asked.  There was one guy - the original author - who was able (or allowed) to make changes to the script language, and it was controlled by distributing only the .exe for the script parser.  Nobody else had access to the parser's source code.

    But wait, it gets better.  He had crappy version control, so we were always tripping over bugs and when he fixed bugs in the parser he didn't send an updated exe to everybody.  The only way to know what version your .exe was, was to run it and type a version command.  If you didn't have the right type of script to go with that version of the parser, it would puke before you could even type a command, which meant you generally had to have several "stub" scripts laying around that you could test the .exe with.  He had no QA, but would tweak things and send us an .exe to test and see if he fixed what we complained about. 

    He maintained a document that explained the script functions, but it was in .txt format and was the typical engineer stream of consciousness format.  At one point I massaged the thing into MS Word format with a table of contents so that you could at least search and find things, but he would only ever update his .txt version.

    So over a 20 year evolution of this nightmare, one net result is this:  one guy who learned quite a bit about Windows programming, and a cadre of developers (us, not the customers) who gained 10+ years of experience programming a custom script language not used in any other company.

    So.... how many WTF's have you guys been able to count?  



  • Ok, so the issue is "how to deprecate a custom scripting language that seemed a good idea at the time".

    Well the obvious way is to start by writing the new way of doing things, and all new work is done this way. The advantage of a scripting language over a high-level language is mostly that it is quicker to make changes because it doesn't have to go through the build cycle. And not that it is easier to program in.

    When, in 1997, I worked for a company called IMA (who are now bust so they can't sue) they used their own proprietary scripting language called EDGE. This was developed within their own GUI application. You then compiled your scripts and ran them in a different GUI called GEO. If you had a team developing and one of them was testing in GEO, it restricted others from committing their scripts (I think) or it might have just been compiling... but I do recall there was a restriction. I did think it was a WTF'y way to develop at the time and wondered why they didn't just write DLLs (or shared objects for UNIX) then develop the client apps in Visual C++ or even Visual Basic, which were the popular tools at the time. However bad you think VB might have been, it wasn't as bad as using their own tool. My main role during that year was working on side products. In particular they wanted computer-telephony integration and I got a system working to integrate with the API of a telephony switch that could be integrated into EDGE as well. Their product also allowed DDE to integrate so I got their scripts working with some other products. DDE was rather dated even in 1997, with OLE Automation being more practical, and ActiveX was just starting to be promoted as the new way to embed controls.

    When, in 1999, I wrote an inhouse scripting language called XER it was a different situation entirely in that the purpose of the scripts was to generally edit to perform the job in hand rather than build into some released system. I was only there for 6 months but I know the system was used for about 10 years, i.e. was still used in 2009. It was really a bit like SQL with a few extras and a variety of output formats. Actually, it was intended originally to just replace an old system and be used in one place, but being the way I am, I often see a bigger scope for my products and for once in my career my manager agreed with my attitude and the product spread all over the company as other departments heard about it and said they could use such a system.

    That I was there for just 6 months was pretty much the policy of contracting in companies like that at the time - you brought in an "expert" to start-up a system then had your permies extend and maintain it. I left before I could get bored and found another contract immediately. Unfortunately they didn't have the same attitude towards me as my previous one, paid me 10% more than the previous job had but treated me as a junior and an idiot and when the senior boss, who had changed 3 times, told me they thought it was a good idea to terminate, my immediate response was that was the first good suggestion I had heard from them for sometime.

    I was one of those developers at the investment bank I referred to in the other topic, between September 2007 and November 2008... One thing you learnt was that you can't change the habits of the people using your scripting language. They will write their scripts in the intuitive way and you have to make that, if not the best way, at least a reasonable way. You don't know what the scripter will do next, but you can guess, e.g. if they access a sequence with s[0] you can pretty much guess they are going to iterate across the entire sequence using random access, so you had better make sure random access is constant time. Even if it wasn't for the first call, you can change your representation (without telling them) to make it be so. These were things I pointed out to the person who had written the system before me, and as I result I got through some huge improvements which were not always noticed because they weren't all new features / functions.

    For many of the other jobs, where I have had a say, I prefer an IOC (inversion of control) method of programming. This involves writing a "config" which can begin to look a bit like a scripting language. In essence you write libraries, then in the script, objects are created from the classes within the library using the "builder" design pattern. Some of these objects are "runnable" and the application is a two-phase process. The first is the "load" phase where it constructs the model from the config, and the second phase is running it. Normally I also give an option to "test" the load by allowing you to specify a load-phase only.

    Within the config, there is a concept of a library, a class, an object, a list (sequential container) and a map (associative container). Some runnables have containers of other runnables and there are both serial and parallel versions of these.

    However there are severe limitations which does not make it into a scripting language as such, even if it can start to look a bit like one.

    Firstly, there is not normally an sequence flow based on conditions (if, while...).

    I said there is a concept of classes but you can't create your own, you only set up your own alias for a class defined in the code specifying what builder you are loading from which library. You then create objects of this class specifying the parameters it requires, which may also be objects. (An object may be a parameter to more than one other object and you cannot specify circular references, so A cannot compile with B and B with A).

    What distinguishes this, and XER, from some of the other cases is that there is no standard scripting language way of doing either of these, and both require a scripting language if you wish to be able to dynamically change without a rebuild. IOC works specifically well with automated tests, where you do remain very flexible. (The only bit of WTF'ery I once put in was getting my tests to output "Quack quack oops!!" if they failed along with the error it encountered, of course...British readers old enough to remember Dave Lee Travis would understand the significance).

     



  • There's nothing wrong with developing one's own domain-specific language. It's something that seems like a WTF to the poorly educated. People who have actually taken classes in language theory mostly know better.



  • @bridget99 said:

    There's nothing wrong with developing one's own domain-specific language. It's something that seems like a WTF to the poorly educated. People who have actually taken classes in language theory mostly know better.

    Every time I read a bridget99 comment, I find myself dazed and confused. Then I realize who wrote it.



  • @Ben L. said:

    Every time I read a bridget99 comment, I find myself dazed and confused.
     

    You're not alone.


Log in to reply