What happens when data analysts design a Scripting language
-
I am currently working in a job where producing huge sets of nicely formatted Tables[1] is my day to day job. It is somewhat boring, but I am learning programming on the side so its overall ok.
My only real gripe is the "table language" we are using.
Generally the process looks like this:
data comes to me in a SPSS Datafile (think of a statiscial analysis package)
I am writing a script in the table language
The tool writes this data to a nicely formatted pdf or excel file.
Unfortunately the designers of the Table language are terribly bad a designing languages:- there are "macros" to reduce the amount of script you have to write. Unfortuately, any error in the macro error is simply announced as "macro error" - very helpfull when your macro contains 500+ lines. You can look at the finished macro - but it is written to a single linve of text which can become quite long (10.000 characters....).
- Macros can handle "variables" unfortunately, you have to be careful with the names. If you have a variable called "Top" and another one called "TopMost" the substitution process will replace all places where TopMost is called with Top. If I called the developer he was really suprised that not every language works like this.
- There is no concept of arrays, making it impossible to avoid huge copy & paste orgies, because without arrays, the macros are not really useful.
- Much worse is the fact, that there are builtin variables like the Tablename, which can be written, but not read - again crippling macros because you cannot append text to the end of the Tablename, which would be great in cases there a multiple tables, where only part of the Name changes.
- The PDF files are not directly created, but are written as .ps files.
- and: The damned thing is ancient and writes its excel tables via com.interop calls directly to excel. Which creates all sorts of fun of you want to work with your computer while it creates tables. And it is horrible slow. They recently ported this behaviour to OpenOffice, so my suggestion of writing xlsx files directly was obviously lost on them.
[1] [mod - but not nicely formatted forum posts it seems? Brick-text tidied up with <br /> markup. May I suggest the use of the preview tab until you get used to the foibles of the forum - if ever. PJH]
- there are "macros" to reduce the amount of script you have to write. Unfortuately, any error in the macro error is simply announced as "macro error" - very helpfull when your macro contains 500+ lines. You can look at the finished macro - but it is written to a single linve of text which can become quite long (10.000 characters....).
-
Could you perhaps ignore the thing's inbuilt macro facility and instead use a real macro processor for your script volume reduction needs? Or abandon macros altogether and use a real scripting language like JS or Python or Perl to emit automated and nicely parameterized copy and paste orgies in table language? Or ignore the whole clusterfuck entirely and use a tool that works instead?
-
@HerrDerSchatten said:
If you have a variable called "Top" and another one called "TopMost" the substitution process will replace all places where TopMost is called with Top.
I cannot divine the meaning of this structure of words. What substitution process?
-
When are people going to learn saving time and money, by choosing not to learn an existing language/methodology to achieve your goals, and instead inventing your own language/methodology, doesn't actually work.
The only effects are to increase the training time of new hires, increase the rate at which talented developers leave your company, further embed comparatively untalented (but trained) staff, and decrease the chance you can make your technology work with anything else (and thus increasing the likelihood that further decisions will be made to extend the monolithic in-house monster to include some missing feature, instead of using something cheaper that already does what you want).
-
Ok, I try to explain it:
Imagine you have to following code
#macro #Test (&Var &VarSecond)
compute variableTest1 = &Var;
compute variableTest2 = &VarSecond;
#endmacro
If executed, the macro should grab the values from the two variables and compute two new variables.
But, when invoked, you get this:
#macro #Test (&TheActualVariable &TheActualVariable2)
compute variableTest1 = &TheActualVariable;
compute variableTest2 = &TheActualVariableSecond;
#endmacro
The problem occurs in the Last line: It blissfully applies a simple string substitution and replaces all "&Var" occurences - even if there are other Variables with "&Varxxxxxx".
[mod - last time I'm going to do this - PJH]
-
He buttumed. And when you buttume, you make an butt out of U and me.
-
@HerrDerSchatten said:
Ok, I try to explain it:
On the basis of your first two posts, and since you can't seem to be bothered checking your own posts after, let alone during, composing them, I'll just let the others complain at you publicly the next few times until you get it.
Cleaning up twice after you is, I believe, sufficient:
Before picture:
-
Can you explain me why that post looked so bad?
In the editor, it look quite normal and I hit submit before going to lunch, sorry
-
The bad thing about this language is, that there a few alternatives.
There is the SPSS Table module - but its workflow ist actually far worse and it's syntax is next to impossible to write manually.
There are other software packages like WinCross - but I have never used them, so I cannot tell if they are better.
Otherwise: I have a feeling that the entire business of table software will going down in the near future. At the moment several companies are deploying much nicer "online reports" (tables and finished charts), which are superior to the current tools.
-
@HerrDerSchatten said:
Can you explain me why that post looked so bad?
In the editor, it look quite normal and I hit submit before going to lunch, sorryYou're probably using Chrome, at which point the rich editor doesn't work. Or you weren't using the rich editor in some other browser. At which point, you have to use html to get paragraphs, etc.
The preview tab is your readers' friend.
-
Honestly: I would wish to abandon all this piece of shit - but it is not possible: Several ongoing projects have 100,000 lines of code. Also, the Language does a good job of computing and displaying statistical data - it would be a nightmare to graft it on another language.
-
@eViLegion said:
When are people going to learn saving time and money, by choosing not to learn an existing language/methodology to achieve your goals, and instead inventing your own language/methodology, doesn't actually work.
It's a toss-up between "never" and "just before the Rapture".
-
@HerrDerSchatten said:
Honestly: I would wish to abandon all this piece of shit - but it is not possible: Several ongoing projects have 100,000 lines of code. Also, the Language does a good job of computing and displaying statistical data - it would be a nightmare to graft it on another language.
Port your whole shit to Tableau in a couple weeks, show it to your boss, promotion.
