How to import a SPSS Database into our Web application
-
Where I worked, we used SPSS for data handling and presentation, which is a shitty Java Tool in itself but gets the job done.
However, our management decided to build a new web application for data handlung and presentation, because we did a lot of our work online and our customers could view our results much more easily.
I was hired around that time and was a little bit shocked to find out, that there where no specifications, no one talked to our team about this and there where only one developer who worked on this project.
My job was two fold: work in the old "SPSS department" and do normal day to day work there AND create a list of requirements from that work, which is then fed to the developer.
I did as I was told and produced a huge list of requirements which where non negotiable, because our team needed theese functions.
However, when I presented this list to our technical management, the list was truncated to 1/5 of the items, on the grounds of "we did not need these functions which the new shiny tool!" or "these functions are seldom used, so we can ignore them!".
Then this truncated web application was put to work and I should show my coworkers how to use it - there was huge resentment because they could not finish their work due to the lack of features. Especially the "seldom used" functions where needed in practically every project - maybe not every function, but every project contained one seldem used function.
A new meeting was held an the technical management tried to blame me, because i did not find out the requirments - when I showed him the List of the last meeting, he shut up. I also warned them, that the application overall was too slow and cumbersome to be extended as needed. Then our CEO stepped in a blissfully declared "hey, that is easy stuff: only bivariate statistics. What can be so hard?!? - simply do it and shut up" - so we went forth and our programmer tried to add the features, but was totally overwhelmed by the number of features and their complex interactions.
From the beginning, I was pushing for a method to import SPSS data into this application, so that shortcomings of the application could be circumvented by doing the work in SPSS - this was finally approved but the way it was implemented was hilarious:
The application itself is a php script so our developer called R (a statistical package) from php, which opened the datafile and presented tables of the datafile which where then parsed by php.
This worked, sort of - but was slow as hell and error prone.
He could have used the SPSS c files, SPSS python support or a dozen other methods. Heck, SPSS can output data to a database if someone would have asked!
So the tool drained our development resources, and another project of mine could not be finished, because I was forbidden to use SPSS for it and the company took a dive.
So I was let go due to financial concerns and seeked greener pastures elsewhere.
But: last week they called in again: They want to use my other project, which was 99% finished. But because the web application is still unusuable they want to use SPSS for it. They asked if I could finish it - they accepted a rate which is twice my original rate....
-
All this talk of SPSS and Java threw me for a moment, until I remembered that I haven't needed to keep up... SPSS is nearly as old as me, and I have colleagues young enough to be my children and old enough to not be students anymore. SPSS certainly wasn't in Java when it was first released, in 1968...
How time flies when you're having fun...
-
@HerrDerSchatten said:
they accepted a rate which is twice my original rate....
Then you should have asked for four times and they would probably have accepted three times as a compromise...
-
@HerrDerSchatten said:
The application itself is a php script so our developer called R (a statistical package) from php, which opened the datafile and presented tables of the datafile which where then parsed by php.
This worked, sort of - but was slow as hell and error prone.
Were you guys aware of Rserve? We use it quite a lot, and its rather popular with our users.
@HerrDerSchatten said:But: last week they called in again: They want to use my other project, which was 99% finished. But because the web application is still unusuable they want to use SPSS for it. They asked if I could finish it - they accepted a rate which is twice my original rate....
At least there's a happy ending! <FX:hugs>
-
They switched around 3-4 years ago and it was a relief, sort of:
Windows can now CHANGE SIZE - useful when your variable name is longer then 4 characters.
There a fewer gross violations of the windows UI.
On the other hand: I am working with SPSS 19 and it is slow as hell and while the script editor now offers syntax highlithting, you can watch while it paints the characters (fun if your syntax file is bigger than ~500 lines)
.
Also: some things like "right click a variable and do a frequency via pop up menu" will probably never be implemented.
-
No, we where not awars of that. What does it? I will google, but I am interested in an insiede view.
Basically: We did everything for ourselves, which was a recipe for wasted time...
-
@HerrDerSchatten said:
But: last week they called in again: They want to use my other project, which was 99% finished. But because the web application is still unusuable they want to use SPSS for it. They asked if I could finish it - they accepted a rate which is twice my original rate....They are in trouble..so expect it to be difficult to get them to pay...invoice often, get the invoices signed by the manager you are working for, make sure you can take them to Small Claims Court (frequent invoices means amounts under the Small Claims limit).
