How often do you write scripts?

archivator

Yesterday I had to move one cell phone's phonebook to another brand new phone. First, let me assure that LG do not make it simple to download their PC sync software. How does removing the battery and entering the IMEI in a proprietary support application sound? Come on, it's not like I don't know what the phone model is.

Anyway, I find myself using this half-baked PC Sync application which is little more than barely functional. Of course, its sole export format is CSV. And they didn't even get that one right. Every single field including the phone numbers had quotes surrounding it. Naturally, the Samsung software I had to import it in could not deal with that.

Being the inpatient guy that I am, instead of searching for CSV parsing options in Excel, I threw together exactly 11 lines of Python code to strip the quotes (as I knew there weren't any quotes in anyone's name or email address) and gave the Samsung software a usable file.

That brings me to my question, how often do you find yourself writing such small scripts to use with desktop applications? I'm mostly referring to non-IT tasks.

morbiuswilters

I find myself frequently writing scripts to "bridge" CLI apps. I use Linux and even then only 2 GUI apps, so there aren't many opportunities to write scripts for desktop software. I suppose GreaseMonkey scripts also count. I've accumulated many of these to automate certain tasks or enhance functionality of web apps I use.

Oh, and FYI, double-quoting all fields in CSV is valid. I believe this is the default behavior for Excel, too. So the problem there wasn't the LG app, but the Samsung one for not recognizing proper CSV.

archivator

@morbiuswilters said:

Oh, and FYI, double-quoting all fields in CSV is valid. I believe this is the default behavior for Excel, too. So the problem there wasn't the LG app, but the Samsung one for not recognizing proper CSV.

Apparently, you're right. From RFC 4180:

field = (escaped / non-escaped)
escaped = DQUOTE *(TEXTDATA / COMMA / CR / LF / 2DQUOTE) DQUOTE
non-escaped = *TEXTDATA

Nevertheless, double quotting every field is simply being lazy and, as experience shows, might lead to confusion. Oh, and the RFC mentions that Excel does not use quotes at all (don't know what version they're referring to). Follow the leader, right?

Anyway, I'm just glad I'm not buying a phone anytime soon.

morbiuswilters

@archivator said:

Nevertheless, double quotting every field is simply being lazy...

To be fair, if people weren't so lazy we wouldn't have computers to begin with. Meanwhile, almost every advancement in computing has come from someone wanting to make something easier or quicker, so they have more time for recreation. I understand your point, though, I just like pointing out that laziness can be a virtue.

chebrock

@morbiuswilters said:

I understand your point, though, I just like pointing out that laziness can be a virtue.

I'm working on a little CRM tool for a client, it has two different types of contacts. I ended up writing a fair share of conditionals if it we're one type or the other, which generally is lame, all the while thinking to myself that the code would be alot nicer if I had been forced to consider 20 types, and had to be polymorphic/generic, instead of the testing of one type or another. Basically my application would have been written better if I was forced to be more lazy.

Why write a script for this? Why not open it in an editor and do a Find-And-Replace?

danixdefcon5

@archivator said:

How does removing the battery and entering the IMEI in a proprietary support application sound?

*#06# is your friend. That'll give you the IMEI in any GSM-compliant cellphone.

Anyway, on the topic, my previous job sometimes included analyzing logs, which meant I had to parse through 400Mb log files and extract information on these. Our boss was an old-school UNIX man, so he did everything with obscure ksh scripts that made us dizzy just trying to understand what they did.

My solution was simpler: I just ran the logs through sed/awk, outputting a CSV-compliant document, and then imported that into PostgreSQL. Once inside a DBMS, doing stuff like "give me how many HTTP 500 errors we got in the last 45 minutes, grouped by remote IP address" becomes a simple SQL query.

But yeah, most of the time, if I'm presented with some kind of text file, the odds are that I'm going to go down the script route; though for processing data in text files, I'll usually jump to Java, or just do the CSV/import thing if its simple enough.

danixdefcon5

@Salami said:

Why write a script for this? Why not open it in an editor and do a Find-And-Replace?

Using vi:

:s/\"//g

Using scripts:

sed quoted.csv -e "s/\"//g" > unquoted.csv

Depends on how big your text file is. If its large enough, you might find your editor locking up/dying, or eating all resources just trying to load the file. If it can be done with vi, then you can do it with sed as well, and save yourself the Find-and-Replace process.

Lingerance

@danixdefcon5 said:

My solution was simpler: I just ran the logs through sed/awk, outputting a CSV-compliant document, and then imported that into PostgreSQL. Once inside a DBMS, doing stuff like "give me how many HTTP 500 errors we got in the last 45 minutes, grouped by remote IP address" becomes a simple SQL query.

AFAIK most *nix httpd's can use syslog for logging, so this would have been simpler: syslog (-ng) to postgreSQL database howto (presuming you needed to do checks frequently, if not then yes, your solution is simpler).

bjolling

@archivator said:

That brings me to my question, how often do you find yourself writing such small scripts to use with desktop applications? I'm mostly referring to non-IT tasks.

I usually write small tools in .NET instead of creating scripts, just to have that little more functionality. My wife came home the other day with a PDF, extracted from her company's time-tracking application. No decent formatting whatsoever so I had to create a small state machine just to group related lines and output them to CSV for further processing in Excel.

I then proposed to her boss to rewrite their time-tracking system (writte in Borland Powerbuilder 16-bit) for a small fee, but he declined.