Never, Ever Use Real Life Data For Testing



  • The text book web site we wrote for our customer was selling fairly well for them.  They were quite pleased with the performance, but more importantly they were really pleased with some of the extra bits we put in to solve customer problems.  Their calls had been reduced more than 50% with the new release, and the new tools we gave the customer service people reduced solve times considerably as well.  Of course, no system is perfect, and customer service finally got a call they couldn’t solve.

    One of their new schools couldn’t use their activation code to access the site. It kept telling them that the school was already a customer.  But there was no related data or users according to the customer service representative who contacted us. A quick database check confirmed that.  Since the school was but a pilot for city with the possibility of the other schools buying in later, our customer’s sales person (who mainly got paid via commission) asked us to escalate the problem to a high priority level. We were called into the project manager’s office to discuss the problem and delegate.  The problem was explained, a few possible explanations were discussed, and eventually a plan put in place to figure out the problem.  The project manager sent the school information via email and we hustled back to find out the problem.

    Anytime a school designated as “new” was signed up, a special activation code is given to the customer. Once a user is added the school loses its “new” status and only the first user can add additional users.  This was done deliberately to avoid multiple purchases of the same product at the same school, and it worked for the most part.  Here the problem was that a user already existed for the school, one that the customer service representatives were not allowed to view due to privacy concerns.

    As soon as I saw the school in the email, I knew what the problem was.  I had entered my son, attached to his real life school, as a test user.  Technically, my son was in charge of the entire school’s accounts.  Since he was a student, customer service could not see him.  I removed my son’s account and the activation code worked fine after that.   After a lengthy explanation to the customer, I personally called the school’s district administrative offices and explained the situation to them as well so they would realize they were not using a buggy product.  

    Of course, schools being overreactive, my wife soon got a call that she needed to come in. She got to the school to find my very scared son in the principal’s office for “hacking.”  Of course, the principal was as confused as to exactly why my son was in his office. Especially since my son was SIX at the time.  Apparently an administrator at the district’s office had interpreted my explanation into my son hacking into the system, but never bothered to check his age before reporting him.  While the principal was trying to figure out why one of his first-graders was being accused of hacking (he actually said to my wife at one point, “Hell, if he broke into the school’s system I’m going to put him in charge of it,”) my wife called me to let me know what was going on, and fortunately it didn’t take long to put the matter to rest after that.  

    We now use a non-existent school with made up student names for all testing purposes. The entire school district now uses the product. And my son (who is now 9) loves to remind me about the only time he got sent to the principal's office was my fault.



  • Ah, if I had a nickel. What is it with school IT staff and hacker paranoia? Do anything besides use MS Word and the provided web browser and they're immediately convinced you're hacking something.



  • Great story.  Probably front-page-worthy; did you submit this to Alex?

    Reminds me of the story about Denis Leary having to meet with his son's teacher because his son was singing his (Denis') song "Asshole" in class.  



  • Should have just let your son take the fall for you.

     

    Seriously, though, why even mention him at all?  The error was your fault and adding in the details about your son was likely to confuse the issue.  Tell the customer what they need to hear and exclude unnecessary details whenever possible.



  • @cconroy said:

    Great story.  Probably front-page-worthy; did you submit this to Alex?
    What ever happened to Best of The Sidebar? No posts good enough?



  •  I remember writing a 'batch' file for a VAX system to help me out whilst in college. It was very simple, but it worked by bypassing the usual user interface and doing stuff without my input, since I got bored of pressing the same keys all the time. I had to manually run it for it to do anything, so its not like it would automatically do something. However, I got pulled into an office with the head of the college when someone from IT noticed the script. They had printed it out, circled the entire script, and wrote "V. Naughty!!!" next to it. When I explained to them so I couldn't upset any of the system files or damage the network because I only had basic priviledges and no physical access to anything apart from a Wyse terminal, they shrugged it of and cautioned me for doing it anyway as apparently I was only supposed to be using commands given to me in writing. 

    So much for going to college to actually try and learn something. 

    However, your story is much better, considering the kid was probably more confused than the principal about what he had done. But yes, why not just "Sorry, I created an account already at your school, I'll just delete it, then it'll work as normal" rather than telling them about your kid? 



  •  But careful, don't use overly fake data either. I remember at one job I put "Jenny Burro Donkey" as one of the customers in my development database, as well as other similar juvenile and silly, but totally PG names. Well, during a demo to the board of our Fortune 500 company, our technical team lead apparently wanted to show new features which has not yet been migrated to test to the manager doing the demo, and so he sent him the link to my machine to look at.... and they accidentaly ended up using that link for the formal presentation.

    Apparently, the result of the demo was some company bigwig saying, fire whoever is responsible for the ass jokes.

    Well, I didn't know for quite some time how very, very close I was to being fired that day. Apparenly it took quite a few folks standing up for me, and finally my project manager threatening to quit if they fired me to save my job.

     



  •  So TRWTF is that you were either a.) testing in production, or b.) migrated your test data to the production database? 

     

    Using real life data for testing is fine, the question is: why the fuck did your test data make it into the production database?



  •  @amischiefr said:

    Using real life data for testing is fine, the question is: why the fuck did your test data make it into the production database?  

     

    I think you mean 'how' rather than 'why' and it's probably because they needed to 'test' the production database to make sure it worked just like the development database does ;)

    I've seen over zealous testers do this kind of thing before. "Yes it worked on the development database, but how can we be 100% sure it'll work on the production database?".



  • @kraagenskul said:

    After a lengthy explanation to the customer, I personally called the school’s district administrative offices and explained the situation to them

    Ta-dah.. the Real WTF(tm). What the HELL were you thinking?



  • The US Census provides a list of female & male first names, as well as a list of the common family names from the 2000 census, ranked by probability.



    Whenever I need to make a fake person up for testing, I pick a random first name (of a random gender!) and a random last name.

    There is a chance it's a real person of course, but I can justify my technique if anyone ever raises a fuss. But usually you end up with some strange combinations, like Heriberto Waggy or Takisha Xaimoungkhoun



    http://www.census.gov/genealogy/names/names_files.html



  • ♿ (Parody)

    @Zecc said:

    @cconroy said:
    Great story.  Probably front-page-worthy; did you submit this to Alex?
    What ever happened to Best of The Sidebar? No posts good enough?

    Great story, indeed -- if anything qualifies for Best of, this certainly does!

    And I have no idea what happened to Best of... I think I gave up after a few weeks of "eh material" and forgot about it.



  • @anothercontractor said:

    There is a chance it's a real person of course, but I can justify my technique if anyone ever raises a fuss. But usually you end up with some strange combinations, like Heriberto Waggy or Takisha Xaimoungkhoun
    How did you get my name?!



  • @anothercontractor said:

    The US Census provides a list of female & male first names, as well as a list of the common family names from the 2000 census, ranked by probability.

    Whenever I need to make a fake person up for testing, I pick a random first name (of a random gender!) and a random last name.
    There is a chance it's a real person of course, but I can justify my technique if anyone ever raises a fuss. But usually you end up with some strange combinations, like Heriberto Waggy or Takisha Xaimoungkhoun

    http://www.census.gov/genealogy/names/names_files.html

    Note to self: First useful thing ever read in the Side Bar.



  • Yes., I know I screwed up by mentioning my son's name to the school administrative office, but I did it to help out our customer's sales guy; he was worried it was going to screw up his sale.   I never in a million years would have guessed it could be interperted as my son (again, SIX years old) hacking.

     As for the test data, it was something I entered to A) make sure everything work, and  B) have an account I could log into if the customer complained about something being broken. I actually don't see how you cannot have some sort of test data even on a production system, especially one with privacy concerns. We also are not technically allowed access to the student's information.   It's just making sure the test data is isolated and fake is the lesson I learned.



  • @kraagenskul said:

    She got to the school to find my very scared son in the principal’s office for “hacking.”  Of course, the principal was as confused as to exactly why my son was in his office. Especially since my son was SIX at the time
    Heh,

    crackers... gotta catch them while there's still time. Otherwise they start playing games with nuclear-arsenal-controlling computers and all hell breaks loose.



  • @DOA said:

    Otherwise they start playing games with nuclear-arsenal-controlling computers and all hell breaks loose.
     

    Ooh! Global Thermonuclear War! I'll have a bit of that


Log in to reply