Lazy Bastards



  • So I come in to the office this morning and find an email waiting for me. It seems that we received an insurance application yesterday containing a character that the Sql Server XML parser doesn't like (a '~'). Because of this, the application couldn't make it through the submit process and the agent was rightfully pissed off. The workaround is simple; point a debugger at production and manually submit the policy by removing the offending character from the in-memory XML object (count the WTFs in that statement). Visual Studio makes it incredibly easy to do what I just described. I would expect that anyone who dares call himself a .NET dev could work it out.




    It turns out that the problem was identified about 10 minutes after I left the office yesterday. Instead of just fixing it, four people got together and decided that they had no idea how to do so. I've had to do it several times in the past, so they wrote me the aforementioned email and went about their usual WTFery. Today, as I'm in the middle of fixing this, the furious agent calls up and says that the customers decided to abandon the policy because they were tired of waiting on our buggy system. Great job assholes. Had they just done their jobs, we probably wouldn't have lost the business.




    </rant>



  •  Out of curiosity, are those four code monkeys developers overpaid, or are they paid in peanuts? One case could worsen the WTF, the other, while not being an excuse to their lack of ability, could help explain why it happened.



  • [quote user="Renan "C#" Sousa"]Out of curiosity, are those four code monkeys developers overpaid, or are they paid in peanuts?[/quote]

    At this point, I'd say that paying them in peanuts would still be overpaying them. One of the four is our resident PMP. The other three are experienced devs.



  • Sounds to me like your XML parser is shit and your process is shit.

    Maybe now that it's been proven that the shittiness leads to lost revenue, there will be some motivation to make improvements.



  • @Rootbeer said:

    Sounds to me like your XML parser is shit and your process is shit.

    Maybe now that it's been proven that the shittiness leads to lost revenue, there will be some motivation to make improvements.

    I'm long since finished hoping for permission to make improvements. That way lies madness.



  • @Smitty said:

    It seems that we received an insurance application yesterday containing a character that the Sql Server XML parser doesn't like (a '~').

    ... wait wait wait, what?

    @Smitty said:

    a character that the Sql Server XML parser doesn't like (a '~').

    ... but, wha? XML, what the wha bu (incoherent gibberish)

    @Smitty said:

    The workaround is simple; point a debugger at production and manually submit the policy by removing the offending character from the in-memory XML object

    (sobs quietly)



  • @arotenbe said:

    @Smitty said:
    The workaround is simple; point a debugger at production and manually submit the policy by removing the offending character from the in-memory XML object

    What's really awesome about this situation is that these bad characters inevitably arrive in name fields (of which we have only five or so). Why, you might ask, don't we employ some validators and prevent these offending characters up front? A regular expression to allow A-Z (upper and lowercase) doesn't seem too fucking hard to me. Alas, the decision makers can't (won't?) hear the voice of reason, so here I am.



  • I'm with arotenbe.   Maybe it would be less funny if this became the focus of the thread, but I just have to know, is there any particular reason why a tilde breaks the XML parser?



  • @Xyro said:

    I'm with arotenbe.   Maybe it would be less funny if this became the focus of the thread, but I just have to know, is there any particular reason why a tilde breaks the XML parser?

    Also, why is there a ~ character in the data at all? This has happened before, apparently, and I am certain that the salesmen have been told "don't enter stupid punctuation in fields x,y,z".



  • @Xyro said:

    I'm with arotenbe.   Maybe it would be less funny if this became the focus of the thread, but I just have to know, is there any particular reason why a tilde breaks the XML parser?

    If it were at the beginning of a tag, maybe? Tag names have to start with a letter.



  • @Xyro said:

    I'm with arotenbe.   Maybe it would be less funny if this became the focus of the thread, but I just have to know, is there any particular reason why a tilde breaks the XML parser?

    We're currently encoding the XML as UTF-8. Somebody suggested that this could be the problem. Rather than let us investigate, though, the decision makers blame Microsoft and prefer to have an expensive programmer deal with the problem manually. Again and again.



  • @Medezark said:

    Also, why is there a ~ character in the data at all? This has happened before, apparently, and I am certain that the salesmen have been told "don't enter stupid punctuation in fields x,y,z".

    The rant we usually get from the agents is that "my customer's name is saldaña, not saldana. Why can't I type it in correctly?". Probably a valid question.



  • The tilde is a mighty, nobel, and perfectly valid UTF-8 character.  Your company isn't using the field names as XML tags is it?

    <this is="totally fine">~<this>
    <~>this is not</~>



  • Ooooh, so this isn't about a tilde, but about characters outside of ASCII.   That would make more sense.  So the input is Latin-1 or Unicode, but the parser is attempting to validate character data from a different encoding. Characters 127-255 in Latin-1 aren't compatible with UTF-8, so if the parser was making sure no binary nonsense was being fed into the database, it would have rightly barfed.

    Ok, on with the regularly scheduled WTF.



  •  @Smitty said:

    @Medezark said:
    Also, why is there a ~ character in the data at all? This has happened before, apparently, and I am certain that the salesmen have been told "don't enter stupid punctuation in fields x,y,z".

    The rant we usually get from the agents is that "my customer's name is saldaña, not saldana. Why can't I type it in correctly?". Probably a valid question.

    It's more than merely valid if the customer is an upscale eatery called "Los Años Del Oro" (The Golden Years) and your arbitrary restriction means they're getting stuff addessed to "The Golden Assholes" instead.



  • @da Doctah said:

     @Smitty said:

    @Medezark said:
    Also, why is there a ~ character in the data at all? This has happened before, apparently, and I am certain that the salesmen have been told "don't enter stupid punctuation in fields x,y,z".

    The rant we usually get from the agents is that "my customer's name is saldaña, not saldana. Why can't I type it in correctly?". Probably a valid question.

    It's more than merely valid if the customer is an upscale eatery called "Los Años Del Oro" (The Golden Years) and your arbitrary restriction means they're getting stuff addessed to "The Golden Assholes" instead.

    That would mean the salesmen are typing "Los An~os Del Oro" or "Los A~nos Del Oro" for the tilde to show up as a tilde. Which is still not "Los Años Del Oro". As addressed earlier. It's the encoding that's causing the issue. Until the coding is corrected the "dont type non English Characters, regardless, until we can code for non English characters" would still stand.

    This is a complex issue. Ethnic Diversity aside, an argument could be made that within a given country a specific language and it's character sets should be adhered to and considered the "standard". I seem to remember a story about the building of a tower. . . . .



  • @Smitty said:

    @Medezark said:
    Also, why is there a ~ character in the data at all? This has happened before, apparently, and I am certain that the salesmen have been told "don't enter stupid punctuation in fields x,y,z".

    The rant we usually get from the agents is that "my customer's name is saldaña, not saldana. Why can't I type it in correctly?". Probably a valid question.

     

    Well, duh.

    So the problem is not the input, and I can guarantee you the problem is not in SQL server choking on a certain character. It might be that ~ has to be embedded in <![CDATA[ tags (although I don't remember), the main point is that something between input and storage has gone wrong - ie the program. ie the programmers. See where this is going?



  • @Xyro said:

    Ooooh, so this isn't about a tilde, but about characters outside of ASCII.   That would make more sense.  So the input is Latin-1 or Unicode, but the parser is attempting to validate character data from a different encoding. Characters 127-255 in Latin-1 aren't compatible with UTF-8, so if the parser was making sure no binary nonsense was being fed into the database, it would have rightly barfed.

    Ok, on with the regularly scheduled WTF.

     

    The tilde by itself isn't extended ascii, it's character 126.  Of course a character with a tilde like ñ is not representable by a single UTF-8 byte.

     If it's things like ñ that break it, then I agree, it's simply a matter of people using data in your run of the mill non-Unicode character set and pretending like you can just try to interpret the byte sequence as UTF-8.  Then the real WTF is that no encoding of UTF-8 is happening at all; it's only the fact that UTF-8 is a superset of 7-bit ascii that keeps the system working at all.



  • @b-redeker said:

    It might be that ~ has to be embedded in <![CDATA[ tags

    <![CDATA[ only spares you the need to mask <, > and &. Instead, you need to mask ]]>. Other than that, it makes no difference whether you use it or not.



  • To all of you people getting confused about the tilde character: this is not about the tilde character which is totally fine, but instead some other character like escape (27). Microsoft's XML parser does not like it. A solution that I used is to create a wrapper TextReader that filters out the bad characters and use that to create an XmlReader. And from that you can read into a DOM or do whatever.



  • @SlyEcho said:

    To all of you people getting confused about the tilde character: this is not about the tilde character which is totally fine, but instead some other character like escape (27). Microsoft's XML parser does not like it. A solution that I used is to create a wrapper TextReader that filters out the bad characters and use that to create an XmlReader. And from that you can read into a DOM or do whatever.

     

     But the OP specified a ~ character as the culprit.



  • I feel your pain... Last week, we had a finance app submitted via XML. The software it came from is a 3rd party app from a vendor that does just finance broker software, but the broker rang us to complain, rather than going through the developers (the agreement with this particular software company is they're 1st line support for their clients).


    Turns out, the software can't correctly escape an ampersand that was in the customer's employer's name. We told them to refer it back to the vendor, although I helpfully corrected the dodgy XML and submitted the app myself to save them hassle as its one of our larger (i.e. "damned important") brokers.


    I bet my life that the real WTF was doing that bit myself because they won't go back to the developers (and will complain to us the next time it happens...)



  • @Medezark said:

    @SlyEcho said:
    To all of you people getting confused about the tilde character: this is not about the tilde character which is totally fine, ...

    But the OP specified a ~ character as the culprit.

    I thought the OP meant it as an example of a character. But if ~ characters are rejected then the OP is TRWTF.

    Now that I have read it again, it seems I was wrong. Microsoft's XML parser has already read the file and he is manipulating the DOM using a debugger.


Log in to reply