XML endianness problems



  • Second Life, (a sort of 3D online virtual world), recently moved some of its client-server communications to LLSD (a home-brewed XML-based serialisation format). Reasonable enough.

    Unfortunately, IP addresses and certain other integer values are transmitted as base64-encoded binary data. What's more, they use the opposite endianness to the one standardised on for the older UDP-based communications. The icing on the cake was when they changed the endianness of the IP address (but not any of the other fields) in the LLSD-based format at some point between the changes being put up on the beta-testing servers and being rolled out to the real systems. The official client coped OK, having been modified correctly; the unofficial (but quietly officially-approved) libsecondlife project didn't fare so well.

    (This has not been previewed; I'm a Konqueror user and I don't want to lose the post again. Besides, The Daily WTF's continual use of blocking XMLHttpRequests even when nothing is being done in the tab in question tends to cause Konqueror to stop responding after a while...)



  • Maybe there's a subliminal reason for the "LSD" part of the name...  (Related thought:  I've always wondered where Estridge and team came up with 5150 as the model number for the first PC ...)

     
    Okay, I'm reaching.  It's late on Friday (any time after lunch is "Late" on Friday).  And yes, I know that the 5150 was a follow-on to the 5100.  Just enjoy the joke.  And if you don't get the joke, [url=http://en.wikipedia.org/wiki/5150_%28Involuntary_psychiatric_hold%29]this article[/url] may help.

    On reflection, I seem to have lost myself.  Oh well.  Back to work.
     



  • @makomk said:

    Second Life, (a sort of 3D online virtual world), recently moved some of its client-server communications to LLSD (a home-brewed XML-based serialisation format)....
    [b]W[/b ]ith all these digits floating around I must conclude that LLSD is short for [i]Lotsa LSD[/i].

     



  • Um... I'm not normally one of those guys who goes around bashing XML.... but this case... I'd have to say "the problem is XML".  Really.

    When you consider how much effort Blizzard goes through to keep their packets compact, so that their servers can perform efficiently and the clients can survive on low bandwidth (playable over a modem!) its just sort of laughable to hear someone changing TO XML in their protocol.

    If ever there was a time for hand-crafted bit-packed binary encoding, its client/server communication on MMO games.



  • You don't even need to hand-craft the packet. Use ASN.1 and some decent compiler that produces PER. And from what I heard it's less overhead to read PER than BER.



  • @Sgt. Zim said:


    Okay, I'm reaching.  It's late on Friday (any time after lunch is "Late" on Friday).  And yes, I know that the 5150 was a follow-on to the 5100.  Just enjoy the joke.  And if you don't get the joke, [url=http://en.wikipedia.org/wiki/5150_%28Involuntary_psychiatric_hold%29]this article[/url] may help.
     

    I don't get it.  The 5150 PC was developed in Florida, not California.

     



  • So, um, let me get this straight. They replace the binary protocol they use for real time communication with some kind of home-brewed XMl format that uses Base64 to encode... binary information?

    SELECT * FROM Sense
    > 0 rows returned

     



  • Damn, they should've just gone straight for RFC 3252.



  • @SilverDirk said:

    Um... I'm not normally one of those guys who goes around bashing XML.... but this case... I'd have to say "the problem is XML". Really.

    When you consider how much effort Blizzard goes through to keep their packets compact, so that their servers can perform efficiently and the clients can survive on low bandwidth (playable over a modem!) its just sort of laughable to hear someone changing *TO* XML in their protocol.

    If ever there was a time for hand-crafted bit-packed binary encoding, its client/server communication on MMO games.

     Hence why they're only moving *some* of their communications over to XML-over-HTTP (mostly low-frequency, low-data ones where latency isn't so important). They've also moved texture uploads to HTTP (*not* base64-encoded, thank god!) and texture downloads are supposed to be moving over soon.

    Oh, and you really wouldn't want to play Second Life over a modem. The need to stream textures (amongst other things) makes it very bandwidth-heavy.


Log in to reply