Twitter's API sucks



  • I've been working all day with Twitter. It sucks. And I hate it. And I want it to die. In a fire.

    So here's a simple requirement: search Twitter for a term, say, "shoes", then for each user who tweeted about shoes find out how many followers they have. Sound simple, right?

    So Twitter has a simple API for performing searches which is easy-to-use, returns the data in a sane format, and generally is not-WTF-y at all. At least, at first glance. Oh, and BTW, this also happens to be the only API for performing searches.

    Digging into Twitter's other API, it has a nice users/lookup function. You pass in a list of users, either identified by screen name or by user ID, and it returns the information we need (along with a bunch of other stuff we don't need, but can ignore.) It requires authentication, which is OAuth, which is a GIGANTIC PAIN IN THE ASS, but that's not so much of a WTF as they need to identify who's sending the request for rate-limiting. So far so good, right?

    So you write the program, and for some reason a bunch of the UserIDs returned from the search API are showing up as "not found" in the other API. WTF? Go digging again, and you notice this page linked from the search API documentation. Of course! User IDs from the search API are completely different than User IDs used by every other API because that makes fucking sense. (The link is basically kind of an explanation of why that is, which I guess is important, but it doesn't mention the more important issue: when the fuck are you going to fix it?! Twitter bought Summarize in July 2008, and it's still broken.)

    Ok, fine, so we have to look up our users by Screen Name instead of User ID. Ah, but one of the quirks is that Twitter users can change their Screen Name at any time... so the user with the Screen Name "Super69Womyn" right now might not necessarily be the same user who mentioned the search term 2 weeks ago. And since the search API's User IDs are about as useful as just sending back Rand(), there is absolutely no way to correctly link a tweet from a search result with a current user.

    It cannot be done. At all.


    The whale has never failed harder.

    Jesus fuck this is an awful API. Until I came across this issue, my only complaints were:

    1) OAuth is a fucking pig and needs to die in a fire

    2) The API documentation is fucking beyond awful (I'm guessing Twitter devs are open source fans, from the state of the documentation and their mostly-ignored bug tracker)
    To give an example, look at the users/lookup link above... great is shows an example of what the results look like, but note it doesn't define what any of those field labels mean. Nor is there any kind of thesaurus to define terms as Twitter uses them. Especially egregious are fields like "following" which are boolean... what the holy shit does that mean? And "favourites_count", is that the number of time the user has been favorited by others, or the number of favorites they have? (I got this one wrong and made an ass out of myself this morning.)

    3) The API is missing a lot of more advanced functions that are really impossible to implement in a client (for example, reverse-lookup to find how many users favorited a tweet-- you can go through a user's list of followers and ask each follower whether they favorited the tweet, but that doesn't help if the tweet was favorited by a non-follower-- that's literally impossible to get via the API, as far as I can tell)

    Those things are annoying, but not that big a deal.


  • 🚽 Regular

    I've found Facebook's API to suck even more. Always changing, docs not matching reality, major problems with server-side caching that makes debugging very difficult, and no sandbox to work with. It's a complete disaster. It especially sucks when you have to save face whenever something breaks because some API change broke what previously worked and you have to run around looking through the dev blog for that one "btw" from a dev that says "X will no longer work after our release TOMORROW."



  • Our clients are slowly getting into the twitter thing, so I find myself writing tweet-querying stuff now and then. The simple stuff— getting some tweets— works like a charm, but apparently anything more advanced becomes troublesome.

    Thanks for the heads up.



  • There is an open bug about this. But don't worry, they are definitely planning to fix this, as soon as they have time. Some dev said that in 2008, so it must be true.



  • @RHuckster said:

    I've found Facebook's API to suck even more. Always changing, docs not matching reality, major problems with server-side caching that makes debugging very difficult, and no sandbox to work with. It's a complete disaster. It especially sucks when you have to save face whenever something breaks because some API change broke what previously worked and you have to run around looking through the dev blog for that one "btw" from a dev that says "X will no longer work after our release TOMORROW."

    Tomorrow? If only. "Oh, and btw, if Y doesn't work for you, it's because we dropped that functionality yesterday, and we're not planning to put it back; neener neener."



  • @RHuckster said:

    I've found Facebook's API to suck even more.
     

    Forget the API. Just look at the semantic markup which they want you to put in your pages.

    <meta property="og:type" content="movie"/>
    That's fine if you don't care about your pages being valid XHTML. If you do care, then tough: the "meta" element doesn't have a "property" attribute.

  • 🚽 Regular

    @pjt33 said:

    Forget the API. Just look at the semantic markup which they want you to put in your pages.

    <meta property="og:type" content="movie"/>

    That's fine if you don't care about your pages being valid XHTML. If you do care, then tough: the "meta" element doesn't have a "property" attribute.

     

    That's only the tip of the iceberg. Up until last year, they had that god-awful FBML crap. It was especially bad because with that, there was no way you could really 'test" your pages until you put them on some stupid makeshift "testing" page... but it broke the concept of the development process because you were effectively exposing your experimental and under-construction app to the world, just hoping nobody would wander into your public test page.



  • @pjt33 said:

    @RHuckster said:

    I've found Facebook's API to suck even more.
     

    Forget the API. Just look at the semantic markup which they want you to put in your pages.

    <meta property="og:type" content="movie"/>

    That's fine if you don't care about your pages being valid XHTML. If you do care, then tough: the "meta" element doesn't have a "property" attribute.

     

    I never want to miss an opportunity to rant about facebook, but in this case, it's not their fault. It's just no pure XHTML, but it's perfectly valid XHTML+RDFa, which is another W3C-sanctioned standard. If you use the right doctype, you can check it with W3C's validator just fine.

     



  • @PSWorx said:

    I never want to miss an opportunity to rant about facebook, but in this case, it's not their fault. It's just no pure XHTML, but it's perfectly valid XHTML+RDFa, which is another W3C-sanctioned standard. If you use the right doctype, you can check it with W3C's validator just fine.
     

    I have a vague memory of having tried this and it not working, but either way that's the kind of thing they should document. Instead they tell you to use namespaces for og: and fbml: which are completely pointless because you're not actually using og:meta or fbml:meta elements.



  • @blakeyrat said:

    So Twitter has a simple API for performing searches [ ... ]

    Go digging again, and you notice this page linked from the search API documentation.

    Admittedly I was forewarned, but the very first thing I noticed on clicking your first link (to the search api documentation) was this big warning right up at the top of the page, before you even see any of the parameters or anything:

    [quote user="a simple API for performing searches"]Warning: The user ids in the Search API are different from those in the REST API (about the two APIs). This means that the to_user_id and from_user_id field vary from the actualy user id on Twitter.com. Applications will have to perform a screen name-based lookup with the users/lookup method to get the correct user id if necessary.

    [/quote] Did they add that since you posted, or did your eye skip over it thinking it was irrelevant boilerplate?

     


  • 🚽 Regular

    @PSWorx said:

    I never want to miss an opportunity to rant about facebook, but in this case, it's not their fault. It's just no pure XHTML, but it's perfectly valid XHTML+RDFa, which is another W3C-sanctioned standard. If you use the right doctype, you can check it with W3C's validator just fine.
     

    I'd argue that it's easier to just use the perfectly valid "name" attribute on the meta tag. The W3C does not have a whitelist of valid values for that attribute, and it just makes sense to follow the same conventions as everyone else. All the cool kids are doing it, even IE is doing it now.



  • Ever since Twitter started making their own clients, they're letting the public API slip by the wayside.

    Such a shame, considering I loved the Twitter API circa early 2009 when they didn't even have any of this OAuth shit and the documentation was a bit nicer.



  • @DaveK said:

    Did they add that since you posted, or did your eye skip over it thinking it was irrelevant boilerplate?

    No; it was there, but I was using the API through a library with its own documentation that did not make mention of it. I didn't look at the Twitter documentation (what little there is of it) until I had problems.



  • @blakeyrat said:

    @DaveK said:
    Did they add that since you posted, or did your eye skip over it thinking it was irrelevant boilerplate?

    No; it was there, but I was using the API through a library with its own documentation that did not make mention of it. I didn't look at the Twitter documentation (what little there is of it) until I had problems.

    Heh, yet another case of a TDWTF submission getting edited to make it sound more dramatic at the expense of precise factual accuracy!



Log in to reply