Oops, Twitter Does it Again

Oops I did it again - Britney SpearsI don’t think Twitter is listening. In fact, I remember a few of them mentioning they don’t read the news or blogs so that it doesn’t affect their work. Perhaps it’s about time they start. Today I blogged over on LouisGray about a cool new change to the e-mail notifications on Twitter where they now show a user’s profile image and follower/friend data along with the new follower e-mails they send to you. (and dang it – MG beat me to the news. He’s good!)

What is scary though is what actually happened behind the scenes in this change. Today Twitter, again without notice to developers, completely changed the format of both their new follower and DM e-mails from plain text, to HTML multi-part format, completely breaking any app that was relying on those formats to parse and process new follows or direct messages.

What’s funny is that the very apps I was saying Twitter was venturing into competing ground with, Topify, and Twimailer, are the very types of apps that would have been broken by this change. In addition, apps like Greg Lavallee’s addNetflix app are now broken because they were relying on the plain-text format of the new DM e-mails. In my previous post about Twitter doing this, Greg commented, stating, “When I first read this post last month I thought, “well, if you code your application well, it should take into account potential changes from Twitter.” I also thought that Twitter would warn us about bigger changes. Wrong on both counts.” Many apps are relying on these e-mails, some of them probably completely unaware their apps are broken at the moment.

This issue was brought up on the Twitter developers list this afternoon by TwitReport developer, “TjL” (“Can Twitter Please Pick a From, and Stick With It?”). Evidently the new format also broke the new follower statistics for his app, and has happened multiple times causing him to have to re-educate his users to re-do their filters every time. Matt Sanford, Twitter API Team member responded explaining,

“We had changed the from address to try and improve bounce reporting and prevent being marked as spam by major ISPs. When we added the HTML formatting we found that we needed a consistent address for the ‘always display images’ option in many clients so we changed things around again. Hopefully this will be the last change as it causes us a bunch of work as well. I’ll keep an eye out for future changes and try and let people know.”

The conversation went on to discuss further elements of the e-mail, and Sanford suggested they were going to change the e-mails again after the discussion. I think TjL reflected the developer community’s frustration exactly when he responded to the further changes,

“Seriously? I’ve already started telling people to change their filters
and now they’re going to break *again*.

This is why daddy drinks.

All kidding aside, I don’t understand how a change like this gets
pushed out without the left hand knowing WTF the right hand is doing —
which is what it looks like (from an outsider’s perspective) happened.

IMO/FWIW: You’ve gotten too big to make these sorts of changes without
more consideration and communication. It makes me look bad as a
developer, and it makes Twitter look bad.

The irony is that you’re a company built around communication.”

Twitter has got to change their ways – on my blog posts about this I’m seeing comment after comment of developers now refusing to develop on the Twitter development platform because of their lack of warning during changes like this. The thing is I’m not complaining about rate limits or Twitter scalability or anything like that at all when I’m complaining. As developers, we simply want a little bit of communication before changes go out. I actually like Twitter. I have a business with components built on it so I want it to succeed. I also think the Twitter dev team has done an outstanding job building out this amazing API. The only area they’re failing in right now is communication. We need a) a clear developer ToS, and b) warning before changes go in, or come out. Developers have been amazingly patient for the most part regarding this, but I know there is frustration.

I want to be clear that I love what the Twitter API team is doing. I really like and respect Alex and Matt and the rest of the team working extremely hard, often to the late hours of the night working on this stuff. I’m not sure where the fault lies, but I do hope they are listening. We need some warning on this stuff guys.

There is still no official announcement on the Developer mailing list, nor any official blog post by Twitter on the e-mail changes.

26 thoughts on “Oops, Twitter Does it Again

  1. Nice post. I just want to add the minor annoyance they caused to anyone who applied a filter to the emails from Twitter. The change to the email killed my filter. Minor annoyance for me to fix, but these minor annoyances add up.

    Like

  2. Someone should tell them about how great TWITTER is for keeping a diverse group of interested parties informed about a topic :: It's free … they should give it try.

    Like

  3. SaltyDroid, there is an @twitterapi account that they should be using for
    stuff like this, but it hasn't been used in every case. There is still no
    announcement of this on that account. The other problem they have is that
    they have about 3 or 4 blogs, and 3 or 4 Twitter accounts. Which ones do I
    follow for the information I need?

    Like

  4. Twitter e-mails include several different headers to describe the e-mail such as “X-Twittersenderscreenname: “

    And as I recall when I began working on my own app, they suggested using these headers instead of formats of the e-mail address / subject / message body / etc. as their format wasn't necessarily set in stone.

    As for twimailer and topify, i was using topify until I read a blog post/comment somewhere that mentioned the danger of doing that. The problem comes from if these services should happen to shut down and you needed to do a password reset. How do you do a password reset to an e-mail address you don't access and no longer works?

    Like

  5. From the API Wiki: Design Patterns

    How should my bot follow users that follow it or inspect direct messages?

    By default, Twitter sends an email the first time a user follows you or sends you a direct message. For your bot-building pleasure, we've added the following headers to those emails:

    * X-TwitterEmailType – will be 'is_following' or 'direct_message'
    * X-TwitterCreatedAt – ex: Thu Aug 07 15:17:15 -0700 2008
    * X-TwitterSenderScreenName – ex: 'bob'
    * X-TwitterSenderName – ex: 'Bob Smith'
    * X-TwitterSenderID – ex: 12345
    * X-TwitterRecipientScreenName – ex: 'john'
    * X-TwitterRecipientName – ex: 'John Doe'
    * X-TwitterRecipientID – ex: 67890
    * X-TwitterDirectMessageID – ex: 2346346

    Like

  6. This is a change to the *email format* we're talking about here, not an API change?

    Isn't the idea to use the API to hook your funky little application into Twitter instead. Using emails to interact seems pretty dicey to me, because of the kind of reasons above.

    Drew

    Like

  7. Pierre, any idea if these headers were modified? I'm pretty sure I had
    Google filtering on those at one point and just recently all my filters
    seemed to break with the new e-mail formats.

    Like

  8. Drew, I agree. Twitter has actually recommended this as a suggested way to
    detect new followers though, since they don't provide an API for new
    followers. There have been multiple discussions on this on the Twitter dev
    mailing list and even an entire spec on how to parse the e-mails.

    Like

  9. Not to my knowledge, although for my personal twitter I turned off notifications a few days ago. After I stopped using topify, I just stick with the DMs from my app which sends hourly stats on who i've gained and who i've lost.

    Like

  10. Jesse don't be such a baby. This is exactly the way facebook treated it's devs with changes. If you can't take the heat move on.

    Like

  11. I'm not a developer or a fan of twitter. I say let twitter continue to sow the seeds of their own demise. I'm a big believer of pay it forward, karma whatever else you want to call it. The faster it occurs the faster people start to adopt friendfeed. Friendfeed Rocks.

    Like

  12. Jesse,

    Why would this in any way impact you? You would never be parsing the text of the e-mail, when they offer the full headers with cleanly parsable details … right?

    I've compared the old and new headers and see no differences … and with those there, any changes to the page format don't matter.

    What exactly did you choose to do in your code that was broken by the e-mail change? I'm curious of the details …

    Like

  13. Sam yes, and Facebook has changed their ways since. They were wrong for not
    announcing changes before as
    well, and I was complaining about them back then too. This isn't a
    very hard thing to do and it's very amateurish to not be notifying
    people on stuff like this. Twitter's been doing this type of stuff a
    lot longer than Facebook did.

    Like

  14. Scott, this one didn't affect me personally. It did affect multiple other
    developers though. The problem here is Twitter is still not announcing
    changes before they release them in the wild. This is poor and amateur
    practice if you ask me. If this one doesn't affect you another release will
    until they change their release practices.

    Like

  15. Jesse,

    FAN*tastic post! You captured the essence of the situation brilliantly – love it! One of the “beauties” of Twitter is that it's been touted again and again a developer's dream .. and party attributable to it's success. Given that, it's especially ironic for Twitter to make these changes so willy-nilly, without any apparent regard for how it impacts both developers AND users.

    As a user, I'm impacted because it's annoying. Hard to read on my phone and just about as annoying delivered in email. Simple is often better and, IMHO, in this case, simple was infinitely better.

    Thanks again for a TERRIFIC article.

    @shellykramer

    Like

  16. The bottom line is Twitter is free and slowly but surely they will offer all the features that all the other Twitspin off products offer. Making them irrelevant. So I think people should do less worrying about how Twitter is messing up and screwed them and more of how long it will be before the gravy train is OVER. Great article!!

    Like

  17. The bottom line is Twitter is free and slowly but surely they will offer all the features that all the other Twitspin off products offer. Making them irrelevant. So I think people should do less worrying about how Twitter is messing up and screwed them and more of how long it will be before the gravy train is OVER. Great article!!

    Like

  18. Jesse,

    FAN*tastic post! You captured the essence of the situation brilliantly – love it! One of the “beauties” of Twitter is that it's been touted again and again a developer's dream .. and party attributable to it's success. Given that, it's especially ironic for Twitter to make these changes so willy-nilly, without any apparent regard for how it impacts both developers AND users.

    As a user, I'm impacted because it's annoying. Hard to read on my phone and just about as annoying delivered in email. Simple is often better and, IMHO, in this case, simple was infinitely better.

    Thanks again for a TERRIFIC article.

    @shellykramer

    Like

  19. I'm not a developer or a fan of twitter. I say let twitter continue to sow the seeds of their own demise. I'm a big believer of pay it forward, karma whatever else you want to call it. The faster it occurs the faster people start to adopt friendfeed. Friendfeed Rocks.

    Like

  20. Not to my knowledge, although for my personal twitter I turned off notifications a few days ago. After I stopped using topify, I just stick with the DMs from my app which sends hourly stats on who i've gained and who i've lost.

    Like

Leave a comment