Perhaps calling it Twitter’s new API is a bit of a misnomer, since Twitter announced User Streams and Site Streams in April already. It is still in beta though, and developers have only been given access to it last month. On top of that, we are not allowed to build public apps with it yet. So, in a sense, it is new (if, like me, you’re a bit old fashioned)

So, what’s the deal with these new API’s anyway?

In the past (which is now) Twitter applications have had (or still have) to request data off the Twitter timeline one batch at a time, using the Twitter REST API. Each batch consists of 20 – 200 tweets. In other words, the developer of an app will decide to poll Twitter, say, once every few minutes, effectively asking Twitter if there are any new Tweets. This is like registering a poste restante postal address, and going to the Post Office each time you think you might have post.

To make things more difficult, these API requests are rate limited. That means you’re not allowed to go to the Post Office as often as you like. If you do, the security guard will close the door in your face when you try and go in, and only let other people in. Sounds frustrating, huh?

See why the Stream API have developers sipping their Caffeinated Root Beer with renewed vigor?

Suddenly, you get a post box in your front door. And every time a letter arrives it finds its way into your house, as if by magic! This is what User Streams are all about. Instead of making numerous calls to the Twitter API, a connection is made, and kept open. Over this connection, Twitter then sends any relevant tweets as soon as they get it. It is the future! No more rate limiting!

Site Streams are even more powerful. This is a single connection over which numerous users’ tweets are sent. Think, for instance, of an application with a number of users. The application won’t have to create a User Stream connection for each one of its Users, but it only needs to open a single Site Stream and all its users’ tweets are sent over that stream, leaving the sorting for the application. This is like a block of flats sharing a kind of premium post box – one that gets special attention from the Post Office.

So, what will the end user experience be?

No more insane refreshing to try and see all your Tweets in real time – by default, any well written application will deliver your timeline in realtime. Imagine the user experience on searches, especially ones that return a lot of results. At the moment the result is delivered in large batches of results, a while apart. It makes it very difficult to follow it in realtime.

In future you’ll see these as they arrive at Twitter, one by one, giving a very good indication of the speed at which they arrive. The future is now – it’s time to tweet like an addict.