RAAK

The story so far …

In last week’s tech post I focused on Twitter influence measuring service Klout’s inability to stop bots from acquiring high klout scores. Klout responded positively, and got involved in the conversation.

Joe Fernandez, CEO of Klout, raised the philosophical question of whether giving bots a high influence score is a good or a bad thing. Shouldn’t it be possible for a bot to get a high Klout score? Is it not possible for a bot to have a high influence level?

To recap, the test probes in this experiment are four bots that Tweet random slightly humorous quotes once every minute, once every five minutes, once every 15 minutes and once every 30 minutes respectively.

Testing Peerindex

So, this week, as promised, I registered my four little soldiers on peerindex.net, and within 24 hours the scores were in. As predicted by Azeem Azhar, Peerindex’s CEO, the bots scored reassuringly low. (See his comment on the previous post).

Let’s look at a screen shot of the first bot (the one with the highest score, which reached a score of 52 on Klout)

Quite interesting, isn’t it? At a first glance, it really seems like they got it spot-on. Except for one thing … the realness count (indication of whether the account is a human or a bot) is 75% – quite high for a bot, especially since, according to their score documentation, they start the score at 50%, and increase it only based on clues that the account is human.

Let’s look at the other bots’ realness count:

All of them score 75% – an indication that this metric is, as it should be, based (directly or indirectly) on Tweet content. Since the bots Tweet from the same set of quotes, this makes sense. It’s not as reassuring as our first impression though …

Then, while looking at one of my other accounts, I saw something quite disturbing. A highly acclaimed South African author and singer/songwriter I’m following, Koos Kombuis, has a peerindex of 0 …

… and a klout score of 58! His large klout score makes a lot more sense, given his celebrity profile and the size of his actively participating online audiences.

Koos also seems to have only a 35% likelyhood to be a human. Koos, are you still in there?

This had me quite puzzled for a while, until, Tuesday at #DevNest, Peerindex’s Product Manager, Simon Cast, revealed that a large part of their metric is based on the links you include in your Tweets. None of my bots ever tweet any links, and Koos Kombuis also rarely tweets links.

This actually means that testing Peerindex against these bots doesn’t mean much – a proper test will be to set up a set of bots that tweet automated content and links, and test Peerindex against those.

Let’s take a moment to investigate the philosophy behind ignoring linkless Tweets, though …

I believe, by discounting Tweets that don’t contain any links, Peerindex is ignoring a large part of what is important on Twitter. Koos Kombuis, for instance, uses Twitter for widely followed creative projects, like his most recent, #Twitterdawn, a novel written entirely by means of Twitter.

Stephen Fry had a very interesting feed to follow last year, as he followed the transfer of four Northern White Rhino’s from a zoo in the Czech Republic to a nature reserve in Kenya. During this period, Fry tweeted a lot, and his feed mostly consisted of news that he was witnessing first hand, in other words, linkless.

Having said this, in a recent Cinch interview by Robert Scoble, Azeem@Peerindex sums it up quite well:

… we have these obvious monsterous failings, which I’m quite happy to talk about … there’s a guy who’s won the nobel prize in chemistry, and his Peerindex is 0 … that doesn’t reflect the real world … you have a guy like Clay Shirky, he blogs once a month, a million people read his blogposts. he doesn’t have as high a Peerindex as somebody who Tweets his stuff out … these are huge holes in our data index and in our analytics … we’re a startup, we’re going to get better at doing that, but right now, today, if you’re trying to find topical authority for people on Peerindex, it’s not a bad place to start …

Both Klout and Peerindex are busy with pioneering work. In a way, they are to Social Networks what Google was to the Web in 1998.

There are obvious flaws in their respective approaches, not because of negligence or stupidity, but because what they’re trying to do is dangerously close to linguistically perfect artificial intelligence. It’s never been achieved before.