What’s Your Real Twitter Cred? – A Better Metric

[UPDATE 12/29/08, 13:45 PST – new content added at the bottom]

This past weekend Seesmic CEO Loic Le Meur caused a bit of a fracas when he made a simple feature request for Twitter search. He wanted to be able to sort Twitter searches by the number of followers that a given user has, using that measure as some sort of indication of a user’s “authority”. That comment generated a surprisingly large and passionate amount of feedback from bloggers and Twitter users. Frankly, I was shocked that people felt so strongly about it, and I think Michael Arrington said it all with this post title: “Bloggers lose the plot over Twitter search“.

Be that as it may, it brings up a point that has long been talked about in the Twitterverse: how does one determine a Twitter user’s influence or authority?

Setting aside for a moment the question of whether or not we should even care about “authority” or influence on Twitter, I think that anyone advancing the notion of followers or followees as an accurate measure of influence is completely missing the mark. Twitter is social and interactive. The follower/followee model is overly simplistic. Sure, number of followers is interesting, since anyone with 15,000 followers has a “louder” voice than someone with 7, but what we still need is a dynamic, interactive measure.

Om Malik just posted an article about a study carried out by Bernardo Huberman (et al.) from HP’s Social Computing lab which examined the relationship between followers, followees (people a Twitter user follows), and “friends”. The study defined friends as people to whom a user has sent at least two @-replies.

As I have long suspected, measuring “friends”, as defined in the study, proved more meaningful a metric than simply counting followers or followees. A brief excerpt from Om’s post summarizes the study’s results:

On Twitter, [Huberman] found that regardless of the number of followers or followees, there were very few friends in a personal Twitter circle. He used a very weak definition of “friend” — anyone to whom a user has directed a post at least twice. And because of that, Huberman says that in order to “influence a person’s absorption of content, there is a need to find the hidden social network; the one that matters when trying to rely on word of mouth to spread an idea, a belief, or a trend.”

Huberman’s study found that:

• Users with a large number of followers are not necessarily those with very large number of total posts.
• Even though the number of friends initially increases as the number of followees increases, after a while the number of friends starts to saturate and stays nearly constant.
• The number of people a user actually communicates with eventually stops increasing while the number of followees can continue to grow indefinitely.

Interesting results, but still pretty obvious/intuitive. At least this study is getting closer to the heart of Twitter interaction and the influence its users have. After all, Twitter is a social tool, so just counting followers or followees doesn’t nearly capture the interactive nature of the service. Measuring “friends” (as defined above) gets closer, but I maintain that this still misses two very important measures. If, as Om assumes, we’re using the traditional definition of authority as the “power to influence or command thought, opinion or behavior”, then I propose that the following metrics are even more useful:

• Number of @-replies per post for a given user
• Number of “retweets” per post

As I stated in my comment on Om’s original post, I believe that one of the most important factors in a measure of authority or influence is the extent to which a user is able to get people thinking or talking about a topic. By that logic, a tweet which sparks a conversation and/or a large volume of replies should be given more weight than a tweet which goes out quietly and generates no responses.

To create a normalized metric which translates well across users with different numbers of followers we would ultimately need to collect data and see what kind of conclusions can be drawn from the metric above. We might find that it’s best to take the number of replies per post and multiply it by the ratio of followers who replied to the total number of followers. Or we might come to a different conclusion altogether.

The second metric I mentioned above is the number of re-tweets that a given post generates. Although they are less conversational in nature, re-tweets are interesting because they highlight tweets which someone not only found worthwhile, but found so compelling that (s)he wanted to pass it on to others. In fact, re-tweets are the core of Twitter’s ability to spread news at lightning speed. For example, in the recent terrorist attacks in Mumbai, Twitter was as good (if not better) a source of breaking news than any major cable news network.

I maintain, then, that any credible measure of Twitter authority has to take into account the spread of a re-tweeted post throughout the “Twitterverse”.

One of the main reasons I think my metrics are better than the “friend” metric defined in the HP study is because one can directly control (or even game) the friend count, whereas the reply/retweet metric is dependent on how other people perceive a given user.

For example, since “friends” are defined simply as people to whom one has directed two or more replies, I could start replying to tons of people throughout the Twitterverse, thereby increasing my friend count, but not really adding anything substantive to the community. Conversely, the reply/retweet metric is dependent on my ability to inspire and influence others. If I’m blathering on about pocket lint all day, my followers probably won’t find that noteworthy. Whereas if I’m funny, or a thought leader, or offering breaking news stories, my ability to inspire replies and retweets is likely to be much higher.

The bottom line is that while the study from HP’s Social Computing Lab is getting much closer to determining the influence of any given Twitter user, we really need more data to fully understand the interactive nature of Twitter and its users’ influence.

I would really like to see a more complete study which looks at the metrics I identified above. In fact, I have half a mind to get back to my statistics roots and put that ol’ masters degree to use in analyzing these metrics to see what conclusions can be drawn. If anyone wants to collaborate, drop me a line!

[UPDATE 12/29/08 – 13:45 PST]
It might not be obvious from my post above, so I wanted to go one step further and say that I don’t really think any single metric is sufficient to measure Twitter influence. While I strongly believe that two measures that I highlight above are more meaningful than follower count in a search for a proper influence metric, I think that they need to be looked at in context, and in combination with other measures (including follow numbers, total reach of a given tweet, etc.).

As a simple example, take the tweet from Loic which started this whole mess. Clearly it’s had a pretty major impact among the “Twitterati” and has elicited passionate feedback. But that didn’t just play out on Twitter. No, it’s been blogged, and retweeted, and remixed, and discussed both online and off (some would say ad nauseam). So perhaps any measure of Twitter “authority” is incomplete if it is confined to only measuring activity that plays out on Twitter.

One can extrapolate this problem even further and say that it’s directly related to the problem of measuring the full impact of anything that takes place on a blog, social network, or perhaps even the Internet. How does one measure the full impact of pay-per-impression advertising? Or a viral YouTube video? Or a popular blog post? Perhaps the best we can hope for is a good approximation.