In my articles and research, I have sometimes used Alexa’s traffic rankings as a way to get at least some information on how much traffic a certain website has. Lately I’ve been getting comments by people saying that Alexa is inaccurate. It’s not exactly news to me, but…

The truth is, Alexa IS inaccurate. Grossly inaccurate. I’ve always thought that Alexa can still be of use in certain situations. I did a bit of research today (I’ve gotten the idea from a good little article on Digg and Alexa on Mashable.com, thank you), and I’ve came to the conclusion that Alexa is not only inaccurate, but also useless. Its data figures are almost random. Read on.

Alexa supposedly works like this: it measures what Alexa toolbar users do, and gets its data from there. And I always thought: ok, it’s a percentage of internet users, same as users of Firefox are a percentage of total number of internet users, so it might not be perfectly accurate, but it’s still accurate to some degree, right? Wrong. Alexa churns out completely wrong and seemingly random numbers.

Well, how does Alexa calculate traffic, then? There are many speculations about it. The truth is, after just a little research, I don’t even want to know. It certainly doesn’t seem connected to, you know, the actual traffic. Some people say that it can be manipulated by using Alexa toolbar, as in, you and two of your friends install it and you get a huge boost in Alexa rankings. Which seems to be true, but also doesn’t explain how the damn thing works, because it would implicate that basically noone is using Alexa toolbar anymore. Which also might be true. Like I’ve said, it’s time to stop caring.

You know why? Because Alexa isn’t the only free service that estimates web traffic. There are several alternatives out there, most notably SnapShot, Quantcast, Google Trends. This gives us the opportunity to simply try out a couple of websites and see how the traffic ranking matches accross different graphs. You guessed it: it matches to a reasonable degree in the mentioned three other services, but it’s completely bonkers in Alexa. Which leads me to the conclusion that any of these services is better than Alexa (and probably the best idea for any serious research would be to do a side-by-side comparison of all of them - someone make that mash-up, please?).

And now, for the cold, hard facts. These numbers need little or no explanation. In the following three instances, I took three websites with predictably comparable traffic but focused on completely different areas. I didn’t set this up to make Alexa look bad, I just came up with 9 sites from the top of my head. In all three tests, Alexa had major, inexplicable differences from the other two-traffic measuring sites. Here are the results (btw, the timeframes don’t match, but the differences are so huge that it doesn’t really matter):

flurl.com, pitchforkmedia.com, metacafe.com - everyone besides Alexa sees Metacafe on top here. Also, Alexa shows a big hump in Flurl’s traffic, starting with August 2006, which is nowhere to be seen in the other graphs.

penny-arcade.com, pvponline.com, dilbert.com - Yay, an online comic battle! Google trends said it doesn’t have enough volume to compare, so its results are omitted here. The results are similar accross the three graphs, with one notable difference: Snapshot and Quantcast say that Penny-Arcade’s traffic is slowly falling, while Alexa sees it as rising.

digg.com, facebook.com, imdb.com - I took a couple of heavyweights for the last one, and boy did they cause a mess for our poor traffic measuring sites. Fact: Everyone except Alexa puts Digg FAR behind the other two, in the third place. Fact: Facebook fares pretty horrible in Alexa’s graph compared to others. However, since the graphs generally don’t match here, I can only conclude that at least two of the traffic-measuring sites have guessed horribly wrong here. My estimate would be that Google trends and Snapshot are on the right track, although I don’t have the facts to back it up. One thing is for sure: Alexa is dead-wrong about Facebook here, because Facebook is heavily analyzed lately and many analysts put in the top 10 of all websites.

OK, so what have we learned here? Well, although my methods aren’t exactly scientific, I’d say that this short analysis gives a good estimate about the quality of Alexa’s rankings. And the estimate is: they suck. Even if you’re going to compare Alexa’s data with the other 3 sites mentioned here, you’ll still get useless results because Alexa’s rankings are so far off that it would just ruin your graph. Conclusion: although I’ve used Alexa’s rankings for some analysis in the past, I’ll either steer clear of it in the future or triple-check it with these other services.

This entry was posted on Monday, December 18th, 2006 at 2:41 pm and is filed under Reviews, Web, Features. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.