Why Alexa sucks.
In my articles and research, I have sometimes used Alexa’s traffic rankings as a way to get at least some information on how much traffic a certain website has. Lately I’ve been getting comments by people saying that Alexa is inaccurate. It’s not exactly news to me, but…
The truth is, Alexa IS inaccurate. Grossly inaccurate. I’ve always thought that Alexa can still be of use in certain situations. I did a bit of research today (I’ve gotten the idea from a good little article on Digg and Alexa on Mashable.com, thank you), and I’ve came to the conclusion that Alexa is not only inaccurate, but also useless. Its data figures are almost random. Read on.
Alexa supposedly works like this: it measures what Alexa toolbar users do, and gets its data from there. And I always thought: ok, it’s a percentage of internet users, same as users of Firefox are a percentage of total number of internet users, so it might not be perfectly accurate, but it’s still accurate to some degree, right? Wrong. Alexa churns out completely wrong and seemingly random numbers.
Well, how does Alexa calculate traffic, then? There are many speculations about it. The truth is, after just a little research, I don’t even want to know. It certainly doesn’t seem connected to, you know, the actual traffic. Some people say that it can be manipulated by using Alexa toolbar, as in, you and two of your friends install it and you get a huge boost in Alexa rankings. Which seems to be true, but also doesn’t explain how the damn thing works, because it would implicate that basically noone is using Alexa toolbar anymore. Which also might be true. Like I’ve said, it’s time to stop caring.
You know why? Because Alexa isn’t the only free service that estimates web traffic. There are several alternatives out there, most notably SnapShot, Quantcast, Google Trends. This gives us the opportunity to simply try out a couple of websites and see how the traffic ranking matches accross different graphs. You guessed it: it matches to a reasonable degree in the mentioned three other services, but it’s completely bonkers in Alexa. Which leads me to the conclusion that any of these services is better than Alexa (and probably the best idea for any serious research would be to do a side-by-side comparison of all of them - someone make that mash-up, please?).
And now, for the cold, hard facts. These numbers need little or no explanation. In the following three instances, I took three websites with predictably comparable traffic but focused on completely different areas. I didn’t set this up to make Alexa look bad, I just came up with 9 sites from the top of my head. In all three tests, Alexa had major, inexplicable differences from the other two-traffic measuring sites. Here are the results (btw, the timeframes don’t match, but the differences are so huge that it doesn’t really matter):
flurl.com, pitchforkmedia.com, metacafe.com - everyone besides Alexa sees Metacafe on top here. Also, Alexa shows a big hump in Flurl’s traffic, starting with August 2006, which is nowhere to be seen in the other graphs.
![]()
![]()
![]()
penny-arcade.com, pvponline.com, dilbert.com - Yay, an online comic battle! Google trends said it doesn’t have enough volume to compare, so its results are omitted here. The results are similar accross the three graphs, with one notable difference: Snapshot and Quantcast say that Penny-Arcade’s traffic is slowly falling, while Alexa sees it as rising.
![]()
![]()
![]()
digg.com, facebook.com, imdb.com - I took a couple of heavyweights for the last one, and boy did they cause a mess for our poor traffic measuring sites. Fact: Everyone except Alexa puts Digg FAR behind the other two, in the third place. Fact: Facebook fares pretty horrible in Alexa’s graph compared to others. However, since the graphs generally don’t match here, I can only conclude that at least two of the traffic-measuring sites have guessed horribly wrong here. My estimate would be that Google trends and Snapshot are on the right track, although I don’t have the facts to back it up. One thing is for sure: Alexa is dead-wrong about Facebook here, because Facebook is heavily analyzed lately and many analysts put in the top 10 of all websites.
OK, so what have we learned here? Well, although my methods aren’t exactly scientific, I’d say that this short analysis gives a good estimate about the quality of Alexa’s rankings. And the estimate is: they suck. Even if you’re going to compare Alexa’s data with the other 3 sites mentioned here, you’ll still get useless results because Alexa’s rankings are so far off that it would just ruin your graph. Conclusion: although I’ve used Alexa’s rankings for some analysis in the past, I’ll either steer clear of it in the future or triple-check it with these other services.
December 27th, 2006 at 1:12 pm
Let’s just hope that the vast majority of internet publishers come to realise this all-to-open fact!
January 19th, 2007 at 6:51 pm
I laughed when you asked who actually uses the alexa toolbar c:
My experience shows traffic rankings depend entirely on the users of the providers of the rankings. I get much more referrals from MSN Live that Google, and MSN ranks me higher.
February 19th, 2007 at 12:16 am
[…] Like many other folks, I think Alexa data is just plain wrong. […]
April 20th, 2007 at 4:14 am
[…] I’ve recently voiced my opinion that bigcos and some of the essential components of Web 2.0, especially mashups and sharing (all kinds of sharing) simply don’t mix. With the recent news of Amazon suing Statsaholic, the poor little mashup that simply took Alexa’s (largely useless) data and made it better looking - and after they’ve done everything in their power to shut down the service - this seems even more obvious. […]
June 11th, 2007 at 1:04 pm
Stan,
We recently undertook a similar analysis and concluded Alexa was using a random number generator to produce the graphs. This was the only theory that consistently held up to a variety of tests.
June 11th, 2007 at 2:12 pm
[…] So why I still don’t think Alexa traffic ranking will do anything good to my blog? I agree with few of the bloggers out there. Why Alexa sucks. 3 Reasons Why Alexa Sucks (And They Know It!) […]
June 12th, 2007 at 2:45 pm
@Steve: hey, i bet at least 5% of it is real solid data (:
November 4th, 2007 at 10:33 am
[…] berguna untuk melihat seberapa ramaikah web/blog yang kita kelola selama ini. Akan tetapi ada juga beberapa org yang kontra terhadap kesimpulan itu. Mereka menilai bahwa perhitungan ranking alexa tidak akurat […]
December 21st, 2007 at 12:27 am
[…] it dead, even when there’s no other indication (except an Alexa graph, and we all know how reliable those are) that the company is actually doing bad. Case in point: […]
August 29th, 2008 at 5:59 am
Good post, may I use it?
October 6th, 2008 at 10:09 pm
[…] berguna untuk melihat seberapa ramaikah web/blog yang kita kelola selama ini. Akan tetapi ada juga beberapa org yang kontra terhadap kesimpulan itu. Mereka menilai bahwa perhitungan ranking alexa tidak akurat […]
October 21st, 2008 at 12:30 am
[…] berguna untuk melihat seberapa ramaikah web/blog yang kita kelola selama ini. Akan tetapi ada juga beberapa org yang kontra terhadap kesimpulan itu. Mereka menilai bahwa perhitungan ranking alexa tidak akurat […]
March 5th, 2009 at 1:06 am
The factors that are determinant in a website’s http://www.alexa.com/data/details/main?url=www.fortunehotels.in Alexa rankings apart from the data collected from the users of the Alexa toolbar are no where clearly mentioned. This somehow brings in a question of doubt and credibility issues as far as the Alexa rankings are concerned. However, even today the Alexa toolbar has the largest chunk in determining the ranking of a website.
April 6th, 2009 at 9:15 am
frying oil filter siemens business notebook pro v 5315
April 18th, 2009 at 7:52 am
uk number one singles history
jobserver.exe
June 5th, 2009 at 4:08 pm
what a great post, i like it
July 3rd, 2009 at 11:39 am
This was very nice to read. I will share this with my friends on Facebook.