Use Crowdsourcing to Identify Popular Usability Tools

It would be very helpful for all web developers to know which usability tools were most popular. Ideally, we would all know which user interface design tools were used by successful website developers and usability professionals. There are so many new tools now available (at least 300), that it would be difficult to do detailed evaluations of them all.

One way to ascertain a tool’s popularity is to see how many are using the product’s website. We assumed that the more people accessing a tool’s website, the more that were using that product. In other words, the better the rankings, the more popular the product. This is a form of ‘crowdsourcing’.

Crowdsourcing refers to obtaining needed information by soliciting contributions from a large group of online users. In this case, we combined the efforts of a large number of computer professionals who use the Web to find, use and buy usability software tools. This provides an opportunity for each contributor to unknowingly add a small portion to the overall result. In this case, the final result comes from a (loosely defined) group of people who have the common interest of improving the usability or user experience (UX) of online products.

Ideally, we would have an up-to-date count of how many of each tool was purchased or downloaded on a daily, weekly or monthly basis. But this information was not available to us. Consequently, we used an estimate of website popularity. Although it is rather crude, we used the site and page rankings produced by three different methods (WebRank Stats, Google Page Rank and Alexa) to estimate popularity of both usability tools and usability books. Each of these metrics tries to estimate the relative popularity of certain websites, and less directly, with the usability tools they represent.

‘WebRank Stats’ is a service for webmasters, search engine optimization (SEO) specialists and Internet surfers. This tool combines many web analytics from other sites, and then creates a number between 1 and 9, with 9 being the most popular.

‘Google PageRank’ is an algorithm developed to determine the relative importance of specific sites. The tool shows a website’s rank as a number from 0 to 10, with 10 being the most accessed.

The ‘Alexa Site Rank’ metric ranks sites based primarily on tracking a sample set of internet traffic. Major users of Alexa’s toolbar include the Internet Explorer, Firefox and Google Chrome web browsers.  The Alexa toolbar displays a website’s rank, where lower is better.

The webusability.com website presents all three of the ranks just discussed for each usability tool (almost 300 tools). The data that were collected as of August, 2013 were used to calculate the correlation between each pair of the three ranking methods. We found that Google PageRank and WebRank Stats had the highest correlation with each other of 0.72. The Google PageRank and WebRank Stats correlations with Alexa Rank were lower at -0.43 and -0.42. The minus sign indicates that the higher the Google PageRank and WebRank Stats rank, the lower the Alexa ranking. The fact that none of the correlations were a perfect 1.00 suggests that each ranking method was measuring something different, which argues in favor of including all three metrics.