Introduction
We have been using a new, very effective methodology for conducting usability tests on websites. Over the past couple of years, we have been evaluating, refining, and enhancing ‘FirstClick’ usability testing. FirstClick testing focuses on the user’s first click on the first page that users encounter in a website. We then compare the success of that click with whether or not the user successfully completes the task.

 

Method
We used Bailey’s Usability Testing Environment (UTE) to collect data during the tests. The tool enabled us to (a) present the task scenarios to each user, (b) determine which link was selected (clicked) on the first page, (c) follow the user’s path to determine whether each user got to the target page, and (d) record whether the user was able to successfully answer a multiple-choice question related to the target page. This provided the data we needed to do the FirstClick analyses.

 

Conclusion
In addition, those scenarios that had incorrect first clicks tended to take longer to complete, and required more page views. In general, we found that participants were about twice as likely to succeed if they selected the correct response on the first page with which they had to deal. The average correct/incorrect ratio was 1.9. ions.

 

This approach offers many benefits. The first major benefit is that it allows (encourages) the use of a much wider selection of tasks in each one-hour test than has been used in the past. A traditional usability test usually includes 10-15 task scenarios, but a FirstClick usability test can include 50 or more scenarios. A second benefit is that designers/developers can systematically begin their testing with the homepage, and then after making the necessary improvements, move on to major second-level pages.

 

Results
We analyzed the results of data collected on twelve usability studies on different websites, each with varying amounts and types of content, a range of subject matter complexity, and distinct user interfaces. The results showed that if the first click was correct, the chances of getting the entire scenario correct was .87 (almost 90% were successful). On the other hand, if the first click was incorrect, the chance of eventually getting the scenario correct was only .46, which is less than a 50-50 chance of being successful.

 

In addition, those scenarios that had incorrect first clicks tended to take longer to complete, and required more page views. In general, we found that participants were about twice as likely to succeed if they selected the correct response on the first page with which they had to deal. The average correct/incorrect ratio was 1.9.

 

A third benefit is that the FirstClick test results can show clearly which of the scenarios elicits the worst performance. The poor performers are the tasks that require the most attention. The close relationship between FirstClick performance and full Homepage-to-Content performance suggests that by staying focused on optimizing the participant’s first clicks first, it will provide major steps toward substantially improving a site.

 

References
Bailey, R.W. (2007), User Interface Update – 2007: An annual review of the usability literature for practitioners, Computer Psychology Training Course.

Bailey, R.W., Wolfson, C.A. and Nall, J. (2008), Revising a Homepage: Applying Usability Methods that Guarantee Success, Proceedings of the UPA.

Bailey, R.W., Wolfson, C.A. and Nall, J., Redesign of the CDC Website, In T.S. Tullis and B. Albert, Measuring the User Experience, Boston: Morgan Kaufmann Publishers, 2008, 252-262.

Wolfson, C.A., Bailey, R.W., Nall, J. and Koyani, S. (2008), Contextual card sorting (or FirstClick testing): A new methodology for validating information architectures, Proceedings of the UPA.