Test and Evaluation | Web Usability

Conducting a Set of Usability Tests: The Final (Pre-Launch) Tests

04/23/2014 by Bob Bailey Leave a Comment

There were certain design decisions on both wireframes that led to better performance, and we used the best from both wireframes to produce a final wireframe. The final wireframe then was used as the basis for developing a graphic prototype. The graphic prototype, with images, colors, and a variety of type fonts for headers, was used for the final pre-launch usability test.

The participants were a mixture of federal and non-federal employees. The majority of participants included the primary audience for this site, which were healthcare providers, public health professionals and consumers. Again, these tests were conducted using Bailey’s Usability Testing Environment (UTE).

The pre-launch usability tests were divided into two parts: a pilot test and the final test. The pilot test was conducted on Monday using 18 participants and 56 task scenarios that were divided into three categories: 24 FirstClick from the homepage, 24 FirstClick from one of the new second-level pages, and 9 ‘homepage to the actual content page’ (all clicks). All participants saw all scenarios in one-hour in-person testing sessions. [Read more…]

Conducting a Set of Usability Tests: Wireframing and ‘FirstClick’ Testing

04/16/2014 by Bob Bailey Leave a Comment

Once the problems were better understood, and several solutions had been proposed, we created several competing wireframes to see which would best elicit the success levels we were seeking. All of these eventually were combined into three homepage wireframes (A, B, and C). After only one test, B was eliminated and the final two wireframes (A and C) were tested ‘head-to-head’.

We had 65 participants attempt to complete 136 scenarios (68 using Wireframe A and 68 using Wireframe C). Each participant spent about one hour completing the scenarios using Wireframe A and then Wireframe C, or Wireframe C then Wireframe A. [Read more…]

Conducting a Set of Usability Tests: The Baseline Test

04/09/2014 by Bob Bailey Leave a Comment

The original Baseline Test was used to establish the human performance and user satisfaction levels for the existing site. We also used the baseline test to help us understand some of the major usability issues that would help to guide future changes to the new homepage. This was consistent with our usability and user-centered design approach.

The Baseline tests took place in August 2006 at three different locations in the United States (Washington DC; Atlanta, Georgia; Ogden, Utah). All were in-person usability tests, and were conducted in government usability labs. Because of the type of data we were collecting, we elected not to use remote usability testing.

The participants were both federal employees and people who had no affiliation with the government. We tested a total of 68 participants that included: [Read more…]

Conducting a Set of Usability Tests: Setting-Up for the Tests

04/02/2014 by Bob Bailey Leave a Comment

A few years ago (2007), while consulting with the Centers for Disease Control and Prevention (CDC), they elected to revise and update their website, CDC.gov. One of their primary goals during this redesign process was to optimize the usability of the homepage, and some of the second-level and third-level pages.

We organized a six-person user experience team^[1], and created a plan to ensure that major, meaningful usability-related activities would be appropriately carried out. Over a six-month period, we conducted the following major usability activities:

Conducting a review of past usability studies on the current CDC.gov site
Interviewing users, stakeholders, partners, and web staff
Conducting detailed analyses of web, search and call logs
Analyzing the user survey data from the American Customer Satisfaction Index
Surveying the ideas and attitudes of CDC leadership, employees and web staff
Conducting a card-sort activity
Conducting parallel design sessions
Producing a series of wireframes
Creating graphically-oriented prototypes
Conducting several usability test and evaluation activities

[Read more…]

100 Usability and UX Testing Tools

02/05/2014 by Bob Bailey Leave a Comment

We conducted an exhaustive review of software tools on the Web created for conducting usability tests and/or usability evaluations. We found over 100 tools.

As it relates to usability testing, the terms ‘test’ and ‘evaluation’ have very different meanings. When conducting a test, you can collect considerable objective data; but when conducting an evaluation, you are usually limited to collecting a limited amount of subjective or ‘softer’ observational data. The ‘tests’ provide an opportunity to make changes, then to retest, and then to statistically determine if reliable improvements actually have occurred. The ‘evaluations’ can assist you in identifying possible weaknesses in a site, but are less helpful in providing an objective way to statistically measure any improvements once changes are made.

To make it convenient for you to find the tool in which you are most interested, we developed a way to filter and sort all the tools.

[Read more…]

FirstClick Usability Testing

10/08/2013 by Bob Bailey 32 Comments

In 2006, the Centers for Disease Control and Prevention (CDC) elected to revise and update their website (CDC.gov). One of their primary goals during this redesign process was to optimize the usability of the homepage, as well as some of the second-level pages. As part of this usability testing process, Cari Wolfson and I conducted a series of usability tests – one of these was the first FirstClick usability test. A more detailed account of this testing program can be found in a later blog posting, and is also found as Case Study 10.3 in Tom Tullis’ and Bill Albert’s book, Measuring the User Experience (2008).

We studied the factors contributing most to each person’s success on a task. By looking at click streams and success rates, we found an interesting relationship between the user’s success in making their first click and their eventual success with the entire task.

[Read more…]