Website Performance Benchmark & Customer Research – Stage 2

In November 2011, the results came in for Stage 2 of our user testing. We were eagerly anticipating these results because our two proposed designs for the home page and search results page were going to be under scrutiny. Several scenarios were possible – either one design would be the clear favourite, both designs would be  equally favoured; or people would like neither design   and we would be back to the drawing board.

So what happened? Both home pages were well received, yet there was no one clear preference. 77% liked the Home page 1 design (faceted search where popular services are displayed and have a dedicated search tool) and 79% liked the Home page 2 design (one large search box).

Design 1 - faceted search

Design 2 - single search

Compared to Stage 1 testing (which looked at the current version of vic.gov.au) where the overall satisfaction was 57%, this is a clear improvement. Key findings from Stage 1 highlighted that end users had difficulty knowing where to start to find information on our current home page – and this is the key difference. Both of the proposed designs offered the users a clear starting point to find information and clear navigation.

Testing slightly differed to Stage 1 due to the fact that a functional website was not available. The number of tasks were reduced as users did not have a functional website and were shown the alternative design at the end of their tasks. For testing, html pages were created for each of the proposed designs for the home page and corresponding search results pages. This would results in users using their mouse to indicate where they would start on the home page to find the information but not actually completing the task. The percentages indicate the high confidence and satisfaction (Home page 1, Home page 2).

This second round of testing also set some tasks for users to achieve, though the number of tasks was lower as a full website using the new design was not available yet. Overall results and satisfaction for the tasks also scored well:

1. Click where you would start to find the jobs home page for the Department of Health. (66%, 78% confidence and satisfaction)
2. Click on the section of the page that you would normally click, if you were looking to find information on the Great Victorian Bike Ride. (82%, 80% confidence and satisfaction)
3. Click on the section of the page that you would normally click, if you were looking to find a phone number for the food safety hotline. (83%, 80% confidence and satisfaction)

After careful analysis of the results, we have chosen the design with the one large search box. The single search box does not require people to identify what type of information they are searching for, or where it comes from, yet the search will be very sophisticated and produce results from five different repositories of data. Our rationale for this choice is:

• It answers the biggest perceived problem from the first round of testing which was “ I don’t know where to start”
• People told us they really liked the clarity and simplicity of the uncluttered page
• While the faceted search works well now as a design, as new services become available  we could easily fall into the trap of creating a cluttered home page again.

So this is the road we are heading down – we look forward to Stage 3 testing which will include some functional wire-frames of the new design.