Stage 3 testing occurred during July 2012 as we had a functional website with our new search engine working in a User Acceptance Testing environment. Our new home page (see image below) features the large search box that users liked in the Stage 2 testing.
We were keen to test the same scenarios using our new search and to see whether users liked and understood our new-look search results page that features categorised search results. See an example of our new search results page below.
Our new search engine was now searching 5 repositories:
- Contacts & Services Directory
- Events Calendar
- Ask a question
Search results were displayed in a redesigned page clearly indicating which repository a search result was sourced from. If there were no results from a repository, then that particular repository did not display.
We were also keen to see whether search remained more popular than browse. We have kept the browse functionality on our new site, but have removed the persistent navigation that occurs on our current site. Instead we are now providing a dedicated browse page – see image below.
So what happened this time? Initial perception of the new home page was positive. Users felt that there was a definite clear starting point on the homepage. The large search box is a clear call to action for the user to type in their search term(s) and search the site. There was some ambivalence about whether the home page was cluttered. This is something that we will continue to test in Stage 4 (post-launch). As users got used to the website, they became more successful in completing the tasks. Task 1 had low satisfaction (9%) but the final task was 68% satisfaction.
Generally, users searched (Task 1 – 34%, Task 2 – 48%, Task 3 – 37%). Many users also used the new feature “Recent search terms”.
Several factors affected the overall satisfaction scores:
- Search engine still in the process of fine-tuning (at time of testing)
- Federal sites were favoured in search results over Victorian State sites. Going forward the new site will not include federal sites.
Some key findings were:
- 74% of users thought the home page had the right mix of information and direct links
- Users found the redesigned search results page logical. There was moderate satisfaction with the results page.
- Task 1:63% reported that results page was logical
- Task 2: 91% reported that results page was logical – more logical on second visit
- Task 3: 88% reported that results page was logical
- ‘Browse by topic’ page: meets customer’s needs. There were strong scores across all questions: layout, being uncluttered, and having an appealing design
Subsequent in-house user testing after all the issues with search had been rectified yielded much better results. Again, users generally searched (Task 1 – 100%, Task 2 – 37%, Task 3 – 100%) and where the users did not use search, they used the new feature “Recent search terms”. Task success was:
- Task 1 – 88%
- Task 2 – 100%
- Task 3 – 88%
The new site will be launched soon and one month after launch we will commence Stage 4 testing. We will continually test our new site to ensure that it meets users’ needs and that their satisfaction levels remain high.