Measure how content performs

Before you begin

What is performance measurement?

Performance measurement is the deliberate act of setting goals and finding out if your digital presence (for example, a website) is meeting them.

Tip: Measure social media and online engagement

More information on how to measure social media and online engagement can be found in the How-to guide: How to use social media and the How-to guide: How to set up a public consultation.

 

Why measure performance?

Continuous improvement

Every service must aim for continuous improvement. It’s important to have discussions about a service’s strengths and weaknesses regularly. The right metrics will make sure your decisions to improve your service are supported by evidence (data).

Benchmark performance

The metrics proposed in this document can help you benchmark performance with other countries, like the Gov.UK and the analytics.us.gov.

Qualitative feedback is also valid

Quantitative measures can’t measure everything. Information on how to get qualitative feedback can be found in the How-to guide: How to do user experience (UX) research.

What you find is also crucial for the WoVG

Even though you’ll be collecting data for your website, a WoVG view of all digital activity allows for better decisions and planning by considering, for example, the kinds of online services citizens prefer, and the ones they prefer to use face-to-face.

What does the Victorian Government recommend?

All Victorian Government online services should develop goals and methods of measuring success when planning a service or product. These goals and measures should also be reviewed regularly throughout the service lifecycle.

Consider the entire user journey. Include both online and offline channels. Also, include where users go when they can’t complete a task.

What standards must be met?

 

Google analytics tracking codes

To allow for a WoVG view of online service delivery, each website you operate must report into the WoVG analytics account using the WoVG analytics tracking code. Check your department’s intranet to find out who looks after analytics or contact your Digital Management Committee (DMC) or its equivalent.

Note: For your social media channels, we suggest you monitor these using a tool designed for social media analytics, such as SproutSocial.

 

Back to top

Step 1: How to decide what to measure

If you’re a service and product owner, at a minimum, you should measure these five key performance indicators:

User satisfaction

Are users satisfied with the service you’ve provided?

Time to completion

The average time it takes for a user to get from the start to the end of a transaction.

Completion rate

The percentage of started transactions completed. A crucial performance measure.

Cost per transaction

The total estimated cost of your service per month (this varies for some services) divided by the number of completed transactions. Cost per transaction measures on their own may give you a rough guide for later cost reductions but it’s the portfolio of services against cost that is the really critical measure.

Digital take-up

The percentage of people using your service online in relation to other channels, for example, paper or telephone.

Tip: Are these measures always relevant?

For some services and products, not all of these measures are relevant. For example, for some content sites where a user task is limited to one page, it may only be useful to measure user satisfaction on that page. For other content sites, you may be able to aggregate individual pages together to measure completion rate, or time to complete

Time-to-successful-completion metrics are not always customer focused. For example someone may begin a registration process but not have the time to complete it all. They may take a week and not be unhappy about that. Countering this is the cost to your systems of having a transaction ‘open’ for an extended period of time.

 

How to measure relevance

Consider your goals for the website, and ask yourself:

  • Findable: After the user perceived the product as valuable (offline), could they find the product or service in Google etc quickly and easily?
  • Accessible: After the user found the product or service, was there anything that prevented them from accessing it (technology, language)?
  • Usable: After the user accessed product or service, could they complete their task? Did they complete it quickly and easily?

How to measure the success of your content

With digital products or services that mostly contain information, measuring success is more difficult.

Combining a range of metrics and collecting user feedback will help you find out if your content is actually helping people do what they need to do. If not, why not? And do you have the ability to act on this feedback?

Use these metrics to measure the relevance and success of your digital content  

Metric

Category (relevance, success)

Measurement tool

Percentage of users who entered the [task] page directly via an alias, referral, organic search, campaign, section landing page, homepage or internal search. (Note: Only shows you who found it, not who tried and failed.)
 
Findable, Valuable Google analytics or another dedicated analytics tool
Percentage of pageviews with search Usable Google analytics or another dedicated analytics tool
 
Percentage of users who exited the website via the [task] completion page Usable Google analytics or another dedicated analytics tool
 
Percentage of video/audio watched Valuable, Usable Google analytics or another dedicated analytics tool
 
Comments and feedback provided to other channels (call centre, email inboxes) Findable, Usable Dependant on the channel

 

How to measure if transactions are successful

When a user completes a task, such as making a payment, submitting a form or downloading a document, this is called a ‘transaction’.

The transactions might also be part of a longer series of smaller transactions, such as case management activity. Measuring what happens to your transactions will help you discover how well your online product or service is helping people complete important tasks.

Use the table to decide the mix of metrics, categories, and measurement tools you need. Once you decide what to measure, you'll need to make a judgment as to what success looks like for your digital presence. For example, if users abandon a particular step of a multi-part form it indicates something is amiss, and you need to research the user experience  to find out what.

The same applies if you notice a high number of user errors — but you'll have to decide how many user errors are the trigger for more thinking on your part.

Other countries use much the same metrics, such as the Gov.UK and the analytics.us.gov.

Use these metrics to measure if digital (online) transactions are successful

Metric

Category (relevance, success)

Measurement tool

Percentage of abandonment of each step of [task]
 
Accessible, Usable Google analytics or another dedicated analytics tool
Percentage of user errors of each field [task]
 
Accessible, Usable, Satisfying Google analytics or another dedicated analytics tool
Comments and feedback provided to other channels (call centre, email inboxes) Findable, Usable

Dependant on the channel

 

You can also use metrics like the frequency of search requests, search result clickthroughs, search result clickthrough order (to determine accuracy and relevancy of search results) and analysis of search terms to see if people are finding what they’re looking for.

Some sites will measure by linear navigation up and down information trees to measure the effectiveness of their site’s layout. It’s the equivalent of people wandering around on the shop floor of a retail store – where someone observes this and asks ’Can I help you?’.

Back to top

Step 2: How to measure performance

Measurement 1: User satisfaction

User satisfaction measures if people are satisfied with your online service. By tracking user satisfaction you can find out what users think about the service and exactly what parts cause them problems or friction. (Although in some cases you might discover a pain point you can't do anything about.)

As a minimum, place a form measuring the helpfulness of your page at the end of every content page (above the footer), with the option for them to add a comment.  Here's what we recommend:

Screen grab of a helpfulness survey with a send button not submit

Our experience with this binary ('yes' or 'no') approach is that is simplifies the task of giving feedback. As the feedback tends to be specific, it provides more useful feedback than more complicated approaches, such as choosing from five or even three responses.

Measuring performance for multi-stage forms

At a minimum, offer your user a feedback option once they see the transaction ‘success’ confirmation page. A more robust approach is to capture data on people who abandon a form part way through. To do this, offer them the option to give feedback all the way through — not just when they've successfully submitted the form. 

Tip: Tracking content for the whole service 'journey'

Services can have many different points of contact for users, and several stages. For example, email , a phone call to your call centre, or a visit to an office. Make sure there’s a way to collect feedback from each point of contact, not just the digital elements, such as a form on web page.

 

Measurement 2: Time to completion

Time to complete measures how long it takes for someone to get from the start to the end of a transaction.

Time to complete is often measured by setting a cookie that stores the user’s session start time and updates every time the user completes an action as part of the transaction.

Measurement 3: Completion rate

To calculate your completion rate:

  • count the number of completed transactions
  • divide it by the total number of transactions (including partially completed or failed ones)
  • show the result as a percentage

Tip: Measuring your page load times might alert you to a poor user experience that isn’t showing up in your stats for completion rates.

 

When a transaction is completed

A transaction is completed when someone finishes the task that your digital service provides. For example, when a user:

  • submits an application
  • makes a payment
  • makes a booking

The service should tell the user the transaction is complete with a confirmation or ‘thank you’ message.

A user’s application doesn’t have to be successful for you to consider the transaction complete. For example, a transaction could be considered complete when the user has submitted an application — even if their application is unsuccessful.

You’ll dramatically improve your users’ completion rates if you design effective error handling, for example, the options your service will offer the user if they have to go back a step in a form.

When a transaction has failed

A failed transaction is when a user gives up before reaching a defined end page (often called a ‘confirmation page’). This includes when a user abandons the service from an error page.

Configure your service to measure completion rate

To make it easier to analyse user behaviour, give each step in your service a unique URL so you can identify them in your analytics tool. This becomes more important when you allow people to maintain their transaction state across multiple devices, and if they continue a transaction from another device.

Measure from the start of a transaction

Most services will have a start page, which should be the only way to access the service. You should usually only consider a transaction to have started when the user proceeds from your service’s start page.

Make sure users can’t bypass the start page via links or search engine results.

Know when a service ends

Services often have multiple end points. You must have appropriate end pages at each of them.

For example:

  • one for those who are eligible (that tells them what to do next)
  • one for those who are not eligible (that suggests other forms of support)

When a user reaches one of these end pages, you must consider the transaction to have finished.

Counting users who 'save and return'

If your service allows users to save a transaction and finish it later, make sure you can match transaction starts to their subsequent completions.

For example, in Google Analytics you can use cohort analysis and custom dimensions to track the number of times a user returns to complete a transaction on your service.

Measurement 4: Cost per transaction

The cost to government each time someone completes the task your service provides is known as ’cost per transaction’.

Some great advice on measuring cost per transaction is on the Gov.UK website. 

Calculating your service’s cost per transaction allows you to:

  • measure how cost-efficient your service is
  • use your results to change the service and make it more cost-efficient

Tip: While we usually try and move people to digital because it is assumed to be cheaper, it’s not always true.

It’s the full cost of a transaction across all your service channels that's important to understand.

Some transactions might take seconds online, but take several weeks to complete offline, for example, receive a grant application online, then offline, assess the applications, interview and notify each applicant.

 

Measurement 5: Digital take-up

You should plan to continually increase digital take-up if the situation warrants it, for example, you need to move traffic from one channel to another.

Digital take-up is the percentage of people using government services online in relation to other channels, for example, paper or telephone.

  • find the number of completed digital transactions over any fixed period (include digital transactions where assisted digital support was used)
  • divide that number by the total number of transactions from all channels in the same period
  • show the result as a percentage

Step 3: How to set up and report performance measurement for the WoVG account with Google Analytics  

Use these instructions to comply with the mandatory standard for Victorian government for adding the Google Analytics tracking code to your website.

Before you start, have your Google Analytics Universal Access (UA) number from your site’s account ready. If you need to set up a new UA, follow these instructions from the Google website.

Installing the Google Analytics tracking code for WoVG

The task has 3 parts:

  • part 1: update a JavaScript file (using the template ) and save it to your JS library

analytics-include.txt (~40kb)

  • part 2: check you have a JQuery library file in your JS library
  • part 3: insert JavaScript references to the JQuery and the analytics JS files in the footer of each web page or template

Part 1. Update the JavaScript file with your details

For this task you'll edit the JS file supplied (analytics-include.txt) using a text editor like Notepad. Save the analytics-include.txt file to your computer. 

To update the JS file (analytics-include.txt) in a simple text editor (don't use MS Word):

Step 1: At around line 30, change the 'rollup' statement,

'var DEPTprofileNumber = "UA-000000-2"; // Set Department roll up profile number - if none exists, replace with "UA-000000-0"'.

Your department may be using a ‘roll up profile number’ to aggregate analytics for all its subsites, in the same way the WoVG Analytics account does. Contact your department's digital admin to find out what your department's UA/profile number is, and overwrite UA-000000-2 with the department UA number. If none exists, replace 'UA-000000-2' with 'UA-000000-0'. 

Step 2:  At around line 31, is where you need to overwite the 'UA-000000-3' with your Google Anaytics UA number for your site. ; ‘var SITEprofileNumber = "UA-000000-3"; //'.

Step 3. Insert your site’s web address where it occurs in the code. There are five places. To find these, search for ‘yoursite.vic.gov.au’. Note also the comments in the JS file about not using ‘www’.

Step 4. At around line 40: Replace 'dedjtr' in ‘var dept = "dedjtr"' with your department’s initials (the JS file lists every valid option. Don’t use any other code). Make sure to read the notes in the code about subsites too.

Step 5. Now you've updated the JS file, save it to a folder on your web server that any browser can access. Make sure to save it with a .js extension.

Parts 2 and 3: Check the JQuery library works and update the footer of the web page

Now that you've an up-to-date JS file you can call from any HTML page, you’ll need to add two lines of JavaScript to the footer in your web page. (The better option of course is to add this to the footer of your content templates in your CMS.)

In the footer: 

  • make sure you list the location of the newly updated JS file (analytics-include.js) and its name. 
  • check you have a JQuery library set up and working (linked to from the JQuery in the footer of every page)

Here’s a sample JS code for a page footer. Note the JQuery reference must come before the JS reference to the analytics script.

<script src="http://yoursite.vic.gov.au/_path/to/js_
file/jquery-1.9.1.min.js"></script><!-- jquery-1.9.1.min.js -->
<script src="http://yoursite.vic.gov.au/_path/to/js_file/
analytics-include.js"></script><!-- analytics-include.js -->

Lastly, view some pages on your site and check your Google Analytics account to check it's working (it might take an hour or two).

For more information: contact@dpc.vic.gov.au

Step 4: Check you meet the minimum requirements

Here's the checklist of our essential (mandatory) internal and external requirements. Use the checklist to:

  • see if you've done everything you need to do to get your work signed off
  • help a vendor work out what they have to do
  • help a colleague understand what we have to do, and what we can't do

I have:

  • defined what’s important and actually useful to measure
  • set targets for the next financial year
  • implemented and tested ways to track each measurement
  • set up my analytics account to send statistics to the WoVG account

Glossary

Transactions

Transactions are an exchange of:

  • money
  • information
  • permission
  • goods or services

Transactions on your service will have clearly defined start and end points. They will also result in a change to a government system, for example, someone’s personal information or details of an interaction are stored in a register or database.

The following activities don’t have clearly defined start and end points and aren't recorded in a government system, so they're not transactions:

  • general advice or enquiries
  • informal complaints
  • visits to websites

Back to top

 

Related How-to guides

How to use social media

How to create a content strategy and manage content online

How to do user experience (UX) research

How to design and develop a digital presence

Other good reads and tools

Digital Transformation Agency's (DTA) general guide to performance testing

UK Gov Digital Service Standard (see criteria 15, 16 and 17)

US Gov Digital Analytics Program

Join the conversation on digital

Get advice and share your insights with other digital practitioners on the WoVG Digital Group on Yammer (VPS access only).

Can’t access Yammer? Contact us by email: contact@dpc.vic.gov.au (We may post your comment on Yammer for general discussion. Please tell us if that’s not OK).