Behavioural insights case study: supporting VET students

We tested whether sending behaviourally informed SMSs to students would increase student retention.

The problem

The vocational education and training (VET) system is vital to building a strong economy and a healthy society. It seeks to equip students with the skills they need to succeed in the workforce and contributes to a pipeline of talent to meet the needs of employers in a growing economy.

In 2018, around 240,000 students enrolled in VET courses in Victorian Technical and Further Education Institutions (TAFEs). However, many students who enrol in Victoria’s VET system drop out early or fail to complete their course. In some courses fewer than half of the students who enrol go on to complete their course. This represents a lost opportunity for the students and potential employers, and a poor return on public investment in the sector.

What we did

In 2018 we partnered with the Department of Education and Training and three TAFEs to test whether sending behaviourally informed SMSs to students would increase student retention.

Focussing on courses for ‘in-demand’ skills that have a history of low student retention (e.g. early childhood education and community services), we worked with TAFEs to send students personalised SMSs every week for up to 18 weeks. To assess the impact of the messages, we ran a randomised controlled trial where students were randomly allocated to one of three groups:

  • Control Group: Students received a single introductory SMS informing them that they may receive weekly SMSs as part of a trial of new student support services. It provided the opportunity to opt out of receiving further SMSs.
  • Group A: Supportive SMSs: Students received the same introductory message as the Control group, plus weekly SMSs. These were designed to support student engagement drawing on concepts such as motivation, positive reinforcement, self-determination, growth mindset, grit, practical advice, and planning prompts.
  • Group B: Supportive and personalised SMSs: Students received the same series of SMSs as those in Group A, with additional references to the job associated with their specific course (e.g. nurse, childcare worker).

Text message example: Hey Kerry, staying positive is #keytosuccess in your course. Remember to focus on what you CAN do, and build on what you DO know. Practice = confidence = success!

Phone image with text message example: Hey Kerry, staying positive is #keytosuccess in your course. Remember to focus on what you CAN do, and build on what you DO know. Practice = confidence = success!

What we found

Overall, the results showed there was no difference in student retention across the three groups. Students who received the behaviourally informed messages (Groups A and B) were no more or less likely to continue in their course for the period of the SMSs than those students who did not receive the behaviourally informed SMSs (Control group).

Equally, students who received SMSs with references to the job associated with the course (Group B) were no more or less likely to continue in the course than those who received messages without such references (Group A).

This figure shows the proportion of VET students who completed their course. 91.4% of the students in the Control group completed their course; 89.9% of the students in Group A completed their course; 91.8% of the students in Group B completed their course. The proportion of students who completed their course was not statistically different in Group A and B compared to the Control group (p > 0.05).

We also ran a survey at the end of the trial to gauge student feedback. The results indicate that the vast majority of students (75%) thought the SMSs were a good idea and should continue in the future.

We found that students valued the SMSs containing study tips and motivational messages above others.

Some of the qualitative responses from students further emphasise these points:

“Great little motivational tactic, was helpful and useful with information that I didn’t know I needed. Keep it going!”

“I enjoyed the motivational messages and the ones that reminded me that I wasn’t alone and that my whole class were in the same position.”

Other qualitative data showed that many students engaged deeply with the SMSs and that they contributed to student wellbeing. This was particularly evident for students who were finding their studies challenging.

What's next?

While the results of this trial did not show that the messages produced a significant impact on student retention, there are several positive outcomes that can guide future work.

It showed the potential to integrate the behavioural insights approach with TAFEs’ strategic focus on student experience. It demonstrated the value of testing innovative policy solutions on a small scale to build an evidence base prior to wider adoption.

The project also helped build the capability of staff at both the Department of Education and Training and TAFEs to apply a behavioural approach on an ongoing basis.

Thanks

We thank our partner the Department of Education and Training.

Updated