This Fire Services Outcomes Framework – Progress Report provides the Country Fire Authority (CFA) and Fire Rescue Victoria (FRV) Quarter 4 (Q4) FY 2022-23 fire services performance indicator results, in line with FSIM’s legislative requirements under section 141 of the Fire Rescue Victoria Act 1958 (FRV Act). 

This report presents quarterly results provided by CFA and FRV against their respective performance measures and agency commentary on quarterly performance. Where possible, FSIM has included historical data against measures provided by agencies from 1 July 2020 as a reference for the reader, noting that historical data is significantly limited due to changes in agencies’ outcomes frameworks in Q2[1]  and other incidents outlined below.

FSIM noted in Quarters 2 and 3 that a lack of data impacted FRV’s ability to report performance results and progress toward outcomes. While FRV continues to experience some issues accessing data following the cyber-attack incident in December 2022, this quarter they have been able to report against 22 (of 26) indicators, a significant improvement from previous quarters in FY2022-23. Additionally, to address some remaining information gaps in Q4, FRV obtained data from Emergency Services Telecommunications Authority (ESTA) and undertook comparative reporting of 5 measures relating to false alarms, structure fires, and response times. FRV report that work continues to restore reporting systems.

Reporting on outcomes

Section 140(1) of the FRV Act requires CFA and FRV to each prepare an outcomes framework that sets out their respective outcomes-based fire service performance measures. In developing and refining their outcomes frameworks, CFA and FRV each determined their strategic organisational priorities and the associated indicators and measures of performance to be included in those frameworks. 

Tracking the progress of agencies towards their outcomes is a long-term proposition and trends or patterns may only be observable over an extended period. FSIM continues to update its outcomes reporting approach and may highlight particular indicators in the summary by exception where a result:

• shows a continuing off-track trend over repeated quarters
• demonstrates a notable achievement
• varies more than 5% from the agencies baseline or average over time
• is a significant change from the previous quarter (positive or negative).

FSIM notes that some performance measures are influenced by factors outside the control of CFA or FRV and in some instances, agencies may not be able to provide substantiated reasons, supported by evidence for changes to results.

In future, FSIM will work with agencies to identify outcomes that may benefit from a deeper assessment of progress, taking into account relevant data limitations and outcome framework amendments.  

Maturing outcomes frameworks

The CFA and FRV Outcomes Frameworks signal the strategic priorities of each organisation, and each provides a set of indicators and measures to demonstrate progress towards their respective strategic goals. The ongoing review and refinement of both Outcomes Frameworks is important to assist agencies to assess progress towards delivering against their objectives and support informed, evidence-based decision making. Updates to the agencies’ outcome frameworks for Q4 are outlined below.          
 

CFA’s Outcomes Framework 2022-23

CFA has reported on a total of 29 quarterly indicators and 17 annual indicators. Baselines for eight indicators will be introduced in future quarterly outcomes framework reports when historical data becomes available.[2] CFA introduced targets for five indicators to its current Outcomes Framework (2022-23).[3] Baseline data will continue to be provided in CFA outcomes progress reporting where targets are introduced, providing additional context. 

FRV’s Outcomes Framework

Ongoing impacts of the December 2022 cyber-attack on FRV’s systems (outlined below) continues to limit FRV’s ability to report retrospectively and continued to impact reporting this quarter. However, in Q4, FRV were able to provide data against 22 of its 26 indicators. This is a significant increase compared to seven indicators FRV reported on in Q3 2022-23. FRV also reported on 36 out of 46 measures. Measures are aimed to inform progress against a given indicator.

During the fourth quarter, FRV introduced a year-on-year (YoY) status where quantitative data is available to highlight positive and negative trends, allowing for a comparison between the annual results of FY 2021-22 and FY 2022-23.

FRV report that a YoY status could not be reported for 17 measures out of 36 reported in Q4. This is a result of the cyber-attack on FRV[4] leading to insufficient data to calculate a year-on-year comparison and industrial actions restricting reporting of certain measures[5] from the period of 27 October 2022 to 30 April 2023.

Outcomes framework Q4 results commentary overview

FSIM highlights indicators with a variance of more than 5% from the agencies baseline or average over time and indicators that demonstrated a notable achievement. As reported above, FRV’s data limitations constrain FSIM’s ability to assess FRV quarterly progress on all indicators. 

Indicators with a variance of more than 5% from the agencies baseline or average over time

Fire suppression

CFA reported an improvement in indicator 2.2.3 (average time spent to suppress structure fires) from 73 minutes 1 second in Q3 to 65 minutes in Q4. Despite this improvement, CFA did not meet the 58 minutes 4 seconds baseline in Q4. CFA has not identified any specific trends or factors that have driven the 2022-23 Q4 performance for this indicator. CFA recognised that there are a significant range of factors that can influence this outcome including fire progression at time of arrival, building type, timeliness of supporting services (such as gas and power) and the presence of hazardous materials. FSIM understand that many of these factors are unable to be quantified.

FSIM notes the Q4 result in 2022-23 reflects increased time spent suppressing structure fires when compared to Q4 results from FY 2020-21 and FY 2021-22. 

Response to structure fire

In Q4, due to the cyber-attack, FRV have replaced Australian Incident Reporting System (AIRS) data with Emergency Services Telecommunications Authority (ESTA) data and have reported a decline in quarterly and annual average performance for response to structure fire incidents within 7.7 mins (measure 2.1.3a).[6] FSIM observes that the Q4 result was 6.87% below the comparable quarterly results in Q4 2021-22 and 4.3% below the annual average results recorded for 2021-22. FRV advise that the electronic automated Station Turnout System was compromised during the December 2022 cyber-attack, and the return to manual dispatch processes, such as pagers to dispatch crews, increased the time it took firefighters to turnout from the station. FRV reported that the time taken for firefighters to arrive on scene after leaving the station remains constant. FRV advise that manual processes will continue to be used to dispatch crews until the FRV electronic Station Turnout System is back online. 

Firefighter demographics

In Q4, FRV have reported on two annual indicators that demonstrate the age profile of FRV staff (3.2.3a) and the age profile of staff turnover (3.4.2a). Measure 3.2.3a (workforce profile) indicates that 1,242 of 4,809 (25.8%) total operational and non-operational FRV staff are aged above 55 years. In addition, 1,084 of 4,809 (22.5%) are aged between 45-55 years old. FRV have also reported that 116 staff over the age of 55 years have exited FRV in 2022-23 (measure 3.4.2a), which includes 90 operational and 26 non-operational staff. Based on the age profile for 2022-23 (annual results), FSIM recognises that in 10 years-time, FRV will have a significant number of retiring staff.

FRV recognises the age profile of its operational staff and the need to plan and prepare for significant workforce retirements over the next five years. FSIM notes that under the Year Two to Five Fire Services Reform Implementation Plan – action 5.4 is developing a 5-year FRV Strategic Workforce Plan (Operations) for the period 2023-2027 to address its workforce requirements.

Indicators that demonstrated a notable achievement

After Action Reviews

In Q4, CFA recorded a significant increase in results for 4.4.1 (number of After-Action Reviews) compared to Q1, Q2 and Q3. 

CFA report that this increase in performance is due to additional CFA facilitator training and major flood events that required the facilitation of After-Action Reviews and the CFA post-seasonal debrief. FSIM notes that an increase in After Action Reviews informs continuous improvement in operational service delivery for the CFA and the wider emergency management sector.  

Smoke alarms

In Q4, CFA and FRV have reported on results for smoke alarm campaigns during the 2022-23 FY.   

CFA reported a significant improvement in performance for the number of homes with operational smoke alarms (indicator 2.3.6), which includes the installation and maintenance of smoke alarms for vulnerable people who require assistance to maintain or install smoke alarms in their homes. Key contributors to this improvement includes CFA investment in skills and resources available through training provided to CFA members and the use of digital service delivery tools for reporting and smoke alarm installation visits. CFA advises that there are a variety of approaches that regions use to inform this result, for instance South-West Region continued to grow and develop a Residential Advice Visit Service and CFA brigades in South-East Region ran a campaign to identify people at higher risk and ensure they had working smoke alarms.

FRV report on results from an online survey of Victorian homeowners (conducted between 8 and 19 September 2022) to determine the effectiveness of the joint CFA and FRV smoke alarm awareness campaign. FRV report that the campaign received a good response from the community. The survey results informed recommendations on future smoke alarms awareness campaigns considering advertising channels and message refinement.

FSIM supports CFA and FRV collaboration in providing smoke alarm awareness and preparedness initiatives to improve community safety from fire in the home.

Footnotes

Definitions

Reference to fire services agencies

References to the fire services in this document relate specifically to CFA and FRV. Where more than one fire services agency is referenced in this document, they are listed in alphabetical order.

Comparison between agencies’ performance against published Outcomes Frameworks

As the two outcomes frameworks are based on the agencies’ own strategic and operational priorities and data, comparisons between the agencies should not be made.

Legislative requirements to report on outcomes

The performance measures that CFA and FRV report on are set by the agencies in their outcomes frameworks. Section 140(1) of the FRV Act requires CFA and FRV to each prepare an outcomes framework that sets out their respective outcomes-based fire service performance measures. Section 140(4) of the FRV Act further requires CFA and FRV to prepare and submit to FSIM a quarterly update that sets out their performance against their performance indicators.

Reporting and data limitations

CFA has reported on its Year Three Outcomes Framework in Q4. FSIM will continue to use historical data published in previous quarterly progress reports to track indicator trends over time where indicator business rules and baseline calculations in the Year Three Outcomes Framework remain consistent with earlier CFA outcomes framework iterations.

FRV cyber-attack

Throughout Q2, Q3 and Q4, FRV’s ability to report on outcomes performance indicators and measures has been significantly reduced due to the cyber-attack in December 2022. Prior to the cyber-attack, FRV reported on operational performance measures using AIRS data. FRV reported that its access to AIRS remained unavailable in Q4. Although FRV continue to experience the effects of the cyber-security attack, FRV has obtained data from ESTA to report on five operational performance measures in Q4, including:

• 1.2.3 No. of properties with six or more false alarms on different days (rolling 12 months)
• 1.3.4 No. of structure fire incidents
• 1.3.3 No. of incidents other than structure fires and other non-fire related incidents
• 2.1.3a Percentage of response to structure fire incidents within 7.7 minutes
• 2.1.3b Percentage of response to EMR (emergency medical response) within 9.2 minutes.

FRV advises that it continues to work towards restoring AIRS functionality to facilitate future and retrospective reporting.

FRV has provided data and commentary against 36 of its 46 measures. There are seven measures where one or more quarters are not reported in the 2022-23 FY.[1] There are eight measures where annual results for 2021-22 FY (8)[2] are not recorded due to the cyber-attack.

Industrial action impacting data quality

FRV reported industrial action between 27 October 2022 and 20 April 2023 has impacted results reported on under measure 1.3.3 (Decrease in preventable non-structure fires and other hazard related incidents). Under the industrial action, certain ‘False alarms and False calls’ and ‘Good intent’ calls are not reported on scene at the time of the incident. Such calls are reported under the category of 'Other’ during the industrial action time-period which has affected reported results for Q2, Q3 and Q4.

Report structure

This report comprises FSIM summary (section one) and the CFA and FRV performance measure results received for this quarter (sections four to six). Where possible, FSIM presents performance results and historical trends. FSIM also includes CFA and FRV explanations of measures (business rules) and where appropriate, provides confirmation of whether the agency met or did not meet targets or baselines. FSIM also includes CFA and FRV commentary on indicator progress.

Publication of reports

FSIM publishes quarterly reports at Fire Services Implementation Monitor publications.    
 

In Q4, CFA has reported on quarterly and annual indicator progress against the CFA Outcomes Framework 2022-23. For indicators that are reported annually, CFA’s Year Three Outcomes Framework uses an annual baseline which calculates the average result for results over a yearly specified time-period (1, 2, 3 or 5 years). CFA’s Year Three Outcomes Framework considers baseline data (where available) which is a calculation of the average result (1, 2, 3 or 5 years) over the equivalent period. Where baselines are reported in lieu of targets, CFA’s aim is to either ‘increase’, ‘decrease’ or ‘maintain’ (as appropriate) performance with consideration to baseline data.

CFA categorises Year Three Outcomes Framework indicators into four domains:

1. Strong relationships to keep communities safe
2. A responsive and innovative volunteer service
3. High performing volunteer and paid workforce
4. A continuous improvement culture.

Data is available from page 8 in the report document.

In Q4, FRV report on both quarterly and annual results, including the annual results from the previous financial year (FY 2021-22) for some indicators.

FRV categorises Year Two Outcomes Framework^[1] measures into three domains: Prevention and Preparedness; Response and Recovery; and Organisational Excellence. FRV began reporting outcomes performance for the FY2022-23 in Q4.

Due to the cyber-attack on FRV IT systems, FRV’s ability to report this quarter is limited to 22 (36 measures) of its 26 indicators. The reported results for each measure are compared to the results for FY 2021-22, providing an update on FRV’s performance and progress toward achieving the outcomes in each domain. Year-on-year (YoY) status have been included (where available) for each measure to compare performance on an annual basis.

FRV does not report a baseline or target for its indicators and measures and does not qualify results as “on track” or “meeting baseline”.

Data is available from page 30 in the report document.