This Fire Services Outcomes Framework – Progress Report provides the Country Fire Authority (CFA) and Fire Rescue Victoria (FRV) Quarter 2 (Q2) FY 2022–23 fire services performance indicator results, in line with FSIM’s legislative requirements under section 141 of the Fire Rescue Victoria Act 1958 (FRV Act).

This report presents quarterly results provided by CFA and FRV against their respective performance measures and agency commentary on quarterly performance. FSIM has included historical data against measures provided by agencies, where possible, dating back to the start of reform (1 July 2020) as a reference for the reader. FSIM notes that, in Q2, historical data is limited due to changes in agencies’ outcomes frameworks and other incidents outlined below.

Reporting on Outcomes

Section 140(1) of the FRV Act requires CFA and FRV to each prepare an outcomes framework that sets out their respective outcomes-based fire service performance measures. In developing their outcomes frameworks, CFA and FRV each determined their own strategic organisational priorities and the associated indicators and measures of performance to be included in those frameworks.

Tracking the progress of agencies towards their outcomes is a long-term proposition and many trends or patterns are only observable over an extended period. For this reason, FSIM has updated its outcomes reporting approach. For quarters one to three, FSIM will highlight particular indicators in the summary by exception where a result:

• shows a continuing off-track trend over repeated quarters
• demonstrates a notable achievement
• is a significant change from the previous quarter (positive or negative).

FSIM notes that some performance measures are influenced by factors outside the control of CFA or FRV and in some instances, agencies may not be able to provide substantiated reasons for changes to results within the three categories listed above. FSIM will continue to work with agencies to explore clearer linkages and explanations of contributing factors to results as their outcomes reporting continues to develop.

Starting in Q4 FY 2022–23, FSIM will initiate an annual overview of agency outcome progress. With the foundational elements of each agencies’ outcomes frameworks now in place, agencies can begin to develop a deeper understanding of trend data and meaningful insights towards outcomes performance.

Maturing outcomes frameworks

The CFA and FRV Outcomes Frameworks signal the strategic priorities of each organisation, and each provides a set of measures to demonstrate progress towards their respective strategic goals. The ongoing review and refinement of both Outcomes Frameworks is important to assist agencies to assess progress towards delivering against their objectives and support informed, evidence-based decision making.

CFA’s Year Three Outcomes Framework

In Q2, CFA commenced reporting on its Year Three Outcomes Framework. The CFA Year Three Outcomes Framework has refined 23 indicators from the CFA Year Two Outcomes Framework, which included updating business rules for three¹ indicators and revising all baselines.² The revised baselines will provide an opportunity to better compare quarterly and seasonal variation through time. The CFA Year Three Outcomes Framework also includes five³ new indicators that CFA considers important to inform strategic outcomes.

In this report, all CFA indicator results are reported as a quarterly total which provides consistency and clarity when comparing results across time⁴. CFA developed an annual target for one indicator⁵ and is considering targets for some indicators which may be used in future reporting. For some indicators, targets will provide a clear point of focus to better monitor and evaluate the effectiveness of CFA’s ability to meet its strategic objectives.

Given CFA’s transition to the Year Three Outcomes Framework report and resulting changes to baselines, FSIM has not reported historical trends for updated Year Three Outcomes Framework indicators. Further, FSIM has not included any baselines in the trend analysis charts as the data sets have altered. Given the changes to baselines across CFA’s indicators, FSIM has not elevated any specific issues against the above criteria to the summary.

FRV’s Outcomes Framework

FRV finalised its Outcomes Framework in FY 2021-22. Given the impacts of the recent cyber-attack on FRV’s systems (outlined below) reporting this quarter is very limited. FRV have reported against five measures.

FRV cyber-attack

A cyber-attack on FRV in December 2022 has had a severe impact on FRV’s information and computer technology systems and FRV’s Q2 FY 2022–23 reporting. As a result, FRV can only report on five measures in Q2.

FRV is undertaking work to investigate the cyber-attack and restore the IT environment as quickly as possible and in the meantime, FRV reports that it is developing and implementing interim solutions to maintain system functionality and data collection functions, where possible. However, the timeframe for retrieval of data stored in those systems, and access to systems and applications is still unknown. FRV advises that, at this point, it is unclear which data for the period of the outage can be extracted and reported on retrospectively.

FRV reports implementing business continuity measures, workarounds and contingencies to ensure that service delivery continues.

FRV is also exploring developing and implementing interim reporting solutions using data provided by ESTA to restore essential incident reporting functionality within FRV. This will include data for the period of the outage being made available to be reported on retrospectively.

FRV reports that crews continue to turn out through the use of mobile phones, pagers and radios and that it has maintained a range of corporate support functions during the cyber-incident. FRV advises it is publishing regular updates for the community and stakeholders on its website at www.frv.vic.gov.au.

Outcomes framework Q2 results commentary overview

Indicators showing a continued off-track trend:

Service Level Agreements (SLAs)

FRV’s ‘Service Level Agreement’ indicator monitors the number of SLAs (also known by both agencies as Service Level Deeds of Agreements) that are agreed to in-principle by CFA and FRV. FSIM considers that completion of clear, agreed, and implemented SLAs between agencies underpins the broader success of the Fire Services Reform, particularly supporting "an effective and productive working relationship across the fire services and other emergency service agencies, building on the principle of interoperability”.⁶ These SLAs provide a foundation for role clarity and functions for volunteers and staff in both agencies and are required to harmonise procedures.

FRV confirmed that the Q2 result for this indicator remains at 88% and advises that CFA and FRV are working collaboratively to finalise the Tools and Trade Lease agreement. FRV reports ongoing collaboration in conducting service and maintenance testing of breathing apparatus, despite cyber-attack disruptions, which corresponds with the Protective Equipment SLA. FRV also notes that further work needs to be conducted to agree on the Operational Communications SLA, with discussions continuing in February. Finally, FRV reports that the cyber-attack impacted FRV’s ability to progress the Dangerous Goods and Major Hazard Facilities SLA in Q2.

Footnotes

[1] CFA indicators with updated business rules: 2.3.6, 3.2.3, 4.2.2

[2] Revised baselines are reported using a quarterly average.

[3] CFA new indicators: 2.3.2, 2.3.3, 3.3.3, 3.4.3, 4.1.1

[4] In previous quarters, CFA reported results for 11 indicators (1.2.1, 2.1.1, 2.1.2, 2.3.1, 2.3.4, 2.3.5, 3.1.1, 3.1.3, 3.1.4, 3.1.5, 4.4.1) as a YTD cumulative result.

[5] An annual target has been developed for indicator 2.3.6.

[6] Year Two to Five Fire Services Reform Implementation Plan,pg 25

Definitions

Reference to fire services agencies

References to the fire services in this document relate specifically to CFA and FRV. Where more than one fire services agency is referenced in this document, they are listed in alphabetical order.

Comparison between agencies’ performance against published Outcomes Frameworks

As the two outcomes frameworks are based on the agencies’ own strategic and operational priorities and data, comparisons between the agencies should not be made.

Legislative requirements to report on outcomes

The performance measures that CFA and FRV report on are set by the agencies in their outcomes frameworks. Section 140(1) of the FRV Act requires CFA and FRV to each prepare an outcomes framework that sets out their respective outcomes-based fire service performance measures. Section 140(4) of the FRV Act further requires CFA and FRV to prepare and submit to FSIM a quarterly update that sets out their performance against their performance indicators.

Reporting and data limitations

Q2 is the first quarter CFA has reported on its Year Three Outcomes Framework. As reported above, CFA reports that the Year Three Outcomes Framework refines indicators from the Year Two Outcomes Framework and introduces some new indicators. Where appropriate, FSIM will continue to use historical data published in previous quarterly progress reports to track indicator trends over time however, in this report, FSIM has not included any baselines in the trend analysis charts as the data sets have altered.

As noted above, FRV reports that the cyber-security breach of its systems has significantly reduced its capability to report on outcomes and output data. FRV has provided commentary of five measures this quarter.

Report structure

This report comprises FSIM summary (section one) and the CFA and FRV performance measure results received for this quarter (sections four to six). Where possible, FSIM presents CFA performance results with trend data from 1 July 2020. FSIM also includes CFA and FRV explanations of measures (business rules) and where appropriate, provides confirmation of whether the agency met or did not meet target or baselines. FSIM also includes CFA and FRV commentary on indicator progress.

Fire Services Implementation Monitor (FSIM)

FSIM was appointed in 2020 in accordance with the FRV Act. FSIM’s functions under the FRV Act are to assess the effectiveness of agencies in delivering against Implementation Plan actions and provide independent assurance to government and the community on the progress made towards modern fire services providing for a safer Victoria. FSIM is required to prepare and publish quarterly reports on CFA and FRV Outcomes Frameworks measures under s141 of the FRV Act.

Publication of reports

FSIM publishes quarterly reports at Fire Services Implementation Monitor publications | Victorian Government.

CFA’s Year Three Outcomes Framework uses a quarterly baseline which calculates the average result for Q2 over a specified time period (1, 2, 3 or 5 years). CFA reports the use of baseline data in lieu of specified targets for many measures. CFA’s target therefore is to ‘increase’ or ‘decrease’ (as appropriate) and improve performance with consideration to baseline data.

CFA will not be retrospectively applying the Year Three Outcomes Framework reporting rules or recalculating baseline data using the CFA’s year three methodology for reporting periods prior to Q2. CFA categorises Year Three Outcomes Framework indicators into four domains:

1. strong relationships to keep communities safe
2. a responsive and innovative volunteer service
3. high performing volunteer and paid workforce
4. a continuous improvement culture.

Data is available from page 7 in the report document.

2022-23 Fire Services Outcomes Framework Progress Report Quarter 2

PDF 3.69 MB

(opens in a new window)

FRV report that the majority of indicator data is unavailable this quarter. However, FRV have reported on five Year Two Outcomes Framework indicators in Q2.

FRV categorises Year Two Outcomes Framework measures into three domains:

• Prevention and Preparedness
• Response and Recovery, and
• Organisational Excellence.

In Q2, FRV provided commentary against indicators under domain three: Organisational Excellence.

Data is available from page 23 in the report document.

2022-23 Fire Services Outcomes Framework Progress Report Quarter 2

PDF 3.69 MB

(opens in a new window)