vic_logo

Review implementation activities - MARAM framework

An implementation review is a way to check that the implementation activities are being successfully embedded into your organisation.

Download and print the PDF or read the accessible version below.

Introduction

Successful implementation means your organisation’s workforce is consistently identifying, screening, assessing and managing family violence risk according to its MARAM responsibilities, and working collaboratively with other services and practitioners to participate in ongoing risk management.

This guide suggests three ways to review the success of implementation activities.

  1. A qualitative review involves asking staff members questions, and reflecting on and analysing their responses. This helps your organisation understand how the workforce is experiencing implementation, which is important because they have close contact with service users. The qualitative review could take place at a workshop with key staff members and managers, by anonymous online survey, focus groups or during an all-staff forum.
  2. A case file checklist involves making a random selection of representative case files from across the organisation and staff members, and assessing them according to a checklist of information.
  3. A quantitative review is a data-based review to determine whether policies and procedures are being applied in practice.

Steps to review implementation activities

  • Determine when to undertake the review – review your implementation plan to evaluate how implementation of the MARAM Framework has tracked against expected timelines, and the outputs you would expect to see at the point of time of the review.
  • Decide how to undertake the review – determine the questions to be asked in a qualitative review, and the forum for the review (i.e. anonymous staff survey, workshops, all staff forum). Select an agreed number of random case files. Run a data report (where possible).
  • Analyse the results – analyse the feedback and data obtained, and consider the impact on the further planned efforts for implementation and alignment.
  • Review policies, procedures, practice guides and tools – based on the feedback and data obtained, review the organisation’s documentation to identify what further changes may be required to improve the feedback received.
  • Share the results – ensure the people responsible for the next MARAM organisational self-audit have the results of the review available to help assess current progress.
  • Close the feedback loop – report the outcome of the review and the next steps that will be taken to the organisation, governance groups and staff members.

When to review

Deciding when to conduct an implementation review is a fine balance – too early and a lot of implementation issues may not be resolved; too late and you may not be able to accurately capture lessons learned.

The review can be scheduled as part of the project plan at suitable junctures of implementation.

Repeat the review at key points to ensure continuous improvement.

  • Tips for conducting the qualitative review and analysing the responses

    • Ask for openness. Emphasise the importance of being open and honest, and make sure people aren't concerned about reprisal. Consider making the survey anonymous.
    • Be objective. Describe what has happened in objective terms, and then focus on improvements.
    • Document success. Document practices and procedures that led to project successes, and make recommendations for applying them to similar future projects.
    • Look with hindsight. Pay attention to the ‘unknowns’ (now known) that may have increased implementation risks. Develop a way of looking out for these in future projects.
    • Focus on the future. Remember, the purpose is to focus on the future, not to assign blame for what happened in the past. This is not the time to focus on any one person or team.
    • Look at both positives and negatives – identify key achievements and challenges when looking at lessons learnt.

    Suggested qualitative questions

    The questions are designed for a staff survey, they can be adapted for use in a workshop or other forum. The questions are only suggestion, and you should modify them to suit your organisation and the specific implementation activities under review.

    If appropriate to your service, consider a survey or forum of service-users to ascertain their experience of how the organisation responded to family violence in their circumstances

    Pillar 1: A shared understanding of family violence

    Question 1

    How would you rate your understanding of:

    • what behaviour constitutes family violence
    • the drivers of family violence
    • the evidence-based risk factors
    • how issues of diversity and intersectionality intersect with family violence
    • the nature of perpetrator accountability

    [Options: very low, low, medium, high, very high for each bullet point]

    Question 2

    Do you know where to find foundational knowledge about family violence within our organisation’s policies and procedures?

    [Options: yes/no]

    Question 3

    Do you believe our organisation is welcoming and respectful to all people regardless of identity?

    [Options: yes/no]

    [Free text: if no, what could we do better?]

    Question 4

    Do you understand the term intersectionality?

    [Options: yes/no]

    Pillar 2: Confidence and skills in applying MARAM responsibilities

    Question 1

    Which of the following MARAM responsibilities apply to your role? Please select all that apply.

    [Options: respectful, sensitive and safe engagement, screening all clients for family violence, screening for family violence where there are observable signs of trauma, brief risk assessment, intermediate risk assessment, intermediate risk management, information sharing with other services, secondary consultation, making referrals, comprehensive risk assessment, comprehensive risk management and safety planning, Contribute to coordinated and collaborative risk management including ongoing risk assessment]

    Question 2

    For each of the responsibilities chosen a question: how confident do you feel in performing this responsibility?

    [Options: not at all confident, not very confident, a bit confident, very confident]

    Note: only certain questions will be relevant dependent on the MARAM responsibilities held.

    Question 3 (responsibility 1 and 2)

    How confident are you in recognising observable signs of trauma that may indicate that family violence is occurring?

    [Options: not at all confident, not very confident, a bit confident, very confident]

    Question 4 (responsibility 1 and 2)

    How confident are you in knowing what to do in your organisation if you believe family violence is occurring?

    [Options: not at all confident, not very confident, a bit confident, very confident]

    Question 5 (responsibility 3 and 7)

    When completing a risk assessment, how often do you:

    • ask the victim survivor for their own assessment of the level of risk, fear and safety
    • assess against evidence-based risk factors
    • request information from other agencies to inform the risk assessment
    • give a risk rating based on structured professional judgment
    • complete a separate risk assessment for children
    • make referrals for the client
    • record client identities
    • use intersectional analysis
    • ask identity specific risk questions (comprehensive assessment only)

    [Options: always, most of the time, some of the time, never]

    Question 6 (responsibility 4–8)

    When managing a client’s risk of family violence, how often do you:

    • re-assess risk on a regular basis
    • share information with other agencies
    • request information from other agencies
    • coordinate risk management strategies with other agencies
    • impose restrictions on perpetrators
    • provide protective measures for victim survivors

    [Options: always, most of the time, some of the time, never]

    Question 7 (responsibility 5)

    How confident are you in seeking secondary consultations?

    [Options: not at all confident, not very confident, a bit confident, very confident]

    Question 8 (responsibility 5)

    How confident are you in making referrals?

    [Options: not at all confident, not very confident, a bit confident, very confident]

    Question 9 (responsibility 6)

    How confident are you in identifying risk relevant information which should be shared proactively (without a request)?

    [Options: not at all confident, not very confident, a bit confident, very confident]

    Question 10 (responsibility 6)

    How confident are you in following the procedures in place to share risk-relevant information internally/externally in response to a request?

    [Options: not at all confident, not very confident, a bit confident, very confident]

    Question 11 (responsibility 9 and 10)

    How often are you working with external services to provide a coordinated and collaborative response to family violence risk assessment and management?

    [Options: always, most of the time, some of the time, never]

    Question 12 (all responsibilities)

    What barriers do you experience in performing MARAM responsibilities? (allow multiple choices):

    [Options: not enough time, inadequate training (MARAM-specific or supporting training), resistance from clients, lack of guidance from manager, lack of support from organisation, don’t know relevant services to work with, relevant services lack capacity, relevant services don’t exist, not part of my job role, other]

    Impact of risk assessment and risk management practice on service users

    Question 1

    Do you think the family violence risk management process is helping to reduce risk to victim survivors? Why or why not?

    Question 2

    Do you think the family violence risk assessment and management process in your organisation gives victim survivors a better understanding of the level of risk they are facing from the perpetrator? Why or why not?

    Question 3

    Do you think the family violence risk assessment and risk management process is helping to keep perpetrators in view and accountable? Why or why not?

    Question 4

    Do you think the family violence risk management practices you use decrease the risk of the perpetrator committing family violence again? Why or why not?

    Question 5

    Do you think your risk management and information sharing practices are keeping perpetrators of family violence in view? Why or why not?

    Question 6

    What barriers do you think exist in decreasing the risk of perpetrators committing family violence?

    [free text]

    Policies, procedures, practice guidance and tools

    Question 1

    Do you feel supported by your manager and the organisation in responding to family violence in respect of service users? Why or why not?

    Question 2

    Have you received adequate training on the use of practice guidance and tools?

    [Options: yes/no]

    Question 3

    Have you received adequate training in responding to children, Aboriginal and other diverse communities in addition to MARAM training?

    [Options: yes/no]

    Pillar 3: Understanding the MARAM responsibilities including training

    Question 1

    Do you understand what MARAM responsibilities apply in your role?

    [Option: yes/no]


    Question 2

    Have you participated in any training on MARAM?

    If yes

    • who provided the training?
    • how would you rate the effectiveness of the training?

    [Options: very poor, poor, moderate, high, very high]

    If no, have you experienced any of the following barriers to accessing MARAM training?

    [Options: lack of time, employer not willing to support training and development, employer not able to support training and development (i.e. funding issues), lack of flexible training options (i.e. online training or after-hours training), Not aware of any training that exists, resourcing issues (i.e. inability for organisation to provide backfill for my role while I am on training, cost, location, child care, accessibility, cultural appropriateness, other, please specify]

    Question 3

    Do you feel confident in applying the knowledge from training into your practice?

    [Option: yes/no]

    [Free text – if no, what further training do you require? (examples, professional development, internal support, additional resources, face to face training, on the job training)]

    Pillar 4: Impact of responding to family violence on staff members

    Question 1

    Are you aware of the supports in place for you, as a staff member, if you experience family violence?

    [Options: yes/no]

    Question 2

    Have you participated in any communities of practice related to family violence risk assessment and management? If yes, please provide name:

    Question 3

    Do you know what data you are required to gather when family violence is identified?

    [Option: yes/no]

  • Note: When conducting a review of case files, choose a random selection of files across different practitioners and programs/departments to ensure a broad overview of practices.

    A high score suggests MARAM procedures are well embedded into usual practice. Low to medium scores suggest further work is required to understand why staff members are not following policies and procedures, and what further changes are needed to improve best practice.

    Risk assessment checklist:

    • evidence of use of structured professional judgement
    • recorded evidence of a victim survivor self-assessed level of risk
    • recorded evidence-based risk factors
    • information sharing – request made of another framework organisation
    • information sharing – voluntary/proactive sharing undertaken
    • asked relevant identity-specific questions
    • evidence of an intersectional analysis
    • separate risk assessment undertaken for each child

    Risk management checklist:

    • regular and updated risk assessments undertaken
    • secondary consultations take place
    • evidence of multi-agency involvement and coordination
    • evidence of referrals to relevant services
    • information sharing – request made of another framework organisation
    • information sharing – voluntary sharing undertaken
    • safety plan completed for each victim survivor, including children

    Information sharing checklist:

    • information provided to victim survivor about how their information can/will be shared
    • consent obtained (where relevant, if not relevant or views sought, mark as completed)
    • record keeping completed as per organisational policy (in line with legislation)
  • Family Violence Data Collection Framework

    The Royal Commission into Family Violence (RCFV) identified that limited data about the experience of Victorians affected by family violence contributes to poor outcomes. In response to this, the Victorian Government created the Family Violence Data Collection Framework (the FVDCF).

    The FVDCF includes:

    • a common set of family violence and demographic data items
    • definitions and standards
    • advice on reporting practices that will make information on family violence more useful

    The FVDCF supports the collection of more robust data, but does not fix broader data quality issues and limitations. It is expected that the FVDCF may be included voluntarily as part of any systems updates. Under a maturity model, organisations are not expected to collect the data identified here immediately, but can work towards collecting this data in the future.

    Tools for Risk Assessment and Management (TRAM)

    Online versions of the MARAM risk assessment tools are available through the Tools for Risk Assessment and Management (TRAM) platform and the Specialist Homelessness Information Platform (SHIP). These systems allow users to complete, store and print MARAM risk assessments in an online environment. This provides a basis for easy reporting on the number, status and overall risk level of risk assessments entered into these systems. SHIP is used by specialist family violence and homelessness service providers and includes the MARAM tools automatically (starting July 2020).

    TRAM has been developed as an alternative solution for non-SHIP users. TRAM can be accessed free of charge by all Framework organisations provided MARAM training requirements have been met.

    Both TRAM and SHIP can aide in the collection of data as identified within this section.  If your organisation is interested in adopting or trialling TRAM, please contact tram@familysafety.vic.gov.au

    Screening data

    Total number of service users:

    • screened for family violence
    • identified as a victim survivor
    • identified as a perpetrator
    • identified as an adolescent using violence

    Note: breaking down data of victim survivor, perpetrator and adolescent using family violence into demographics of gender / Aboriginal and/or Torres Strait Islander / CALD / LGBTIQ / person with disability / rural / older person.

    Risk assessment data

    Number of cases where risk assessment has taken place, and as a percentage of those cases:

    • where a referral has been made
    • where secondary consultations have taken place
    • with requests for information made to another framework organisation
    • where voluntary information sharing took place
    • went on to complete a risk management safety plan

    For risk assessments of victim survivors, a breakdown of the risk factors present in each case, and any identity demographics recorded

    Total number of risk assessments broken down into categories of ‘at risk’, ‘elevated risk’, ‘serious risk’ and ‘serious risk – requires immediate protection’

    Risk management data

    Number of cases where management has taken place, and as a percentage of those cases:

    • where a referral has been made
    • where secondary consultations have taken place
    • requests for information made to another framework organisation
    • voluntary information sharing took place

    Referrals and secondary consultations data

    • Total referrals, by type of framework organisation:
    • made to other framework organisations
    • received from other framework organisations
    • Total number of requests for, or provision of, secondary consultations related to family violence

    Information-sharing data

    Total information-sharing requests received:

    • including type of framework organisation making the request
    • number of declined requests

    Total information-sharing requests made:

    • Including type of framework organisation the request was made to
    • Number of declined requests

    Number of voluntary shares of information

Using the data to inform a further MARAM organisational self-audit

The data collected as part of an implementation review should inform the next use of the MARAM organisational self-audit.

Example case study: 

  • Greater than 75 per cent of staff surveyed said they are not at all confident, not very confident or a bit confident in seeking secondary consultations.
  • A case file audit revealed very few cases had referrals made to external organisations.
  • Risk assessment data contains low levels of information sharing when compared with the number of cases where family violence was identified.

This data is relevant to Milestones 2D and 2E in the MARAM organisational self-audit tool on secondary consultations, referrals and information sharing. While an organisation may have enacted policies, procedures and practice guidance, and put in place appropriate forms, the data suggests staff members do not feel confident in putting these into practice.

The assessment for Milestones 2D and 2E is likely to be between some progress (staff members are not familiar with the changes made and aligning to MARAM in their course of their work) or good progress (there is room for further work to be undertaken including to ensure more consistent application by staff members).

The notes section may include comments and insight into why it is thought staff members are not confident in responsibilities 5 and 6 and explore what appropriate implementation activities should follow – such as further training, improved resources, managerial support and so on.

Reviewed 26 June 2020

Was this page helpful?