National review of community risk methodology across the UK Fire and Rescue Service

Since the 2004 Fire and Rescue Services Act, fire and rescue services have been legally obliged to produce integrated risk management plans. While the legislation only covers England, Wales, Scotland and Northern Ireland have similar arrangements and slightly different terminology, so the National Fire Chiefs Council (NFCC) uses the broader term of ‘risk management plans’ (RMP) to capture them all.

In the intervening 15 years, no national review of RMPs has been carried out. Fire and rescue services have complied with the National Framework (or devolved equivalent) to ensure that the RMP is consulted on and published on a regular, mostly three-year basis. It is timely for a review to take place and consideration given to the way in which RMPs have assisted fire and rescue services to identify, manage and then mitigate risks using the resources available to them at a local level.

The NFCC has taken a leadership role and commissioned research into RMPs as part of its Community Risk Programme (CRP)¹. The CRP is one of three programmes run by the NFCC’s Central Programme Office.

1 Report Citation: Hill, R., Andrews, S., Murphy, P. and Lakoma, K. (2019). National Review of Community Risk Methodology across the UK Fire and Rescue Service. London: National Fire Chiefs Council.

Approach

In 2018, academics from Nottingham Trent University (NTU) started work on a review of community risk methodology across UK fire and rescue services. They developed a questionnaire to gather information and data about RMPs and received responses from 43/50 fire and rescue services. This is a very good response rate (see figure 1). Building on this questionnaire and through the development of case studies and bringing together some practitioners, the resulting report is a review of current practice; it identifies good practice, gaps in current practice and suggests future direction.

 

 

The questionnaire was in two parts. Section one explored the approaches to risk assessments and the remaining sections explored prevention; protection; response; nature and disposition of resources; and evaluation of fire and rescue activities. The researchers developed a set of questions to assess the quality of risk management plans. They drew on a range of academic practice to analyse and interrogate the data.

The researchers note: ‘The aim of the methodology is to create an evidence base to support professional, informed judgements about resourcing to address or mitigate that risk, so this report is written from this perspective’.

Experience of Using FSEC

The questionnaire asked fire and rescue services whether they used the Fire Services Emergency Cover (FSEC) toolkit. FSEC was developed by the government after the 2004 Fire and Rescue Services Act; it is an electronic system that assists fire and rescue services in assessing emergency response options and targeting prevention and protection activities as part of an RMP.

Out of the 43 services that responded to the questionnaire, 36 said they had used FSEC at some point, six services reported they had never used it and 19 confirmed that they are no longer doing so.

The researchers conclude: ‘It currently has limited use and it appears to have been superseded. There could be a positive reason for this, in that services are finding better datasets and techniques. Alternatively, it could be that they have stopped doing some of the analysis that was formerly possible with the previous toolkit. The final possibility is that changes in organisational structure have resulted in organisational memory loss’.

As a result of this finding, the researchers suggest conducting further research to explore the reasons why fire and rescue services do not currently use the toolkit. Such research could explore the features that fire and rescue services may find useful in a community risk assessment toolkit, and could conduct extensive user experience testing once a prototype has been developed, to ensure that the toolkit operates as expected with FRS personnel across the UK who will be responsible for community risk assessments.

The researchers provide some caution: ‘Tools will only be used where it is beneficial (or necessary) for FRSs to use it, so we would recommend that it is clear to FRSs how the toolkit will make their roles easier, and what it can contribute to the effectiveness of their practice’.

 

“A national fire database would have value to enable fire and rescue services to conduct analysis with larger, national datasets rather than the limited local ones they have at present”

 

Data

The research report looks at the role of data in informing RMPs. ‘Data should underpin all aspects of community risk management’. Data is used to identify hazards and those at risk. Fire and rescue services use varying number of datasets to inform risk planning. ‘We do advocate that each FRS should be using national and regional datasets to identify trends and patterns which can inform their own local risk profile.’ Figure 2 shows the toolkits, models and data sources used in risk assessments (2a) and response time standards (2b).

 

Figure 2a

 

Figure 2b

 

 

Through their analysis of the results of the questionnaire, the researchers conclude: ‘There is a general lack of insight into what data are specifically used from the data sources reported, how fire and rescue services are using data sources to assess risk, and particularly how fire and rescue services are combining different sources of data to build a more complete picture of community risk’.

The researchers suggest that fire and rescue services should be more critical about datasets provided commercially and to be better clients mostly through collective purchasing power. Currently individual fire and rescue services purchase datasets or set up data sharing agreements with partners at a local level. The Exeter dataset is the only example of national dataset that is divided up and given to fire and rescue services. Previous research carried out for the Chief Fire Officers Association concluded that the value of the Exeter dataset (containing data about over 65s registered with a GP) was to be found when it was layered with other datasets to give a more sophisticated view on risk in an area.

That same CFOA research found that fire and rescue service analysts had many different ways of managing datasets and integrating them. The NTU researchers confirmed this finding and concluded: ‘There should be a clear method for integrating sources of data into a single analysis’.

The other national dataset is the data collected by the government in England on incidents. This is held in the Incident Response System (IRS). Fire and rescue services use their own local incident data to assist in analysing risk but are unable to see beyond their own boundaries because the data is held nationally and not accessible.

This leads the researchers to the conclusion that a national fire database would have value to enable fire and rescue services to conduct analysis with larger, national datasets rather than the limited local ones they have at present. That database could contain both public data such as IRS but also provide access to commercial data through national procurement of datasets like Experian’s MOSAIC. Many services do not use the latter because it is too expensive.

The findings of the NTU researchers when they looked at the use of data to inform RMPs is echoed in the summary of findings from HMICFRS’s first tranche of inspections of fire and rescue services.

The NTU report recognises that the demands of HMICFRS may lead to improvements as they require fire and rescue services to provide data on an ongoing basis for inspection. The vast amount of data that HMICFRS is collecting for its inspection purposes may well form the basis of the national fire database that the researchers suggest in their report. ‘Consequently, this will increase the sophistication of the analysis and evaluation available to FRSs, which will increase the potential to accurately and elaborately identify priority areas of community risk in UK FRSs’.

Use of Evidence

While data are used to assess risk and identify those at risk, the use of evidence in decision making ensures that resources are used effectively and efficiently. More than half of the respondents to the questionnaire reported rarely or never using academic research to inform responses, with the same number reporting rarely or never using other partners’ research to inform response plans.

Fire and rescue services are good at identifying sources but do not demonstrate how they use the evidence to inform decision making. This is a key area of interest for HMICFRS.

The NTU report concludes: ‘The key message from this analysis is that most FRSs are either not sharing the number and ways in which they are using evidence to inform RMP, or they are not drawing across the range of available evidence when informing their decision making around community risk management’.

 

“The NTU research provides, for the first time, a national view on how RMPs are being developed in individual fire and rescue services”

 

Methodology

The researchers looked at three key elements of developing an RMP: the balance of using risk and demand to assess what resources to use; the definition of what that risk is; and the costs of the consequences of fire should it occur.

The questionnaire sought to establish the extent to which fire and rescue services base their RMPs on risk and demand. ‘From the submissions, it is also evident that the majority of fire and rescue services base their risk management on risk, with only five basing their risk management on risk and demand, and no FRS basing their risk management solely on demand’.

The concept of risk is perceived differently by different fire and rescue services. There is no accepted national definition of risk. The researchers point to the need to model both risk and demand based on evidence and data. As highlighted above there are weaknesses in both the way that data is acquired, analysed and used and in how evidence from external sources is used to develop RMPs.

For some, the priority is to model based on the likelihood of an event occurring. Others model based on the consequences of an event happening (ie survivability), while others still look at the economic cost of an event occurring’. And on that latter point, the economic cost of fire is very out of date and the researchers suggest that the 2008 model be updated to reflect modern risks and monetary values.

Evaluation

Like HMICFRS, the researchers found little evidence of evaluation taking place in fire and rescue services. Being able to describe the effectiveness of interventions to reduce risk is critical in an RMP and having a robust evaluation method is part of the process. Evaluation should be a constant to maintain an up to date picture of what is effective. Evaluation is a process rather than a piece of work which takes place at the end of a project.

There is not a one size fits all approach that would apply equally to all areas of fire and rescue business, but there are common strands of evaluation activity that would which is why the researchers argue for the creation of a ‘compendium of evaluation methodologies’. This would allow fire and rescue services to choose the evaluation methods most appropriate for the task at hand.

In addition to this compendium, the researchers also suggest the creation of an annual conference of good practice and a place to keep that good practice. In terms of the latter, ukfrs.com, home of National Operational Guidance, may be a good place to start.

Practitioner Wish List

As part of NTU’s work, they set up a Technical Working Group (TWG) to look at the outcomes from the questionnaires and provide feedback to ensure that sector knowledge and expertise were fed into the final analysis.

The TWG were asked to consider whether a new toolkit, something to replace FSEC, would be of value and if so, what should it be able to do. The TWG provide a wish list below.

Toolkit needs to:

• Allow for a RMP that is compliant against statutory duties of fire and rescue services and expectations placed on them through frameworks, guidance and codes of practice etc.
• Support fire and rescue services in developing a RMP which supports NFCC Strategy around RMP (risk not demand linked to funding).
• Generate capacity, data, expertise, understanding and competence at a national level to support development of RMP.
• Provide sufficient data and be flexible to use with a range of data sets both internal and external to the FRS to support. This also includes the ability to drill down data sets to provide FRSs with the opportunity to target activities more effectively and efficiently.
• Supports political understanding and local communication, engagement and consultation in the creation of RMPs.
• Provide a greater research and evidence base to support decision making throughout the RMP process in the areas of RPP.
• Bring together good practice across the UK.

 

In addition, there is opportunity for FRSs to take a joined-up approach to procure software where economies of scale opportunities can be identified as opposed to individual FRSs holding individual contracts with software providers.

Conclusion

The NTU research provides, for the first time, a national view on how RMPs are being developed in individual fire and rescue services. It shows that the creation of a national tool to assist fire and rescue services only works if it adds value to the work already taking place at a local level. There is much to be learned from the FSEC experience and there is more work required to understand that before a new toolkit based approach is agreed.

Agreeing a national definition of risk will provide a firm foundation on which to develop RMPs. The value of data and evidence to underpin RMPs is clear: staff need access to national datasets based on meaningful data standards. Staff also need to have skills and training to carry out their work on RMPs. The discovery phase of the NFCC Digital and Data Programme has also come to similar conclusions about the need to respond to the professional needs of this group of staff.

Looking at national approaches to procuring datasets may alleviate some local financial burdens and provide all fire and rescue services with the same tools to assess local risk. The same applies to non-commercial datasets held by partners. There is a delicate balance between the need to resource to risk with the need to resource to demand and some of that may be alleviated by having access to an up to date model of the cost of fire and other rescue activities.

Knowing what works and why is also a valuable part of RMPs. There is no point in spending resource on an activity or intervention if there is no evidence to suggest it works. Ingraining evaluation into all aspects of RMPs is important and is something that the NFCC Digital and Data Programme also identified in its discovery work. The NTU research, allied with the discovery work in the Digital and Data Programme, confirm a complementary programme of work that will go a long way to improve the risk management planning of fire and rescue services.