A ‘one size fits all’ approach does not work for NEXES to use Key Performance Indicators (KPIs) on emergency services in different countries. The key obstacle is that KPIs are defined to measure a single effect. The solution is to focus on the differences between emergency services, not the similarities. By decoupling effect measurements from the KPIs, we gain flexible KPIs.
This is the third article in a series of six about the NEXES Key Performance Indicators (KPIs). The previous article described that the ‘one size fits all’ approach where the same KPIs are applied to all emergency services, is deemed unfair, disrespectful and demotivating.
The standard structure of one KPI is to measure a single effect accompanied by a certain target value, e.g. “average cost of production per widget should be less than 1.50 €”, or “arrival time of first responders at incident location should be less than 10 minutes”, or “average call waiting time should be less than 10 seconds”. The situation unfolds that traditional KPIs are not easily applied to multiple, heterogeneous, organisations. Our ‘apples and oranges’ are emergency services that are organised differently and each has their own plan for implementation of NEXES solutions.
Our solution lies in decoupling effects from KPIs, by describing a common denominator in the KPI which is to be specified in detail in the effect measurements by the single, different parties. Traditionally, a KPI expresses a single effect, e.g. “Good fruit has an orange colour” which may have a score for all the fruit in our oranges orchard (say, 95% is of the correct orange colour). Our suggestion is that a flexible KPI has a statement “All fruit has to look good” which has a score (say, 95% of the fruit looks good). The KPI is in practice, however, based on multiple effect measurements by the partaking organisations. For example, “Fruit does not have brown spots”, “Oranges have an orange colour”, and “Apples have a red colour”. Now, an apple orchard can select the effect measurements that are relevant for itself: an apple orchard can thus ignore the effect measurement on oranges.
However, the apple orchard may face another problem with the effect measurement “Apples have a red colour”. A quick inspection of apple trees shows that apples can have many different colours: green, red, yellow, pink, and combinations thereof. It would be useful for the apple orchard to adapt the effect measurement on apples. Instead of only using the reference value “red colour”, the orchard may decide to use “red, green, yellow, pink colour and combinations thereof” as the reference value. The question arises: who can decide to change the effect measurement? Must we decide on this, or should the decision be left to an orchard? In general, the orchards are in the best position to choose the best adaptation for their fruit: we’ll ask the orchards to tailor their effect measurements and be transparent about their adjustments.
Both the orange orchard and the apple orchard can be scored with the KPI on good fruit. Each orchard can use a relevant effect measurement, which can be tailored to their specific situation.
Within the NEXES Action, we used this idea to define flexible NEXES KPIs. First, a set of 11 KPIs is defined where each KPI is mandatory. That means each emergency service must have a score for all KPIs. A KPI is not anymore defined as a single effect, rather each flexible KPI is dependent on multiple effect measurements. The score of a KPI can be calculated by aggregating the scores for the relevant and tailored effect measurements. An effect measurement has a method to, as the name suggests, measure an effect. This method can be tailored including setting the scoring formula.
Overview of the 11 KPIs in the 5 categories, all effect measurements are not shown.
The current 58 effect measurements are not shown in full detail in the picture above. The 11 NEXES KPIs are grouped into categories as follows:
- Emergency Calls:
- KPI#01: Contact Alternatives. Assessment of diversity in how the Emergency Services can be contacted.
- KPI#02: Scalability. Assessment of impact of increasing number of calls to emergency services.
- KPI#03: Call Handling. Assessment of how the call is processed, including forwarding the call.
- Defining the Situation:
- KPI#04: Call validation. Assessment of validity of a call as an emergency call.
- KPI#05: Communication with Citizens. Assessment of the total conversation communication with callers, including citizens experiencing disability, impairment or special needs (elderly, tourists and non-native speakers).
- KPI#06: Information Management. Assessment of the accuracy of the acquired location and other incident information.
- Coordinating Response Activities:
- KPI#07: Dispatching First Responders. Assessment of the effectiveness and efficiency of single and multi-agency incident response.
- KPI#08: Communication to First Responders. Assessment of timeliness and accuracy of interaction and sharing information with FRs from single and multiple agencies.
- Communication to Citizens:
- KPI#09: Alerting. Assessment on the effectiveness and efficiency of providing information to the general public and specific groups of citizens.
- KPI#10: Instructing. Assessment on the effectiveness and efficiency of providing instructions to citizens involved in an incident to prevent further escalation, fatalities, and/or damages.
- Customer Satisfaction:
- KPI#11: Customer Satisfaction. Assessment of the perception of emergency services and citizens, including those experiencing disability, impairment or special needs, on the performance of the emergency services.
Are these flexible KPIs and effect measurements better? The feedback we got so far:
- These KPIs are fair. The effect measurements can be made relevant or irrelevant. An emergency service can thus select those effect measurements that relate to its own plans.
- These KPIs are respectful. The effect measurements can be tailored to respect the local situation.
- These KPIs are motivating. The emergency services can each adapt the effect measurements to their needs.
The next articles delve into the tailoring of effect measurements and comparing KPIs from emergency services.
This blog is number 3 in a series of six articles on the NEXES Key Performance Indicators and Effect Measurements. When you wish to delve deeper into the NEXES Action and its solution to comparing apples and oranges we recommend to read the deliverable D2.4. Below is the list of all the articles in the series:
Dr. Niek Wijngaards works for AIMTech Consulting Limited in the United Kingdom and True Information Solutions in the Netherlands as senior consultant and solution architect. His focus on user-centered innovation and his work on intelligent systems and scenario-based robust decision-making provides a sound basis for the development of the NEXES flexible KPI structure, making it possible to rigorously compare apples, oranges and indeed the entire contents of a fruit basket. Niek can be contacted at n.wijngaards AT aimtech DOT co DOT uk for KPI and fruit-related questions.
Copyright © 2016, NEXES RIA, All Rights Reserved. The NEXES Research and Innovation Action has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 653337. The work on the NEXES Key Performance Indicators is co-authored by the Action partners and has benefited from the constructive comments by the reviewers. See the NEXES LinkedIn group 
for an overview of NEXES colleagues. All images Copyright © NEXES unless stated otherwise.