The Flexible KPIs as a tactical aid for emergency services
The NEXES Flexible KPIs are suitable for measuring the impact of total conversation (NGES) on emergency services in Europe and beyond, taking into account the individual differences among emergency services. The Flexible KPIs support planning and roadmap activities by providing insight in progress achieved, and allowing retro-active re-evaluation.
The NEXES Flexible KPIs should be used as a tactical aid for continuous improvement of (NG)ES performance.
The Flexible KPIs are set up for measuring both achievement and sustaining qualitative and quantitative impact of NGES. The innate ability in the Flexible KPIs for re-evaluation supports what-if analysis and other monitoring and planning techniques. The Flexible KPIs have been designed for use during long(er) time periods, supporting continuous improvement of the performance of NGES.
Advantages & Disadvantages
As with any methodology the Flexible KPIs have advantages and disadvantages that each organisation must take into account:
Advantages:
- Standardised KPIs and categories, customisable priorities and effect measurements.
- Respect autonomy of emergency services regarding their focus, time, budget, technology adoption roadmap and other characteristics.
- Transparent about customization, fostering comparison with other KPI evaluations.
- Applicable to one emergency service organisation multiple times to evaluate impact as a measure of progress.
- Applicable to multiple emergency services to measure individual progress and determine joint progress.
- Provide guidance on quantitative and qualitative data to be collected.
- Include perspectives of both emergency services’ professionals as well as citizens.
- Separation of data gathering versus evaluation: making it possible to re-evaluate gathered (current and historical) data (‘measure once, evaluate many times’).
Disadvantages:
- Different to the standard EENA KPIs.
- Strategic decisions needed when applying Flexible KPIs regarding priorities and evaluation parameters.
- Emergency services may be unfamiliar with socio-technical evaluation and/or with qualitative evaluations.
Effort estimation
Clearly the application of the Flexible KPIs requires effort. It is beyond scope of the NEXES Action to collect data for quantitative effort estimation of the Flexible KPI implementation. Based on the experiences during the NEXES pilots and evaluation thereof, a qualitative effort estimation can be provided, as shown in the table below:
| Activity | Effort estimation | Brief rationale |
|---|---|---|
| Determining application of Flexible KPIs | Moderate effort / Moderate effort | First time: Includes strategic decisions on priorities and ‘what to measure. Subsequently revisiting KPIs and effect measurements and re-determining relevance requires similar effort. |
| Obtaining measurement data | Most effort / Moderate effort | First time is most effort, including setting up measurement methods, etc. Subsequent obtaining of measurement data becomes simpler. |
| Evaluating measurement data | Moderate effort / Little effort | Moderate effort for the first time to set up analysis and evaluation methods, subsequently little effort as procedure can be repeated. |
| Evaluating KPIs | Least effort | A simple spreadsheet can easily support KPI evaluations. |
| Re-evaluating measurement data | Little effort | Is repeat of earlier defined procedure, albeit with different priorities and parameters. |
| Re-evaluating KPIs | Least effort | As simple as evaluating the KPIs |
Wrap-up
Taking all considerations into account the we recommend that the Flexible KPIs are used by emergency services, policy-makers and decision-makers when adopting NGES technologies to monitor progress and achievement of the NGES effects and impact, including citizen perspectives.
When you are interested to read more on the Flexible KPIs, you can read Deliverable D2.4, as well as the evaluations of the NEXES pilots. For more information on all NEXES Recommendations, we encourage you to read Deliverable D2.5. All these deliverables can be found at the NEXES website.
Thank you for reading and enjoy making the most from the Flexible KPIs for your organisation!
On behalf of the NEXES Action and all NEXES’ colleagues who contributed to the Flexible KPIs as well as their application to and analysis in pilots and demonstrations,
Niek Wijngaards
This blog is number 5 in a series of five articles on the NEXES KPIs Lessons Learned and Recommendations. When you wish to delve deeper into the NEXES Action and its solution to comparing apples and oranges we recommend to read the deliverable D2.4, for more information on the NEXES Recommendations we recommend to read the deliverable D2.5. Below is the list of all the articles in the series:
- ‘Comparing apples and oranges’: The evaluation setting
- Flexible KPIs: Assuring fair, respectful and motivating evaluation
- Playing the ‘what-if’ game: Monitoring and planning with Flexible KPIs
- A public playing field: Pragmatic exploitation and availabaility
- Continuous Improvement: The Flexible KPIs as a tactical aid for emergency services
Dr. Niek Wijngaards works for AIMTech Consulting Limited in the United Kingdom and True Information Solutions in the Netherlands as senior consultant and solution architect. His focus on user-centered innovation and his work on intelligent systems and scenario-based robust decision-making provides a sound basis for the development of the NEXES Flexible KPI structure. Niek can be contacted at n.wijngaards AT aimtech DOT co DOT uk for Flexible KPI and recommendation-related questions.
Copyright © 2018, NEXES Research and Innovation Action, All Rights Reserved. The NEXES Action has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 653337. The work on the NEXES Key Performance Indicators is co-authored by the Action partners and has benefited from the constructive comments by the reviewers. See the NEXES LinkedIn group 
for an overview of NEXES colleagues. All figures Copyright © NEXES unless stated otherwise and all images Copyright © www.openclipart.org unless stated otherwise.