‘Comparing Apples and Oranges’

By   April 21, 2018
0

The evaluation setting

With Flexible KPIs it is possible to compare apples and oranges which translates into the NEXES context as: Emergency services (ES) are very different yet need support to adopt Next-Generation Emergency Services (NGES) capabilities. The NEXES Research and Innovation Action aims to facilitate this adoption process by providing reference technology implementations. The adoption of NGES technology is further supported by an evaluation framework for NGES performance and recommendations on adoption of NGES capabilities.

NGES adoption is facilitated by having each Emergency Service organisation monitor and plan their own progress while aiming for the intended socio-technical impact of NGES.

Policy-makers and decision-makers should steer on ‘the effects and impact’ of NGES, not steer on specific technologies. Citizens should be involved in the evaluation given their integral role in the performance of Emergency Services.

Emergency services (ES) within Europe and beyond have many aspects in common but also significant areas of difference. The commonalities primarily concern the task for emergency services of preventing and mitigating incidents, large and small. This is reflected in their major business processes:

  • Handling emergency calls
  • Defining the situation
  • Coordinating response activities
  • Communication to citizens

At the same time, many differences are identifiable between emergency services, e.g. concerning their management structures and reporting / accountability structures, budgets, technological installed-base, area(s) of responsibility (Sapeurs Pompiers in France are not identical to Fire and Rescue Services in the UK for example), environmental conditions (coastal, airport, river, mountains, …), technology replacement strategy and more.

In this series of articles we employ two example emergency service organisations. These examples are entirely fictional and are solely used for illustrative purposes. Below is a brief characterisation of the two example emergency service organisations.

Emergency Service Organisation A Emergency Service Organisation B
  • First responders from fire brigade, police and emergency medical services.
  • Responsible for a large rural area, populated with small towns.
  • Area is mountainous with forests.
  • Major industry: agriculture, tourism (skiing, hiking).
  • First responders from fire brigade, police and emergency medical services.
  • Responsible for major urban area consisting of a large city situated at the sea with a major harbour and river.
  • Area is a delta plain including many smaller canals.
  • Major industry: harbour, industry, financial services.

Meanwhile the NEXES Action has shown a reference implementation for Next-Generation Emergency Services (NGES), showcasing four advanced capabilities:

  • Total conversation: being able to make video calls, use interactive chat, use VOIP (wifi-calls) to call emergency services.
  • Exchange of additional information: GPS location information of the device, as well as being able to send additional medical and/or other information including photos and videos when calling emergency services.
  • Automated emergency calls: being able to have e.g. vehicles make automatic emergency calls during an accident.
  • Improved interoperability: between organisations within and among European countries and beyond by providing templates for emergency apps for mobile phones as well as reference implementations of technology needed by emergency services.

Although the implementation is of a technical nature, its social implications are extensive, enabling citizens, including tourists and non-native speakers, as well as people experiencing impairments, to engage in contact with emergency services in Europe and beyond. Key indicators are improvements in communication with citizens, situational awareness and, thereby, improved incident handling.

While the NEXES Action draws to a close, it is important to support emergency services who are going to adopt NGES implementations. Emergency services engaging in technological change often require multi-year plans that require frequent monitoring and often need re-planning. This long(er) timespan poses additional challenges given the uncertain and unpredictable (changes in) decisions by policy-makers, changes in budgets, available technology, et cetera. The future is hard to predict but opportunities and challenges may arise from the complex emergency services landscape both in Europe and beyond. Additional support is beneficial, given that the NGES impact is not monitored by the traditional EENA Key Performance Indicators, especially concerning the expected qualitative improvements in (e.g.) situation awareness.

The example emergency service organisations, A (rural) and B (urban), both intend to obtain the NEXES reference technologies from one or more suppliers to become Next-Generation Emergency Services. They are in need of a method to monitor their progress and determine a means to implement these technologies while taking into account their own technology roadmap, budgets and other constraints from policy-makers, decision-makers and other stakeholders. These needs are discussed in subsequent articles.

Policy-makers and decision-makers of emergency services are encouraged to steer on effects of NGES, such that each emergency service organisation retains autonomy in its own change-processes. Policy-makers and decision-makers can sidestep the pitfall of micro-managing emergency service organisations’ technological and organisational change. Citizens, a major stakeholder of emergency services, also have a role in the qualitative and quantitative evaluation of NGES performance and are thus part of the evaluation setting.

For the example emergency service organisations this implies that policy-makers and decision-makers should not steer on, e.g., “you must use product X for video-calling and instant-messaging”. This places emphasis on a specific technology, which may or may not fit and achieve implied effects. Rather, steering should happen by expressing the desired effects, e.g. “Achieve 5% of video-calls of your total call volume in the next year” and “Your technology choice must ensure interoperability with products X and Y”. The first approach is akin to micro-management and gives less incentives for organisations to achieve progress. The latter approach focuses on the desired effects while respecting the autonomy of organisations in determining how to best achieve these effects.

The NEXES Action also offers support for emergency services engaging in adopting NGES technologies. This support is aligned with the commonalities of emergency services, while also being adaptable to the individual differences of each emergency service organisation. Both technical and social performances must be taken into account including a multi-stakeholder perspective.

This article has sketched the evaluation setting, introducing two example emergency service organisations and briefly described the role and intent of the NEXES Action. The next article discusses an evaluation method to each emergency service organisation can use.

This blog is number 1 in a series of five articles on the NEXES KPIs Lessons Learned and Recommendations. When you wish to delve deeper into the NEXES Action and its solution to comparing apples and oranges we recommend to read the deliverable D2.4, for more information on the NEXES Recommendations we recommend to read the deliverable D2.5. Below is the list of all the articles in the series:

Photo of Niek Wijngaards. Dr. Niek Wijngaards works for AIMTech Consulting Limited in the United Kingdom and True Information Solutions in the Netherlands as senior consultant and solution architect. His focus on user-centered innovation and his work on intelligent systems and scenario-based robust decision-making provides a sound basis for the development of the NEXES Flexible KPI structure. Niek can be contacted at n.wijngaards AT aimtech DOT co DOT uk for Fleixble KPI and recommendation-related questions.


EC FlagNEXES action logo Copyright © 2018, NEXES Research and Innovation Action, All Rights Reserved. The NEXES Action has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 653337. The work on the NEXES Key Performance Indicators is co-authored by the Action partners and has benefited from the constructive comments by the reviewers. See the NEXES LinkedIn group LinkedIn Logo for an overview of NEXES colleagues. All figures Copyright © NEXES unless stated otherwise and all images Copyright © www.openclipart.org unless stated otherwise.