There is a great post on NextBillion.net that summarizes recent advancements M-health teams in frontier markets are making in what they call "remote monitoring". But tech firms have already learned that tracking key performance indicators and not considering customer focused quality indicators only gets half of the story needed to make lasting improvements. More after the jump.
Products used in frontier markets often are hard to track, because their users are in remote areas far from their designers. By linking a product like an efficient cook stove or a vaccine centrifuge to a cell phone network, designers are able to track and receive data on product usage and performance. They use this data to improve product design.
Cell phone companies built this approach into their product testing approach: phones in their networks often collect and send information back to identify issues in the mobile network. However, telecoms have learned over time that remote monitoring alone only provides part of the answer.
When telcos set up the metrics they would grade themselves on, they tracked and aligned their improvements based on the performance indicators (KPIs) that define engineering metrics, not metrics linked to the customer. As a result, they might establish a KPI that requires an SMS to be delivered to the competitor’s network within 10 seconds of hitting send, but customers might still think the service was terrible because it had taken 20 minutes for the recipient to receive that message, and over time customers would leave that company, even though KPIs indicated that the Telco was doing great.
Ultimately, all of these metrics come down to the people who are using these products and services (the stakeholders), so measurement metrics have to focus on their perceptions. To do this correctly, there needs to be a metric interface that connects the needs of the customer and the performance of the product/service. (If you have ever worked in sales, this is similar to the concept of feature versus benefit).
Quality of experience (QoE) Quality of experience reflects the desired experience that an organization wants its stakeholders to receive. It is a subjective measurement that customers establish, and is wrapped up in the psychology of tradeoffs. At a minimum, an acceptable overall QoE is one where the alternative they could switch to is less acceptable. This normally draws from an established stakeholder experience design that was created in close collaboration with stakeholders or customers, and is measured using customer feedback.
KPIs are essential to measuring the performance of the product or service. Ultimately, stakeholders using the service have to weigh in on these results and if the KPI results fit their changing needs. This is what is traditionally measured by most groups—and it is the kind of data that remote monitoring can yield.
Key Quality Indicators (KQI) provides the interface between the QoE and the KPIs. These are the metrics that show that the desired quality of experience is being fulfilled. Each KQI should align to both defined points inside the QoE and the KPIs.
Remote monitoring brings data into product design that had been hard to collect before. It is a foundational set of metrics that can have long lasting impact, if it is paired with a more robust customer experience design, and tracked according to stakeholders’ perceived quality of experience. But using it is only part of the story. Organizations that focus in on how performance metrics are affected will, in the long run, end up having to reprogram their remote monitoring systems. If instead, they focus on building out their metrics using KQIs that lead to a total QoE, they will find more value in the KPIs they apply.
I will often say that customer experience moves way beyond monitoring and evaluation. But people often focus on this first, because it is an obvious application for CX. Reviewing this approach fleshes out what I describe here. Traditional M&E will focus on KPIs and, ideally KQIs. But designing an end state total quality of experience that organizations want to achieve is a much more strategic process, and needs to be central to the management of people engaging with stakeholders, and the strategic design of projects.