Blog

Doctorlink scores highest but more regulation needed as not all symptom checker tools are created equal concludes new research

Accuracy and Performance of Symptom Trackers thumbnail

Recent studies have shown that more of us than ever before are turning to online and digital tools to manage and improve our health. Unsurprisingly, COVID-19 has played a part, accelerating digital adoption across the globe.

Online Symptom Checkers

One of the ways digital tools can help is through the use of symptom checkers. These tools have the potential to help the growing workload of clinicians by helping practices to allocate resources more effectively. A study in 2019 predicted that by 2028/29, there will be a shortfall of 11,500 general practitioners (GPs) in the UK[1] – posing a serious threat to the sustainability of primary care services. In June 2021 alone, appointments in general practice were up just over 13%, compared to the same month before the pandemic.

These web or app-based tools can also help patients understand the likely causes of their symptoms. Some go further by also triaging patients – directing patients to the most appropriate care setting for their symptoms i.e. GP appointment, self-care at home, trip to A&E.

Understanding Performance

We set out to understand just how well existing online symptom checkers perform. To do that, we enlisted the expertise of specialist independent data consultancy, Methods Analytics, and their findings have recently undergone a rigorous peer review and been subsequently published on PLOS ONE.

The study set out to understand:

  1. The performance of symptom checkers as diagnostic and triage tools.
  2. The potential impact of symptom checkers on clinical capacity and utilisation.
  3. The variations in performance across selected symptom trackers available today.

Researchers ran 50 clinical cases through twelve publicly available symptom checkers to test their efficacy. The overall result showed significant room for improvement, with the correct diagnosis being present in the top five diagnoses in 51% of cases.

Doctorlink emerges best-of-breed

Doctorlink, part of the HealthHero family of companies, was one of only two symptom checkers that proved to be consistently accurate – with the correct diagnosis listed first in 68% of cases, compared to an average of 38% and in the top 5 diagnoses in 86% of cases, versus the average of 51%.

Doctorlink continued to outperform other providers ranking first with 90% accuracy of its triage, and 100% on the safety of its triage advice – compared to averages of 58% and 83%. And when reviewing the relationship between triage accuracy and patient safety, Doctorlink also scored highest. This relationship is an important measurement, as a good digital triage tool needs to be reliable and provide clinically appropriate advice with necessary safeguards in place. And the outcome of this study shows that Doctorlink has successfully struck the right balance.

Additionally, evidence shows that Doctorlink was the most efficient in ensuring patient safety without having a negative impact on healthcare resources. When compared against NICE guidelines, other symptom assessment tools in the survey recommended a higher level of care than necessary in over 50% of cases. In comparison, Doctorlink, only recommended a higher level of care than necessary in only 18% of cases. Furthermore, where self-care was the most appropriate solution, Doctorlink recommended patients use a healthcare service in only 12.5% of cases, compared to an average of 52% across all tools.

Understanding whether online symptom assessment tools are recommending the right level of care is key in analysing the impact digital tools have on the healthcare system. With clinical workloads continuing to rise, referring minor conditions that could be managed at home only places a further strain on finite resources.

Overall, the study concluded that with the exception of high performing outliers, like Doctorlink, symptom checker tool performance is below was would be expected of medical devices in any other field. And as most tools are typically risk averse and the accuracy varies considerably, the benefits symptom assessment tools bring to clinicians remains unclear, in terms of how much they really mitigate capacity challenges.

A Need For Greater Regulation

Despite its own strong performance, Doctorlink is committed to championing greater rigour and assessment of tools, like online symptom checkers, to ensure that all such tools consistently deliver a safe service to patients and prove a valuable tool for clinicians. This commitment is also evident in Doctorlink’s recent collaboration with the Care Quality Commission (CQC) to help shape future policy guidelines and regulatory frameworks for the use of digital health products across the UK.

Dr Ben Littlewood-Hillsden, Chief Medical Officer of Doctorlink commented: “Healthcare is a high stakes industry, with clinical teams being responsible for the lives of their patients. This is an argument in itself for why we must mandate the use of proper tools that provide consistently accurate results. These have the potential to be transformational, but they simply aren’t at the moment because many are just not sophisticated enough to be truly fit-for- purpose.”

These findings also echo an ORCHA (Organisation for the Review of Care and Health Apps) report published earlier this year, which stated 80% of health apps available don’t meet standards for quality.

To learn more about Doctorlink and the performance of its online symptom assessment, get in touch.

[1] Beech J, Bottery S, Charlesworth A, Evans H, Gershlick B, Hemmings N, et al. Closing the Gap: Key Areas for Action on the Health and Care Workforce. London: The Health Foundation, The King’s Fund & Nuffield Trust, 2019 [Internet] KingsFund [cited 2020 Apr 12]. Available from: https://www.kingsfund.org.uk/publications/closing-gap-health-care-workforce