Apps for prescribers: how safe and effective are they?

Medical apps are increasingly being used by prescribers to support their day-to-day practice. But with huge numbers of new apps becoming available all the time, and no formal accreditation process in place, how can prescribers assess their safety and usefulness?


If you need information about a drug in a hurry, what is your first port of call? Increasingly, the answer for many prescribers is their smartphone. Smartphone medical applications (apps) can provide clinicians with evidence-based information in an easily portable, readily accessible form on a wide range of issues at the point of care. This can include instant access to the BNF and the BNF for Children (BNFC), providing practical guidance on prescribing, dispensing and administering medicines, with vital information on indications and dosage, cautions, interactions, side-effects and contraindications. Other medical apps, widely available from different sources, also provide medical information and advice on prescribing, some with dosage calculators for a wide range of conditions and some that present topics in pictures, tables, algorithms and videos.

Small wonder then that growing numbers of clinical prescribers regard them as an invaluable resource to support their work. In addition to the numerous studies relating to the use of apps to be found on PubMed (, one recent survey for the Royal College of Physicians (RCP) of junior doctors (in press) revealed that about half of the 1200 respondents believed that apps were essential to their work, and “supporting prescribing” was one of the main categories they found useful.

NICE and the Royal Pharmaceutical Society, which collaborated in producing apps for the BNF and BNFC, have seen their apps constantly remain in the top 15 downloaded medical apps since they were launched in 2012. The average monthly downloads for the BNF app rose from 5620 in 2014/15 to 8672 in 2015/16 and average monthly sessions rose from 307,015 to 380,000. The corresponding BNFC average monthly download figures rose from 2006 to 4170 and average monthly sessions rose from 56,331 to 71,916 over the same period. The apps, automatically available to all NHS employees via their Athens account, are updated monthly.

The huge daily access rate of the BNF app is an even greater measure of its success, says Vincent Doyle, NICE associate director – design and development, information management and technology. Mr Doyle says: “We had many experts curating, reading and trying out the BNF app before it went live and we felt that was quite important. There was a higher risk factor considered at NICE – rightly so because it was new territory – and we felt we could not risk anything being incorrect. I think if people are cautious, careful and quality assured of the content, the benefits of a digital application can be tremendous.” Meanwhile the NICE Guidance app, which is open to the public, currently has 1000 new downloads per month and an average of 42,000 active sessions per month.

Are apps safe?

While they have great potential to improve care for patients, not all apps are as rigorously tested as those of NICE. There are huge variations in the quality, reliability and data protection in these apps, which provide potential dangers, says Professor Jeremy Wyatt. As clinical adviser on new technologies at the RCP, and professor of digital healthcare and director at the Wessex Institute of Health and Research, University of Southampton, he is well qualified to talk about the benefits and risks of apps.

Professor Wyatt says the developers of apps range from “literally, a teenager in his bedroom” with no medical knowledge, maybe downloading information from the web to help a school friend with diabetes, to huge, well-established organisations with highly qualified medical experts, high standards of editorial judgement and excellent reputations they need to guard.

He adds: “This is one of the big challenges because there are zero barriers to market entry and therefore the quality and standards vary enormously. We have empirical evidence showing there are some misleading, in fact potentially dangerous, apps and others that are of very high quality.”

Safety checklist

Professor Wyatt calls misplaced overconfidence about apps ‘apptimism’ and says that the current methods of assessment – user rating, professional reviews, developer self-certification and regulation – are inadequate. Together with his team, Professor Wyatt has introduced the RCP Health Informatics Unit 18-point checklist (see Table 1) in a bid to help clinicians stay safe by properly evaluating the “structure, functions and impact” of selected medical apps.1


Table 1. Royal College of Physicians (RCP) Health Informatics Unit 18-point clinical app quality checklist (reproduced from Wyatt et al. 20151)

The RCP’s recent survey also revealed that nearly half of respondents were concerned about some aspects of app quality – and their concerns are justified, believes Professor Wyatt. He explains: “For example, a study of 23 calculator apps for converting opioid drug dose equivalents2 found dangerously large variations in calculated dose, with fewer than half the apps recognising that the conversion formula used should depend on the actual dose as well as on the drugs concerned.” Another study found that the delete key on many apps does not work correctly, so this can lead to inadvertent and unnoticed errors.3

The increased use of medical apps has also raised important ethical questions relating to privacy and the sharing of information, particularly where apps are used to store a patient’s data – often in an unencrypted form. As Professor Wyatt remarks: “Devices get lost, they get borrowed by other members of the family or peers, etc. Professional, thoughtful app developers will consider that risk – those who aren’t do not.” One study examining privacy issues of “accredited health and wellness” apps on the NHS Health Apps Library4 (which closed in 2015) found that 89 per cent (70 of the 79 apps studied) transmitted information to online services, half of which included strong identifiers, and four apps (5 per cent) sent health-related information together with personal identifiers without encryption (see Table 2).


Table 2. Findings of the study by Huckvale et al,4 on unaddressed privacy risks in accredited health and wellness apps

Evaluating new apps

Another significant challenge is the huge volume of apps constantly appearing: 1000 new apps a day on the Apple iTunes platform alone, translating to about 80 new apps a day relating to health and lifestyle. Maintaining existing apps is also demanding because if they are not maintained they will be deleted from the app store after about six months, which can lead to a perverse incentive to make trivial changes and increase their ranking, notes Professor Wyatt.

Individual clinicians can use the RCP structured checklist to examine apps and check out their safety, but how can clinicians generally navigate their way safely through the huge sea of apps appearing on the scene? A project led by Public Health England and NICE aims to solve the problem. It is designed to develop a new endorsement model to replace the NHS Apps Library, which closed after being swamped by the volume of new apps and amid controversy about the poor-quality, insecure apps it was recommending.

The development of a new endorsement model for healthcare apps is one of the workstreams set up by the National Information Board (NIB) to implement the NHS Personalised Health and Care 2020 framework,5 launched in November 2014. It includes a four-step endorsement process, ranging from self-assessment by app developers through some form of crowd sourcing to an independent evaluation of an app by an official body of appropriate experts (for the small number of apps that get through the rigorous tests).

The aim of NIB Workstream 1.26 is to provide “citizens” with access to a set of health and care digital apps that have been endorsed by the NHS to enable them to “manage their health better and deal more effectively with illness and disability”. The most recent report on the plan in November 2015, quotes Public Health England’s deputy director for digital, Diarmaid Crean, as saying there was “a 50 per cent chance we won’t pull this off because it’s so complicated.”7 At that time, the plan was to have the process for accrediting apps in place by March 2016, but more recently, it appears to have disappeared from the radar and its future is unclear.

How do you choose your app?

Dr Pradeep Dhal, a physician at Berrywood Hospital, Northamptonshire Healthcare NHS Foundation Trust, treats patients with mental health conditions, who often have polypharmacy, so he needs a medical app that enables him to check for potential interactions between drugs. He has more than one but the one he tends to use most, he says, is Lexicomp (, a cut-down version of which can be accessed via UpToDate (

So where does Lexicomp come from? Andre Rebelo, global public relations and analyst manager at UpToDate, Wolters Kluwer Health, which owns the Lexicomp drug database, says the apps are funded entirely by individual subscribers and hospital/trust subscriptions and not by sponsors. He stresses that the apps are subjected to a rigorous editorial process by 53 medical doctors on the staff and “more than 6300 physician authors, editors and peer reviewers from around the world”. He adds: “Our unparalleled due diligence has earned the trust and confidence of over 1.1m clinicians worldwide.”

Dr Dhal says he has chosen to use the stand-alone Lexicomp app because he wants to have access to its extensive interactions facility that enables him to investigate interactions relating to psycho­pharmacology. So why not use the free BNF app? He explains: “The BNF app is like a book, so you have to open that chapter, then go looking in it to find whether there’s any interaction or not – whereas with the Lexicomp you just add in and check. It pulls up and lists the potential interactions and classifies them according to severity. You can look at a single medication and it will pull up all the potential interactions in a really helpful colour-coded way: red is a really big problem, a light yellow is a minor problem. You obviously still need to exercise clinical judgement but it’s helpful as a quick ‘go-to’ that’s in my pocket all the time. I don’t need an internet connection to be able to look at that database and I don’t have to carry all my books around with me.”

What Lexicomp does not give you is the UK listed cautions or contraindications, so Dr Dhal then turns to the BNF. He no longer has a paper BNF, but uses the app because it’s updated monthly and is in his pocket. He also uses the BNFC app because the patients he is currently working with are aged under 18 years.

The other decision-making app he uses is the BMA Best Practice decision support app for clinical information that is regularly updated and has a rigorous checking process.

Do his colleagues use medical apps? It seems not. As well as working as a clinical doctor, he teaches undergraduate medical students, and he observes: “The only group I know who use them is medical students and that’s because from the word go, they’re given access to them, so it’s embedded in their learning.”

Is it changing the way he works? “Definitely. Because I am a meticulous person I tend to apply that to prescribing and psychopharmacology, so it’s natural for me to think: ‘What’s this medicine going to do for someone, will it cause a problem with other medicines they’re taking?’ It’s really helpful then not to have to break off and go to books or ask other people. You have all the information at your fingertips.”

GPs develop an app-etite

GPs, too, are discovering the value of apps. Professor Maureen Baker, chair of the RCGP, says: “Medical apps and gadgets can be hugely useful for both patients and healthcare professionals – and there is certainly potential to use such technology to support prescribing professionals, including GPs.


Top: the NICE guidance app has an average of 42,000 active sessions per month; the BNF app attracts around 380,000 sessions per month

“To have something like the BNF in app form, for instance, is very convenient, and allows GPs to compare different drugs and doses, without having to spend time looking things up in a book. When we are working to the constraints of a standard 10-minute consultation, anything that maximises the time we have with patients can be beneficial.

“Any apps or devices need to be developed with patient safety and confidentiality as a priority. Prescribing is a core skill for GPs, involving years of training, and we do have concerns about people who aren’t medically trained having access to tools that can be used to make decisions about their own, or other people’s, health.

“However, as long as medical apps are used professionally and in line with safeguarding guidance, then there is no reason that they should be seen as anything other than a positive move forward as we work within an increasingly tech-savvy NHS.

“The potential of technology works both ways and GPs are also using IT to make it easier for patients to order repeat prescriptions as part of Patient Online initiatives, which also offer patients the ability to book appointments online and view their summary medical records.”

What about wider advantages?

One retrospective study by Harvard School of Public Health, published in 2012, examined the impact of UpToDate computerised clinical management systems on outcomes of care in US hospitals.8 It found a small but consistent association between the use of UpToDate and reduced length of stay, lower risk-adjusted mortality rates for some conditions and better quality performance for every condition on the Hospital Quality Alliance metrics. It was in the smaller, nonteaching institutions where the benefits of UpToDate seemed most pronounced, compared with the bigger teaching hospitals, where the benefits seemed “small or nonexistent”, the researchers concluded. However, UpToDate points out that these small effects translate into larger ones when multiplied by large hospital populations and extrapolated out to the NHS.

Professor Wyatt notes that when an app appears to work well on sample data, the most reliable way to test if it helps in managing patients would be a large, well-designed randomised controlled trial (RCT). However, his search in September 2015 found only 20 app RCTs and he stresses there is an urgent need for more.

Declaration of interests

None to declare.

1. Wyatt JC, et al. What makes a good clinical app? Introducing the RCP Health Informatics Unit checklist. Clin Med 2015;15(6):519–21.
2. Haffey F, et al. A comparison of the reliability of smartphone apps for opioid conversion. Drug Saf 2013;36(2):111–7.
3.Thimbleby H, et al. Unreliable numbers: error and harm induced by bad design can be reduced by better design. J Royal Soc Interface 2015. DOI: 10.1098/rsif.2015.0685.
4. Huckvale K, et al. Unaddressed privacy risks in accredited health and wellness apps: a cross-sectional systematic assessment. BMC Med 2015;13:214. http://bmcmedicine.
5. NHS. Personalised health and care 2020. Using data and technology to transform outcomes for patients and citizens. A framework for action. November 2014.
6. National Information Board. Workstream 1.2: providing citizens with access to an endorsed set of NHS and social care apps. March 2015.
7. App endorsement: 50% chance of success.
8. Isaac T, et al. Use of UpToDate and outcomes in US hospitals. J Hosp Med 2012;7(2):85–90.


Apps for prescribers: how safe and effective are they?

Add yours ↓
Web design and marketing agency Leamington Spa