Illusion or reality? Does SMS messaging really help the medicine go down?

Mobile health solutions or mHealth encompasses all of the technology and software that combined with mainly smartphones, provide individuals to measure, record, monitor, analyse and report the state of their health. The ultimate aim of course is to keep people well and to help them to get better in the event of illness.

It is easy with mHealth to confuse the enormous potential for technology to bring about change, with its actual ability to do so. Whilst the advances in connected devices and the promise of technology such as Google’s smart contact lenses that will monitor blood sugar levels of diabetics, there is very little evidence that these devices will bring about significant changes in the outcomes that matter, namely an improvement in the health of the person using the device.

The problem with research in this area is that mHealth is such a broad area, covering such a large range of technologies that are applied to a large range of uses. Whilst it is entirely possible that an app on a mobile phone could help in the management of a very specific condition, being able to determine this experimentally is very difficult. There are few rigorous “randomised controlled trials” involving mHealth and those that have been conducted confirm the difficulty of drawing definite conclusions from the promise of this technology.

This was certainly the case in a recently reported meta-analysis study on the effectiveness of text messages in helping chronic disease patients take their medication. This report ended up looking at 16 randomised clinical trials that used a variety of different of different types of message, sent a different number of times, to patients suffering different conditions in a variety of different clinical settings and countries.

The authors of the report ended up coming to the conclusion that text messaging “doubled” the odds of patients with chronic disease sticking to their medication program. On the face of it, this is compelling evidence that a simple and inexpensive technique of reminding people to take their medications is extraordinarily effective. This conclusion then lead to articles declaring

“Text messages are vital tool for increasing medication adherence in chronic disease patients”.

Before getting to the problems with this type of analysis and how it was applied in this case in particular, what may seem extraordinary is that the conclusions of at least one of the studies included in the meta-analysis was “telephone messaging intervention with alerts and reminders sent to mobile phones did not improve compliance with therapy in patients with hypertension”

One may ask how it is possible to get from a study that shows absolutely no effect of text messaging to an overall conclusion that text messaging is effective?

In an invited commentary, three Canadian researchers detailed the many deficiencies of the meta-analysis. They were concerned that the individual trials used self-report of medication adherence which is an unreliable way to measure whether someone is really taking their medications, especially in the case where people know that they are in the intervention group and not the control i.e. they were the ones getting the text messages. Probably more damning was the fact that to show statistical significance, the results from the 16 trials were pooled, despite the fact that each of the groups of people differed substantially from each other.

Was it really reasonable to group patients who had high blood pressure with other groups of people who had HIV? In considering whether a statistical result is significant or not, one has to be able to suggest a reasonable theoretical basis for why that effect should be observed. In the case of text messages, do they act as simple reminders or are they operating at a motivational level? If the latter, are they acting as extrinsic motivators, i.e. are they responding out of guilt or obligation as a response to the messages or do they operate in an intrinsic way, helping people to see the benefits of taking the medications for example?

With a meta-analysis of widely different studies, there is no way of knowing what any of the underlying mechanisms could possibly have been. It is a good example where the principles of data exploration are taken in the absence of any real concern for where that data came from or whether it was in fact at all valid to combine the separate studies.

Unlike clinical trials involving drugs, the trial can be conducted where neither the patients or the doctors know if an individual is getting the drug or the placebo. In studies with text messaging, it is immediately obvious and so it is very hard to remove the placebo effect.

One consistent characteristic of reports on mHealth are that they always end with a comment similar to the concluding remark of these authors that more “High quality trials measuring clinical outcomes are needed.”

David Glance, Director of UWA Centre for Software Practice, University of Western Australia

This article was originally published on The Conversation. Read the original article.