Pint of Science: Oh Data, Where Art Thou?
May 15, 2017
In this post, I provide some background on the health data talk I gave on May 15, 2017, at Pint of Science, Edinburgh. (Slides)
The central argument of the talk is that any data we collect about health and wellbeing have no meaning in themselves – they need to be interpreted in context. Take step counts, for example. Measuring step counts is a somewhat inexact science, because the signals picked up by the accelerometers in a phone or a dedicated pedometer or actigraph need to be converted into the metric of steps (Berendsen et al, 2014; Fulk et al, 2014). Rating threads about pedometers like the FitBit or Jawbone often contain disappointed comments about bad measurements (too many steps counted, too few steps counted, failure to detect stair climbing).
Step counts also need to be interpreted in the context of the person who is taking the steps. 6000 steps in a day is impressive for somebody who barely walks, but an indication of a lazy day for somebody who usually averages 10000 or more.
So, we need to bear two contexts in mind if we want to interpret objective data such as step counts, the context of measurement in which the data were acquired, and the context of the person who generated the data.
When estimating the probability p(cause | symptom) that somebody has a certain condition, such as depression, given the signs they exhibit, such as activity levels measured in step counts, it’s worth considering several related probabilities:
- p(symptom). The probability that somebody exhibits the symptom. If the symptom is very common, it’s unlikely to be a strong indicator for the cause, especially if it can have multiple causes. A classic example is the humble cough, which can be a sign of the common cold or an indicator of lung cancer.
- p(cause). The probability that the cause occurs. This is the old adage “When you hear hoofbeats, think horses, not zebras.” Unfortunately, rare diseases are more frequent than one might think.
- p(symptom | cause). When you look at the diagnostic criteria for most illnesses, you will often find a list of several symptoms, together with the qualification “if two or more of these indicators are present, then …”
Even worse, diseases commonly occur together (Mokraoui et al., 2016), and some of these may have overlapping symptoms.
So, what should we do when we read about yet another algorithm that can diagnose depression? First of all, every diagnosis, in particular when it comes from algorithms, should be treated as a working hypothesis. In fact, some diseases, such as dementia, can only be diagnosed with absolute certainty after a person has died and their brain has been autopsied (Toledo et al., 2013). Secondly, even if the measurements we take are objective and repeatable, we can only make sense of them in the context in which they were taken, which includes both the person and the (measurement) process.
What do you think – is objectivity possible? Am I too pessimistic?
Berendsen, B. A., Hendriks, M. R., Meijer, K., Plasqui, G., Schaper, N. C., & Savelberg, H. H. (2014). Which activity monitor to use? Validity, reproducibility and user friendliness of three activity monitors. BMC Public Health, 14(1), 749. https://doi.org/10.1186/1471-2458-14-749
Fulk, G. D., Combs, S. A., Danks, K. A., Nirider, C. D., Raja, B., & Reisman, D. S. (2014). Accuracy of 2 activity monitors in detecting steps in people with stroke and traumatic brain injury. Physical Therapy, 94(2), 222–9. https://doi.org/10.2522/ptj.20120525
Mokraoui, N.-M., Haggerty, J., Almirall, J., & Fortin, M. (2016). Prevalence of self-reported multimorbidity in the general population and in primary care practices: a cross-sectional study. BMC Research Notes, 9(1), 314. https://doi.org/10.1186/s13104-016-2121-4
Toledo, J. B., Van Deerlin, V. M., Lee, E. B., Suh, E., Baek, Y., Robinson, J. L., … Trojanowski, J. Q. (2013). A platform for discovery: The University of Pennsylvania Integrated Neurodegenerative Disease Biobank. Alzheimer’s & Dementia, null(null). https://doi.org/10.1016/j.jalz.2013.06.003