January 13, 2026 | 2 minute read
How to Evaluate Technologies for Health Behavior Change in HCI Research
by Predrag Klasnja, Sunny Consolvo and & Wanda Pratt
Text Exploration
In this article, the authors argue that because efficacy studies of a health intervention take a long time and require a lot of resources, they are impractical for HCI researchers; instead, HCI researchers should encourage the evaluations of different parts of their work, and the work should be judged in peer review on these other qualities.
The authors begin by identifying that the “obvious” way to assess the introduction of a new health intervention should be in seeing if the intervention actually worked as intended; they view this as too limited. Instead, success should be based on a view of effectiveness that “tailors outcome measures to the particular intervention strategies a technology employs.”
The basis of this argument appears to be grounded in how peer-review of work at conferences like CHI assess studies. The authors argue that conducting a comprehensive study to understand behavior change “is often not feasible” because work is done on “early-stage and error-prone systems.” Yet medical professionals view behavior change as only truly observable when an intervention has been studied for 24 months or more. HCI studies should therefore not claim that a participant has truly changed their habits, unless such a longitudinal study has been conducted. The authors declare that “few of us, we suspect, really believe that a multi-year evaluation is needed as an initial evaluation of a novel HCI technology.” They again indicate that the problem is operational: that long-term studies require significant resources and large sample sizes, and these are challenging.
Instead of focusing on large studies that prove efficacy of claimed behavior change, it is argued that researchers should focus on identifying the way bugs, technical difficulties, and lacking features impede the use of a system. The authors conclude that uncovering issues like this, along with sentiment related to the use of a system, is “one of the most important contributions that HCI can make to the design of technologies for behavior change.”
Really trying not to denigrate papers we're reading, but what a thin argument: we shouldn't use the recommended and generally accepted way of assessing interventions because it's hard and time consuming? This paper seems to exist solely to change the way other papers are reviewed, probably so they don't get rejected from conferences anymore.