This past week we were invited to attend the mHealth Connect Conference sponsored by the Stanford Mobilize Center. During the conference we were excited to meet Karla Gleichauf, a Research Data Analyst at Fitbit. Karla was presenting some of her recent work exploring the impact and efficacy of the Reminders To Move (RTM) feature. For those of you not familiar with this feature, on certain Fitbit devices users are able to set a time window during which they can receive notifications to encourage obtaining a minimum of 250 steps every hour. If you haven’t met your 250 steps goal for a given hour, your Fitbit will gently vibrate and display a message reminding you to get up and get those steps in.
Why remind you to move? It won’t come as a surprise to those of you following the physical activity literature in the last decade, but sedentary behavior has become a particularly important behavior to understand, measure, and intervene on. Time spent in sedentary behaviors has been shown to be related to a variety of negative health consequences causing some to ask, “Is sitting the new smoking?” The number of interventions that researchers are testing in order to encourage more all day movement have been increasing over time as well. We were very interested to learn about Karla’s work on understanding if Fitbit’s RTM feature was having any impact. What follows is a brief interview we conducted with Karla. Responses have been edited and slightly condensed for clarity.
Karla, it was great to learn about your research. Before we dive into your findings, can you tell me a little bit about how the Reminders To Move feature came about?<
Karla: A group of researchers, designers, and product managers had been thinking about how to tackle sedentary behavior for a few years and we started to push for it to become a feature. But, before we did any massive deployment, we conducted a few small scale experiments of some prototypes. We enlisted some friends and family to try a feature prototype to see how it worked on a small set of people before making a “real” feature.
So, we had this small group testing it, and the early results were really promising! The short story was that our testers were really enjoying the reminders they became significantly less sedentary. This gave us enough information to go back to the larger team and make the case for Reminders to Move (RTM) as a widely available feature. Fitbit successfully launched the feature in April of 2016.
So here we are one year later and you’ve done some work to see if those reminders are making a difference. How did this research project start?
Good question. So, that beta testing program we initiated, like I said, it was mostly friends and family that took part. They’re super engaged people, and we wanted to understand if their experience would be born out in the “real world” with our broader user base. So we set out to design a study that could explore whether RTMs were making a difference, and if so, what was the magnitude of that difference.
On of the most interesting part of this research was your methodology. Can you explain a bit more about how you structured the study?
We compared tens of thousands of Fitbit users’ behavior before and after they had RTM. This allowed us to explore each user’s historical data to understand their baseline behavior and to measure the increase in steps they took each hour and each day. In order to reduce bias and confounding factors such as seasonality, we created a control group of users who never received RTM using propensity score matching, a commonly used statistical technique in clinical and medical studies. Using this methodology, we determined how much more active users became both initially with RTM and after a few months with RTM to assess the "stickiness" of RTM.
So what did you find? Do the reminders work?
They do! When we split up our sample to look at how the RTMs impacted people who were really sedentary prior to having the feature (meaning they get 250 steps in three or less workday hours), we found great results! As you can see in the poster, this group of sedentary people really reacted to the reminders, with 70% moving more after just two weeks. And this behavior was persistent, 60% kept their higher daily activity levels at the two-month mark. These sedentary users were moving more, receiving fewer reminders during their day, and getting more daily steps - almost 450 steps per day more on average at the two-month mark than their baseline daily steps.
Now, I know that 450 steps (approximately 0.25 miles) doesn’t seem like a lot, but that’s actually a huge deal at this population level. Based on internal Fitbit data, we can see when changes occur during the year, because of seasonality or other events or trends. At this scale, we’re used to seeing daily steps change by about 40 steps from week to week. Reminders to Move are associated with an over 10x improvement over just the normal ebb and flow of activity over time. We feel this is significant, not just for showing that this feature works, but in helping people sustain positive behavior change and live healthier, more active lives.
As you know, we primarily work with researchers and clinicians that are using Fitbit devices in their studies. What can they learn from your process?
I think people might jump to the fact that we have such a large number of users and large amounts of data to work with, but remember, this all started with a pretty small group of people and that small trial and it’s data led us to where we are. We’re lucky to have the data we have, but there is a also strong case for doing these type of studies at much smaller scales.
The other piece of advice is that when you’re doing this type of analysis you have to be patient and make sure you’re asking the right question. I mentioned that we looked at how the RTMs impacted sedentary users. Well, we didn’t start with that question. We took a step back and asked ourselves, “Who are these reminders for? Who are we really trying to help?” By taking that step back during the exploratory phase we were able to refine and hone our process so we could tell the right story.
So what’s next? Anything else you all are thinking about that you can share?
Well, I can say that on a broad level we’re really interested in becoming more personalized. We think that using data to provide better guidance and personal interventions is really interesting. You’re already starting to see some of that with our new guidance tools. We’re also really interested in learning more about how we can use our data to keep individuals motivated and engaged in their health.
We did a short survey recently that asked our users about their favorite features, and Reminders To Move was near the top. So we’re excited to explore more and showcase what works. This research we showed… it’s just the beginning.
Thank you to Karla and the Fitbit research team for speaking with us about this exciting new research.
Are you interested in conducting research like with with Fitbit devices? Do you have a novel idea for an intervention or clinical trial using wearables? We can help. Get in touch today to learn more about how we can help!