Fitbit claims its users cost their employers a lot less in health care — here’s why that study is flawed

People who used Fitbits cost their employer nearly 25 percent less in health care costs, according to a new study from Fitbit. At first blush this appears to be a tantalizing result for companies looking to save money, but a closer look shows that gains aren’t as dramatic as they appear.

Here’s the claim: Fitbit says that employees at a large company cost almost $1,300 less to employers when they used a Fitbit for two years. These numbers look great, but there are flaws in how the study was conducted.

The study began with a first year where no one had a Fitbit; this data was used as a baseline for the “before and after.” Afterward, people who opted to be studied received a partially subsidized Fitbit. Fitbit then compared this group’s insurance claims against a control for the next two years.  This is what’s called an opt-in study — of the roughly 20,000 employees at the company, Fitbit recruited about 900 volunteers.

In general, opt-in studies like this one create the possibility that those who join are already healthy and that’s why their medical claims are going down. Fitbit worked to prevent this by creating a control group that roughly matches the age, gender and previous health information of the participants. This is the only ethical way to run a study in the real world. Still, it’s best to have a kind of study called a randomized control trial, where half the people who want to participate don’t get a Fitbit at all. Randomized controlled trials are considered the gold standard because they help ensure that the results are from the Fitbit, rather than differences between groups of participants.

All this matters because of how popular Fitbit is with companies that have “corporate wellness” programs. Companies provide health insurance to their employees. If their employees get sick often, they need to pay more. So, around 80 percent of companies do things like subsidize gym memberships to encourage everyone to be fit. Businesses including Target, IBM, and Bank of America have all bought subsidized Fitbits for employees, so it’s important to figure out if this is working.

With that in mind, let’s take a closer look at the numbers. The Fitbit group saw a 24.7 percent reduction in their medical claims, and they did cost the company less. But those in the control group also saw a 9.3 percent reduction. Everyone’s health care costs declined.

An employee could opt out of the study at any time, according to Amy McDonough, vice president of Fitbit Group Health. More people started the program than completed it, though Fitbit has not kept track of how many left. Those who dropped out stopped sharing their activity data, but they were still included in the final results as a “less active” user, or someone who logged fewer than 182 days.

Finally, people had to use their Fitbits pretty consistently to see a gain. About 350 participants used the gadget for at least 274 non-consecutive days over two years. These were the ones who saw a big reduction in medical costs. The difference was much smaller for people who logged less than those 274 days. And there was little significant change for these “less active” users.

It’s common sense that you can’t benefit from Fitbit if you stop using Fitbit. But this is a key problem that raises questions about the long-term effectiveness of activity trackers as a corporate wellness strategy. Abandonment rates are high across the board. One research firm reported that over a third of people ditch their “smart wearables” within six months. Another study published just today — which used the Fitbit Zip specifically — concluded that it doesn’t really help health, probably in part because 90 percent of people abandoned it after a year. If people won’t continue using the Fitbit, the initial numbers might not matter. Having medical costs go down at the beginning and then rise again when employees abandon the gadget is not the ideal outcome for any company.

Fitbit has been getting a lot of heat recently. A recent JAMA study found that fitness trackers in general can backfire when it comes to weight loss by providing a false sense of security. The Verge also discovered bugs that resulted in accurate distances during a recent review of the new Charge 2 wearable. In response to today’s study on how the Fitbit doesn’t work, a spokesperson said that, “We are confident in the positive results our millions of users have seen from using Fitbit products.” This is all well and good, but we need to take a closer look at where these positive results come from, and how long they last.

Correction: After publication, Fitbit emailed to say it had miscommunicated the status of participants who dropped out. They were, in fact, included in the final results.