Skip to content
Wearables & measurementResearch article3 min read

Six wrist and phone trackers: good on heart rate and steps, poor on calories

MoveVitals Editorial Collective · Study published · Reading updated

Original paper:Evaluating the Validity of Current Mainstream Wearable Devices in Fitness Tracking Under Various Physical Activities: Comparative Study. · JMIR mHealth and uHealth · 2018 · Original paper ↗

Key takeaway

In 44 healthy people wearing 6 devices and 2 phone apps, heart rate, steps, distance and sleep duration had a mean absolute percentage error (MAPE) of about 0.10, but energy use reached up to 0.44. Results differed widely by brand and activity, and these are older models, so they do not carry over to today's watches.

Study design

Research article

Reading basis

Published abstract

Reading focus

Design & applicability

The research question

Wearables are marketed for fitness and health monitoring, but their value depends on how close they are to a reference measurement. The authors compared several popular devices and phone apps against gold-standard measurements across resting, walking, running, cycling and sleeping.

Who and how

Comparative accuracy study of 44 healthy participants, each wearing 6 devices (Apple Watch 2, Samsung Gear S3, Jawbone Up3, Fitbit Surge, Huawei Talk Band B3, Xiaomi Mi Band 2) and 2 smartphone apps (Dongdong and Ledongli) at the same time. Five indicators were measured: heart rate, number of steps, distance, energy consumption and sleep duration, during resting, walking, running, cycling and sleeping. The reference was manual measurement of heart rate, steps, distance and sleep, and energy from oxygen consumption. Accuracy was summarised as the mean absolute percentage error (MAPE).

What it found

The authors’ reported results. Read each estimate with its population and uncertainty.

  1. 1
    Heart rate, number of steps, distance and sleep duration had a MAPE of approximately 0.10 across devices, which the authors called rather high accuracy. Energy consumption was poor, with a MAPE of up to 0.44.
  2. 2
    Devices differed a lot on the same indicator. Step counting varied most (Apple Watch 2: 0.42; Dongdong: 0.01), and heart rate varied least (Samsung Gear S3: 0.34; Xiaomi Mi Band 2: 0.12).
  3. 3
    Differences across activity states were small for the same indicator: MAPE ranged from 0.08 to 0.17 for distance and from 0.41 to 0.48 for energy.
  4. 4
    The best single results were Samsung Gear S3 for resting heart rate (MAPE 0.04), Dongdong for steps while walking (0.01), Fitbit Surge for distance while cycling (0.04) and Huawei Talk Band B3 for energy while walking (0.17).

Limits & uncertainty

  1. 1
    Only 44 healthy people were tested, so the results may not hold for older adults, people with illness, or different skin tones and wrist sizes.
  2. 2
    The activities were seminatural, and the abstract does not say how long each was or how sleep was verified against the manual reference.
  3. 3
    These are specific device models from that period; software and sensors have since changed, so the brand rankings should not be used to choose a current device.
  4. 4
    This reading is based on the abstract only.

What to take from it

Heart rate, steps, distance and sleep time from consumer trackers were reasonably close to reference methods in this study, but calorie estimates were not. Treat calories burned as a rough guide at best, and do not base decisions on the exact figure. Brand and activity both mattered.

Reading your own data

Use your tracker's heart rate, steps and sleep duration for trends, and look at calories only as a loose estimate. Compare days using the same device, worn the same way, in similar activities, since error differed by device and by activity.

How to read this evidence

The index does not establish a precise design. Read the methods before making causal claims; check population, comparator, outcomes and uncertainty in the source.

Authors’ structured abstract English

The attributed English abstract is reproduced under the recorded open licence. It is source material, not a MoveVitals interpretation. The prepared analysis above separates the authors’ findings from our interpretation.

Background

Wearable devices have attracted much attention from the market in recent years for their fitness monitoring and other health-related metrics; however, the accuracy of fitness tracking results still plays a major role in health promotion.

Objective

The aim of this study was to evaluate the accuracy of a host of latest wearable devices in measuring fitness-related indicators under various seminatural activities.

Methods

A total of 44 healthy subjects were recruited, and each subject was asked to simultaneously wear 6 devices (Apple Watch 2, Samsung Gear S3, Jawbone Up3, Fitbit Surge, Huawei Talk Band B3, and Xiaomi Mi Band 2) and 2 smartphone apps (Dongdong and Ledongli) to measure five major health indicators (heart rate, number of steps, distance, energy consumption, and sleep duration) under various activity states (resting, walking, running, cycling, and sleeping), which were then compared with the gold standard (manual measurements of the heart rate, number of steps, distance, and sleep, and energy consumption through oxygen consumption) and calculated to determine their respective mean absolute percentage errors (MAPEs).

Results

Wearable devices had a rather high measurement accuracy with respect to heart rate, number of steps, distance, and sleep duration, with a MAPE of approximately 0.10, whereas poor measurement accuracy was observed for energy consumption (calories), indicated by a MAPE of up to 0.44. The measurements varied for the same indicator measured by different fitness trackers. The variation in measurement of the number of steps was the highest (Apple Watch 2: 0.42; Dongdong: 0.01), whereas it was the lowest for heart rate (Samsung Gear S3: 0.34; Xiaomi Mi Band 2: 0.12). Measurements differed insignificantly for the same indicator measured under different states of activity; the MAPE of distance and energy measurements were in the range of 0.08 to 0.17 and 0.41 to 0.48, respectively. Overall, the Samsung Gear S3 performed the best for the measurement of heart rate under the resting state (MAPE of 0.04), whereas Dongdong performed the best for the measurement of the number of steps under the walking state (MAPE of 0.01). Fitbit Surge performed the best for distance measurement under the cycling state (MAPE of 0.04), and Huawei Talk Band B3 performed the best for energy consumption measurement under the walking state (MAPE of 0.17).

Conclusions

At present, mainstream devices are able to reliably measure heart rate, number of steps, distance, and sleep duration, which can be used as effective health evaluation indicators, but the measurement accuracy of energy consumption is still inadequate. Fitness trackers of different brands vary with regard to measurement of indicators and are all affected by the activity state, which indicates that manufacturers of fitness trackers need to improve their algorithms for different activity states.

Sources & attribution

Evaluating the Validity of Current Mainstream Wearable Devices in Fitness Tracking Under Various Physical Activities: Comparative Study.

Xie J, Wen D, Liang L, Jia Y, Gao L, Lei J.

JMIR mHealth and uHealth · 2018-04-12

DOI: 10.2196/mhealth.9754 ↗PubMed · 29650506 ↗Europe PMC · Recorded licence: cc by ↗

Abstract © the original authors, reproduced under the recorded open licence. Translations support reading; use the English record to check terminology and numerical results. Our methodological interpretation is separate from the authors’ conclusions.

MoveVitals editorial interpretation · educational content · no individual clinical review recorded

Sources checked 2026-11-01

Practical guides

Read in context

·Cohort study

Activity–stroke and heart attack curves differ by accelerometer processing method

Among 90,237 UK Biobank adults free of cardiovascular disease at baseline, followed until December 2022 for 1,298 strokes and 2,031 myocardial infarctions, the shape of the link between moderate-vigorous physical activity (MVPA) and each outcome depended on how the accelerometer data were processed. For stroke, hazard fell in a straight line only with the machine-learning metric; for myocardial infarction, two metrics gave a curved decline and activity counts a straight one. The abstract gives no hazard ratios, so it shows that methods disagree, not which one is right, and the findings are observational associations.

Preventive Medicine

In-depth explainersRead study →
·Validation study/review

Smartwatch data as forensic evidence: heart rate is best validated, the rest uneven

A systematic review and meta-analysis of studies on smartwatches from four major brands, framed around digital forensics, found that heart rate was both the most studied and the most accurate measure. Step count, distance, energy expenditure, activity classification and sleep were less consistently validated, and results varied by brand, protocol and reporting. The abstract gives no pooled numbers, and most studies were run in healthy people in laboratories, so it cannot say how a particular watch performs for a particular person in daily life.

Forensic Science International

In-depth explainersRead study →
·Overview of reviews

Activity trackers: about 1,800 more steps a day, but small effects on other outcomes

An umbrella review of 39 systematic reviews and meta-analyses, covering 163,992 participants, found that activity trackers were associated with more physical activity (SMD 0.3–0.6), equating to about 1,800 extra steps and 40 minutes more walking per day, and with a bodyweight reduction of about 1 kg. Effects on blood pressure, cholesterol, glycosylated haemoglobin, quality of life and pain were typically small and often non-significant. The abstract does not say what else came with the trackers in these programs or how much the reviews overlap, so the gains cannot be credited to the device alone.

The Lancet Digital Health

In-depth explainersRead study →
·Validation study/review

Seven sleep trackers: good at spotting sleep, weak on wake and sleep stages

In a sleep laboratory, 34 healthy young adults wore a subset of seven consumer sleep trackers for three nights. Against polysomnography every device detected sleep very well (sensitivity 0.93 or higher) but caught wake poorly (specificity 0.18–0.54), and five of the seven matched or beat research actigraphy (specificity 0.39). Sleep-stage estimates were inconsistent, and the participants were healthy adults aged 18 to 35 in a fixed 8-hour bed window, with each device tested in only 11 to 20 of them, so the results say little about people with sleep problems.

Sleep

In-depth explainersRead study →
Wearables & measurement →