RESEARCH REVIEW: Wrist-worn tech: investment or waste of money?
With the popularity of wearable tech showing no signs of abating, its accuracy once again comes under scrutiny.
Title: Accuracy of Heart Rate Watches: Implications for Weight Management
Authors: Drs Wallen and colleagues. (Centre for Research on Exercise, Physical Activity and Health, Qld. Australia)
Source: PLOS ONE, May 2016. Available free online: journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0154420
Introduction: Every time we publish a new Research Review, we receive five to 10 emails pertaining to the article we have chosen, which is manageable – indeed welcomed. However, when we published our Summer 2015 Research Review on ‘wearable technology’, we set records with regard to feedback! So after lengthy discussions on its pro’s and con’s, we decided to re-visit this hot topic, focusing on the accuracy of these devices for assessing exercise heart rate and the associated energy expenditure.
Let us premise this article by stating that the equipment in our laboratories for accurately assessing heart rate (via electrocardiography) and energy expenditure (via indirect calorimetry using a metabolic cart) costs thousands of dollars! More specifically, a quality, brand name, new metabolic cart will cost anywhere from $35,000 and an additional $10,000 or so for an ECG to determine heart rate (and ECG). Now, common sense must prevail here… if these wearable technology devices for your wrist cost approximately $399 (FitBit Surge for example) to $799 (such as Garmin Fenix 3HR), you can’t realistically expect the same accuracy as a metabolic cart or ECG… or can you?
There is no doubt that these devices are extremely popular in the health and fitness industry (worldwide sales estimated at $12.6 billion USD by 2018), and are continuing to improve in their accuracy. Here, we review research that was conducted at the University of Queensland (UQ), a prestigious institution that conducts a large number of studies each year in the areas of health and fitness.
Matthew Wallen, a PhD candidate at UQ, and colleagues recently investigated the accuracy of four popular wrist-worn monitors (Apple Watch, FitBit Charger HR, Samsung Gear S and Mia Alpha) with regard to heart rate (resting and exercise) and energy expenditure in healthy individuals.
Methods: A total of 22 active participants (males and females, average VO2max ~ 50.1ml/kg/min) aged in their mid 20’s volunteered to participate. All participants followed the same testing protocol which consisted of resting in a supine position (5mins), then sitting (5mins), standing (5mins), Bruce treadmill protocol (9mins), sitting (5mins), cycling at 25 Watts on a stationary bike (six 3-minute stages) and then sitting (5mins). Subjects repeated the above testing protocol twice, wearing two of the wrist monitors (one on each wrist) during each visit. There was a seven day break between the two exercise protocols.
Maximal oxygen consumption (VO2max) was assessed via indirect calorimetry, with the gas analysers (O2 and CO2) and volume calibrated prior to each test. A portable gas analysis system (MetaMax 3B) was used to asses expired gases to estimate energy expenditure during the test protocols. A 3-lead ECG was used to determine continuous heart rate during the test protocols. The four watches were individualised according to the manufacturer’s instructions for the individual’s age, gender and anthropometrical data.
Results: All of the participants completed all of the assessments. With regard to heart rate, the most accurate of the watches (compared to simultaneous ECG) was the Apple Watch which was only (on average) 1.3bpm lower than the ECG. The other heart rate watches also underestimated the actual (ECG) heart rate by varying degrees: Mio ALPHA (-4.3bpm); Samsung Gear S (-7.1bpm); and the FitBit Charge HR (-9.3bpm).
With regard to energy expenditure for the test protocol, all of the devices underestimated the actual energy expenditure. The most accurate was the Samsung Gear S, which only underestimated the true energy expenditure by approximately -9 per cent. This was followed by the FitBit Charge HR (-20 per cent), Mia ALPHA (-34 per cent) and the Apple Watch (-43 per cent).
The authors list the devices’ accuracy for heart rate and energy expenditure in an easy-to-understand table, which can be referenced in Figure 2 (page 6) of the publication.
Conclusions: The authors conclude that all four devices underestimated heart rate (by between 1 and 9 per cent depending on device). Additionally, energy expenditure was also underestimated for all devices, but by far greater degrees (with average device difference ranging from 9 to 43 per cent below the gas analyser results). These are considered poor estimates of energy expenditure.
Pros: This is a good study using established and reliable methods to compare the watches’ accuracy with regard to heart rate and energy expenditure. The authors further comment that the accuracy of the devices may be reduced during higher intensity exercise or while completing resistance training due to movement artifact (error due to movement of the body tissue during measurement).
Given the devices all underestimated heart rate, caution should be exercised when advising clients who may be at risk during higher intensity exercise to base their heart rate upon these devices. This pertains not only to high intensity exercise, but also to exercise at an upper personal limit for your clients.
Would we buy one of these devices? Absolutely; however, the purchase would be based upon our specific information requirements, i.e. the device with the worst accuracy at reporting heart rate would not be a first choice for ‘at risk’ clients.
Cons: It would have been interesting if the authors had completed the testing protocol with low intensity exercise, high intensity exercise and when completing resistance training exercises. These paradigms would make for a good follow-up study.
Associate Professor Mike Climstein, PhD FASMF FACSM FAAESS is one of Australia’s leading Accredited Exercise Physiologists and researchers. firstname.lastname@example.org
Joe Walsh, MSc is a sport and exercise scientist. As well as working for Charles Darwin and Bond Universities, he is a director of Fitness Clinic in Five Dock, Sydney. fitnessclinic.com.au