Amazon announces Halo, a fitness band and app that scans your body and voice

By wazup on August 27, 2020 Comments Closed / 100 views

Amazon announces Halo, a fitness band and app that scans your body and voice

Amazon is getting into the health gadget market with a new fitness band and subscription service called Halo. Unlike the Apple Watch or even most basic Fitbits, the Amazon Halo band doesn’t have a screen. The app that goes along with it comes with the usual set of fitness tracking features along with two innovative — and potentially troubling — ideas: using your camera to create 3D scans for body fat and listening for the emotion in your voice.

The Halo band will cost $99.99 and the service (which is required for Halo’s more advanced features) costs $3.99 per month. Amazon is launching it as an invite-only early access program today with an introductory price of $64.99 that includes months of the service for free. The Halo service is a separate product that isn’t part of Amazon Prime.

The lack of a screen on the Halo band is the first indicator that Amazon is trying to carve out a niche for itself that’s focused a little less on sports and exercise and a little more on lifestyle changes. Alongside cardio, sleep, body fat, and voice tone tracking, a Halo subscription will offer a suite of “labs” developed by partners. They’re short challenges designed to improve your health habits — like meditation, improving your sleep habits, or starting up basic exercise routines.

The Halo band “is not a medical device,” Amazon tells me. As such, it hasn’t submitted the device to the FDA for any sort of approval, including the lighter-touch “FDA clearance” that so many other fitness bands have used.

The Amazon Halo intro video | Source: Amazon

The Halo Band hardware

The Halo Band consists of a sensor module and a band that clicks into it on top. It’s a simple concept and one we’ve seen before. The lack of a display means that if you want to check your steps or the time, you’ll need to strap something else to your wrist or just check your phone.

The three color options for the Halo Band

Amazon will sell lots of different band styles and colors

The band lacks increasingly standard options like GPS, Wi-Fi, or a cellular radio, another sign that it’s meant to be a more laid-back kind of tracker. It has an accelerometer, a temperature sensor, a heart rate monitor, two microphones, an LED indicator light, and a button to turn the microphones on or off. The microphones are not for speaking to Alexa, by the way, they’re there for the voice tone feature. There is explicitly no Alexa integration.

It communicates with your phone via Bluetooth, and it should work equally well with both iPhones and Android phones. The three main band colors that will be sold are onyx (black), mineral (light blue), and rose gold (pink-ish).

There will of course be a series of optional bands so you can choose one to match your style — and all of them bear no small resemblance to popular Apple Watch bands. The fabric bands will cost $19.99 and the sport bands will be $15.99.

Amazon intends for users to leave the Halo Band on all the time: the battery should last a full week and the sensor is water resistant up to 5ATM. Amazon calls it “swimproof.”

But where the Halo service really differentiates itself is in two new features, called Body and Tone. The former uses your smartphone camera to capture a 3D scan of your body and then calculate your body fat, and the latter uses a microphone on the Halo Band to listen to the tone of your voice and report back on your emotional state throughout the day.

Halo’s body scanning feature, which calculates body fat percentage.

Body Scans

Body scans work with just your smartphone’s camera. The app instructs you to wear tight-fitting clothing (ideally just your underwear) and then stand back six feet or so from your camera. Then it takes four photos (front, back, and both sides) and uploads them to Amazon’s servers where they’re combined into a 3D scan of your body that’s sent back to your phone. The data is then deleted from Amazon’s servers.

Once you have the 3D scan, Amazon uses machine learning to analyze it and calculate your body fat percentage. Amazon argues that body fat percentage is a more reliable indicator of health than either weight or body mass index. Amazon also claims that smart scales that try to measure body fat using bioelectrical impedance are not as accurate as its scan. Amazon says it did an internal study to back up those claims and may begin submitting papers to peer-reviewed medical journals in the future.

Finally, once you have your scan, the app will give you a little slider you can drag your finger on to have it show what you would look like with more or less body fat.

Halo uses AI in Amazon’s cloud to create the 3D scan and calculate body fat percentage

That feature is meant to be educational and motivational, but it could also be literally dangerous for people with body dysmorphic disorder, anorexia, or other self-image issues. I asked Amazon about this directly and the company says that it has put in what it hopes are a few safeguards: the app recommends you only scan yourself every two weeks, it won’t allow the slider to show dangerously low levels of body fat, and it has information about how low body fat can increase your risk for certain health problems. Finally, although anybody 13 years of age and up can use the Halo Band, the body scan feature will only be allowed for people 18 or older.

Halo listens to your voice and guesses at your emotional state

Tracking the tone of your voice

The microphone on the Amazon Halo band isn’t meant for voice commands; instead it listens to your voice and reports back on what it believes your emotional state was throughout the day. If you don’t opt in, the microphone on the Band doesn’t do anything at all.

Once you opt in, the Halo app will have you read some text back to it so that it can train a model on your voice, allowing the Halo band to only key in on your tone and not those around you. After that, the band will intermittently listen to your voice and judge it on metrics like positivity and energy.

It’s a passive and intermittent system, meaning that you can’t actively ask it to read your tone, and it’s not listening all of the time. You can also mute the mic at any time by pressing the button until a red blinking LED briefly appears to show you it’s muted.

Amazon is quick to note that your voice is never uploaded to any servers and never heard by any humans. Instead, the band sends its audio snippets to your phone via Bluetooth, and it’s analyzed there. Amazon says that the Halo app immediately deletes the voice samples after it analyzes it for your emotional state.

It picks up on the pitch, intensity, rhythm, and tempo of your voice and then categorizes them into “notable moments” that you can go back and review throughout the day. Some of the emotional states include words like hopeful, elated, hesitant, bored, apologetic, happy, worried, confused, and affectionate.

We asked Amazon whether this Tone feature was tested across differing accents, gender, and cultures. A spokesperson says that it “has been a top priority for our team” but that “if you have an accent you can use Tone but your results will likely be less accurate. Tone was modeled on American English but it’s only day one and Tone will continue to improve.”

Data Privacy

Both the Body and Tone features are innovative uses of applied AI, but they are likely to set off any number of privacy alarm bells. Amazon says that it is being incredibly careful with user data. The company will post a document detailing every type of data, where it’s stored, and how to delete it.

Every feature is opt-in, easy to turn off, and it’s easy to delete data. For example, there’s no requirement you create a body scan and even if you do, human reviewers will never see those images. Amazon says the most sensitive data like body scans and Tone data are only stored locally (though photos do need to temporarily be uploaded so Amazon’s servers can build the 3D model). Amazon isn’t even allowing Halo to integrate with other fitness apps like Apple Health at launch.

Some of the key points include:

Your Halo profile is distinct from your Amazon account — and will need to be individually activated with a second factor like a text message so that anybody else that might share your Amazon Prime can’t get to it.
You can download and delete any data that’s stored in the cloud at any time, or reset your account to zero.
Body scans and tone data can be individually deleted separately from the rest of your health data.
Body scans are only briefly uploaded to Amazon’s servers then deleted “within 12 hours” and scan images are never shared to other apps like the photo gallery unless you explicitly export an image.
Voice recordings are analyzed locally on your phone and then deleted. “Speech samples are processed locally and never sent to the cloud,” Amazon says, adding that “Tone data won’t be used for training purposes.”
Data can be shared with third parties, including some partners like WW (formerly Weight Watchers). Data generated by the “labs” feature is only shared as anonymous aggregate info.

Halo’s activity tracker does more than count steps

Activity and sleep tracking

The body scanning and tone features might be the most flashy (or, depending on your perspective, most creepy) parts of Halo, but the thing you’ll likely spend the most time watching is your activity score.

Amazon’s Halo app tracks your cardio fitness on a weekly basis instead of daily — allowing for rest days. It does count steps, but on a top level what you get is an abstracted score (and, of course, a ring to complete) that’s more holistic. Just as Google did in 2018, Amazon has worked with the American Heart Association to develop the abstracted Activity score.

The Halo band uses its heart monitor to distinguish between intense, moderate, and light activity. The app combines those to ensure you’re hitting a weekly target. Instead of the Apple Watch’s hourly “stand” prompts, the Halo app tracks how long you have been “sedentary.” If you go for more than 8 hours without doing much (not counting sleep), the app will begin to deduct from your weekly activity score.

The Halo band can automatically detect activities like walking and running, but literally every other type of exercise will need to be manually entered into the app. The whole system feels less designed for workout min-maxers and more for people who just want to start being more active in the first place.

Speaking of heart tracking, the Halo band doesn’t proactively alert you to heart conditions like a-fib, nor does it do fall detection.

The Halo band’s sleep tracking similarly tries to create an abstracted score, though you can dig in and view details on your REM sleep and other metrics. One small innovation that the Halo band shares with the new Fitbit is temperature monitoring. It uses a three-day baseline when you are sleeping and from there can show a chart of your average body temperature when you wake up.

Halo Labs, partnerships, and the subscription

Finally, Amazon has partnered with several third parties to create services and studies to go along with the Halo service. For example, if your health care provider’s system is compatible with Cerner, you can choose to share your body fat percentage with your provider’s electronic medical records system. Amazon says it will also be a fully subsidized option for the John Hancock Vitality wellness program.

The flagship partnership is with WW, which syncs up data from Halo into WW’s own FitPoints system. WW will also be promoting the Halo Band itself to people who sign up for its service.

There are dozens of lower-profile partnerships, which will surface in the Halo app as “Labs.” Many of the labs will surface as four-week “challenges” designed to get you to change your health habits. Partners creating Labs range from Mayo Clinic, Exhale, Aaptiv, Lifesum, Headspace, and more. So there might be a lab encouraging you to give yoga a try, or a set of advice on sleeping better like kicking your pet out of your bedroom.

Amazon says each Lab needs to be developed with “scientific evidence” of its effectiveness and Amazon will audit them. Data crated from these challenges will be shared with those partners, but only in an aggregated, anonymous way.

Virtually all the features discussed here are part of the $3.99/month Halo subscription. If you choose to let it lapse, the Halo band will still do basic activity and sleep tracking.

In charging a monthly subscription, Amazon is out on a limb compared to most of its competitors. Companies like Fitbit and Withings offer some of the same features you can get out of the Halo system, including sleep tracking and suggestions for improving your fitness. They also have more full-featured bands with displays and other functionality. And of course there’s the Apple Watch, which will have deeper and better integrations with the iPhone than will ever be possible for the Halo band.

Overall, Halo is a curious mix. Its hardware is intentionally less intrusive and less feature-rich than competitors, and its pricing strategy puts Amazon on the hook for creating new, regular content to keep people subscribed (exercise videos seem like a natural next step). Meanwhile, the body scanning feature goes much further than other apps in directly digitizing your self-image — which is either appealing or disturbing depending on your relationship to your self image. And the emotion tracking with Tone is completely new and more than a little weird.

The mix is so eclectic that I can’t possibly guess who it might appeal to. People who are more serious about exercise and fitness will surely want more than what’s on offer in the hardware itself, and people who just sort of want to be a little more active may balk at the subscription price. And since the Halo band doesn’t offer the same health alerts like fall detection or abnormal heart rate detection, using it as a more passive health monitor isn’t really an option either.

That doesn’t mean the Halo system can’t succeed. Amazon’s vision of a more holistic health gadget is appealing, and some of its choices in how it aggregates and presents health data is genuinely better than simple step counting or ring completion.

We won’t really know how well the Halo system does for some time, either. Amazon’s opening it up as an early access program for now, which means you need to request to join rather than just signing up and buying it.