Alexa Plus arrives with promise but plenty of questions

The biggest surprise at Amazon’s press conference last week was the lack of hardware announcements. Traditionally, Amazon announces dozens of new gadgets at its events, but this time, Amazon spent 70 minutes talking about software. Specifically, Alexa Plus, its new generative AI-powered Alexa voice assistant. And that was exactly the right move.

Over the past decade, the company has spent way too much money building cheap hardware for Alexa that no one really likes, developing home robots and flying indoor cameras no one really needs, and wasting efforts on failed ways for people to interact with Alexa (the Loop, the Microwave, the Clock, and so on), all while the core technology itself stagnated.

Imagine what could have been if Amazon had focused all that time and money on making Alexa really good. For a start, we’d be a lot closer to Jeff Bezos’ original vision of replicating Star Trek’s “Computer” and much further away from what Alexa essentially is today, a very expensive-to-build timer.

But with the launch of Alexa Plus, Amazon has finally taken a big step toward that goal. The voice assistant has been “one hundred percent re-architected,“ Amazon’s head of Devices and Services, Panos Panay, said in an interview with The Verge. “Alexa did need to get better. It did need to grow. It did need to get to the next level,” he said. “Our goal wasn’t just to have a chatbot; it was to have Alexa be able to get stuff done.”

According to the demos Amazon showed, Alexa Plus is conversational, proactive, capable of reason and inference, and can learn from context. It’s seemingly far more capable than its predecessor – which is sticking around, if you’d rather the devil you know.

But the new Alexa can plan a date, book a restaurant, and text the babysitter. It can create a travel itinerary and add it to everyone’s calendar, read your study guide and test you on the answers, remember your favorite movies and food, as well as find your smart devices and take action, Panay said during the event.

It’s not just that Alexa has a whole host of new abilities; it’s also better at doing everything it used to do, but without you having to use precise phrasing or constantly repeat the word “Alexa.”

Now, instead of Skills, we have Experts, which Alexa’s various large language models can smartly tap into to deliver on your requests however you phrase them – from ordering food and turning on lights to telling you who has been at your door and what to cook for dinner. The company claims to have tens of thousands of integrations via APIs as well as new “agentic experiences” that let Alexa navigate and fill out web forms on your behalf to do things like book a cleaner or repairman.

As part of the new software push, we’re also finally getting a new revamped Alexa app, which will host Alexa Plus on your phone, and a new alexa.com website where you can interact with the assistant from the comfort of a keyboard – both of which should be better input methods for Alexa’s new multi-modal capabilities. All of this could take the voice assistant closer to its goal of being a personal assistant rather than just a household utility.

This long-overdue laser focus on improving the core software product could be down to Panay’s influence, who was installed shortly after the first attempt to launch the new Alexa in September 2023. With a background in hardware development, it’s possible he came in, took a look at the array of middling to poor products in the Echo line-up, and issued a stop order.

Panay told The Verge that his vision for new Echo hardware is products that people are passionate about and fall in love with. “The design vision is a little bit more consistency in look and feel and … I believe in screens. I think they matter in a massive way. So you’ll see some investment there.” Shortly after the event, Amazon CEO Andy Jassy confirmed that “beautiful hardware” is coming this fall. No one has ever described current Echo hardware as “beautiful.”

However, the most likely reason for the intense effort to launch the new Alexa is external pressures from the likes of OpenAI’s ChatGPT, Gemini creeping into Google Home, and Apple Intelligence slowly coming to Siri. For a decade, Amazon has coasted along as the “best” voice assistant largely by virtue of its vast interoperability. Alexa could work with almost everything – but that didn’t mean it always did.

Yet, this new, smarter Alexa is still just a promise. The demos I saw were very much on the rails, even though they were happening live. Aside from a few interactions I had with Alexa Plus, no one got to try all the new capabilities and won’t until late March at the earliest. A footnote on the Alexa Plus page indicates it won’t be coming all at once either, stating: “Some features represented are in development and will be released in future updates.”

We still don’t really know what Alexa will do once it’s in millions of homes. This is not your old Alexa. Amazon achieved this transformation by rebuilding the voice assistant from the ground up. According to Panay, there is none of the “classic” Alexa in there. “There was a way to do that – keep the old Alexa and have a new Alexa that doesn’t do anything but is a great conversationalist. But that seemed like it would miss the point,” he said.

Instead, Amazon has built a new voice assistant on foundational LLMs, developing a way to integrate it with APIs to give LLM Alexa the ability to do everything classic Alexa could. This structural change is why it took so long, said Panay.

“Take a timer,” he said. “A timer has moved from ‘Alexa, set a timer for …’, to ‘Alexa, I’m cooking a turkey,’ and Alexa asks you, ‘Do you want me to set a timer?’” And she sets it for the right time for the size of your bird. But to call that timer is not a trivial task, he says. “The engineering team had to hook that API to the LLM and make sure it can do it all without hallucinating,” he said.

This brings me to the biggest question: Will an Alexa powered by LLMs be safe to use in our smart homes, where it has access to our thermostats, our ovens, our door locks, and other vital infrastructure in our homes? Panay says he doesn’t have any fear that Alexa Plus will hallucinate in people’s homes. But, he said, “There’s always risk. I can’t say there isn’t going to be risk.”

This is likely why we may not see all these new features at once. “You need a level of accuracy in smart homes that’s unmatched, unprecedented in the industry. We need to lock the door when you ask,” he said. “We’ve done this for years, so we understand it. How you manage the prompts, how you retrain the models, how you make sure that the Alexa models are fundamentally trained for specific smart home scenarios so hallucinations don’t happen. Take a bit of time. Make sure it’s right. No feature will be released unless we know it’s 100 percent going to get you there.”