Check out this smart voice assistant demo hacked along with GPT-3 and Siri
A developer has put together a GPT-3 demo that attempts to improve Apple’s Siri voice assistant and enable many more conversational voice commands. In a video posted on Redditshows developer Mate Marschalko the assistant controlling his HomeKit smart home devices and responding to questions in response to relatively vague directions that today’s voice assistants like Siri would normally have a hard time understanding.
For example, the AI is shown turning on Marschalko’s lights in response to the voice prompt “I just noticed I’m recording this video in the dark at the office, can you do something about that?” Later in the video, Marschalko asks the assistant to set his bedroom to a temperature that would “help me sleep better”, and he responds by setting his bedroom thermostat to 19 degrees Celsius.
“Just noticed I’m recording this video in the dark at the office, can you do something about that?”
In a accompanying blog post, Marschalko explains how the demo works. Essentially, it uses Apple’s Shortcuts app to communicate between Siri, GPT-3, and its HomeKit-enabled smart home devices. A voice command to Siri causes Shortcuts to send a long prompt to the AI service requesting an answer in a machine-readable format. Once it gets a response, Shortcuts analyzes it to control smart home devices and/or respond via Siri. The blog post is worth reading in its entirety for a more complete explanation.
It’s an intriguing demo and shows what voice assistants could be capable of by integrating this new generation of AI technology, but it’s not perfect. For starters, Marschalko says each command costs $0.014 per API request sent to GPT-3. The video is also being edited, so it’s unclear if the demo works 100 percent of the time or if comments are selectively shown. A more intelligent backend won’t help either if the speech recognition software can’t understand what words a user is saying in the first place.
The demo raises bigger questions about how we want voice assistants to work. Do you really want to have a conversation with a voice assistant, or do you just want to be able to bark short and simple commands to control it like a robot? Marschalko himself also points to the risk of the assistant saying “unexpected things” in a continued Reddit comment:
Nevertheless, the demo still offers fascinating hints about what kind of interactions with voice assistants will soon be possible as Apple, Amazon and Google integrate this technology into their services. And it’s impressive that these tools are so open and approachable that a relatively slick demo like this can be hacked with widely available consumer software.