Waymo is testing Gemini as an in-car AI assistant in its robotaxis

Wong writes that the assistant is tasked with having a clear identity and purpose: “a friendly and helpful AI companion integrated into an autonomous Waymo vehicle” whose primary purpose is “to improve the rider’s experience by providing useful information and assistance in a safe, reassuring and unobtrusive manner.” The bot is instructed to use clear, simple language and avoid technical jargon, and is instructed to keep its responses concise in one to three sentences.

According to the system prompts, when a rider activates the assistant via the in-car screen, Gemini can choose from a series of pre-approved greetings personalized with the rider’s first name. The system also has access to contextual data about the rider, such as how many Waymo rides they have taken.

The prompts currently allow Gemini to access in-car functions such as temperature, lights and music. Notably absent from the features list are volume control, route changes, seat adjustment and window controls, Wong points out. If a rider asks for a feature that Gemini has no control over, the bot should respond with « aspirational phrases, » such as, « It’s not something I can do yet. »

Interestingly, the assistant is instructed to make a clear distinction between its identity as Gemini, the AI ​​bot, and the autonomous driving technology (the Waymo Driver). So when you answer a question like, “How do you see the road?” Gemini should not say, « I’m using a combination of sensors, » but should instead respond, « The Waymo Driver uses a combination of sensors… »

The system prompts include a series of fascinating tidbits, such as how the bot should handle questions about competitors like Tesla or the now-defunct Cruise, or which keywords will cause it to stop talking.

The assistant should also avoid speculating about, explaining, confirming, denying or commenting on real-time driving actions or specific driving events. So if a passenger asks about a video they’ve seen of a Waymo hitting something, the bot is instructed not to answer right away and to deflect.

“Your role is not to be a spokesperson for propulsion system performance, and you should not adopt a defensive or apologetic tone,” the prompt reads.

The assistant in the car may answer general knowledge questions, such as about the weather, the height of the Eiffel Tower, what time the local Trader Joe’s closes and who won the last World Series. You are not allowed to take real-world actions such as ordering food, making reservations, or handling emergencies.

Waymo isn’t the only company integrating AI assistants into self-driving vehicles. Tesla is doing something similar with xAI’s Grok. However, the two different car assistants have different functions. Gemini seems programmed to be more pragmatic and ride-oriented, while Grok is presented more as a buddy in the car who can have long conversations and remember the context of previous questions.