LLM Message Summarization
Android Auto - Google
While communication is currently the most popular use case for the Assistant while driving (for example, read message CUJ accounts for 47% of AAP traffic), UXR shows that users still regularly use their communication apps on their phones, which poses safety risks and results in lower platform penetration.
Among the 68 million daily incoming messages on AAP, users only read 3% of messages through Assistant. Besides the common user understanding issue, Assistant fell short in complex messages, such as multiple messages, group messages, and long messages. For example, for group messages, accounting for 40% of all incoming messages, Assistant reads each message in the thread sequentially, which is ineffective and sometimes disruptive. UXR indicates that users want Assistant to recognize and read only important messages.
Here we introduce message summary feature that highlights the most important information, so users can easily consume the messages and respond appropriately. As a result, we expect higher user engagement and platform penetration.
About
Principles
-
Large language models (LLMs) should be used sparingly and only when necessary, it should run with minimal or no TPU cost
-
Aim for solutions that use minimal processing power and be voice modality optimized (eg avoid verbose responses).
-
Improve what users already know and use. Rather than creating new user intents which require discovery & education, focus on optimizing experiences users are already familiar with.
-
Let the ser call the shots, unless you’re we have high confidence in their preferences
Considerations
Driver Distraction
Designing for the vehicle is never as straight forward as it may seem. In addition to designing beautiful implementations, we also must take into consideration the environment we are designing for, and automotive tends to be the most complex. Take into account limitations around on-screen and verbal string length, font/icon/button sizes, no animations, all features must be accessible within three taps, and more.
Various Screen Sizes & Inputs
It all begins with an idea. Maybe you want to launch a business. Maybe you want to turn a hobby into something more. Or maybe you have a creative project to share with the world. Whatever it is, the way you tell your story online can make all the difference.
LLMs
The big unknown. As google learns more about LLM capabilities, we must take into consideration cost, hallucinations, limited control of the outcome, and ensuring key information is pulled from messages, every time.
Various Platforms
Designs must scale for both Android Auto Projected (AAP), and Google Automotive Services (GAS) implementations. AAP integrations are 100% owned and controlled by Google, while GAS implementations enable full OEM control and customizability,
Launch
We recently launched the beta for Message Summarization, where Google Assistant provides summaries of both group messages and long text messages. This feature is particularly useful for drivers who receive a high volume of messages and need to stay focused on the road.
This new capability was released in version 14.52 of the Google app, which is currently in beta testing.
What People Are Saying
“I was impressed with how [message summaries] gave the important key takeaways from the lengthy messages and left everything else that wasn't really needed behind.”
— Naquan T. | 36| Buford, GA
“[Message summaries] save time, energy, take less of your focus off the road…I can just get the quick rundown [of my messages] and then continue on with my drive.”
-Micha U. | 35 | Farmington, NM
“Overall, [message summaries] do a good job at relating the content in a condensed, shorter format than the full message.”
-Dmitriy Z. | 38 | Lake Villa, IL
UX Learnings
-
Large Language Models (LLMs) might occasionally disregard instructions in the "Preamble" that specify the desired word or sentence count. Be aware of this potential limitation when designing prompts.
-
Tester-provided examples can directly enhance the model over time. This continuous learning cycle leads to consistently better results.
-
Participants felt message summaries can miss important details, the tone of a message, the context, and the nuances of a conversation.
Because of this, many (8/21) preferred to hear full messages:
From work contacts
From close personal contacts
When the message is personal in nature (about themselves or someone they know, includes feelings and thoughts)
When there is creativity, humor, or sensitive information in the message
-
Some users said the summary was too concise and didn’t give them enough details to know how to reply
When the message included multiple topics, it missed one of them entirely
Several participants reported the summary missed an important point in the message
-
Some participants (6/21) indicated they would always prefer to hear the full message instead of a summary because:
They don’t trust AI or Google to know what is important to them, or to capture the most important information
They read the full message after each summary anyway
They want to make sure they understood what was said
It’s not hard to read the full message, and they can stop reading/listening if it’s not important