On August 13, 2024, Google began rolling out the Gemini Live feature to Android devices. Initially exclusive to the paid Gemini Advanced tier, the feature was soon extended to free Gemini users. By Friday, September 27, 2024, it reached my Samsung Galaxy A55. After testing it, here are my thoughts. My phone’s system language is set to English (United States), and all tests were conducted using the free version of Gemini Live, so I cannot confirm any differences for Gemini Advanced users.
Table of Contents
What is Gemini Live
The following is taken from the Google Blog Gemini Live rollout announcement:
“Gemini Live is a mobile conversational experience that lets you have free-flowing conversations with Gemini. Want to brainstorm potential jobs that are well-suited to your skillset or degree? Go Live with Gemini and ask about them. You can even interrupt mid-response to dive deeper on a particular point, or pause a conversation and come back to it later. It’s like having a sidekick in your pocket who you can chat with about new ideas or practice with for an important conversation.
Gemini Live is also available hands-free: You can keep talking with the Gemini app in the background or when your phone is locked, so you can carry on your conversation on the go, just like you might on a regular phone call.”
Launching and Configuring Gemini Live
Gemini Live hasn’t reached all free users yet. To check if it’s available on your device, open the Gemini app and look for the “Open Gemini Live” button at the bottom-right of the screen or next to the image picker button. You can also find this button in the Assistant window after invoking the assistant. For me, using the older version of the assistant doesn’t allow access to the Gemini app, so having Gemini as your default assistant is crucial to use the Live feature.
If you don’t see the button, you’ll need to wait, but make sure both your Gemini and Google apps are updated.
When launching Gemini Live for the first time, you’ll receive a brief introduction to the feature. You can either read or listen to this information, as it’s automatically read aloud. Activating the feature is straightforward, and the prompts are easy to follow. If your assistant language is set to U.S. English, you’ll be able to select your preferred voice. You can navigate between voices using the next and previous buttons or by swiping left and right—two fingers for screen reader users, and one finger for others. UK English users won’t see this option as there’s currently only one available voice. I had my assistant language set to UK English, but after switching to U.S. English, the voice selection appeared immediately upon reopening Gemini Live.
If you’re using a screen reader, Gemini Live will display a message upon first launch advising you to use earphones to prevent confusion between what you are saying and what the screen reader is speaking. I’m not sure if this message appears for Jieshuo users who do not use the TalkBack spoofing service, though.
During a conversation, you can interrupt Gemini by simply speaking. Additionally, there are buttons to pause or close the conversation. The button labels are placed alongside their descriptions, which requires performing swiping gestures twice—once to hear the button and another time for its description. Tapping the button will activate it.
After closing a Gemini Live conversation, either by tapping the close button or using the back gesture or back button, you can review the conversation messages. However, after each message, you have to navigate through all related options, making swiping through the messages less convenient. Adding accessibility actions in this window is essential.
In the Gemini app, you can access settings by tapping your signed-in account name near the top-right of the screen. Within the settings, you can adjust options like the voice used, whether speaking interrupts Gemini’s responses, and whether the conversation continues after the screen is locked.
My Gemini Live Impressions
What I Like
The idea is cool and intuitive
It’s easy to fall in love with the hands-free experience Gemini Live offers, especially with the ability to interrupt a response without touching the screen, making continuous conversations much smoother. You can also change topics seamlessly by discussing something new or asking an unrelated question. The responses are usually quick, enhancing the real-time conversation experience.
I particularly like that I can lock the screen and keep the conversation going. Even if I leave the app, the conversation continues, allowing me to handle other tasks on my phone while talking to Gemini Live. When Gemini runs in the background, it’s easy to end the conversation from the notification shade or return to its window from the recent apps screen.
If you stay silent for about a minute, the conversation is put on hold, and you’ll need to resume it using the “Resume” button.
A Silent Quiet Screen Reader During the Conversation
I’m impressed that Google got it right this time with how TalkBack behaves during conversations. TalkBack doesn’t interrupt you or Gemini during the chat. Even if you receive a notification, TalkBack won’t read it automatically, which enhances the overall experience. The Jieshuo screen reader also remains silent during the chat.
While navigating window controls within the conversation, Gemini didn’t get confused by the screen reader’s speech. However, I recommend using headphones if you plan to multitask while using the feature to prevent Gemini from capturing your speech and misinterpreting it as part of the prompts.
What I Dislike
Gemini Shortcomings
My main concern when using Gemini Live is the AI service it is based on. As conversations progress, it’s easy to encounter contradictory responses, hallucinations, and incorrect information.
A prominent example involves the statements about what the feature can do. I asked Live if it could alert me when a specific period of time passed, and it confidently stated that it would. However, it didn’t notify me when the time was reached. It seemed unaware that it would go on hold and couldn’t provide a response, making it unable to remind me. Even when I shortened the duration and asked it to tell me when the time was up, it still couldn’t do it.
Another instance occurred when I asked if it could sing. With full confidence, it suggested singing “Happy Birthday” as a test song. Unsurprisingly, it failed to sing and later confessed that this capability was still not possible.
Additionally, Gemini Live insisted there was no pause button in the window for pausing the conversation when I inquired about this option.
While I understand that this is a new service, it is not unreasonable to expect it to be trained to recognize its limitations and clearly communicate what it can and cannot do.
Gemini is very cautious when discussing political topics; it even refused to tell me who the current president of the United States is. I tested this with another country and received a similarly reluctant response.
Moreover, you shouldn’t be surprised if you receive different responses at different times. When I asked Gemini about the latest news, it initially stated that it could not provide updates. However, later in another chat, after asking about the current weather, I inquired again about the news, and it provided the most important event happening in my country. I couldn’t request more details, though, as it suggested I do a Google search instead. Strangely, when I asked for another news item, it fabricated a fake event related to reactions to the previously discussed topic. This raises serious concerns about the reliability of the sources used for news and how easily it can miss facts or create false information. It’s clear that Gemini still struggles to deliver accurate results, especially concerning current affairs.
Gemini Live can remind you of its unreliability through unexpected mistakes. For instance, while giving me a quiz about Android, it inexplicably provided information about the first iPhones, despite the question being about the first Android version. When I indicated that the topic was unrelated, it apologized. In another instance, while discussing famous Lebanese food, it stated that it couldn’t be used to provide medical advice. Again, when I noted that this was unrelated, it offered an apology.
The Limited Current Abilities
Setting aside the previously mentioned issues with Gemini as a service, Gemini Live offers a groundbreaking way to engage with an assistant, with the potential to elevate the overall experience of interaction.
However, Live still has many limitations. Currently, it cannot respond to what is displayed on the screen, and you cannot ask questions specific to the app you are using.
The range of actions Gemini Live cannot perform exceeds those it can. For example, you cannot use it for basic tasks like turning Bluetooth on or off. Moreover, the tasks it can perform are often executed poorly. I asked it to set an alarm, but because I was in the conversation when the time arrived, I didn’t receive any ringtone or notification indicating that the alarm had been activated.
Translation is another area where Gemini Live could improve. When I requested a translation from English to Arabic, it provided the results but used its English voice. After I asked it to speak the result in Arabic, it correctly switched to the Arabic voice. However, when I asked it to translate from Arabic to English, it asked for the text but remained silent until I began speaking in English again.
Less Natural engagement than ChatGPT Voice Conversations
I still don’t have access to the advanced voice mode of ChatGPT, but even without it, I find that the style ChatGPT uses while speaking feels more natural and intuitive. While Gemini Live excels in the hands-free experience—especially with its ability to interrupt responses and function while the screen is locked, along with its quick responses and successful conversation initiation—the conversation with GPT feels more human-like. GPT engages better in discussions and incorporates expressions that enhance the interaction. For reference, you can check out this post.
Gemini Live offers responses similar to regular Gemini interactions, but without the need to tap the microphone button every time you want to send a reply. While this is convenient, adding a bit more liveliness to the tone and style could enhance certain discussions without being overly exaggerated.
No Way to Start/Stop Gemini Live Using Your Voice
Although Gemini Live stated that I could ask the Assistant to launch it using a voice command, this was not true. I couldn’t get the Assistant to open it at all; the only method was to use the button. I should note here, though, that it is possible to launch Gemini Live through connected Google Pixel Buds while the screen is unlocked. Also, there is no separate icon for Live among the apps, and no widget can be added to the home screen for quick access.
Closing a conversation isn’t better. Using prompts like “stop” doesn’t close the conversation; it will simply stay silent until you say something or until it is automatically paused after about a minute passes. I’m unsure if the conversation closes after a certain amount of time, but it would be a useful addition to have a direct command to close the conversation.
Jieshuo’s Accessibility Volume Ongoing Bug
The bug affecting Google Assistant—whether using the old assistant or Gemini—also appears in Gemini Live. When the accessibility channel is used for Jieshuo’s volume, spoken messages cannot be captured. For now, using the media volume for Jieshuo suffices, and during my testing, I didn’t need to force stop Jieshuo to make Live work again.
Final Thoughts
Separating Gemini Live from the Gemini Advanced paid subscription is a welcome development. Gemini Live has significant potential and various use cases; even with its current limitations, it can assist in brainstorming, provide useful suggestions, and facilitate discussions on different topics. However, it’s essential to recognize its limitations as a service based on an AI model. With AI, one should not be deceived by a confident tone—always verify the facts and information provided.
As the backbone of the Live feature, any improvements in the Gemini model will positively impact the feature itself. Additionally, enhancing Live with capabilities like context awareness, voice activation, and the inclusion of more voices with different English accents should significantly improve the user experience.
It’s clear that more updates are expected for Gemini Live in the future. It will be interesting to see how these updates shape the user experience and what will be available in the free version. Attention should also be paid to its compatibility with screen readers in upcoming versions, particularly given the accessibility glitches that occasionally affect Google apps. Speaking of accessibility, it remains uncertain whether the previously mentioned Jieshuo accessibility volume usage bug will ever be resolved, and whether actions on messages will be implemented.
Comments