Gemini Live Camera Stream and Screen Sharing Impressions: Is Gemini Ready to Be the AI Assistant for the Blind?

I had been waiting impatiently for the Gemini camera streaming and screen sharing features to be rolled out to my device ever since I saw Google’s announcement about extending their availability to free users. Although I briefly tried the preview version, the idea of having these features directly built into the Gemini Android appgave me a fresh, strong feeling of excitement, as a blind person who believes that AI can play a major role in identifying surroundings and assisting with tasks that require vision.

Now, about a week has passed since I received the two features. While I’m still very optimistic about their presence, I’ve had the chance to experience how certain limitations impact their effectiveness, some of which are quite serious. In this article, I’m sharing my quick impressions, along with the observed limitations of the screen sharing and live stream features of Gemini Live.

Note: I mainly used the Gemini 2.5 Flash model (the 2.5 Pro model can be selected, but its use is very limited in the free plan).

Screen Sharing

What Is Screen Sharing and How to Use It

Screen sharing allows Gemini to access your screen so it can answer your questions based on the on-screen content. To enable screen sharing:

1. Invoke the Gemini assistant using your preferred gesture or the wake word while you’re on the screen you want to share, then select the Share this screen button.
2. Alternatively, you can open Gemini Live by using the button that appears when invoking the assistant, or from within the Gemini app. Then, navigate to the screen you want to share, open the Live window from the notification shade, select the item for screen sharing, and confirm to start casting.

After successfully starting screen sharing, you can begin asking your questions. When you’re done, you can:

• Find the Live notification in the notification shade and end the session from there,
• Navigate to the Gemini app and stop sharing,
• Or simply lock the screen; this ends the sharing session and puts Live on hold.

Note that the notification might not appear after unlocking the screen, so your only option in that case is to reopen the Gemini app to either resume or end the conversation. If you choose to resume, screen sharing will not start automatically; you’ll need to manually reshare the screen.

Screen Sharing Observations and Limitations

Sharing the screen enriches the conversation with Gemini. It allows the assistant to identify on-screen content and respond to relevant questions about that content. During my tests, Gemini was able to tell me which icon was currently focused and even described the screen reader’s focus indicator. It also provided information about specific items on the screen. A sighted family member tested the feature while playing Minecraft and asked questions based on his progress, Gemini responded with useful information.

One aspect of screen sharing that especially caught my interest was sharing the camera app screen to get a description of the viewfinder. I told Gemini that I wanted to take a photo of the street view from a window and kept asking for assistance until the street was in focus. It then told me that I could take the picture.

Although screen sharing is a valuable feature, it has some limitations.

Sharing Stops When Media Plays

If you were hoping to use screen sharing to describe YouTube videos as they play, prepare to be disappointed. When Gemini detects that media is playing, screen sharing ends immediately and the conversation is paused. I tested this with several apps—including a music player, and while playing media in Firefox, Chrome, and YouTube—and the result was the same. This limitation removes a potentially important use case for screen sharing.

Misidentifications

No matter how advanced Gemini’s image recognition is, it still misidentifies things—and more commonly, it fails to determine the correct positioning of items. On my home screen, for example, it stated that Chrome and Telegram were next to each other, but that wasn’t the case. While they are close, they are not adjacent. Knowing the exact position of an item can be necessary, especially when dealing with certain inaccessible screens.

No Screen Sharing if an App Doesn’t Allow Screenshots or Screen Recording

On Android, app developers can choose to block screenshots and screen recordings. While this is completely understandable, it can be frustrating for blind users, as it prevents getting image descriptions and using Gemini’s screen sharing feature. If Gemini tells you that the screen is black, you’re likely dealing with one of these apps. WhatsApp, for example, prevents screen sharing in windows where profile photos are visible.

Camera Live Streaming

What Is Camera Streaming?

Camera or video streaming is a feature that allows Gemini to access the feed from your phone’s camera, enabling you to ask questions based on what Gemini sees. To enable video streaming, open a Live conversation and select the appropriate button.

Observations and Limitations

Despite the importance of screen sharing, it’s not the highlight of the experience; the crown goes to camera live streaming, which has the potential to be a real game changer.

When this feature works well, it can be impressive. It can assist in reading text, identifying items, describing people, and more. During my testing, Gemini described people and told me whether they were facing the camera. It identified when someone was wearing sunglasses, described the colors of their clothing, and even noted their gender. It recognized what food someone was eating and described what another person was doing.

Using the phrase “I am blind” often transforms Gemini into a more helpful and considerate assistant; it tries to provide detailed descriptions and as much information as possible.

However, the limitations I encountered make it clear that the feature still isn’t ready to replace a sighted assistant.

So Many Wrong Assumptions and Misidentifications

I should state from the outset that I am not the best at holding a phone’s camera. However, I don’t see that as a justification for the many misidentifications I encountered. My tests were conducted in a risk-free environment with prior sighted verification of all items shown to Gemini. The real concern is when someone uses the feature independently and relies on Gemini’s camera capabilities without external confirmation.

I showed Gemini three medicine boxes—it correctly identified two but gave completely incorrect information about the third. In another case, it identified a custard box as a dishwashing powder. It even acknowledged that the word custard was written on the box, yet insisted it was a dishwashing product until I asked how it could say that when the label clearly stated otherwise. At that point, it retreated with the typical AI-style apology.

The problems continued with brand misidentifications. A can of tuna, for example, was labeled with a completely unrelated brand name that bore no textual similarity to the actual label.

Gemini’s confidence went as far as identifying the expiration date of a medicine—and unsurprisingly, the date had nothing to do with the actual one. In that instance, when I initially showed it a medicine box, it said it couldn’t find the expiration date, which was a better response than making one up. But when I later showed it a different medicine to identify its name, it suddenly “remembered” my earlier question and confidently provided an incorrect expiration date.

It’s completely normal for AI to misidentify things, but what makes matters worse is the nature of the responses. Gemini often speaks with such confidence that an unsuspecting user might take its answers at face value—especially when it repeats the same wrong information multiple times. To be fair, though, I must say that sometimes it announced that the information I asked about couldn’t be deduced from the photo, suggesting moving the camera or the object. It even tried to give me guidance on how to move the camera when I prompted it to do so, but often the guidance it provided was very general and imprecise.

Confusion When Background Speech Is Detected

To Gemini’s credit, it is one of the assistants that handles noisy environments better than most. Compared to ChatGPT, which is easily interrupted in such scenarios, Gemini performs more reliably. However, it’s not perfect.

During my testing, simply hearing someone speaking Arabic in the background was enough to switch Gemini’s responses to Arabic, even when I was asking questions in English. This behavior was consistently observed.

Another limitation appeared when I asked it to describe what was on my TV screen without lowering the volume. It couldn’t continue with the description without being interrupted by the TV’s audio.

For a more effective conversation, it’s best to be in a quiet environment, though that’s not always possible. It’s worth noting, however, that general background noise (like traffic) typically doesn’t interfere, as long as it doesn’t involve speech.

No Automatic Updates or Continuous Video Feed

It is important to understand that when using Gemini’s screen sharing and live camera features, Gemini is not viewing a continuous stream. Instead, it analyzes the current view only after receiving a prompt. While it can relate scenes to provide comparisons or answer follow-up questions on the same topic, this limitation means Gemini cannot continuously monitor the camera feed or provide real-time updates without being prompted.

However, Gemini doesn’t always make this clear. It is not uncommon to hear Gemini promise to notify you when something you asked about comes into view—only for that promise to go unfulfilled. For example, it once told me it would alert me when the street was in view as I prepared to take a picture, but then I was met with silence.

This also applies to screen sharing. Although you can navigate screens and ask questions about them, don’t expect Gemini to provide automatic updates for now.

Final Remarks

Google took the lead by expanding Gemini Live’s camera feed and screen sharing features to free tier users, opening new possibilities for blind individuals who need better descriptions of their surroundings and assistance in understanding the visual world they interact with. However, in its current state, Gemini Live cannot yet serve as the fully reliable assistant many are searching for. The main issue lies in incorrect identifications that are delivered with full confidence.

How the camera is held and its relative position are critical in what Gemini perceives and how it interprets the scene. Therefore, if you are totally blind, it is important to familiarize yourself with these basics before relying on Gemini or other AI services for directions or for judging distances between objects.

To be clear, while Gemini Live’s features are impressive, they must be used with caution, as relying on them without verification could lead to damage or harm. The tendency of AI to hallucinate and confidently present incorrect information remains a notable shortcoming. Although improvements are strongly felt with newer, more advanced models, these issues still exist and cannot be ignored.

Although this article has focused more on the limitations, this approach aims to raise awareness among unsuspecting users about these shortcomings. It is easy to be attracted to such services without considering possible consequences. Nevertheless, I encourage everyone with access to these features to take advantage of their availability to improve independence, while always verifying critical information through more reliable means.