Demonstrating Be My AI: where AI is more than 2 letters to catch attention

Since its early days, Be My Eyes has proven itself as one of the most effective initiatives in the blind community, serving as a connection between blind users and volunteers from all around the world who could easily lend their eyes to help blind users accomplish various tasks. When the GPT-4 AI model was introduced, Be My Eyes announced Be My AI, an addition to the Be My Eyes app that enables image descriptions using the GPT-4 AI model. The announcement materialized soon after as a feature in the Be My Eyes on iOS, as usual. Later, the feature reached Android, lacking the essential feature of sharing images directly within the app. With the introduction of Be My Eyes 2.2 for Android, the missing sharing feature was added. Let’s take a look at Be My AI and how to use it.

What is Be My AI:

Be My AI is a feature within the Be My Eyes app that leverages the capabilities of the GPT-4 AI model to describe image content and present the results in conversational human language.

While other blind-related apps provide detailed image descriptions, including Microsoft’s Seeing AI, which was released on Android on the same day that Be My AI became publicly available on the platform, Be My AI distinguishes itself by offering users the ability to ask follow-up questions. This means that after an image is described, users can inquire about specific details or seek suggestions based on the image content.

How to Use Be My AI:

Since Be My AI is a feature within the Be My Eyes app, you’ll need to install Be My Eyes first. Once installed, launch the Be My Eyes app and log in. Setting up Be My Eyes is a straightforward task.

Once setup is complete, you can start benefiting from Be My AI by either taking pictures to describe them or sharing images from other apps to be described.

Taking Pictures:

1. Open the Be My Eyes app and activate the Be My AI tab, located among the available tabs at the bottom of the screen.
2. You’ll encounter a simple window with a button to take a picture. Note that no detection occurs before capturing the image, so you cannot use Be My AI to assist with focusing things or people, or to identify what is in the camera view.
3. After capturing the photo, wait for the description result to be announced. If you’re using TalkBack, you’ll hear the result automatically. However, if you’re using Jieshuo, you’ll need to locate the message containing the description on the screen or by swiping to read it.
4. If you wish to discuss the image further, activate “ask more.” Here, you can write your message and send it, and you can accompany it with another image if needed, useful for comparisons or clarifying unclear details.
5. You’ll be notified that the AI is writing a message, and then you’ll hear the response automatically through your screen reader. You can easily review it along with previous messages.
6. You can continue asking more questions or tap on “call a volunteer” to seek human assistance.
7. When you’re finished with the chat, simply go back to end it. Currently, you’ll be asked to rate your experience with the chat, where you can choose either that you had a good experience or that you faced issues. However, you won’t be prompted to include further details.

Sharing Images from External Apps:

If you come across an image, such as a photo sent to you via a messaging service, you can send it to be described using Be My AI. Simply use the share option (you can learn more about sharing on Android here) and then select “describe with Be My Eyes.” Wait until the image is uploaded and analyzed. The same window and options described above when taking pictures within the app apply here as well.

Testing and Remarks:

The results obtained using Be My AI are impressive. The GPT4 model already demonstrates significant potential in recognizing image contents and presenting them in a friendly, human-like manner. The ability to ask follow-up questions makes a big difference, opening the door for numerous helpful use cases. It’s worth noting that compared to Google’s Gemini or Microsoft’s Co-pilot, which also utilize GPT4, Be My AI exhibits less strictness when dealing with medical papers. For instance, when I showed a blood test and asked about any abnormalities, I received a message with sufficient details. When I specifically inquired about a certain result, I also received relevant information. Of course, it consistently reminded me of the importance of consulting a doctor to examine those results, but it effectively fulfilled the task of reading the tests and answering related questions.

Although Be My AI is a highly useful feature, it’s essential to pay attention to a few points:

1. Be My AI utilizes GPT AI capabilities, meaning that GPT’s weaknesses, such as hallucinations, misidentifications, and other limitations, should be considered when using the feature.
2. Encountering errors when sharing images with the Be My AI service is not uncommon. This may be due to high load or other issues. In such cases, perseverance is necessary until you can successfully get the image described. It’s important to remember that this service is provided for free before lodging any complaints.
3. Currently, there is no way to save a photo after receiving the description. Adding such an option would be very useful in certain situations, especially after ensuring that the captured image is what was intended, such as a document or a particular scene.
4. When encountering an error while taking a photo, users need to retake the photo again. Adding an option to keep the photo and try again would be a beneficial addition.

Despite these points, Be My AI service is another groundbreaking feature presented by Be My Eyes. Even in its current state, the feature is capable of making a positive impact in the lives of many blind people. Writing about projects like Be My Eyes always fills me with pride and excitement. As AI models evolve and advance, it’s reassuring to know that there are individuals and companies willing to provide AI capabilities to blind people worldwide without hefty subscription fees.