Google has started rolling out version 16.0 of the TalkBack screen reader. This release introduces new image description features powered by Gemini AI and new options for navigating tables. As usual, the update is being rolled out in stages and may take up to a week to reach your device. You can still install the update through an APK, but do so at your own risk.
Table of Contents
Gemini AI Image Description Enhancements
TalkBack 16.0 introduces two new features for detailed image descriptions: describing the entire screen and asking follow-up questions.
Describing the Entire Screen
Previously, detailed image descriptions only worked for the currently focused item. With TalkBack 16.0, a new option allows capturing a screenshot of the entire screen and sending it to Gemini for description. To use this, open the TalkBack menu and select “Describe screen.” You can also assign a gesture for this feature in the Customize gestures section of TalkBack settings.
Asking Follow-Up Questions
TalkBack now supports follow-up questions after receiving an image description, whether it’s for the focused item or the whole screen. In the description results window, double-tap “Ask Gemini” to speak your question, or “Type question” to enter it manually.
- When using voice, the question is sent automatically after you stop talking.
- When typing, press the Done key on Gboard or use the appropriate braille gesture to send the question.
Answers are spoken aloud when received, and you can review the conversation through touch exploration or swiping. Next to each description or answer, there are thumbs up and thumbs down buttons to give feedback. To close the window, press the Close button or use the Back gesture.
Initial Observations
It’s too early for an in-depth review, but quick tests show that answers are received quickly. The Gemini model used is capable of describing people, including estimated age and skin tone. However, we noticed that its knowledge base is outdated—it revealed a September 2021 knowledge cutoff during one answer.
We also observed misidentification of some home screen icons, including popular ones.
Google introduced table navigation by columns and rows in TalkBack 16.0. When a table is in focus, select the rows and columns reading control to start table navigation. Then swipe right or left to navigate between columns, or swipe down or up to navigate between rows.
To exit table navigation and return swipes to their normal functions, switch to any other reading control. Note that you can add “columns” and “rows” as reading control items, allowing you to switch to the desired item and navigate using up and down swipes. Additionally, you can assign gestures to move to the next or previous row, and to the next or previous column.
It is also possible to start table navigation automatically when a table is focused by enabling the option “Automatically switch to table navigation reading control when entering a table” found in Advanced settings.
Repeat and Spell Last Spoken Phrase Can Now Be Assigned to Gestures
With TalkBack 16.0, it is now possible to assign the existing TalkBack functions Repeat last spoken phrase and Spell last spoken phrase to any available gesture. To do this, open TalkBack settings and select Customize gestures. Then choose the desired gesture and double-tap the function to assign it.
Toggleable Grouping of Repeated Symbols
TalkBack used to group repeated symbols by announcing the number of occurrences followed by the symbol name. In TalkBack 16.0, this behavior has been made toggleable. To enable or disable symbol grouping when a symbol appears four or more consecutive times in the focused text, go to TalkBack settings > Verbosity, and toggle the Count repeated symbols switch.
Note that this option does not seem to be compatible with most TTS engines. It appears to work only when using the Google TTS engine. In testing with Acapela and AutoTTS, symbols were read individually. With Google TTS, the feature works as expected, and emojis are also included.
Hardware Keyboard Improvements
New Modifier Key for Physical Keyboard Shortcuts
It is now possible to use the Search key on an external keyboard as the modifier key. To select your preferred modifier key, go to TalkBack settings > Advanced settings > Keyboard shortcuts, then activate “Choose modifier key”.
New Shortcut Keys to Move Between Visited Links
You can now press Modifier key + V to move to the next visited link, and Modifier key + Shift + V to move to the previous one.
Reading and Reviewing System Messages on the Connected Braille Display
You can now read system messages, such as toasts or notifications, directly on your braille display.
When a message appears, it will be shown on the braille display. You can dismiss it by pressing Dot 8 or using the routing buttons.
To access missed messages, press Space + Dots 1-3-4-5-7-8 to open the notification history.
Quick Notes
- We have observed a bug where TalkBack starts scrolling automatically when navigating to the previous or next item in a window. The scrolling either stops on its own or continues until the screen is touched. This odd behavior has so far only been noticed in the GSMArena app. Tests on several mainstream apps show no sign of this bug. It is still unclear whether this is a general issue, especially since the device on which we observed the bug is a Samsung device.
- This update applies only to Google TalkBack, not Samsung TalkBack. Although it is possible to use Google TalkBack on Samsung devices (and we do use it), the official version for Samsung devices is Samsung TalkBack. If you have installed the Android Accessibility Suite, you will receive the new update from the Play Store.
- We may update this post if we observe other changes.
witch featchers will be supported on witch devices? like, will a non google phone get thees improvements
They will. Though it also depends on your phone’s manufacturer.
like, e.g oppo, like the a80 or the a5 pro. witch featchers would work
Well with me both the image description and the screen descriptions are on point and to a large extent the talkback descriptions surpass the likes of piccy bot and many others. That makes me to disagree with talkback descriptions are based on old data 2021 or so.
Screen d scription doesn.t appear on my fones with talkback new version.
It works only i th android 15?