Android Accessibility: A Blueprint for Google in Spring 2024 and Beyond, in 10 Items

Google, listen closely to the winds of change,

For in your hands lies the power to rearrange.

Spring forth, nurture, and let TalkBack sing,

In this symphony of access, let every heart take wing.

— Poem written by GPT-4 and Microsoft Copilot.

Spring is almost upon us at least in the Northern hemisphere. Each spring reminds us of the need to take a closer look at our plans, make new pledges or learn new habits. Many people do these at the beginning of January, but the concept of renewal or renascence jibes with the spring more aptly.

So it would be a good, so-called springy idea to prepare a blueprint of accessibility-oriented actions for Google. What should Google do in Spring 2024 and beyond to improve accessibility for blind and visually impaired users? Let’s dive in. Will Google listen? Before actually diving in, let’s be more sanguine, but not Panglossian, given the nature of the season.

1. Open up TalkBack’s beta-testing program to the whole universe

What’s the rationale behind Google’s decision to limit TalkBack’s beta-testing to a couple of countries, namely the USA? This is really beyond my ken. After all, Apple, quite notorious for its culture of secrecy, has opened up both its iOS and its VoiceOver beta-testing program to the whole world – noo Developer fees required. The risks and glitches of beta-testing aside, one just needs an iPhone to beta-test iOS betas which also provide access to the latest TalkBack betas.

Now that the Google Accessibility Suite, dearly known by its TalkBack component, has a dedicated but geo-restricted beta-testing team, it would be wise to let everyone in the world with a relatively new Android handset, or whatever phone which runs one of the more recent Android releases, to join the beta-testing program to provide feedback via whatever means Google deems appropriate. As brilliantly pointed out on AccessibleAndroid a while ago – link already provided,

If there is a sincere desire to elevate TalkBack as a central component of Android’s accessibility to the next level, a genuine collaboration between the global blind Android community and Google should occur. This collaboration cannot be achieved with the current Google strategies. Marginalizing capable, experienced individuals simply because they are not residents of a particular country shows disrespect, to say the least.

2. Fully embrace TalkBack and accessibility actions

Google added TalkBack actions as a reading control in TalkBack 13.1. They can also be brought up from the TalkBack menu, or assigned to a gesture. The idea is to replicate the way Rotor actions make life easier for visually impaired iOS users. However, TalkBack actions, or what many developers come to know as accessibility actions, haven’t been fully embraced by Google. Most Google Android apps like Maps, Youtube, Messages, and Phone don’t utilize TalkBack actions – Gmail seems to be an exception. As such, one wonders why Google tends to support VoiceOver actions on iOS quite consistently and drop TalkBack actions on Android like a hot potato. Google should really get to the bottom of this unfulfilled vow.

3. Prevent TalkBack from speaking while recording voice messages

I call this the bane of TalkBack usage. In apps like Telegram, Instagram and WhatsApp, recording voice messages has been complicated by the fact that TalkBack speaks, or reads the screen prompts or notifications, while recording is in progress. This is why you might hear TalkBack when listening to audio messages recorded by most Android users. Google should eliminate this bug, once and for all. Temporarily muting TalkBack isn’t the optimal solution because it makes locating the Record button difficult for many users. And if Jieshuo (AKA CSR) can handle this situation appropriately, Google has no reason to delay the fix.

4. Pull Lookout out of its miserable oblivion

What’s happened to Google’s response to Microsoft’s Seeing AI, Google Lookout? Now that both Seeing AI and Be My Eyes/AI are Android, how is it that they offer a lot more AI-centric features compared with Lookout? At the time of this writing, Lookout was updated on Feb. 20, 2024, and the What’s new section for the app states the following:

Get detailed image descriptions using the new AI-powered caption feature in Images mode, available in English only. Try out the Image Question & Answer feature available in the US, UK and Canada only.

Ah, really? In tandem with Google’s freakish geo-restrictions, the status quo severely hampers the use and adoption of Lookout in most countries and regions. Is the accessibility team, or department, powerful enough to bring about a change in this regard? Does Google really care about the fate of its Lookout offering? Let’s see.

5. Make high-quality offline Google TTS voices available to TalkBack users, without the intervention of third-party solutions

Google TTS, a major component of Speech Recognition & Synthesis, has come a long way. It supports many languages, and a very good number of screen reader users prefer it to other solutions on Android because of the combination of its overall responsiveness, intonation and accurate pronunciations. However, TalkBack can’t use its high-quality offline voices without paid third-party solution like Auto TTS, Multilingual TTS, or a free solution like Voicify, a component of Tech Freedom. Check here on how to use Voicify.

If TalkBack can gain access to high-quality offline voices via these solutions and loopholes, why doesn’t Google make this process easier? For those who have experienced iOS and want to taste Android, high-quality voices mean a lot. Google shouldn’t discourage users by refusing to open the gateway to its more enhanced and more natural voice offerings. It’s worth mentioning that the popular Android screen reader known as Jieshuo (or CSR) can directly use these voices, further questioning Google’s rationale behind limiting TalkBack.

6. Add more languages to Speech Recognition & Synthesis, including Persian

Yes, it might sound overly personal, but now that Speech Recognition & Synthesis supports right-to-left languages like Arabic and Hebrew, why doesn’t Google add Persian (Farsi) support? It’s a bit absurd to be forced to install eSpeak NG on Android phones in order to gain access to languages like Persian for speech synthesis. Of course, it’s not just Persian. Google TTS, or Speech Recognition & Synthesis, doesn’t support many Eastern European languages either. So Google should step up its so-called TTS game to bridge the language gap. And if you ask many TalkBack users, they’ll tell you that Google TTS isn’t very good at language detection and switching, something which forces them to use the aforementioned Auto TTS app. Google should rectify this, too.

Sticking to the concept of speech and languages, it’s hard to believe that TalkBack doesn’t have a pronunciation dictionary. This is in contrast with virtually all major screen readers on Windows and iOS. Even Jieshuo offers a full-fledged pronunciation dictionary. So, in short, TalkBack requires a facility to correct mispronunciations regardless of how accurate TTS engines have become in terms of pronunciation.

7. Clear up the Google TalkBack VS. the Samsung TalkBack mess

A lot has been written and said about this entangling issue – just check this and this to get a sense of what headaches and troubles the status quo has stirred. Samsung is one of the biggest, if not the most prominent face of Android. So it’s idiotic to purchase a Samsung phone only to find out that its TalkBack is behind the official Google TalkBack in terms of features. After all, where has Google’s partnership with Samsung to provide a more streamlined TalkBack offering gone? True enough, Samsung also carries a fair share of the blame here. But Google can rectify the situation by approaching Samsung if there’s a will.

8. Divide the Controls item, in TalkBack’s reading controls, to make accessing edit boxes, combos and check boxes easier

To borrow my own sentences from an article I wrote earlier, one of TalkBack’s reading controls is called, well, controls”. The problem with this reading item is that it piles many useful web-centric controls together. So when this reading item is selected via the three-finger up/down swipes, TalkBack can move to the next or previous button, edit box, combo box, radio button, check box, etc. This means that TalkBack can’t move to one of these controls quickly if desired.

And, speaking of reading controls, Google should allow users to customize the items they want to appear here. I might want edit boxes there, and someone else might want landmarks. Needless to say, TalkBack currently doesn’t detect landmarks on webpages even with its “controls” reading item – yet another major omission. But if a hardware physical keyboard gets attached, TalkBack can easily navigate to buttons, landmarks, edit boxes, etc. This is equally puzzling, and Google should allow the same level of access to controls on the web with or without a physical keyboard.

9. Improve Braille support for TalkBack

Admittedly, this doesn’t belong here, towards the bottom of the blueprint. But since I’m not a Braille user, I had to focus on other items.

TalkBack should provide a so-called gesture commander to create Braille commands users need. For instance, users should be allowed to jump to the next level 3 heading on a website with a custom Braille shortcut, make their own keystroke to navigate by row or column in a table, and so forth. Moreover, with TalkBack’s Braille support, it’s not easy to immediately know when there’s a so-called swipe-down action for items on the web. It’s also no secret that Android’s Braille table support requires a serious overhaul. And, last but not least, TalkBack should support more Braille displays over Bluetooth.

10. Share some of that AI prowess with TalkBack for offline image descriptions

Finally, TalkBack 14.1 offers the much needed offline description of images and graphics. Google describes it this way:

AI-generated descriptions of images (English only).

So Google clearly indicates that the descriptions are AI-generated. But regardless of whatever sort of AI which powers them, the descriptions are mostly bad, very short, and inaccurate, more than what one might expect from a typical AI description. It’s downright awkward and embarrassing to have most women described in photos as “your wife!” What’s more, the descriptions for photos are quite short and fail to provide the essential details, person identifiers or the major aspects of the surroundings. Compared with VoiceOver’s image descriptions on iOS, Google’s approach, which aptly requires downloading some data for offline use, is in need of a true shakeup.

With Gemini taking over almost all aspects of Google’s activities, and with some Gemini modes becoming available for offline, on-device use, expecting Google to share some of that power with TalkBack is only the major logical step in the right direction.

In conclusion…

This blueprint offers 10 major items for Google to look at and, hopefully, to implement in the months ahead. I could have been a rambunctious pettifogger by including more items in my list. Stuff like:

• not moving the cursor when an element, like a heading, doesn’t exist on the screen and one tries to locate the next or the previous element via TalkBack,
• the addition of a continuous reading mode from the current cursor position not from the next item, like virtually all other screen readers,
• the implementation of direct touch-typing for TalkBack users, similar to what VoiceOver offers on iOS,
• the implementation of a gesture to report the cursor position on the screen – again like all other screen readers,

etc. In addition, another worrying trend worth pointing out is the fact that Google apps are gradually facing issues with other screen readers like CSR. For example, Google Assistant cannot be utilized when using the accessibility volume in Jieshuo. Normal voice search and sound search widget work as expected. If Google wants to promote inclusivity and equal access, this should be stopped or fixed whether or not it is deliberate.

However – and despite all of these extra issues which paint a rather gloomy picture, in the spirit of the season, let’s keep the major numbers in the article intact. If you need further non-techie evidence about my emphasis on the number 10, check this article which talks about 10 reasons why spring is the best season.

Back to our own accessibility-oriented discussion, I don’t expect Google to take care of all of these issues, bugs and feature requests in the months or the year ahead. However, I’d very much like to see, and feel, Google’s tendency and volition to move towards a more accessible Android by taking tangible and meaningful steps. So, to put the finishing touches to this blueprint or plan of actions, let’s see what GPT-4 and Microsoft Copilot have in store for Google and its accessibility efforts in the form of a poetic piece of advice:

In the budding spring, where tech blossoms anew,

Google, listen closely, for here’s what you must do:

TalkBack, your companion for the sightless soul,

Needs a fresh coat of petals to make it whole.

Plant seeds of empathy, let them take root,

In every line of code, in every pixel’s pursuit.

For TalkBack isn’t just an app—it’s a bridge,

Connecting sight and sound, across the digital ridge.

And so, Google, tend this garden with care,

For in the spring of innovation, let TalkBack declare:

“Accessibility blooms when hearts and minds unite,

Guiding blind souls toward a more inclusive light.”