A Beginner’s Guide to the Types of Screen Reader Gestures on Android

Have you read some of the TalkBack reviews on the Play Store, especially those with a 1-star rating? If so, you’ve likely encountered frustrated individuals who stated that enabling this built-in software made their phone practically unusable.

Have you ever found yourself in a situation where someone is staring at you while you’re using your phone, looking surprised and unable to understand what’s happening and what you’re trying to do, to the point that they may ask if you need help?

Sighted people may not be aware that when a screen reader is active, many things change, including how items are activated and the functionality of swiping gestures. The screen reader has its own focus that enables navigation and interaction with on-screen elements. Additionally, the operating system introduces specific onscreen gestures for the screen reader to utilize.

Now, let’s explore the various types of on-screen screen reader gestures available on Android.

What are screen reader gestures?

Gestures are a set of movements and taps used by the screen reader to perform specific tasks. These tasks are typically determined by the user’s needs and preferences. In the absence of a screen reader, a swipe is often used to perform actions on an item or to scroll through a list or switch to a different page, such as on the home screen. However, when a screen reader is active, swipes are interpreted as gestures for the screen reader.
For instance, if you’re using TalkBack, swiping right on the screen with one finger is for moving to the next available on-screen element, while swiping left corresponds to navigating to the previous one.
The purpose of these gestures is to simplify tasks for screen reader users and prevent accidental actions.
Moreover, these gestures empower users to perform actions by executing specific gestures, such as opening the notification shade, copying, pasting, and more.

When using a screen reader, you can activate the focused item by double-tapping anywhere on the screen. Focusing can be achieved either by using a gesture to navigate between on-screen elements or by placing your finger directly on the location of the element, or putting it on the screen and moving it until it reaches the item you want to activate.

Types of screen reader gestures available on Android:

1-finger gestures:

With the release of Android 4.1 (Jelly Bean), Google introduced the 1-finger gestures. These remained the only available gestures for several years until Google added multi-finger gestures.
The 1-finger gestures include both 1-directional and 2-directional swipes, resulting in a total of 16 unique gestures.

One directional swipes:

Those swipes are either left, right, down or up. They are performed anywhere on the screen. whether the finger performs the swipe on a blank area or on the actual position of an element doesn’t make any difference.
These swipes do not need to cover a large portion of the screen or be lengthy; even short lines are sufficient to register as swipes.

Swiping back and forth:

These gestures consist of swiping to one direction and then returning in the opposite direction in one continuous motion without pauses or lifting the finger from the screen.
They can be performed as up and down, down and up, right and left, or left and right swipes, and there’s no requirement to draw them as long lines.

Angular gestures:

There are 8 angular gestures:

• Swipe right then down.
• Swipe right then up.
• Swipe left then down.
• Swipe left then up.
• Swipe up then right.
• Swipe up then left.
• Swipe down then right.
• Swipe down then left.

These gestures can be executed by either drawing the first short line and then drawing the second line or by smoothly curving in the required direction, all without removing your finger from the screen or pausing. Long lines are not necessary.
For example, to perform the swipe down and then right, you can either draw a short line down and continue to the right without pausing, or slide your finger down and then curve it towards the right.

Notes:

1. It’s recommended for the swipes of the one-finger gestures to be quick. Slow movement can be interpreted as screen exploration since moving the finger on the screen is commonly used for discovering on-screen items.
2. After completing the gesture you are drawing don’t hold your finger on the screen, to ensure the gesture registers properly and isn’t mistaken for screen exploration.
3. Typically, executing those gestures is flexible. There are no strict rules for a successful gesture other than moving in the correct directions and avoiding slow movement or long pauses in gestures consisting of 2 swipes. However, it’s essential to be aware that the Jieshuo screen reader has additional gestures like edge gestures and long swipes, which can sometimes be triggered by a slight pause during a gesture. By default, these gestures are disabled, but if you choose to enable them, you need to be more deliberate in your gesture execution to avoid confusion with regular gestures. For example, if you activate the swipe down then long swipe right gesture, the regular swipe down then right should be a quick, short movement without pausing between the swipes or after the next swipe. To perform the long gesture, you make the right swipe longer to cover more screen area, and you can make a slight hold between the two lines or after the right swipe, but the hold should be very brief.

Multi-finger gestures:

Google didn’t introduce multi-finger gestures until Android 11. However, Samsung had them exclusively for its Voice Assistant screen reader before that.
Despite the debut of multi-finger gestures with Android 11, Google Talkback couldn’t utilize them until the release of Android 12. Other screen readers could use them, but they caused issues on non-Samsung and non-Pixel devices. On Samsung phones running Android 11, third-party screen readers and Samsung’s version of Talkback were able to use the gestures normally, unlike Google’s Talkback, which couldn’t use them until Android 12.

The addition of multi-finger gestures was welcomed by many, especially those who were not comfortable with using the one-finger available gestures. These new gestures made it possible to have more options for users to customize and assign actions.
Moreover, people transitioning from iOS were more accustomed to multi-finger gestures rather than the one-finger gestures. Multi-finger gestures make use of 2, 3, and 4 fingers.

Swiping:

These gestures involve swiping in any of the four directions using 3 or 4 fingers together. The 2-finger swipes are reserved as replacement gestures for the one-finger swipes when no screen reader is running. For example, scrolling in a list, typically done with a one-finger swipe, is achieved using a two-finger swipe when a screen reader is active.

Tapping:

These gestures are executed by tapping, double tapping, or triple tapping the screen with 2, 3, or 4 fingers.

Tapping and Holding:

Similar to the previous gestures, these involve tapping the screen with 2, 3, or 4 fingers but with the added step of holding the fingers slightly before lifting them off the screen.

Notes:

1. The sensitivity of the screen can influence how gestures are registered. If you find yourself frequently failing to execute gestures, consider performing them near the middle of the screen.
2. If you are a beginner user, it’s recommended that you give yourself some time to become accustomed to gestures. Talkback offers a feature called “Practice Gestures,” which is found in “Customize Gestures” within the Talkback settings. It is also accessed by tapping the screen once with four fingers. Spend some time there familiarizing yourself with how these gestures are executed.
3. This post is intended to provide a quick overview for beginners regarding the available onscreen gestures. It is not meant for an in-depth discussion of what each gesture does by default, or the specific additional gestures third-party screen readers offer. Nevertheless, if you are a blind Android user, it is recommended to take the time to customize gestures to your liking, regardless of the screen reader you are using. When comparing Jieshuo to other screen readers, it stands out as having the most versatile gesture system, allowing you to assign numerous actions for gestures, including extensions and tools.
4. Some Android user interfaces may offer on-screen gestures to perform certain actions. If these gestures conflict with your screen reader, consider turning them off.
5. If you are a sighted person reading this, hopefully, the next time you see someone drawing lines on the screen or tapping it strangely, you won’t be astonished and will be wise in your judgment, setting aside the notion of them being crazy, or coming from outer space.