OCR Functions and Settings in Jieshuo Screen Reader

Jieshuo Screen Reader utilizes Optical Character Recognition (OCR) to offer text recognition features. These include recognizing text in the currently focused item, the entire screen, or a virtual screen designed to improve accessibility for otherwise inaccessible app screens. Jieshuo supports several OCR engines, including custom ones, with a dedicated settings section.

Recognize Text of the Current Focused Element and Recognize Text on the Current Screen

The key difference between these two functions lies in their scope. The current focus recognition captures a screenshot of the currently focused element and sends it to the OCR engine for recognition. In contrast, the entire screen recognition captures a screenshot of the entire screen and attempts to recognize all the text displayed. Both functions are available in the main menu, can be assigned to gestures, or activated through the voice asistant.

However, for current focus recognition, there are additional ways to perform the function:

1. Using the OCR and Translation Navigation Type
   After adding OCR and translation as a navigation type and switching to it, the gesture to move to the next item or increase the value translates the currently focused text, while the gesture to move to the previous item or decrease the value performs OCR on the focused item.
2. From the Recognition Menu
   The recognition menu, part of the functions menu but also accessible from the main menu, includes a “Text Recognition” function. Initially intended to recognize and translate the currently focused text, it now acts as a focus text recognition tool only.

Recognize Subtitles Function

This function performs continuous OCR recognition on a specific portion of the screen, as defined by the user. It is designed primarily for recognizing video subtitles.

Virtual Screen

The virtual screen feature aims to enhance the accessibility of inaccessible app screens. The results vary across apps but generally enable better focus on items and their activation. It also uses OCR to recognize text, including the names of controls.

OCR-Related Settings

To access OCR settings, navigate to “Screen reader settings” > “Advanced settings”, then activate “OCR Settings”. Available options are:

• Use a dark background for the virtual screen: This option targets partially sighted users and determines whether a dark background is used when displaying results in both the “Virtual screen” and “Recognize text on the current screen” functions.
• Automatically read the recognized text when using the virtual screen or the current screen recognition: Unlike the current focus recognition, which is always read automatically, it is possible to choose whether the automatic announcement should occur for the entire screen and virtual screen recognition. Note that for these two functions, the results are displayed in a new window, whereas with the current focus recognition, the text is only read aloud, and you must select recognition results from the recognition menu to read them again.
• Automatically use OCR: Automatic OCR tries to perform text recognition whenever an item that lacks a label is focused.
• Auto image description: When enabled, this option also acts on unlabeled focused items, but with one important difference—the method used to recognize the item. While the automatic OCR uses the selected OCR engine to recognize any text in the item, the auto image description uses the Blue Heart AI model to try to describe the item.
• OCR engine: Here, the user selects the engine to be used for all OCR-related functions. It includes:
  • Tencent Cloud: This is an online engine that supports many languages. Testing it with both English and Arabic, I find it delivers acceptable results, though not as accurate as the OCR features in popular apps. It gets the job done, especially since it is supported through the screen reader itself.
  • Vivo: The latest supported engine. It is also an online engine but supports fewer languages. English is included, but I still find the Tencent engine more reliable.
  • Custom Baidu OCR data: If this engine is selected, required data such as the API key must be filled in by the user.
  • Custom OCR extensions: If the user wants to use a custom extension and has the necessary data and capabilities to create the OCR extension, they select this option.
  • Offline OCR: This offline OCR engine is for users who don’t want to rely on the internet for text recognition. It supports fewer languages compared to the Tencent Cloud default engine but is much faster, with results delivered almost instantaneously. It is the choice for privacy-conscious users. Supported languages include English, mixed Chinese and English, Japanese, Korean, Russian, German, French, Italian, Portuguese, and Spanish.
  • OCR language: If you have selected Tencent Cloud or the offline engine, you specify the recognition language here.
• Filter keywords while recognizing subtitles: Here, users can replace misidentified common errors with the correct text, so it is used instead. Note that I have no information on whether this works only for the subtitle recognition function or for all OCR-related functions.
• Subtitle recognition default landscape/portrait mode: These options specify the part of the screen that should be recognized when using the subtitle recognition, based on whether the screen orientation is landscape or portrait. There is also an option to always ask, so the position is determined by the user every time the function is used.
• Baidu OCR: Data related to the Baidu OCR engine is added here for users who selected this engine to be able to use it.
• Edit custom OCR extensions: Here, custom OCR engine support extensions are created.

Notes:

1. When performing the current screen recognition or virtual screen, results are provided in a new window with the ability to click items. For example, if a recognized text belongs to a certain item on the screen, tapping on the resulting text clicks the item. The results also include an edit option that opens Granular Editing mode, allowing users to edit the recognized text to copy or export it as a text file.
2. All OCR-related functions and settings are paid. Users must be subscribed to the premium version to access them.
3. The inquire by voice functions can be also helpful in recognizing text by using prompts to read the text included in the image. Additionally, other image and video related description functions can detect text sometimes.
4. The recognition menu includes a CAPTCHA recognition function that users may try if they encounter the old less-used text insertion captchas. It also includes an OCR function with the “exam recognition” name, however, I am unable to get how the function works exactly and what it tries to detect, compared to the text recognition functions discussed above.

Audio Demonstration