Journey to make Swalekhan Accessible

Recently, I got chance to hear Uma Badve’s interview on 95 Big FM radio channel. Extremely happy to learn that Bandya and his Big FM team are bringing the Swalekhan initiative by Niwant Andh Mukta Vikasalaya (NAMV) in front of people in Pune. Swalekhan is an app intended to train visually challenged at their schooling age to learn to type. The larger vision is to eventually get students to appear for exams on their own without depending on scribes.

Swalekhan app had to be built in Android. NAMV had informed us about this. Simple reasons being desktop computers are more expensive whereas Android devices aren’t. Desktop computers aren’t portable whereas Android devices are. Josh Software must get the credit for Android development and Rails server application development that they carried out to make this app a reality.

At the same time, it was an exciting journey for Tekvision being involved in Accessibility Consulting of this app. I would like to bring to light the Accessibility support that was added to the app and how it got added.

Choosing a keyboard layout

As a first step, students need to learn to type in native Marathi language. First came the choice of keyboard. Although many out there might be used to transliteration, that isn’t a good choice from blind student’s standpoint. For those who don’t know the term transliteration, it means using English characters and English pronunciation rules to closely match the native language pronunciation so that a Transliteration Service can produce the native language characters. In case the Transliteration Service doesn’t find exact match, it produces an auto-complete suggestion list with words belonging to native language. So why not transliteration?

1. Transliteration needs more number of keystroke presses. For example “कमल नमन कर” will be typed as “kamal naman kar”. Actual characters in native language script including spaces are 10, whereas transliteration requires 15 characters including spaces to have the same content typed out.
2. Transliteration causes confusion in context of vowels. For example “Mala gadi mahit ahe” can have “mala” transliterated to either “माला” or “मला” and “gadi” transliterated to either “गाडी” or “गडी”.

On the other hand, Inscript keyboard layout provides advantages like follows.

1. One-to-one mapping of native Marathi character to the key being pressed.
2. Various operating systems today make it available by default as one of the keyboard layouts including Windows Operating System and IOS. Even Android has many apps that provide Inscript layout typing for native Indian languages including Marathi.

These advantages of Inscript keyboard over transliteration typing made Inscript layout a default choice to be taught to end users.

Explicit use of physical keyboard

Any visually challenged individual gets better acquainted with input devices through tactile feel. The on-screen keyboards in Android were accessible but aren’t suitable for typing fast. Any touch interface to be made accessible has the challenge about “Explore-By-Touch” which impacts the speed of interacting. A physical keyboard was hence necessary ingredient. Android’s support to physical keyboard came as a surprise during the prototyping phase before we approached Josh Software.

Even though you had inscript keyboard app installed, connecting and typing using physical keyboard always printed English characters. We had to depend on keydown event to translate keycodes to their Unicode equivalents as it would happen in a desktop environment. In a way, reinventing the keyboard input because of dependency on physical keyboard layout.

Teaching special keys like Alt, Windows, Capslock

This was challenging as these keys are interpreted by operating system and do not even reach your keydown event you might have written in your Android app. We researched on this and came up with a work-around. We had to build an Accessibility Service which allowed listening to these special keys even before operating system would interpret. This service then sent  Intents across to Swalekhan app and then let the app teach those keys. Eventually, we discovered that Android Accessibility Services cannot be prioritized over one another. With no such feature, it was up to the operating system to decide which of the Accessibility Service gets this information. So with this solution implemented, some devices worked well whereas some other devices didn’t quite provide the key down information to our Accessibility Service on priority.

Non-text color contrast

Just like any other typing tutor app, Swalekhan also needed to use images to teach words or characters representing certain words. For example, “प पक्ष्याचा”. In such cases, we needed to ensure that we provide images that would match color contrast schemes. It was Chandrashekhar Sahu from Josh Software who provided us inputs on using Font Files to produce vector graphic images. That’s what was done and Unicode characters were used to represent many such images including non-printable keyboard keys.

Use of audio files with no dependency on Text-To-Speech engine.

For those who aren’t aware, a visually challenged individual uses computers or smartphones using Text-To-Speech (TTS) Software called Screen Reader. For native language speakers, understanding the English accent along with the monotonous tone of the TTS becomes difficult. As Swalekhan app was intended for school students, instead of letting the students face the monotonous tone of TTS, it was decided to provide them audio feedback. We then recommended importantForAccessibility=”no” for text fields and text views. This allowed us to not let a running screen reader read out the content while an audio file is available to provide the same content.

Screen reader like behavior expected from audio files

As discussed in previous section, audio files were chosen over TTS. There was another drawback of current TTS and that was speech feedback for typed vowels. In Marathi, vowels are expressed in 2 ways. Firstly, the full vowels like “आ” and secondly, when suffixed to a consonant like “पा”. In the latter case, the vowel in example is referred to as “काना”. While typing, TTS didn’t differentiate between pronouncing “आ” and “काना” and both were pronounced as “आ”. This made it difficult for a screen reader user to identify whether he has typed “काना” or “आ”.

There was another challenge. While editing or navigating text with arrows, it was important to provide feedback about cursor position, when to pronounce syllables over characters, differentiating delete key erasing vs backspace key erasing, etc. All such challenges prompted almost screen reader like feedback mechanisms that would be needed for a text editor.

Writing and recording content

With not just a tutorial content being expressed, with visually challenged students as end users, the content had to cater to multiple aspects.

1. Never assume examples that are explicitly visual in nature. For example, it is easy to explain cursor/insertion point to sighted person – whenever arrow keys are pressed or mouse is clicked at character position, they can see insertion point moving. Try explaining cursors to completely visually impaired individual who has at max handled braille when it boils down to written text. How does one teach cursors? Content had to bring up examples like “Finger placed on braille character while reading is same as cursor on screen.”
2. With every feedback mechanism needing audio and sound feedback, lot of grammatical phrases had to be further broken down to provide an experience which wasn’t monotonous in nature.
3. Additional set of sounds had to be recorded for entertaining students for their scores and evaluations.
4. Lesson content had to maintain brevity while catering to audio-only way of teaching. With no explicit dependency on visuals, achieving such concise content was a challenge.

With 78 lessons in all including 12 intermittent games, writing and recording task was also shouldered by Tekvision.

Conclusion

In all, one might think typing tutor development would be relatively straight forward given numerous typing tutors already available out there. But building such an app with Accessibility in mind needs a different approach and thought process to achieve the best accessible user experience for the end users. Swalekhan app elaborates this to a great extent. Right from content writing for 78 lessons, to ensuring grammatically correct broken phrases, to screen reader like text editor behavior, to technical intricacies in Android Accessibility, to implementing never before tried keyboard-only input on Android, etc all was a memorable journey helping us contribute our little bit to this great cause!