From pixels to taxels: a new world of opportunities for the blind

At the heart of the BLINDPAD (Personal Assistive Device for BLIND and visually impaired people) project was a single question: How can graphical contents be made accessible through touch? With current technologies essentially relying on visual user interfaces and graphical information, the sense of touch has been neglected, and the need for tactile technologies to facilitate the inclusion of visually-impaired people in modern society has never been so important.

As Dr Luca Brayda, researcher at the Robotics, Brain and Cognitive Sciences Department of the Italian Institute of Technology, explains, the BlindPAD aims to become to digital graphical information and communication what Braille is to text.

What are the main problems with current technology and its use of (or lack thereof) the sense of touch?

Blind people need to use the residual sense of touch to understand information – something not at all achieved for graphics. Overcoming that issue would require the fabrication of a tactile tablet for blind persons, which has been a challenge for decades. This is the concept of ‘sensory substitution’: Much like pixels, one digital image can be formed by a grid of small tactile pins (‘taxels’) that can be programmed to be ‘up’ or ‘down’, therefore forming a bas-relief that can be sensed and understood with the hands.

However, making a dense array of taxels with sufficient force and displacement to be easily felt by any user – that is also low power, fast and compact – is a major technological challenge. Due to the complexity of drive electronics or the lack of performance of actuators, none of the previous technological solutions was shown to be scalable, have sufficient performance and be portable.

How does the BlindPAD represent a breakthrough in this regard?

The BlindPAD has successfully been built and field-tested. It is a system comprising a new tactile tablet, software translating images into tactile representations and a series of exercises that together make digital graphical information accessible to blind and visually-impaired persons.

We have verified the effectiveness of this system especially at the developmental age. Our results contribute to various fields of research. First, to the research field of material engineering – as we have shown that it is possible to build arrangements of small elements able to deliver high forces but with small components. Then, to the field of experimental psychology and cognitive neuroscience, in which the potential abilities of persons with sensory deficits are still debated and to a certain extent underestimated or at least unexploited.

It doesn’t stop there. The project advanced the field of computer-human interaction: since Tactile feedback on non-flat screens had received attention before BlindPAD, but produced very few solutions going beyond academic experimental setups. More generally, it advanced the field of computer-assisted rehabilitation where no standards exist. The access to digital information can be a breakthrough in making rehabilitation practices semi-autonomous.

What can you tell us about BlindPAD components and the way it works?

Our tactile display is 12 cm x 15 cm in size, consisting of 192 electromagnetic, independent taxels. Each taxel is very fast in changing state (up/down), allowing static and moving patterns to be displayed.

Together, the taxels can form arbitrary simple sketches: graphs, symbols, emoticons, conventional signs. Conventional visual contents can therefore be learnt through both vision and touch.

What use cases did you aim to assess with your serious games and why?

We considered two paramount use cases: learning mathematics and learning unknown spaces from maps. We have shown that the BlindPAD successfully trains visual-spatial working memory, complex mental operations and mathematical concepts and helps picture maps of unknown rooms, helping people to find their own position in a real environment.

We believe that learning mathematics is crucial at developmental age: poor knowledge of scientific content undermines skills such as logical reasoning and spatial awareness. A bigger issue is mobility, since tactile maps available in public places are rare, often too complex, costly and sometimes not accessible. An instrument used to display maps dynamically, even lightly like what is being done with Google Maps, does not exist and blind persons are locked out from modern society.

Almost no blind persons specialise in scientific-oriented high schools because of this issue. However, it is known that blind people can develop quite accurate spatial skills. The problem is not how to develop knowledge about space, but rather how to access information about it.

Did the results live up to your initial expectations?

Yes. We expected to make spatial knowledge better with the BlindPAD, and our experiments confirmed that. Although resolution can be improved, a lot of spatial tasks with about 200 taxels can be efficiently performed. The prototype as it is can be used within rehabilitation contexts.

What can you tell us of your commercial strategy? When can the BlindPAD be expected on the market?

The BlindPAD system was built in three years, from scratch: It now comprises a hardware tactile device, software running on most PCs and tablets, and a series of exercises that stimulate abilities linked to spatial working memory, spatial processing, logics and mathematical reasoning.

The system, as it is, can be used in rehabilitation centres, potentially with very high impact on rehabilitation practices. Our prototypes are rather easy to reproduce, therefore the technology can be exploited by transferring it to a start-up. With a relatively low effort in terms of engineering, it can become a product. Further funding could allow us to launch a product on the market within less than two years.

BLINDPAD
Funded under FP7-ICT.
Project website
Project video
CORDIS webpage