Researchers have overcome a major challenge in bionic robotics and developed a sensor that, with the assistance of artificial intelligence, can slide over Braille and accurately read Braille at twice the speed of humans. The technology could be applied to robotic hands and prosthetics, providing fingertip sensitivity comparable to humans.
Human fingertips are incredibly sensitive. They can convey details of objects as small as half the width of a human hair, discern subtle differences in surface texture, and apply the right amount of force to grab an egg or a 20-pound (9-kilogram) bag of dog food without slipping.
As cutting-edge electronic skins begin to incorporate more and more biomimetic features, dynamic human-like interactions like swiping will become increasingly important. However, despite advances in soft robotics, reproducing the sensitivity of human fingertips in robots has been difficult.
Researchers at the University of Cambridge in the UK have taken a step closer to reality with an approach that combines vision-based tactile sensors with artificial intelligence to detect features at high resolution and speed.
"The softness of human fingertips is one of the reasons why we are able to grip things with the right amount of pressure," said Parth Potdar, lead author of the study. "Softness is a useful property for robotics, but you also need a lot of sensor information, and having both at the same time is tricky, especially when dealing with flexible or deformable surfaces."
Researchers have set themselves a challenging task: develop a robotic "fingertip" sensor that can slide along a fingertip like a human finger and thereby read Braille. This is an ideal test. The sensor needs to be highly sensitive because each dot representing a letter is packed very closely together.
David Hardman, co-author of the study, said: "There are currently robotic Braille readers, but they can only read one letter at a time, which is different from the way humans read. The way existing robotic Braille readers work is static: they touch a letter pattern, read it, pull it up from the surface, move it over, lower it to the next letter pattern, and so on. What we want is something more realistic and efficient."
So the researchers created a robotic sensor with cameras at the "fingertips." Considering that the sliding action of the sensor can cause motion blur, the researchers used a machine learning algorithm trained on a set of real static images that were synthetically blurred to "deblur" the image. Once the motion blur is removed, the computer vision model can detect and classify each letter.
"This is a problem for roboticists because removing motion blur requires a lot of image processing, which is time-consuming and labor-intensive," Potdar said.
The use of trained machine learning algorithms means the robot sensor can read Braille at a speed of 315 words per minute with an accuracy of 87.5%, twice as fast as a human reader and with a similar accuracy. The researchers say this is much faster than previous studies, and the approach can be scaled with more data and more complex model architectures, allowing for better performance at higher speeds.
"Considering we used fake blur to train the algorithm, it was surprisingly accurate at reading Braille," Hardman said. "We found a good balance between speed and accuracy, and so did human readers."
While the sensor is not designed for assistive technology, its ability to read Braille quickly and accurately bodes well for the development of robotic hands or prosthetics with sensitivity comparable to human fingertips, the researchers say. They hope to scale up their technology to the size of a humanoid hand or skin.
"Braille reading speed is a good way to measure the dynamic performance of tactile sensing systems, so our findings could be applied in areas beyond braille, such as detecting surface texture or slippage in robotic manipulation," Potdar said.
The video below, produced by the University of Cambridge, shows how researchers developed a Braille reading sensor.