Abstract:
For a long time, whether it is a TV, a monitor, or a smartphone, images and sounds often come from different locations. Viewers see characters speaking on the screen, but the sound usually comes from speakers below the screen or from external sound equipment. Now, a Korean research team has developed a new display technology in an attempt to completely solve this problem, allowing sound to be emitted directly from the sound position in the picture, thereby bringing a more immersive audio-visual experience.
Researchers at the Korea Institute of Mechanical and Materials (KIMM) recently announced an "audio-visual fusion display technology" based on a modular grid structure. This technology installs multiple vibration actuators on the back of the display panel, allowing the screen itself to become a large speaker and able to precisely control sound to emit from specific areas of the screen.
This means that when someone is speaking in the upper left corner of the screen, his or her voice will also be synchronously transmitted from the corresponding area, as if coming directly from the position of the character's mouth. As the content of the picture changes, the sound emission area can also move in real time. For example, when a sphere flies rapidly from one side of the screen to the other, the sound will also move along the same trajectory, thereby enhancing the sense of space and realism.
Researchers point out that humans naturally associate sound sources with visual targets in the real world. People can determine the location of dropped objects and can tell who is speaking with their eyes closed because the sound itself is directional. However, the visual information and sound sources of existing display devices are usually separated from each other, and this inconsistency always affects the immersive experience.
Although stereo and surround sound systems have been able to create a sense of direction through multiple speakers, the sound still mainly comes from the devices arranged around the screen, and it is difficult to completely correspond to the specific position in the picture on large monitors. There have been some previous solutions to allow the display panel to emit sound directly, but the sound output point is usually fixed and is easily limited by the screen size and frequency range.
The KIMM team’s new solution achieves dynamic control of the sound emission position and a scalable design. The prototype device created by the researchers is approximately 100 inches in size and consists of a transparent LED film and an aluminum panel, with 80 voice coil motors (VCMs) installed behind it. These motors will move back and forth after receiving electrical signals, causing the panel to vibrate and push the air to form sound waves. The principle is similar to that of a traditional speaker diaphragm.
However, getting a specific area of the screen to sound is not easy. Because vibrations propagate throughout the panel, adding to or canceling out other vibrations, it's difficult to control exactly where the sound is coming from. To solve this problem, the research team first analyzed the response characteristics of each area of the panel to vibration, and then used a mathematical algorithm called the "Moore-Penrose pseudoinverse method" to calculate the driving voltage required for each motor, thereby accurately focusing the vibration energy on the target area.
This method does not simply start the nearest motor, but uses multiple motors to work together to form a local vibration field on the screen. When the sound source in the video moves, the vibration field will also move synchronously to achieve a dynamic tracking effect.
Real-time sound localization also faces another challenge. The vibration waves in the panel will affect each other, and the transmission efficiency of different motors to different areas is also different. When the system constantly recalculates the position of a moving target, the control process may become unstable.
In this regard, the research team developed a set of selective excitation algorithms to exclude motors with low vibration transmission efficiency from the calculation, and also designed a technology that can smoothly connect continuous acoustic signals to reduce noise and interruptions during sound movement.
In order to further improve the level of automation, the researchers also introduced sound source separation and localization technology based on artificial intelligence. AI can automatically identify the sound target in the video screen, extract the corresponding sound from the complex audio signal, and then accurately project it to the corresponding position on the screen without manual intervention.
Test results show that the prototype system has an accuracy of 99.9% in selecting the target sound area. The difference in vibration amplitude between the active area and the inactive area reaches up to 35 decibels, proving that the system can highly concentrate vibrations in designated areas and effectively suppress vibrations in other areas. Tests using an acoustic camera also confirmed that the sound output position can continuously follow a moving target on the screen.
Relevant research results have been published in the Journal of Mechanical Science and Technology. Currently, this technology has demonstrated good vibration field positioning capabilities in the low and medium frequency ranges below 500 Hz.
In order to adapt to display devices of different sizes, the research team adopted a modular grid design. Multiple display modules can be freely combined or split as needed, so in theory it can be used for home TVs and can also be extended to commercial display scenarios such as large LED display walls and digital signage. The researchers also said that when multiple modules work simultaneously, the system can produce a low-frequency effect similar to that of a large woofer.
The research team believes that unifying sound and images on the same surface can significantly enhance the audience's immersive experience. In addition to the television field, this technology is also expected to be used in scenarios such as media arts, stage performances, exhibition displays, and vehicle-mounted display systems in the future.
Dr. Jin Dongjun, the leader of the research project, said that the team plans to further expand this technology to modular LED splicing display systems and promote its development into a new generation of core technology suitable for a wide range of display devices.
However, the researchers also admitted that although the system can simulate some of the effects of a large bass unit, it is still difficult to completely replace a professional independent sound system in terms of overall sound quality, sound pressure performance and full-band coverage capabilities. Even so, this technology successfully solves the long-standing spatial misalignment problem of "pictures in one place and sound in another", making everything on the screen look and sound more real and natural.
Comments