It may be hard for you to imagine that among the more than 7,000 active languages ​​in the world, only a few hundred have enjoyed the "favor" of modern speech technology. The vast majority of speakers of human languages—from native African tribesmen, people in the Amazon rainforest, to old people in rural towns still speaking ancient dialects—have always lived outside the narrative of the digital age.

Of the more than 7,000 human languages, only a few are heard by modern speech technology, and now this inequality may be broken. The Omnilingual ASR system released by Meta can recognize more than 1,600 languages ​​and can quickly learn new languages ​​through a small number of examples. With open source and community co-creation as its core, this technology gives every voice a chance to be on the stage of AI.


Voice assistants, automatic subtitles, real-time translation, these conveniences brought by AI seem to be only available for a few "mainstream" languages, and the rest of the language community is still blocked from the technological door.

This digital divide now has a disruptor.

The Meta artificial intelligence research team recently released the Omnilingual ASR system, a family of AI models that can automatically recognize and transcribe speech in more than 1,600 languages, allowing almost all human languages ​​to be "understood" by machines.


This system is shared with the world in an open source manner, and can be expanded into new languages ​​by the community, giving every voice a chance to be on the AI ​​stage.


1,600 languages, just the beginning

The Omnilingual ASR launched by Meta this time has set a new record for the number of languages ​​covered by speech recognition, supporting more than 1,600 languages, including 500 languages ​​that have never been transcribed by any AI system before.

In comparison, OpenAI’s open source Whisper model only supports 99 languages, while Omnilingual ASR almost improves this number by an order of magnitude.


For many people around the world who speak minority languages, this is undoubtedly a "digital revenge": for the first time, their mother tongue has the possibility of being fluently understood by AI.

The recognition performance of this system has reached leading levels in many languages.

According to data provided by Meta, among the more than 1,600 languages ​​tested, 78% have a recognition error rate (CER) lower than 10%. If you look at languages ​​trained with more than 10 hours of speech data, this ratio reaches 95%.

Even for low-resource languages ​​with extremely sparse training corpora, 36% still achieved a CER of less than 10%.


These numbers mean that Omnilingual ASR not only has broad coverage, but also delivers practical and high-quality transcription results in most languages.

However, 1,600 languages ​​is not the end of Omnilingual ASR.

The greater significance is that it breaks the limitation of the fixed and rigid language range supported by the previous ASR model, allowing language coverage to move from "quantitative" to "scalable".

Omnilingual ASR draws on the ideas of large language models (LLM) and introduces a zero-sample "context learning" mechanism.

This means that even if a language is not initially in the support list, users can instantly teach the model a new language during inference by providing a few audio clips and corresponding text in that language as examples.

There is no need to spend months collecting large corpora or professional deep learning training. You can learn new languages ​​with simple few-shot learning.

With this innovative paradigm, the potential language coverage of Omnilingual ASR is suddenly expanded.

Officials say that in theory, the system can be expanded to more than 5,400 languages, covering almost all human languages ​​with written records!

No matter how unpopular the spoken language is, as long as there is a corresponding writing system and a few examples, it has a chance to be captured and recorded by Omnilingual ASR.

In the field of AI speech recognition, this is a paradigm shift from static closure to dynamic adaptation - the model is no longer bound to the preset language list during training, but has become a flexible and open framework that encourages local communities to add new languages ​​on their own.

For those groups who have been absent from the technology map for a long time, this is tantamount to having a key to the door that can "unlock" new languages ​​at any time.

Open source and community, breaking the language gap

Another distinguishing feature of Omnilingual ASR is its open source and community-driven nature.

Meta chose to make this huge multilingual ASR system completely open source on GitHub, using the Apache 2.0 license to release the model and code.


Whether researchers, developers or corporate organizations, you can use, modify, and commercialize this model for free without worrying about cumbersome licensing restrictions.

Compared with previous "semi-open source" models with additional terms for some AI models, Omnilingual ASR's open attitude can be described as very magnanimous, setting an example for the democratization of technology.

In order to benefit all language communities, Meta not only opened up the model, but also simultaneously released a huge multilingual speech data set - the Omnilingual ASR corpus.

The corpus contains transcribed speech data for 350 languages ​​with scarce corpora, covering many languages ​​that were previously “lost” in the digital world.

All data are available under CC-BY license.

Developers and scholars can use these valuable resources to train and improve speech recognition models suitable for local needs.

This initiative will undoubtedly help those languages ​​​​that lack large-scale annotated corpora to cross the data threshold, giving "small languages" the opportunity to make big achievements.

Omnilingual ASR can cover an unprecedented breadth of languages ​​and is inseparable from the support of global cooperation.

During the development process, Meta worked with local language organizations and communities to collect a large number of speech samples.

They cooperate with the Mozilla Foundation's Common Voice project, Africa's Lanfrica/NaijaVoices and other organizations to recruit native speakers from remote areas to record voices.

To ensure that the data are diverse and relevant to life, these recordings often use open-ended questions, allowing the speakers to express their everyday thoughts freely.

All participants received fair compensation and followed culturally sensitive guidelines for collection.

This model of community co-creation gives Omnilingual ASR profound linguistic knowledge and cultural understanding, and also demonstrates the project's humanistic care: technological development does not and should not condescendingly "save" small languages, but rather cooperates with local communities to make themselves the protagonists of language digitization.

In terms of technical specifications, Meta provides a series of models of different sizes to adapt to diverse application scenarios: from lightweight models with about 300 million parameters (suitable for low-power devices such as mobile phones) to powerful models with up to 7 billion parameters (pursuing ultimate accuracy).

The model architecture uses the self-supervised pre-trained wav2vec 2.0 speech encoder (expanded to 7 billion parameter scale) to extract universal audio features, and combines two decoder strategies: one is traditional CTC decoding, and the other is a large model text decoder integrated with Transformer. The latter gives the model powerful context learning capabilities.


Huge models require massive amounts of data to support - Omnilingual ASR training uses more than 4.3 million hours of speech audio, covering materials in 1,239 languages.

This is one of the largest and most diverse speech training corpora ever created. Such a large amount of data coupled with the long-tail language corpus contributed by the community ensures that the model can learn robust speech representations for various languages, and has a good generalization basis even for languages ​​that it has never seen before.

As the research paper points out, "No model can cover all the world's languages ​​in advance, but Omnilingual ASR allows the community to continue to expand this list with its own data."

This marks that voice AI has the vitality of self-growth and can co-evolve with the rich diversity of human language.

When technology puts aside its arrogance and embraces diversity with an open source attitude, when the voices of every language have the opportunity to be heard and recorded, and when no language is forgotten in the digital world, we are one step closer to truly eliminating the language gap, and only then can human connections truly begin to eliminate boundaries.

References:

https://ai.meta.com/blog/omnilingual-asr-advancing-automatic-speech-recognition