AI that lip-reads 'better than humans'

AI that lip-reads 'better than humans'

Scientists at Oxford University have developed a machine that can lip-read better than humans.

The artificial intelligence system - LipNet - watches video of a person speaking and matches the text to the movement of their mouths with 93% accuracy, the researchers said.

Automating the process could help millions, they suggested.

But experts said the system needed to be tested in real-life situations.

Lip-reading is a notoriously tricky business with professionals only able to decipher what someone is saying up to 60% of the time.


"Machine lip-readers have enormous potential, with applications in improved hearing aids, silent dictation in public spaces, covert conversations, speech recognition in noisy environments, biometric identification and silent-movie processing," wrote the researchers.

They said that the AI system was provided with whole sentences so that it could teach itself which letter corresponded to which lip movement.

To train the AI, the team - from Oxford University's AI lab - fed it nearly 29,000 videos, labelled with the correct text. Each video was three seconds long and followed a similar grammatical pattern.

While human testers given similar videos had an error rate of 47.7%, the AI had one of just 6.6%.

The fact that the AI learned from specialist training videos led some on Twitter to criticise the research.

Writing in OpenReview, Neil Lawrence pointed out that the videos had "limited vocabulary and a single syntax grammar".

"While it's promising to perform well on this data, it's not really groundbreaking. While the model may be able to read my lips better than a human, it can only do so when I say a meaningless list of words from a highly constrained vocabulary in a specific order," he writes.

The project was partially funded by Google's artificial intelligence firm DeepMind.



Add Comment

all comments

  Other news

more
California judge rules against Trump’s ban on sanctuary cities

California judge rules against Trump’s ban on sanctuary cities..

21-Jul, 19:12

A federal judge in San Francisco, California has refused to reinstate...

China holds Canadians 'for smoking marijuana'

China holds Canadians 'for smoking marijuana'..

21-Jul, 16:52

Chinese authorities have arrested two Canadians in the capital Beijing...

Rise of Comic-Con

Rise of Comic-Con..

21-Jul, 21:00

San Diego's Comic-Con International, happening this weekend, is an annual...

Britain's Got Talent winner Ashleigh Butler 'REPLACES' Pudsey with an almost identical dog bearing the same name just hours after paying tribute to dead pooch

Britain's Got Talent winner Ashleigh Butler 'REPLACES' Pudsey with an almost identical dog..

21-Jul, 22:11

Earlier today former Britain's Got Talent winner Ashleigh Butler...

Girl Scout threatened after confronting a neo-Nazi group, police get involved

Girl Scout threatened after confronting a neo-Nazi group, police get involved..

07-May, 14:32

The Czech Interior Ministry says that police are taking steps to ensure...

Trump lambastes Snoop Dogg over music video: 'Jail time!'

Trump lambastes Snoop Dogg over music video: 'Jail time!'..

15-Mar, 14:52

Days after Snoop Dogg's new music video "Lavender" was released,...

Trump sons' expansion of mid-market hotel chain ‘has nothing to do with politics’

Trump sons' expansion of mid-market hotel chain ‘has nothing to do with politics’..

06-Jun, 07:54

The Trump Organization has announced plans to expand its signature hotel...

Who We Know Will Be at the Oscars 2017

Who We Know Will Be at the Oscars 2017..

08-Feb, 07:20

Halle Berry, Kate McKinnon and Samuel L. Jackson will be presenting at...