AI that lip-reads 'better than humans'


AI that lip-reads 'better than humans'

Scientists at Oxford University have developed a machine that can lip-read better than humans.

The artificial intelligence system - LipNet - watches video of a person speaking and matches the text to the movement of their mouths with 93% accuracy, the researchers said.

Automating the process could help millions, they suggested.

But experts said the system needed to be tested in real-life situations.

Lip-reading is a notoriously tricky business with professionals only able to decipher what someone is saying up to 60% of the time.


"Machine lip-readers have enormous potential, with applications in improved hearing aids, silent dictation in public spaces, covert conversations, speech recognition in noisy environments, biometric identification and silent-movie processing," wrote the researchers.

They said that the AI system was provided with whole sentences so that it could teach itself which letter corresponded to which lip movement.

To train the AI, the team - from Oxford University's AI lab - fed it nearly 29,000 videos, labelled with the correct text. Each video was three seconds long and followed a similar grammatical pattern.

While human testers given similar videos had an error rate of 47.7%, the AI had one of just 6.6%.

The fact that the AI learned from specialist training videos led some on Twitter to criticise the research.

Writing in OpenReview, Neil Lawrence pointed out that the videos had "limited vocabulary and a single syntax grammar".

"While it's promising to perform well on this data, it's not really groundbreaking. While the model may be able to read my lips better than a human, it can only do so when I say a meaningless list of words from a highly constrained vocabulary in a specific order," he writes.

The project was partially funded by Google's artificial intelligence firm DeepMind.



Add Comment

all comments

  Other news

more
3 injured after car smashes into crowd outside north London pub (PHOTOS, VIDEOS)

3 injured after car smashes into crowd outside north London pub (PHOTOS, VIDEOS)..

26-Mar, 15:04

Four arrests have been made after a car ploughed into a group of people...

Banbury North signal box demolished despite bid to save it

Banbury North signal box demolished despite bid to save it..

26-Mar, 16:58

The company delayed the demolition of the box to allow railway fans to...

Marnie Simpson admits to signing new lease to continue living with cheating ex Lewis Bloor

Marnie Simpson admits to signing new lease to continue living with cheating ex Lewis Bloor..

26-Mar, 18:54

However the former couple continued to live together in their shared flat...

Benedict Cumberbatch is a dad again! See which Shakespeare character his new son is named after

Benedict Cumberbatch is a dad again! See which Shakespeare character his new son is named after..

26-Mar, 14:45

Actor Benedict Cumberbatch and his wife Sophie Hunter have become proud...

With Travel Ban on Hold, Iraqi Family Arrives in 2nd Attempt to Reach US

With Travel Ban on Hold, Iraqi Family Arrives in 2nd Attempt to Reach US..

06-Feb, 13:36

Iraqi citizen Fuad Sharef Suleman and his family landed in Nashville,...

Mattis Warns NKorea Against Any Attack on US or Its Allies

Mattis Warns NKorea Against Any Attack on US or Its Allies..

03-Feb, 01:42

In an explicit warning to North Korea, U.S. Defense Secretary Jim Mattis...

Paraguay's Central Bank to count 30 tons of Venezuelan bills

Paraguay's Central Bank to count 30 tons of Venezuelan bills..

14-Feb, 20:04

Central Bank employees will count about 30 tons of Venezuelan currency...

More Whales Strand in New Zealand, Bringing Total to 650

More Whales Strand in New Zealand, Bringing Total to 650..

11-Feb, 03:46

A new pod of 240 whales swam aground at a remote New Zealand beach on...