AI that lip-reads 'better than humans'

AI that lip-reads 'better than humans'

Scientists at Oxford University have developed a machine that can lip-read better than humans.

The artificial intelligence system - LipNet - watches video of a person speaking and matches the text to the movement of their mouths with 93% accuracy, the researchers said.

Automating the process could help millions, they suggested.

But experts said the system needed to be tested in real-life situations.

Lip-reading is a notoriously tricky business with professionals only able to decipher what someone is saying up to 60% of the time.


"Machine lip-readers have enormous potential, with applications in improved hearing aids, silent dictation in public spaces, covert conversations, speech recognition in noisy environments, biometric identification and silent-movie processing," wrote the researchers.

They said that the AI system was provided with whole sentences so that it could teach itself which letter corresponded to which lip movement.

To train the AI, the team - from Oxford University's AI lab - fed it nearly 29,000 videos, labelled with the correct text. Each video was three seconds long and followed a similar grammatical pattern.

While human testers given similar videos had an error rate of 47.7%, the AI had one of just 6.6%.

The fact that the AI learned from specialist training videos led some on Twitter to criticise the research.

Writing in OpenReview, Neil Lawrence pointed out that the videos had "limited vocabulary and a single syntax grammar".

"While it's promising to perform well on this data, it's not really groundbreaking. While the model may be able to read my lips better than a human, it can only do so when I say a meaningless list of words from a highly constrained vocabulary in a specific order," he writes.

The project was partially funded by Google's artificial intelligence firm DeepMind.



Add Comment

all comments

  Other news

more
Cheryl makes ?500k donation to the Prince’s Trust to open a centre for disadvantaged children in Newcastle – as boyfriend Liam Payne continues to gush about his girlfriend and baby Bear

Cheryl makes ?500k donation to the Prince’s Trust to open a centre for disadvantaged children..

28-May, 18:03

The money, from Cheryl’s Trust, will enable a modern centre in the Girls...

Bradley Lowery: Ladbible sorry for false death report

Bradley Lowery: Ladbible sorry for false death report..

29-May, 06:48

The family of terminally-ill Sunderland fan Bradley Lowery has accepted...

Israel’s economy heading for disaster, experts warn

Israel’s economy heading for disaster, experts warn..

29-May, 07:08

A report released by the Taub Center for Social Policy Studies in Israel...

How to hang onto your email if you change your name

How to hang onto your email if you change your name..

28-May, 12:16

Here's how to change your email name but not your address. A. Not...

The Latest: Pakistani raids kill 39 suspects after IS attack

The Latest: Pakistani raids kill 39 suspects after IS attack..

17-Feb, 02:20

The Latest on the aftermath of the massive attack by the Islamic State...

As Democrats call for recusal, House intel chair says 'nobody was sneaking around' on WH visit

As Democrats call for recusal, House intel chair says 'nobody was sneaking around' on WH visit..

27-Mar, 20:06

House Intelligence Committee Chairman Devin Nunes defended his visit to...

Boston City Hall Lit Red, White Blue in Anti-Trump Move

Boston City Hall Lit Red, White Blue in Anti-Trump Move..

26-Jan, 02:03

Boston mayor Marty Walsh lashed out at Donald Trump Wednesday afternoon...

Ex-sheriff with Alzheimer's faces trial on corruption counts

Ex-sheriff with Alzheimer's faces trial on corruption counts..

24-Feb, 21:22

Former Los Angeles County Sheriff Lee Baca went on trial on federal...