CLOSE
George Marks/Retrofile/Getty Images
George Marks/Retrofile/Getty Images

Artificial Intelligence Can Read Lips More Accurately Than People

George Marks/Retrofile/Getty Images
George Marks/Retrofile/Getty Images

A new artificial intelligence system can now lip read better than humans, according to New Scientist. Though films and pop culture usually show lip reading as some incredible tool that allows you to decode what anyone says, in practice, it’s fairly spotty: Even for experienced lip readers, one estimate puts the amount of speech you can interpret from someone’s lip movements at a mere 30 percent.

But artificial intelligence researchers from Google’s DeepMind and the University of Oxford’s engineering department have been working on a network that transcribes natural sentences just from visuals of people talking with no audio. It can also transcribe audio with no video. Their pre-publication paper is posted on arXIV [PDF].

The system recognizes syllables and short phrases, and has learned on a far-reaching database called “Lip Reading Sentences,” drawn from a half-dozen BBC programs and containing more than 100,000 sentences and 17,500 words. It works independently with both audio and video, helping it decode speech even if the audio stream is noisy or if the audio and video aren’t perfectly aligned.

This model was significantly more accurate than professional lip readers in a comparative test. The experimenters commissioned professional lip readers from a company that provides transcription services, each with around 10 years of experience lip reading in situations as diverse as videos for court use and national events like the British royal wedding. These lip readers could correctly decipher just 12 percent of the words they saw, while the computer model could decipher almost half of the words accurately. Aside from providing more accurate transcription services, “it is possible that research of this type could discern important discriminative cues that are beneficial for teaching lip reading to the hearing impaired,” the researchers write.

Try out your lip-reading skills with the video below:

[h/t New Scientist]

nextArticle.image_alt|e
WWF
arrow
Animals
Watch an Antarctic Minke Whale Feed in a First-of-Its-Kind Video
WWF
WWF

New research from the World Wildlife Fund is giving us a rare glimpse into the world of the mysterious minke whale. The WWF worked with Australian Antarctic researchers to tag minke whales with cameras for the first time, watching where and how the animals feed.

The camera attaches to the whale's body with suction cups. In the case of the video below, the camera accidentally slid down the side of the minke whale's body, providing an unexpected look at the way its throat moves as it feeds.

Minke whales are one of the smallest baleen whales, but they're still pretty substantial animals, growing 30 to 35 feet long and weighing up to 20,000 pounds. Unlike other baleen whales, though, they're small enough to maneuver in tight spaces like within sea ice, a helpful adaptation for living in Antarctic waters. They feed by lunging through the sea, gulping huge amounts of water along with krill and small fish, and then filtering the mix through their baleen.

The WWF video shows just how quickly the minke can process this treat-laden water. The whale could lunge, process, and lunge again every 10 seconds. "He was like a Pac-Man continuously feeding," Ari Friedlaender, the lead scientist on the project, described in a press statement.

The video research, conducted under the International Whaling Commission's Southern Ocean Research Partnership, is part of WWF's efforts to protect critical feeding areas for whales in the region.

If that's not enough whale for you, you can also watch the full 13-minute research video below:

nextArticle.image_alt|e
iStock
arrow
technology
AI Could Help Scientists Detect Earthquakes More Effectively
iStock
iStock

Thanks in part to the rise of hydraulic fracturing, or fracking, earthquakes are becoming more frequent in the U.S. Even though it doesn't fall on a fault line, Oklahoma, where gas and oil drilling activity doubled between 2010 and 2013, is now a major earthquake hot spot. As our landscape shifts (literally), our earthquake-detecting technology must evolve to keep up with it. Now, a team of researchers is changing the game with a new system that uses AI to identify seismic activity, Futurism reports.

The team, led by deep learning researcher Thibaut Perol, published the study detailing their new neural network in the journal Science Advances. Dubbed ConvNetQuake, it uses an algorithm to analyze the measurements of ground movements, a.k.a. seismograms, and determines which are small earthquakes and which are just noise. Seismic noise describes the vibrations that are almost constantly running through the ground, either due to wind, traffic, or other activity at surface level. It's sometimes hard to tell the difference between noise and legitimate quakes, which is why most detection methods focus on medium and large earthquakes instead of smaller ones.

But better understanding natural and manmade earthquakes means studying them at every level. With ConvNetQuake, that could soon become a reality. After testing the system in Oklahoma, the team reports it detected 17 times more earthquakes than what was recorded by the Oklahoma Geological Survey earthquake catalog.

That level of performance is more than just good news for seismologists studying quakes caused by humans. The technology could be built into current earthquake detection methods set up to alert the public to dangerous disasters. California alone is home to 400 seismic stations waiting for "The Big One." On a smaller scale, there's an app that uses a smartphone's accelerometers to detect tremors and alert the user directly. If earthquake detection methods could sense big earthquakes right as they were beginning using AI, that could afford people more potentially life-saving moments to prepare.

[h/t Futurism]

SECTIONS

arrow
LIVE SMARTER
More from mental floss studios