CLOSE
Original image

How Do Computers Understand Speech?

Original image

More and more, we can get computers to do things for us by talking to them. A computer can call your mother when you tell it to, find you a pizza place when you ask for one, or write out an email that you dictate. Sometimes the computer gets it wrong, but a lot of the time it gets it right, which is amazing when you think about what a computer has to do to turn human speech into written words: turn tiny changes in air pressure into language. Computer speech recognition is very complicated and has a long history of development, but here, condensed for you, are the 7 basic things a computer has to do to understand speech.

1. Turn the movement of air molecules into numbers.


Wikimedia Commons

Sound comes into your ear or a microphone as changes in air pressure, a continuous sound wave. The computer records a measurement of that wave at one point in time, stores it, and then measures it again. If it waits too long between measurements, it will miss important changes in the wave. To get a good approximation of a speech wave, it has to take a measurement at least 8000 times a second, but it works better if it takes one 44,100 times a second. This process is otherwise known as digitization at 8kHz or 44.1kHz.

2. Figure out which parts of the sound wave are speech.

When the computer takes measurements of air pressure changes, it doesn't know which ones are caused by speech, and which are caused by passing cars, rustling fabric, or the hum of hard drives. A variety of mathematical operations are performed on the digitized sound wave to filter out the stuff that doesn't look like what we expect from speech. We kind of know what to expect from speech, but not enough to make separating the noise out an easy task.

3. Pick out the parts of the sound wave that help tell speech sounds apart.


Wikimedia Commons

A sound wave from speech is actually a very complex mix of multiple waves coming at different frequencies. The particular frequencies—how they change, and how strongly those frequencies are coming through—matter a lot in telling the difference between, say, an "ah" sound and an "ee" sound. More mathematical operations transform the complex wave into a numerical representation of the important features.

4. Look at small chunks of the digitized sound one after the other and guess what speech sound each chunk shows.

There are about 40 speech sounds, or phonemes, in English. The computer has a general idea of what each of them should look like because it has been trained on a bunch of examples. But not only do the characteristics of these phonemes vary with different speaker accents, they change depending on the phonemes next to them—the 't' in "star" looks different than the 't' in "city." The computer must have a model of each phoneme in a bunch of different contexts for it to make a good guess.

5. Guess possible words that could be made up of those phonemes.

The computer has a big list of words that includes the different ways they can be pronounced. It makes guesses about what words are being spoken by splitting up the string of phonemes into strings of permissible words. If it sees the sequence "hang ten," it shouldn't split it into "hey, ngten!" because "ngten" won't find a good match in the dictionary.

6. Determine the most likely sequence of words based on how people actually talk.

There are no word breaks in the speech stream. The computer has to figure out where to put them by finding strings of phonemes that match valid words. There can be multiple guesses about what English words make up the speech stream, but not all of them will make good sequences of words. "What do cats like for breakfast?" could be just as good a guess as "water gaslight four brick vast?" if words are the only consideration. The computer applies models of how likely one word is to follow the next in order to determine which word string is the best guess. Some systems also take into account other information, like dependencies between words that are not next to each other. But the more information you want to use, the more processing power you need.

7. Take action

Once the computer has decided which guesses to go with, it can take action. In the case of dictation software, it will print the guess to the screen. In the case of a customer service phone line, it will try to match the guess to one of its pre-set menu items. In the case of Siri, it will make a call, look up something on the Internet, or try to come up with an answer to match the guess. As anyone who has used speech recognition software knows, mistakes happen. All the complicated statistics and mathematical transformations might not prevent "recognize speech" from coming out as "wreck a nice beach," but for a computer to pluck either one of those phrases out of the air is still pretty incredible.

Original image
iStock
arrow
Live Smarter
Trying to Save Money? Avoid Shopping on a Smartphone
Original image
iStock
Today, Americans do most of their shopping online—but as anyone who’s indulged in late-night retail therapy likely knows, this convenience often can come with an added cost. Trying to curb expenses, but don't want to swear off the convenience of ordering groceries in your PJs? New research shows that shopping on a desktop computer instead of a mobile phone may help you avoid making foolish purchases, according to Co. Design. Ying Zhu, a marketing professor at the University of British Columbia-Okanagan, recently led a study to measure how touchscreen technology affects consumer behavior. Published in the Journal of Retailing and Consumer Services, her research found that people are more likely to make more frivolous, impulsive purchases if they’re shopping on their phones than if they’re facing a computer monitor. Zhu, along with study co-author Jeffrey Meyer of Bowling Green State University, ran a series of lab experiments on student participants to observe how different electronic devices affected shoppers’ thinking styles and intentions. Their aim was to see if subjects' purchasing goals changed when it came to buying frivolous things, like chocolate or massages, or more practical things, like food or office supplies. In one experiment, participants were randomly assigned to use a desktop or a touchscreen. Then, they were presented with an offer to purchase either a frivolous item (a $50 restaurant certificate for $30) or a useful one (a $50 grocery certificate for $30). These subjects used a three-point scale to gauge how likely they were to purchase the offer, and they also evaluated how practical or frivolous each item was. (Participants rated the restaurant certificate to be more indulgent than the grocery certificate.) Sure enough, the researchers found that participants had "significantly higher" purchase intentions for hedonic (i.e. pleasurable) products when buying on touchscreens than on desktops, according to the study. On the flip side, participants had significantly higher purchase intentions for utilitarian (i.e. practical) products while using desktops instead of touchscreens. "The playful and fun nature of the touchscreen enhances consumers' favor of hedonic products; while the logical and functional nature of a desktop endorses the consumers' preference for utilitarian products," Zhu explains in a press release. The study also found that participants using touchscreen technology scored significantly higher on "experiential thinking" than subjects using desktop computers, whereas those with desktop computers demonstrated higher scores for rational thinking. “When you’re in an experiential thinking mode, [you crave] excitement, a different experience,” Zhu explained to Co. Design. “When you’re on the desktop, with all the work emails, that interface puts you into a rational thinking style. While you’re in a rational thinking style, when you assess a product, you’ll look for something with functionality and specific uses.” Zhu’s advice for consumers looking to conserve cash? Stow away the smartphone when you’re itching to splurge on a guilty pleasure. [h/t Fast Company]
arrow
Animals
Elusive Butterfly Sighted in Scotland for the First Time in 133 Years

Conditions weren’t looking too promising for the white-letter hairstreak, an elusive butterfly that’s native to the UK. Threatened by habitat loss, the butterfly's numbers have dwindled by 96 percent since the 1970s, and the insect hasn’t even been spotted in Scotland since 1884. So you can imagine the surprise lepidopterists felt when a white-letter hairstreak was seen feeding in a field in Berwickshire, Scotland earlier in August, according to The Guardian.

A man named Iain Cowe noticed the butterfly and managed to capture it on camera. “It is not every day that something as special as this is found when out and about on a regular butterfly foray,” Cowe said in a statement provided by the UK's Butterfly Conservation. “It was a very ragged and worn individual found feeding on ragwort in the grassy edge of an arable field.”

The white-letter hairstreak is a small brown butterfly with a white “W”-shaped streak on the underside of its wings and a small orange spot on its hindwings. It’s not easily sighted, as it tends to spend most of its life feeding and breeding in treetops.

The butterfly’s preferred habitat is the elm tree, but an outbreak of Dutch elm disease—first noted the 1970s—forced the white-letter hairstreak to find new homes and food sources as millions of Britain's elm trees died. The threatened species has slowly spread north, and experts are now hopeful that Scotland could be a good home for the insect. (Dutch elm disease does exist in Scotland, but the nation also has a good amount of disease-resistant Wych elms.)

If a breeding colony is confirmed, the white-letter hairstreak will bump Scotland’s number of butterfly species that live and breed in the country up to 34. “We don’t have many butterfly species in Scotland so one more is very nice to have,” Paul Kirkland, director of Butterfly Conservation Scotland, said in a statement.

Prior to 1884, the only confirmed sighting of a white-letter hairstreak in Scotland was in 1859. However, the insect’s newfound presence in Scotland comes at a cost: The UK’s butterflies are moving north due to climate change, and the white-letter hairstreak’s arrival is “almost certainly due to the warming climate,” Kirkland said.

[h/t The Guardian]

SECTIONS

arrow
LIVE SMARTER
More from mental floss studios