US4769845A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method of recognizing speech using a lip image
Assignee: CARRYLAB KK
•
Filing: 1987-04-06
•
Granted: 1988-09-06
•
Expires: 2007-04-06
Abstract
Abstract A speech recognition method is implemented with an image pickup apparatus such as a TV camera which picks up a lip image during speech and with a small computer which has a small memory capacity and is connected to the TV camera. The computer receives and processes as lip data an image signal from the TV camera which represents the lip image. The lip data is collated with language data stored in the memory of the computer so as to select the language corresponding to the lip data, thereby recognizing the speech. A microphone may also be provided to output to the system a voice waveform signal serving as voice data. This voice data is collated with the language data stored in the memory of the computer to select the language corresponding to the voice data, thereby recognizing the speech on the basis of the language selected using the lip data and using the voice data. Image pattern data and voice pattern data may be extracted and processed for every word, or for every unit sound. With the inventive method, the speech recognition ratio and processing speed are improved, particularly with respect to use of a computer with a small memory capacity.
Inventors
- 👤 Hiroyuki Nakamura
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-28 17:54
CPC Classifications
G; G10; G10L; G10L15/00; G10L15/24; G10L15/25; G06; G06V; G06V40/00; G06V40/10; G06V40/16; G06V40/168
Claims Analysis
Total Claims: 13
Claim 1
Independent Claim
97 words
A method of recognizing speech by inputting a lip image, comprising the steps of: using image pickup means for picking up the lip image from the lips during speech; receiving and processing, using data processing means connected to said image pickup means, lip data from the image pickup means in the form of an image signal indicative of the lip image; collating said lip data with language data previously stored in a first memory provided in said data processing means; selecting from said language data the language corresponding to said lip data; and thereby recognizing the speech.
Claim 2
Independent Claim
97 words
A method of recognizing speech by inputting a lip image, comprising the steps of: using image pickup means for picking up the lip image from the lips during speech; receiving and processing, using data processing means connected to said image pickup means, lip data from the image pickup means in the form of an image signal indicative of the lip image; collating said lip data with language data previously stored in a first memory provided in said data processing means; selecting from said language data the language corresponding to said lip data; and thereby recognizing the speech.
Claim 3
Dependent Claim
51 words
A speech recognizing method according to claim 1, including the steps of inputting said lip image to said data processing means as the lip data for every word, and collating the lip data for every word with said stored language data to select the corresponding language, thereby recognizing the input speech.
Claim 4
Dependent Claim
89 words
A speech recognizing method according to claim 1, including the steps of causing said data processing means to receive the lip image as the lip data for every unit sound; collating the lip data for every unit sound with unit sound data previously stored into a second memory provided in said data processing means; identifying each unit sound corresponding to said lip data; collating a combination of the identified unit sounds with said stored language data; selecting the language corresponding to said combination; and thereby recognizing the input speech.
Claim 5
Dependent Claim
33 words
A speech recognizing method according to claim 1, wherein said image pickup means is a television camera which picks up the lip image and outputs the image data to said data processing means.
Claim 6
Dependent Claim
41 words
A speech recognizing method according to claim 1, further comprising the step of providing microphone means for picking up the sound of the speech and outputting a waveform signal indicative of the waveform of said speech to said data processing means.
Claim 7
Dependent Claim
80 words
A speech recognizing method according to claim 5, further comprising the steps of: receiving said waveform signal in said data processing means as voice data; collating said voice data with language data previously stored in a second memory provided in the data processing means; selecting from said language data the language corresponding to said voice data; and thereby recognizing the speech on the basis of the language selected using said lip data and the language selected using said voice data.
Claim 8
Dependent Claim
138 words
A speech recognizing method according to claim 5, including the steps of causing said data processing means to receive said lip image as the lip data for every word; collating said lip data for every word with said stored language data; selecting from said language data the language corresponding to said lip data; receiving said voice waveform as voice data for every unit sound; collating said voice data for every unit sound with said stored unit sound data; identifying each unit sound corresponding to said voice data; collating a combination of said identified unit sounds with the stored language data; selecting from said language data the language corresponding to said combination of unit sounds; and thereby recognizing the speech on the basis of the language selected using said lip data and the language selected using said voice data.
Claim 9
Dependent Claim
181 words
A voice recognizing method according to claim 5, including the steps of causing said data processing means to receive said lip image as the lip data for every unit sound; collating said lip data for every unit sound with unit sound data previously stored in a second memory provided in said data processing means; identifying each unit sound corresponding to said lip data; collating a first combination of said identified unit sounds corresponding to said lip data with said language data; selecting from said language data the language corresponding to said first combination; receiving said voice waveform as voice data for every unit sound; collating said voice data for every unit sound with said stored unit sound data; identifying each unit sound corresponding to said voice data; collating a second combination of said identified unit sounds corresponding to said voice data with said language data; selecting from said language data the language corresponding to said second combination; and thereby recognizing the speech on the basis of the language selected using said lip data and the language selected using said voice data.
Claim 10
Independent Claim
181 words
A method of speech recognition, comprising the steps of: producing a time-varying image of the lips of a person making a verbal statement, said verbal statement being made up of at least one unit of language; converting said lip image into a signal; extracting from said signal a plurality of characteristic parameters of said lip image which vary with time; accessing reference language data previously stored in a memory, said reference language data including a predetermined plurality of units of language and including for each such unit of language a set of characteristic parameters corresponding thereto; comparing said extracted characteristic parameters representative of said lip image to said stored sets of characteristic parameters so as to successively identify stored sets of characteristic parameters substantially equivalent to successive portions of said extracted characteristic parameters; retrieving the stored unit of language associated with each said identified set of characteristic parameters; and arranging said retrieved units of language in the chronological order of occurrence of said successive portions of said extracted characteristic parameters to thereby produce an accurate representation of the entire verbal statement.
Claim 11
Independent Claim
181 words
A method of speech recognition, comprising the steps of: producing a time-varying image of the lips of a person making a verbal statement, said verbal statement being made up of at least one unit of language; converting said lip image into a signal; extracting from said signal a plurality of characteristic parameters of said lip image which vary with time; accessing reference language data previously stored in a memory, said reference language data including a predetermined plurality of units of language and including for each such unit of language a set of characteristic parameters corresponding thereto; comparing said extracted characteristic parameters representative of said lip image to said stored sets of characteristic parameters so as to successively identify stored sets of characteristic parameters substantially equivalent to successive portions of said extracted characteristic parameters; retrieving the stored unit of language associated with each said identified set of characteristic parameters; and arranging said retrieved units of language in the chronological order of occurrence of said successive portions of said extracted characteristic parameters to thereby produce an accurate representation of the entire verbal statement.
Claim 12
Dependent Claim
14 words
A method according to claim 9, wherein each said language unit is one word.
Claim 13
Dependent Claim
45 words
A method according to claim 9, wherein said extracted characteristic parameters include the area in said lip image of the opening between the lips and the ratio of the vertical dimension to the horizontal dimension of said opening between said lips in said lip image.
References Cited (Backward Citations)
US3383466A
Cited
US4569026A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1991-09-06 | 1992-03-07 | 1992-09-06 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 1995-09-06 | 1996-03-07 | 1996-09-06 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 1999-09-06 | 2000-03-07 | 2000-09-06 | $7700.00 | Window Closed / Lapsed |