US4975960A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Electronic facial tracking and detection system and method and apparatus for automated speech recognition
Assignee: FACE2FACE ANIMATION Inc A CORP OF DELAWARE
•
Filing: 1985-06-03
•
Granted: 1990-12-04
•
Expires: 2005-06-03
Abstract
Abstract The apparatus includes circuitry for obtaining a video image of an individual's face, circuitry for electronically locating and tracking a first feature, such as the nostrils, of the facial image for use as reference coordinates and circuitry responsive to the reference coordinates for locating and tracking a second facial feature, such as the mouth, of the facial image with respect to the first feature. By tracking the location of the nostrils, the apparatus can follow the movement of the mouth, and thereby automatically recognize speech. In a preferred embodiment, the video image is grayscale encoded and the raster lines are smoothed to eliminate noise. The transitions between gray levels of the smoothed image are encoded and the resulting transition code is used to form a contour map of the image from which region parameters are computed which can be compared against stored speech templates to recognize speech. In the preferred embodiment, acoustic speech recognition is combined with visual speech recognition to improve accuracy.
Inventors
- 👤 Eric D. Petajan
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-27 14:52
CPC Classifications
G; G10; G10L; G10L15/00; G10L15/24; G07; G07C; G07C9/00; G07C9/30; G07C9/32; G07C9/37
Claims Analysis
Total Claims: 100
Claim 1
Independent Claim
141 words
An apparatus for producing output indicating words spoken by a human speaker, said apparatus comprising: means for detecting sounds, converting said sounds to electrical signals, analyzing said signals to detect for words, and then producing an electrical acoustic output signal representing at least one spoken word; means for scanning said speaker's face and producing electrical image signals, each said signal representing an image in a sequence of video images of said speaker; means, responsive to each said image signal, for tracking said speaker's mouth by tracking said speaker's nostrils; means for analyzing portions of said image signals, said portions defined by said means for tracking, to produce a video output signal representing at least one visual manifestation of at least one spoken word; means for receiving and correlating said acoustic output signal and said video output signal to produce said output.
Claim 2
Independent Claim
141 words
An apparatus for producing output indicating words spoken by a human speaker, said apparatus comprising: means for detecting sounds, converting said sounds to electrical signals, analyzing said signals to detect for words, and then producing an electrical acoustic output signal representing at least one spoken word; means for scanning said speaker's face and producing electrical image signals, each said signal representing an image in a sequence of video images of said speaker; means, responsive to each said image signal, for tracking said speaker's mouth by tracking said speaker's nostrils; means for analyzing portions of said image signals, said portions defined by said means for tracking, to produce a video output signal representing at least one visual manifestation of at least one spoken word; means for receiving and correlating said acoustic output signal and said video output signal to produce said output.
Claim 3
Dependent Claim
20 words
The apparatus of claim 1, wherein: said means for tracking is responsive to more than one of said image signals.
Claim 4
Dependent Claim
60 words
The apparatus of claim 2, wherein: said means for tracking, means for analyzing said portions, and said means for receiving and correlating are together comprised of: a video processor, at least one computer means operably connected to said video processor and said means for detecting sounds, and at least one memory means accessible by said at least one computer means.
Claim 5
Dependent Claim
53 words
The apparatus of claim 3, wherein: said video processor is comprised of components including a camera control unit operably connected to said means for scanning, a window control unit, a grayscale threshold circuit, a video memory, a smooth and transition code circuit, a video control unit, and a controller, said components operably connected.
Claim 6
Dependent Claim
61 words
The apparatus of claim 1, wherein: said means for tracking, said means for analyzing said portions, and said means for receiving and correlating are together comprised of: a video processor, at least one computer means operably connected to said video processor and said means for detecting sounds and at least one memory means assessable by said at least one computer means.
Claim 7
Dependent Claim
53 words
The apparatus of claim 5, wherein: said video processor is comprised of components including a camera control unit operably connected to said means for scanning, a window control unit, a grayscale threshold circuit, a video memory, a smooth and transition code circuit, a video control unit, and a controller, said components operably connected.
Claim 8
Independent Claim
193 words
An apparatus for producing output indicating words spoken by a human speaker, said apparatus comprising: an acoustic speech recognizer in operative combination with a microphone, for detecting sounds, converting said sounds to electrical signals, analyzing said electrical signals to detect for words, and then producing an electrical acoustic output signal representing an audio manifestation of spoken speech; means for scanning said speaker's face and producing electrical image signals, each said signal representing an image in a sequence of video images of said speaker; means for analyzing said image signals to produce respective location signals in response to a detection of said speaker's nostrils; means, activatable in response to one of said location signals, for defining a portion of one of said image signals for mouth analysis, said portion representing a region on said speaker's face located at a position determined by said speaker's nostrils; means for analyzing portions of said video image signals, responsive to said means for defining, to produce a video output signal, said video output signal representing a visual manifestation of speech; means for receiving and correlating said acoustic output signal and said video output signal to produce said output.
Claim 9
Independent Claim
193 words
An apparatus for producing output indicating words spoken by a human speaker, said apparatus comprising: an acoustic speech recognizer in operative combination with a microphone, for detecting sounds, converting said sounds to electrical signals, analyzing said electrical signals to detect for words, and then producing an electrical acoustic output signal representing an audio manifestation of spoken speech; means for scanning said speaker's face and producing electrical image signals, each said signal representing an image in a sequence of video images of said speaker; means for analyzing said image signals to produce respective location signals in response to a detection of said speaker's nostrils; means, activatable in response to one of said location signals, for defining a portion of one of said image signals for mouth analysis, said portion representing a region on said speaker's face located at a position determined by said speaker's nostrils; means for analyzing portions of said video image signals, responsive to said means for defining, to produce a video output signal, said video output signal representing a visual manifestation of speech; means for receiving and correlating said acoustic output signal and said video output signal to produce said output.
Claim 10
Dependent Claim
34 words
The apparatus of claim 7, wherein: said means for analyzing said image signals is responsive to said one of said image signals and is further responsive to a prior one of said image signals.
Claim 11
Dependent Claim
88 words
The apparatus of claim 8, wherein said means for analyzing respective portions of said video images is comprised of: memory means for storing a series of mouth information data sequences, said series of data sequence representing visual manifestations of spoken words; means for deriving mouth information data from said portions to form a second data sequence; and means for comparing each of said data sequences in said series with said second data sequence to produce said video output signals, wherein said means for comparing includes time warping means.
Claim 12
Dependent Claim
88 words
The apparatus of claim 7, wherein said means for analyzing respective portions of said video images is comprised of: memory means for storing a series of mouth information data sequences, said series of data sequences representing visual manifestations of spoken words; means for deriving mouth information data from said portions to form a second data sequence; and means for comparing each of said data sequences in said series with said second data sequence to produce said video output signals, wherein said means for comparing includes time warping means.
Claim 13
Independent Claim
64 words
An apparatus for electronically detecting a speaker's facial features, the apparatus comprising: means for producing electric signals corresponding to a sequence of video images of the speaker; and means, responsive to one of the signals corresponding to one image in the sequence, for finding the speaker's nostrils and then using the nostrils to define a region of the image for analyzing the speaker's mouth.
Claim 14
Independent Claim
64 words
An apparatus for electronically detecting a speaker's facial features, the apparatus comprising: means for producing electric signals corresponding to a sequence of video images of the speaker; and means, responsive to one of the signals corresponding to one image in the sequence, for finding the speaker's nostrils and then using the nostrils to define a region of the image for analyzing the speaker's mouth.
Claim 15
Dependent Claim
13 words
The apparatus of claim 11 wherein: the region does not include the nostrils.
Claim 16
Dependent Claim
109 words
The apparatus of claim 11, further comprising: means for comparing formations of the mouth in the regions in the sequence of images with data in a first computer memory to produce video speech recognition output; means for detecting sound produced by the speaker, operatively connected to the means for comparing, to associate the detected sound produced by the speaker with data in a second computer memory to produce acoustic speech recognition output; and means, responsive to the video speech recognition output and the acoustic speech recognition output, for combining the outputs to recognize speech, wherein the data in at least one of the computer memories comprises speech syntax information.
Claim 17
Dependent Claim
30 words
The apparatus of claim 11, further comprising: means for lipreading, responsive to a sequence of the regions and information in memory means, to produce output representing video speech recognition data.
Claim 18
Dependent Claim
28 words
The apparatus of claim 14, wherein the means for lipreading is comprised of: means for defining mouth parameters, responsive to the regions in the sequence of video images.
Claim 19
Dependent Claim
33 words
The apparatus of claim 14, wherein the means for lipreading is comprised of: means for defining region parameters relating to the speaker's mouth, responsive to the regions in the sequence of video images.
Claim 20
Dependent Claim
27 words
The apparatus of claim 14, wherein the means for lipreading is comprised of: means for contour coding, responsive to the regions in the sequence of video images.
Claim 21
Dependent Claim
29 words
The apparatus of claim 14, wherein the means for lipreading is comprised of: means for light intensity scale thresholding, responsive to the regions in the sequence of video images.
Claim 22
Dependent Claim
31 words
The apparatus of claim 14, wherein the means for lipreading is comprised of: means for raster smoothing, responsive to the regions in the sequence of video images, to reduce video noise.
Claim 23
Dependent Claim
27 words
The apparatus of claim 14, wherein the means for lipreading is comprised of: means for transition coding, responsive to the regions in the sequence of video images.
Claim 24
Dependent Claim
51 words
The apparatus of claim 14, wherein the means for lipreading is comprised of: means for deriving mouth information time sequences, responsive to the regions in the sequence of video images, the time sequences each having a distance adjusted to standardize the time sequences with the data in the first computer memory.
Claim 25
Dependent Claim
44 words
The apparatus of claim 14, further comprising: means for recognizing acoustic speech, to produce output representing acoustic speech recognition data; and means for combining said video speech recognition data with said acoustic speech recognition data to produce output indicating words spoken by said speaker.
Claim 26
Dependent Claim
28 words
The apparatus of claim 22, wherein the means for lipreading is comprised of: means for defining mouth parameters, responsive to the regions in the sequence of video images.
Claim 27
Dependent Claim
33 words
The apparatus of claim 22, wherein the means for lipreading is comprised of: means for defining region parameters relating to the speaker's mouth, responsive to the regions in the sequence of video images.
Claim 28
Dependent Claim
27 words
The apparatus of claim 22, wherein the means for lipreading is comprised of: means for contour coding, responsive to the regions in the sequence of video images.
Claim 29
Dependent Claim
29 words
The apparatus of claim 22, wherein the means for lipreading is comprised of: means for light intensity scale thresholding, responsive to the regions in the sequence of video images.
Claim 30
Dependent Claim
31 words
The apparatus of claim 22, wherein the means for lipreading is comprised of: means for raster smoothing, responsive to the regions in the sequence of video images, to reduce video noise.
Claim 31
Dependent Claim
27 words
The apparatus of claim 22, wherein the means for lipreading is comprised of: means for transition coding, responsive to the regions in the sequence of video images.
Claim 32
Dependent Claim
51 words
The apparatus of claim 22, wherein the means for lipreading is comprised of: means for deriving mouth information time sequences, responsive to the regions in the sequence of video images, the time sequences each having a distance adjusted to standardize the time sequences with the data in the first computer memory.
Claim 33
Dependent Claim
61 words
The apparatus of claim 11, wherein the means for finding and the using is comprised of: means, responsive to the one of the signals, for forming a data window in a subsequent one of the images in the sequence and limiting the means for finding and then using to searching for the nostrils within the data window of the subsequent image.
Claim 34
Dependent Claim
109 words
The apparatus of claim 30, further comprising: means for comparing formations of the mouth in the regions in the sequence of images with data in a first computer memory to produce video speech recognition output; means for detecting sound produced by the speaker, operatively connected to the means for comparing, to associate the detected sound produced by the speaker with data in a second computer memory to produce acoustic speech recognition output; and means, responsive to the video speech recognition output and the acoustic speech recognition output, for combining the outputs to recognize speech, wherein the data in at least one of the computer memories comprises speech syntax information.
Claim 35
Dependent Claim
30 words
The apparatus of claim 30, further comprising: means for lipreading, responsive to a sequence of the regions and information in memory means, to produce output representing video speech recognition data.
Claim 36
Dependent Claim
28 words
The apparatus of claim 32, wherein the means for lipreading is comprised of: means for defining mouth parameters, responsive to the regions in the sequence of video images.
Claim 37
Dependent Claim
33 words
The apparatus of claim 32, wherein the means for lipreading is comprised of: means for defining region parameters relating to the speaker's mouth, responsive to the regions in the sequence of video images.
Claim 38
Dependent Claim
27 words
The apparatus of claim 32, wherein the means for lipreading is comprised of: means for contour coding, responsive to the regions in the sequence of video images.
Claim 39
Dependent Claim
29 words
The apparatus of claim 32, wherein the means for lipreading is comprised of: means for light intensity scale thresholding, responsive to the regions in the sequence of video images.
Claim 40
Dependent Claim
31 words
The apparatus of claim 32, wherein the means for lipreading is comprised of: means for aster smoothing, responsive to the regions in the sequence of video images, to reduce video noise.
Claim 41
Dependent Claim
27 words
The apparatus of claim 32, wherein the means for lipreading is comprised of: means for transition coding, responsive to the regions in the sequence of video images.
Claim 42
Dependent Claim
51 words
The apparatus of claim 32, wherein the means for lipreading is comprised of: means for deriving mouth information time sequences, responsive to the regions in the sequence of video images, the time sequences each having a distance adjusted to standardize the time sequences with the data in the first computer memory.
Claim 43
Dependent Claim
44 words
The apparatus of claim 32, further comprising: means for recognizing acoustic speech, to produce output representing acoustic speech recognition data; and means for combining said video speech recognition data with said acoustic speech recognition data to produce output indicating words spoken by said speaker.
Claim 44
Dependent Claim
28 words
The apparatus of claim 40, wherein the means for lipreading is comprised of: means for defining mouth parameters, responsive to the regions in the sequence of video images.
Claim 45
Dependent Claim
33 words
The apparatus of claim 40, wherein the means for lipreading is comprised of: means for defining region parameters relating to the speaker's mouth, responsive to the regions in the sequence of video images.
Claim 46
Dependent Claim
27 words
The apparatus of claim 40, wherein the means for lipreading is comprised of: means for contour coding, responsive to the regions in the sequence of video images.
Claim 47
Dependent Claim
29 words
The apparatus of claim 40, wherein the means for lipreading is comprised of: means for light intensity scale thresholding, responsive to the regions in the sequence of video images.
Claim 48
Dependent Claim
31 words
The apparatus of claim 40, wherein the means for lipreading is comprised of: means for raster smoothing, responsive to the regions in the sequence of video images, to reduce video noise.
Claim 49
Dependent Claim
27 words
The apparatus of claim 40, wherein the means for lipreading is comprised of: means for transition coding, responsive to the regions in the sequence of video images.
Claim 50
Dependent Claim
51 words
The apparatus of claim 40, wherein the means for lipreading is comprised of: means for deriving mouth information time sequences, responsive to the regions in the sequence of video images, the time sequences each having a distance adjusted to standardize the time sequences with the data in the first computer memory.
Claim 51
Independent Claim
127 words
A method for producing output indicating words spoken by a human speaker, said method comprising steps of: detecting sounds, converting said sounds to electrical signals, analyzing said signals to detect for words, and then producing an electrical acoustic output signal representing at least one spoken word; scanning said speaker's face and producing electrical image signals, each said signal representing an image in a sequence of video images of said speaker; tracking said speaker's mouth by tracking said speaker's nostrils; analyzing portions of said image signals, said portions defined by said means for tracking, to produce a video output signal representing at least one visual manifestation of at least one spoken word; receiving and correlating said acoustic output signal and said video output signal to produce said output.
Claim 52
Independent Claim
127 words
A method for producing output indicating words spoken by a human speaker, said method comprising steps of: detecting sounds, converting said sounds to electrical signals, analyzing said signals to detect for words, and then producing an electrical acoustic output signal representing at least one spoken word; scanning said speaker's face and producing electrical image signals, each said signal representing an image in a sequence of video images of said speaker; tracking said speaker's mouth by tracking said speaker's nostrils; analyzing portions of said image signals, said portions defined by said means for tracking, to produce a video output signal representing at least one visual manifestation of at least one spoken word; receiving and correlating said acoustic output signal and said video output signal to produce said output.
Claim 53
Dependent Claim
18 words
The method of claim 48, wherein: said tracking is responsive to more than one of said image signals.
Claim 54
Dependent Claim
60 words
The method of claim 49, wherein: said steps of tracking, analyzing said portions, and receiving and correlating are together carried out with a video processor, at least one computer means operably connected to said video processor and said means for detecting sounds and at least one memory means accessible by said at least one computer means and said video processor.
Claim 55
Dependent Claim
68 words
The method of claim 50, wherein: said steps of tracking, analyzing said portions, and receiving and correlating are together carried out with a video processor comprised of components including a camera control unit operably connected to said means for scanning, a window control unit, a grayscale threshold circuit, a video memory, a smooth and transition code circuit, a video control unit, and a controller, said components operably connected.
Claim 56
Dependent Claim
67 words
The method of claim 49, wherein: said steps of tracking, analyzing said portions, and receiving and correlating are carried out with a video processor comprised of components including a camera control unit operably connected to said means for scanning, a window control unit, a grayscale threshold circuit, a video memory, a smooth and transition code circuit, a video control unit, and a controller, said components operably connected.
Claim 57
Dependent Claim
60 words
The method of claim 48, wherein: said steps of tracking, analyzing said portions, and receiving and correlating are together carried out with a video processor, at least one computer means operably connected to said video processor and said means for detecting sounds, and at least one memory means accessible by said at least one computer means and said video processor.
Claim 58
Independent Claim
186 words
A method for producing output indicating words spoken by a human speaker, said method comprising steps of: detecting sounds by means of an acoustic speech recognizer in operative combination with a microphone, converting said sounds to electrical signals, analyzing said electrical signals to detect for words, and then producing an electrical acoustic output signal representing an audio manifestation of spoken speech; scanning said speaker's face and producing electrical image signals, each said signal representing an image in a sequence of video images of said speaker; analyzing said image signals to produce respective location signals in response to a detection of said speaker's nostrils; defining a portion of one of said image signals for mouth analysis in response to one of said location signals, said portion representing a region on said speaker's face located at a position determined by said speaker's nostrils; analyzing portions of said video image signals, responsive to said step of defining, t produce a video output signal, said video output signal representing a visual manifestation of speech; receiving and correlating said acoustic output signal and said video output signal to produce said output.
Claim 59
Independent Claim
186 words
A method for producing output indicating words spoken by a human speaker, said method comprising steps of: detecting sounds by means of an acoustic speech recognizer in operative combination with a microphone, converting said sounds to electrical signals, analyzing said electrical signals to detect for words, and then producing an electrical acoustic output signal representing an audio manifestation of spoken speech; scanning said speaker's face and producing electrical image signals, each said signal representing an image in a sequence of video images of said speaker; analyzing said image signals to produce respective location signals in response to a detection of said speaker's nostrils; defining a portion of one of said image signals for mouth analysis in response to one of said location signals, said portion representing a region on said speaker's face located at a position determined by said speaker's nostrils; analyzing portions of said video image signals, responsive to said step of defining, t produce a video output signal, said video output signal representing a visual manifestation of speech; receiving and correlating said acoustic output signal and said video output signal to produce said output.
Claim 60
Dependent Claim
34 words
The method of claim 54, wherein: said step of analyzing said image signals is responsive to said one of said image signals and is further responsive to a prior one of said image signals.
Claim 61
Dependent Claim
101 words
The method of claim 55, wherein said step of analyzing respective portions of said video images is comprised of: storing a series of mouth information data sequences in memory means, said series of data sequences representing visual manifestations of spoken words; deriving mouth information data from said portions to form a derived data sequence; and comparing each of said data sequences in said series with said derived data sequence to produce said video output signals, wherein said step of comparing includes time warping to equalize the length of said derived data sequence with each of said data sequences in said series.
Claim 62
Dependent Claim
101 words
The method of claim 54, wherein said step of analyzing respective portions of said video images is comprised of: storing a series of mouth information data sequences in memory means, said series of data sequences representing visual manifestations of spoken words; deriving mouth information data from said portions to form a derived data sequence; and comparing each of said data sequences in said series with said derived data sequence to produce said video output signals, wherein said step of comparing includes time warping to equalize the length of said derived data sequence with each of said data sequences in said series.
Claim 63
Independent Claim
56 words
A method for electronically detecting a speaker's facial features, the method comprising steps of: producing electric signals corresponding to a sequence of video images of the speaker; and finding the speaker's nostrils in one image in the sequence and then using the nostrils to automatically define a region of the image for analyzing the speaker's mouth.
Claim 64
Independent Claim
56 words
A method for electronically detecting a speaker's facial features, the method comprising steps of: producing electric signals corresponding to a sequence of video images of the speaker; and finding the speaker's nostrils in one image in the sequence and then using the nostrils to automatically define a region of the image for analyzing the speaker's mouth.
Claim 65
Dependent Claim
21 words
The method of claim 58 wherein: the region, defined in the step of using the nostrils, does not include the nostrils.
Claim 66
Dependent Claim
98 words
The method of claim 58, further comprising steps of: comparing formations of the mouth in the regions in the sequence of images with data in a first computer memory to produce video speech recognition output; detecting sound produced by the speaker and associating the detected sound produced by the speaker with data in a second computer memory to produce acoustic speech recognition output; and combining the outputs to recognize speech, responsive to the video speech recognition output and the acoustic speech recognition output, wherein the data in at least one of the computer memories comprises speech syntax information.
Claim 67
Dependent Claim
31 words
The method of claim 58, further comprising a step of: lipreading, responsive to a sequence of the regions and information in memory means, to produce output representing video speech recognition data.
Claim 68
Dependent Claim
26 words
The method of claim 61, wherein the step of lipreading is comprised of: defining mouth parameters, responsive to the regions in the sequence of video images.
Claim 69
Dependent Claim
31 words
The method of claim 61, wherein the step of lipreading is comprised of: defining region parameters relating to the speaker's mouth, responsive to the regions in the sequence of video images.
Claim 70
Dependent Claim
25 words
The method of claim 61, wherein the step of lipreading is comprised of: contour coding, responsive to the regions in the sequence of video images.
Claim 71
Dependent Claim
27 words
The method of claim 61, wherein the step of lipreading is comprised of: light intensity scale thresholding, responsive to the regions in the sequence of video images.
Claim 72
Dependent Claim
29 words
The method of claim 61, wherein the step of lipreading is comprised of: raster smoothing, responsive to the regions in the sequence of video images, to reduce video noise.
Claim 73
Dependent Claim
25 words
The method of claim 61, wherein the step of lipreading is comprised of: transition coding, responsive to the regions in the sequence of video images.
Claim 74
Dependent Claim
49 words
The method of claim 61, wherein the step of lipreading is comprised of: deriving mouth information time sequences, responsive to the regions in the sequence of video images, the time sequences each having a distance adjusted to standardize the time sequences with the data in the first computer memory.
Claim 75
Dependent Claim
42 words
The method of claim 61, further comprising steps of: recognizing acoustic speech, to produce output representing acoustic speech recognition data; and combining said video speech recognition data with said acoustic speech recognition data to produce output indicating words spoken by said speaker.
Claim 76
Dependent Claim
26 words
The method of claim 69, wherein the step of lipreading is comprised of: defining mouth parameters, responsive to the regions in the sequence of video images.
Claim 77
Dependent Claim
31 words
The method of claim 69, wherein the step of lipreading is comprised of: defining region parameters relating to the speaker's mouth, responsive to the regions in the sequence of video images.
Claim 78
Dependent Claim
25 words
The method of claim 69, wherein the step of lipreading is comprised of: contour coding, responsive to the regions in the sequence of video images.
Claim 79
Dependent Claim
27 words
The method of claim 69, wherein the step of lipreading is comprised of: light intensity scale thresholding, responsive to the regions in the sequence of video images.
Claim 80
Dependent Claim
29 words
The method of claim 69, wherein the step of lipreading is comprised of: raster smoothing, responsive to the regions in the sequence of video images, to reduce video noise.
Claim 81
Dependent Claim
25 words
The method of claim 69, wherein the step of lipreading is comprised of: transition coding, responsive to the regions in the sequence of video images.
Claim 82
Dependent Claim
49 words
The method of claim 69, wherein the step of lipreading is comprised of: deriving mouth information time sequences, responsive to the regions in the sequence of video images, the time sequences each having a distance adjusted to standardize the time sequences with the data in the first computer memory.
Claim 83
Dependent Claim
59 words
The method of claim 58, wherein: the steps of finding and than using are together comprised of: forming, responsive to the one of the signals, a data window in a subsequent one of the images in the sequence and limiting the finding and then using steps to searching for the nostrils within the data window of the subsequent image.
Claim 84
Dependent Claim
94 words
The method of claim 77, further comprising steps of: comparing formations of the mouth in the regions in the sequence of images with data in a first computer memory to produce video speech recognition output; detecting sound produced by the speaker and associating the detected sound produced by the speaker with data in a second computer memory to produce acoustic speech recognition output; and combining the video speech recognition output and the acoustic speech recognition output to recognize speech with the data in at least one of the computer memories comprising speech syntax information.
Claim 85
Dependent Claim
31 words
The method of claim 77, further comprising a step of: lipreading, responsive to a sequence of the regions and information in memory means, to produce output representing video speech recognition data.
Claim 86
Dependent Claim
26 words
The method of claim 79, wherein the step of lipreading is comprised of: defining mouth parameters, responsive to the regions in the sequence of video images.
Claim 87
Dependent Claim
31 words
The method of claim 79, wherein the step of lipreading is comprised of: defining region parameters relating to the speaker's mouth, responsive to the regions in the sequence of video images.
Claim 88
Dependent Claim
25 words
The method of claim 71, wherein the step of lipreading is comprised of: contour coding, responsive to the regions in the sequence of video images.
Claim 89
Dependent Claim
27 words
The method of claim 79, wherein the step of lipreading is comprised of: light intensity scale thresholding, responsive to the regions in the sequence of video images.
Claim 90
Dependent Claim
29 words
The method of claim 79, wherein the step of lipreading is comprised of: raster smoothing, responsive to the regions in the sequence of video images, to reduce video noise.
Claim 91
Dependent Claim
25 words
The method of claim 79, wherein the step of lipreading is comprised of: transition coding, responsive to the regions in the sequence of video images.
Claim 92
Dependent Claim
49 words
The method of claim 79, wherein the step of lipreading is comprised of: deriving mouth information time sequences, responsive to the regions in the sequence of video images, the time sequences each having a distance adjusted to standardize the time sequences with the data in the first computer memory.
Claim 93
Dependent Claim
42 words
The method of claim 79, further comprising steps of: recognizing acoustic speech, to produce output representing acoustic speech recognition data; and combining said video speech recognition data with said acoustic speech recognition data to produce output indicating words spoken by said speaker.
Claim 94
Dependent Claim
26 words
The method of claim 87, wherein the step of lipreading is comprised of: defining mouth parameters, responsive to the regions in the sequence of video images.
Claim 95
Dependent Claim
31 words
The method of claim 87, wherein the step of lipreading is comprised of: defining region parameters relating to the speaker's mouth, responsive to the regions in the sequence of video images.
Claim 96
Dependent Claim
25 words
The method of claim 87, wherein the step of lipreading is comprised of: contour coding, responsive to the regions in the sequence of video images.
Claim 97
Dependent Claim
27 words
The method of claim 87, wherein the step of lipreading is comprised of: light intensity scale thresholding, responsive to the regions in the sequence of video images.
Claim 98
Dependent Claim
29 words
The method of claim 87, wherein the step of lipreading is comprised of: raster smoothing, responsive to the regions in the sequence of video images, to reduce video noise.
Claim 99
Dependent Claim
25 words
The method of claim 87, wherein the step of lipreading is comprised of: transition coding, responsive to the regions in the sequence of video images.
Claim 100
Dependent Claim
49 words
The method of claim 87, wherein the step of lipreading is comprised of: deriving mouth information time sequences, responsive to the regions in the sequence of video images, the time sequences each having a distance adjusted to standardize the time sequences with the data in the first computer memory.
References Cited (Backward Citations)
US3192321A
Cited
US3383466A
Cited
US3829614A
Cited
US3903357A
Cited
US3953669A
Cited
US4405940A
Cited
US4286115A
Cited
US4449189A
Cited
US4566126A
Cited
US4546383A
Cited
US4757541A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1993-12-03 | 1994-06-04 | 1994-12-04 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 1997-12-03 | 1998-06-04 | 1998-12-04 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2001-12-03 | 2002-06-04 | 2002-12-04 | $7700.00 | Window Closed / Lapsed |