US5608839A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Sound-synchronized video system
Assignee: Google LLC
•
Filing: 1994-06-21
•
Granted: 1997-03-04
•
Expires: 2014-06-21
Abstract
Abstract A stream of unsynchronized audio signal, representing speech and video signal of a speaker, is processed by decoding the signal, memorizing a plurality of visemes corresponding to phonemes in the audio signal, fetching from the plurality visemes corresponding to phonemes in the audio signal, and imparting a synchronism to the video signal and audio signal by applying the fetched visemes to the unsynchronized video signal of the stream in synchronism with corresponding phonemes in the audio signal of the stream. According to an embodiment, the fetching step includes fetching visemes of the lip movement. The system is suitable for use in a videophone.
Inventors
- 👤 Homer H. Chen
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-08 10:10
CPC Classifications
H; H04; H04N; H04N21/00; H04N21/40; H04N21/43; H04N21/434; H04N21/4341; H04N21/4302; H04N21/4307; H04N21/43072; H04N21/439; H04N21/4394; H04N21/44; H04N21/44008; G; G10; G10L; G10L21/00; G10L21/06; G10L21/10; G10L2021/105; G11; G11B; G11B2220/00; G11B2220/90; H04N19/00; H04N19/20
Claims Analysis
Total Claims: 18
Claim 1
Independent Claim
83 words
A method of processing a stream of unsynchronized audio and video signals of a speaker, comprising the steps of: decoding the audio and video signals; memorizing a plurality of visemes corresponding to phonemes in the decoded audio signal; fetching visemes corresponding to phonemes in the audio signal; and imparting a synchronism to the decoded video and audio signals by applying the fetched visemes to the unsynchronized video signal of the stream in synchronism with corresponding phonemes in the audio signal of the stream.
Claim 2
Independent Claim
83 words
A method of processing a stream of unsynchronized audio and video signals of a speaker, comprising the steps of: decoding the audio and video signals; memorizing a plurality of visemes corresponding to phonemes in the decoded audio signal; fetching visemes corresponding to phonemes in the audio signal; and imparting a synchronism to the decoded video and audio signals by applying the fetched visemes to the unsynchronized video signal of the stream in synchronism with corresponding phonemes in the audio signal of the stream.
Claim 3
Dependent Claim
58 words
A method as in claim 1, wherein the memorizing step includes continuously updating the plurality of visemes corresponding to phonemes by continuously extracting visemes from the decoded video signal during the decoding of the audio and video signals and updating the memory of the visemes with the extracted visemes during the decoding of the audio and video signals.
Claim 4
Dependent Claim
17 words
A method as in claim 1, wherein the fetching step includes fetching visemes of the lip movement.
Claim 5
Dependent Claim
102 words
A communication method, comprising the steps of: transmitting and receiving streams of synchronized video and audio signals at a multiplicity of stations; at each of said stations, decoding said signals; at a plurality of said multiplicity of stations, the steps of: memorizing a plurality of visemes corresponding to phonemes in the decoded audio signal; fetching from the plurality visemes corresponding to phonemes in the audio signal; and imparting a synchronism to the video signal and audio signal by applying the fetched visemes to the unsynchronized video signal of the stream in synchronism with corresponding phonemes in the audio signal of the stream.
Claim 6
Dependent Claim
102 words
A communication method, comprising the steps of: transmitting and receiving streams of synchronized video and audio signals at a multiplicity of stations; at each of said stations, decoding said signals; at a plurality of said multiplicity of stations, the steps of: memorizing a plurality of visemes corresponding to phonemes in the decoded audio signal; fetching from the plurality visemes corresponding to phonemes in the audio signal; and imparting a synchronism to the video signal and audio signal by applying the fetched visemes to the unsynchronized video signal of the stream in synchronism with corresponding phonemes in the audio signal of the stream.
Claim 7
Dependent Claim
58 words
A method as in claim 4, wherein the memorizing step includes continuously updating the plurality of visemes corresponding to phonemes by continuously extracting visemes from the decoded video signal during the decoding of the audio and video signals and updating the memory of the visemes with the extracted visemes during the decoding of the audio and video signals.
Claim 8
Dependent Claim
16 words
A method as in claim 4, wherein the fetching step includes fetching visemes of lip movement.
Claim 9
Independent Claim
98 words
An apparatus for processing a stream of audio and video signals of a speaker, comprising: a signal decoder responsive to the audio and video signals; a visime memory responsive to a plurality of visemes corresponding to phonemes in the audio signal from the signal decoder; a video signal and audio signal synchronizer coupled to said visime memory and responsive to the plurality visemes corresponding to phonemes in the audio signal and to the unsynchronized video signal of the stream so as to impart synchronism of the video signal with corresponding phonemes in the audio signal of the stream.
Claim 10
Independent Claim
98 words
An apparatus for processing a stream of audio and video signals of a speaker, comprising: a signal decoder responsive to the audio and video signals; a visime memory responsive to a plurality of visemes corresponding to phonemes in the audio signal from the signal decoder; a video signal and audio signal synchronizer coupled to said visime memory and responsive to the plurality visemes corresponding to phonemes in the audio signal and to the unsynchronized video signal of the stream so as to impart synchronism of the video signal with corresponding phonemes in the audio signal of the stream.
Claim 11
Dependent Claim
34 words
An apparatus as in claim 7, wherein the signal decoder includes an audio output for outputting the audio signal from the stream and a video output for outputting the video signal from the stream.
Claim 12
Dependent Claim
15 words
An apparatus as in claim 7, wherein the synchronizer responds to visemes of lip movement.
Claim 13
Dependent Claim
22 words
An apparatus as in claim 7, wherein said visime memory is responsive continuously to new visemes corresponding to said phonemes during decoding.
Claim 14
Dependent Claim
114 words
A communication system, comprising: a plurality of stations each having means for transmitting and receiving video and audio signals; a communications network linking said stations; a signal decoder at each of said stations; at a plurality of said stations: a visime memory coupled to said decoding means and responsive to a plurality of visemes corresponding to phonemes in the audio signal; a video signal and audio signal synchronizer coupled to said visime memory and responsive to the plurality visemes corresponding to phonemes in the audio signal and to the unsynchronized video signal of the stream so as to impart synchronism of the video signal with corresponding phonemes in the audio signal of the stream.
Claim 15
Dependent Claim
114 words
A communication system, comprising: a plurality of stations each having means for transmitting and receiving video and audio signals; a communications network linking said stations; a signal decoder at each of said stations; at a plurality of said stations: a visime memory coupled to said decoding means and responsive to a plurality of visemes corresponding to phonemes in the audio signal; a video signal and audio signal synchronizer coupled to said visime memory and responsive to the plurality visemes corresponding to phonemes in the audio signal and to the unsynchronized video signal of the stream so as to impart synchronism of the video signal with corresponding phonemes in the audio signal of the stream.
Claim 16
Dependent Claim
34 words
An apparatus as in claim 11, wherein said signal decoder includes an audio output for outputting the audio signal from the stream and a video output for outputting the video signal from the stream.
Claim 17
Dependent Claim
15 words
An apparatus as in claim 11, wherein the synchronizer responds to visemes of lip movement.
Claim 18
Dependent Claim
22 words
An apparatus as in claim 10, wherein said visime memory is responsive continuously to new visemes corresponding to said phonemes during decording.
References Cited (Backward Citations)
US4104625A
Cited
US4305131A
Cited
US4757541A
Cited
US4841575A
Cited
JPH01162492A
Cited
GB2231246A
Cited
GB2250405A
Cited
US5327521A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2000-03-03 | 2000-09-02 | 2001-03-04 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2004-03-03 | 2004-09-02 | 2005-03-04 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2008-03-03 | 2008-09-02 | 2009-03-04 | $7700.00 | Window Closed / Lapsed |