US4841575A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Image encoding and synthesis
Assignee: British Telecommunications PLC
•
Filing: 1986-11-14
•
Granted: 1989-06-20
•
Expires: 2006-11-14
Abstract
Abstract Visual images of the face of a speaker are processed to extract during a learning sequence a still frame of the image and a set of typical mouth shapes. Encoding of a sequence to be transmitted, recorded etc. is then achieved by matching the changing mouth shapes to those of the set and generating codewords identifying them. Alternatively, the codewords may be generated to accompany real or synthetic speech using a look- up table relating speech parameters to codewords. In a receiver, the still frames and set of mouth shapes are stored and received codewords used to select successive mouth shapes to be incorporated in the still frame.
Inventors
- 👤 William J. Welsh
- 👤 Brian A. Fenn
- 👤 Paul Challener
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-24 21:45
CPC Classifications
G; G06; G06T; G06T9/00; G06T9/001; G10; G10L; G10L21/00; G10L21/06; G10L21/10; G10L2021/105; H; H04; H04N; H04N19/00; H04N19/20
Claims Analysis
Total Claims: 21
Claim 1
Independent Claim
139 words
An apparatus for encoding a moving image including a human face comprising: means for receiving video input data; means for output of data representing one frame of the image; identification means arranged in operation for each frame of the image to identify that part of the input data corresponding to the mouth of the face represented and (a) in a first phase of operation to compare the mouth data parts of each frame with those of other frames to select a representative set of mouth data parts, to store the selected parts and to output the selected parts; (b) in a second phase to compare the mouth data part of each frame with those of the stored set and to generate a codeword indicating which member of the set the mouth data part of that frame most closely resembles.
Claim 2
Independent Claim
139 words
An apparatus for encoding a moving image including a human face comprising: means for receiving video input data; means for output of data representing one frame of the image; identification means arranged in operation for each frame of the image to identify that part of the input data corresponding to the mouth of the face represented and (a) in a first phase of operation to compare the mouth data parts of each frame with those of other frames to select a representative set of mouth data parts, to store the selected parts and to output the selected parts; (b) in a second phase to compare the mouth data part of each frame with those of the stored set and to generate a codeword indicating which member of the set the mouth data part of that frame most closely resembles.
Claim 3
Dependent Claim
52 words
An apparatus according to claim 1 in which the identification means is arranged in operation firstly to identify that part of one frame of input data corresponding to the mouth of the face represented and to identify the mouth part of successive frames by auto-correlation with data from the said one frame.
Claim 4
Dependent Claim
66 words
An apparatus according to claim 1 or 2 in which the identification means is arranged in operation during the first phase to store a first mouth data part and then for the mouth data parts of each successive frame to compare it with the first and any other stored mouth data part and if the result of the comparison exceeds a threshold value, to store it.
Claim 5
Dependent Claim
33 words
An apparatus according to claim 1, or 2 in which the comparison of mouth data is carried out by subtraction of individual picture element values and summing the absolute values of the differences.
Claim 6
Dependent Claim
25 words
An apparatus according to claim 1, or 2 including means for tracking the face position within the image and generating coded data representing that position.
Claim 7
Independent Claim
68 words
An apparatus according to any one of the preceding claims 1 or 2, in which the identification means is arranged during the second phase in the event that the result of the comparison between a mouth data part and that one of the stored set which it most closely resembles exceeds a predetermined threshold to output that data part and store it as part of the stored set.
Claim 8
Independent Claim
143 words
An apparatus according to any one of the preceding claims 1 or 2 further including identification means arranged in operation for each frame of the image to identify that part of the input data corresponding to the eyes of the face represented and (a) in the first phase of operation to compare, for each frame, the eye data part thus identified with those of other frames to eye data parts of each frame with those of other frames to select a representative set of eye data parts, to store the selected parts and to output the selected parts; (b) in the second phase to compare the eye data part of each frame with those of the stored set of eye data parts and to generate a codeword indicating which member of the set the eye data part of that frame most closely resembles.
Claim 9
Dependent Claim
115 words
An image transmission system comprising an encoding apparatus according to any one of claims 1 or 2 and a receiver for decoding a moving image including a human face, the receiver comprising: frame store means for receiving and storing data representing one frame of the image; means for repetitive readout of the frame store to produce a video signal; further store means receiving and storing a set of selected mouth data parts; and control means arranged in operation to receive input codewords and in response to each codeword to read out the corresponding mouth data part from the further store and to effect insertion of that data into the data supplied to the readout means.
Claim 10
Dependent Claim
97 words
A receiver for decoding a moving image including a human face, the receiver comprising: frame store means for receiving and storing data representing one frame of the image; means for repetitive readout of the frame store to produce a video signal; further store means receiving and storing a set of selected mouth data parts; and control means arranged in operation to receive input codewords and in response to each codeword to read out the corresponding mouth data part from the further store and to effect insertion of that data into the data supplied to the readout means.
Claim 11
Dependent Claim
97 words
A receiver for decoding a moving image including a human face, the receiver comprising: frame store means for receiving and storing data representing one frame of the image; means for repetitive readout of the frame store to produce a video signal; further store means receiving and storing a set of selected mouth data parts; and control means arranged in operation to receive input codewords and in response to each codeword to read out the corresponding mouth data part from the further store and to effect insertion of that data into the data supplied to the readout means.
Claim 12
Dependent Claim
22 words
A receiver according to claim 9 in which the control means is arranged to overwrite the frame store with the mouth data.
Claim 13
Dependent Claim
22 words
A receiver according to claim 9 in which the control means is arranged to supply the mouth data to the readout means.
Claim 14
Dependent Claim
28 words
A receiver according to claim 9, 10 or 11 including means responsive to input data to effect corresponding movement of the face within the area of the image.
Claim 15
Dependent Claim
25 words
A receiver according to claim 9, 10 or 11 including means arranged to effect random movement of the face within the area of the image.
Claim 16
Dependent Claim
116 words
A speech synthesiser including means for synthesis of a moving image including a human face, comprising; (a) means for storage and output of the image of a face; (b) means for storage and output of a set of mouth data blocks each corresponding to the mouth area of the face and representing a respective different mouth shape; (c) speech synthesis mans responsive to input information identifying words or parts of words to be spoken; (d) means storing a table relating words or part of words to codewords identifying mouth data blocks or sequences thereof; (e) control means responsive to the said input information to select for output the corresponding codewords or codeword sequences from the table.
Claim 17
Dependent Claim
116 words
A speech synthesiser including means for synthesis of a moving image including a human face, comprising; (a) means for storage and output of the image of a face; (b) means for storage and output of a set of mouth data blocks each corresponding to the mouth area of the face and representing a respective different mouth shape; (c) speech synthesis mans responsive to input information identifying words or parts of words to be spoken; (d) means storing a table relating words or part of words to codewords identifying mouth data blocks or sequences thereof; (e) control means responsive to the said input information to select for output the corresponding codewords or codeword sequences from the table.
Claim 18
Dependent Claim
72 words
A synthesiser according to claim 14 in which the speech synthesis means includes means arranged in operation for processing and queuing the input information, the queue including flag codes indicating changes in mouth shape, and responsive to each flag code to transmit to the control means an indication following generation of the speech preceding that code in the queue, whereby the control means may synchronise the codeword output to the synthesised speech.
Claim 19
Independent Claim
112 words
An apparatus for synthesis of a moving image, comprising: (a) means for storage and output of the image of a face; (b) means for storage and output of a set of mouth data blocks each corresponding to the mouth area of the face and representing a respective different mouth shape; (c) an audio input for receiving speech signals and frequency analysis means responsive to such signals to produce sequences of spectral parameters; (d) means storing a table relating spectral parameter sequences to codewords identifying mouth data blocks or sequences thereof; (e) control means responsive to the said spectral parameters to select for output the corresponding codewords or codeword sequences from the table.
Claim 20
Independent Claim
112 words
An apparatus for synthesis of a moving image, comprising: (a) means for storage and output of the image of a face; (b) means for storage and output of a set of mouth data blocks each corresponding to the mouth area of the face and representing a respective different mouth shape; (c) an audio input for receiving speech signals and frequency analysis means responsive to such signals to produce sequences of spectral parameters; (d) means storing a table relating spectral parameter sequences to codewords identifying mouth data blocks or sequences thereof; (e) control means responsive to the said spectral parameters to select for output the corresponding codewords or codeword sequences from the table.
Claim 21
Dependent Claim
107 words
An apparatus according to claim 14, 15 or 16 including a receiver for decoding a moving image including a human face, the receiver comprising: frame store means for receiving and storing data representing one frame of the image; means for repetitive readout of the frame store to produce a video signal; further store means receiving and storing a set of selected mouth data parts; and control means arranged in operation to receive input codewords and in response to each codeword to read out the corresponding mouth data part from the further store and to effect insertion of that data into the data supplied to the readout means.
References Cited (Backward Citations)
US3662374A
Cited
US4104625A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1992-06-19 | 1992-12-19 | 1993-06-20 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 1996-06-19 | 1996-12-19 | 1997-06-20 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2000-06-19 | 2000-12-19 | 2001-06-20 | $7700.00 | Window Closed / Lapsed |