US5874997A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Measuring and regulating synchronization of merged video and audio data
Assignee: FutureTel Inc
•
Filing: 1994-08-29
•
Granted: 1999-02-23
•
Expires: 2014-08-29
Abstract
Abstract This invention concerns real-time assembly of a compressed audio-visual system data stream (22) so the audio and video data may be subsequently presented in synchronism. Assembly of a system data stream (22) in accordance with present invention interleaves packets of data selected from a compressed audio bitstream (16) with packets of data selected from a compressed video bitstream (18). If frames of video data being assembled into the system data stream (22) advance too far ahead of the audio data being assembled into the system data stream (22), then all the data for a single frame of the video signal is omitted from the system data stream (22). Conversely, if frames of video data being assembled into the system data stream (22) lag too far behind the audio data being assembled into the system data stream (22), then a second copy of all the data for a single frame of the video signal is assembled into the system data stream (22).
Inventors
- 👤 Stephen G Haigh
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-08 15:47
CPC Classifications
H; H04; H04N; H04N21/00; H04N21/20; H04N21/23; H04N21/236; H04N21/2368; H04N21/242; H04N21/40; H04N21/43; H04N21/4302; H04N21/4305; H04N21/4307; H04N21/43072; H04N21/434; H04N21/4341
Claims Analysis
Total Claims: 14
Claim 1
Independent Claim
384 words
A method for real-time assembly of an encoded system data stream which is decodable by a decoder into decoded video pictures and into a decoded audio signal, the system data stream being assembled so the decoder presents the decoded audio signal substantially in synchronism with the decoded video pictures, the system data stream being assembled by interleaving packets of data selected from a compressed audio bitstream with packets of data selected from a compressed video bitstream, the compressed audio bitstream being generated by compressing an audio signal that is sampled at a pre-specified audio sampling rate ("PSASR"), the compressed video bitstream being generated by compressing a sequence of frames of a video signal having a pre-specified video frame rate ("PSVFR"), the method comprising the steps of: a. before commencing assembly of the system data stream, computing an expected encoded audio-video ratio ("EEAVR") which equals of the PSASR divided by the PSVFR; b. embedding in the system data stream a system header ("SH") in which both a system -- audio -- lock -- flag and a system -- video -- lock -- flag are set to indicate respectively that a specified, constant relationship exists between an audio sampling rate and a system clock reference ("SCR"), and a specified, constant relationship exists between a video picture rate and the SCR; c. repetitively selecting a packet of data respectively from the compressed audio bitstream or from the compressed video bitstream for assembly into the system data stream; d. repetitively embedding into the system data stream, together with each packet selected respectively from the compressed audio bitstream or from the compressed video bitstream, a presentation time-stamp ("PTS"); e. computing an actual encoded audio-video ratio ("AEAVR") which equals a number that represents a count of all the samples ("NOS") of the audio signal that have been received for compression divided by a total number of frames ("NOF") of the video signal that have been received for compression; f. computing an encoded frame error value ("EFEV") by first subtracting the EEAVR from the AEAVR to obtain a difference of ratios ("DOR"), and then multiplying the DOR thus computed by NOF; and g. if the EFEV is less than a pre-specified negative error value ("PSNEV"), omitting from the system data stream all data for an entire frame of the video signal.
Claim 2
Independent Claim
384 words
A method for real-time assembly of an encoded system data stream which is decodable by a decoder into decoded video pictures and into a decoded audio signal, the system data stream being assembled so the decoder presents the decoded audio signal substantially in synchronism with the decoded video pictures, the system data stream being assembled by interleaving packets of data selected from a compressed audio bitstream with packets of data selected from a compressed video bitstream, the compressed audio bitstream being generated by compressing an audio signal that is sampled at a pre-specified audio sampling rate ("PSASR"), the compressed video bitstream being generated by compressing a sequence of frames of a video signal having a pre-specified video frame rate ("PSVFR"), the method comprising the steps of: a. before commencing assembly of the system data stream, computing an expected encoded audio-video ratio ("EEAVR") which equals of the PSASR divided by the PSVFR; b. embedding in the system data stream a system header ("SH") in which both a system -- audio -- lock -- flag and a system -- video -- lock -- flag are set to indicate respectively that a specified, constant relationship exists between an audio sampling rate and a system clock reference ("SCR"), and a specified, constant relationship exists between a video picture rate and the SCR; c. repetitively selecting a packet of data respectively from the compressed audio bitstream or from the compressed video bitstream for assembly into the system data stream; d. repetitively embedding into the system data stream, together with each packet selected respectively from the compressed audio bitstream or from the compressed video bitstream, a presentation time-stamp ("PTS"); e. computing an actual encoded audio-video ratio ("AEAVR") which equals a number that represents a count of all the samples ("NOS") of the audio signal that have been received for compression divided by a total number of frames ("NOF") of the video signal that have been received for compression; f. computing an encoded frame error value ("EFEV") by first subtracting the EEAVR from the AEAVR to obtain a difference of ratios ("DOR"), and then multiplying the DOR thus computed by NOF; and g. if the EFEV is less than a pre-specified negative error value ("PSNEV"), omitting from the system data stream all data for an entire frame of the video signal.
Claim 3
Dependent Claim
44 words
The method of claim 1 comprising the further step of: h. if the EFEV is greater than a pre-specified positive error value ("PSPEV"), assembling into the system data stream a second copy of all the data for an entire frame of the video signal.
Claim 4
Dependent Claim
34 words
The method of claim 2 wherein the PSPEV represents an interval of time which is greater than a time interval required for presentation of one-half of a single frame of the decoded video pictures.
Claim 5
Dependent Claim
56 words
The method of claim 2 wherein during several minutes immediately following commencing assembly of the system data stream omission of a frame of the video signal from the system data stream, and addition of a second copy of all the data for an entire frame of the video signal to the system data stream, are inhibited.
Claim 6
Dependent Claim
51 words
The method of claim 2 comprising the further step of: i. applying a low pass filter to EFEV before determining whether to omit a frame of the video signal from the system data stream, and before determining whether to add a frame of the video signal to the system data stream.
Claim 7
Dependent Claim
34 words
The method of claim 1 wherein the PSNEV represents an interval of time which is greater than a time interval required for presentation of one-half of a single frame of the decoded video pictures.
Claim 8
Dependent Claim
33 words
The method of claim 1 wherein during several minutes immediately following commencing assembly of the system data stream omission of a frame of the video signal from the system data stream is inhibited.
Claim 9
Dependent Claim
34 words
The method of claim 1 comprising the further step of: h. applying a low pass filter to EFEV before determining whether to omit a frame of the video signal from the system data stream.
Claim 10
Independent Claim
402 words
A method for real-time assembly of an encoded system data stream which is decodable by a decoder into decoded video pictures and into a decoded audio signal, the system data stream being assembled so the decoder presents the decoded audio signal substantially in synchronism with the decoded video pictures, the system data stream being assembled by interleaving packets of data selected from a compressed audio bitstream with packets of data selected from a compressed video bitstream, the compressed audio bitstream being generated by compressing an audio signal that is sampled at a pre-specified audio sampling rate"), the compressed video bitstream being generated by compressing a sequence of frames of a video signal having a pre-specified video frame rate ("PSVFR"), the method comprising the steps of: a. before commencing assembly of the system data stream, computing an expected encoded audio-video ratio ("EEAVR") which equals of the PSASR divided by the PSVFR; b. embedding in the system data stream a system header ("SH") in which both a system -- audio -- lock -- flag and a system -- video -- lock -- flag are set to indicate respectively that a specified, constant relationship exists between an audio sampling rate and a system clock reference ("SCR"), and a specified, constant relationship exists between a video picture rate and the SCR; c. repetitively selecting a packet of data respectively from the compressed audio bitstream or from the compressed video bitstream for assembly into the system data stream; d. repetitively embedding into the system data stream, together with each packet selected respectively from the compressed audio bitstream or from the compressed video bitstream, a presentation time-stamp ("PTS"); e. computing an actual encoded audio-video ratio ("AEAVR") which a number that represents a count of all the samples ("NOS") of the audio signal that have been received for compression divided by equals a total number of frames ("NOF") of the video signal that have been received for compression; f. computing an encoded frame error value ("EFEV") by first subtracting the EEAVR from the AEAVR to obtain a difference of ratios ("DOR"), and then multiplying the DOR thus computed by the total number of frames of the video signal that have been received for compression; and g. if the EFEV is greater than a pre-specified positive error value ("PSPEV"), assembling into the system data stream a second copy of all the data for an entire frame of the video signal.
Claim 11
Independent Claim
402 words
A method for real-time assembly of an encoded system data stream which is decodable by a decoder into decoded video pictures and into a decoded audio signal, the system data stream being assembled so the decoder presents the decoded audio signal substantially in synchronism with the decoded video pictures, the system data stream being assembled by interleaving packets of data selected from a compressed audio bitstream with packets of data selected from a compressed video bitstream, the compressed audio bitstream being generated by compressing an audio signal that is sampled at a pre-specified audio sampling rate"), the compressed video bitstream being generated by compressing a sequence of frames of a video signal having a pre-specified video frame rate ("PSVFR"), the method comprising the steps of: a. before commencing assembly of the system data stream, computing an expected encoded audio-video ratio ("EEAVR") which equals of the PSASR divided by the PSVFR; b. embedding in the system data stream a system header ("SH") in which both a system -- audio -- lock -- flag and a system -- video -- lock -- flag are set to indicate respectively that a specified, constant relationship exists between an audio sampling rate and a system clock reference ("SCR"), and a specified, constant relationship exists between a video picture rate and the SCR; c. repetitively selecting a packet of data respectively from the compressed audio bitstream or from the compressed video bitstream for assembly into the system data stream; d. repetitively embedding into the system data stream, together with each packet selected respectively from the compressed audio bitstream or from the compressed video bitstream, a presentation time-stamp ("PTS"); e. computing an actual encoded audio-video ratio ("AEAVR") which a number that represents a count of all the samples ("NOS") of the audio signal that have been received for compression divided by equals a total number of frames ("NOF") of the video signal that have been received for compression; f. computing an encoded frame error value ("EFEV") by first subtracting the EEAVR from the AEAVR to obtain a difference of ratios ("DOR"), and then multiplying the DOR thus computed by the total number of frames of the video signal that have been received for compression; and g. if the EFEV is greater than a pre-specified positive error value ("PSPEV"), assembling into the system data stream a second copy of all the data for an entire frame of the video signal.
Claim 12
Dependent Claim
34 words
The method of claim 9 wherein the PSPEV represents an interval of time which is greater than a time interval required for presentation of one-half of a single frame of the decoded video pictures.
Claim 13
Dependent Claim
42 words
The method of claim 9 wherein during several minutes immediately following commencing assembly of the system data stream addition of a second copy of all the data for an entire frame of the video signal to the system data stream is inhibited.
Claim 14
Dependent Claim
34 words
The method of claim 9 comprising the further step of: h. applying a low pass filter to EFEV before determining whether to add a frame of the video signal to the system data stream.
References Cited (Backward Citations)
US4734764A
Cited
US4847690A
Cited
US4849817A
Cited
US5053860A
Cited
US5231492A
Cited
US5323396A
Cited
US5481543A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-02-22 | 2002-08-24 | 2003-02-23 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-02-22 | 2006-08-24 | 2007-02-23 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-02-22 | 2010-08-24 | 2011-02-23 | $7700.00 | Window Closed / Lapsed |