US5664044A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Synchronized, variable-speed playback of digitally recorded audio and video
Assignee: International Business Machines Corp
•
Filing: 1996-03-27
•
Granted: 1997-09-02
•
Expires: 2016-03-27
Abstract
Abstract Method and system for providing user-controlled, continuous, synchronized variable-speed playback of a previously recorded digital audio/video presentation. The user directly controls the rate of playback and the audio and video remain synchronized. The audio is expanded or compressed using the time domain harmonic scaling method so that the pitch of the audio remains undistorted. Synchronization is maintained by allowing one clock to serve as the master time clock for the system. The clocks which can serve as the master time clock include the audio decoder clock, the video decoder clock, and the system clock. The invention is particularly useful in multimedia display systems designed to display MPEG data.
Inventors
- 👤 Malcolm S. Ware
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-04 23:20
CPC Classifications
H; H04; H04N; H04N21/00; H04N21/40; H04N21/43; H04N21/434; H04N21/4341; G; G11; G11B; G11B20/00; G10; G10H; G10H1/00; G10H1/0033; G10H1/36; G10H1/361; G10H1/368; G11B20/00007; G11B20/10; G11B20/10527; G11B27/00; G11B27/005; G11B27/10; H04N21/4302; H04N21/4307; H04N21/43072; H04N21/47; H04N21/472; H04N21/47217; H04N5/00; H04N5/04; H04N5/06; H04N5/067; H04N5/073; H04N5/76; H04N5/78; H04N5/782; H04N5/783; H04N5/91; H04N5/93; H04N5/937; H04N7/00; H04N7/24; H04N7/52; H04N7/54; H04N7/56; G10H2240/00; G10H2240/011; G10H2240/046; G10H2240/066; G10H2250/00; G10H2250/541; G10H2250/571; G11B20/14; G11B2020/10537; G11B2020/10546; G11B2020/10555; G11B2020/1062; G11B2020/10629; G11B2020/10638; G11B2020/10805; G11B2020/10814; G11B2220/00; G11B2220/20; G11B2220/25; G11B2220/2537; H04N5/44; H04N5/60; H04N5/602; H04N5/84; H04N5/85; H04N9/00; H04N9/79; H04N9/80; H04N9/804; H04N9/8042
Claims Analysis
Total Claims: 22
Claim 1
Independent Claim
313 words
A method of maintaining synchronization of a stream of digital audio data with a stream of digital video data originally recorded at a specified rate in frames per second, FPS, the digital video data being played back on a variable-speed audio/video display system using a user-selected scaling factor, RV, which is set to a proportional value of a recorded rate, the method comprising the steps of: placing at least 2 frames of digital audio in an audio buffer and at least 2 frames of digital video in a video buffer; setting an audio time domain harmonic scaling factor, C, equal to the user-selected scaling factor, RV; calculating an amount of time in seconds, T, required to play back an audio frame with the selected audio time domain harmonic scaling factor C; decoding and displaying, if the video buffer is not empty, a frame of video from the video buffer for T seconds; decoding, scaling and playing, if the audio buffer is not empty, an audio frame from the audio buffer using the audio time domain harmonic scaling factor C so that playback of the audio frame requires an audio playback time period equal to T seconds; reading, after T seconds, and if the end of the stream of digital audio data or the stream of digital video data has not been reached, another frame from the stream of digital audio into the audio buffer and another frame from the stream of digital video into the video buffer; determining whether the audio playback time period of the audio frame played during the preceding time period T was exactly equal to T seconds; adjusting the value C if the audio playback time period was not exactly equal to T seconds; and recalculating the time T if the audio time domain harmonic scaling factor C has changed since the beginning of the preceding time period T.
Claim 2
Independent Claim
313 words
A method of maintaining synchronization of a stream of digital audio data with a stream of digital video data originally recorded at a specified rate in frames per second, FPS, the digital video data being played back on a variable-speed audio/video display system using a user-selected scaling factor, RV, which is set to a proportional value of a recorded rate, the method comprising the steps of: placing at least 2 frames of digital audio in an audio buffer and at least 2 frames of digital video in a video buffer; setting an audio time domain harmonic scaling factor, C, equal to the user-selected scaling factor, RV; calculating an amount of time in seconds, T, required to play back an audio frame with the selected audio time domain harmonic scaling factor C; decoding and displaying, if the video buffer is not empty, a frame of video from the video buffer for T seconds; decoding, scaling and playing, if the audio buffer is not empty, an audio frame from the audio buffer using the audio time domain harmonic scaling factor C so that playback of the audio frame requires an audio playback time period equal to T seconds; reading, after T seconds, and if the end of the stream of digital audio data or the stream of digital video data has not been reached, another frame from the stream of digital audio into the audio buffer and another frame from the stream of digital video into the video buffer; determining whether the audio playback time period of the audio frame played during the preceding time period T was exactly equal to T seconds; adjusting the value C if the audio playback time period was not exactly equal to T seconds; and recalculating the time T if the audio time domain harmonic scaling factor C has changed since the beginning of the preceding time period T.
Claim 3
Dependent Claim
227 words
In a system for playing digitally recorded audio and video data streams, the system having an audio buffer holding two or more audio frames, and a video buffer holding two or more video frames, a method for maintaining synchronization during variable-speed playback of the audio data stream with the video data stream, where the video data stream is being played using a user-selected scaling factor, RV, which is set to a proportional value of an originally recorded rate, the method comprising the steps of: calculating an amount of time in seconds, T, required to play back an audio frame with an audio time domain harmonic scaling factor C, where C is initially set to equal the user-selected scaling factor RV; decoding and displaying, if the video buffer is not empty, a frame of video from the video buffer for T seconds; decoding, scaling and playing, if the audio buffer is not empty, an audio frame from the audio buffer using the audio time domain harmonic scaling factor C so that playback of the audio frame requires an audio playback time period equal to T seconds; and reading, after T seconds, and if the end of the data streams has not been reached, a frame from the stream of digital audio into the audio buffer and a frame from the stream of digital video into the video buffer.
Claim 4
Dependent Claim
227 words
In a system for playing digitally recorded audio and video data streams, the system having an audio buffer holding two or more audio frames, and a video buffer holding two or more video frames, a method for maintaining synchronization during variable-speed playback of the audio data stream with the video data stream, where the video data stream is being played using a user-selected scaling factor, RV, which is set to a proportional value of an originally recorded rate, the method comprising the steps of: calculating an amount of time in seconds, T, required to play back an audio frame with an audio time domain harmonic scaling factor C, where C is initially set to equal the user-selected scaling factor RV; decoding and displaying, if the video buffer is not empty, a frame of video from the video buffer for T seconds; decoding, scaling and playing, if the audio buffer is not empty, an audio frame from the audio buffer using the audio time domain harmonic scaling factor C so that playback of the audio frame requires an audio playback time period equal to T seconds; and reading, after T seconds, and if the end of the data streams has not been reached, a frame from the stream of digital audio into the audio buffer and a frame from the stream of digital video into the video buffer.
Claim 5
Dependent Claim
101 words
The method according to claim 2 further comprising the steps of: determining whether the audio buffer is approaching an underflow condition because the audio playback time period is less than T seconds; and if the audio buffer is approaching an underflow condition, recalculating the audio time domain harmonic scaling factor C according to an equation C=C-A, where A is an audio adjustment factor which is set to a fractional value of C that is large enough to prevent underflow and overflow of the audio buffer, and small enough so that changes in playback speed are perceived as gradual by a user.
Claim 6
Dependent Claim
56 words
The method according to claim 3 further comprising the steps of: determining whether the audio buffer is approaching an overflow condition because the audio playback time period is more than T seconds; and if the audio buffer is approaching an overflow condition, recalculating the audio time domain harmonic scaling factor C according to an equation C=C+A.
Claim 7
Dependent Claim
102 words
The method according to claim 2 further comprising the steps of: determining whether the audio buffer is in approaching an overflow condition because the audio playback time period is more than T seconds; and if the audio buffer is approaching an overflow condition, recalculating the audio time domain harmonic scaling factor C according to an equation C=C+A, where A is an audio adjustment factor which is set to a fractional value of C that is large enough to prevent underflow and overflow of the audio buffer, but small enough so that changes in playback speed are perceived as gradual by a user.
Claim 8
Dependent Claim
266 words
A digital audio/video playback system for playing digitally encoded, simultaneous audio and video at variable speeds while maintaining synchronization between the audio and video, the system comprising: means for receiving a digital audio data stream and digital video data stream from a source; a video buffer capable of holding at least two frames of video, the video buffer connected to the receiving means; an audio buffer capable of holding at least two frames of audio, the audio buffer connected to the receiving means; means for displaying a plurality of video frames in succession; means for generating sounds from electronic signals; and apparatus for maintaining synchronization during playback of the digital audio data stream with the digital video data stream, where the digital video data stream is being played using a user-selected scaling factor, RV, which is set to a proportional value of an originally recorded rate, the apparatus being disposed between the audio and video buffers and the generating and displaying means, the apparatus including: means for calculating an amount of time in seconds, T, required to play back an audio frame with an audio time domain harmonic scaling factor C, where C is initially set to equal RV; a video decoder for decoding a frame of video from the video buffer and sending the frame of video to the display means; and an audio time domain harmonic scaler for decoding, scaling and playing an audio frame from the audio buffer using the audio time domain harmonic scaling factor C so that playback of the audio frame requires an audio playback time period equal to T.
Claim 9
Dependent Claim
266 words
A digital audio/video playback system for playing digitally encoded, simultaneous audio and video at variable speeds while maintaining synchronization between the audio and video, the system comprising: means for receiving a digital audio data stream and digital video data stream from a source; a video buffer capable of holding at least two frames of video, the video buffer connected to the receiving means; an audio buffer capable of holding at least two frames of audio, the audio buffer connected to the receiving means; means for displaying a plurality of video frames in succession; means for generating sounds from electronic signals; and apparatus for maintaining synchronization during playback of the digital audio data stream with the digital video data stream, where the digital video data stream is being played using a user-selected scaling factor, RV, which is set to a proportional value of an originally recorded rate, the apparatus being disposed between the audio and video buffers and the generating and displaying means, the apparatus including: means for calculating an amount of time in seconds, T, required to play back an audio frame with an audio time domain harmonic scaling factor C, where C is initially set to equal RV; a video decoder for decoding a frame of video from the video buffer and sending the frame of video to the display means; and an audio time domain harmonic scaler for decoding, scaling and playing an audio frame from the audio buffer using the audio time domain harmonic scaling factor C so that playback of the audio frame requires an audio playback time period equal to T.
Claim 10
Dependent Claim
102 words
The system of claim 6 wherein the apparatus for maintaining synchronization during playback further includes: means for determining whether the audio buffer is approaching an underflow condition because the audio playback time period is less than T seconds; and means for recalculating the audio time domain harmonic scaling factor C according to an equation C=C-A, where A is an audio adjustment factor which is set to a fractional value of C that is large enough to prevent underflow and overflow of the audio and video buffers, but small enough so that changes in playback speed are perceived as gradual by a user.
Claim 11
Dependent Claim
50 words
The system of claim 7 wherein the apparatus further includes: means for determining whether the audio buffer is approaching an overflow condition because the audio playback time period is more than T seconds; and means for recalculating the audio time domain harmonic scaling factor C according to an equation C=C+A.
Claim 12
Dependent Claim
100 words
The system of claim 6 wherein the apparatus for maintaining synchronization during playback further includes: means for determining whether the audio buffer is approaching an overflow condition because the audio playback time period is more than T seconds; and means for recalculating the audio time domain harmonic scaling factor C according to an equation C=C+A, where A is an audio adjustment factor which is set to a fractional value of C that is large enough to prevent underflow and overflow of the audio buffer, but small enough so that changes in playback speed are perceived as gradual by a user.
Claim 13
Dependent Claim
195 words
Apparatus for maintaining synchronization during variable-speed playback of an audio data stream with a video data stream, where the video data stream is being played using a user-selected scaling factor, RV, which is set to a proportional value of an originally recorded rate, the apparatus comprising: means for receiving digital audio data and digital video data from a source; a video buffer capable of holding at least two frames of video, the video buffer connected to the receiving means; an audio buffer capable of holding at least two frames of audio, the audio buffer connected to the receiving means; means for calculating an amount of time in seconds, T, required to play back an audio frame with an audio time domain harmonic scaling factor C which is set equal to the user-selected scaling factor RV; means for decoding and displaying a frame of video from the video buffer for T seconds; and an audio time domain harmonic scaler for decoding, scaling and playing an audio frame from the audio buffer using the audio time domain harmonic scaling factor C so that playback of the audio frame requires an audio playback time period equal to T.
Claim 14
Dependent Claim
195 words
Apparatus for maintaining synchronization during variable-speed playback of an audio data stream with a video data stream, where the video data stream is being played using a user-selected scaling factor, RV, which is set to a proportional value of an originally recorded rate, the apparatus comprising: means for receiving digital audio data and digital video data from a source; a video buffer capable of holding at least two frames of video, the video buffer connected to the receiving means; an audio buffer capable of holding at least two frames of audio, the audio buffer connected to the receiving means; means for calculating an amount of time in seconds, T, required to play back an audio frame with an audio time domain harmonic scaling factor C which is set equal to the user-selected scaling factor RV; means for decoding and displaying a frame of video from the video buffer for T seconds; and an audio time domain harmonic scaler for decoding, scaling and playing an audio frame from the audio buffer using the audio time domain harmonic scaling factor C so that playback of the audio frame requires an audio playback time period equal to T.
Claim 15
Dependent Claim
92 words
The apparatus of claim 10 further comprising: means for determining whether the audio buffer is approaching an underflow condition because the audio playback time period is less than T seconds; and means for recalculating the audio time domain harmonic scaling factor C according to an equation C=C-A, where A is an audio adjustment factor which is set to a fractional value of C that is large enough to prevent underflow and overflow of the audio buffer, but small enough so that changes in playback speed are perceived as gradual by a user.
Claim 16
Dependent Claim
47 words
The apparatus of claim 11 further comprising: means for determining whether the audio buffer is approaching an overflow condition because the audio playback time period is more than T seconds; and means for recalculating the audio time domain harmonic scaling factor C according to an equation C=C+A.
Claim 17
Dependent Claim
92 words
The apparatus of claim 10 further comprising: means for determining whether the audio buffer is approaching an overflow condition because the audio playback time period is more than T seconds; and means for recalculating the audio time domain harmonic scaling factor C according to an equation C=C+A, where A is an audio adjustment factor which is set to a fractional value of C that is large enough to prevent underflow and overflow of the audio buffer, but small enough so that changes in playback speed are perceived as gradual by a user.
Claim 18
Dependent Claim
275 words
A digital audio/video playback subsystem for playing digitally encoded, simultaneous audio and video at variable speeds while maintaining synchronization between the audio and video, the system comprising: means for receiving a digital audio data stream and a digital video data stream from a source; a video buffer capable of holding at least two frames of video, the video buffer connected to the receiving means; an audio buffer capable of holding at least two frames of audio, the audio buffer connected to the receiving means; means for connection to apparatus for displaying a plurality of video frames in succession; means for connection to apparatus generating sounds from electronic signals; and apparatus for maintaining synchronization during playback of the digital audio data stream with the digital video data stream, where the digital video data stream is being played using a user-selected scaling factor, RV, which is set to a proportional value of an originally recorded rate, the apparatus disposed between the audio and video buffers and the connection means, the apparatus including: means for calculating an amount of time in seconds, T, required to play back an audio frame with an audio time domain harmonic scaling factor C, where C is set equal to the user-selected scaling factor RV; a video decoder for decoding a frame of video from the video buffer and sending the frame of video to the display apparatus connection means; and an audio time domain harmonic scaler for scaling and playing an audio frame from the audio buffer using the audio time domain harmonic scaling factor C so that playback of the audio frame requires an audio playback time period equal to T.
Claim 19
Dependent Claim
275 words
A digital audio/video playback subsystem for playing digitally encoded, simultaneous audio and video at variable speeds while maintaining synchronization between the audio and video, the system comprising: means for receiving a digital audio data stream and a digital video data stream from a source; a video buffer capable of holding at least two frames of video, the video buffer connected to the receiving means; an audio buffer capable of holding at least two frames of audio, the audio buffer connected to the receiving means; means for connection to apparatus for displaying a plurality of video frames in succession; means for connection to apparatus generating sounds from electronic signals; and apparatus for maintaining synchronization during playback of the digital audio data stream with the digital video data stream, where the digital video data stream is being played using a user-selected scaling factor, RV, which is set to a proportional value of an originally recorded rate, the apparatus disposed between the audio and video buffers and the connection means, the apparatus including: means for calculating an amount of time in seconds, T, required to play back an audio frame with an audio time domain harmonic scaling factor C, where C is set equal to the user-selected scaling factor RV; a video decoder for decoding a frame of video from the video buffer and sending the frame of video to the display apparatus connection means; and an audio time domain harmonic scaler for scaling and playing an audio frame from the audio buffer using the audio time domain harmonic scaling factor C so that playback of the audio frame requires an audio playback time period equal to T.
Claim 20
Dependent Claim
100 words
The subsystem of claim 14 wherein the apparatus for maintaining synchronization during playback further includes: means for determining whether the audio buffer is approaching an underflow condition because the audio playback time period is less than T seconds; and means for recalculating the audio time domain harmonic scaling factor C according to an equation C=C-A, where A is an audio adjustment factor which is set to a fractional value of C that is large enough to prevent underflow and overflow of the audio buffer, but small enough so that changes in playback speed are perceived as gradual by a user.
Claim 21
Dependent Claim
50 words
The subsystem of claim 15 wherein the apparatus further includes: means for determining whether the audio buffer is approaching an overflow condition because the audio playback time period is more than T seconds; and means for recalculating the audio time domain harmonic scaling factor C according to an equation C=C+A.
Claim 22
Dependent Claim
100 words
The subsystem of claim 14 wherein the apparatus for maintaining synchronization during playback further includes: means for determining whether the audio buffer is approaching an overflow condition because the audio playback time period is more than T seconds; and means for recalculating the audio time domain harmonic scaling factor C according to an equation C=C+A, where A is an audio adjustment factor which is set to a fractional value of C that is large enough to prevent underflow and overflow of the audio buffer, but small enough so that changes in playback speed are perceived as gradual by a user.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2000-09-01 | 2001-03-03 | 2001-09-02 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2004-09-01 | 2005-03-03 | 2005-09-02 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2008-09-01 | 2009-03-03 | 2009-09-02 | $7700.00 | Window Closed / Lapsed |