US6072832A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Audio/video/computer graphics synchronous reproducing/synthesizing system and method
Assignee: NEC Corp
•
Filing: 1997-10-24
•
Granted: 2000-06-06
•
Expires: 2017-10-24
Abstract
Abstract In an audio/video/computer graphics data synchronous reproducing/synthesizing system, a demultiplexer separates a bit stream in which an audio signal, a video signal, and computer graphics data are compressed and multiplexed, into compressed audio and video signal streams, audio and video signal time reference values, and a compressed computer graphics data stream. An audio PLL generates a first decoding clock. An audio decoder decodes the audio signal. An audio memory stores the decoded audio signal. A modulator modulates the audio signal in accordance with sound source control information. A video PLL generates a second decoding clock. A video decoder decodes the video signal. A video memory stores the decoded video signal. A CG decoder decodes the computer graphics data and event time management information. A CG memory stores the decoded computer graphics data. An event generator generates an event driving instruction. A detector detects viewpoint movement of an observer. A rendering engine receives the video signal, the computer graphics data, the event driving instruction, and viewpoint movement data and outputs a synthesized image of the video signal and the computer graphics data and the sound source control information. The method for this system is also disclosed.
Inventors
- 👤 Jiro Katto
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-02 13:24
CPC Classifications
H; H04; H04N; H04N21/00; H04N21/80; H04N21/85; H04N21/854; H04N21/8543; G; G09; G09G; G09G5/00; H04N21/40; H04N21/41; H04N21/422; H04N21/43; H04N21/4302; H04N21/4305; H04N21/4307; H04N21/43074; H04N21/434; H04N21/4341; H04N21/4348; H04N21/439; H04N21/44; H04N21/45; H04N21/462; H04N21/4622; H04N21/47; H04N21/472; H04N21/47205; H04N21/81; H04N21/8146
Claims Analysis
Total Claims: 14
Claim 1
Independent Claim
367 words
An audio/video/computer graphics data synchronous reproducing/synthesizing system comprising: separation means for separating a bit stream in which an audio signal, a video signal, and computer graphics data are compressed and multiplexed, into a compressed audio signal stream, an audio signal time reference value, a compressed video signal stream, a video signal time reference value, and a compressed computer graphics data stream; first clock generation means for generating a first decoding clock on the basis of the audio signal time reference value from said separation means; first decoding means for decoding the audio signal from the compressed audio signal stream from said separation means and the first decoding clock from said first clock generation means; first storage means for storing the decoded audio signal from said first decoding means; modulation means for modulating the audio signal from said first storage means in accordance with sound source control information; second clock generation means for generating a second decoding clock on the basis of the video signal time reference value from said separation means; second decoding means for decoding the video signal from the compressed video signal stream from said separation means and the second decoding clock from said second clock generation means; second storage means for storing the decoded video signal from said second decoding means; third decoding means for decoding the computer graphics data and event time management information from the compressed computer graphics data stream from said separation means; third storage means for storing the decoded computer graphics data from said third decoding means; event generation means for generating an event driving instruction on the basis of the second decoding clock from said second clock generation means and the event time management information from said third decoding means; detection means for detecting viewpoint movement of an observer using a pointing device; and rendering means for receiving the video signal stored in said second storage means, the computer graphics data stored in said third storage means, the event driving instruction from said event generation means, and viewpoint movement data from said detection means, and outputting a synthesized image of the video signal and the computer graphics data and the sound source control information used by said modulation means.
Claim 2
Dependent Claim
367 words
An audio/video/computer graphics data synchronous reproducing/synthesizing system comprising: separation means for separating a bit stream in which an audio signal, a video signal, and computer graphics data are compressed and multiplexed, into a compressed audio signal stream, an audio signal time reference value, a compressed video signal stream, a video signal time reference value, and a compressed computer graphics data stream; first clock generation means for generating a first decoding clock on the basis of the audio signal time reference value from said separation means; first decoding means for decoding the audio signal from the compressed audio signal stream from said separation means and the first decoding clock from said first clock generation means; first storage means for storing the decoded audio signal from said first decoding means; modulation means for modulating the audio signal from said first storage means in accordance with sound source control information; second clock generation means for generating a second decoding clock on the basis of the video signal time reference value from said separation means; second decoding means for decoding the video signal from the compressed video signal stream from said separation means and the second decoding clock from said second clock generation means; second storage means for storing the decoded video signal from said second decoding means; third decoding means for decoding the computer graphics data and event time management information from the compressed computer graphics data stream from said separation means; third storage means for storing the decoded computer graphics data from said third decoding means; event generation means for generating an event driving instruction on the basis of the second decoding clock from said second clock generation means and the event time management information from said third decoding means; detection means for detecting viewpoint movement of an observer using a pointing device; and rendering means for receiving the video signal stored in said second storage means, the computer graphics data stored in said third storage means, the event driving instruction from said event generation means, and viewpoint movement data from said detection means, and outputting a synthesized image of the video signal and the computer graphics data and the sound source control information used by said modulation means.
Claim 3
Dependent Claim
89 words
An apparatus according to claim 1, further comprising: first buffer means for buffering the compressed audio signal stream from said separation means and outputting the compressed audio signal stream to said first decoding means; second buffer means for buffering the compressed video signal stream from said separation means and outputting the compressed video signal stream to said second decoding means; and third buffer means for buffering the compressed computer graphics data stream from said separation means and outputting the compressed computer graphics data stream to said third decoding means.
Claim 4
Dependent Claim
29 words
An apparatus according to claim 1, further comprising fourth storage means for storing the synthesized image of the video signal and the computer graphics data from said rendering means.
Claim 5
Dependent Claim
93 words
An apparatus according to claim 3, further comprising video deformation means for deforming the video signal from said second storage means on the basis of two-dimensional projection information of an object, and wherein said rendering means outputs the two-dimensional projection information of said object on which the video signal is mapped to said video deformation means, and said fourth storage means overwrites the deformed video signal from said video deformation means on the synthesized image of the video signal and the computer graphics data from said rendering means and stores the synthesized image.
Claim 6
Dependent Claim
20 words
An apparatus according to claim 1, further comprising delay means for delaying the modulated audio signal from said modulation means.
Claim 7
Dependent Claim
365 words
An audio/video/computer graphics data synchronous reproducing/synthesizing system comprising: separation means for separating a bit stream in which an audio signal and a video signal are compressed and multiplexed, into a compressed audio signal stream, an audio signal time reference value, a compressed video signal stream, and a video signal time reference value; first clock generation means for generating a first decoding clock on the basis of the audio signal time reference value from said separation means; first decoding means for decoding the audio signal from the compressed audio signal stream from said separation means and the first decoding clock from said first clock generation means; first storage means for storing the decoded audio signal from said first decoding means; modulation means for modulating the audio signal from said first storage means in accordance with sound source control information; second clock generation means for generating a second decoding clock on the basis of the video signal time reference value from said separation means; second decoding means for decoding the video signal from the compressed video signal stream from said separation means and the second decoding clock from said second clock generation means; second storage means for storing the decoded video signal from said second decoding means; third decoding means for decoding computer graphics data and event time management information from a compressed computer graphics data stream which is not multiplexed with the compressed audio arid video signal streams; third storage means for storing the decoded computer graphics data from said third decoding means; event generation means for generating an event driving instruction on the basis of the second decoding clock from said second clock generation means and the event time management information from said third decoding means; detection means for detecting viewpoint movement of an observer using a pointing device; and rendering means for receiving the video signal stored in said second storage means, the computer graphics data stored in said third storage means, the event driving instruction from said event generation means, and viewpoint movement data from said detection means, and outputting a synthesized image of the video signal and the computer graphics data and the sound source control information used by said modulation means.
Claim 8
Dependent Claim
365 words
An audio/video/computer graphics data synchronous reproducing/synthesizing system comprising: separation means for separating a bit stream in which an audio signal and a video signal are compressed and multiplexed, into a compressed audio signal stream, an audio signal time reference value, a compressed video signal stream, and a video signal time reference value; first clock generation means for generating a first decoding clock on the basis of the audio signal time reference value from said separation means; first decoding means for decoding the audio signal from the compressed audio signal stream from said separation means and the first decoding clock from said first clock generation means; first storage means for storing the decoded audio signal from said first decoding means; modulation means for modulating the audio signal from said first storage means in accordance with sound source control information; second clock generation means for generating a second decoding clock on the basis of the video signal time reference value from said separation means; second decoding means for decoding the video signal from the compressed video signal stream from said separation means and the second decoding clock from said second clock generation means; second storage means for storing the decoded video signal from said second decoding means; third decoding means for decoding computer graphics data and event time management information from a compressed computer graphics data stream which is not multiplexed with the compressed audio arid video signal streams; third storage means for storing the decoded computer graphics data from said third decoding means; event generation means for generating an event driving instruction on the basis of the second decoding clock from said second clock generation means and the event time management information from said third decoding means; detection means for detecting viewpoint movement of an observer using a pointing device; and rendering means for receiving the video signal stored in said second storage means, the computer graphics data stored in said third storage means, the event driving instruction from said event generation means, and viewpoint movement data from said detection means, and outputting a synthesized image of the video signal and the computer graphics data and the sound source control information used by said modulation means.
Claim 9
Dependent Claim
84 words
An apparatus according to claim 6, further comprising: first buffer means for buffering the compressed audio signal stream from said separation means and outputting the compressed audio signal stream to said first decoding means; second buffer means for buffering the compressed video signal stream from said separation means and outputting the compressed video signal stream to said second decoding means; and third buffer means for buffering the compressed computer graphics data stream which is not multiplexed with the compressed audio and video signal streams.
Claim 10
Dependent Claim
29 words
An apparatus according to claim 6, further comprising fourth storage means for storing the synthesized image of the video signal and the computer graphics data from said rendering means.
Claim 11
Dependent Claim
93 words
An apparatus according to claim 8, further comprising video deformation means for deforming the video signal from said second storage means on the basis of two-dimensional projection information of an object, and wherein said rendering means outputs the two-dimensional projection information of said object on which the video signal is mapped to said video deformation means, and said fourth storage means overwrites the deformed video signal from said video deformation means on the synthesized image of the video signal and the computer graphics data from said rendering means and stores the synthesized image.
Claim 12
Dependent Claim
20 words
An apparatus according to claim 6, further comprising delay means for delaying the modulated audio signal from said modulation means.
Claim 13
Dependent Claim
132 words
An audio/video/computer graphics data synchronous reproducing/synthesizing method, comprising the steps of: separating a bit stream in which an audio signal, a video signal, and computer graphics data are compressed and multiplexed, into compressed audio and video signal streams, audio and video signal identification reference values, and a compressed computer graphics data stream; generating a signal clock from the audio and video identification reference values; decoding the audio and video signals from the compressed audio and video signal streams; respectively, using the generated signal clock, decoding event time reference information from the compressed computer graphics data stream, and generating an event driving instruction from the generated signal clock and the decoded event time reference information to synchronize a synthesized image of the video signal and the computer graphics data with the audio signal.
Claim 14
Dependent Claim
132 words
An audio/video/computer graphics data synchronous reproducing/synthesizing method, comprising the steps of: separating a bit stream in which an audio signal, a video signal, and computer graphics data are compressed and multiplexed, into compressed audio and video signal streams, audio and video signal identification reference values, and a compressed computer graphics data stream; generating a signal clock from the audio and video identification reference values; decoding the audio and video signals from the compressed audio and video signal streams; respectively, using the generated signal clock, decoding event time reference information from the compressed computer graphics data stream, and generating an event driving instruction from the generated signal clock and the decoded event time reference information to synchronize a synthesized image of the video signal and the computer graphics data with the audio signal.
References Cited (Backward Citations)
JPH07212653A
Cited
US5774206A
Cited
US5812791A
Cited
US5896140A
Cited
US5933597A
Cited
US5949436A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2003-06-06 | 2003-12-06 | 2004-06-06 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2007-06-06 | 2007-12-06 | 2008-06-06 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2011-06-06 | 2011-12-06 | 2012-06-06 | $7700.00 | Window Closed / Lapsed |