Programmable audio-video synchronization method and apparatus for multimedia systems

Claim 2

A multimedia system having an audio/video decoding/decompressing circuit for decoding an encoded/compressed data stream, the audio/video decoding/decompressing circuit comprising: a parser for separating the encoded/compressed data stream into encoded/compressed audio data, encoded/compressed video data, an audio time value, and a video time value, wherein each of the audio time value and the video time value comprise a time stamp for the encoded/compressed audio data and the encoded/compressed video data respectively; a memory having addressable storage, coupled to the parser, addressing addressable storage to selectively store and read the encoded/compressed audio data, the encoded/compressed video data, or decoded/decompressed video data to and from addressable storage; a first decoder, coupled to the memory, for decoding the encoded/compressed video data and generating the decoded/decompressed video data; and a synchronizer, coupled to the first decoder and the memory, for comparing the video time value with the audio time value and when the comparison exceeds a predetermined threshold, substantially synchronizing the generation of the decoded/decompressed video data by the first decoder with the addressing of addressable storage by the memory.

Claim 3

The audio/video decoding circuit of claim 1 wherein the synchronizer determines that the video time value exceeds the audio time value by the predetermined threshold, and a sequence of addressing addressable storage containing decoded/decompressed video data by the memory is modified by repeating a sequence of addresses to substantially synchronize the generation of the decoded/decompressed video data by the first decoder with the addressing of addressable storage by the memory.

Claim 4

The audio/video decoding circuit of claim 1 wherein the synchronizer determines that the audio time value exceeds the video time value by the predetermined threshold, and a sequence of addressing addressable storage containing encoded/compressed video data by the memory is modified by skipping a sequence of addresses to substantially synchronize the generation of the decoded/decompressed video data by the first decoder with the addressing of addressable storage by the memory.

Claim 5

The audio/video decoding circuit of claim 1 further comprising a second decoder, coupled to the memory, for decoding the encoded/compressed audio data into decoded/decompressed audio data.

Claim 6

The audio/video decoding circuit of claim 1 further comprising a graphics controller, coupled to the first decoder and a video monitor, for displaying the decoded/decompressed video data as an image on the video monitor.

Claim 7

The audio/video decoding circuit of claim 1 further comprising a graphics controller, coupled to the first decoder and a digital display, for displaying the decoded/decompressed video data as an image on the digital display.

Claim 8

The audio/video decoding circuit of claim 1 further comprising a television signal encoder, coupled to the first decoder and a television display, for generating television data from the decoded/decompressed video data and displaying the television data as an image on the television display.

Claim 9

The audio/video decoding circuit of claim 4 further comprising a digital to analog converter, coupled to the second decoder for generating an analog audio signal from the decoded/decompressed audio data.

Claim 10

The audio/video decoding circuit of claim 8 further comprising an audio amplifier, coupled to the digital to analog converter, for amplifying the magnitude of the analog audio signal.

Claim 13

The method of claim 10 further comprising the step of: skipping M compressed video data frames to substantially synchronize a compressed video data frame with a compressed audio data packet when it is determined in the first comparing step that the video display is lagging the audio playback by more than a lag threshold.

Claim 14

The method of claim 11 wherein the lag threshold is three frames of a video frame display time and the value of M is three.

Claim 15

The method of claim 11 wherein the lag threshold is one frame of a video frame display time and the value of M is one.

Claim 16

The method of claim 11 wherein the lag threshold is one half of a video frame display time and the value of M is one.

Claim 17

The method of claim 11 wherein the lag threshold is one half of a video frame display time and the value of M is one half.

Claim 18

The method of claim 10 further comprising the step of: comparing the normalized audio time value with the video time value to determine whether the video display is leading the audio playback and requires synchronization.

Claim 19

The method of claim 16 further comprising the step of: repeating N decompressed video data frames to substantially synchronize a decompressed video data frame with a decompressed audio data packet when it is determined in the second comparing step that the video display is leading the audio playback by more than a lead threshold.

Claim 20

The method of claim 17 wherein the lead threshold is three frames of a video frame display time and the value of N is three.

Claim 21

The method of claim 17 wherein the lead threshold is one frame of a video frame display time and the value of N is one.

Claim 22

The method of claim 17 wherein the lead threshold is one half of a video frame display time and the value of N is one.

Claim 23

The method of claim 17 wherein the lead threshold is one half of a video frame display time and the value of N is one half.

Claim 26

The method of claim 22 further comprising the step of: skipping M compressed video data frames to substantially synchronize a compressed video data frame with a compressed audio data packet when it is determined in the first comparing step that the video display is lagging the audio playback by more than a lag threshold.

Claim 27

The method of claim 23 wherein the lag threshold is two and one half frames of a video frame display time and the value of M is three.

Claim 28

The method of claim 23 wherein the lag threshold is one frame of a video frame display time and the value of M is one.

Claim 29

The method of claim 23 wherein the lag threshold is one half of a video frame display time and the value of M is one.

Claim 30

The method of claim 23 wherein the lag threshold is one half of a video frame display time and the value of M is one half.

Claim 31

The method of claim 22 further comprising the step of: comparing the normalized video time value with the audio time value to determine whether the video display is leading the audio playback and requires synchronization.

Claim 32

The method of claim 28 further comprising the step of: repeating N decompressed video data frames to substantially synchronize a decompressed video data frame with a decompressed audio data packet when it is determined in the second comparing step that the video display is leading the audio playback by more than a lead threshold.

Claim 33

The method of claim 29 wherein the lead threshold is three frames of a video frame display time and the value of N is three.

Claim 34

The method of claim 29 wherein the lead threshold is one frame of a video frame display time and the value of N is one.

Claim 35

The method of claim 29 wherein the lead threshold is one half of a video frame display time and the value of N is one.

Claim 36

The method of claim 29 wherein the lead threshold is one half of a video frame display time and the value of N is one half.

Claim 37

An audio/video synchronization circuit for substantially synchronizing video display images having a plurality of compressed and decompressed video data frames, with audio playback having a plurality of compressed and decompressed audio data packets, the audio/video synchronization circuit comprising: a counter initialized to an audio time value upon receiving the audio time value within an audio/video encoded data stream and incremented by a clock to generate a normalized audio time value, the audio time value comprising a time stamp; a memory means for storing a video time value upon receiving the video time value within the audio/video encoded data stream, the video time value also comprising a time stamp; and a subtracter coupled to the memory means, for subtracting the video time value stored in said memory means from the normalized audio time value to generate a time drift value, for comparing the time drift value with a positive lag threshold to determine whether the video display is lagging the audio playback by the lag threshold and when exceeding the lag threshold skipping M compressed video data frames to substantially synchronize a compressed video data frame with a compressed audio data packet, and for comparing the time drift value with the negative value of a lead threshold to determine whether the video display is leading the audio playback by the lead threshold and when exceeding the lead threshold repeating N decompressed video data frames to substantially synchronize a decompressed video data frame with a decompressed audio data packet.

Claim 38

An audio/video synchronization circuit for substantially synchronizing video display images having a plurality of compressed and decompressed video data frames, with audio playback having a plurality of compressed and decompressed audio data packets, the audio/video synchronization circuit comprising: a counter initialized to an audio time value upon receiving the audio time value within an audio/video encoded data stream and incremented by a clock to generate a normalized audio time value, the audio time value comprising a time stamp; a memory means for storing a video time value upon receiving the video time value within the audio/video encoded data stream, the video time value also comprising a time stamp; and a subtracter coupled to the memory means, for subtracting the video time value stored in said memory means from the normalized audio time value to generate a time drift value, for comparing the time drift value with a positive lag threshold to determine whether the video display is lagging the audio playback by the lag threshold and when exceeding the lag threshold skipping M compressed video data frames to substantially synchronize a compressed video data frame with a compressed audio data packet, and for comparing the time drift value with the negative value of a lead threshold to determine whether the video display is leading the audio playback by the lead threshold and when exceeding the lead threshold repeating N decompressed video data frames to substantially synchronize a decompressed video data frame with a decompressed audio data packet.

Claim 39

The audio/video synchronization circuit of claim 34 wherein the lag threshold and lead threshold is three frames of a video frame display time and the value for M and N is three.

Claim 40

The audio/video synchronization circuit of claim 34 wherein the lag threshold and lead threshold is one frame of a video frame display time and the value for M and N is one.

Claim 41

The audio/video synchronization circuit of claim 34 wherein the lag threshold and lead threshold is one half of a video frame display time and the value for M and N is one.

Claim 42

The audio/video synchronization circuit of claim 34 wherein the lag threshold and lead threshold is one half of a video frame display time and the value for M and N is one half.

Claim 43

An audio/video synchronization circuit for substantially synchronizing video display images having a plurality of compressed and decompressed video data frames with audio playback having a plurality of compressed and decompressed audio data packets, the audio/video synchronization circuit comprising: a frequency divider for selectively dividing the frequency of a clock in response to a display frame rate to generate a divided frequency clock; a counter, coupled to the frequency divider, initialized to an audio time value upon receiving the audio time value within an audio/video encoded data stream, and incremented by the divided frequency clock to generate a normalized audio time value., wherein the audio time value comprises a time stamp; a first memory means for storing a video time value upon receiving the video time value within the audio/video encoded data stream, wherein the video time value also comprises a time stamp; a second memory means for storing a threshold value; and an arithmetic means selectively coupled to the first memory means, the second memory means, and the counter, for selectively adding or subtracting the threshold value and the video time value to generate a first result and selectively adding or subtracting the normalized audio time value and the first result to generate a second result, wherein the second result indicates that the video display lags the audio playback by more than the threshold value; and a synchronizer for causing M compressed video data frames to be skipped in order to substantially synchronize a decompressed video data frame with a decompressed audio data packet.

Claim 44

An audio/video synchronization circuit for substantially synchronizing video display images having a plurality of compressed and decompressed video data frames with audio playback having a plurality of compressed and decompressed audio data packets, the audio/video synchronization circuit comprising: a frequency divider for selectively dividing the frequency of a clock in response to a display frame rate to generate a divided frequency clock; a counter, coupled to the frequency divider, initialized to an audio time value upon receiving the audio time value within an audio/video encoded data stream, and incremented by the divided frequency clock to generate a normalized audio time value., wherein the audio time value comprises a time stamp; a first memory means for storing a video time value upon receiving the video time value within the audio/video encoded data stream, wherein the video time value also comprises a time stamp; a second memory means for storing a threshold value; and an arithmetic means selectively coupled to the first memory means, the second memory means, and the counter, for selectively adding or subtracting the threshold value and the video time value to generate a first result and selectively adding or subtracting the normalized audio time value and the first result to generate a second result, wherein the second result indicates that the video display lags the audio playback by more than the threshold value; and a synchronizer for causing M compressed video data frames to be skipped in order to substantially synchronize a decompressed video data frame with a decompressed audio data packet.

Claim 45

The audio/video synchronization circuit of claim 39 wherein the audio time value and video time value have selective bits truncated, such that the audio time value and video time value are N bits wide to represent a narrower range of values, the frequency divider is further responsive to the selected bits of the audio time value and video time value being truncated, and the counter and arithmetic means are N+1 bits wide.

Claim 46

The audio/video synchronization circuit of claim 39 wherein the type of compressed video data frames that are caused to be skipped are MPEG encoded video frames within the audio/video encoded data stream.

Claim 47

The audio/video synchronization circuit of claim 39 wherein the type of compressed video data frames that are skipped is a B-type MPEG encoded video frame or a P-type MPEG encoded video frame which immediately precedes an I-type MPEG encoded video frame within the audio/video encoded data stream.

Claim 48

The audio/video synchronization circuit of claim 39 wherein the type of compressed video data frames that are desirable to skip is a B-type MPEG encoded video frame or a P-type MPEG encoded video frame which immediately precedes an I-type MPEG encoded video frame within the audio/video encoded data stream, and when the desirable compressed video data frames that are desirable to skip are not contained within the audio/video encoded data stream for a predetermined period of time, the audio/video synchronization circuit generates an interrupt.

Claim 49

The audio/video synchronization circuit of claim 39 wherein the second result indicates that the video display lags the audio playback by more than the threshold value, the audio video synchronization circuit signals to skip M compressed video data frames of a B-type MPEG encoded video frame or a P-type MPEG encoded video frame which immediately precedes an I-type MPEG encoded video frame within the audio/video encoded data stream to substantially synchronize a decompressed video data frame with a decompressed audio data packet, and when B-type MPEG encoded video frames and P-type MPEG encoded video frames which immediately precede an I-type MPEG encoded video frame to be skipped are unavailable for a predetermined period of time within the audio/video encoded data stream, the audio video synchronization circuit causes a delay of D compressed audio data packets.

Claim 50

The audio/video synchronization circuit of claim 39 wherein the threshold is three frames of a video frame display time and M is three.

Claim 51

The audio/video synchronization circuit of claim 39 wherein the threshold is one frame of a video frame display time and M is one.

Claim 52

The audio/video synchronization circuit of claim 39 wherein the threshold is one half of a video frame display time and M is one.

Claim 53

The audio/video synchronization circuit of claim 39 wherein the threshold is one half of a video frame display time and M is one half.

Claim 54

The audio/video synchronization circuit of claim 39 wherein the second result indicates that the video display leads the audio playback.

Claim 55

The audio/video synchronization circuit of claim 49 wherein the second result indicates that the video display leads the audio playback by more than the threshold value, and the audio video synchronization circuit causes N decompressed video data frames to be repeated in order to substantially synchronize a decompressed video data frame with a decompressed audio data packet.

Claim 56

The audio/video synchronization circuit of claim 50 wherein the threshold is three frames of a video frame display time and N is three.

Claim 57

The audio/video synchronization circuit of claim 50 wherein the threshold is one frame of a video frame display time and N is one.

Claim 58

The audio/video synchronization circuit of claim 50 wherein the threshold is one half of a video frame display time and N is one.

Claim 59

The audio/video synchronization circuit of claim 50 wherein the threshold is one half of a video frame display time and N is one half.

Claim 60

An audio/video synchronization circuit for substantially synchronizing video display images having a plurality of compressed and decompressed video data frames with audio playback having a plurality of compressed and decompressed audio data packets, the audio/video synchronization circuit comprising: a frequency divider for selectively dividing the frequency of a clock in response to a display frame rate to generate a divided frequency clock; a counter, coupled to the frequency divider, initialized to a video time value upon receiving the video time value within an audio/video encoded data stream, and incremented by the divided frequency clock to generate a normalized video time value, wherein the video time value comprises a time stamp; a first memory means for storing an audio time value upon receiving the audio time value within the audio/video encoded data stream, wherein the audio time value also comprises a time stamp; a second memory means for storing a threshold value; and an arithmetic means selectively coupled to the first memory means, the second memory means, and the counter, for selectively adding or subtracting the threshold value and the audio time value to generate a first result and selectively adding or subtracting the normalized video time value and the first result to generate a second result, wherein the second result indicates that the video display lags the audio playback by more than the threshold value; and a synchronization circuit for causing M compressed video data frames to be skipped in order to substantially synchronize a decompressed video data frame with a decompressed audio data packet.

Claim 61

An audio/video synchronization circuit for substantially synchronizing video display images having a plurality of compressed and decompressed video data frames with audio playback having a plurality of compressed and decompressed audio data packets, the audio/video synchronization circuit comprising: a frequency divider for selectively dividing the frequency of a clock in response to a display frame rate to generate a divided frequency clock; a counter, coupled to the frequency divider, initialized to a video time value upon receiving the video time value within an audio/video encoded data stream, and incremented by the divided frequency clock to generate a normalized video time value, wherein the video time value comprises a time stamp; a first memory means for storing an audio time value upon receiving the audio time value within the audio/video encoded data stream, wherein the audio time value also comprises a time stamp; a second memory means for storing a threshold value; and an arithmetic means selectively coupled to the first memory means, the second memory means, and the counter, for selectively adding or subtracting the threshold value and the audio time value to generate a first result and selectively adding or subtracting the normalized video time value and the first result to generate a second result, wherein the second result indicates that the video display lags the audio playback by more than the threshold value; and a synchronization circuit for causing M compressed video data frames to be skipped in order to substantially synchronize a decompressed video data frame with a decompressed audio data packet.

Claim 62

The audio/video synchronization circuit of claim 55 wherein the audio time value and video time value have selective bits truncated, such that the audio time value and video time value are N bits wide to represent a narrower range of values, the frequency divider is further responsive to the selected bits of the audio time value and video time value being truncated, and the counter and arithmetic means are N+1 bits wide.

Claim 63

The audio/video synchronization circuit of claim 55 wherein the threshold is three frames of a video frame display time and M is three.

Claim 64

The audio/video synchronization circuit of claim 55 wherein the threshold is one frame of a video frame display time and M is one.

Claim 65

The audio/video synchronization circuit of claim 55 wherein the threshold is one half of a video frame display time and M is one.

Claim 66

The audio/video synchronization circuit of claim 55 wherein the threshold is one half of a video frame display time and M is one half.

Claim 67

The audio/video synchronization circuit of claim 55 wherein the second result indicates that the video display leads the audio playback.

Claim 68

The audio/video synchronization circuit of claim 61 wherein the second result indicates that the video display leads the audio playback by more than the threshold value, and the audio video synchronization circuit causes N decompressed video data frames to be repeated in order to substantially synchronize a decompressed video data frame with a decompressed audio data packet.

Claim 69

The audio/video synchronization circuit of claim 62 wherein the threshold is three frames of a video frame display time and N is three.

Claim 70

The audio/video synchronization circuit of claim 62 wherein the threshold is one frame of a video frame display time and N is one.

Claim 71

The audio/video synchronization circuit of claim 62 wherein the threshold is one half of a video frame display time and N is one.

Claim 72

The audio/video synchronization circuit of claim 62 wherein the threshold is one half of a video frame display time and N is one half.

Claim 73

A multimedia system for reading, decoding, and decompressing encoded audio/video data into video data for display on a plurality of display types and into audio data for reproduction on a plurality of sound reproduction devices, the multimedia system comprising: a reader, receiving the encoded audio/video data, for reading the encoded audio/video data and generating a first encoded audio/video bit stream; a first memory for storing the first encoded audio/video bit stream; a processor, coupled to the reader, the first memory, and a system bus, for receiving the first encoded audio/video bit stream and writing first encoded audio/video bit stream into and a first memory, and reading the first encoded audio/video bit stream from the first memory being stored therein and outputing the first encoded audio/video bit stream onto the bus; and an audio/video decoding circuit, coupled to the system bus, for receiving the first encoded audio/video bit stream and for decoding and decompressing the encoded audio/video data into video data and audio data, the audio/video decoding circuit comprising: a parser for separating the encoded data stream into encoded audio data, encoded video data, an audio time value, and a video time value, the audio time value and the video time value comprising a time stamp; a second memory having addressable storage, coupled to the parser, coupled to the parser, addressing addressable storage to selectively store and read the encoded audio data, the encoded video data, or decoded video data to and from addressable storage; a first decoder, coupled to the second memory, for decoding and decompressing encoded video data and generating decompressed video data in response to the video time value; a synchronizing circuit, coupled to the first decoder and the second memory, for comparing the video time value with the audio time value and when the comparison exceeds a predetermined threshold, substantially synchronizing the generation of the decompressed video data by the first decoder with the addressing of addressable storage by the second memory; and a second decoder, coupled to the second memory, for decoding and decompressing encoded audio data to generate decompressed audio data.

Claim 74

A multimedia system for reading, decoding, and decompressing encoded audio/video data into video data for display on a plurality of display types and into audio data for reproduction on a plurality of sound reproduction devices, the multimedia system comprising: a reader, receiving the encoded audio/video data, for reading the encoded audio/video data and generating a first encoded audio/video bit stream; a first memory for storing the first encoded audio/video bit stream; a processor, coupled to the reader, the first memory, and a system bus, for receiving the first encoded audio/video bit stream and writing first encoded audio/video bit stream into and a first memory, and reading the first encoded audio/video bit stream from the first memory being stored therein and outputing the first encoded audio/video bit stream onto the bus; and an audio/video decoding circuit, coupled to the system bus, for receiving the first encoded audio/video bit stream and for decoding and decompressing the encoded audio/video data into video data and audio data, the audio/video decoding circuit comprising: a parser for separating the encoded data stream into encoded audio data, encoded video data, an audio time value, and a video time value, the audio time value and the video time value comprising a time stamp; a second memory having addressable storage, coupled to the parser, coupled to the parser, addressing addressable storage to selectively store and read the encoded audio data, the encoded video data, or decoded video data to and from addressable storage; a first decoder, coupled to the second memory, for decoding and decompressing encoded video data and generating decompressed video data in response to the video time value; a synchronizing circuit, coupled to the first decoder and the second memory, for comparing the video time value with the audio time value and when the comparison exceeds a predetermined threshold, substantially synchronizing the generation of the decompressed video data by the first decoder with the addressing of addressable storage by the second memory; and a second decoder, coupled to the second memory, for decoding and decompressing encoded audio data to generate decompressed audio data.

Claim 75

The multimedia system of claim 67 wherein the synchronizer determines that the video time value exceeds the audio time value by the predetermined threshold, and a sequence of addressing addressable storage containing decoded/decompressed video data by the memory is modified by repeating a sequence of addresses to substantially synchronize the generation of the decoded/decompressed video data by the first decoder with the addressing of addressable storage by the memory.

Claim 76

The multimedia system of claim 67 wherein the synchronizer determines that the audio time value exceeds the video time value by the predetermined threshold, and a sequence of addressing addressable storage containing encoded video data by the memory is modified by skipping a sequence of addresses to substantially synchronize the generation of the decoded/decompressed video data by the first decoder with the addressing of addressable storage by the memory.

Claim 77

The multimedia system of claim 67 further comprising a graphics controller, coupled to the first decoder and a video monitor, for displaying the decoded/decompressed video data as an image on the video monitor.

Claim 78

The multimedia system of claim 67 further comprising a graphics controller, coupled to the first decoder and a digital display, for displaying the decoded/decompressed video data as an image on the digital display.

Claim 79

The multimedia system of claim 67 further comprising a television signal encoder, coupled to the first decoder and a television display, for generating television data from the decoded/decompressed video data and displaying the television data as an image on the television display.

Claim 80

The multimedia system of claim 67 further comprising a digital to analog converter, coupled to the second decoder for generating an analog audio signal from the decoded/decompressed audio data.

Claim 81

The multimedia system of claim 67 further comprising an audio amplifier, coupled to the digital to analog converter, for amplifying the magnitude of the analog audio signal.