Indexing video by detecting speech and music in audio

Claim 2

A video processing device, comprising: an audio event detecting means for detecting audio events in a video data; and a memory communicating with said audio event detecting means and storing video data and audio data corresponding to said video data, said memory further storing a predetermined energy threshold, a predetermined ZCR variance threshold, and a predetermined ZCR amplitude span threshold; wherein said audio event detecting means includes an energy detector, which detects an audio event in said audio data by measuring an energy content of said audio data and indexes said video data at about a beginning of said audio event and, wherein said audio event detecting means further includes a ZCR detector generating a ZCR value from said audio data and comparing a ZCR variance and a ZCR value span of said audio data to the predetermined ZCR variance threshold and to said predetermined ZCR amplitude span threshold, respectively, if said energy content satisfies said predetermined energy threshold.

Claim 3

The device of claim 1 , wherein said audio event detecting means indexes said video data by extracting and storing one or more representative video frames.

Claim 4

The device of claim 1 wherein said audio event detecting means indexes said video data by inserting index data into said video data.

Claim 5

The device of claim 1 , wherein said audio event detecting means indexes said video data by saving one or more index pointers.

Claim 6

The device of claim 1 , wherein said audio event detecting means indexes said video data by recording one or more time stamps.

Claim 7

The device of claim 1 , wherein said audio event comprises speech.

Claim 8

The device of claim 1 , wherein said audio event comprises music.

Claim 9

The device of claim 1 , wherein said video processing device comprises a video recorder device.

Claim 10

The device of claim 1 , wherein said video processing device comprises a video editor device.

Claim 11

The device of claim 1 , wherein said video processing device comprises a video authoring device.

Claim 12

A video processing device, comprising: a processor; an audio event detector communicating with said processor, wherein said audio event detector includes an energy detector communicating with said processor and a ZCR detector communicating with said processor; and a memory communicating with said processor, said memory storing video data and audio data corresponding to said video data; said memory further storing a predetermined energy threshold, a predetermined ZCR amplitude span threshold, and a predetermined ZCR variance threshold; wherein said audio event detector detects an audio event in said audio data by measuring an energy content of said audio data, generating a ZCR value from said audio data, and comparing a ZCR variance and ZCR value span of said audio data to the predetermined ZCR variance threshold and to the predetermined ZCR amplitude span threshold, respectively, if said energy content satisfies said predetermined energy threshold and wherein said processor indexes said video data at about a beginning of said audio event.

Claim 13

A video processing device, comprising: a processor; an audio event detector communicating with said processor, wherein said audio event detector includes an energy detector communicating with said processor and a ZCR detector communicating with said processor; and a memory communicating with said processor, said memory storing video data and audio data corresponding to said video data; said memory further storing a predetermined energy threshold, a predetermined ZCR amplitude span threshold, and a predetermined ZCR variance threshold; wherein said audio event detector detects an audio event in said audio data by measuring an energy content of said audio data, generating a ZCR value from said audio data, and comparing a ZCR variance and ZCR value span of said audio data to the predetermined ZCR variance threshold and to the predetermined ZCR amplitude span threshold, respectively, if said energy content satisfies said predetermined energy threshold and wherein said processor indexes said video data at about a beginning of said audio event.

Claim 14

The device of claim 11 , wherein said video processing device comprises a video recorder device.

Claim 15

The device of claim 11 , wherein said video processing device comprises a video editor device.

Claim 16

The device of claim 11 , wherein said video processing device comprises a video authoring device.

Claim 17

The device of claim 11 , wherein said processor indexes said video data by extracting and storing one or more representative video frames.

Claim 18

The device of claim 11 , wherein said processor indexes said video data by inserting index darn into said video data.

Claim 19

The device of claim 11 , wherein said processor Indexes said video data by saving one or more Index pointers.

Claim 20

The device of claim 11 , wherein said processor Indexes said video data by recording one or more time stamps.

Claim 21

The device of claim 11 , wherein said memory stores a predetermined set of speech harmonics thresholds, and wherein said audio event detector further comprises: a spectrum detector communicating with said processor and generating a frequency spectrum from said audio data; wherein said audio event detector compares harmonic frequency components in said frequency Spectrum to said predetermined set of speech harmonies thresholds and detects a speech audio event if said harmonic frequency components fall within said predetermined set of speech harmonics thresholds if said ZCR value span satisfies said predetermined ZCR amplitude spun threshold, if a variance between said ZCR value and one or more previous ZCR values satisfies the predetermined ZCR variance threshold, and if said energy content satisfies than said predetermined energy threshold.

Claim 22

The device of claim 19 , wherein said spectrum detector comprises an FFT processor.

Claim 23

The device of claim 11 , wherein said memory stores a predetermined frequency change threshold and wherein said audio event detector further comprises: a spectrum detector communicating with said processor and generating a frequency spectrum from said audio data; a peak detector communicating with said processor and said spectrum detector, said peak detector receiving said frequency spectrum, detecting frequency peaks in said frequency spectrum and generating a frequency peak output; and wherein said audio event detector compares frequency peaks in two or more frequency peak outputs and detects a music audio event if said frequency peaks in said two or more frequency peak outputs are substantially stable and if said energy content is greater than said predetermined energy threshold.

Claim 24

The device of claim 21 , wherein said spectrum detector comprises an FFT processor.

Claim 25

The device of claim 21 , wherein said music detector compares frequency peaks in two or more consecutive frequency peak outputs.

Claim 28

The method of claim 24 , with the step of detecting said audio event further comprising detecting a speech audio event in said audio data.

Claim 29

The method of claim 24 , with the step of detecting said audio event further comprising the steps of: comparing harmonic frequency components of said audio data to a predetermined set of speech harmonics thresholds if said ZCR variance and said ZCR value span satisfies said predetermined ZCR variance threshold and said predetermined ZCR amplitude span threshold, respectively; and detecting a speech audio event if said harmonic frequency components are within said predetermined speech harmonics range.

Claim 30

The method of claim 24 , with the step of detecting said audio event further comprising detecting a music audio event in said audio data.

Claim 31

The method of claim 24 , with the step of detecting said audio event further comprising the steps of: comparing frequency peaks in two or more consecutive frequency spectra if said energy content is greater than said predetermined energy threshold; and detecting a music audio event if said frequency peaks in said two or more consecutive frequency spectra are substantially stable.

Claim 32

The method of claim 24 , with the step of indexing further comprising indexing said video data at about a beginning of a semantically meaningful video scene.

Claim 33

The method of claim 24 , with the step of indexing further comprising extracting and storing said one or more representative video frames.

Claim 34

The method of claim 24 , with the step of indexing further comprising inserting index data into said video data.

Claim 35

The method of claim 24 , with the step of indexing further comprising saving one or more index pointers.

Claim 36

The method of claim 24 , with the step of indexing further comprising storing one or more time stamps.