Significant scene detection and frame filtering for a visual indexing system

Claim 1

A system for detecting significant scenes and selecting keyframes of source video, comprising: a receiver for receiving source video as macroblocks including blocks of DCT coefficients and frames; a summer calculating a sum for each type of block within a macroblock based on: ##EQU6## where: k is the number of macroblocks in width of a frame, k=1 to Frame-width/16, j is the number of macroblocks in height of a frame, j=1 to Frame-height/16, I is the number of blocks in a macroblock, I=1 to number of blocks in macroblock, DCT1 k ,j and DCT2 k ,j are DCT coefficients for the specified macroblock for a previous and a current frame, respectively, and ABS is an absolute value function; a threshold calculator calculating a first threshold and a second threshold; a first comparator for comparing each SUM[i] to the first and second thresholds and saving the current frame as a keyframe in a frame memory if SUM[i] is greater than the first threshold, incrementing a static scene counter if less than the second and saving the previous frame in a temporary memory, and resetting the static scene counter otherwise; and a second comparator for comparing the static scene counter to a predetermined number and transferring the most previous video frame saved in the temporary memory to the frame memory as a keyframe.

Claim 2

A system for detecting significant scenes and selecting keyframes of source video, comprising: a receiver for receiving source video as macroblocks including blocks of DCT coefficients and frames; a summer calculating a sum for each type of block within a macroblock based on: ##EQU6## where: k is the number of macroblocks in width of a frame, k=1 to Frame-width/16, j is the number of macroblocks in height of a frame, j=1 to Frame-height/16, I is the number of blocks in a macroblock, I=1 to number of blocks in macroblock, DCT1 k ,j and DCT2 k ,j are DCT coefficients for the specified macroblock for a previous and a current frame, respectively, and ABS is an absolute value function; a threshold calculator calculating a first threshold and a second threshold; a first comparator for comparing each SUM[i] to the first and second thresholds and saving the current frame as a keyframe in a frame memory if SUM[i] is greater than the first threshold, incrementing a static scene counter if less than the second and saving the previous frame in a temporary memory, and resetting the static scene counter otherwise; and a second comparator for comparing the static scene counter to a predetermined number and transferring the most previous video frame saved in the temporary memory to the frame memory as a keyframe.

Claim 4

A system for detecting significant scenes and selecting keyframes of source video, comprising: a receiver for receiving source video as macroblocks including blocks of DCT coefficients and frames; a summer calculating a sum for each macroblock based on: ##EQU9## k is the number of macroblocks in width of a frame, k=1 to Frame-width/16, j is the number of macroblocks in height of a frame, j=1 to Frame-height/16, I is the number of blocks in a macroblock, I=1 to number of blocks in macroblock, DCT1 k ,j,i and DCT2 k ,j,i are DCT coefficients for the specified macroblock and block for a previous and a current video frame, respectively, and ABS is an absolute value function; a threshold calculator calculating a first threshold and a second threshold; a first comparator for comparing each SUM to the first and second thresholds and saving the current frame as a keyframe in a frame memory if SUM is greater than the first threshold, incrementing a static scene counter if less than the second and saving the previous frame in a temporary memory, and resetting the static scene counter otherwise; and a second comparator for comparing the static scene counter to a predetermined number and transferring the most previous video frame saved in the temporary memory to the frame memory as a keyframe.

Claim 5

A system for detecting significant scenes and selecting keyframes of source video, comprising: a receiver for receiving source video as macroblocks including blocks of DCT coefficients and frames; a summer calculating a sum for each macroblock based on: ##EQU9## k is the number of macroblocks in width of a frame, k=1 to Frame-width/16, j is the number of macroblocks in height of a frame, j=1 to Frame-height/16, I is the number of blocks in a macroblock, I=1 to number of blocks in macroblock, DCT1 k ,j,i and DCT2 k ,j,i are DCT coefficients for the specified macroblock and block for a previous and a current video frame, respectively, and ABS is an absolute value function; a threshold calculator calculating a first threshold and a second threshold; a first comparator for comparing each SUM to the first and second thresholds and saving the current frame as a keyframe in a frame memory if SUM is greater than the first threshold, incrementing a static scene counter if less than the second and saving the previous frame in a temporary memory, and resetting the static scene counter otherwise; and a second comparator for comparing the static scene counter to a predetermined number and transferring the most previous video frame saved in the temporary memory to the frame memory as a keyframe.

Claim 7

A system for detecting significant scenes and selecting keyframes of source video, comprising: a receiver for receiving source video as macroblocks including blocks of DCT coefficients and frames; a summer calculating a sum for each type of block within a macroblock based on: ##EQU12## where: k is the number of macroblocks in width of a frame, k=1 to Frame-width/16, j is the number of macroblocks in height of a frame, j=1 to Frame-height/16, I is the number of blocks in a macroblock, I=1 to number of blocks in macroblock, DCT1 k ,j and DCT2 k ,j are DCT coefficients for the specified macroblock for a previous and a current frame, respectively, and ABS is an absolute value function; a threshold calculator calculating a first threshold and a second threshold; a first comparator for comparing each SUM[i] to the first and second thresholds and saving the current frame as a keyframe in a frame memory if SUM[i] is greater than the first threshold, incrementing a static scene counter if less than the second and saving the previous frame in a temporary memory, and resetting the static scene counter otherwise; and a second comparator for comparing the static scene counter to a predetermined number and transferring the most previous video frame saved in the temporary memory to the frame memory as a keyframe.

Claim 8

A system for detecting significant scenes and selecting keyframes of source video, comprising: a receiver for receiving source video as macroblocks including blocks of DCT coefficients and frames; a summer calculating a sum for each type of block within a macroblock based on: ##EQU12## where: k is the number of macroblocks in width of a frame, k=1 to Frame-width/16, j is the number of macroblocks in height of a frame, j=1 to Frame-height/16, I is the number of blocks in a macroblock, I=1 to number of blocks in macroblock, DCT1 k ,j and DCT2 k ,j are DCT coefficients for the specified macroblock for a previous and a current frame, respectively, and ABS is an absolute value function; a threshold calculator calculating a first threshold and a second threshold; a first comparator for comparing each SUM[i] to the first and second thresholds and saving the current frame as a keyframe in a frame memory if SUM[i] is greater than the first threshold, incrementing a static scene counter if less than the second and saving the previous frame in a temporary memory, and resetting the static scene counter otherwise; and a second comparator for comparing the static scene counter to a predetermined number and transferring the most previous video frame saved in the temporary memory to the frame memory as a keyframe.

Claim 10

A system for detecting significant scenes and selecting keyframes of source video, comprising: a receiver for receiving source video as macroblocks including blocks of DCT coefficients and frames; a summer calculating a sum for each type of block within a macroblock based on: ##EQU15## where: k is the number of macroblocks in width of a frame, k=1 to Frame-width/16, j is the number of macroblocks in height of a frame, j=1 to Frame-height/16, I is the number of blocks in a macroblock, I=1 to number of blocks in macroblock, MB1k,j and MB2 k ,j macroblocks for a previous and a current video frame, respectively, and Mbdiff is a function that determines the number of blocks which are different from each other between two macroblocks, and outputs a first value if this difference is higher than a certain threshold and a second value otherwise; said summer comprising: a subsum calculator calculating a subsum for each block, where subsum[i] j ,k =ABS(DCT1[i] j ,k -DCT2[i] j ,k) and DCT1 k ,j and DCT2 k ,j are DCT coefficients for the specified macroblock; a subsum comparator comparing each subsum to a first predetermined subsum threshold and if greater than the first predetermined subsum threshold, incrementing a first block counter and comparing each subsum to a second predetermined subsum threshold and if less than the second predetermined subsum threshold, incrementing a second block counter; a block comparator comparing the first block counter and if above a first predetermined block threshold, incrementing a first macroblock counter and resetting the first block counter for analysis of a next macroblock, and comparing the second block counter and if above a second predetermined block threshold, incrementing a second macroblock counter and resetting the second block counter for analysis of a next macroblock; a macroblock comparator for comparing the first macroblock counter and if the first macroblock counter is above a first predetermined frame threshold, saving the current frame in frame memory and resetting the first macroblock counter for analysis of a next frame, and comparing the second macroblock counter and if the second macroblock counter is above a second predetermined frame threshold, setting a frame counter and resetting the second macroblock counter; and a frame comparator comparing the frame counter to a predetermined scene threshold and if greater than the predetermined threshold, saving the previous frame in frame memory as a keyframe.

Claim 11

A system for detecting significant scenes and selecting keyframes of source video, comprising: a receiver for receiving source video as macroblocks including blocks of DCT coefficients and frames; a summer calculating a sum for each type of block within a macroblock based on: ##EQU15## where: k is the number of macroblocks in width of a frame, k=1 to Frame-width/16, j is the number of macroblocks in height of a frame, j=1 to Frame-height/16, I is the number of blocks in a macroblock, I=1 to number of blocks in macroblock, MB1k,j and MB2 k ,j macroblocks for a previous and a current video frame, respectively, and Mbdiff is a function that determines the number of blocks which are different from each other between two macroblocks, and outputs a first value if this difference is higher than a certain threshold and a second value otherwise; said summer comprising: a subsum calculator calculating a subsum for each block, where subsum[i] j ,k =ABS(DCT1[i] j ,k -DCT2[i] j ,k) and DCT1 k ,j and DCT2 k ,j are DCT coefficients for the specified macroblock; a subsum comparator comparing each subsum to a first predetermined subsum threshold and if greater than the first predetermined subsum threshold, incrementing a first block counter and comparing each subsum to a second predetermined subsum threshold and if less than the second predetermined subsum threshold, incrementing a second block counter; a block comparator comparing the first block counter and if above a first predetermined block threshold, incrementing a first macroblock counter and resetting the first block counter for analysis of a next macroblock, and comparing the second block counter and if above a second predetermined block threshold, incrementing a second macroblock counter and resetting the second block counter for analysis of a next macroblock; a macroblock comparator for comparing the first macroblock counter and if the first macroblock counter is above a first predetermined frame threshold, saving the current frame in frame memory and resetting the first macroblock counter for analysis of a next frame, and comparing the second macroblock counter and if the second macroblock counter is above a second predetermined frame threshold, setting a frame counter and resetting the second macroblock counter; and a frame comparator comparing the frame counter to a predetermined scene threshold and if greater than the predetermined threshold, saving the previous frame in frame memory as a keyframe.

Claim 13

A system for filtering frames from a video source, comprising: a receiver receiving frames in a frame memory and macroblocks comprising blocks of DCT coefficients in a temporary memory; a block signature creator for creating a block signature for each block; a region grower for detecting regions of each frame and a respective size; a centroid determiner for determining a centroid of each region; a region signature creator for assigning each region one of the block signatures of the blocks within the region, the respective size, and the centroid; a frame signature creator for creating a frame signature for a first frame, based on the region signatures; and a frame comparator for comparing the first frame to a second frame to determine a frame difference and saving or discarding the frame based on the frame difference.

Claim 14

A system for filtering frames from a video source, comprising: a receiver receiving frames in a frame memory and macroblocks comprising blocks of DCT coefficients in a temporary memory; a block signature creator for creating a block signature for each block; a region grower for detecting regions of each frame and a respective size; a centroid determiner for determining a centroid of each region; a region signature creator for assigning each region one of the block signatures of the blocks within the region, the respective size, and the centroid; a frame signature creator for creating a frame signature for a first frame, based on the region signatures; and a frame comparator for comparing the first frame to a second frame to determine a frame difference and saving or discarding the frame based on the frame difference.