Image processing for analyzing video content

Claim 2

An automated method of determining a sequence of intensity measures, each intensity measure corresponding to each of a sequence of recorded video frames, the method comprising using a programmed computer to: determine a motion activity measure for each video frame of said sequence to provide a sequence of motion measures; filter the sequence of motion activity measures; provide a sequence of intensity measures using said filtered sequence of motion activity measures; determine a measure of cut frequency to provide a sequence of cut frequency measures; filter the sequence of cut frequency measures; and combine the filtered measures to provide said sequence of intensity measures.

Claim 3

An automated method according to claim 1 , in which the recorded video frames have an associated audio track and further comprising using said computer to: determine a measure of sound energy in high frequencies for the audio track to provide a sequence of high frequency measures; filter the sequence of sound energy measures; normalize the filtered sequence of sound energy measures; and combine the normalized sequence of sound energy measures with the filtered measures to provide said sequence of intensity measures.

Claim 4

An automated method according to claim 1 , which also forms an affect measure for a sequence of recorded video frames having an associated audio track, by further comprising use of said computer to: determine a pitch measurement for the audio track to provide a sequence of pitch measures; filter the sequence of pitch measures; provide a sequence of valence measures using said filtered sequence of pitch measures; and combine the sequence of valence measures with the sequence of intensity measures to provide a sequence of affect measures.

Claim 5

An automated method of selecting frames from a sequence of recorded video frames comprising using said computer to determine a sequence of intensity measures corresponding to each of the sequence of video frames using the method according to claim 1 , and selecting frames from the sequence of video frames for which the intensity measure is greater than a predetermined threshold.

Claim 6

An automated method of selecting frames from a sequence of recorded video frames comprising using said computer to determine a sequence of intensity measures corresponding to each of the sequence of video frames using the method according to claim 1 , and in which the threshold is determined to select a predetermined number of frames.

Claim 7

An automated method according to claim 3 , in which the affect measure is generated by using said computer to: define ranges of the intensity measures and valence measures; and count for respective combinations of intensity range and valence range the number of video frames whose measures fall into the respective range combination, to produce a multi-element affect measure.

Claim 8

An automated method according to claim 6 , in which the counts are normalized by division by the number of frames in the video sequence.

Claim 9

An automated method according to claim 6 , including forming a generic affect measure by combining the affect measures obtained for a plurality of video frame sequences.

Claim 10

An automated method according to claim 6 , including the step of comparing an affect measure obtained for one sequence of video frames with the affect measure obtained for other video sequences.

Claim 11

An automated method of forming an affect measure for a sequence of recorded video frames having an associated audio track, said method comprising using a programmed computer to: determine a sequence of intensity measures using a method according to claim 2 ; determine a pitch measurement for the audio track to provide a sequence of pitch measures; filter the sequence of pitch measures; provide a sequence of valence measures using said filtered sequence of pitch measures; and combine the sequence of valence measures with the sequence of intensity measures to provide a sequence of affect measures.

Claim 12

An automated method of generating an affect measure for a sequence of recorded video frames having an accompanying sound track, said method comprising using a programmed computer to: (a) generate a sequence of intensity measures for the frames of the sequence, said intensity measures depending on the degree of visual activity in the sequence; (b) generate a sequence of valence measures for the frames of the sequence, said valence measures depending on the pitch of the sound track; (c) define ranges of the intensity measures and valence measures; and (d) count for respective combinations of intensity range and valence range the number of video frames whose measures fall into the respective range combination, to produce a multi-element affect measure.

Claim 13

An automated method of generating an affect measure for a sequence of recorded video frames having an accompanying sound track, said method comprising using a programmed computer to: (a) generate a sequence of intensity measures for the frames of the sequence, said intensity measures depending on the degree of visual activity in the sequence; (b) generate a sequence of valence measures for the frames of the sequence, said valence measures depending on the pitch of the sound track; (c) define ranges of the intensity measures and valence measures; and (d) count for respective combinations of intensity range and valence range the number of video frames whose measures fall into the respective range combination, to produce a multi-element affect measure.

Claim 14

An automated method according to claim 11 , in which the counts are normalized by division by the number of frames in the recorded video sequence.

Claim 15

An automated method according to claim 11 , including forming a generic affect measure by combining the affect measures obtained for a plurality of recorded video frame sequences.

Claim 16

An automated method according to claim 11 , including the step of comparing an affect measure obtained for one sequence of video frames with the affect measure obtained for other recorded video sequences.

Claim 17

An automated method of determining a sequence of intensity measures, each intensity measure corresponding to each of a sequence of recorded video frames having an associated audio track, the method comprising using a programmed computer to: determine a motion activity measure for each frame of said sequence to provide a sequence of motion measures; filter the sequence of motion activity measures; provide the sequence of intensity measures using said filtered sequence of motion activity measures; determine a measure of sound energy in high frequencies for the audio track to provide a sequence of high frequency measures; filter the sequence of sound energy measures; normalize the filtered sequence of sound energy measures; and combine the normalized sequence of sound energy measures with the filtered measures to provide said sequence of intensity measures.

Claim 18

An automated method of determining a sequence of intensity measures, each intensity measure corresponding to each of a sequence of recorded video frames having an associated audio track, the method comprising using a programmed computer to: determine a motion activity measure for each frame of said sequence to provide a sequence of motion measures; filter the sequence of motion activity measures; provide the sequence of intensity measures using said filtered sequence of motion activity measures; determine a measure of sound energy in high frequencies for the audio track to provide a sequence of high frequency measures; filter the sequence of sound energy measures; normalize the filtered sequence of sound energy measures; and combine the normalized sequence of sound energy measures with the filtered measures to provide said sequence of intensity measures.

Claim 19

An automated method of determining a sequence of intensity measures, each intensity measure corresponding to each of a sequence of recorded video frames having an associated audio track and then forming an affect measure for a sequence of said video frames, said method comprising using a programmed computer to: determine a motion activity measure for each frame of said sequence to provide a sequence of motion measures; filter the sequence of motion activity measures; provide the sequence of intensity measures using said filtered sequence of motion activity measures; determine a pitch measurement for the audio track to provide a sequence of pitch measures; filter the sequence of pitch measures; provide a sequence of valence measures using said filtered sequence of pitch measures; and combine the sequence of valence measures with the sequence of intensity measures to provide a sequence of affect measures.

Claim 20

An automated method of determining a sequence of intensity measures, each intensity measure corresponding to each of a sequence of recorded video frames having an associated audio track and then forming an affect measure for a sequence of said video frames, said method comprising using a programmed computer to: determine a motion activity measure for each frame of said sequence to provide a sequence of motion measures; filter the sequence of motion activity measures; provide the sequence of intensity measures using said filtered sequence of motion activity measures; determine a pitch measurement for the audio track to provide a sequence of pitch measures; filter the sequence of pitch measures; provide a sequence of valence measures using said filtered sequence of pitch measures; and combine the sequence of valence measures with the sequence of intensity measures to provide a sequence of affect measures.