Patent US6751354B2 successfully refreshed from Google Patents.
US6751354B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Methods and apparatuses for video segmentation, classification, and retrieval using image class statistical models
Assignee: Fujifilm Business Innovation Corp
•
Filing: 1999-03-11
•
Granted: 2004-06-15
•
Expires: 2019-03-11
Abstract
Abstract Techniques for classifying video frames using statistical models of transform coefficients are disclosed. After optionally being decimated in time and space, image frames are transformed using a discrete cosine transform or Hadamard transform. The methods disclosed model image composition and operate on grayscale images. The resulting transform matrices are reduced using truncation, principal component analysis, or linear discriminant analysis to produce feature vectors. Feature vectors of training images for image classes are used to compute image class statistical models. Once image class statistical models are derived, individual frames are classified by the maximum likelihood resulting from the image class statistical models. Thus, the probabilities that a feature vector derived from a frame would be produced from each of the image class statistical models are computed. The frame is classified into the image class corresponding to the image class statistical model which produced the highest probability for the feature vector derived from the frame. Optionally, frame sequence information is taken into account by applying a hidden Markov model to represent image class transitions from the previous frame to the current frame. After computing all class probabilities for all frames in the video or sequence of frames using the image class statistical models and the image class transition probabilities, the final class is selected as having the maximum likelihood. Previous frames are selected in reverse order based upon their likelihood given determined current states.
Inventors
- 👤 Jonathan T. Foote
- 👤 Lynn Wilcox
- 👤 Andreas Girgensohn
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 16:56
CPC Classifications
G; G06; G06V; G06V20/00; G06V20/40; G06V20/48; G06F; G06F18/00; G06F18/20; G06F18/24; G06F18/241; G06F18/2415; G06F18/29; G06F18/295; G06V10/00; G06V10/70; G06V10/764; G06V10/84; G06V10/85
Claims Analysis
Total Claims: 63
Claim 1
Independent Claim
114 words
A method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; and selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices, the number of d coefficient positions being less than the number of total coefficient positions in the v rows and h columns.
Claim 2
Independent Claim
114 words
A method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; and selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices, the number of d coefficient positions being less than the number of total coefficient positions in the v rows and h columns.
Claim 3
Dependent Claim
16 words
A method as in claim 1 , wherein the d coefficient positions are selected by truncation.
Claim 4
Dependent Claim
18 words
A method as in claim 1 , wherein the d coefficient positions are selected by principal component analysis.
Claim 5
Dependent Claim
18 words
A method as in claim 1 , wherein the d coefficient positions are selected by linear discriminant analysis.
Claim 6
Dependent Claim
15 words
A method as in claim 1 , wherein the transform comprises a discrete cosine transform.
Claim 7
Dependent Claim
14 words
A method as in claim 1 , wherein the transform comprises a Hadamard transform.
Claim 8
Independent Claim
162 words
A method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices; computing a variance matrix having v rows and h columns of variance positions, each variance position having a variance associated with it, where each variance is computed from t transform coefficients, the t transform coefficients being in a corresponding coefficient position of the t transform matrices; and wherein the selecting-d-coefficient-positions step includes the step of: selecting d coefficient positions corresponding to variance positions of the variance matrix having d highest variances.
Claim 9
Independent Claim
162 words
A method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices; computing a variance matrix having v rows and h columns of variance positions, each variance position having a variance associated with it, where each variance is computed from t transform coefficients, the t transform coefficients being in a corresponding coefficient position of the t transform matrices; and wherein the selecting-d-coefficient-positions step includes the step of: selecting d coefficient positions corresponding to variance positions of the variance matrix having d highest variances.
Claim 10
Independent Claim
166 words
A method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; and selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices; computing a mean matrix having v rows and h columns of mean positions, each mean position having a mean associated with it, where each mean is computed from the t transform coefficients, the t transform coefficients being in a corresponding coefficient position of the t transform matrices; and wherein the selecting-d-coefficient-positions step includes the step of: selecting the d coefficient positions corresponding to mean positions of the mean matrix having d highest mean magnitudes.
Claim 11
Independent Claim
166 words
A method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; and selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices; computing a mean matrix having v rows and h columns of mean positions, each mean position having a mean associated with it, where each mean is computed from the t transform coefficients, the t transform coefficients being in a corresponding coefficient position of the t transform matrices; and wherein the selecting-d-coefficient-positions step includes the step of: selecting the d coefficient positions corresponding to mean positions of the mean matrix having d highest mean magnitudes.
Claim 12
Independent Claim
159 words
A method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; and selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices; computing a covariance matrix having v×h rows and v×h columns of covariance positions, each covariance position having a covariance associated with it, where each covariance is computed from t transform coefficients, the t transform coefficients being in a corresponding coefficient position of the t transform matrices; and wherein the selecting-d-coefficient-positions step includes the step of: selecting d coefficient positions corresponding to covariance positions of the covariance matrix.
Claim 13
Independent Claim
159 words
A method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; and selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices; computing a covariance matrix having v×h rows and v×h columns of covariance positions, each covariance position having a covariance associated with it, where each covariance is computed from t transform coefficients, the t transform coefficients being in a corresponding coefficient position of the t transform matrices; and wherein the selecting-d-coefficient-positions step includes the step of: selecting d coefficient positions corresponding to covariance positions of the covariance matrix.
Claim 14
Independent Claim
81 words
A method of generating a video image class statistical model, the method comprising the steps of: determining a first d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and determining a first d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 15
Independent Claim
81 words
A method of generating a video image class statistical model, the method comprising the steps of: determining a first d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and determining a first d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 16
Independent Claim
142 words
A method of generating a video image class statistical model, the method comprising the steps of: determining a first d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and determining a first d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions; determining a second d-entry mean vector; determining a second d-entry variance feature vector; and determining a mixture weight for combining the first and second d-entry mean vectors; wherein the video image class statistical model is a Gaussian mixture; wherein the first and second d-entry mean vectors and the first and second d-entry variance feature vectors are computed using an expectation-maximization algorithm.
Claim 17
Independent Claim
142 words
A method of generating a video image class statistical model, the method comprising the steps of: determining a first d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and determining a first d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions; determining a second d-entry mean vector; determining a second d-entry variance feature vector; and determining a mixture weight for combining the first and second d-entry mean vectors; wherein the video image class statistical model is a Gaussian mixture; wherein the first and second d-entry mean vectors and the first and second d-entry variance feature vectors are computed using an expectation-maximization algorithm.
Claim 18
Independent Claim
95 words
A method of generating s video image class statistical models, the method comprising the steps of: for each of the s video image classes, determining a d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and for each of the s video image classes, determining a d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 19
Independent Claim
95 words
A method of generating s video image class statistical models, the method comprising the steps of: for each of the s video image classes, determining a d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and for each of the s video image classes, determining a d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 20
Dependent Claim
22 words
A method as in claim 12 , wherein each of the s d-entry mean vector refers to identical d transform coefficient positions.
Claim 21
Dependent Claim
17 words
A method as in claim 12 , wherein the image class statistical model is a Gaussian distribution.
Claim 22
Dependent Claim
80 words
A method as in claim 12 , further comprising the step of: determining a class transition probability matrix having s rows and s columns of class transition probability positions, each class transition probability position having a class transition probability associated with it, wherein an ith-row-jth-column class transition probability position represents a probability of transferring from an ith video image class to a jth video image class during a frame transition from a previous video frame to a current video frame.
Claim 23
Dependent Claim
108 words
A method for classifying a video frame into one of s video image classes, the method comprising the steps of: retrieving a d-entry feature vector corresponding to the video frame; for each of the s video image classes, using the d-entry feature vector to compute an image class probability of the d-entry feature vector being produced by a corresponding one of s image class statistical models; and classifying the video frame into the video image class corresponding to a maximum image class probability. 17 .A method as in claim 16 , further comprising the step of: prior to the retrieving step, performing a transform on the video frame.
Claim 24
Dependent Claim
25 words
A method as in claim 16 , wherein the step of computing an image class probability is performed by applying a Gaussian probability distribution function.
Claim 25
Dependent Claim
26 words
A method as in claim 16 , wherein the step of computing an image class probability is performed by applying a Gaussian mixture probability distribution function.
Claim 26
Independent Claim
108 words
A method for classifying a video frame into one of s video image classes, the method comprising the steps of: retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector; for each of the s video image classes, generating a previous image class pointer corresponding to a maximum product of a previous image class probability and class transition probability.
Claim 27
Independent Claim
108 words
A method for classifying a video frame into one of s video image classes, the method comprising the steps of: retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector; for each of the s video image classes, generating a previous image class pointer corresponding to a maximum product of a previous image class probability and class transition probability.
Claim 28
Dependent Claim
28 words
A method as in claim 20 , further comprising the step of: classifying the video frame into the video image class corresponding to a maximum image class probability.
Claim 29
Dependent Claim
28 words
A method as in claim 21 , further comprising the step of: classifying a previous frame into the video image class indicated by the previous image class pointer.
Claim 30
Independent Claim
167 words
A method for segmenting a series of video frames into one of s video image classes, the method comprising the steps of: for each of the video frames in the series, retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector; and generating a previous image class pointer corresponding to a maximum product of a previous image class probability and class transition probability; for a last video frame in the series, classifying the video frame into the video image class corresponding to a maximum image class probability; and for each of the video frames except the last video frame in the series, classifying a previous frame into the video image class indicated by the previous image class pointer.
Claim 31
Independent Claim
167 words
A method for segmenting a series of video frames into one of s video image classes, the method comprising the steps of: for each of the video frames in the series, retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector; and generating a previous image class pointer corresponding to a maximum product of a previous image class probability and class transition probability; for a last video frame in the series, classifying the video frame into the video image class corresponding to a maximum image class probability; and for each of the video frames except the last video frame in the series, classifying a previous frame into the video image class indicated by the previous image class pointer.
Claim 32
Independent Claim
81 words
A method of determining similarity of a video frame using an image class statistical model, comprising the steps of: retrieving a feature vector corresponding to the video frame; retrieving a mean vector of the image class statistical model; and subtracting the mean vector from the feature vector to produce a difference vector; classifying the video frame as similar if a magnitude of the difference vector is less than a predetermined multiple of a standard deviation of the image class statistical model.
Claim 33
Independent Claim
81 words
A method of determining similarity of a video frame using an image class statistical model, comprising the steps of: retrieving a feature vector corresponding to the video frame; retrieving a mean vector of the image class statistical model; and subtracting the mean vector from the feature vector to produce a difference vector; classifying the video frame as similar if a magnitude of the difference vector is less than a predetermined multiple of a standard deviation of the image class statistical model.
Claim 34
Dependent Claim
33 words
A method as in claim 24 , further comprising the step of: computing an image class probability of the video frame being produced by an image class statistical model using the feature vector.
Claim 35
Dependent Claim
34 words
A method as in claim 25 , comprising the steps of: computing a logarithm of the image class probability; and displaying the logarithm of the image class probability in relation to the video frame.
Claim 36
Dependent Claim
120 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; and selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices.
Claim 37
Dependent Claim
120 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; and selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices.
Claim 38
Dependent Claim
109 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of generating a video image class statistical model, the method comprising the steps of: determining a first d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and determining a first d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 39
Dependent Claim
109 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of generating a video image class statistical model, the method comprising the steps of: determining a first d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and determining a first d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 40
Dependent Claim
123 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of generating s video image class statistical models, the method comprising the steps of: for each of the s video image classes, determining a d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and for each of the s video image classes, determining a d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 41
Dependent Claim
123 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of generating s video image class statistical models, the method comprising the steps of: for each of the s video image classes, determining a d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and for each of the s video image classes, determining a d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 42
Dependent Claim
111 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method for classifying a video frame into one of s video image classes, the method comprising the steps of: retrieving a d-entry feature vector corresponding to the video frame; for each of the s video image classes, using the d-entry feature vector to compute an image class probability of the d-entry feature vector being produced by a corresponding one of s image class statistical models; and classifying the video frame into the video image class corresponding to a maximum image class probability.
Claim 43
Dependent Claim
111 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method for classifying a video frame into one of s video image classes, the method comprising the steps of: retrieving a d-entry feature vector corresponding to the video frame; for each of the s video image classes, using the d-entry feature vector to compute an image class probability of the d-entry feature vector being produced by a corresponding one of s image class statistical models; and classifying the video frame into the video image class corresponding to a maximum image class probability.
Claim 44
Dependent Claim
107 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method for classifying a video frame into one of s video image classes, the method comprising the steps of: retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector.
Claim 45
Dependent Claim
107 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method for classifying a video frame into one of s video image classes, the method comprising the steps of: retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector.
Claim 46
Dependent Claim
195 words
A computer readable storage medium, comprising computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method for segmenting a series of video frames into one of s video image classes, the method comprising the steps of: for each of the video frames in the series, retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector; and generating a previous image class pointer corresponding to a maximum product of a previous image class probability and class transition probability; for a last video frame in the series, classifying the video frame into the video image class corresponding to a maximum image class probability; and for each of the video frames except the last video frame in the series, classifying a previous frame into the video image class indicated by the previous image class pointer.
Claim 47
Dependent Claim
195 words
A computer readable storage medium, comprising computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method for segmenting a series of video frames into one of s video image classes, the method comprising the steps of: for each of the video frames in the series, retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector; and generating a previous image class pointer corresponding to a maximum product of a previous image class probability and class transition probability; for a last video frame in the series, classifying the video frame into the video image class corresponding to a maximum image class probability; and for each of the video frames except the last video frame in the series, classifying a previous frame into the video image class indicated by the previous image class pointer.
Claim 48
Dependent Claim
109 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of determining similarity of a video frame using an image class statistical model, comprising the steps of: retrieving a feature vector corresponding to the video frame; retrieving a mean vector of the image class statistical model; and subtracting the mean vector from the feature vector to produce a difference vector; classifying the video frame as similar if a magnitude of the difference vector is less than a predetermined multiple of a standard deviation of the image class statistical model.
Claim 49
Dependent Claim
109 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of determining similarity of a video frame using an image class statistical model, comprising the steps of: retrieving a feature vector corresponding to the video frame; retrieving a mean vector of the image class statistical model; and subtracting the mean vector from the feature vector to produce a difference vector; classifying the video frame as similar if a magnitude of the difference vector is less than a predetermined multiple of a standard deviation of the image class statistical model.
Claim 50
Dependent Claim
128 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; and selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices.
Claim 51
Dependent Claim
128 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method for selecting a d-entry feature set for video classification from t training images, each of the t training images having v rows and h columns of sub-images, the method comprising the steps of: computing t transform matrices by performing a transform on each of the t training images, each transform matrix having v rows and h columns of coefficient positions, each coefficient position having a transform coefficient associated with it; and selecting d coefficient positions as the d-entry feature set based upon the transform coefficients in the t transform matrices.
Claim 52
Dependent Claim
117 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of generating a video image class statistical model, the method comprising the steps of: determining a first d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and determining a first d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 53
Dependent Claim
117 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of generating a video image class statistical model, the method comprising the steps of: determining a first d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and determining a first d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 54
Dependent Claim
131 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of generating s video image class statistical models, the method comprising the steps of: for each of the s video image classes, determining a d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and for each of the s video image classes, determining a d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 55
Dependent Claim
131 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of generating s video image class statistical models, the method comprising the steps of: for each of the s video image classes, determining a d-entry mean vector having d mean positions, each mean position having a mean associated with it, wherein each mean position corresponds to a transform coefficient position in a transform matrix; and for each of the s video image classes, determining a d-entry variance feature vector having d variance positions, each variance position having a variance associated with it, wherein each variance position corresponds to one of the mean positions.
Claim 56
Dependent Claim
119 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method for classifying a video frame into one of s video image classes, the method comprising the steps of: retrieving a d-entry feature vector corresponding to the video frame; for each of the s video image classes, using the d-entry feature vector to compute an image class probability of the d-entry feature vector being produced by a corresponding one of s image class statistical models; and classifying the video frame into the video image class corresponding to a maximum image class probability.
Claim 57
Dependent Claim
119 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method for classifying a video frame into one of s video image classes, the method comprising the steps of: retrieving a d-entry feature vector corresponding to the video frame; for each of the s video image classes, using the d-entry feature vector to compute an image class probability of the d-entry feature vector being produced by a corresponding one of s image class statistical models; and classifying the video frame into the video image class corresponding to a maximum image class probability.
Claim 58
Dependent Claim
115 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method for classifying a video frame into one of s video image classes, the method comprising the steps of: retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector.
Claim 59
Dependent Claim
115 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method for classifying a video frame into one of s video image classes, the method comprising the steps of: retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector.
Claim 60
Dependent Claim
203 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said computer readable program code for programming the computer system to perform a method for segmenting a series of video frames into one of s video image classes, the method comprising the steps of: for each of the video frames in the series, retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector; and generating a previous image class pointer corresponding to a maximum product of a previous image class probability and class transition probability; for a last video frame in the series, classifying the video frame into the video image class corresponding to a maximum image class probability; and for each of the video frames except the last video frame in the series, classifying a previous frame into the video image class indicated by the previous image class pointer.
Claim 61
Dependent Claim
203 words
A computer system, comprising: a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said computer readable program code for programming the computer system to perform a method for segmenting a series of video frames into one of s video image classes, the method comprising the steps of: for each of the video frames in the series, retrieving a d-entry feature vector corresponding to the video frame; and for each of the s video image classes, using the d-entry feature vector, s previous image class probabilities, a corresponding one of s class transition probability vectors, and a corresponding one of s image class statistical models to compute an image class probability for the d-entry feature vector; and generating a previous image class pointer corresponding to a maximum product of a previous image class probability and class transition probability; for a last video frame in the series, classifying the video frame into the video image class corresponding to a maximum image class probability; and for each of the video frames except the last video frame in the series, classifying a previous frame into the video image class indicated by the previous image class pointer.
Claim 62
Dependent Claim
118 words
A computer system, comprising: a processor; and a processor readable storage medium having a processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of determining similarity of a video frame using an image class statistical model, comprising the steps of: retrieving a feature vector corresponding to the video frame; retrieving a mean vector of the image class statistical model; and subtracting the mean vector from the feature vector to produce a difference vector classifying the video frame as similar if a magnitude of the difference vector is less than a predetermined multiple of a standard deviation of the image class statistical model.
Claim 63
Dependent Claim
118 words
A computer system, comprising: a processor; and a processor readable storage medium having a processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of determining similarity of a video frame using an image class statistical model, comprising the steps of: retrieving a feature vector corresponding to the video frame; retrieving a mean vector of the image class statistical model; and subtracting the mean vector from the feature vector to produce a difference vector classifying the video frame as similar if a magnitude of the difference vector is less than a predetermined multiple of a standard deviation of the image class statistical model.
References Cited (Backward Citations)
US5794788A
Cited
US5699449A
Cited
US5872865A
Cited
US6430306B2
Cited
US5850352A
Cited
US5870502A
Cited
US6075875A
Cited
US6134348A
Cited
US5905500A
Cited
US6072542A
Cited
US6178260B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2007-06-15 | 2007-12-15 | 2008-06-15 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2011-06-15 | 2011-12-15 | 2012-06-15 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2015-06-15 | 2015-12-15 | 2016-06-15 | $7700.00 | Window Closed / Lapsed |