US6611622B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Object recognition system and process for identifying people and objects in an image of a scene
Assignee: Microsoft Technology Licensing LLC
•
Filing: 1999-11-23
•
Granted: 2003-08-26
•
Expires: 2019-11-23
Abstract
Abstract An object recognition system and process that identifies people and objects depicted in an image of a scene. In general, this system and process entails first creating model histograms of the people and objects that it is desired to identify in the image. Then, the image is segmented to extract regions which likely correspond to the people and objects being identified. A histogram is computed for each of the extracted regions, and the degree of similarity between each extracted region histogram and each of the model histograms is assessed. The extracted regions having a histogram that exhibits a degree of similarity to one of the model histograms which exceeds a prescribed threshold is designated as corresponding to the person or object associated with that model histogram. Finally, the histogram computed for any extracted region of the image that is designated as corresponding to a person or object associated with a model histogram can be stored as an additional model histogram associated with that person or object. Preferably, the foregoing general system and process is repeated for subsequently generated images of the scene, so that the identity of people and objects can be monitored over time as they move into and about the scene. In addition, preferably color images of the scene and color histograms are employed in the object recognition system and process.
Inventors
- 👤 John Krumm
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 09:55
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/70; G06V; G06V10/00; G06V10/40; G06V10/56; G06V40/00; G06V40/10; G06V40/16; G06V40/161; G06T2207/00; G06T2207/30; G06T2207/30196
Claims Analysis
Total Claims: 35
Claim 1
Independent Claim
175 words
A computer-implemented process for identifying a person or object in an image of a scene, said process comprising using a computer to perform the following process actions: creating model histograms of people and objects that it is desired to identify in said image of the scene; segmenting said image to extract regions which correspond to at least one person or object whose identity it is desired to determine; for each region extracted from the image, computing a histogram for the extracted region, respectively assessing the degree of similarity between the extracted region histogram and each of said model histograms, and designating each extracted region having a histogram that exhibits a degree of similarity to a one of the model histograms which exceeds a prescribed threshold as corresponding to the person or object associated with that model histogram; and storing the histogram computed for any extracted region of the image that is designated as corresponding to a person or object associated with a model histogram as an additional model histogram associated with that person or object.
Claim 2
Dependent Claim
175 words
A computer-implemented process for identifying a person or object in an image of a scene, said process comprising using a computer to perform the following process actions: creating model histograms of people and objects that it is desired to identify in said image of the scene; segmenting said image to extract regions which correspond to at least one person or object whose identity it is desired to determine; for each region extracted from the image, computing a histogram for the extracted region, respectively assessing the degree of similarity between the extracted region histogram and each of said model histograms, and designating each extracted region having a histogram that exhibits a degree of similarity to a one of the model histograms which exceeds a prescribed threshold as corresponding to the person or object associated with that model histogram; and storing the histogram computed for any extracted region of the image that is designated as corresponding to a person or object associated with a model histogram as an additional model histogram associated with that person or object.
Claim 3
Dependent Claim
48 words
The process of claim 1 , further comprising the process actions of: designating each extracted region having a histogram that does not exhibit a degree of similarity to any of the model histograms which exceeds the prescribed threshold as corresponding to the person or object of unknown identity.
Claim 4
Dependent Claim
42 words
The process of claim 1 , further comprising the process action of repeating said segmenting, computing, assessing, designating and storing actions for successive images of the scene so as to track the identity of persons and objects in the scene over time.
Claim 5
Dependent Claim
35 words
The process of claim 1 , wherein the image is a color image of the scene, and the model histograms and histograms computed for each of region extracted from the color image are color histograms.
Claim 6
Dependent Claim
166 words
The process of claim 4 , wherein the process action of creating model histograms of people and objects that it is desired to identify in said image of the scene, comprises the actions of: capturing one or more model images of the people and objects that it is desired to identify; segmenting said model image to extract model regions which correspond to each of said people and objects whose identity it is desired to determine; and for each model region, determining the actual colors exhibited by the pixels of the model region; dividing the overall gamut of actual colors exhibited by the pixels of the extracted model region into a series of discrete color ranges, hereinafter referred to as quantized color categories; respectively assigning each pixel of the extracted model region to the quantized color category into which the actual color of the pixel falls, and establishing a count of the number of pixels of the extracted model region assigned to the same quantized color category.
Claim 7
Dependent Claim
109 words
The process of claim 5 , wherein the process action of computing a histogram for each of region extracted from the image, comprises the actions of: determining the actual colors exhibited by the pixels of the extracted region; dividing the overall gamut of actual colors exhibited by the pixels of the extracted region into a series of discrete color ranges, hereinafter referred to as quantized color categories; respectively assigning each pixel of the extracted region to the quantized color category into which the actual color of the pixel falls; and establishing a count of the number of pixels of the extracted region assigned to the same quantized color category.
Claim 8
Dependent Claim
50 words
The process of claim 6 , wherein the process actions of dividing the overall gamut of actual colors exhibited by the pixels of the extracted model regions and extracted image regions into a series of discrete color ranges, comprises the action of employing the same quantized color categories for each.
Claim 9
Dependent Claim
112 words
The process of claim 7 , wherein the process action of respectively assessing the degree of similarity between the extracted region histogram and each of the model histograms for each histogram associated with an extracted region, comprises the actions of: respectively comparing the pixel count from each quantized color category of the histogram derived from the extracted region to the pixel count from the corresponding quantized color category of each model histograms; identifying the smaller of the two counts in each quantized color category for each pair of histograms compared; and summing the smaller counts from each quantized color category to produce a separate similarity value for each pair of histograms compared.
Claim 10
Dependent Claim
134 words
The process of claim 8 , wherein the process action of designating each extracted region having a histogram that exhibits a degree of similarity to one of the model histograms which exceeds a prescribed threshold as corresponding to the person or object associated with that model histogram, comprises the actions of: for each extracted region, normalizing the similarity value associated with each pair of compared histograms by dividing it by a maximum possible similarity value to produce a match quality indicator; identify the largest match quality indicator associated with each extracted region; for each extracted region, designating the extracted region to be the person or object associated with the model histogram used in part to compute the largest match quality indicator for that region whenever the largest match quality indicator exceeds the prescribed threshold.
Claim 11
Dependent Claim
119 words
The process of claim 2 , wherein the process action of designating each extracted region having a histogram that does not exhibit a degree of similarity to one of the model histograms which exceeds the prescribed threshold as corresponding to the person or object of unknown identity, comprises the actions of: for each extracted region, normalizing the similarity value associated with each pair of compared histograms by dividing it by a maximum possible similarity value to produce a match quality indicator; identify the largest match quality indicator associated with each extracted region; for each extracted region, designating the extracted region to be an unknown person or object whenever the largest match quality indicator does not exceeds the prescribed threshold.
Claim 12
Dependent Claim
205 words
An object recognition system for identifying a person or object in an image of a scene, comprising: a general purpose computing device; a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to, (a) create model histograms of people and objects that it is desired to identify in said image of the scene, (b) segment said image to extract regions which correspond to at least one person or object whose identity it is desired to determine, (c) compute a histogram for each of region extracted from the image, (d) for each histogram associated with an extracted region, respectively assess the degree of similarity between the extracted region histogram and each of said model histograms, (e) designate each extracted region having a histogram that exhibits a degree of similarity to one of the model histograms which exceeds a prescribed threshold as corresponding to the person or object associated with that model histogram, and (f) store the histogram computed for any extracted region of the image that is designated as corresponding to a person or object associated with a model histogram as an additional model histogram associated with that person or object.
Claim 13
Dependent Claim
205 words
An object recognition system for identifying a person or object in an image of a scene, comprising: a general purpose computing device; a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to, (a) create model histograms of people and objects that it is desired to identify in said image of the scene, (b) segment said image to extract regions which correspond to at least one person or object whose identity it is desired to determine, (c) compute a histogram for each of region extracted from the image, (d) for each histogram associated with an extracted region, respectively assess the degree of similarity between the extracted region histogram and each of said model histograms, (e) designate each extracted region having a histogram that exhibits a degree of similarity to one of the model histograms which exceeds a prescribed threshold as corresponding to the person or object associated with that model histogram, and (f) store the histogram computed for any extracted region of the image that is designated as corresponding to a person or object associated with a model histogram as an additional model histogram associated with that person or object.
Claim 14
Dependent Claim
48 words
The system of claim 11 , further comprising the program modules for: designating each extracted region having a histogram that does not exhibit a degree of similarity to any of the model histograms which exceeds the prescribed threshold as corresponding to the person or object of unknown identity.
Claim 15
Dependent Claim
38 words
The system of claim 11 , further comprising a program module for repeating actions (b) through (f) for successive images of the scene so as to track the identity of persons and objects in the scene over time.
Claim 16
Dependent Claim
35 words
The system of claim 11 , wherein the image is a color image of the scene, and the model histograms and histograms computed for each of region extracted from the color image are color histograms.
Claim 17
Dependent Claim
165 words
The system of claim 14 , wherein the program module for creating model histograms of people and objects that it is desired to identify in said image of the scene, comprises sub-modules for: capturing one or more model images of the people and objects that it is desired to identify; segmenting said model image to extract model regions which correspond to each of said people and objects whose identity it is desired to determine; and for each model region, determining the actual colors exhibited by the pixels of the model region; dividing the overall gamut of actual colors exhibited by the pixels of the extracted model region into a series of discrete color ranges, hereinafter referred to as quantized color categories; respectively assigning each pixel of the extracted model region to the quantized color category into which the actual color of the pixel falls, and establishing a count of the number of pixels of the extracted model region assigned to the same quantized color category.
Claim 18
Dependent Claim
108 words
The system of claim 15 , wherein the program module for computing a histogram for each of region extracted from the image, comprises sub-modules for: determining the actual colors exhibited by the pixels of the extracted region; dividing the overall gamut of actual colors exhibited by the pixels of the extracted region into a series of discrete color ranges, hereinafter referred to as quantized color categories; respectively assigning each pixel of the extracted region to the quantized color category into which the actual color of the pixel falls; and establishing a count of the number of pixels of the extracted region assigned to the same quantized color category.
Claim 19
Dependent Claim
49 words
The system of claim 16 , wherein the sub-modules for dividing the overall gamut of actual colors exhibited by the pixels of the extracted model regions and extracted image regions into a series of discrete color ranges, comprises a sub-module for employing the same quantized color categories for each.
Claim 20
Dependent Claim
111 words
The system of claim 17 , wherein the program module for respectively assessing the degree of similarity between the extracted region histogram and each of the model histograms for each histogram associated with an extracted region, comprises sub-modules for: respectively comparing the pixel count from each quantized color category of the histogram derived from the extracted region to the pixel count from the corresponding quantized color category of each model histograms; identifying the smaller of the two counts in each quantized color category for each pair of histograms compared; and summing the smaller counts from each quantized color category to produce a separate similarity value for each pair of histograms compared.
Claim 21
Dependent Claim
133 words
The system of claim 18 , wherein the program module for designating each extracted region having a histogram that exhibits a degree of similarity to one of the model histograms which exceeds a prescribed threshold as corresponding to the person or object associated with that model histogram, comprises sub-modules for: for each extracted region, normalizing the similarity value associated with each pair of compared histograms by dividing it by a maximum possible similarity value to produce a match quality indicator; identify the largest match quality indicator associated with each extracted region; for each extracted region, designating the extracted region to be the person or object associated with the model histogram used in part to compute the largest match quality indicator for that region whenever the largest match quality indicator exceeds the prescribed threshold.
Claim 22
Dependent Claim
118 words
The system of claim 12 , wherein the program module for designating each extracted region having a histogram that does not exhibit a degree of similarity to one of the model histograms which exceeds the prescribed threshold as corresponding to the person or object of unknown identity, comprises sub-modules for: for each extracted region, normalizing the similarity value associated with each pair of compared histograms by dividing it by a maximum possible similarity value to produce a match quality indicator; identify the largest match quality indicator associated with each extracted region; for each extracted region, designating the extracted region to be an unknown person or object whenever the largest match quality indicator does not exceeds the prescribed threshold.
Claim 23
Dependent Claim
214 words
A computer-readable memory for causing a computer to perform an object recognition procedure for identifying a person or object in an image of a scene, comprising: a general purpose computing device; a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to, (a) create model histograms of people and objects that it is desired to identify in said image of the scene, (b) segment said image to extract regions which correspond to at least one person or object whose identity it is desired to determine, (c) compute a histogram for each of region extracted from the image, (d) for each histogram associated with an extracted region, respectively assess the degree of similarity between the extracted region histogram and each of said model histograms, (e) designate each extracted region having a histogram that exhibits a degree of similarity to one of the model histograms which exceeds a prescribed threshold as corresponding to the person or object associated with that model histogram, and (f) store the histogram computed for any extracted region of the image that is designated as corresponding to a person or object associated with a model histogram as an additional model histogram associated with that person or object.
Claim 24
Dependent Claim
214 words
A computer-readable memory for causing a computer to perform an object recognition procedure for identifying a person or object in an image of a scene, comprising: a general purpose computing device; a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to, (a) create model histograms of people and objects that it is desired to identify in said image of the scene, (b) segment said image to extract regions which correspond to at least one person or object whose identity it is desired to determine, (c) compute a histogram for each of region extracted from the image, (d) for each histogram associated with an extracted region, respectively assess the degree of similarity between the extracted region histogram and each of said model histograms, (e) designate each extracted region having a histogram that exhibits a degree of similarity to one of the model histograms which exceeds a prescribed threshold as corresponding to the person or object associated with that model histogram, and (f) store the histogram computed for any extracted region of the image that is designated as corresponding to a person or object associated with a model histogram as an additional model histogram associated with that person or object.
Claim 25
Dependent Claim
49 words
The computer-readable memory of claim 21 , further comprising the program modules for: designating each extracted region having a histogram that does not exhibit a degree of similarity to any of the model histograms which exceeds the prescribed threshold as corresponding to the person or object of unknown identity.
Claim 26
Dependent Claim
39 words
The computer-readable memory of claim 21 , further comprising a program module for repeating actions (b) through (f) for successive images of the scene so as to track the identity of persons and objects in the scene over time.
Claim 27
Dependent Claim
36 words
The computer-readable memory of claim 21 , wherein the image is a color image of the scene, and the model histograms and histograms computed for each of region extracted from the color image are color histograms.
Claim 28
Dependent Claim
166 words
The computer-readable memory of claim 24 , wherein the program module for creating model histograms of people and objects that it is desired to identify in said image of the scene, comprises sub-modules for: capturing one or more model images of the people and objects that it is desired to identify; segmenting said model image to extract model regions which correspond to each of said people and objects whose identity it is desired to determine; and for each model region, determining the actual colors exhibited by the pixels of the model region; dividing the overall gamut of actual colors exhibited by the pixels of the extracted model region into a series of discrete color ranges, hereinafter referred to as quantized color categories; respectively assigning each pixel of the extracted model region to the quantized color category into which the actual color of the pixel falls, and establishing a count of the number of pixels of the extracted model region assigned to the same quantized color category.
Claim 29
Dependent Claim
109 words
The computer-readable memory of claim 25 , wherein the program module for computing a histogram for each of region extracted from the image, comprises sub-modules for: determining the actual colors exhibited by the pixels of the extracted region; dividing the overall gamut of actual colors exhibited by the pixels of the extracted region into a series of discrete color ranges, hereinafter referred to as quantized color categories; respectively assigning each pixel of the extracted region to the quantized color category into which the actual color of the pixel falls; and establishing a count of the number of pixels of the extracted region assigned to the same quantized color category.
Claim 30
Dependent Claim
50 words
The computer-readable memory of claim 26 , wherein the sub-modules for dividing the overall gamut of actual colors exhibited by the pixels of the extracted model regions and extracted image regions into a series of discrete color ranges, comprises a sub-module for employing the same quantized color categories for each.
Claim 31
Dependent Claim
112 words
The computer-readable memory of claim 27 , wherein the program module for respectively assessing the degree of similarity between the extracted region histogram and each of the model histograms for each histogram associated with an extracted region, comprises sub-modules for: respectively comparing the pixel count from each quantized color category of the histogram derived from the extracted region to the pixel count from the corresponding quantized color category of each model histograms; identifying the smaller of the two counts in each quantized color category for each pair of histograms compared; and summing the smaller counts from each quantized color category to produce a separate similarity value for each pair of histograms compared.
Claim 32
Dependent Claim
134 words
The computer-readable memory of claim 28 , wherein the program module for designating each extracted region having a histogram that exhibits a degree of similarity to one of the model histograms which exceeds a prescribed threshold as corresponding to the person or object associated with that model histogram, comprises sub-modules for: for each extracted region, normalizing the similarity value associated with each pair of compared histograms by dividing it by a maximum possible similarity value to produce a match quality indicator; identify the largest match quality indicator associated with each extracted region; for each extracted region, designating the extracted region to be the person or object associated with the model histogram used in part to compute the largest match quality indicator for that region whenever the largest match quality indicator exceeds the prescribed threshold.
Claim 33
Dependent Claim
119 words
The computer-readable memory of claim 22 , wherein the program module for designating each extracted region having a histogram that does not exhibit a degree of similarity to one of the model histograms which exceeds the prescribed threshold as corresponding to the person or object of unknown identity, comprises sub-modules for: for each extracted region, normalizing the similarity value associated with each pair of compared histograms by dividing it by a maximum possible similarity value to produce a match quality indicator; identify the largest match quality indicator associated with each extracted region; for each extracted region, designating the extracted region to be an unknown person or object whenever the largest match quality indicator does not exceeds the prescribed threshold.
Claim 34
Dependent Claim
177 words
A computer-implemented process for identifying a person or object in an image of a scene, said process comprising using a computer to perform the following process actions: creating model histograms of people and objects that it is desired to identify in said image of the scene; segmenting said image to extract regions which correspond to at least one person or object whose identity it is desired to determine; for each region extracted from the image, computing a histogram for the extracted region, respectively assessing the degree of similarity between the extracted region histogram and at least some of said model histograms, and designating each extracted region having a histogram that exhibits a degree of similarity to a one of the model histograms which exceeds a prescribed threshold as corresponding to the person or object associated with that model histogram; and storing the histogram computed for any extracted region of the image that is designated as corresponding to a person or object associated with a model histogram as an additional model histogram associated with that person or object.
Claim 35
Dependent Claim
177 words
A computer-implemented process for identifying a person or object in an image of a scene, said process comprising using a computer to perform the following process actions: creating model histograms of people and objects that it is desired to identify in said image of the scene; segmenting said image to extract regions which correspond to at least one person or object whose identity it is desired to determine; for each region extracted from the image, computing a histogram for the extracted region, respectively assessing the degree of similarity between the extracted region histogram and at least some of said model histograms, and designating each extracted region having a histogram that exhibits a degree of similarity to a one of the model histograms which exceeds a prescribed threshold as corresponding to the person or object associated with that model histogram; and storing the histogram computed for any extracted region of the image that is designated as corresponding to a person or object associated with a model histogram as an additional model histogram associated with that person or object.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2006-08-25 | 2007-02-24 | 2007-08-26 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2010-08-25 | 2011-02-24 | 2011-08-26 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2014-08-25 | 2015-02-24 | 2015-08-26 | $7700.00 | Window Closed / Lapsed |