US6282317B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method for automatic determination of main subjects in photographic images
Assignee: Monument Peak Ventures LLC
•
Filing: 1998-12-31
•
Granted: 2001-08-28
•
Expires: 2018-12-31
Abstract
Abstract A method for detecting a main subject in an image, the method comprises: receiving a digital image; extracting regions of arbitrary shape and size defined by actual objects from the digital image; grouping the regions into larger segments corresponding to physically coherent objects; extracting for each of the regions at least one structural saliency feature and at least one semantic saliency feature; and integrating saliency features using a probabilistic reasoning engine into an estimate of a belief that each region is the main subject.
Inventors
- 👤 Jiebo Luo
- 👤 Stephen Etz
- 👤 Amit Singhal
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-07 20:59
CPC Classifications
G; G06; G06T; G06T11/00; G06T11/60; G06F; G06F18/00; G06F18/20; G06F18/29; G06T7/00; G06T7/10; G06T7/11; G06T7/143; G06T7/194; G06V; G06V10/00; G06V10/20; G06V10/25; G06V10/40; G06V10/46; G06V10/462; G06V10/70; G06V10/84; G06V20/00; G06V20/10; G06V40/00; G06V40/10; G06V40/16; G06V40/161; G06V40/162
Claims Analysis
Total Claims: 39
Claim 1
Independent Claim
86 words
A method for detecting a main subject in an image, the method comprising the steps of: a) receiving a digital image; b) extracting regions of arbitrary shape and size defined by actual objects from the digital image; c) extracting for each of the regions at least one structural saliency feature and at least one semantic saliency feature; and, d) integrating the structural saliency feature and the semantic feature using a probabilistic reasoning engine into an estimate of a belief that each region is the main subject.
Claim 2
Independent Claim
86 words
A method for detecting a main subject in an image, the method comprising the steps of: a) receiving a digital image; b) extracting regions of arbitrary shape and size defined by actual objects from the digital image; c) extracting for each of the regions at least one structural saliency feature and at least one semantic saliency feature; and, d) integrating the structural saliency feature and the semantic feature using a probabilistic reasoning engine into an estimate of a belief that each region is the main subject.
Claim 3
Dependent Claim
32 words
The method as in claim 1 , wherein step (b) includes using a color distance metric defined in a color space, a spatial homogeneity constraint, and a mechanism for permitting spatial adaptivity.
Claim 4
Dependent Claim
28 words
The method as in claim 1 , wherein the structural saliency feature of step (c) includes at least one of a low-level vision feature and a geometric feature.
Claim 5
Dependent Claim
58 words
The method as in claim 1 , wherein step (c) includes using either individually or in combination a color, brightness and/or texture as a low-level vision feature; a location, size, shape, convexity, aspect ratio, symmetry, borderness, surroundedness and/or occlusion as a geometric feature; and a flesh, face, sky, grass and/or other green vegetation as the semantic saliency feature.
Claim 6
Dependent Claim
31 words
The method as in claim 1 , wherein step (d) includes using a collection of human opinions to train the reasoning engine to recognize the relative importance of the saliency features.
Claim 7
Dependent Claim
30 words
The method as in claim 1 , wherein step (c) includes using either individually or in combination a self-saliency feature and a relative saliency feature as the structural saliency feature.
Claim 8
Dependent Claim
72 words
The method as in claim 6 , wherein step (c) includes using an extended neighborhood window to compute a plurality of the relative saliency features, wherein the extended neighborhood window is determined by the steps of: (c1) finding a minimum bounding rectangle of a region; (c2) stretching the minimum bounding rectangle in all four directions proportionally; and (c3) defining all regions intersecting the stretched minimum bounding rectangle as neighbors of the region.
Claim 9
Dependent Claim
78 words
The method as in claim 4 , wherein step (c) includes using a centrality as the location feature, wherein the centrality feature is computed by the steps of: (c1) determining a probability density function of main subject locations using a collection of training data; (c2) computing an integral of the probability density function over an area of a region; and, (c3) obtaining a value of the centrality feature by normalizing the integral by the area of the region.
Claim 10
Dependent Claim
37 words
The method as in claim 4 , wherein step (c) includes using a hyperconvexity as the convexity feature, wherein the hyperconvexity feature is computed as a ratio of a perimeter-based convexity measure and an area-based convexity measure.
Claim 11
Dependent Claim
28 words
The method as in claim 4 , wherein step (c) includes computing a maximum fraction of a region perimeter shared with a neighboring region as the surroundedness feature.
Claim 12
Dependent Claim
48 words
The method as in claim 4 , wherein step (c) includes using an orientation-unaware borderness feature as the borderness feature, wherein the orientation-unaware borderness feature is categorized by the number and configuration of image borders a region is in contact with, and all image borders are treated equally.
Claim 13
Dependent Claim
48 words
The method as in claim 4 , wherein step (c) includes using an orientation-aware borderness feature as the borderness feature, wherein the orientation-aware borderness feature is categorized by the number and configuration of image borders a region is in contact with, and each image border is treated differently.
Claim 14
Dependent Claim
31 words
The method as in claim 4 , wherein step (c) includes using the borderness feature that is determined by what fraction of an image border is in contact with a region.
Claim 15
Dependent Claim
32 words
The method as in claim 4 , wherein step (c) includes using the borderness feature that is determined by what fraction of a region border is in contact with an image border.
Claim 16
Dependent Claim
19 words
The method as in claim 1 , wherein step (d) includes using a Bayes net as the reasoning engine.
Claim 17
Dependent Claim
31 words
The method as in claim 1 , wherein step (d) includes using a conditional probability matrix that is determined by using fractional frequency counting according to a collection of training data.
Claim 18
Dependent Claim
33 words
The method as in claim 1 , wherein step (d) includes using a belief sensor function to convert a measurement of a feature into evidence, which is an input to a Bayes net.
Claim 19
Dependent Claim
27 words
The method as in claim 1 , wherein step (d) includes outputting a belief map, which indicates a location of and a belief in the main subject.
Claim 20
Independent Claim
98 words
A method for detecting a main subject in an image, the method comprising the steps of: a) receiving a digital image; b) extracting regions of arbitrary shape and size defined by actual objects from the digital image; c) grouping the regions into larger segments corresponding to physically coherent objects; d) extracting for each of the regions at least one structural saliency feature and at least one semantic saliency feature; and, e) integrating the structural saliency feature and the semantic feature using a probabilistic reasoning engine into an estimate of a belief that each region is the main subject.
Claim 21
Independent Claim
98 words
A method for detecting a main subject in an image, the method comprising the steps of: a) receiving a digital image; b) extracting regions of arbitrary shape and size defined by actual objects from the digital image; c) grouping the regions into larger segments corresponding to physically coherent objects; d) extracting for each of the regions at least one structural saliency feature and at least one semantic saliency feature; and, e) integrating the structural saliency feature and the semantic feature using a probabilistic reasoning engine into an estimate of a belief that each region is the main subject.
Claim 22
Dependent Claim
32 words
The method as in claim 19 , wherein step (b) includes using a color distance metric defined in a color space, a spatial homogeneity constraint, and a mechanism for permitting spatial adaptivity.
Claim 23
Dependent Claim
22 words
The method as in claim 19 , wherein step (c) includes using either individually or in combination non-purposive grouping and purposive grouping.
Claim 24
Dependent Claim
34 words
The method as in claim 19 , wherein step (d) includes using either individually or in combination at least one low-level vision feature and at least one geometric feature as the structural saliency feature.
Claim 25
Dependent Claim
58 words
The method as in claim 19 , wherein step (d) includes using either individually or in combination a color, brightness and/or texture as a low-level vision feature; a location, size, shape, convexity, aspect ratio, symmetry, borderness, surroundedness and/or occlusion as a geometric feature; and a flesh, face, sky, grass and/or other green vegetation as the semantic saliency feature.
Claim 26
Dependent Claim
31 words
The method as in claim 19 , wherein step (e) includes using a collection of human opinions to train the reasoning engine to recognize the relative importance of the saliency features.
Claim 27
Dependent Claim
30 words
The method as in claim 19 , wherein step (d) includes using either individually or in combination a self-saliency feature and a relative saliency feature as the structural saliency feature.
Claim 28
Dependent Claim
72 words
The method as in claim 25 , wherein step (d) includes using an extended neighborhood window to compute a plurality of the relative saliency features, wherein the extended neighborhood window is determined by the steps of: (c1) finding a minimum bounding rectangle of a region; (c2) stretching the minimum bounding rectangle in all four directions proportionally; and, (c3) defining all regions intersecting the stretched minimum bounding rectangle as neighbors of the region.
Claim 29
Dependent Claim
78 words
The method as in claim 23 , wherein step (d) includes using a centrality as the location feature, wherein the centrality feature is computed by the steps of: (c1) determining a probability density function of main subject locations using a collection of training data; (c2) computing an integral of the probability density function over an area of a region; and, (c3) obtaining a value of the centrality feature by normalizing the integral by the area of the region.
Claim 30
Dependent Claim
37 words
The method as in claim 23 , wherein step (d) includes using a hyperconvexity as the convexity feature, wherein the hyperconvexity feature is computed as a ratio of a perimeter-based convexity measure and an area-based convexity measure.
Claim 31
Dependent Claim
28 words
The method as in claim 23 , wherein step (d) includes computing a maximum fraction of a region perimeter shared with a neighboring region as the surroundedness feature.
Claim 32
Dependent Claim
48 words
The method as in claim 23 , wherein step (d) includes using an orientation-unaware borderness feature as the borderness feature, wherein the orientation-unaware borderness feature is categorized by the number and configuration of image borders a region is in contact with, and all image borders are treated equally.
Claim 33
Dependent Claim
48 words
The method as in claim 23 , wherein step (d) includes using an orientation-aware borderness feature as the borderness feature, wherein the orientation-aware borderness feature is categorized by the number and configuration of image borders a region is in contact with, and each image border is treated differently.
Claim 34
Dependent Claim
31 words
The method as in claim 23 , wherein step (d) includes using the borderness feature that is determined by what fraction of an image border is in contact with a region.
Claim 35
Dependent Claim
32 words
The method as in claim 23 , wherein step (d) includes using the borderness feature that is determined by what fraction of a region border is in contact with an image border.
Claim 36
Dependent Claim
19 words
The method as in claim 19 , wherein step (e) includes using a Bayes net as the reasoning engine.
Claim 37
Dependent Claim
31 words
The method as in claim 19 , wherein step (e) includes using a conditional probability matrix that is determined by using fractional frequency counting according to a collection of training data.
Claim 38
Dependent Claim
33 words
The method as in claim 19 , wherein step (e) includes using a belief sensor function to convert a measurement of a feature into evidence, which is an input to a Bayes net.
Claim 39
Dependent Claim
27 words
The method as in claim 19 , wherein step (e) includes outputting a belief map, which indicates a location of and a belief in the main subject.
References Cited (Backward Citations)
US4975975A
Cited
US5862260A
Cited
US5809322A
Cited
US6014461A
Cited
US5724456A
Cited
US5850352A
Cited
US5983218A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2004-08-27 | 2005-02-26 | 2005-08-28 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2008-08-27 | 2009-02-26 | 2009-08-28 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2012-08-27 | 2013-02-26 | 2013-08-28 | $7700.00 | Window Closed / Lapsed |