Patent US6671405B1 successfully refreshed from Google Patents.
US6671405B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method for automatic assessment of emphasis and appeal in consumer images
Assignee: Intellectual Ventures Assets 20 LLC
•
Filing: 1999-12-14
•
Granted: 2003-12-30
•
Expires: 2019-12-14
Abstract
Abstract An image is automatically assessed with respect to certain features, wherein the assessment is a determination of the degree of importance, interest or attractiveness of the image. First, a digital image is obtained corresponding to the image. Then one or more quantities are computed that are related to one or more features in the digital image, including one or more features pertaining to the content of the digital image. The quantities are processed with a reasoning algorithm that is trained on the opinions of one or more human observers, and an output is obtained from the reasoning algorithm that assesses the image. More specifically, the reasoning algorithm is a Bayesian network that provides a score which, when done for a group of images, selects one image as the emphasis image. The features pertaining to the content of the digital image include people-related features and/or subject-related features. Moreover, additional quantities may be computed that relate to objective measures of the digital image, such as colorfulness and/or sharpness.
Inventors
- 👤 Andreas E. Savakis
- 👤 Stephen Etz
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:24
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/60; G06T7/66; G06V; G06V10/00; G06V10/40; G06V10/56; G06V40/00; G06V40/10; G06V40/16; G06V40/161; G06V40/162
Claims Analysis
Total Claims: 46
Claim 1
Independent Claim
166 words
A method for assessing an image with respect to certain features, wherein said assessment is a determination of the overall degree of importance, interest or attractiveness of a whole image, said method comprising the steps of: (a) obtaining a digital image corresponding to the image; (b) computing one or more quantities related to one or more features in the digital image, including one or more features pertaining to the content of the digital image; (c) processing said one or more quantities with a reasoning algorithm that is trained on the opinions of one or more human observers; (d) providing an output from the reasoning algorithm that assesses the image, wherein said output comprises a score for assessing the overall degree of importance, interest or attractiveness of the whole image; and (e) ordering the image relative to other images in term of its score, so that an extreme score corresponds to an image having a greater overall degree of importance, interest or attractivenss than the other images.
Claim 2
Independent Claim
166 words
A method for assessing an image with respect to certain features, wherein said assessment is a determination of the overall degree of importance, interest or attractiveness of a whole image, said method comprising the steps of: (a) obtaining a digital image corresponding to the image; (b) computing one or more quantities related to one or more features in the digital image, including one or more features pertaining to the content of the digital image; (c) processing said one or more quantities with a reasoning algorithm that is trained on the opinions of one or more human observers; (d) providing an output from the reasoning algorithm that assesses the image, wherein said output comprises a score for assessing the overall degree of importance, interest or attractiveness of the whole image; and (e) ordering the image relative to other images in term of its score, so that an extreme score corresponds to an image having a greater overall degree of importance, interest or attractivenss than the other images.
Claim 3
Dependent Claim
28 words
The method as claimed in claim 1 wherein the features pertaining to the content of the digital image include at least one of people-related features and subject-related features.
Claim 4
Dependent Claim
32 words
The method as claimed in claim 1 wherein step (b) further includes computing one or more quantities related to one or more objective features pertaining to objective measures of the digital image.
Claim 5
Dependent Claim
19 words
The method as claimed in claim 3 wherein the objective features include at least one of colorfulness and sharpness.
Claim 6
Dependent Claim
27 words
The method as claimed in claim 1 wherein the reasoning algorithm in step (c) is trained at least in part from ground truth studies of candidate images.
Claim 7
Dependent Claim
15 words
The method as claimed in claim 1 wherein the reasoning algorithm is a Bayesian network.
Claim 8
Dependent Claim
24 words
The method as claimed in claim 1 wherein step (d) provides an output from the reasoning algorithm that scores the assessment of the image.
Claim 9
Dependent Claim
44 words
The method as claimed in claim 1 wherein step (a) obtains a plurality of digital images corresponding to a plurality of images and steps (b)-(d) are performed on each of the digital images such that an assessment is performed for each of the images.
Claim 10
Dependent Claim
26 words
The method as claimed in claim 8 wherein the assessments are ranked such that one image with the highest rank is selected as an emphasis image.
Claim 11
Dependent Claim
27 words
The method as claimed in claim 1 wherein the assessment of the image is used in an automatic albuming algorithm to arrange a page layout of images.
Claim 12
Independent Claim
171 words
A method for determining the appeal of an image with respect to certain self-salient features, wherein said appeal is an assessment of the overall degree of importance, interest or attractiveness of a whole individual image, said method comprising the steps of: (a) obtaining a digital image corresponding to the image; (b) computing one or more quantities related to one or more self-salient features in the digital image, including one or more self-salient features pertaining to the content of the digital image; (c) processing said one or more quantities with a reasoning algorithm that is trained on the opinions of one or more human observers; (d) providing an output from the reasoning algorithm that assesses the appeal of the image, wherein said output comprises a score for assessing the overall degree of importance, interest or attractiveness of the whole individual image; and (e) ordering the image relative to other images in term of its score, so that an extreme score corresponds to an image having a greater appeal than the other images.
Claim 13
Independent Claim
171 words
A method for determining the appeal of an image with respect to certain self-salient features, wherein said appeal is an assessment of the overall degree of importance, interest or attractiveness of a whole individual image, said method comprising the steps of: (a) obtaining a digital image corresponding to the image; (b) computing one or more quantities related to one or more self-salient features in the digital image, including one or more self-salient features pertaining to the content of the digital image; (c) processing said one or more quantities with a reasoning algorithm that is trained on the opinions of one or more human observers; (d) providing an output from the reasoning algorithm that assesses the appeal of the image, wherein said output comprises a score for assessing the overall degree of importance, interest or attractiveness of the whole individual image; and (e) ordering the image relative to other images in term of its score, so that an extreme score corresponds to an image having a greater appeal than the other images.
Claim 14
Dependent Claim
29 words
The method as claimed in claim 11 wherein the self-salient features pertaining to the content of the digital image include at least one of people-related features and subject-related features.
Claim 15
Dependent Claim
32 words
The method as claimed in claim 11 wherein step (b) further includes computing one or more quantities related to one or more self-salient features pertaining to the quality of the digital image.
Claim 16
Dependent Claim
27 words
The method as claimed in claim 13 wherein the self-salient features related to the quality of the digital image include at least one of colorfulness and sharpness.
Claim 17
Dependent Claim
27 words
The method as claimed in claim 11 wherein the reasoning algorithm in step (c) is trained at least in part from ground truth studies of candidate images.
Claim 18
Dependent Claim
15 words
The method as claimed in claim 11 wherein the reasoning algorithm is a Bayesian network.
Claim 19
Dependent Claim
24 words
The method as claimed in claim 11 wherein step (d) provides an output from the reasoning algorithm that scores the appeal of the image.
Claim 20
Independent Claim
171 words
A method for determining the emphasis of an image with respect to certain features, wherein said emphasis is an assessment of the degree of importance, interest or attractiveness of an individual image relative to other images in a group of images, said method comprising the steps of: (a) obtaining a plurality of digital images corresponding to the group of images; (b) computing one or more quantities related to one or more features in each digital image, including one or more self-salient features pertaining to the content of each digital image; (c) processing the one or more quantities with a reasoning algorithm that is trained on the opinions of one or more human observers; (d) providing an output from the reasoning algorithm that assesses the emphasis of each image and computes an emphasis score for each image; and (e) ordering the images in the group of images in terms of their emphasis score, so that an extreme score corresponds to an image that is characterized as an emphasis image for the group.
Claim 21
Independent Claim
171 words
A method for determining the emphasis of an image with respect to certain features, wherein said emphasis is an assessment of the degree of importance, interest or attractiveness of an individual image relative to other images in a group of images, said method comprising the steps of: (a) obtaining a plurality of digital images corresponding to the group of images; (b) computing one or more quantities related to one or more features in each digital image, including one or more self-salient features pertaining to the content of each digital image; (c) processing the one or more quantities with a reasoning algorithm that is trained on the opinions of one or more human observers; (d) providing an output from the reasoning algorithm that assesses the emphasis of each image and computes an emphasis score for each image; and (e) ordering the images in the group of images in terms of their emphasis score, so that an extreme score corresponds to an image that is characterized as an emphasis image for the group.
Claim 22
Dependent Claim
29 words
A method as claimed in claim 18 wherein the self-salient features related to the content of the digital images include at least one of people-related features and subject-related features.
Claim 23
Dependent Claim
32 words
The method as claimed in claim 18 wherein step (b) further includes computing one or more quantities related to one or more self-salient features pertaining to the quality of the digital images.
Claim 24
Dependent Claim
31 words
The method as claimed in claim 20 wherein the self-salient features related to the quality of the digital images include at least one of colorfulness and sharpness of the digital images.
Claim 25
Dependent Claim
51 words
The method as claimed in claim 18 wherein step (b) further includes computing one or quantities related to one or more relative-salient features in the digital images pertaining to one or more of the representative quality and the uniqueness of an individual digital image relative to the group of digital images.
Claim 26
Dependent Claim
21 words
The method as claimed in claim 22 wherein the representative quality pertains to the representative quality in terms of color content.
Claim 27
Dependent Claim
18 words
The method as claimed in claim 22 wherein the uniqueness pertains to the uniqueness of picture aspect format.
Claim 28
Dependent Claim
27 words
The method as claimed in claim 22 wherein the reasoning algorithm in step (c) is trained at least in part from ground truth studies of candidate images.
Claim 29
Dependent Claim
18 words
The method as claimed in claim 18 wherein the reasoning algorithm in step (c) is a Bayesian network.
Claim 30
Dependent Claim
182 words
A computer program product for assessing an image with respect to certain features, wherein said assessment is a determination of the overall degree of importance, interest or attractiveness of a whole image, said program product comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of: (a) obtaining a digital image corresponding to the image; (b) computing one or more quantities related to one or more features in the digital image, including one or more features pertaining to the content of the digital image; (c) processing said one or more quantities with a reasoning algorithm that is trained on the opinions of one or more human observers; (d) providing an output from the reasoning algorithm that assesses the image, wherein said output comprises a score for assessing the overall degree of importance, interest or attractiveness of the whole image; and (e) ordering the image relative to other images in terms of its score, so that an extreme score corresponds to an image having a greater overall degree of importance, interest or attractiveness than the other image.
Claim 31
Dependent Claim
182 words
A computer program product for assessing an image with respect to certain features, wherein said assessment is a determination of the overall degree of importance, interest or attractiveness of a whole image, said program product comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of: (a) obtaining a digital image corresponding to the image; (b) computing one or more quantities related to one or more features in the digital image, including one or more features pertaining to the content of the digital image; (c) processing said one or more quantities with a reasoning algorithm that is trained on the opinions of one or more human observers; (d) providing an output from the reasoning algorithm that assesses the image, wherein said output comprises a score for assessing the overall degree of importance, interest or attractiveness of the whole image; and (e) ordering the image relative to other images in terms of its score, so that an extreme score corresponds to an image having a greater overall degree of importance, interest or attractiveness than the other image.
Claim 32
Dependent Claim
30 words
The computer program product as claimed in claim 27 wherein the features pertaining to the content of the digital image include at least one of people-related features and subject-related features.
Claim 33
Dependent Claim
34 words
The computer program product as claimed in claim 27 wherein step (b) further includes computing one or more quantities related to one or more objective features pertaining to objective measures of the digital image.
Claim 34
Dependent Claim
21 words
The computer program product as claimed in claim 29 wherein the objective features include at least one of colorfulness and sharpness.
Claim 35
Dependent Claim
29 words
The computer program product as claimed in claim 27 wherein the reasoning algorithm in step (c) is trained at least in part from ground truth studies of candidate images.
Claim 36
Dependent Claim
17 words
The computer program product as claimed in claim 27 wherein the reasoning algorithm is a Bayesian network.
Claim 37
Dependent Claim
26 words
The computer program product as claimed in claim 27 wherein step (d) provides an output from the reasoning algorithm that scores the assessment of the image.
Claim 38
Dependent Claim
46 words
The computer program product as claimed in claim 27 wherein step (a) obtains a plurality of digital images corresponding to a plurality of images and steps (b)-(d) are performed on each of the digital images such that an assessment is performed for each of the images.
Claim 39
Dependent Claim
28 words
The computer program product as claimed in claim 34 wherein the assessments are ranked such that one image with the highest rank is selected as an emphasis image.
Claim 40
Dependent Claim
29 words
The computer program product as claimed in claim 27 wherein the assessment of the image is used in an automatic albuming algorithm to arrange a page layout of images.
Claim 41
Independent Claim
116 words
A method for determining the colorfulness of a digital image based on the chrominance component of its image pixels, said method comprising the steps of: (a) transforming the digital image to luminance/chrominance space; (b) dividing the chromaticity space into a plurality of bins; (c) associating each image pixel with one of the bins if its chrominance component lies within the bounds of that bin; (d) computing the level of saturation for each pixel in each bin; (e) determining the number of high saturation pixels in each bin; (f) identifying each bin where the number of high saturation pixels exceed a threshold as an active bin; and (g) determining the colorfulness from the number of active bins.
Claim 42
Independent Claim
116 words
A method for determining the colorfulness of a digital image based on the chrominance component of its image pixels, said method comprising the steps of: (a) transforming the digital image to luminance/chrominance space; (b) dividing the chromaticity space into a plurality of bins; (c) associating each image pixel with one of the bins if its chrominance component lies within the bounds of that bin; (d) computing the level of saturation for each pixel in each bin; (e) determining the number of high saturation pixels in each bin; (f) identifying each bin where the number of high saturation pixels exceed a threshold as an active bin; and (g) determining the colorfulness from the number of active bins.
Claim 43
Independent Claim
141 words
A method for determining compositional aspects of a digital image based on the outcome of main subject detection, said method comprising the steps of: (a) generating a main subject belief map indicative of a main subject in the digital image; (b) computing a variance measure of the pixels belonging to the main subject belief map; (c) determining a centrality measure of the main subject based on a distance between an image center and a centroid of a portion of the pixels belonging to the main subject belief map; (d) determining a compactness measure of the main subject by evaluating the proportion of a bounding box occupied by said portion of the pixels belonging to the main subject belief map; and (e) evaluating the goodness of the compositional aspects of the digital image by reference to the variance, centrality and compactness measures.
Claim 44
Independent Claim
141 words
A method for determining compositional aspects of a digital image based on the outcome of main subject detection, said method comprising the steps of: (a) generating a main subject belief map indicative of a main subject in the digital image; (b) computing a variance measure of the pixels belonging to the main subject belief map; (c) determining a centrality measure of the main subject based on a distance between an image center and a centroid of a portion of the pixels belonging to the main subject belief map; (d) determining a compactness measure of the main subject by evaluating the proportion of a bounding box occupied by said portion of the pixels belonging to the main subject belief map; and (e) evaluating the goodness of the compositional aspects of the digital image by reference to the variance, centrality and compactness measures.
Claim 45
Dependent Claim
54 words
The method as claimed in claim 38 wherein the pixels belonging to the main subject belief map are clustered into a plurality of regions based on their probability of being part of the main subject and the highest probability region provides said portion of the pixels that are considered for steps (c) and (d).
Claim 46
Dependent Claim
39 words
The method as claimed in claim 39 further including determining the size of the main subject in relation to the proportion of a central area of the digital image that is occupied by pixels from the highest probability region.
References Cited (Backward Citations)
US4396903A
Cited
US5418895A
Cited
US5424945A
Cited
US5875265A
Cited
US5855482A
Cited
US5983218A
Cited
US6282317B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2006-12-29 | 2007-06-30 | 2007-12-30 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2010-12-29 | 2011-06-30 | 2011-12-30 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2014-12-29 | 2015-06-30 | 2015-12-30 | $7700.00 | Window Closed / Lapsed |