US5894309A
Expired / Lapsed
US
⏱️ 0.0 years remaining
System for modifying lighting in photographs
Assignee: Mitsubishi Electric Research Laboratories Inc
•
Filing: 1997-02-27
•
Granted: 1999-04-13
•
Expires: 2017-02-27
Abstract
Abstract A system for creating lighting for a photograph generates a large number ofmages based upon structures determined from the photograph and randomly selected light positions, types and directions. One or more photographs are analyzed to determine the three-dimensional structures in the photographs. These structures are represented as gray surfaces, to which lighting is applied. The images are culled to provide a set of images which best spans the lighting space represented by the large number of images. The culling process is iterative; at each iteration, the image which is most dissimilar from a nearest neighbor in the selected set is added to the selected set. The images are organized in a hierarchical structure. A user interface allows the user to review and select images in the hierarchical structure. The images selected by the user are combined to create a final image with composite lighting. The lighting specifications can be saved for future reference by the lighting system or other applications. The lighting of the gray surface images are used to modify the intensity of pixels in the photograph in order to change lighting in the photograph.
Inventors
- 👤 William T. Freeman
- 👤 Joseph Marks
- 👤 Paul Beardsley
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-08 22:39
CPC Classifications
G; G06; G06T; G06T15/00; G06T15/50
Claims Analysis
Total Claims: 15
Claim 1
Independent Claim
123 words
A system for adding lighting to a photograph comprising: structure determining means for determining three-dimensional structures in the photograph and representing the structures in an original image; light generation means for generating a plurality of images with lighting based upon the original image without lighting and data regarding a plurality of lights; culling means for selecting a set of said plurality of images with lighting; selection means for selecting at least one of said plurality of images with lighting from said set; display means for displaying an image formed from said at least one of said plurality of images with lighting; and photograph modifying means for modifying the intensity of pixels in the photograph based upon at least one of the displayed images.
Claim 2
Independent Claim
123 words
A system for adding lighting to a photograph comprising: structure determining means for determining three-dimensional structures in the photograph and representing the structures in an original image; light generation means for generating a plurality of images with lighting based upon the original image without lighting and data regarding a plurality of lights; culling means for selecting a set of said plurality of images with lighting; selection means for selecting at least one of said plurality of images with lighting from said set; display means for displaying an image formed from said at least one of said plurality of images with lighting; and photograph modifying means for modifying the intensity of pixels in the photograph based upon at least one of the displayed images.
Claim 3
Dependent Claim
111 words
The system for adding lighting to photograph according to claim 1, wherein said light generation means includes: means for selecting at least one light-hook surface in said image; means for selecting at least one light type; position means for randomly selecting a plurality of light positions at a location on said at least one light-hook surface; type means for selecting one of said at least one light type for each light position selected by said position means; and image generation means for generating said plurality of images with lighting, each of said images with lighting being based upon said original image, one of said light positions and a corresponding light type.
Claim 4
Dependent Claim
50 words
The system for adding lighting to a photograph according to claim 1, wherein said culling means includes: means for determining an illumination level for each of said plurality of light images with lighting; and means for deleting light images with lighting which have an illumination level below a predetermined threshold.
Claim 5
Dependent Claim
34 words
The system for adding lighting to a photograph according to claim 1, wherein said culling means selects said set so as to best cover a lighting space representing said plurality of images with lighting.
Claim 6
Dependent Claim
104 words
The system for adding lighting to a photograph according to claim 4, wherein said culling means includes: means for selecting a first image for said set from said plurality of images with lighting and removing said first image from said plurality of images with lighting; difference means for determining a difference for each of said images with lighting, said difference representing a smallest difference between each of said images with lighting and images in said set; and selection means for selecting one of said images with lighting having a greatest difference for said set and removing said selected image from said plurality of images.
Claim 7
Dependent Claim
72 words
The system for adding lighting to a photograph according to claim 1, wherein said selection means includes: organizing means for organizing said images in a hierarchical structure; level display means for displaying images in said hierarchical structure based upon user selection of an image at a higher level in said hierarchical structure; and selection means for selecting one of said images displayed by the level display means based upon a user input.
Claim 8
Independent Claim
101 words
A method for adding lighting to a photograph comprising the steps of: generating an original image of three-dimensional structures in the photograph represented as gray surfaces; generating a plurality of images with lighting based upon the original image without lighting and data regarding a plurality of lights; selecting a set of said plurality of images with lighting; selecting at least one of said plurality of images with lighting from said set; displaying an image formed from said at least one of said plurality of images with lighting; and modifying an intensity of pixels in the photograph based upon said displayed image.
Claim 9
Independent Claim
101 words
A method for adding lighting to a photograph comprising the steps of: generating an original image of three-dimensional structures in the photograph represented as gray surfaces; generating a plurality of images with lighting based upon the original image without lighting and data regarding a plurality of lights; selecting a set of said plurality of images with lighting; selecting at least one of said plurality of images with lighting from said set; displaying an image formed from said at least one of said plurality of images with lighting; and modifying an intensity of pixels in the photograph based upon said displayed image.
Claim 10
Dependent Claim
86 words
The method for adding lighting to a photograph according to claim 7, wherein said generating step generates images having a first resolution level; the method further comprising before the second selecting step, a step of generating a second set of images with lighting based upon said set of images, said second set of images having a second resolution level which is higher than said first resolution level; and wherein said second selecting step selects one of said plurality of images from said second set of images.
Claim 11
Dependent Claim
68 words
The method for adding lighting to a photograph according to claim 7, wherein said generating step includes the steps of: randomly selecting a plurality of light positions at locations on at least one light-hook surface; selecting at least one light type for each light position; and generating said plurality of images with lighting based upon said original image, one of said light positions, and a corresponding light type.
Claim 12
Dependent Claim
50 words
The method for adding lighting to a photograph according to claim 7, wherein said first selection step includes the steps of: determining an illumination level for each of said plurality of light images with lighting; and deleting light images with lighting which have an illumination level below a predetermined threshold.
Claim 13
Dependent Claim
116 words
The method for adding lighting to a photograph according to claim 7, wherein said first selection step includes the steps of: selecting a first image for said set from said plurality of images with lighting and removing said first image from said plurality of images with lighting; determining a difference for each of said images with lighting, said difference representing a smallest difference between each of said images with lighting and images in said set; and selecting one of said images with lighting having a greatest difference for said set and removing said selected image from said plurality of images; and repeating said determining and selecting steps until the set includes a predetermined number of images.
Claim 14
Dependent Claim
66 words
The method for adding lighting to a photograph according to claim 7, wherein said second selection step includes the steps of: organizing said images in a hierarchical structure; displaying images in said hierarchical structure based upon user selection of an image at a higher level in said hierarchical structure; and selecting one of said images displayed by the level display means based upon a user input.
Claim 15
Dependent Claim
98 words
The method for adding lighting to a photograph according to claim 12, wherein said organizing step includes: determining a first subset of images from said set of images which are most distinctive from each other; determining a plurality of second subsets of images from said set of images, each of said second subsets being most similar to one of said images in said first subset; and determining a plurality of third subsets of images from said set of images, each of said third subsets being most similar to one of said images in said plurality of second subsets.
References Cited (Backward Citations)
US5422986A
Cited
US5739819A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-04-12 | 2002-10-12 | 2003-04-13 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-04-12 | 2006-10-12 | 2007-04-13 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-04-12 | 2010-10-12 | 2011-04-13 | $7700.00 | Window Closed / Lapsed |