US6151009A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus for merging real and synthetic images
Assignee: Carnegie Mellon University
•
Filing: 1996-08-21
•
Granted: 2000-11-21
•
Expires: 2016-08-21
Abstract
Abstract A method for merging real and synthetic images in real time is comprised of the steps of providing a first signal containing depth and image information per pixel about a real image. A second signal containing depth and image information per pixel about a synthetic image is provided. The depth information corresponding to the real image and the depth information corresponding to the synthetic image for each pixel are compared. Based on the comparison, either the image information corresponding to the real image or the image information corresponding to the synthetic image is selected and combined. Because the image information is compared based on depth, any interaction such as occluding, shadowing, reflecting, or colliding can be determined and an appropriate output generated
Inventors
- 👤 Takeo Kanade
- 👤 Kazuo Oda
- 👤 Masaya Tanaka
- 👤 Hiroshi Kano
- 👤 Atsushi Yoshida
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:04
CPC Classifications
G; G06; G06T; G06T15/00; G06T15/50; G06T15/506; G06T15/10; G06T15/40; G06T15/405; G06T15/60; H; H04; H04N; H04N5/00; H04N5/222; H04N5/2224; H04N5/262; H04N5/272
Claims Analysis
Total Claims: 36
Claim 1
Independent Claim
141 words
A method of merging a real image and a synthetic image in real time, comprising: providing a first signal containing depth and image information per pixel about said real image; providing a second signal containing depth and image information per pixel about said synthetic image; comparing said depth information corresponding to said real image and said depth information corresponding to said synthetic image for each pixel; selecting one of said image information corresponding to said real image and said image information corresponding to said synthetic image in response to said comparison step for each pixel; combining the selected information to produce an output signal; calculating an interaction between images in the first signal and the second signal, wherein said interaction is selected from the group consisting of occlusions, collisions, shadows, and reflections; and generating output signals based on said calculated interaction.
Claim 2
Independent Claim
141 words
A method of merging a real image and a synthetic image in real time, comprising: providing a first signal containing depth and image information per pixel about said real image; providing a second signal containing depth and image information per pixel about said synthetic image; comparing said depth information corresponding to said real image and said depth information corresponding to said synthetic image for each pixel; selecting one of said image information corresponding to said real image and said image information corresponding to said synthetic image in response to said comparison step for each pixel; combining the selected information to produce an output signal; calculating an interaction between images in the first signal and the second signal, wherein said interaction is selected from the group consisting of occlusions, collisions, shadows, and reflections; and generating output signals based on said calculated interaction.
Claim 3
Dependent Claim
18 words
The method of claim 1 wherein said generating step includes the step of generating an audible output signal.
Claim 4
Dependent Claim
18 words
The method of claim 1 wherein said generating step includes the step of generating a visible output signal.
Claim 5
Dependent Claim
52 words
The method of claim 1 additionally comprising the step of producing an image mask for an object in said first signal and wherein said comparing step includes the step of comparing said depth information corresponding to only said image mask and said depth information corresponding to said synthetic image for each pixel.
Claim 6
Dependent Claim
51 words
The method of claim 1 additionally comprising the step of producing an image mask for an object in said second signal, and wherein said comparing step includes the step of comparing depth information corresponding to said real image and said depth information corresponding to only said image mask for each pixel.
Claim 7
Independent Claim
139 words
A method of merging images in real time, comprising: providing first and second signals containing depth and image information per pixel about first and second images, wherein said first and second images are selected from the group consisting of real images and synthetic images; producing an image mask for an object in said second signal; comparing said depth information corresponding to said first image and said depth information corresponding to said second image for each pixel, said comparing includes comparing depth information corresponding to said first image and said depth information corresponding to only said image mask for each pixel; selecting one of said image information corresponding to said first image and said image information corresponding to said second image in response to said comparison step for each pixel; and combining the selected information to produce an output signal.
Claim 8
Independent Claim
139 words
A method of merging images in real time, comprising: providing first and second signals containing depth and image information per pixel about first and second images, wherein said first and second images are selected from the group consisting of real images and synthetic images; producing an image mask for an object in said second signal; comparing said depth information corresponding to said first image and said depth information corresponding to said second image for each pixel, said comparing includes comparing depth information corresponding to said first image and said depth information corresponding to only said image mask for each pixel; selecting one of said image information corresponding to said first image and said image information corresponding to said second image in response to said comparison step for each pixel; and combining the selected information to produce an output signal.
Claim 9
Dependent Claim
46 words
The method of claim 6 additionally comprising the steps of calculating an interaction between images in the first signal and the second signal and generating output signals based on said calculated interaction, wherein said interaction is selected from the group consisting of collisions, shadows, and reflections.
Claim 10
Dependent Claim
18 words
The method of claim 7 wherein said generating step includes the step of generating an audible output signal.
Claim 11
Dependent Claim
18 words
The method of claim 7 wherein said generating step includes the step of generating a visible output signal.
Claim 12
Independent Claim
92 words
A method of determining interactions between objects in a real image and objects in a synthetic image, comprising: providing a first signal containing depth and image information per pixel about said real image; providing a second signal containing depth and image information per pixel about said synthetic image; comparing said depth information corresponding to said real image and said depth information corresponding to said synthetic image for each pixel; and calculating interaction based on said comparison step, wherein said interaction is selected from the group consisting of occlusions, collisions, shadows, and reflections.
Claim 13
Independent Claim
92 words
A method of determining interactions between objects in a real image and objects in a synthetic image, comprising: providing a first signal containing depth and image information per pixel about said real image; providing a second signal containing depth and image information per pixel about said synthetic image; comparing said depth information corresponding to said real image and said depth information corresponding to said synthetic image for each pixel; and calculating interaction based on said comparison step, wherein said interaction is selected from the group consisting of occlusions, collisions, shadows, and reflections.
Claim 14
Independent Claim
141 words
A method of merging a real image and a synthetic image in real time, comprising: providing a first signal containing depth and image information per pixel about said real image; providing a second signal containing depth and image information per pixel about said synthetic image; producing an image mask for an object in said first signal; comparing said depth information corresponding to said real image and said depth information corresponding to said synthetic image for each pixel, said comparing includes comparing said depth information corresponding to only said image mask and said depth information corresponding to said synthetic image for each pixel; selecting one of said image information corresponding to said real image and said image information corresponding to said synthetic image in response to said comparison step for each pixel; and combining the selected information to produce an output signal.
Claim 15
Independent Claim
141 words
A method of merging a real image and a synthetic image in real time, comprising: providing a first signal containing depth and image information per pixel about said real image; providing a second signal containing depth and image information per pixel about said synthetic image; producing an image mask for an object in said first signal; comparing said depth information corresponding to said real image and said depth information corresponding to said synthetic image for each pixel, said comparing includes comparing said depth information corresponding to only said image mask and said depth information corresponding to said synthetic image for each pixel; selecting one of said image information corresponding to said real image and said image information corresponding to said synthetic image in response to said comparison step for each pixel; and combining the selected information to produce an output signal.
Claim 16
Dependent Claim
47 words
The method of claim 11 additionally comprising the steps of calculating an interaction between images in the first signal and the second signal and generating output signals based on said calculated interaction, wherein said interaction is selected from the group consisting of occlusions, collisions, shadows, and reflections.
Claim 17
Dependent Claim
18 words
The method of claim 12 wherein said generating step includes the step of generating an audible output signal.
Claim 18
Dependent Claim
18 words
The method of claim 12 wherein said generating step includes the step of generating a visible output signal.
Claim 19
Independent Claim
140 words
A method of merging a real image and a synthetic image in real time, comprising: providing a first signal containing depth and image information per pixel about said real image; providing a second signal containing depth and image information per pixel about said synthetic image; producing an image mask for an object in said second signal; comparing said depth information corresponding to said real image and said depth information corresponding to said synthetic image for each pixel, said comparing includes comparing depth information corresponding to said real image and said depth information corresponding to only said image mask for each pixel; selecting one of said image information corresponding to said real image and said image information corresponding to said synthetic image in response to said comparison step for each pixel; and combining the selected information to produce an output signal.
Claim 20
Independent Claim
140 words
A method of merging a real image and a synthetic image in real time, comprising: providing a first signal containing depth and image information per pixel about said real image; providing a second signal containing depth and image information per pixel about said synthetic image; producing an image mask for an object in said second signal; comparing said depth information corresponding to said real image and said depth information corresponding to said synthetic image for each pixel, said comparing includes comparing depth information corresponding to said real image and said depth information corresponding to only said image mask for each pixel; selecting one of said image information corresponding to said real image and said image information corresponding to said synthetic image in response to said comparison step for each pixel; and combining the selected information to produce an output signal.
Claim 21
Dependent Claim
47 words
The method of claim 15 additionally comprising the steps of calculating an interaction between images in the first signal and the second signal and generating output signals based on said calculated interaction, wherein said interaction is selected from the group consisting of occlusions, collisions, shadows, and reflections.
Claim 22
Dependent Claim
18 words
The method of claim 16 wherein said generating step includes the step of generating an audible output signal.
Claim 23
Dependent Claim
18 words
The method of claim 16 wherein said generating step includes the step of generating a visible output signal.
Claim 24
Independent Claim
137 words
A method of merging images in real time, comprising: providing first and second signals containing depth and image information per pixel about first and second images, wherein said first and second images are selected from the group consisting of real images and synthetic images; comparing said depth information corresponding to said first image and said depth information corresponding to said second image for each pixel; selecting one of said image information corresponding to said first image and said image information corresponding to said second image in response to said comparison step for each pixel; combining the selected information to produce an output signal; calculating an interaction between images in said first and second signals wherein said interaction is selected from the group consisting of collisions, shadows, and reflections; and generating output signals based on said calculated interaction.
Claim 25
Independent Claim
137 words
A method of merging images in real time, comprising: providing first and second signals containing depth and image information per pixel about first and second images, wherein said first and second images are selected from the group consisting of real images and synthetic images; comparing said depth information corresponding to said first image and said depth information corresponding to said second image for each pixel; selecting one of said image information corresponding to said first image and said image information corresponding to said second image in response to said comparison step for each pixel; combining the selected information to produce an output signal; calculating an interaction between images in said first and second signals wherein said interaction is selected from the group consisting of collisions, shadows, and reflections; and generating output signals based on said calculated interaction.
Claim 26
Dependent Claim
18 words
The method of claim 19 wherein said generating step includes the step of generating an audible output signal.
Claim 27
Dependent Claim
18 words
The method of claim 19 wherein said generating step includes the step of generating a visible output signal.
Claim 28
Dependent Claim
52 words
The method of claim 19 additionally comprising the step of producing an image mask for an object in said first signal and wherein said comparing step includes the step of comparing said depth information corresponding to only said image mask and said depth information corresponding to said second image for each pixel.
Claim 29
Dependent Claim
51 words
The method of claim 19 additionally comprising the step of producing an image mask for an object in said second signal, and wherein said comparing step includes the step of comparing depth information corresponding to said first image and said depth information corresponding to only said image mask for each pixel.
Claim 30
Independent Claim
98 words
A method of determining interactions between objects in a first image and objects in a second image, comprising: providing first and second signals containing depth and image information per pixel about said first and second images, wherein said first and second images are selected from the group consisting of real images and synthetic images; comparing said depth information corresponding to said first image and said depth information corresponding to said second image for each pixel; and calculating an interaction based on said comparison step, wherein said interaction is selected from the group consisting of collisions, shadows, and reflections.
Claim 31
Independent Claim
98 words
A method of determining interactions between objects in a first image and objects in a second image, comprising: providing first and second signals containing depth and image information per pixel about said first and second images, wherein said first and second images are selected from the group consisting of real images and synthetic images; comparing said depth information corresponding to said first image and said depth information corresponding to said second image for each pixel; and calculating an interaction based on said comparison step, wherein said interaction is selected from the group consisting of collisions, shadows, and reflections.
Claim 32
Independent Claim
137 words
A method of merging images in real time, comprising: providing first and second signals containing depth and image information per pixel about first and second images, wherein said images are selected from the group consisting of real images and synthetic images; producing an image mask for an object in said first signal; comparing said depth information corresponding to said first image and said depth information corresponding to said second image for each pixel, said comparing includes comparing said depth information corresponding to only said image mask and said depth information corresponding to said second image for each pixel; selecting one of said image information corresponding to said first image and said image information corresponding to said second image in response to said comparison step for each pixel; and combining the selected information to produce an output signal.
Claim 33
Independent Claim
137 words
A method of merging images in real time, comprising: providing first and second signals containing depth and image information per pixel about first and second images, wherein said images are selected from the group consisting of real images and synthetic images; producing an image mask for an object in said first signal; comparing said depth information corresponding to said first image and said depth information corresponding to said second image for each pixel, said comparing includes comparing said depth information corresponding to only said image mask and said depth information corresponding to said second image for each pixel; selecting one of said image information corresponding to said first image and said image information corresponding to said second image in response to said comparison step for each pixel; and combining the selected information to produce an output signal.
Claim 34
Dependent Claim
46 words
The method of claim 25 additionally comprising the steps of calculating an interaction between images in the first signal and the second signal and generating output signals based on said calculated interaction, wherein said interaction is selected from the group consisting of collisions, shadows, and reflections.
Claim 35
Dependent Claim
18 words
The method of claim 26 wherein said generating step includes the step of generating an audible output signal.
Claim 36
Dependent Claim
18 words
The method of claim 26 wherein said generating step includes the step of generating a visible output signal.
References Cited (Backward Citations)
US4875097A
Cited
US5379129A
Cited
US5353068A
Cited
US5280337A
Cited
US5400080A
Cited
US5408272A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2003-11-21 | 2004-05-22 | 2004-11-21 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2007-11-21 | 2008-05-22 | 2008-11-21 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2011-11-21 | 2012-05-22 | 2012-11-21 | $7700.00 | Window Closed / Lapsed |