Patent US7230653B1 successfully refreshed from Google Patents.
US7230653B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus for real time insertion of images into video
Assignee: Disney Enterprises Inc
•
Filing: 2000-11-07
•
Granted: 2007-06-12
•
Expires: 2020-11-07
Abstract
Abstract The present invention comprises a real time image insertion system for video utilizes a three-dimensional model of at least one target area within a site for identifying the target area within an original video image of the site, and for rendering a target image and inserting it into the target area of the video image. The target area may be a real, existing surface of the site or an imaginary surface defined only in the model. The model is rendered from the position and angle of a camera that generated the video image. By using a three dimensional model of the site to generate the target image, the resulting image with the inserted target material appears more realistic.
Inventors
- 👤 Kenneth J. Overton
- 👤 Mark S. Muench
- 👤 Charles H. Hayden, JR.
- 👤 Norman S. Carmichael
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:58
CPC Classifications
G; G06; G06T; G06T19/00; G06T15/00; G06T15/10; G06T15/20; H; H04; H04N; H04N5/00; H04N5/222; H04N5/262; H04N5/272; H04N5/2723
Claims Analysis
Total Claims: 34
Claim 1
Independent Claim
157 words
A method for inserting a target image into an image stream of a real site taken by a camera, the method comprising: receiving a stream of original images taken by a camera of a real, three-dimensional site; for each original image in the image stream, identifying a background area within the original image from at least one target area in which a target image will be inserted; rendering, for each at least one target area, a target image based on at least a productions three-dimensional model of the at least one target area within the site and the camera's position and pointing direction, wherein the target area is comprised of a virtual surface that does nor exist within the site; identifying occlusions within the at least one target area of the original image; and combining the background of the original image, the at least one target image, and the image of the occlusions into an output image.
Claim 2
Independent Claim
157 words
A method for inserting a target image into an image stream of a real site taken by a camera, the method comprising: receiving a stream of original images taken by a camera of a real, three-dimensional site; for each original image in the image stream, identifying a background area within the original image from at least one target area in which a target image will be inserted; rendering, for each at least one target area, a target image based on at least a productions three-dimensional model of the at least one target area within the site and the camera's position and pointing direction, wherein the target area is comprised of a virtual surface that does nor exist within the site; identifying occlusions within the at least one target area of the original image; and combining the background of the original image, the at least one target image, and the image of the occlusions into an output image.
Claim 3
Dependent Claim
43 words
The method of claim 1 , wherein identifying a background area within the original image includes identifying at least one target area within the image based on a predefined model of selected surfaces of the site and the camera's position and pointing direction.
Claim 4
Dependent Claim
60 words
The method of claim 2 , wherein identifying at least one target area within the image comprises: receiving camera parameters, the camera parameters including parameters indicating the pointing direction of the camera; rendering the predefined model from the camera's position, based on the camera parameters; and generating from the rendering a mask for defining the target area within the image.
Claim 5
Dependent Claim
19 words
The method of claim 3 , wherein the camera parameters included parameters indicating the camera's position within the site.
Claim 6
Dependent Claim
15 words
The method of claim 3 , wherein the camera's position is predefined within the model.
Claim 7
Dependent Claim
47 words
The method of claim 1 , wherein rendering, for at least one target area, a target image comprises: receiving camera parameters, the camera parameters including parameters indicating the pointing direction of the camera; and rendering the predefined model based at least in part on the camera parameters.
Claim 8
Dependent Claim
15 words
The method of claim 6 , wherein the camera's position is predefined within the model.
Claim 9
Dependent Claim
19 words
The method of claim 6 , wherein the camera parameters includes parameters indicating the camera's position within the site.
Claim 10
Dependent Claim
19 words
The method of claim 8 wherein the camera parameters are encoded onto a video signal generated by the camera.
Claim 11
Dependent Claim
49 words
The method of claim 1 wherein identifying occlusions within the image of the target area includes: rendering the predefined model of the site with a reference image inserted in each of the at least one target areas; and comparing the reference image to the target area within original image.
Claim 12
Dependent Claim
31 words
The method of claim 1 further including encoding the output image and camera parameters onto an output video signal, the camera parameters including parameters indicating the pointing direction of the camera.
Claim 13
Independent Claim
119 words
An apparatus for inserting a target image into video of a real site taken by a camera, the apparatus comprising: a model renderer for generating, a synthetic image based on a predefined three-dimensional reference model of a target area with a site from a known position of a camera, the target area including a virtual surface that does not exist within the site, the synthetic image having a target image inserted in the target area; an image separator for masking from a video image contained within a frame of a video signal generated by the camera, the target area to create a masked background image; and an image combiner for combining the masked background image with the target image.
Claim 14
Independent Claim
119 words
An apparatus for inserting a target image into video of a real site taken by a camera, the apparatus comprising: a model renderer for generating, a synthetic image based on a predefined three-dimensional reference model of a target area with a site from a known position of a camera, the target area including a virtual surface that does not exist within the site, the synthetic image having a target image inserted in the target area; an image separator for masking from a video image contained within a frame of a video signal generated by the camera, the target area to create a masked background image; and an image combiner for combining the masked background image with the target image.
Claim 15
Dependent Claim
94 words
The apparatus of claim 12 further comprising: a second model renderer for generating a second synthetic image based on the predefined reference model of the site, the second synthetic image having a reference image inserted into the target area; and an occlusion separator for comparing the reference image of the second synthetic image to the target area of the video image and generating an image of occlusions within the target area of the video image; wherein the combiner further combines the image of the occlusions with the masked background image and the target image.
Claim 16
Independent Claim
274 words
A method for inserting a target image into an image stream of a real site taken by a camera, the method comprising: receiving a stream of original images taken by a camera of a real site; for each original image in the image stream; identifying a background area within the original image from at least one target area in which a target image will be inserted by identifying at least one target area within the image based on a predefined model of selected surfaces of the site and the camera's position and pointing direction, the target area including a virtual surface that does not exist within the site, wherein identifying at least one target area within the image includes: receiving camera parameters, the camera parameters including parameters indicating the pointing direction of the camera; rendering the predefined model from the camera's position, based on the camera parameters; and generating from the rendering a mask for defining the target area within the image; rendering, for each at least one target area, a target image based on at least a predefined three-dimensional model of the at least one target area within the site and the camera's position and pointing direction, the camera's position being predefined within the model, wherein rendering includes receiving camera parameters, the camera parameters including parameters indicating the pointing direction of the camera, and rendering the predefined model based at least in part on the camera parameters; identifying occlusions within the at least one target area of the original image; combining the background of the original image, the at least one target image, and the image of the occlusions into an output image.
Claim 17
Independent Claim
274 words
A method for inserting a target image into an image stream of a real site taken by a camera, the method comprising: receiving a stream of original images taken by a camera of a real site; for each original image in the image stream; identifying a background area within the original image from at least one target area in which a target image will be inserted by identifying at least one target area within the image based on a predefined model of selected surfaces of the site and the camera's position and pointing direction, the target area including a virtual surface that does not exist within the site, wherein identifying at least one target area within the image includes: receiving camera parameters, the camera parameters including parameters indicating the pointing direction of the camera; rendering the predefined model from the camera's position, based on the camera parameters; and generating from the rendering a mask for defining the target area within the image; rendering, for each at least one target area, a target image based on at least a predefined three-dimensional model of the at least one target area within the site and the camera's position and pointing direction, the camera's position being predefined within the model, wherein rendering includes receiving camera parameters, the camera parameters including parameters indicating the pointing direction of the camera, and rendering the predefined model based at least in part on the camera parameters; identifying occlusions within the at least one target area of the original image; combining the background of the original image, the at least one target image, and the image of the occlusions into an output image.
Claim 18
Dependent Claim
49 words
The method of claim 14 wherein identifying occlusions within the image of the target area includes: rendering the predefined model of the site with a reference image inserted in each of the at least one target areas; and comparing the reference image to the target area within original image.
Claim 19
Dependent Claim
31 words
The method of claim 14 further including encoding the output image and camera parameters onto an output video signal, the camera parameters including parameters indicating the pointing direction of the camera.
Claim 20
Independent Claim
149 words
A method for inserting a target image into an image stream of a real site taken by a camera, the method comprising: receiving a stream of original images taken by a camera of a real site; for each original image in the image stream, identifying a background area within the original image from at least one target area in which a target image will be inserted; rendering, for each at least one target area, a target image based on at least a predefined three-dimensional model of the at least one target area within the site and the camera's position and pointing direction, the three-dimensional model being of less than the entire site; identifying occlusions within the at least one target area of the original image; and combining the background of the original image, the at least one target image, and the image of the occlusions into an output image.
Claim 21
Independent Claim
149 words
A method for inserting a target image into an image stream of a real site taken by a camera, the method comprising: receiving a stream of original images taken by a camera of a real site; for each original image in the image stream, identifying a background area within the original image from at least one target area in which a target image will be inserted; rendering, for each at least one target area, a target image based on at least a predefined three-dimensional model of the at least one target area within the site and the camera's position and pointing direction, the three-dimensional model being of less than the entire site; identifying occlusions within the at least one target area of the original image; and combining the background of the original image, the at least one target image, and the image of the occlusions into an output image.
Claim 22
Dependent Claim
43 words
The method of claim 17 , wherein identifying a background area within the original image includes identifying at least one target area within the image based on a predefined model of selected surfaces of the site and the camera's position and pointing direction.
Claim 23
Dependent Claim
60 words
The method of claim 18 , wherein identifying at least one target area within the image comprises: receiving camera parameters, the camera parameters including parameters indicating the pointing direction of the camera; rendering the predefined model from the camera's position, based on the camera parameters; and generating from the rendering a mask or defining the target area within the image.
Claim 24
Dependent Claim
19 words
The method of claim 19 , wherein the camera parameters included parameters indicating the camera's position within the site.
Claim 25
Dependent Claim
15 words
The method of claim 19 , wherein the camera's position is predefined within the model.
Claim 26
Dependent Claim
47 words
The method of claim 17 , wherein rendering, for at least one target area, a target image comprises: receiving camera parameters, the camera parameters including parameters indicating the pointing direction of the camera; and rendering the predefined model based at least in part on the camera parameters.
Claim 27
Dependent Claim
15 words
The method of claim 22 , wherein the camera's position is predefined within the model.
Claim 28
Dependent Claim
19 words
The method of claim 22 , wherein the camera parameters includes parameters indicating the camera's position within the site.
Claim 29
Dependent Claim
19 words
The method of claim 24 wherein the camera parameters are encoded onto a video signal generated by the camera.
Claim 30
Dependent Claim
49 words
The method of claim 17 wherein identifying occlusions within the image of the target area includes: rendering the predefined model of the site with a reference image inserted in each of the at least one target areas; and comparing the reference image to the target area within original image.
Claim 31
Dependent Claim
31 words
The method of claim 17 further including encoding the output image and camera parameters onto an output video signal, the camera parameters including parameters indicating the pointing direction of the camera.
Claim 32
Independent Claim
115 words
An apparatus for inserting a target image into video of a real site taken by a camera, the apparatus comprising: a model renderer for generating a synthetic image based on a predefined three-dimensional reference model including a target area within a site from a known position of a camera, the three-dimensional model being of less than the entire site, the synthetic image having a target image inserted in the target area; an image separator for masking from a video image contained within a frame of a video signal generated by the camera, the target area to create a masked background image; and an image combiner for combining the masked background image with the target image.
Claim 33
Independent Claim
115 words
An apparatus for inserting a target image into video of a real site taken by a camera, the apparatus comprising: a model renderer for generating a synthetic image based on a predefined three-dimensional reference model including a target area within a site from a known position of a camera, the three-dimensional model being of less than the entire site, the synthetic image having a target image inserted in the target area; an image separator for masking from a video image contained within a frame of a video signal generated by the camera, the target area to create a masked background image; and an image combiner for combining the masked background image with the target image.
Claim 34
Dependent Claim
94 words
The apparatus of claim 28 further comprising: a second model renderer for generating a second synthetic image based on the predefined reference model of the site, the second synthetic image having a reference image inserted into the target area; and an occlusion separator for comparing the reference image of the second synthetic image to the target area of the video image and generating an image of occlusions within the target area of the video image; wherein the combiner further combines the image of the occlusions with the masked background image and the target image.
References Cited (Backward Citations)
US4084184A
Cited
US4693378A
Cited
US4970666A
Cited
US5353392A
Cited
US5515485A
Cited
US5479597A
Cited
US5264933A
Cited
US5566251A
Cited
US5715018A
Cited
US5903317A
Cited
WO1995010919A1
Cited
US6297853B1
Cited
US5424770A
Cited
US5923365A
Cited
US5543856A
Cited
EP0792068B1
Cited
US5640320A
Cited
US5731846A
Cited
US5491517A
Cited
US6122013A
Cited
US5436672A
Cited
WO1996005689A2
Cited
US5627915A
Cited
FR2730837A1
Cited
US5600366A
Cited
US5850352A
Cited
US5808695A
Cited
US5953076A
Cited
US6522787B1
Cited
WO1997003517A1
Cited
US6384871B1
Cited
US6292227B1
Cited
WO1997009822A1
Cited
WO1997012480A2
Cited
US5892554A
Cited
WO1997026758A1
Cited
US6208387B1
Cited
US6366316B1
Cited
US5917553A
Cited
US6441864B1
Cited
US6100925A
Cited
WO1998028906A2
Cited
US6446261B1
Cited
US6262778B1
Cited
EP0907145A2
Cited
WO1999026415A1
Cited
US6393162B1
Cited
US6487721B1
Cited
US6327004B1
Cited
WO1999048285A1
Cited
US6266100B1
Cited
US6731788B1
Cited
US6525741B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2010-06-11 | 2010-12-11 | 2011-06-12 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2014-06-11 | 2014-12-11 | 2015-06-12 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2018-06-11 | 2018-12-11 | 2019-06-12 | $7700.00 | Window Closed / Lapsed |