US6504569B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
2-D extended image generation from 3-D data extracted from a video sequence
Assignee: Grass Valley USA LLC
•
Filing: 1998-04-22
•
Granted: 2003-01-07
•
Expires: 2018-04-22
Abstract
Abstract A method of generating a 2-D extended image from a video sequence representing a natural 3-D scene first determines motion parameters for a camera that recorded the scene with respect to a bakcground object from the video sequence using a structure-from-motion algorithm. The motion parameters include a rotation matrix, a translation vector and a depth map representing the depth of each point in the background object from the camera. Next from the motion parameters and depth map the 2-D extended image is generated for the background object as a composition of the images from the video sequence using a plane perspective projection technique. The background object may be layered as a function of depth and flatness criteria to form a set of layered 2-D extended images for the background object from the video sequence.
Inventors
- 👤 Radu S. Jasinschi
- 👤 Ali Tabatabai
- 👤 Thumpudi Naveen
- 👤 Paul T. Babic-Vovk
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 13:25
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/20; G06T7/246; G06T7/50; G06T7/55; G06T7/579; G06T2200/00; G06T2200/32; G06T2207/00; G06T2207/10; G06T2207/10016; Y; Y10; Y10S; Y10S345/00; Y10S345/949; Y10S345/95
Claims Analysis
Total Claims: 19
Claim 1
Independent Claim
186 words
A method of generating a 2-D extended image from a video sequence representing a natural 3-D scene comprising the steps of: for a background object segmented from the video sequence, determining motion parameters for a camera that recorded the video sequence with respect to a 3-D coordinate system, and from the motion parameters determining a depth map representing the depth of each point in the background object from the camera; and from the motion parameters and depth map, generating the 2-D extended image for the background object as a composition of contiguous images from the video sequence, wherein the determining step comprises the steps of: extracting feature points for the background object from an image of the video sequence; tracking the features points for the background object into a next image of the video sequence to produce feature point correspondence pairs; performing a structure-from-motion algorithm on the feature point correspondence pairs to produce a rotation matrix and a translation vector as the motion parameters as well as a depth value for each feature point in the image; and from the depth values generating the depth map.
Claim 2
Independent Claim
186 words
A method of generating a 2-D extended image from a video sequence representing a natural 3-D scene comprising the steps of: for a background object segmented from the video sequence, determining motion parameters for a camera that recorded the video sequence with respect to a 3-D coordinate system, and from the motion parameters determining a depth map representing the depth of each point in the background object from the camera; and from the motion parameters and depth map, generating the 2-D extended image for the background object as a composition of contiguous images from the video sequence, wherein the determining step comprises the steps of: extracting feature points for the background object from an image of the video sequence; tracking the features points for the background object into a next image of the video sequence to produce feature point correspondence pairs; performing a structure-from-motion algorithm on the feature point correspondence pairs to produce a rotation matrix and a translation vector as the motion parameters as well as a depth value for each feature point in the image; and from the depth values generating the depth map.
Claim 3
Dependent Claim
25 words
The method as recited in claim 1 further comprising the step of pre-segmenting the background object from the video sequence prior to the determining step.
Claim 4
Dependent Claim
43 words
The method as recited in claim 2 further comprising the step of detecting scene cuts in the video sequence to divide the video sequence into a plurality of scene shots, each scene shot being processed separately by the pre-segmenting, determining and generating steps.
Claim 5
Dependent Claim
29 words
The method as recited in claim 1 wherein the tracking step further comprises the step of removing outliers from the feature point correspondence pairs prior to the performing step.
Claim 6
Dependent Claim
48 words
The method as recited in claim 1 wherein the performing step comprises the steps of: pre-processing the feature point correspondence pairs to produce a plurality of sets of estimated motion parameters; and selecting from among the sets of estimated motion parameters a best set as the motion parameters.
Claim 7
Dependent Claim
47 words
The method as recited in claim 5 wherein the selecting step comprises the step of computing an overall velocity estimation error from input and output velocity vectors for every feature point, with the combination that minimizes the overall velocity estimation error being selected as the best set.
Claim 8
Dependent Claim
28 words
The method as recited in claim 5 wherein the selecting step comprises the step of statistically finding the best set from among the sets of estimated motion parameters.
Claim 9
Dependent Claim
35 words
The method as recited in claim 1 further comprising the step of deciding from the motion parameters and depth map whether to proceed with the generating step based upon a flatness criterion for the object.
Claim 10
Dependent Claim
28 words
The method as recited in claim 8 wherein the deciding step further comprises the step of providing a segmentation signal for use in further segmenting the background object.
Claim 11
Independent Claim
170 words
A method of generating a 2-D extended image from a video sequence representing a natural 3-D scene comprising the steps of: for a background object segmented from the video sequence, determining motion parameters for a camera that recorded the video sequence with respect to a 3-D coordinate system, and from the motion parameters determining a depth map representing the depth of each point in the background object from the camera; and from the motion parameters and depth map, generating the 2-D extended image for the background object as a composition of contiguous images from the video sequence, wherein the generating step comprises the steps of: predicting a next image of the background object as a coarse predicted image from a current version of the 2-D extended image; refining the coarse predicted image to generate a predicted image; pasting the predicted image to the current version of the 2-D extended image to generate the 2-D extended image; and repeating the predicting, refining and pasting steps until an end condition is achieved.
Claim 12
Independent Claim
170 words
A method of generating a 2-D extended image from a video sequence representing a natural 3-D scene comprising the steps of: for a background object segmented from the video sequence, determining motion parameters for a camera that recorded the video sequence with respect to a 3-D coordinate system, and from the motion parameters determining a depth map representing the depth of each point in the background object from the camera; and from the motion parameters and depth map, generating the 2-D extended image for the background object as a composition of contiguous images from the video sequence, wherein the generating step comprises the steps of: predicting a next image of the background object as a coarse predicted image from a current version of the 2-D extended image; refining the coarse predicted image to generate a predicted image; pasting the predicted image to the current version of the 2-D extended image to generate the 2-D extended image; and repeating the predicting, refining and pasting steps until an end condition is achieved.
Claim 13
Dependent Claim
34 words
The method as recited in claim 10 wherein the predicting step comprises the step of integrating the current version of the 2-D extended image with the motion parameters to produce the coarse predicted image.
Claim 14
Dependent Claim
26 words
The method as recited in claim 10 further comprising deciding from the 2-D extended image relative to a first image whether the end condition is achieved.
Claim 15
Dependent Claim
33 words
The method as recited in claim 10 further comprising deciding from the output of a scene-cut detector to which the video sequence is input whether a scene-cut has occurred as the end condition.
Claim 16
Dependent Claim
25 words
The method as recited in claim 10 further comprising the step of pre-segmenting the background object from the video sequence prior to the determining step.
Claim 17
Dependent Claim
43 words
The method as recited in claim 14 further comprising the step of detecting scene cuts in the video sequence to divide the video sequence into a plurality of scene shots, each scene shot being processed separately by teh pre-segmenting, determining and generating steps.
Claim 18
Dependent Claim
35 words
The method as recited in claim 10 further comprising the step of deciding from the motion parameters and depth map whether to proceed with the generating step based upon a flatness criterion for the object.
Claim 19
Dependent Claim
28 words
The method as recited in claim 16 wherein the deciding step further comprises the step of providing a segmentation signal for use in further segmenting the background object.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2006-01-06 | 2006-07-08 | 2007-01-07 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2010-01-06 | 2010-07-08 | 2011-01-07 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2014-01-06 | 2014-07-08 | 2015-01-07 | $7700.00 | Window Closed / Lapsed |