Patent US6392135B1 successfully refreshed from Google Patents.
Patent US6236402B1 successfully refreshed from Google Patents.
US6236402B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Display techniques for three-dimensional virtual reality
Assignee: Nokia of America Corp
•
Filing: 1998-09-25
•
Granted: 2001-05-22
•
Expires: 2018-09-25
Abstract
Abstract A limitation of using two-dimensional images, such as videos or photographs, to represent portions of a three-dimensional world occurs when the user moves within the world and views the world from a location different than from the original context of the two-dimensional image, i.e., from a location different than the image's ideal viewing point (IVP). View changes result in the image not aligning well with the surrounding objects of the three-dimensional world. This limitation is overcome by distorting the two-dimensional image so as to adjust the image's vanishing point(s) in accordance with the movement of the user using a pyramidic panel structure. In this manner, as the user moves away from the ideal viewing point, the distortions act to limit the discontinuities between the two-dimensional image and its surroundings. To minimize the depth profile of the pyramidic panel structure, the structure may be segmented into sections and each section translated towards, or away from, the user's viewpoint.
Inventors
- 👤 John T. Edmark
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 18:12
CPC Classifications
G; G06; G06T; G06T15/00; G06T15/10
Claims Analysis
Total Claims: 35
Claim 1
Independent Claim
165 words
A method for use in processing a view of a three-dimensional world in which a first portion of said world is modeled as computer graphics and a second portion of said world is represented by a two-dimensional image texture-mapped on a panel, comprising the steps of: determining the current viewpoint of the user, V; dividing the panel into triangular regions; distorting the triangular regions to form pyramidic panels such that a corresponding vanishing point, P, of a portion of the two-dimensional image moves as a function of the current viewpoint of the user; segmenting each of said pyramidic panels into a plurality of sections; translating each of said plurality of sections of said pyramidic panels towards, or away from, said current viewpoint of the user, V; texture-mapping the two-dimensional image onto the plurality of sections of the pyramidic panels; and as the user moves within the three-dimensional world, repeating the above steps so as to limit discontinuities between the two-dimensional image and the computer graphics.
Claim 2
Independent Claim
165 words
A method for use in processing a view of a three-dimensional world in which a first portion of said world is modeled as computer graphics and a second portion of said world is represented by a two-dimensional image texture-mapped on a panel, comprising the steps of: determining the current viewpoint of the user, V; dividing the panel into triangular regions; distorting the triangular regions to form pyramidic panels such that a corresponding vanishing point, P, of a portion of the two-dimensional image moves as a function of the current viewpoint of the user; segmenting each of said pyramidic panels into a plurality of sections; translating each of said plurality of sections of said pyramidic panels towards, or away from, said current viewpoint of the user, V; texture-mapping the two-dimensional image onto the plurality of sections of the pyramidic panels; and as the user moves within the three-dimensional world, repeating the above steps so as to limit discontinuities between the two-dimensional image and the computer graphics.
Claim 3
Dependent Claim
40 words
The invention as defined in claim 1 wherein said segmenting step includes resegmenting said pyramidic panels into a greater number of smaller sections until the depth profile of the pyramidic panel structure formed from said panels reaches a predetermined level.
Claim 4
Dependent Claim
31 words
The invention as defined in claim 1 wherein an outer edge of each of said plurality of sections of said pyramidic panels is located on the surface of a predetermined plane.
Claim 5
Dependent Claim
20 words
The invention as defined in claim 3 wherein said predetermined plane is the panel onto which the two-dimensional image texture-mapped.
Claim 6
Dependent Claim
31 words
The invention as defined in claim 1 wherein the center of each of said plurality of sections of said pyramidic panels is substantially located at the surface of a predetermined plane.
Claim 7
Dependent Claim
21 words
The invention as defined in claims 5 wherein said predetermined plane is the panel onto which the two-dimensional image is texture-mapped.
Claim 8
Dependent Claim
81 words
The invention as defined in claim 1 further comprising scaling each of said plurality of sections of said pyramidic panels in accordance with the following relationship S t = S p T t T p , where S p is the size of the section; S t is the size of the translated section; T p is distance the section from the user's viewpoint, V; and T t is the distance to the translated section from the user's viewpoint, V.
Claim 9
Dependent Claim
64 words
The invention as defined in claim 1 further comprising determining a vector, {overscore (C)}, corresponding to the direction of a portion of a path contained within the two-dimensional perspective image, and projecting toward the panel the vector, {overscore (C)}, from the image's ideal viewing point, IVP, the intersection of said vector, {overscore (C)}, with the panel being denoted as the image's vanishing point, P.
Claim 10
Dependent Claim
55 words
The invention as defined in claim 1 wherein said distorting of the triangular regions in said distorting step includes determining a new vanishing point, P′, for said two-dimensional image in accordance with the following relationship P′=P+{overscore (T)}, wherein {overscore (T )}is a vector from the image's ideal viewing point, IVP, to the current viewpoint, V.
Claim 11
Dependent Claim
29 words
The invention as defined in claim 1 further comprising the step of calibrating the two-dimensional perspective image as a function of the dimensions of the surroundings within the world.
Claim 12
Dependent Claim
18 words
The invention as defined in claim 1 wherein said two-dimensional perspective image is a frame of a video.
Claim 13
Dependent Claim
16 words
The invention as defined in claim 1 wherein said two-dimensional perspective image is a still picture.
Claim 14
Independent Claim
257 words
A method for use in processing a view of a three-dimensional world in which a first portion of said world is modeled as computer graphics and a second portion of said world is represented by a two-dimensional image texture-mapped on a panel, said two-dimensional image including an object depicted in perspective, said image being such that features of the object closer to a predetermined point of the image are farther away from a user's viewpoint, comprising the steps of: determining a vector, {overscore (C)}, corresponding to the direction of said perspective object in the three-dimensional world; projecting towards said panel the vector, {overscore (C)}, from the two-dimensional image's ideal viewing point, IVP, the intersection of said vector, {overscore (C)}, with the panel being denoted as the image's vanishing point, P; segmenting said panel into triangular regions intersecting at the image's vanishing point, P; determining the current viewpoint, V, of the user and projecting a vector, {overscore (T)}, from the image's ideal viewing point, IVP, to the current viewpoint, V; determining a new vanishing point for the two-dimensional image in accordance with the following relationship P′=P+{overscore (T)}; distorting the triangular regions in the space of the three-dimensional world such that they intersect at the new vanishing point, P′; segmenting each of said triangular regions into a plurality of sections; translating each of said plurality of sections of said triangular regions towards, or away from, said current viewpoint of the user, V; and texture-mapping the two-dimensional image in the triangular regions onto the corresponding sections of said triangular regions.
Claim 15
Independent Claim
257 words
A method for use in processing a view of a three-dimensional world in which a first portion of said world is modeled as computer graphics and a second portion of said world is represented by a two-dimensional image texture-mapped on a panel, said two-dimensional image including an object depicted in perspective, said image being such that features of the object closer to a predetermined point of the image are farther away from a user's viewpoint, comprising the steps of: determining a vector, {overscore (C)}, corresponding to the direction of said perspective object in the three-dimensional world; projecting towards said panel the vector, {overscore (C)}, from the two-dimensional image's ideal viewing point, IVP, the intersection of said vector, {overscore (C)}, with the panel being denoted as the image's vanishing point, P; segmenting said panel into triangular regions intersecting at the image's vanishing point, P; determining the current viewpoint, V, of the user and projecting a vector, {overscore (T)}, from the image's ideal viewing point, IVP, to the current viewpoint, V; determining a new vanishing point for the two-dimensional image in accordance with the following relationship P′=P+{overscore (T)}; distorting the triangular regions in the space of the three-dimensional world such that they intersect at the new vanishing point, P′; segmenting each of said triangular regions into a plurality of sections; translating each of said plurality of sections of said triangular regions towards, or away from, said current viewpoint of the user, V; and texture-mapping the two-dimensional image in the triangular regions onto the corresponding sections of said triangular regions.
Claim 16
Dependent Claim
45 words
The invention as defined in claim 13 wherein said segmenting step of said triangular regions includes resegmenting said triangular regions into a greater number of smaller sections until the depth profile of the pyramidic panel structure formed from said triangular regions reaches a predetermined level.
Claim 17
Dependent Claim
20 words
The invention as defined in claim 13 wherein said predetermined point is substantially near the center of the two-dimensional image.
Claim 18
Dependent Claim
28 words
The invention as defined in claim 13 further comprising displaying the texture-mapped two-dimensional image merged with the first portion of said world that is modeled as computer graphics.
Claim 19
Dependent Claim
28 words
The invention as defined in claim 13 further comprising the step of calibrating the two-dimensional image as a function of the dimensions of the surroundings within the world.
Claim 20
Dependent Claim
31 words
The invention as defined in claim 13 wherein an outer edge of each of said plurality of sections of said triangular regions is located on the surface of a predetermined plane.
Claim 21
Dependent Claim
21 words
The invention as defined in claims 18 wherein said predetermined plane is the panel onto which the two-dimensional image is texture-mapped.
Claim 22
Dependent Claim
30 words
The invention as defined in claim 13 wherein the center of each of said plurality of sections of said triangular regions is located on the surface of a predetermined plane.
Claim 23
Dependent Claim
21 words
The invention as defined in claim 20 wherein said predetermined plane is the panel onto which the two-dimensional image is texture-mapped.
Claim 24
Dependent Claim
82 words
The invention as defined in claim 13 further comprising scaling each of said plurality of sections of said triangular regions in accordance with the following relationship S t = S p T t T p , where S p is the size of the section; S t is the size of the translated section; T p is distance to the section from the user's viewpoint, V; and T t is the distance to the translated section from the user's viewpoint, V.
Claim 25
Independent Claim
157 words
An apparatus for use in processing a view of a three-dimensional world in which a first portion of said world is modeled as computer graphics and a second portion of said world is represented by a two-dimensional perspective image, said apparatus comprising: means for determining the current viewpoint of the user, V; means for dividing the panel into triangular regions; as the user moves within the three-dimensional world, means for dynamically distorting the triangular regions to form pyramidic panels such that a corresponding vanishing point, P, of a portion of the two-dimensional image moves as a function of the current viewpoint of the user; means for segmenting each of said pyramidic panels into a plurality of sections; means for translating each of said plurality of sections of said pyramidic panels towards, or away from, said current viewpoint of the user, V; and means for texture-mapping the two-dimensional image onto the plurality of sections of the pyramidic panels.
Claim 26
Independent Claim
157 words
An apparatus for use in processing a view of a three-dimensional world in which a first portion of said world is modeled as computer graphics and a second portion of said world is represented by a two-dimensional perspective image, said apparatus comprising: means for determining the current viewpoint of the user, V; means for dividing the panel into triangular regions; as the user moves within the three-dimensional world, means for dynamically distorting the triangular regions to form pyramidic panels such that a corresponding vanishing point, P, of a portion of the two-dimensional image moves as a function of the current viewpoint of the user; means for segmenting each of said pyramidic panels into a plurality of sections; means for translating each of said plurality of sections of said pyramidic panels towards, or away from, said current viewpoint of the user, V; and means for texture-mapping the two-dimensional image onto the plurality of sections of the pyramidic panels.
Claim 27
Dependent Claim
31 words
The invention as defined in claim 23 wherein an outer edge of each of said plurality of sections of said pyramidic panels is located on the surface of a predetermined plane.
Claim 28
Dependent Claim
21 words
The invention as defined in claim 24 wherein said predetermined plane is the panel onto which the two-dimensional image is texture-mapped.
Claim 29
Dependent Claim
30 words
The invention as defined in claim 23 wherein the center of each of said plurality of sections of said pyramidic panels is located on the surface of a predetermined plane.
Claim 30
Dependent Claim
85 words
The invention as defined in claim 23 further comprising means for scaling each of said plurality of sections of said pyramidic panels in accordance with the following relationship S t = S p T t T p , where S p is the size of the section; S t is the size of the translated section; T p is distance to the original section from the user's viewpoint, V; and T t is the distance to the translated section from the user's viewpoint, V.
Claim 31
Dependent Claim
68 words
The invention as defined in claim 23 further comprising means for determining a vector, {overscore (C)}, corresponding to the direction of a portion of a path contained within the two-dimensional perspective image, and means for projecting toward the panel the vector, {overscore (C)}, from the image's ideal viewing point, IVP, the intersection of said vector, {overscore (C)}, with the panel being denoted as the image's vanishing point, P.
Claim 32
Dependent Claim
54 words
The invention as defined in claim 23 wherein said means for distorting the triangular regions includes means for determining a new vanishing point, P′, for said two-dimensional image in accordance with the following relationship P′=P+{overscore (T)}, wherein {overscore (T)} is a vector from the image's ideal viewing point, IVP, to the current viewpoint, V.
Claim 33
Dependent Claim
28 words
The invention as defined in claim 23 further comprising means for calibrating the two-dimensional perspective image as a function of the dimensions of the surroundings within the world.
Claim 34
Dependent Claim
18 words
The invention as defined in claim 23 wherein said two-dimensional perspective image is a frame of a video.
Claim 35
Dependent Claim
16 words
The invention as defined in claim 23 wherein said two-dimensional perspective image is a still picture.
References Cited (Backward Citations)
US5124693A
Cited
US5261041A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2004-05-21 | 2004-11-20 | 2005-05-22 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2008-05-21 | 2008-11-20 | 2009-05-22 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2012-05-21 | 2012-11-20 | 2013-05-22 | $7700.00 | Window Closed / Lapsed |