US20030197779A1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Video-teleconferencing system with eye-gaze correction
Assignee: Microsoft Technology Licensing LLC
•
Filing: 2002-04-23
•
Granted: —
•
Expires: 2022-04-23
Abstract
Abstract Correcting for eye-gaze in video communication devices is accomplished by blending information captured from a stereoscopic view of the conferee and generating a virtual image of the conferee. A personalized face model of the conferee is captured to track head position of the conferee. First and second video images representative of a first conferee taken from different views are concurrently captured. A head position of the first conferee is tracked from the first and second video images. Matching features and contours from the first and second video images are ascertained. The head position as well as the matching features and contours from the first and second video images are synthesized to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Inventors
- 👤 Zhengyou Zhang
- 👤 Ruigang Yang
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:43
CPC Classifications
H; H04; H04N; H04N7/00; H04N7/14; H04N7/141; H04N7/142; H04N7/144
Claims Analysis
Total Claims: 47
Claim 1
Independent Claim
110 words
1 . A method, comprising: concurrently, capturing first and second video images representative of a first conferee taken from different views; tracking a head position of the first conferee from the first and second video images; ascertaining features and contours from the first video image that match features and contours from the second video image; and synthesizing the head position as well as the features and contours from the first and second video images that match to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 2
Dependent Claim
110 words
1 . A method, comprising: concurrently, capturing first and second video images representative of a first conferee taken from different views; tracking a head position of the first conferee from the first and second video images; ascertaining features and contours from the first video image that match features and contours from the second video image; and synthesizing the head position as well as the features and contours from the first and second video images that match to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 3
Dependent Claim
21 words
2 . The method as recited in claim 1 , further comprising storing a personalized face model of the first conferee.
Claim 4
Dependent Claim
56 words
3 . The method as recited in claim 1 , further comprising storing a personalized face model of the first conferee and evaluating the first and second video images with respect to the personalized face model of the first conferee to monitor feature points from the first and second video images to track the head position.
Claim 5
Dependent Claim
51 words
4 . The method as recited in claim 1 , wherein ascertaining features from the first video image that matches features from the second video image comprises rectifying the first and second video images and locating features from the first and second video images that reside on epipolar lines that match.
Claim 6
Dependent Claim
51 words
5 . The method as recited in claim 1 , wherein ascertaining contours from the first video image that matches contours from the second video image comprises rectifying the first and second video images and locating contours from the first and second video images that reside on epipolar lines that match.
Claim 7
Dependent Claim
58 words
6 . The method as recited in claim 1 , wherein synthesizing the head position as well as the features and contours from the first and second video images comprises morphing the head position as well as the features and contours from the first and second images to generate the virtual image video stream of the first conferee.
Claim 8
Dependent Claim
62 words
7 . The method as recited in claim 1 , wherein synthesizing the head position as well as the features and contours from the first and second video images comprises blending multi-texture features associated with the head position as well as the features and contours from the first and second images to generate the virtual image video stream of the first conferee.
Claim 9
Dependent Claim
22 words
8 . One or more computer-readable media comprising computer-executable instructions that, when executed, perform the method as recited in claim 1 .
Claim 10
Dependent Claim
93 words
9 . A method, comprising: storing a personalized face model of a first conferee; concurrently, capturing first and second video images representative of the first conferee taken from different views; evaluating the first and second video images with respect to the personalized face model of the first conferee to ascertain three dimensional information; and synthesizing the three dimensional information to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 11
Dependent Claim
93 words
9 . A method, comprising: storing a personalized face model of a first conferee; concurrently, capturing first and second video images representative of the first conferee taken from different views; evaluating the first and second video images with respect to the personalized face model of the first conferee to ascertain three dimensional information; and synthesizing the three dimensional information to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 12
Dependent Claim
50 words
10 . The method as recited in claim 9 , wherein evaluating the first and second video images with respect to the personalized face model of the first conferee further comprises tracking feature points from the first and second video images to monitor a head position of the first conferee.
Claim 13
Dependent Claim
31 words
11 . The method as recited in claim 9 , further comprising ascertaining features and contours from the first video image that match features and contours from the second video image.
Claim 14
Dependent Claim
52 words
12 . The method as recited in claim 9 , further comprising ascertaining features and contours from the first video image that match features and contours from the second video image and synthesizing the three dimensional information with features and contours to generate the virtual image video stream of the first conferee.
Claim 15
Dependent Claim
102 words
13 . The method as recited in claim 9 , further comprising ascertaining features and contours from the first video image that match features and contours from the second video image and synthesizing the three dimensional information with features and contours to generate the virtual image video stream of the first conferee, and wherein synthesizing the three dimensional information as well as the features and contours from the first and second video images comprises morphing the head position as well as the features and contours from the first and second images to generate the virtual image video stream of the first conferee.
Claim 16
Dependent Claim
106 words
14 . The method as recited in claim 9 , further comprising ascertaining features and contours from the first video image that match features and contours from the second video image and synthesizing the three dimensional information with features and contours to generate the virtual image video stream of the first conferee, and wherein synthesizing the three dimensional information as well as the features and contours from the first and second video images comprises blending multi-texture features associated with the head position as well as the features and contours from the first and second images to generate the virtual image video stream of the first conferee.
Claim 17
Dependent Claim
22 words
15 . One or more computer-readable media comprising computer-executable instructions that, when executed, perform the method as recited in claim 9 .
Claim 18
Dependent Claim
118 words
16 . A system, comprising: means for concurrently capturing first and second video images representative of a first conferee taken from different views; means for tracking a head position of the first conferee from the first and second video images; means for ascertaining features and contours from the first video image that match features and contours from the second video image; and means for synthesizing the head position as well as the features and contours from the first and second video images that match to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 19
Dependent Claim
118 words
16 . A system, comprising: means for concurrently capturing first and second video images representative of a first conferee taken from different views; means for tracking a head position of the first conferee from the first and second video images; means for ascertaining features and contours from the first video image that match features and contours from the second video image; and means for synthesizing the head position as well as the features and contours from the first and second video images that match to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 20
Dependent Claim
27 words
17 . The system as recited in claim 16 , further comprising means for storing a personalized face model of the first conferee in a memory device.
Claim 21
Dependent Claim
64 words
18 . The system as recited in claim 16 , further comprising means for storing a personalized face model of the first conferee in a memory device and means for evaluating the first and second video images with respect to the personalized face model of the first conferee to monitor feature points from the first and second video images to track the head position.
Claim 22
Dependent Claim
58 words
19 . The system as recited in claim 16 , wherein the means for ascertaining features from the first video image that matches features from the second video image comprises means for rectifying the first and second video images and means for locating features from the first and second video images that reside on epipolar lines that match.
Claim 23
Dependent Claim
58 words
20 . The system as recited in claim 16 , wherein the means for ascertaining contours from the first video image that matches contours from the second video image comprises means for rectifying the first and second video images and means for locating contours from the first and second video images that reside on epipolar lines that match.
Claim 24
Dependent Claim
63 words
21 . The system as recited in claim 16 , wherein the means for synthesizing the head position as well as the features and contours from the first and second video images comprises means for morphing the head position as well as the features and contours from the first and second images to generate the virtual image video stream of the first conferee.
Claim 25
Dependent Claim
67 words
22 . The system as recited in claim 16 , wherein the means for synthesizing the head position as well as the features and contours from the first and second video images comprises means for blending multi-texture features associated with the head position as well as the features and contours from the first and second images to generate the virtual image video stream of the first conferee.
Claim 26
Dependent Claim
130 words
23 . A video-teleconferencing system, comprising: a head pose tracking module, configured to receive first and second video images representative of a first conferee concurrently taken from different views and track head position of the first conferee; a stereo module, configured to receive the first and second video images representative of the first conferee concurrently taken from different views and match non-rigid parts observed from the first and second video images; and a view synthesis module, configured to synthesize the head position as well as the matching non-rigid parts from the first and second video images to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 27
Dependent Claim
130 words
23 . A video-teleconferencing system, comprising: a head pose tracking module, configured to receive first and second video images representative of a first conferee concurrently taken from different views and track head position of the first conferee; a stereo module, configured to receive the first and second video images representative of the first conferee concurrently taken from different views and match non-rigid parts observed from the first and second video images; and a view synthesis module, configured to synthesize the head position as well as the matching non-rigid parts from the first and second video images to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 28
Dependent Claim
32 words
24 . The system as recited in claim 23 , wherein the stereo module is further configured to match features of the first conferee observed from the first and second video images.
Claim 29
Dependent Claim
25 words
25 . The system as recited in claim 23 , wherein the stereo module is further configured to rectify the first and second video images.
Claim 30
Dependent Claim
30 words
26 . The system as recited in claim 23 , wherein the stereo module is further configured to perform contour matching of the of the first and second video images.
Claim 31
Dependent Claim
61 words
27 . The system as recited in claim 23 , wherein the view system module is configured to synthesize the head position as well non-rigid parts from the first and second video images by morphing the head position as well as the non-rigid parts form the first and second images to generate the virtual image video stream of the first conferee.
Claim 32
Dependent Claim
65 words
28 . The system as recited in claim 23 , wherein the view system module is configured to synthesize the head position as well non-rigid parts from the first and second video images by blending multi-texture features associated with the head position as well as the non-rigid parts form the first and second images to generate the virtual image video stream of the first conferee.
Claim 33
Dependent Claim
137 words
29 . One or more computer-readable media having stored thereon computer executable instructions that, when executed by one or more processors, causes the one or more processors of a computer system to: concurrently, capture first and second video images representative of a first conferee taken from different views; track a head position of the first conferee from the first and second video images; ascertain features and contours from the first video image that match features and contours from the second video image; and synthesize the head position as well as the features and contours from the first and second video images that match to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 34
Dependent Claim
137 words
29 . One or more computer-readable media having stored thereon computer executable instructions that, when executed by one or more processors, causes the one or more processors of a computer system to: concurrently, capture first and second video images representative of a first conferee taken from different views; track a head position of the first conferee from the first and second video images; ascertain features and contours from the first video image that match features and contours from the second video image; and synthesize the head position as well as the features and contours from the first and second video images that match to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 35
Dependent Claim
35 words
30 . One or more computer-readable media as recited in claim 29 , further comprising computer executable instructions that, when executed, direct the computer system to store a personalized face model of the first conferee.
Claim 36
Dependent Claim
70 words
31 . One or more computer-readable media as recited in claim 29 , further comprising computer executable instructions that, when executed, direct the computer system to store a personalized face model of the first conferee and evaluate the first and second video images with respect to the personalized face model of the first conferee to monitor feature points from the first and second video images to track the head position.
Claim 37
Dependent Claim
66 words
32 . One or more computer-readable media as recited in claim 29 , further comprising computer executable instructions that, when executed, direct the computer system to, rectify the first and second video images and locate features from the first and second video images that reside on epipolar lines that match when ascertaining features from the first video image that matches features from the second video image.
Claim 38
Dependent Claim
66 words
33 . One or more computer-readable media as recited in claim 29 , further comprising computer executable instructions that, when executed, direct the computer system to, rectify the first and second video images and locate contours from the first and second video images that reside on epipolar lines that match when ascertaining contours from the first video image that matches contours from the second video image.
Claim 39
Dependent Claim
73 words
34 . One or more computer-readable media as recited in claim 29 , further comprising computer executable instructions that, when executed, direct the computer system to morph the head position as well as the features and contours from the first and second images to generate the virtual image video stream of the first conferee when synthesizing the head position as well as the features and contours from the first and second video images.
Claim 40
Dependent Claim
77 words
35 . One or more computer-readable media as recited in claim 29 , further comprising computer executable instructions that, when executed, direct the computer system to blend multi-texture features associated with the head position as well as the features and contours from the first and second images to generate the virtual image video stream of the first conferee, when synthesizing the head position as well as the features and contours from the first and second video images.
Claim 41
Dependent Claim
120 words
36 . One or more computer-readable media having stored thereon computer executable instructions that, when executed by one or more processors, causes the one or more processors of a computer system to: store a personalized face model of a first conferee; concurrently, capture first and second video images representative of the first conferee taken from different views; evaluate the first and second video images with respect to the personalized face model of the first conferee to ascertain three dimensional information; and synthesize the three dimensional information to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 42
Dependent Claim
120 words
36 . One or more computer-readable media having stored thereon computer executable instructions that, when executed by one or more processors, causes the one or more processors of a computer system to: store a personalized face model of a first conferee; concurrently, capture first and second video images representative of the first conferee taken from different views; evaluate the first and second video images with respect to the personalized face model of the first conferee to ascertain three dimensional information; and synthesize the three dimensional information to generate a virtual image video stream of the first conferee that makes the first conferee appear to be making eye contact with a second conferee who is watching the virtual image video stream.
Claim 43
Dependent Claim
64 words
37 . One or more computer-readable media as recited in claim 36 , further comprising computer executable instructions that, when executed, direct the computer system to, track feature points from the first and second video images to monitor a head position of the first conferee when evaluating the first and second video images with respect to the personalized face model of the first conferee.
Claim 44
Dependent Claim
45 words
38 . One or more computer-readable media as recited in claim 36 , further comprising computer executable instructions that, when executed, direct the computer system to, ascertain features and contours from the first video image that match features and contours from the second video image.
Claim 45
Dependent Claim
66 words
39 . One or more computer-readable media as recited in claim 36 , further comprising computer executable instructions that, when executed, direct the computer system to ascertain features and contours from the first video image that match features and contours from the second video image and synthesizing the three dimensional information with features and contours to generate the virtual image video stream of the first conferee.
Claim 46
Dependent Claim
104 words
40 . One or more computer-readable media as recited in claim 36 , further comprising computer executable instructions that, when executed, direct the computer system to ascertain features and contours from the first video image that match features and contours from the second video image and synthesize the three dimensional information with features and contours, and morph the head position as well as the features and contours from the first and second images to generate the virtual image video stream of the first conferee when synthesizing the three dimensional information as well as the features and contours from the first and second video images.
Claim 47
Dependent Claim
67 words
41 . One or more computer-readable media as recited in claim 36 , further comprising computer executable instructions that, when executed, direct the computer system to, ascertain features and contours from the first video image that match features and contours from the second video image and synthesize the three dimensional information with the features and contours to generate the virtual image video stream of the first conferee.
References Cited (Backward Citations)
No citation records indexed.
USPTO Annuity & Maintenance Windows
Maintenance schedule is calculated for granted US utility patents.