Picture generator and picture generation method

Claim 2

A picture generator, comprising: mobile photographing means for photographing a performer in front of a predetermined background having a predetermined color to obtain first video data, said background including a plurality of inconspicuous markers each having a color similar to the predetermined color of the predetermined background; camera parameter calculating means for acquiring viewpoint information of said photographing means including three-dimensional position and orientation of the photographing means on the basis of predetermined world coordinates; first video data generating means for generating second video data corresponding to a projected picture of a predetermined object to which perspective transformation in a case viewed from the viewpoint of said photographing means acquired in said camera parameter calculating means is applied; data delay means for acquiring third video data delayed corresponding to time for processing said first video data in said camera parameter calculating means and said first video data generating means; first key signal generating means for generating a first key signal for separating a first picture element having the predetermined color of the predetermined background or a color similar thereto and a second picture element which is not the predetermined color of the predetermined background, based on said first video data; second key generating means for generating a second key signal for separating the first picture element having the predetermined color of the predetermined background or a color similar thereto and the second picture element which is not the predetermined color of the predetermined background, based on said first video data and for acquiring a picture switching signal for fetching either of said first video data or said second video data based upon said first video data; first video data synthesizing means for acquiring first synthetic video data by synthesizing said third video data and said second video data on the basis of said first key signal; second video data synthesizing means for acquiring second synthetic video data by switching between said first video data and said second video data on the basis of said second key signal and synthesizing them; and picture display means for displaying a picture for assisting the performance of said performer based upon said second synthetic video data.

Claim 3

A picture generator according to claim 1, wherein said mobile photographing means further comprises a turning base, on which the mobile photographing means is put, which turns said mobile photographing means using a gear, and wherein an orientation of said mobile photographing means is based upon a quantity in which said gear is rotated.

Claim 4

A picture generator comprising: mobile photographing means for photographing a performer in front of a predetermined background having a predetermined color to obtain first video data, said background including a plurality of inconspicuous markers each having a color similar to the predetermined color of the predetermined background; camera parameter calculating means for acquiring viewpoint information of said photographing means including three-dimensional position and orientation of the photographing means on the basis of predetermined world coordinates using three-dimensional position and orientation of said markers obtained from said first video data on the basis of coordinates relative to said photographing means, and predetermined three-dimensional position information of said markers relative to said world coordinates; first video data generating means for generating second video data corresponding to a projected picture of a predetermined object to which perspective transformation in a case viewed from the viewpoint of said photographing means acquired in said camera parameter calculating means is applied; camera parameter forecasting means for forecasting the viewpoint of said photographing means at current time or at time posterior to the current time based upon the viewpoint information of said photographing means acquired in said camera parameter calculating means; second video data generating means for generating fourth video data corresponding to the projected picture of said predetermined object to which perspective transformation in a case viewed from the viewpoint forecast in said camera parameter forecasting means of said photographing means is applied; data delay means for acquiring third video data delayed corresponding to time for processing said first video data in said camera parameter calculating means and said first video data generating means; first key signal generating means for generating a first key signal for separating a first picture element having the predetermined color of the predetermined background or a color similar thereto and a second picture element which is not the predetermined color of the predetermined background, based on said first video data; second key generating means for generating a second key signal for separating the first picture element having the predetermined color of the predetermined background or a color similar thereto and the second picture element which is not the predetermined color of the predetermined background, based on said first video data and for acquiring a picture switching signal for fetching either of said first video data or said fourth video data based upon said first video data; first video data synthesizing means for acquiring first synthetic video data by synthesizing said third video data and said second video data on the basis of said first key signal; second video data synthesizing means for acquiring second synthetic video data by switching between said first video data and said fourth video data on the basis of said second key signal and synthesizing them; and picture display means for displaying a picture for assisting the performance of said performer based upon said second synthetic video data.

Claim 5

A picture generator comprising: mobile photographing means for photographing a performer in front of a predetermined background having a predetermined color to obtain first video data, said background including a plurality of inconspicuous markers each having a color similar to the predetermined color of the predetermined background; camera parameter calculating means for acquiring viewpoint information of said photographing means including three-dimensional position and orientation of the photographing means on the basis of predetermined world coordinates using three-dimensional position and orientation of said markers obtained from said first video data on the basis of coordinates relative to said photographing means, and predetermined three-dimensional position information of said markers relative to said world coordinates; first video data generating means for generating second video data corresponding to a projected picture of a predetermined object to which perspective transformation in a case viewed from the viewpoint of said photographing means acquired in said camera parameter calculating means is applied; camera parameter forecasting means for forecasting the viewpoint of said photographing means at current time or at time posterior to the current time based upon the viewpoint information of said photographing means acquired in said camera parameter calculating means; second video data generating means for generating fourth video data corresponding to the projected picture of said predetermined object to which perspective transformation in a case viewed from the viewpoint forecast in said camera parameter forecasting means of said photographing means is applied; data delay means for acquiring third video data delayed corresponding to time for processing said first video data in said camera parameter calculating means and said first video data generating means; first key signal generating means for generating a first key signal for separating a first picture element having the predetermined color of the predetermined background or a color similar thereto and a second picture element which is not the predetermined color of the predetermined background, based on said first video data; second key generating means for generating a second key signal for separating the first picture element having the predetermined color of the predetermined background or a color similar thereto and the second picture element which is not the predetermined color of the predetermined background, based on said first video data and for acquiring a picture switching signal for fetching either of said first video data or said fourth video data based upon said first video data; first video data synthesizing means for acquiring first synthetic video data by synthesizing said third video data and said second video data on the basis of said first key signal; second video data synthesizing means for acquiring second synthetic video data by switching between said first video data and said fourth video data on the basis of said second key signal and synthesizing them; and picture display means for displaying a picture for assisting the performance of said performer based upon said second synthetic video data.

Claim 6

The picture generator according to claim 3, wherein said camera parameter forecasting means linearly forecasts a position of the photographing means at current time based upon positional information (T1, T2) of the photographing means at the past two times according to a formula T2+(T2-T1), and wherein said camera parameter forecasting means linearly forecasts an orientation of the photographing means at current time based upon orientation information (R1, R2) of the photographing means at the past two times according to a formula R2+(R2-R1).

Claim 7

A picture generation method, comprising the steps of: acquiring first video data by photographing a performer in front of a predetermined background having a predetermined color with mobile photographing means, said background including a plurality of inconspicuous markers each having color similar to the predetermined color of the background; acquiring viewpoint information of said photographing means including three-dimensional position and orientation of the photographing means on the basis of predetermined world coordinates using three-dimensional position information of said markers obtained from said first video data on the basis of coordinates relative to said photographing means, and predetermined three-dimensional position information of said markers relative to said world coordinates; generating second video data corresponding to a projected picture of a predetermined object to which perspective transformation in a case viewed from the viewpoint of said photographing means is applied; acquiring third video data by delaying said first video data corresponding to processing time in the step for acquiring the viewpoint information of said photographing means and the step for generating said second video data; generating a first key signal for separating a first picture element having the predetermined color of the predetermined background or a color similar thereto and a second picture element which is not the predetermined color of the predetermined background, based on said first video data; generating a second key signal for separating the first picture element having the predetermined color of the predetermined background or a color similar thereto and the second picture element which is not the predetermined color of the predetermined background, based on said first video data and for acquiring a picture switching signal for fetching either of said first video data or said second video data based upon said first video data; acquiring first synthetic video data by synthesizing said third video data and said second video data on the basis of said first key signal; acquiring second synthetic video data by switching between said first video data and said second video data on the basis of said second key signal and synthesizing them; and displaying a picture for assisting the performance of said performer based upon said second synthetic video data.

Claim 8

A picture generation method, comprising the steps of: acquiring first video data by photographing a performer in front of a predetermined background having a predetermined color with mobile photographing means, said background including a plurality of inconspicuous markers each having color similar to the predetermined color of the background; acquiring viewpoint information of said photographing means including three-dimensional position and orientation of the photographing means on the basis of predetermined world coordinates using three-dimensional position information of said markers obtained from said first video data on the basis of coordinates relative to said photographing means, and predetermined three-dimensional position information of said markers relative to said world coordinates; generating second video data corresponding to a projected picture of a predetermined object to which perspective transformation in a case viewed from the viewpoint of said photographing means is applied; acquiring third video data by delaying said first video data corresponding to processing time in the step for acquiring the viewpoint information of said photographing means and the step for generating said second video data; generating a first key signal for separating a first picture element having the predetermined color of the predetermined background or a color similar thereto and a second picture element which is not the predetermined color of the predetermined background, based on said first video data; generating a second key signal for separating the first picture element having the predetermined color of the predetermined background or a color similar thereto and the second picture element which is not the predetermined color of the predetermined background, based on said first video data and for acquiring a picture switching signal for fetching either of said first video data or said second video data based upon said first video data; acquiring first synthetic video data by synthesizing said third video data and said second video data on the basis of said first key signal; acquiring second synthetic video data by switching between said first video data and said second video data on the basis of said second key signal and synthesizing them; and displaying a picture for assisting the performance of said performer based upon said second synthetic video data.

Claim 9

The picture generating method of claim 5, further comprising the steps of: turning said mobile photographing means using a gear, wherein the orientation of the photographing means is based upon a quantity in which the gear is rotated.

Claim 10

A picture generation method comprising the steps of: acquiring first video data by photographing a performer in front of a predetermined background having a predetermined color with mobile photographing means said background including a plurality of inconspicuous markers each having color similar to the predetermined color of the background; acquiring viewpoint information of said photographing means including three-dimensional position and orientation of the photographing means on the basis of predetermined world coordinates using three-dimensional position information of said markers obtained from said first video data on the basis of coordinates relative to said photographing means, and predetermined three-dimensional position information of said markers relative to said world coordinates; generating second video data corresponding to a projected picture of a predetermined object to which perspective transformation in a case viewed from the viewpoint of said photographing means is applied; acquiring third video data by delaying said first video data corresponding to processing time in the step for acquiring the viewpoint information of said photographing means and the step for generating said second video data; forecasting a viewpoint of said photographing means at current time or at time posterior to the current time based upon the viewpoint information of said photographing means acquired in the step for acquiring the viewpoint of said photographing means; generating fourth video data corresponding to the projected picture of said predetermined object to which perspective transformation in a case viewed from said forecast viewpoint of said photographing means is applied; generating a first key signal for separating a first picture element having the predetermined color of the predetermined background or a color similar thereto and a second picture element which is not the predetermined color of the predetermined background, based on said first video data; generating a second key signal for separating the first picture element having the predetermined color of the predetermined background or a color similar thereto and the second picture element which is not the predetermined color of the predetermined background, based on said first video data and for acquiring a picture switching signal for fetching either of said first video data or said fourth video data based upon said first video data; acquiring first synthetic video data by synthesizing said third video data and said second video data on the basis of said first key signal; acquiring second synthetic video data by switching between said first video data and said fourth video data on the basis of said second key signal and synthesizing them; and displaying a picture for assisting the performance of said performer based upon said second synthetic video data.

Claim 11

A picture generation method comprising the steps of: acquiring first video data by photographing a performer in front of a predetermined background having a predetermined color with mobile photographing means said background including a plurality of inconspicuous markers each having color similar to the predetermined color of the background; acquiring viewpoint information of said photographing means including three-dimensional position and orientation of the photographing means on the basis of predetermined world coordinates using three-dimensional position information of said markers obtained from said first video data on the basis of coordinates relative to said photographing means, and predetermined three-dimensional position information of said markers relative to said world coordinates; generating second video data corresponding to a projected picture of a predetermined object to which perspective transformation in a case viewed from the viewpoint of said photographing means is applied; acquiring third video data by delaying said first video data corresponding to processing time in the step for acquiring the viewpoint information of said photographing means and the step for generating said second video data; forecasting a viewpoint of said photographing means at current time or at time posterior to the current time based upon the viewpoint information of said photographing means acquired in the step for acquiring the viewpoint of said photographing means; generating fourth video data corresponding to the projected picture of said predetermined object to which perspective transformation in a case viewed from said forecast viewpoint of said photographing means is applied; generating a first key signal for separating a first picture element having the predetermined color of the predetermined background or a color similar thereto and a second picture element which is not the predetermined color of the predetermined background, based on said first video data; generating a second key signal for separating the first picture element having the predetermined color of the predetermined background or a color similar thereto and the second picture element which is not the predetermined color of the predetermined background, based on said first video data and for acquiring a picture switching signal for fetching either of said first video data or said fourth video data based upon said first video data; acquiring first synthetic video data by synthesizing said third video data and said second video data on the basis of said first key signal; acquiring second synthetic video data by switching between said first video data and said fourth video data on the basis of said second key signal and synthesizing them; and displaying a picture for assisting the performance of said performer based upon said second synthetic video data.

Claim 12

The picture generating of claim 7, wherein said step of forecasting the viewpoint of said photographing means further comprising the steps of: linearly forecasting a position of the photographing means at current time based upon positional information (T1, T2) of the photographing means at the past two times according to a formula T2+(T2-T1), and linearly forecasting an orientation of the photographing means at current time based upon orientation information (R1, R2) of the photographing means at the past two times according to a formula R2+(R2-R1).

Claim 13

A picture generator, comprising: a mobile camera for photographing a performer in front of a predetermined background having a predetermined color to obtain first video data, said background including a plurality of inconspicuous markers each having a color similar to the predetermined color of the predetermined background; a camera parameter calculator for acquiring viewpoint information of said mobile camera including three-dimensional position and orientation of the mobile camera on the basis of predetermined world coordinates; a first video data generator for generating second video data corresponding to a projected picture of a predetermined object to which perspective transformation in a case viewed from the viewpoint of said mobile camera acquired in said camera parameter calculator is applied; a data delay device for acquiring third video data delayed corresponding to time for processing said first video data in said camera parameter calculator and said first video data generator; a first key signal generator for generating a first key signal for separating a first picture element having the predetermined color of the predetermined background or a color similar thereto and a second picture element which is not the predetermined color of the predetermined background, based on said first video data; a second key generator for generating a second key signal for separating the first picture element having the predetermined color of the predetermined background or a color similar thereto and the second picture element which is not the predetermined color of the predetermined background, based on said first video data and for acquiring a picture switching signal for fetching either of said first video data or said second video data based upon said first video data; a first video data synthesizer for acquiring first synthetic video data by synthesizing said third video data and said second video data on the basis of said first key signal; a second video data synthesizer for acquiring second synthetic video data by switching between said first video data and said second video data on the basis of said second key signal and synthesizing them; and a picture display for displaying a picture for assisting the performance of said performer based upon said second synthetic video data.

Claim 14

A picture generator, comprising: a mobile camera for photographing a performer in front of a predetermined background having a predetermined color to obtain first video data, said background including a plurality of inconspicuous markers each having a color similar to the predetermined color of the predetermined background; a camera parameter calculator for acquiring viewpoint information of said mobile camera including three-dimensional position and orientation of the mobile camera on the basis of predetermined world coordinates; a first video data generator for generating second video data corresponding to a projected picture of a predetermined object to which perspective transformation in a case viewed from the viewpoint of said mobile camera acquired in said camera parameter calculator is applied; a data delay device for acquiring third video data delayed corresponding to time for processing said first video data in said camera parameter calculator and said first video data generator; a first key signal generator for generating a first key signal for separating a first picture element having the predetermined color of the predetermined background or a color similar thereto and a second picture element which is not the predetermined color of the predetermined background, based on said first video data; a second key generator for generating a second key signal for separating the first picture element having the predetermined color of the predetermined background or a color similar thereto and the second picture element which is not the predetermined color of the predetermined background, based on said first video data and for acquiring a picture switching signal for fetching either of said first video data or said second video data based upon said first video data; a first video data synthesizer for acquiring first synthetic video data by synthesizing said third video data and said second video data on the basis of said first key signal; a second video data synthesizer for acquiring second synthetic video data by switching between said first video data and said second video data on the basis of said second key signal and synthesizing them; and a picture display for displaying a picture for assisting the performance of said performer based upon said second synthetic video data.

Claim 15

A picture generator according to claim 9, wherein said mobile camera further comprises a turning base, on which the mobile camera is put, which turns said mobile camera using a gear, and wherein an orientation of said mobile camera is based upon a quantity in which said gear is rotated.

Claim 16

A picture generator comprising: a mobile camera for photographing a performer in front of a predetermined background having a predetermined color to obtain first video data, said background including a plurality of inconspicuous markers each having a color similar to the predetermined color of the predetermined background; a camera parameter calculator for acquiring viewpoint information of said mobile camera including three-dimensional position and orientation of the mobile camera on the basis of predetermined world coordinates using three-dimensional position and orientation of said markers obtained from said first video data on the basis of coordinates relative to said mobile camera, and predetermined three-dimensional position information of said markers relative to said world coordinates; a first video data generator for generating second video data corresponding to a projected picture of a predetermined object to which perspective transformation in a case viewed from the viewpoint of said mobile camera acquired in said camera parameter calculator is applied; a camera parameter forecaster for forecasting the viewpoint of said mobile camera at current time or at time posterior to the current time based upon the viewpoint information of said mobile camera acquired in said camera parameter calculator; a second video data generator for generating fourth video data corresponding to the projected picture of said predetermined object to which perspective transformation in a case viewed from the viewpoint forecast in said camera parameter forecaster of said mobile camera is applied; a data delay device for acquiring third video data delayed corresponding to time for processing said first video data in said camera parameter calculator and said first video data generator; a first key signal generator for generating a first key signal for separating a first picture element having the predetermined color of the predetermined background or a color similar thereto and a second picture element which is not the predetermined color of the predetermined background, based on said first video data; a second key generating means for generating a second key signal for separating the first picture element having the predetermined color of the predetermined background or a color similar thereto and the second picture element which is not the predetermined color of the predetermined background, based on said first video data and for acquiring a picture switching signal for fetching either of said first video data or said fourth video data based upon said first video data; a first video data synthesizer for acquiring first synthetic video data by synthesizing said third video data and said fourth video data on the basis of said first key signal; a second video data synthesizer for acquiring second synthetic video data by switching between said first video data and said fourth video data on the basis of said second key signal and synthesizing them; and a picture display for displaying a picture for assisting the performance of said performer based upon said second synthetic video data.

Claim 17

A picture generator comprising: a mobile camera for photographing a performer in front of a predetermined background having a predetermined color to obtain first video data, said background including a plurality of inconspicuous markers each having a color similar to the predetermined color of the predetermined background; a camera parameter calculator for acquiring viewpoint information of said mobile camera including three-dimensional position and orientation of the mobile camera on the basis of predetermined world coordinates using three-dimensional position and orientation of said markers obtained from said first video data on the basis of coordinates relative to said mobile camera, and predetermined three-dimensional position information of said markers relative to said world coordinates; a first video data generator for generating second video data corresponding to a projected picture of a predetermined object to which perspective transformation in a case viewed from the viewpoint of said mobile camera acquired in said camera parameter calculator is applied; a camera parameter forecaster for forecasting the viewpoint of said mobile camera at current time or at time posterior to the current time based upon the viewpoint information of said mobile camera acquired in said camera parameter calculator; a second video data generator for generating fourth video data corresponding to the projected picture of said predetermined object to which perspective transformation in a case viewed from the viewpoint forecast in said camera parameter forecaster of said mobile camera is applied; a data delay device for acquiring third video data delayed corresponding to time for processing said first video data in said camera parameter calculator and said first video data generator; a first key signal generator for generating a first key signal for separating a first picture element having the predetermined color of the predetermined background or a color similar thereto and a second picture element which is not the predetermined color of the predetermined background, based on said first video data; a second key generating means for generating a second key signal for separating the first picture element having the predetermined color of the predetermined background or a color similar thereto and the second picture element which is not the predetermined color of the predetermined background, based on said first video data and for acquiring a picture switching signal for fetching either of said first video data or said fourth video data based upon said first video data; a first video data synthesizer for acquiring first synthetic video data by synthesizing said third video data and said fourth video data on the basis of said first key signal; a second video data synthesizer for acquiring second synthetic video data by switching between said first video data and said fourth video data on the basis of said second key signal and synthesizing them; and a picture display for displaying a picture for assisting the performance of said performer based upon said second synthetic video data.

Claim 18

The picture generator according to claim 11, wherein said camera parameter forecaster linearly forecasts a position of the mobile camera at current time based upon positional information (T1, T2) of the mobile camera at the past two times according to a formula T2+(T2-T1), and wherein said camera parameter forecaster linearly forecasts an orientation of the mobile camera at current time based upon orientation information (R1, R2) of the mobile camera at the past two times according to a formula R2+(R2-R1).