Method and apparatus of facial image conversion by interpolation/extrapolation for plurality of facial expression components representing facial image

Claim 2

An image information conversion method comprising the steps of: extracting, from facial image information of a plurality of one-dimensional different facial expressions of a predetermined human being, a plurality of facial expression components for each said facial expression corresponding to respective projective components for a plurality of coordinate axes on facial image representation space taking the entire face as a processing unit, calculating a plurality of synthesized facial components corresponding to facial image information of a synthesized facial expression at an arbitrary rate for specified arbitrary two of said different facial expressions by interpolation/extrapolation of a facial expression component corresponding to said two different facial expressions, and generating synthesized facial image information according to said plurality of synthesized facial expression components.

Claim 3

The facial image information conversion method according to claim 1, wherein said facial expression components and said synthesized facial expression components are expansion coefficients of corresponding facial image information expanded by Karhunen-Loeve Transform.

Claim 4

The facial image information conversion method according to claim 1, wherein said facial expression components and said synthesized facial expression components are expansion coefficients of corresponding facial image information expanded by discrete cosine transform.

Claim 5

A facial image information conversion method comprising the steps of: producing an average facial expression component difference between each facial expression by an average operation for a plurality of human beings according to facial expression components extracted from facial image information of a plurality of different facial expressions, deforming arbitrary applied facial image information so that a plurality of reference points establish a predetermined position relationship with the entire face taken as a processing unit to convert the same into normalization facial image information, extracting a plurality of one-dimensional facial expression components corresponding to respective projective components for a plurality of coordinate axes on facial image representation space of said normalization facial image information, calculating a plurality of synthesized facial expression components corresponding to a specified facial expression representation rate on the basis of an average facial expression component difference between each facial expression and said facial expression component corresponding to said applied facial image information, according to a facial expression of said applied facial image information and a specified facial expression, generating normalization synthesized facial image information according to said plurality of synthesized facial expression components, and deforming said normalization synthesized facial image information so that arrangement of reference points thereof are identical to the arrangement of reference points for original said applied facial image information to generate synthesized facial image information.

Claim 6

A facial image information conversion method comprising the steps of: producing an average facial expression component difference between each facial expression by an average operation for a plurality of human beings according to facial expression components extracted from facial image information of a plurality of different facial expressions, deforming arbitrary applied facial image information so that a plurality of reference points establish a predetermined position relationship with the entire face taken as a processing unit to convert the same into normalization facial image information, extracting a plurality of one-dimensional facial expression components corresponding to respective projective components for a plurality of coordinate axes on facial image representation space of said normalization facial image information, calculating a plurality of synthesized facial expression components corresponding to a specified facial expression representation rate on the basis of an average facial expression component difference between each facial expression and said facial expression component corresponding to said applied facial image information, according to a facial expression of said applied facial image information and a specified facial expression, generating normalization synthesized facial image information according to said plurality of synthesized facial expression components, and deforming said normalization synthesized facial image information so that arrangement of reference points thereof are identical to the arrangement of reference points for original said applied facial image information to generate synthesized facial image information.

Claim 7

The facial image information conversion method according to claim 4, wherein said facial expression components and said synthesized facial expression components are expansion coefficients of corresponding facial image information expanded by Karhunen-Loeve Transform.

Claim 8

The facial image information conversion method according to claim 4, wherein said facial expression components and said synthesized facial expression components are expansion coefficients of corresponding facial image information expanded by discrete cosine transform.

Claim 9

A facial image information conversion method comprising the steps of: specifying a plurality of reference points for each reference face image information with respect to a plurality of reference face image information of a different facial expression for a plurality of human beings, deforming said reference face image information so that said plurality of reference points established a predetermined position relationship with the entire face taken as a processing unit for every said reference face image information and converting the same into normalization reference facial image information, extracting a plurality of one-dimensional facial expression components corresponding to respective projective components for a plurality of coordinate axes on facial image representation space of each said normalization reference facial image information, and obtaining an average facial expression component for each said facial expression component with respect to said plurality of human beings for each facial expression and said average facial expression component difference between facial expressions, specifying a plurality of reference points for arbitrary applied facial image information with the entire face taken as a processing unit, deforming said applied facial image information so that said plurality of reference points establish a predetermined position relationship with the entire face taken as a processing unit, and converting the same into normalization facial image information, extracting a plurality of facial expression components corresponding to respective projective components for a plurality of coordinate axes on image representation space of said normalization facial image information, calculating a plurality of synthesized facial expression components corresponding to a specified facial expression component from said average facial expression component difference and said facial expression component corresponding to said applied facial image information, according to a facial expression of said applied facial image information and a specified facial expression, generating normalization synthesized facial image information according to said plurality of synthesized facial expression components, and deforming said normalization synthesized facial image information so that arrangement of reference points thereof are identical to arrangement of reference points for original said applied facial image information to generate synthesized facial image information.

Claim 10

A facial image information conversion method comprising the steps of: specifying a plurality of reference points for each reference face image information with respect to a plurality of reference face image information of a different facial expression for a plurality of human beings, deforming said reference face image information so that said plurality of reference points established a predetermined position relationship with the entire face taken as a processing unit for every said reference face image information and converting the same into normalization reference facial image information, extracting a plurality of one-dimensional facial expression components corresponding to respective projective components for a plurality of coordinate axes on facial image representation space of each said normalization reference facial image information, and obtaining an average facial expression component for each said facial expression component with respect to said plurality of human beings for each facial expression and said average facial expression component difference between facial expressions, specifying a plurality of reference points for arbitrary applied facial image information with the entire face taken as a processing unit, deforming said applied facial image information so that said plurality of reference points establish a predetermined position relationship with the entire face taken as a processing unit, and converting the same into normalization facial image information, extracting a plurality of facial expression components corresponding to respective projective components for a plurality of coordinate axes on image representation space of said normalization facial image information, calculating a plurality of synthesized facial expression components corresponding to a specified facial expression component from said average facial expression component difference and said facial expression component corresponding to said applied facial image information, according to a facial expression of said applied facial image information and a specified facial expression, generating normalization synthesized facial image information according to said plurality of synthesized facial expression components, and deforming said normalization synthesized facial image information so that arrangement of reference points thereof are identical to arrangement of reference points for original said applied facial image information to generate synthesized facial image information.

Claim 11

The facial image information conversion method according to claim 7, wherein said facial expression components and said synthesized facial expression components are expansion coefficients of corresponding facial image information expanded by Karhunen-Loeve Transform.

Claim 12

The facial image information conversion method according to claim 7, wherein said facial expression components and said synthesized facial expression components are expansion coefficients of corresponding facial image information expanded by discrete cosine transform.

Claim 13

An image information conversion apparatus comprising: means for extracting, from facial image information of a plurality of different facial expressions of a predetermined human being, a plurality of one-dimensional facial expression components for each said facial expression corresponding to respective projective components for a plurality of coordinate axes on facial image representation space with the entire face taken as a processing unit, means for calculating a plurality of synthesized facial expression components corresponding to facial image information of a synthesized facial expression at an arbitrary rate for specified arbitrary two of said different facial expressions by interpolation/extrapolation of a facial expression component corresponding to said two different facial expressions, and generating synthesized facial image information according to said plurality of synthesized facial expression components.

Claim 14

An image information conversion apparatus comprising: means for extracting, from facial image information of a plurality of different facial expressions of a predetermined human being, a plurality of one-dimensional facial expression components for each said facial expression corresponding to respective projective components for a plurality of coordinate axes on facial image representation space with the entire face taken as a processing unit, means for calculating a plurality of synthesized facial expression components corresponding to facial image information of a synthesized facial expression at an arbitrary rate for specified arbitrary two of said different facial expressions by interpolation/extrapolation of a facial expression component corresponding to said two different facial expressions, and generating synthesized facial image information according to said plurality of synthesized facial expression components.

Claim 15

The facial image information conversion apparatus according to claim 10, wherein said facial expression components and said synthesized facial expression components are expansion coefficients of corresponding facial image information expanded by Karhunen-Loeve Transform.

Claim 16

The facial image information conversion apparatus according to claim 10, wherein said facial expression components and said synthesized facial expression components are expansion coefficients of corresponding facial image information expanded by discrete cosine transform.

Claim 17

A facial image information conversion apparatus comprising: storage means for storing an average facial expression component difference between each facial expression provided by an average operation for a plurality of human beings according to facial expression components extracted from facial image information of a plurality of different facial expressions, means for deforming arbitrary applied facial image information so that a plurality of reference points establish a predetermined position relationship with the entire face taken as a processing unit, means for extracting a plurality of one-dimensional facial expression components corresponding to respective projective components for a plurality of coordinate axes on facial image representation space of said normalization facial image information, means for calculating a plurality of synthesized facial expression components corresponding to a specified facial expression representation rate from an average facial expression component difference between each facial expression and said facial expression component corresponding to said applied facial image information according to a facial expression of said applied facial image information and a specified facial expression, means for generating normalization synthesized facial image information according to said plurality of synthesized facial expression components, and deforming said normalization synthesized facial image information so that arrangement of reference points of said normalization synthesized facial image information is identical to the arrangement of reference points of original said applied facial image information.

Claim 18

A facial image information conversion apparatus comprising: storage means for storing an average facial expression component difference between each facial expression provided by an average operation for a plurality of human beings according to facial expression components extracted from facial image information of a plurality of different facial expressions, means for deforming arbitrary applied facial image information so that a plurality of reference points establish a predetermined position relationship with the entire face taken as a processing unit, means for extracting a plurality of one-dimensional facial expression components corresponding to respective projective components for a plurality of coordinate axes on facial image representation space of said normalization facial image information, means for calculating a plurality of synthesized facial expression components corresponding to a specified facial expression representation rate from an average facial expression component difference between each facial expression and said facial expression component corresponding to said applied facial image information according to a facial expression of said applied facial image information and a specified facial expression, means for generating normalization synthesized facial image information according to said plurality of synthesized facial expression components, and deforming said normalization synthesized facial image information so that arrangement of reference points of said normalization synthesized facial image information is identical to the arrangement of reference points of original said applied facial image information.

Claim 19

The facial image information conversion apparatus according to claim 13, wherein said facial expression components and said synthesized facial expression components are expansion coefficients of corresponding facial image information expanded by Karhunen-Loeve Transform.

Claim 20

The facial image information conversion apparatus according to claim 13, wherein said facial expression components and said synthesized facial expression components are expansion coefficients of corresponding facial image information expanded by discrete cosine transform.