Method and computer program product for determining an area of importance in an image using eye monitoring information

Claim 3

The method of claim 1 , wherein the step of determining an area of importance in the captured archival image based upon the eye information comprises determining an eye gaze direction path based upon said eye gaze direction information and identifying an area of importance based upon the determined eye gaze direction path.

Claim 4

The method of claim 1 , wherein the area of importance is determined by identifying a concentration of the eye gaze direction information within a region of the archival image.

Claim 5

The method of claim 1 , wherein the step of determining an area of importance in the captured archival image comprises determining an eye gaze direction path based upon the obtained eye gaze direction information and identifying a predetermined pattern of the eye gaze direction information.

Claim 6

The method of claim 1 , wherein the eye information further includes pupil size information captured during the image capture sequence and the step of determining an area of importance in the captured archival image based upon the eye information comprises determining the area of importance based upon the eye gaze direction information and the pupil size information.

Claim 7

The method of claim 1 , wherein the step of determining an area of importance in the captured archival image based upon the eye information comprises determining the area of importance based upon the eye information and analysis of the archival image.

Claim 8

The method of claim 1 , wherein the step of determining the area of importance in the captured archival image based upon the eye information comprises detecting a sequence of fixations in the eye gaze direction information and determining the area of importance based upon the sequence of fixations.

Claim 9

The method of claim 7 , wherein the step of determining the area of importance in the captured archival image based upon the eye information comprises detecting a sequence of fixation in the eye gaze direction information, clustering the sequence of fixations, assigning a weight to each cluster, and selecting an area of importance based upon the weights of the clusters.

Claim 10

The method of claim 8 , wherein the step of determining the weight of each cluster comprises determining the weight of each cluster based upon the order of the fixations in each cluster.

Claim 11

The method of claim 8 , wherein the step of determining the weight of each cluster further comprises determining the weight of each cluster based upon the order of fixations in each cluster and the duration of the fixations within each cluster.

Claim 12

The method of claim 8 , wherein the step of determining the weight of each cluster comprises determining the weight of each cluster based upon the duration of the fixations within each cluster.

Claim 13

The method of claim 8 , wherein the step of determining the weight of each cluster comprises determining the weight of each cluster based at least in part upon a pattern of the eye gaze direction information between clusters.

Claim 14

The method of claim 1 , wherein the step of determining the area of importance comprises identifying at least one of a pattern of positive importance and a pattern of negative importance in eye gaze direction information.

Claim 17

The method of claim 14 , wherein the step of determining an area of importance in the captured archival image based upon the eye gaze direction path relative to the captured archival image comprises determining the area of importance in the captured archival image based upon eye gaze direction information that is determined to reflect eye gaze information within the captured archival image.

Claim 18

The method of claim 14 , wherein the step of determining the eye gaze direction path relative to the captured archival image comprises determining the eye gaze direction path by adjusting the eye gaze direction information based upon the scene composition change information so that the eye gaze direction path is recorded relative to the scene image captured to obtain the captured archival image.

Claim 19

The method of claim 14 , wherein the context information comprises a composite image containing image information composited from a set of evaluation images captured during composition of the captured archival image and wherein said eye gaze direction information is recorded relative to the composite image, the method further comprising the step of registering the position of the archival image with respect to the composite image wherein the step of determining the path of eye gaze direction relative to the archival image is based upon the registered position of the archival image with respect to the composite image.

Claim 20

The method of claim 14 , wherein the context information comprises adjustable zoom setting information and wherein the step of determining a path of eye gaze direction relative to the archival image comprises determining the path of eye gaze information based upon the captured eye gaze direction, the scene composition change information and the adjustable zoom setting information.

Claim 21

The method of claim 14 , wherein the context information characterizes changes in a field of view of the camera and the step of determining a path of eye gaze direction relative to the archival image comprises adjusting the path of eye gaze direction during a composition phase of the archival image capture process to compensate for changes in the field of view of the camera.

Claim 22

The method of claim 14 , wherein the context information comprises a set of evaluation images captured during the image capture sequence, and wherein the step of determining a path of eye gaze direction relative to the archival image comprises analyzing the set of evaluation images and determining the path of eye gaze direction relative to the archival image based upon the eye gaze direction information and the analysis of the set of evaluation images.

Claim 23

The method of claim 14 , wherein the eye gaze information comprises eye information captured from more than one eye wherein the path of eye gaze direction is determined relative to the archival image based upon the captured eye gaze direction information from each eye and scene composition change information.

Claim 24

The method of claim 14 , wherein the eye information and context information are recorded on a photosensitive medium further comprising the step of extracting the eye information and context information from the photosensitive medium and associating the eye information and context information with an archival image recorded on the photosensitive medium.

Claim 25

The method of claim 14 , wherein the eye information includes at least one image of an eye of a user, further comprising the step of determining the identity of the user of the camera system based upon the eye images and to associate the user identity with the scene image.

Claim 26

The method of claim 14 , wherein the context information includes location information and wherein the steps of determining an area of importance in the archival image comprises determining the area of importance based upon the path of eye gaze direction relative to the archival image and the location information.

Claim 27

A computer program product for determining an area of importance in captured archival image, the computer program product comprising a computer readable storage medium having a computer program stored thereon for performing the steps of successively: capturing a plurality of eye images, in whole or part, when viewing a scene image to be photographed; capturing the scene image during an image capture sequence, to obtain the captured archival image; determining eye information from the plurality of eye images, including eye gaze direction information which describes eye gaze movement during the image capture sequence; determining an area of importance in the captured archival image based upon the eye information; generating area importance data; and associating the area importance data with the archival image.

Claim 28

A computer program product for determining an area of importance in captured archival image, the computer program product comprising a computer readable storage medium having a computer program stored thereon for performing the steps of successively: capturing a plurality of eye images, in whole or part, when viewing a scene image to be photographed; capturing the scene image during an image capture sequence, to obtain the captured archival image; determining eye information from the plurality of eye images, including eye gaze direction information which describes eye gaze movement during the image capture sequence; determining an area of importance in the captured archival image based upon the eye information; generating area importance data; and associating the area importance data with the archival image.

Claim 29

The computer program product of claim 25 , wherein the step of determining an area of importance in the archival image based upon the eye information comprises determining an eye gaze direction path based upon said eye gaze direction information and identifying an area of importance based upon the determined eye gaze direction path.

Claim 30

The computer program product of claim 25 , wherein the area of importance is determined by identifying a concentration of the eye gaze direction information within a region of the archival image.

Claim 31

The computer program product of claim 25 , wherein the step of determining an area of importance in the archival image comprises determining an eye gaze direction path based upon the obtained eye gaze direction information wherein the area of importance is determined by identifying a predetermined pattern of the eye gaze direction path.

Claim 32

The computer program product of claim 25 , wherein the eye information further includes pupil size information captured during the image capture sequence and the step of determining an area of importance in the archival image based upon the eye gaze direction path comprises determining the area of importance based upon the eye gaze direction information and the pupil size information.

Claim 33

The computer program product of claim 25 , further comprising the step of analyzing the archival image wherein the step of determining an area of importance in the archival image based upon the eye information comprises determining the area of importance based upon the eye information and the analysis of the archival image.

Claim 34

The computer program product of claim 25 , wherein the step of determining the area of importance in the archival image based upon the eye information comprises detecting a sequence of fixations in the eye information and determining the area of importance based upon the sequence of fixations.

Claim 35

The computer program product of claim 25 , wherein the step of determining the area of importance in the archival image based upon the eye information comprises detecting a sequence of fixations in the eye gaze direction information, clustering the sequence of fixations, assigning a weight to each cluster, and selecting an area of importance based upon the weights of the clusters.

Claim 36

The computer program product of claim 32 , wherein the step of determining the weight of each cluster comprises determining the weight of each cluster based upon the order of the fixations in each cluster.

Claim 37

The computer program product of claim 32 , wherein the step of determining the weight of each cluster further comprises determining the weight of each cluster based upon the order of fixations in each cluster and the duration of the fixations within each cluster.

Claim 38

The computer program product of claim 32 , wherein the step of determining the weight of each cluster comprises determining the weight of each cluster based upon the duration of the fixations within each cluster.

Claim 39

The computer program product of claim 32 , wherein the step of determining the weight of each cluster comprises determining the weight of each cluster based at least in part upon a pattern of the eye gaze information between clusters.

Claim 40

The computer program product of claim 25 , wherein the step of determining the area of importance in the archival image based upon the eye information comprises identifying at least one of a pattern of positive importance and a pattern of negative importance in eye gaze direction information.

Claim 41

A computer program product for determining an area of importance in captured archival image the computer program product comprising a computer readable storage medium having a computer program stored thereon for performing the steps of: capturing a plurality of eye images, in whole or part, when viewing a scene image to be photographed; capturing the scene image during an image capture sequence, to obtain the captured archival image; determining eye information, including eye gaze direction information which describe eye gaze movement during the image capture sequence and is determined from the plurality of eye images, and context information, including scene composition change information which describes scene composition adjustments during the image capture sequence and is used to interpret the eye images; determining an eye gaze direction path relative to the captured archival image based upon the eye gaze direction information and scene composition change information; determining an area of importance in the captured archival image based upon the eye gaze direction path; generating area importance data; and associating the area importance data with the captured archival image.

Claim 42

A computer program product for determining an area of importance in captured archival image the computer program product comprising a computer readable storage medium having a computer program stored thereon for performing the steps of: capturing a plurality of eye images, in whole or part, when viewing a scene image to be photographed; capturing the scene image during an image capture sequence, to obtain the captured archival image; determining eye information, including eye gaze direction information which describe eye gaze movement during the image capture sequence and is determined from the plurality of eye images, and context information, including scene composition change information which describes scene composition adjustments during the image capture sequence and is used to interpret the eye images; determining an eye gaze direction path relative to the captured archival image based upon the eye gaze direction information and scene composition change information; determining an area of importance in the captured archival image based upon the eye gaze direction path; generating area importance data; and associating the area importance data with the captured archival image.

Claim 43

The computer program product of claim 38 , wherein the step of determining an area of importance in the archival image based upon the eye gaze direction path comprises determining the area of importance in the archival image based only upon eye gaze direction information that is determined to reflect eye gaze information within the archival image.

Claim 44

The computer program product of claim 38 , wherein the step of determining a path of eye gaze direction relative to the archival image comprises determining the path of eye gaze direction by adjusting the eye gaze direction information based upon the scene composition change information so that the determined path of eye gaze direction is recorded relative to the scene captured in the archival image.

Claim 45

The computer program product of claim 38 , wherein the context information comprises a composite image containing image information composited from a set of evaluation images captured during composition of the archival image and wherein said eye gaze direction information is recorded relative to the composite image; the computer program product further comprising the step of registering the position of the archival image with respect to the composite image and determining the path of eye gaze direction relative to the archival image based upon the registered position of the archival image with respect to the composite image.

Claim 46

The computer program product of claim 38 , wherein the context information comprises adjustable zoom setting information and wherein step of determining a path of eye gaze direction relative to the archival image comprises determining the path of eye gaze information based upon the captured eye gaze direction information, the scene composition change information and the adjustable zoom setting information.

Claim 47

The computer program product of claim 38 , wherein the context information characterizes changes in a field of view and the step of determining a path of eye gaze direction relative to the archival image comprises adjusting the path of eye gaze direction during a composition phase of the archival image capture process to compensate for changes in the field of view of the camera.

Claim 48

The computer program product of claim 38 , wherein the context information comprises a set of evaluation images captured during the image capture sequence, and wherein the step of determining a path of eye gaze direction relative to the archival image comprises analyzing the set of evaluation images and determining the path of eye gaze direction relative to the archival image based upon the eye gaze direction information and the analysis of the set of evaluation images.

Claim 49

The computer program product of claim 38 , wherein the eye information comprises eye information captured from more than one eye wherein the path of eye gaze direction is determined relative to the archival image based upon the captured eye gaze direction information from each eye and scene composition change information.

Claim 50

The computer program product of claim 38 , wherein the eye information and context information are recorded on a photosensitive medium further comprising the step of extracting the eye information and context information from the photosensitive medium and associating the eye information and context information with an archival image recorded on the photosensitive medium.

Claim 51

The computer program product of claim 38 , wherein the eye information includes at least one image of an eye of a user, further comprising the step of determining the identity of the user of the camera system based upon the eye images and associating the user identity with the scene image.

Claim 52

The computer program product of claim 38 , wherein the context information includes location information and wherein the steps of determining an area of importance is based upon the path of eye gaze direction relative to the archival image and the location information.