Computer vision method and apparatus for imaging sensors for recognizing and tracking occupants in fixed environments under variable illumination

Claim 2

A computer vision apparatus for recognizing and tracking occupants in a fixed space including: at least one sensor fixedly positioned to capture light from a fixed space beginning at a time prior to an occupant's entrance and continuing subsequent to an occupant's entrance to form a series of N images having pixels with pixel intensities, the series of N images including a current image N c and a previous image N c−1 , with the previous image N c−1 being the image immediately prior to the current image N c in the series of N subsequent images, the sensor converting the series of N images into electronic data; a processor positioned to receive electronic data from the sensor and to compare the current image N c with the previous image N c−1 in order to dynamically determine the occupant type and occupant position; computing an low-level vision-based feature map of the previous image N c −1 of the fixed space; computing an low-level vision-based feature map of the current image N c of the fixed space; computing a residual low-level vision-based feature map having connected regions withsizes, by calculating the difference between low-level vision-based feature map of the current image N c of the fixed space and the previous image N c−1 of the fixed space, where the pixels corresponding to the occupant in the previous subsequent image have been replaced by corresponding pixels of the current image N c of the fixed space; applying a high pass size filter to remove connected regions of the residual low-level vision-based feature map below a specified size, resulting in a size-filtered residual low-level-vision-based feature map including gaps and a bounded area; filling the bounded area of the size-filtered residual low-level vision-based feature map, resulting in a filled low-level vision-based feature area; calculating two-dimensional moments of the filled low-level vision-based feature area and using the two-dimensional moments to construct an equivalent rectangle with moments equal to that of the filled low-level vision-based feature area; and determining the centroid, orientation, size and aspect ratio of the equivalent rectangle.

Claim 3

A computer vision apparatus for recognizing and tracking occupants in a fixed space as set forth in claim 1 , further including at least one illumination source fixedly positioned to provide illumination to the fixed space and an occupant therein.

Claim 4

A computer vision apparatus for recognizing and tracking occupants in a fixed space as set forth in claim 2 , wherein the illumination source is an IR source.

Claim 5

A computer vision method for recognizing and tracking occupants in a fixed space including the steps of: (a) providing at least one sensor fixedly positioned to capture light reflected from a fixed space beginning at a time prior an occupant's entrance and continuing subsequent to an occupant's entrance to form a series of N images having pixels with pixel intensities, the series of N images including a current image N c and a previous image N c−1 , with the previous image N c−1 being the image immediately prior to the current image N c in the series of N subsequent images, the sensor converting the series of N images into electronic data; and (b) processing the electronic data with a processing system positioned to receive electronic data from the sensor and to compare current image N c with the previous image N c−1 in order to dynamically determine the occupant type and occupant position.

Claim 6

A computer vision method for recognizing and tracking occupants in a fixed space including the steps of: (a) providing at least one sensor fixedly positioned to capture light reflected from a fixed space beginning at a time prior an occupant's entrance and continuing subsequent to an occupant's entrance to form a series of N images having pixels with pixel intensities, the series of N images including a current image N c and a previous image N c−1 , with the previous image N c−1 being the image immediately prior to the current image N c in the series of N subsequent images, the sensor converting the series of N images into electronic data; and (b) processing the electronic data with a processing system positioned to receive electronic data from the sensor and to compare current image N c with the previous image N c−1 in order to dynamically determine the occupant type and occupant position.

Claim 7

A computer vision method for recognizing and tracking occupants in a fixed space as set forth in claim 4 wherein the processing step (b) further includes the steps of: (a) computing an edge map of the previous image N c−1 of the fixed space; (b) computing an edge map of the current image N c of the fixed space; (c) computing a residual edge map having connected regions with sizes, by calculating the difference between edge map of the current image N c of the fixed space and the previous image N c−1 of the fixed space, where the pixels corresponding to the occupant in the previous subsequent image have been replaced by corresponding pixels of the current image N c of the fixed space; (d) applying a high pass size filter to remove connected regions of the residual edge map below a specified size, resulting in a size-filtered residual edge map including gaps and a bounded area; (e) filling the bounded area of the size-filtered residual edge map, resulting in a filled edge area; (f) calculating two-dimensional moments of the filled edge area and using the two-dimensional moments to construct an equivalent rectangle with moments equal to that of the filled edge area; (g) determining the centroid, orientation, size, and aspect ratio of the equivalent rectangle.

Claim 8

A computer vision method for recognizing and tracking occupants in a fixed space as set forth in claim 5 , wherein the sensors include onboard edge extraction and two-dimensional moment computation capability.

Claim 9

A computer vision apparatus for recognizing and tracking occupants in a fixed space as set forth in claim 1 wherein the processing system utilizes the method of claim 6 .

Claim 10

A computer vision method for recognizing and tracking occupants in a fixed space as set forth in claim 6 wherein the filling step (e) further includes the steps of: (a) applying an edge dilation operator to the size-filtered residual edge map to dilate the size-filtered residual edge map to eliminate gaps, resulting in a dilated residual edge map including edge pixels and a bounded area; and (b) vertically filling the area between the edge pixels in the dilated residual edge map using the pixel intensity from the subsequent images resulting in a filled edge area.

Claim 11

A computer vision method for recognizing and tracking occupants in a fixed space as set forth in claim 7 , further including the steps of: (a) partitioning the filled edge area into a plurality N of sub-filled edge areas; (b) calculating two-dimensional moments of each of the N sub-filled edge areas and using the two dimensional moments to construct an equivalent rectangle for each of the N sub-filled edge areas; and (c) determining centroid, orientation, size, and aspect ratio of each of the equivalent rectangles for each of the N sub-filled edge areas.

Claim 12

A computer vision method for recognizing and tracking occupants in a fixed space including the steps of: (a) providing at least one sensor fixedly positioned to capture light reflected from a fixed space beginning at a time prior to an occupant's entrance and continuing subsequent to an occupant's to form a series of N images having pixels with pixel intensities, the series of N images including a current image N c and a previous image N c−1 , with the previous image N c−1 being the image immediately prior to the current image N c in the series of N subsequent images, the sensor converting the series of N images into electronic data; and (b) processing the electronic data with a processing system positioned to receive electronic data from the sensor and to: i. compute an edge map of the previous image N c−1 of the fixed space; ii. compute an edge map of the current image N c of the fixed space; iii. compute a residual edge map having connected regions with sizes, from the difference between the edge map of the current image N c of the fixed space and the previous image N c−1 of the fixed space, where the pixels corresponding to the occupant in the previous subsequent image have been replaced by corresponding pixels of the current image N c of the fixed space; iv. apply a high pass size filter to remove connected regions of the residual edge map below a specified size, resulting in a size-filtered residual edge map including gaps; v. apply an edge dilation operator to the size-filtered residual edge map to dilate the size-filtered residual edge map to eliminate gaps, resulting in a dilated residual edge map including edge pixels; vi. vertically fill the area between the edge pixels in the dilated residual edge map using the pixel intensity from the subsequent images resulting in a filled edge area; vii. calculate two-dimensional moments of the filled edge area and using the two-dimensional moments to construct an equivalent rectangle with moments equal to that of the filled edge area; viii. determine the centroid, orientation, size, and aspect ratio of the equivalent rectangle; ix. partitioning the filled edge area into a plurality N of sub-filled edge areas; x. calculating the two-dimensional moments of each of the N sub-filled edge areas and using the two dimensional moments to construct an equivalent rectangle for each of the N sub-filled edge areas; and xi. determining the centroid, orientation, size, and aspect ratio of each of the equivalent rectangles for each of the N sub-filled edge areas; and (c) providing the centroid, orientation, size, and aspect ratio of the equivalent rectangle and the centroid, orientation, size, and aspect ratio of each of the equivalent rectangles for each of the N sub-filled edge areas as output.

Claim 13

A computer vision method for recognizing and tracking occupants in a fixed space including the steps of: (a) providing at least one sensor fixedly positioned to capture light reflected from a fixed space beginning at a time prior to an occupant's entrance and continuing subsequent to an occupant's to form a series of N images having pixels with pixel intensities, the series of N images including a current image N c and a previous image N c−1 , with the previous image N c−1 being the image immediately prior to the current image N c in the series of N subsequent images, the sensor converting the series of N images into electronic data; and (b) processing the electronic data with a processing system positioned to receive electronic data from the sensor and to: i. compute an edge map of the previous image N c−1 of the fixed space; ii. compute an edge map of the current image N c of the fixed space; iii. compute a residual edge map having connected regions with sizes, from the difference between the edge map of the current image N c of the fixed space and the previous image N c−1 of the fixed space, where the pixels corresponding to the occupant in the previous subsequent image have been replaced by corresponding pixels of the current image N c of the fixed space; iv. apply a high pass size filter to remove connected regions of the residual edge map below a specified size, resulting in a size-filtered residual edge map including gaps; v. apply an edge dilation operator to the size-filtered residual edge map to dilate the size-filtered residual edge map to eliminate gaps, resulting in a dilated residual edge map including edge pixels; vi. vertically fill the area between the edge pixels in the dilated residual edge map using the pixel intensity from the subsequent images resulting in a filled edge area; vii. calculate two-dimensional moments of the filled edge area and using the two-dimensional moments to construct an equivalent rectangle with moments equal to that of the filled edge area; viii. determine the centroid, orientation, size, and aspect ratio of the equivalent rectangle; ix. partitioning the filled edge area into a plurality N of sub-filled edge areas; x. calculating the two-dimensional moments of each of the N sub-filled edge areas and using the two dimensional moments to construct an equivalent rectangle for each of the N sub-filled edge areas; and xi. determining the centroid, orientation, size, and aspect ratio of each of the equivalent rectangles for each of the N sub-filled edge areas; and (c) providing the centroid, orientation, size, and aspect ratio of the equivalent rectangle and the centroid, orientation, size, and aspect ratio of each of the equivalent rectangles for each of the N sub-filled edge areas as output.

Claim 14

A computer vision method for recognizing and tracking occupants in a fixed space as set forth in claim 3 , further including the steps of: (a) repeating steps (a) to (h) of claim 5 a plurality of times; (b) computing the circumscribing circle of the equivalent rectangle during each repetition; (c) comparing the filled edge area generated during the current repetition with the circumscribing circle of the equivalent rectangle computed during the previous repetition; and (d) removing that part of the filled edge area falling outside the circumscribing circle the step of calculating the two-dimensional moments of the filled edge area and using the two-dimensional moments to construct an equivalent rectangle.

Claim 15

A computer vision apparatus for recognizing and tracking occupants in a fixed space as set forth in claim 1 , wherein the low-level vision-based features are edges.