EP0474307A2
Expired / Lapsed
EP
⏱️ 0.0 years remaining
Tracking a moving object
Assignee: Philips Electronics UK Ltd
•
Filing: 1991-09-03
•
Granted: —
•
Expires: 2011-09-03
Abstract
Abstract A method of tracking a moving object in a scene, for example the face of a person in videophone applications, comprises forming an initial template of the face, extracting a mask outlining the face, dividing the template into a plurality (for example sixteen) sub-templates, searching the next frame to find a match with the template, searching the next frame to find a match with each of the sub-templates, determining the displacements of each of the sub-templates with respect to the template, using the displacements to determine affine transform coefficients and performing an affine transform to produce an updated template and updated mask.
Inventors
- 👤 Timothy Ian Paterson Trew
- 👤 Gabriele Carola c/o Philips Electronics Seeling
Bibliographic Details
Application Number
—
Jurisdiction
EP
Data Source
google_patents
Last Synchronized
2026-10-25 20:58
CPC Classifications
H; H04; H04N; H04N7/00; H04N7/14; H04N7/141; G; G01; G01S; G01S3/00; G01S3/78; G01S3/782; G01S3/785; G01S3/786; G01S3/7864; G06; G06T; G06T7/00; G06T7/20; G06T7/223; G06T7/246; G06T7/248; G06V; G06V40/00; G06V40/10; G06V40/16; G06V40/161; G06V40/168; H04N19/00; H04N19/50; H04N19/503; H04N19/51; H04N19/537; H04N19/543; G06T2207/00; G06T2207/10; G06T2207/10016; G06T2207/30; G06T2207/30196; G06T2207/30201; G06V10/00; G06V10/40; G06V10/62; H04N19/20
Claims Analysis
Total Claims: 21
Claim 1
Independent Claim
148 words
A method of tracking a three dimensional object in a scene over a series of two dimensional picture frames, the method comprising: a) forming an initial template including the object to be tracked; b) extracting a mask outlining the object to be tracked; c) dividing the template into a plurality of sub-templates which are not associated with specific features of the object; d) searching each successive frame to find a match with the template in the preceding frame; e) searching each successive frame to find a match with each of the sub-templates in the preceding frame; f) determining in each successive frame the displacements of the positions of each of the sub-templates with respect to that of the template; g) using the displacements to determine coefficients for a bivariate transform capable of mapping the displaced positions; h) performing the transform to produce an updated template and updated mask.
Claim 2
Dependent Claim
37 words
A method as claimed in Claim 1 in which the transform is the affine transform and the method comprises the further step of i) extending the initial template by repeating pixels at the boundary of the object.
Claim 3
Dependent Claim
30 words
A method as claimed in Claim 1 or Claim 2 in which the area of search for each of the sub-templates is a restricted area around the template matched position.
Claim 4
Independent Claim
43 words
A method as claimed in any preceding claim comprising the further step of j) setting the luminance of each pixel within the mask to a first constant value and setting the luminance of each pixel outside the mask to a second constant value.
Claim 5
Independent Claim
33 words
A method as claimed in any preceding claim including the further step of k) discarding each sub-template in which the object to be tracked occupies less than a given proportion of its area.
Claim 6
Independent Claim
49 words
A method as claimed in any preceding claim comprising the further steps of l) discarding any sub-template for which a local match cannot be found, m) determining whether a minimum number of sub-templates remain, and n) where a minimum number remain using their displacements to calculate the transform coefficients.
Claim 7
Dependent Claim
31 words
A method as claimed in Claim 6 in which when less than the minimum number of sub-templates remain the search area for matching the template is increased for the next frame.
Claim 8
Independent Claim
55 words
A method as claimed in any preceding claim comprising the further steps of o) comparing the updated template with the region of the next frame selected by the transformed mask at the position of the template match, and p) using the results of the comparison to control the gain correction term of a Kalman filter.
Claim 9
Dependent Claim
81 words
A method as claimed in Claim 8 wherein step o) includes the steps of o1) for each pixel taking a block of 3x3 pixels having the pixel of interest at its centre; o2) displacing the block by one pixel in each direction in turn; o3) comparing for each displacement the pixels within the updated template with the current picture and calculating a difference value; o4) finding the lowest difference value; and o5) outputting that lowest value as the result of comparison.
Claim 10
Independent Claim
40 words
A method as claimed in any of Claims 7 to 9 in which when a match is not found the transform is again applied to the template to predict the shape and position of the template in the next frame.
Claim 11
Dependent Claim
144 words
Apparatus for tracking a three dimensional object in a scene over a series of two dimensional picture frames, comprising means for forming an initial template including the object to be tracked; means for extracting a mask outlining the object to be tracked; means for dividing the template into a plurality of sub-templates; means for searching each successive frame to find a match with the template in the preceding frame; means for searching each successive frame to find a match with each of the sub-templates in the preceding frame; means for determining in each successive frame the displacements of the positions of each of the sub-templates with respect to that of the template; means for determining, using the displacements, transform coefficients for a bivariate transform capable of mapping the displaced positions; and means for performing the transform to produce an updated template and updated mask.
Claim 12
Dependent Claim
24 words
Apparatus as claimed in Claim 11 comprising means for restricting the area of search around the template matched position for each of the sub-templates.
Claim 13
Dependent Claim
50 words
Apparatus as claimed in Claim 11 or Claim 12 further comprising means for discarding any sub-template for which a local match cannot be found, means for determining whether a minimum number of sub-templates remain, and means for, where a minimum number remain, using their displacements to calculate the transform coefficients.
Claim 14
Dependent Claim
30 words
Apparatus as claimed in Claim 13 comprising means for increasing the search area for matching the template for the next frame when less than the minimum number of sub-templates remain.
Claim 15
Dependent Claim
54 words
Apparatus as claimed in any of Claims 11 to 14 further comprising means for comparing the updated template with the region of the next frame selected by the transformed mask at the position of the template match and using the results of the comparison to control the gain correction term of a Kalman filter.
Claim 16
Dependent Claim
83 words
Apparatus as claimed in Claim 15 in which the comparing means comprises means for, for each pixel, taking a block of 3x3 pixels having the pixel of interest at the centre, means for displacing the block by one pixel in each direction in turn, means for comparing, for each displacement, the updated template with the current picture and calculating a difference value, means for finding the lowest difference value, and means for outputting the lowest difference value as a result of the comparison.
Claim 17
Dependent Claim
34 words
Apparatus as claimed in any of Claim 15 or Claim 16 comprising means for again applying the transform to the template to predict the shape and position of the template in the next frame.
Claim 18
Dependent Claim
107 words
A video telephone terminal comprising a video camera, means for identifying a face within the picture produced by the camera, means for tracking the position of the face in successive picture frames, a codec producing an output to be connected to a telephone line, the codec having a first input to which the output of the video camera is coupled and a second input connected to an output of the tracking means, wherein the codec is arranged to produce an output whose resolution is dependent on the signal at it second input and the tracking means comprises apparatus as claimed in any of Claims 11 to 17.
Claim 19
Independent Claim
93 words
A method of encoding a series of two dimensional pictures representing a scene including a moving three dimensional object comprising the steps of a) generating a video signal representing the series of two dimensional pictures, b) forming an outline of the three dimensional object c) tracking the three dimensional object by means of a method as claimed in any of Claims 1 to 10 and d) encoding the video signal at a higher resolution when representing portions of the picture within the outline than when representing portions of the picture outside the outline.
Claim 20
Dependent Claim
97 words
Apparatus for encoding a series of two dimensional picture frames representing a scene including a moving three dimensional object, the apparatus comprising means for generating a video signal representing the series of two dimensional picture frames, means for forming an outline of the three dimensional object, means for tracking the three dimensional object over a series of picture frames, said tracking means comprising apparatus as claimed in any of Claims 15 to 27 and means for encoding the video signal such that areas within the outline are encoded with a greater resolution than those outside the outline.
Claim 21
Dependent Claim
74 words
Apparatus as claimed in Claim 20 arranged for encoding pictures for recording on record carriers for Interactive Media players wherein the means for forming an outline comprises computing means coupled to a display means and having an input device enabling a user to create an outline around the object, the encoding means is coupled to an emulator, and the emulator is arranged to enable replay of the series of pictures on the display means.
References Cited (Backward Citations)
No citation records indexed.
USPTO Annuity & Maintenance Windows
Maintenance schedule is calculated for granted US utility patents.