← Back to Patent Explorer
US8253799B2 Active / In Force US ⏱️ 1.1 years remaining

Detecting an object in an image using camera registration data indexed to location or camera sensors

Assignee: Sportsmedia Technology Corp
•
Filing: 2007-12-19
•
Granted: 2012-08-28
•
Expires: 2027-12-19
📄 Original PDF 🌐 Google Patents 🔄 Sync Now

Abstract

Abstract An object is detected in images of a live event by storing and indexing camera registration-related data from previous images. For example, the object may be a vehicle which repeatedly traverses a course. A first set of images of the live event is captured when the object is at different locations in the live event. The camera registration-related data for each image is obtained and stored. When the object again traverses the course, for each location, the stored camera registration-related data which is indexed to the location can be retrieved for use in estimating a position of a representation of the object in a current image, such as by defining a search area in the image. An actual position of the object in the image is determined, in response to which the camera registration-related data may be updated, such as for use in a subsequent traversal of the course.

Inventors

  • 👤 Vidya Elangovan
  • 👤 Kenneth A. Milnes
  • 👤 Timothy P. Heidmann

Bibliographic Details

Application Number —
Jurisdiction US
Data Source google_patents
Last Synchronized 2026-11-09 10:06
CPC Classifications
H; H04; H04N; H04N5/00; H04N5/222; H04N5/262; H04N5/265; G; G06; G06T; G06T7/00; G06T7/20; G06T7/246; G06T7/248; G06T7/70; G06T7/73; G06T7/74; G06V; G06V10/00; G06V10/70; G06V10/74; G06V10/75; G06V10/751; H04N5/272; H04N5/2723; A; A63; A63F; A63F2300/00; A63F2300/60; A63F2300/66; A63F2300/663; A63F2300/6692; A63F2300/69; A63F2300/80; A63F2300/8017; G06T2207/00; G06T2207/30; G06T2207/30221; G06T2207/30236; G06V2201/00; G06V2201/08

Claims Analysis

Total Claims: 27
Claim 1
Independent Claim 343 words
A method for processing image data of a live event in which an object repeatedly traverses a circuit, comprising: obtaining images of the live event when the object moves along the circuit during one traversal of the circuit, by panning a camera to follow the object at different locations along the circuit; for each image obtained during the one traversal of the circuit: (a) obtaining time-varying sensor data of the camera comprising time-varying panning data based on the panning, (b) determining a detected position of a representation of the object in the image, (c) determining a corresponding one of the different locations along the circuit using a location detection technique which is independent of the image, (d) determining an expected position of a representation of the object in the image based on the corresponding one of the different locations, (e) determining an inaccuracy between the detected position and the expected position, and (f) based on the inaccuracy, providing updated registration data of the camera comprising an adjusted panning offset to the time-varying panning data; for each location of the different locations during the one traversal of the circuit, storing an entry which indexes the updated registration data to the location; obtaining an additional image of the live event which is captured by the camera when the object is at a location along the circuit during a subsequent traversal of the circuit, by panning the camera to follow the object; determines the location of the object along the circuit during the subsequent traversal of the circuit using a location detection technique which is independent of the additional image; determining one of the different locations during the one traversal of the circuit which is closest to the location along the circuit during the subsequent traversal of the circuit; retrieving the updated registration data from one of the entries which is indexed to the one of the different locations; and responsive to the retrieving, determining an estimate of a position of a representation of the object in the additional image based on the updated registration data.
Claim 2
Independent Claim 343 words
A method for processing image data of a live event in which an object repeatedly traverses a circuit, comprising: obtaining images of the live event when the object moves along the circuit during one traversal of the circuit, by panning a camera to follow the object at different locations along the circuit; for each image obtained during the one traversal of the circuit: (a) obtaining time-varying sensor data of the camera comprising time-varying panning data based on the panning, (b) determining a detected position of a representation of the object in the image, (c) determining a corresponding one of the different locations along the circuit using a location detection technique which is independent of the image, (d) determining an expected position of a representation of the object in the image based on the corresponding one of the different locations, (e) determining an inaccuracy between the detected position and the expected position, and (f) based on the inaccuracy, providing updated registration data of the camera comprising an adjusted panning offset to the time-varying panning data; for each location of the different locations during the one traversal of the circuit, storing an entry which indexes the updated registration data to the location; obtaining an additional image of the live event which is captured by the camera when the object is at a location along the circuit during a subsequent traversal of the circuit, by panning the camera to follow the object; determines the location of the object along the circuit during the subsequent traversal of the circuit using a location detection technique which is independent of the additional image; determining one of the different locations during the one traversal of the circuit which is closest to the location along the circuit during the subsequent traversal of the circuit; retrieving the updated registration data from one of the entries which is indexed to the one of the different locations; and responsive to the retrieving, determining an estimate of a position of a representation of the object in the additional image based on the updated registration data.
Claim 3
Dependent Claim 34 words
The method of claim 1 , further comprising: determining a search area in the additional image based on the estimate of the position; and detecting the representation of the object in the search area.
Claim 4
Dependent Claim 32 words
The method of claim 2 , further comprising: determining and storing an update to the updated registration data based on a position of the representation of the object in the search area.
Claim 5
Dependent Claim 22 words
The method of claim 1 , wherein: each entry indexes the updated registration data to the time-varying sensor data of the camera.
Claim 6
Dependent Claim 23 words
The method of claim 4 , wherein: the time-varying sensor data comprises at least one of time-varying tilt and zoom of the camera.
Claim 7
Dependent Claim 28 words
The method of claim 1 , wherein: the different locations and the location along the circuit during the subsequent traversal of the circuit, are determined using GPS data.
Claim 8
Dependent Claim 36 words
The method of claim 1 , wherein: the different locations and the location along the circuit during the subsequent traversal of the circuit, are determined using radio data by at least one of multi-angulation and multi-lateration.
Claim 9
Dependent Claim 46 words
The method of claim 1 , wherein: the different locations and the location along the circuit during the subsequent traversal of the circuit, are determined using at least one of a time and a distance since the object passed a known location in the live event.
Claim 10
Dependent Claim 16 words
The method of claim 1 , wherein: the different locations are designated locations around the circuit.
Claim 11
Dependent Claim 38 words
The method of claim 1 , further comprising: transforming, using the updated registration data for the additional image, a determined position of the representation of the object in the additional image to a location in the live event.
Claim 12
Dependent Claim 56 words
The method of claim 10 , further comprising: determining an inaccuracy between: a) a location of the object in the live event which is determined independently from the determined position and b) the location which is determined based on the transforming; and applying a correction to location data of the object based on the determined inaccuracy.
Claim 13
Dependent Claim 19 words
The method of claim 1 , wherein: the updated registration data comprises updates to registration parameters of the camera.
Claim 14
Dependent Claim 17 words
The method of claim 1 , wherein: the updated registration data comprises registration parameters of the camera.
Claim 15
Dependent Claim 38 words
The method of claim 1 , wherein: the determining the estimate comprises transforming location data of the object in the live event to a coordinate system of the image using the updated registration data for the additional image.
Claim 16
Dependent Claim 38 words
The method of claim 1 , further comprising: combining a graphic with the additional image, the graphic is located in the additional image based on a determined position of the representation of the object in the additional image.
Claim 17
Dependent Claim 30 words
The method of claim 1 , further comprising: combining a graphic with the additional image, the graphic is located in the additional image based on an orientation of the object.
Claim 18
Dependent Claim 30 words
The method of claim 1 , further comprising: combining a graphic with the additional image, the graphic is located in the additional image based on a perspective of the object.
Claim 19
Dependent Claim 19 words
The method of claim 1 , wherein: the object comprises a vehicle and the circuit is a race track.
Claim 20
Dependent Claim 17 words
The method of claim 1 , wherein: the updated registration data indicates tilt offset of the camera.
Claim 21
Dependent Claim 31 words
The method of claim 1 , wherein: the updated registration data indicates at least one of a zoom field of view map, a camera position, a lens distortion and a focus.
Claim 22
Dependent Claim 33 words
The method of claim 1 , wherein: the obtaining images is performed periodically, once per a time interval which is greater than a time interval at which images are captured by the camera.
Claim 23
Independent Claim 360 words
A system for processing image data of a live event in which an object repeatedly traverses a circuit, comprising: a camera, the camera captures images of the live event when the object is at different locations along the circuit during one traversal of the circuit by movement of the camera to follow the object at different locations along the circuit; and a processing facility which receives the images, the processing facility: a) for each image obtained during the one traversal of the circuit: (i) obtains time-varying sensor data of the camera based on the movement, (ii) determines a detected position of a representation of the object in the image, (iii) determines a corresponding one of the different locations along the circuit using a location detection technique which is independent of the image, (iv) determines an expected position of a representation of the object in the image based on the corresponding one of the different locations, (v) determines an inaccuracy between the detected position and the expected position, and (vi) provides updated registration data of the camera based on the inaccuracy, b) for each location of the different locations during the one traversal of the circuit, stores an entry which indexes the updated registration data to the corresponding one of the different locations, c) obtains an additional image of the live event which is captured by the camera when the object is at a location along the circuit during a subsequent traversal of the circuit, by a pan of the camera to follow the object, d) determines the location of the object along the circuit during the subsequent traversal of the circuit based on a location detection technique which is independent of the additional image, c) determines one of the different locations during the one traversal of the circuit which is closest to the location along the circuit during the subsequent traversal of the circuit, f) performs a retrieval of the updated registration data from one of the entries which is indexed to the one of the different locations, and g) based on the retrieval, determines an estimate of a position of a representation of the object in the additional image.
Claim 24
Independent Claim 360 words
A system for processing image data of a live event in which an object repeatedly traverses a circuit, comprising: a camera, the camera captures images of the live event when the object is at different locations along the circuit during one traversal of the circuit by movement of the camera to follow the object at different locations along the circuit; and a processing facility which receives the images, the processing facility: a) for each image obtained during the one traversal of the circuit: (i) obtains time-varying sensor data of the camera based on the movement, (ii) determines a detected position of a representation of the object in the image, (iii) determines a corresponding one of the different locations along the circuit using a location detection technique which is independent of the image, (iv) determines an expected position of a representation of the object in the image based on the corresponding one of the different locations, (v) determines an inaccuracy between the detected position and the expected position, and (vi) provides updated registration data of the camera based on the inaccuracy, b) for each location of the different locations during the one traversal of the circuit, stores an entry which indexes the updated registration data to the corresponding one of the different locations, c) obtains an additional image of the live event which is captured by the camera when the object is at a location along the circuit during a subsequent traversal of the circuit, by a pan of the camera to follow the object, d) determines the location of the object along the circuit during the subsequent traversal of the circuit based on a location detection technique which is independent of the additional image, c) determines one of the different locations during the one traversal of the circuit which is closest to the location along the circuit during the subsequent traversal of the circuit, f) performs a retrieval of the updated registration data from one of the entries which is indexed to the one of the different locations, and g) based on the retrieval, determines an estimate of a position of a representation of the object in the additional image.
Claim 25
Dependent Claim 72 words
The system of claim 22 , wherein: the movement of the camera comprises a pan of the camera to follow the object at the different locations along the circuit; and for each image obtained during the one traversal of the circuit, the time-varying sensor data of the camera comprises time-varying panning data based on the pan, and the updated camera registration data comprises an adjusted panning offset to the time-varying panning data.
Claim 26
Dependent Claim 362 words
A processor readable storage device comprising processor readable code embodied thereon for programming a processor to perform a method for processing image data of a live event in which an object repeatedly traverses a circuit, the method comprising: obtaining images of the live event when the object is at different locations along the circuit during one traversal of the circuit, the images are captured by panning a camera as the object moves to the different locations; for each image obtained during the one traversal of the circuit: (a) obtaining time-varying sensor data of the camera comprising time-varying panning data based on the panning, (b) determining a detected position of a representation of the object in the image, (c) determining a corresponding one of the different locations along the circuit using a location detection technique which is independent of the image, (d) determining an expected position of a representation of the object in the image based on the corresponding one of the different locations, (e) determining an inaccuracy between the detected position and the expected position, and (f) based on the inaccuracy, providing updated registration data of the camera comprising an adjusted panning offset to the time-varying panning data; for each location of the different locations during the one traversal of the circuit, storing an entry which indexes the updated registration data to the location; obtaining an additional image of the live event which is captured by the camera when the object is at a location along the circuit during a subsequent traversal of the circuit; determining one of the different locations during the one traversal of the circuit which is closest to the location along the circuit during the subsequent traversal of the circuit, by panning the camera to follow the object, the location of the object along the circuit during the subsequent traversal of the circuit is determined using a location detection technique which is independent of the additional image; retrieving the updated registration data from one of the entries which is indexed to the one of the different locations; and responsive to the retrieving, determining an estimate of a position of a representation of the object in the additional image.
Claim 27
Dependent Claim 362 words
A processor readable storage device comprising processor readable code embodied thereon for programming a processor to perform a method for processing image data of a live event in which an object repeatedly traverses a circuit, the method comprising: obtaining images of the live event when the object is at different locations along the circuit during one traversal of the circuit, the images are captured by panning a camera as the object moves to the different locations; for each image obtained during the one traversal of the circuit: (a) obtaining time-varying sensor data of the camera comprising time-varying panning data based on the panning, (b) determining a detected position of a representation of the object in the image, (c) determining a corresponding one of the different locations along the circuit using a location detection technique which is independent of the image, (d) determining an expected position of a representation of the object in the image based on the corresponding one of the different locations, (e) determining an inaccuracy between the detected position and the expected position, and (f) based on the inaccuracy, providing updated registration data of the camera comprising an adjusted panning offset to the time-varying panning data; for each location of the different locations during the one traversal of the circuit, storing an entry which indexes the updated registration data to the location; obtaining an additional image of the live event which is captured by the camera when the object is at a location along the circuit during a subsequent traversal of the circuit; determining one of the different locations during the one traversal of the circuit which is closest to the location along the circuit during the subsequent traversal of the circuit, by panning the camera to follow the object, the location of the object along the circuit during the subsequent traversal of the circuit is determined using a location detection technique which is independent of the additional image; retrieving the updated registration data from one of the entries which is indexed to the one of the different locations; and responsive to the retrieving, determining an estimate of a position of a representation of the object in the additional image.

USPTO Annuity & Maintenance Windows

Annuity Stage Window Opens Payment Deadline Surcharge Period Until Estimated Fee Status
3.5 Years (Year 4) 2015-08-28 2016-02-27 2016-08-28 $2000.00 Window Closed / Lapsed
7.5 Years (Year 8) 2019-08-28 2020-02-27 2020-08-28 $3760.00 Window Closed / Lapsed
11.5 Years (Year 12) 2023-08-28 2024-02-27 2024-08-28 $7700.00 Window Closed / Lapsed

Assigned Categories & Tags

No categories assigned to this patent yet.

Assign Category Tag