US7177447B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Real-time multi-stage infrared image-based tracking system
Assignee: Lockheed Martin Corp
•
Filing: 2003-05-23
•
Granted: 2007-02-13
•
Expires: 2023-05-23
Abstract
Abstract A multi-stage infrared (IR) image-based tracking system is provided that accurately identifies and tracks targets across a variety of different situations and environmental conditions. The system includes primary and secondary trackers. The secondary tracker supports the primary tracker by identifying and tracking targets when the primary tracker loses or fails to identify one or more targets, and helps the primary tracker reacquire valid targets. In an exemplary embodiment of the invention, the primary tracker is a correlation-based tracker, and the secondary tracker is a feature-based tracker. A prescreener is also included, which operates concurrently with the correlation-based and feature-based trackers to generate a list of possible targets. The list can be used to provide a confidence level for the probable targets identified by each of the primary and secondary trackers.
Inventors
- 👤 Teresa Lorae Pace Olson
- 👤 James Joseph Slaski
- 👤 Carl William Sanford
- 👤 Ruey-Yuan Han
- 👤 Casey Leonard Contini
- 👤 Robert Russell Reinig
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 19:22
CPC Classifications
G; G06; G06V; G06V10/00; G06V10/20; G06V10/255; G06V10/70; G06V10/74; G06V10/75; G06V10/751; G06V10/7515
Claims Analysis
Total Claims: 42
Claim 1
Independent Claim
88 words
A multi-stage image-based tracking system, comprising: a first tracker for identifying at least one target within at least one image provided to the system; and a second tracker for identifying the at least one target within the at least one image, wherein the second tracker is a feature-based tracker, wherein the second tracker is initialized during tracking by the first tracker; wherein when the first tracker fails to identify the at least one target, the second tracker provides identification of the at least one target to the first.
Claim 2
Dependent Claim
88 words
A multi-stage image-based tracking system, comprising: a first tracker for identifying at least one target within at least one image provided to the system; and a second tracker for identifying the at least one target within the at least one image, wherein the second tracker is a feature-based tracker, wherein the second tracker is initialized during tracking by the first tracker; wherein when the first tracker fails to identify the at least one target, the second tracker provides identification of the at least one target to the first.
Claim 3
Dependent Claim
46 words
The system of claim 1 , further comprising a prescreener for processing the at least one image to provide a list of possible targets, wherein the first and second trackers identify the at least one target based on the at least one image and the list.
Claim 4
Dependent Claim
21 words
The system of claim 2 , wherein the list is used to corroborate target identification of the first and second trackers.
Claim 5
Dependent Claim
30 words
The system of claim 2 , wherein the list is used to provide a quality measure of a target identification of at least one of the first and second trackers.
Claim 6
Dependent Claim
56 words
The system of claim 2 , further comprising: an automatic target recognition module that receives the list of possible targets from the prescreener, identifies a target from the list of possible targets, and provides the identified target to the first and second trackers when the first and second trackers both lose tracking of an initial target.
Claim 7
Dependent Claim
17 words
The system of claim 5 , wherein the identified target is the same as the initial target.
Claim 8
Dependent Claim
46 words
The system of claim 5 , wherein a device guided by the tracking system continues based upon an identification of the at least one target by the first tracker or the second tracker when both the first and second trackers lose tracking of an initial target.
Claim 9
Dependent Claim
22 words
The system of claim 1 , wherein the prescreener generates the list by integrating edge information in the at least one image.
Claim 10
Dependent Claim
13 words
The system of claim 1 , wherein the image is an infrared image.
Claim 11
Dependent Claim
14 words
The system of claim 1 , wherein the first tracker is a correlation-based tracker.
Claim 12
Dependent Claim
21 words
The system of claim 1 , wherein the at least one image is provided by an image pickup on a missile.
Claim 13
Dependent Claim
37 words
The system of claim 1 , wherein whether the first tracker fails to identify the at least one target is based on a confidence value associated with the at least one target identified by the first tracker.
Claim 14
Dependent Claim
32 words
The system of claim 12 , wherein the confidence value is determined based upon an identification of the at least one target by the first tracker in a predetermined number of images.
Claim 15
Dependent Claim
34 words
The system of claim 1 , wherein when the second tracker fails to identify the at least one target, the first tracker provides identification of the at least one target to the second tracker.
Claim 16
Dependent Claim
36 words
The multi-stage image-based tracking system of claim 1 , wherein the first and second trackers are reinitialized when a confidence value associated with the tracking of the second tracker falls below a second predetermined minimum value.
Claim 17
Dependent Claim
22 words
The multi-stage image-based tracking system of claim 1 , wherein the first tracker and the second tracker track a target in parallel.
Claim 18
Independent Claim
141 words
A method for identifying targets within at least one image using a primary tracker and a secondary tracker, comprising the steps of: tracking at least one target in the at least one image using the primary tracker; evaluating a tracking performance of the primary tracker; initializing the secondary tracker during tracking by the primary tracker; switching tracking control from the primary tracker to the secondary tracker when the tracking performance of the primary tracker falls below a first predetermined minimum value; acquiring and tracking the at least one target using the secondary tracker, the secondary tracker is a feature-based tracker, which calculates a predetermined number of features of the at least one target within the at least one image; switching tracking control from the secondary tracker to the primary tracker after acquiring the at least one target using the secondary tracker.
Claim 19
Independent Claim
141 words
A method for identifying targets within at least one image using a primary tracker and a secondary tracker, comprising the steps of: tracking at least one target in the at least one image using the primary tracker; evaluating a tracking performance of the primary tracker; initializing the secondary tracker during tracking by the primary tracker; switching tracking control from the primary tracker to the secondary tracker when the tracking performance of the primary tracker falls below a first predetermined minimum value; acquiring and tracking the at least one target using the secondary tracker, the secondary tracker is a feature-based tracker, which calculates a predetermined number of features of the at least one target within the at least one image; switching tracking control from the secondary tracker to the primary tracker after acquiring the at least one target using the secondary tracker.
Claim 20
Dependent Claim
47 words
The method of claim 17 , further comprising the steps of: generating a list indicating possible targets in the at least one image using a prescreener; and providing the list to the primary and secondary trackers to enhance the tracking performance of the primary and secondary trackers.
Claim 21
Dependent Claim
85 words
The method of claim 18 , wherein the step of generating the list comprises the steps of: calculating strong edge information in the at least one image using an edge operator; integrating the edge information over the at least one image using a plurality of boxes having different sizes to determine possible targets; and applying a morphological erode operator to the integrated at least one image to calculate centroids of possible targets within the at least one image, wherein the centroids indicate the possible targets.
Claim 22
Dependent Claim
15 words
The method of claim 19 , wherein the edge operator is a Sobel edge operator.
Claim 23
Dependent Claim
92 words
The method of claim 18 , wherein the step of generating comprises: calculating strong edge information in the at least one image using an edge operator; evaluating at least an edge magnitude portion of the strong edge information using atmospheric attenuation weighting to compensate for atmospheric effects; thresholding the compensated strong edge information; integrating the thresholded information over the at least one image using variable box sizes; thresholding the integrated information; and calculating centroids that correspond to potential target objects in the at least one image, based on the thresholded integrated information.
Claim 24
Dependent Claim
48 words
The method of claim 21 , further comprising: determining whether each potential target object is too large or too small to be a valid potential target; based on the step of determining, discarding potential target objects that are too large or too small to be valid potential targets.
Claim 25
Dependent Claim
27 words
The method of claim 21 , wherein the step of calculating centroids is performed by applying a morphological erode operator to subimages in the thresholded integrated information.
Claim 26
Dependent Claim
45 words
The method of claim 23 , further comprising: determining a size of a potential target object represented by a subimage in the thresholded integrated information, by counting a number erodes that the morphological erode operator must perform to reduce the subimage to a single pixel.
Claim 27
Dependent Claim
57 words
The method of claim 21 , wherein: the step of thresholding the integrated information generates a processed version of the at least one image; and the step of calculating centroids comprises calculating centroids of subimages in the processed version of the at least one image, that correspond to potential target objects in the at least one image.
Claim 28
Dependent Claim
21 words
The method of claim 17 , wherein the at least one image is provided by an image pickup on a missile.
Claim 29
Dependent Claim
40 words
The method of claim 17 , wherein tracking control is switched from the primary tracker to the secondary tracker when a distance between the primary and secondary trackers and the at least one target is greater than a predetermined distance.
Claim 30
Dependent Claim
25 words
The method of claim 17 , wherein prior to switching tracking control from the primary tracker to the secondary tracker the secondary tracker is initialized.
Claim 31
Dependent Claim
68 words
The method of claim 28 , wherein the initialization of the secondary tracker comprises: determining whether the tracking performance of the primary tracker is above the predetermined minimum; receiving a list indicating possible targets from a prescreener; computing features in a number of images; calculating statistics of the computed features; and determining whether the secondary tracker identifies valid targets in a predetermined number of the number of images.
Claim 32
Dependent Claim
41 words
The method of claim 17 , comprising: reinitializing the primary and secondary trackers using an output from an automatic target recognition module when a value associated with the tracking performance of the secondary tracker falls below a second predetermined minimum value.
Claim 33
Dependent Claim
19 words
The method of claim 17 , wherein the first tracker and the second tracker track a target in parallel.
Claim 34
Independent Claim
128 words
A method for identifying targets within at least one image comprising the steps of: tracking at least one target in the at least one image using a primary tracker; tracking the at least one target using a secondary tracker; evaluating a tracking performance of the primary tracker; switching tracking control from the primary tracker to the secondary tracker when a confidence value associated with the tracking of the primary tracker falls below a first predetermined minimum value; and reinitializing the primary and secondary trackers using an output from an automatic target recognition module when a confidence value associated with the tracking of the secondary tracker falls below a second predetermined minimum value and the confidence value associated with the primary tracker remains below the first predetermined minimum value.
Claim 35
Independent Claim
128 words
A method for identifying targets within at least one image comprising the steps of: tracking at least one target in the at least one image using a primary tracker; tracking the at least one target using a secondary tracker; evaluating a tracking performance of the primary tracker; switching tracking control from the primary tracker to the secondary tracker when a confidence value associated with the tracking of the primary tracker falls below a first predetermined minimum value; and reinitializing the primary and secondary trackers using an output from an automatic target recognition module when a confidence value associated with the tracking of the secondary tracker falls below a second predetermined minimum value and the confidence value associated with the primary tracker remains below the first predetermined minimum value.
Claim 36
Dependent Claim
21 words
The method of claim 32 , wherein the at least one image is provided by an image pickup on a missile.
Claim 37
Dependent Claim
22 words
The method of claim 32 , wherein the primary tracker is a correlation-based tracker and the secondary tracker is a feature-based tracker.
Claim 38
Dependent Claim
69 words
The method of claim 32 , wherein reinitialization of the primary and secondary trackers comprises: receiving, by the automatic target recognition module, a list of possible targets from a prescreener; identifying, by the automatic target recognition module, a target from the list of possible targets; and providing the identified target to the primary and secondary trackers when the first and second trackers both lose tracking of an initial target.
Claim 39
Independent Claim
202 words
A method for identifying and tracking at least one target within at least one image using a primary tracker and a secondary tracker and an automatic target recognition module, comprising the steps of: tracking at least one target in the at least one image using the primary tracker; tracking the at least one target using the secondary tracker; evaluating a tracking performance of the primary tracker; evaluating a tracking performance of the secondary tracker; switching tracking control from the primary tracker to the secondary tracker when the tracking performance of the primary tracker falls below a first predetermined minimum and the tracking performance of the secondary tracker remains above a second predetermined minimum; switching tracking control to the primary tracker when the tracking performance of the primary tracker rises above the predetermined minimum; switching tracking control to the automatic target recognition module when the tracking performance of the primary tracker is below the first predetermined minimum and the tracking performance of the secondary tracker is below the second predetermined minimum; selecting a target in the at least one image, wherein the automatic target recognition module performs the selection; and identifying the selected target to the primary tracker and to the secondary tracker.
Claim 40
Independent Claim
202 words
A method for identifying and tracking at least one target within at least one image using a primary tracker and a secondary tracker and an automatic target recognition module, comprising the steps of: tracking at least one target in the at least one image using the primary tracker; tracking the at least one target using the secondary tracker; evaluating a tracking performance of the primary tracker; evaluating a tracking performance of the secondary tracker; switching tracking control from the primary tracker to the secondary tracker when the tracking performance of the primary tracker falls below a first predetermined minimum and the tracking performance of the secondary tracker remains above a second predetermined minimum; switching tracking control to the primary tracker when the tracking performance of the primary tracker rises above the predetermined minimum; switching tracking control to the automatic target recognition module when the tracking performance of the primary tracker is below the first predetermined minimum and the tracking performance of the secondary tracker is below the second predetermined minimum; selecting a target in the at least one image, wherein the automatic target recognition module performs the selection; and identifying the selected target to the primary tracker and to the secondary tracker.
Claim 41
Dependent Claim
62 words
The method of claim 36 , wherein a prescreener is also used, further comprising the steps of: providing a list of potential target objects in the at least one image, to each of the primary tracker, the secondary tracker, and the automatic target recognition module; wherein the target selected in the step of selecting is selected from the list of potential targets.
Claim 42
Dependent Claim
26 words
The method of claim 36 , wherein the tracking performance of the primary and secondary trackers is evaluated based on tracking in a number of images.
References Cited (Backward Citations)
US4470048A
Cited
US4550435A
Cited
US4739401A
Cited
US5341142A
Cited
US4849906A
Cited
USH713H
Cited
US5187777A
Cited
US5323472A
Cited
US5235651A
Cited
US5335298A
Cited
US5870486A
Cited
US5647015A
Cited
US5422828A
Cited
US5214433A
Cited
US5325098A
Cited
US5479525A
Cited
US6055334A
Cited
US5651512A
Cited
US5990939A
Cited
US5982930A
Cited
US5809171A
Cited
US6079862A
Cited
US5947413A
Cited
US6005609A
Cited
US5963653A
Cited
US6031568A
Cited
US6042050A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2010-02-12 | 2010-08-14 | 2011-02-13 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2014-02-12 | 2014-08-14 | 2015-02-13 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2018-02-12 | 2018-08-14 | 2019-02-13 | $7700.00 | Window Closed / Lapsed |