US7764808B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
System and method for vehicle detection and tracking
Assignee: Aumovio Autonomous Mobility Germany GmbH
•
Filing: 2004-03-22
•
Granted: 2010-07-27
•
Expires: 2024-03-22
Abstract
Abstract A system and method for detecting and tracking an object is disclosed. A camera captures a video sequence comprised of a plurality of image frames. A processor receives the video sequence and analyzes each image frame to determine if an object is detected. The processor applies one or more classifiers to an object in each image frame and computes a confidence score based on the application of the one or more classifiers to the object. A database stores the one or more classifiers and vehicle training samples. A display displays the video sequence.
Inventors
- 👤 Ying Zhu
- 👤 Dorin Comaniciu
- 👤 Martin Pellkofer
- 👤 Thorsten Köhler
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 11:09
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/20; G06V; G06V10/00; G06V10/20; G06V10/255
Claims Analysis
Total Claims: 96
Claim 1
Independent Claim
145 words
A method for detecting one or more objects belonging to the same object class comprising the steps of: a) receiving a video sequence from a video camera comprised of a plurality of image frames; b) applying one or more component classifiers to detect components of objects in an image frame in the video sequence, wherein the component classifiers include classifiers for detecting object components of different sizes at multiple scales; c) computing a confidence score based in part on the response from the one or more component detectors; d) repeating steps b) and c) to detect components of objects belonging to the same object class in additional images frames in the video sequence; and e) accumulating confidence scores from the component detectors to determine if an object is detected, wherein said method is adapted for detecting moving and stationary objects from a moving video camera.
Claim 2
Independent Claim
145 words
A method for detecting one or more objects belonging to the same object class comprising the steps of: a) receiving a video sequence from a video camera comprised of a plurality of image frames; b) applying one or more component classifiers to detect components of objects in an image frame in the video sequence, wherein the component classifiers include classifiers for detecting object components of different sizes at multiple scales; c) computing a confidence score based in part on the response from the one or more component detectors; d) repeating steps b) and c) to detect components of objects belonging to the same object class in additional images frames in the video sequence; and e) accumulating confidence scores from the component detectors to determine if an object is detected, wherein said method is adapted for detecting moving and stationary objects from a moving video camera.
Claim 3
Dependent Claim
39 words
The method of claim 1 wherein if accumulated confidence scores indicate high confidence of a presence of an object, the method further comprising the step of: identifying the detected components to be an object of a particular object class.
Claim 4
Dependent Claim
12 words
The method of claim 1 wherein the object class is a vehicle.
Claim 5
Dependent Claim
22 words
The method of claim 1 further comprising the step of: if an object is detected, outputting a detection signal and object position.
Claim 6
Dependent Claim
47 words
The method of claim 1 further comprising the steps of: testing geometry constraints on a spatial arrangement of detected components in an image; and applying whole-appearance classifiers an image patch that contains the detected components and which is aligned according to the position of the detected components.
Claim 7
Dependent Claim
14 words
The method of claim 5 wherein the geometry constraints are derived from camera parameters.
Claim 8
Dependent Claim
14 words
The method of claim 5 wherein the geometry constraints are derived from object size.
Claim 9
Dependent Claim
21 words
The method of claim 5 wherein the geometry constraints are derived from a location of an object appearance in the image.
Claim 10
Dependent Claim
32 words
The method of claim 5 wherein the whole appearance classifiers detect entire or partial object appearance, the entire or partial object appearance being aligned according to positioning of at least two components.
Claim 11
Dependent Claim
22 words
The method of claim 1 wherein component classifiers are defined by discriminant features and decision rules which are learned through boosted training.
Claim 12
Dependent Claim
11 words
The method of claim 10 wherein the discriminant features include corners.
Claim 13
Dependent Claim
12 words
The method of claim 10 wherein the discriminant features include horizontal edges.
Claim 14
Dependent Claim
12 words
The method of claim 10 wherein the discriminant features include vertical edges.
Claim 15
Dependent Claim
12 words
The method of claim 10 wherein the discriminant features include horizontal stripes.
Claim 16
Dependent Claim
12 words
The method of claim 10 wherein the discriminant features include vertical stripes.
Claim 17
Dependent Claim
12 words
The method of claim 10 wherein the discriminant features include diagonal stripes.
Claim 18
Dependent Claim
23 words
The method of claim 10 further comprising the step of: performing an online adaptation to adapt a classifier structure to an online pattern.
Claim 19
Dependent Claim
42 words
The method of claim 17 wherein the step of performing an online adaptation further comprises the step of: applying a dynamic switching strategy to direct the detector to take appropriate weak classifiers as discriminants according to auxiliary information about the online pattern.
Claim 20
Dependent Claim
15 words
The method of claim 1 wherein the one or more classifiers include overlapping component classifiers.
Claim 21
Dependent Claim
20 words
The method of claim 19 wherein the overlapping component classifiers comprises four corners representing a rear profile of a vehicle.
Claim 22
Dependent Claim
20 words
The method of claim 19 wherein the overlapping component classifiers comprises four corners representing a frontal profile of a vehicle.
Claim 23
Dependent Claim
20 words
The method of claim 19 wherein the overlapping component classifiers comprises four corners representing a side profile of a vehicle.
Claim 24
Dependent Claim
20 words
The method of claim 20 wherein one of the overlapping component classifiers detects the bottom left corner of a vehicle.
Claim 25
Dependent Claim
20 words
The method of claim 20 wherein one of the overlapping component classifiers detects the bottom right corner of a vehicle.
Claim 26
Dependent Claim
20 words
The method of claim 20 wherein one of the overlapping component classifiers detects the top left corner of a vehicle.
Claim 27
Dependent Claim
20 words
The method of claim 20 wherein one of the overlapping component classifiers detects the top right corner of a vehicle.
Claim 28
Dependent Claim
24 words
The method of claim 20 wherein positioning of the four corners of the rear profile for a vehicle differ for different classes of vehicles.
Claim 29
Dependent Claim
12 words
The method of claim 27 wherein a class of vehicle includes sedans.
Claim 30
Dependent Claim
14 words
The method of claim 27 wherein a class of vehicle includes sports utility vehicles.
Claim 31
Dependent Claim
12 words
The method of claim 27 wherein a class of vehicle includes vans.
Claim 32
Dependent Claim
13 words
The method of claim 27 wherein a class of vehicle includes tractor trailers.
Claim 33
Dependent Claim
12 words
The method of claim 27 wherein a class of vehicle includes trucks.
Claim 34
Dependent Claim
17 words
The method of claim 20 wherein a distance between any two corners of the vehicle is constrained.
Claim 35
Dependent Claim
34 words
The method of claim 33 wherein the constraint between any two corners of the vehicle is scaled based on a distance between the vehicle and a camera capturing the video sequence and camera parameters.
Claim 36
Dependent Claim
31 words
The method of claim 1 wherein an image pyramid of multiple resolutions is used to detect objects of size 2*x, 4*x and so on with the classifier for the size x.
Claim 37
Dependent Claim
22 words
The method of claim 1 wherein the accumulated confidence scores are inferred from confidence scores across multiple frames using a recursive filter.
Claim 38
Dependent Claim
27 words
The method of claim 36 wherein when the accumulated confidence score is a linear combination of the confidence scores of multiple component classifiers and the whole-appearance classifiers.
Claim 39
Dependent Claim
32 words
The method of claim 37 wherein when the confidence score for a principal component classifier is sufficiently high, the confidence score of the remaining component classifiers and the whole-appearance classifier are computed.
Claim 40
Dependent Claim
24 words
The method of claim 2 wherein if an object is detected the method comprising the step of: tracking the object over subsequent image frames.
Claim 41
Dependent Claim
39 words
The method of claim 39 wherein the step of tracking the object further comprises the step of: restricting an area of search in each subsequent image frame based on the location of the object in a current image frame.
Claim 42
Dependent Claim
38 words
The method of claim 39 wherein the step of tracking the object further comprises the step of: determining the optimal classifier scale based on a distance between the object and a camera detecting the object and camera parameters.
Claim 43
Dependent Claim
20 words
The method of claim 1 wherein the confidence scores of component classifiers are computed in a coarse to fine framework.
Claim 44
Dependent Claim
16 words
The method of claim 1 wherein detection is performed on an image pyramid of multiple resolutions.
Claim 45
Dependent Claim
11 words
The method of claim 1 wherein an object class includes pedestrians.
Claim 46
Dependent Claim
11 words
The method of claim 1 wherein an object class includes bicycles.
Claim 47
Dependent Claim
11 words
The method of claim 1 wherein an object class includes motorcycles.
Claim 48
Dependent Claim
14 words
The method of claim 1 wherein object class includes different types of traffic signs.
Claim 49
Independent Claim
128 words
A system for detection and tracking an object comprising: a camera for capturing a video sequence comprised of a plurality of image frames; a processor for receiving the video sequence and analyzing each image frame to determine if an object is detected, said processor applying one or more component classifiers to detect components of objects in each image frame and computing a confidence score based on the response from the one or more component detectors and the result of additional validation, wherein the accumulated confidence scores is inferred from confidence scores across multiple frames using a recursive filter; and a database for storing the one or more classifiers and object training samples, wherein said system is adapted for detecting moving and stationary objects from a moving video camera.
Claim 50
Independent Claim
128 words
A system for detection and tracking an object comprising: a camera for capturing a video sequence comprised of a plurality of image frames; a processor for receiving the video sequence and analyzing each image frame to determine if an object is detected, said processor applying one or more component classifiers to detect components of objects in each image frame and computing a confidence score based on the response from the one or more component detectors and the result of additional validation, wherein the accumulated confidence scores is inferred from confidence scores across multiple frames using a recursive filter; and a database for storing the one or more classifiers and object training samples, wherein said system is adapted for detecting moving and stationary objects from a moving video camera.
Claim 51
Dependent Claim
12 words
The method of claim 48 wherein the object class is a vehicle.
Claim 52
Dependent Claim
12 words
The method of claim 48 wherein the object class is a pedestrian.
Claim 53
Dependent Claim
12 words
The method of claim 48 wherein the object class is a bicycle.
Claim 54
Dependent Claim
12 words
The method of claim 48 wherein the object class is a motorbike.
Claim 55
Dependent Claim
15 words
The method of claim 48 wherein the object class includes different types of traffic signs.
Claim 56
Dependent Claim
26 words
The system of claim 48 wherein the detected components are determined to be an object of a particular object class if the confidence scores are high.
Claim 57
Dependent Claim
17 words
The system of claim 54 wherein if an object is detected, the processor outputs a warning signal.
Claim 58
Dependent Claim
14 words
The system of claim 48 further comprising: a display for displaying the video sequence.
Claim 59
Dependent Claim
51 words
The system of claim 48 wherein the processor further comprises: means for testing geometry constraints on a spatial arrangement of detected components in an image; and means for applying whole-appearance classifiers an image patch that contains the detected components and which is aligned according to the position of the detected components.
Claim 60
Dependent Claim
14 words
The system of claim 57 wherein the geometry constraints are derived from camera parameters.
Claim 61
Dependent Claim
14 words
The system of claim 57 wherein the geometry constraints are derived from object size.
Claim 62
Dependent Claim
21 words
The system of claim 57 wherein the geometry constraints are derived from a location of an object appearance in the image.
Claim 63
Dependent Claim
32 words
The system of claim 57 wherein the whole appearance classifiers detect entire or partial object appearance, the entire or partial object appearance being aligned according to positioning of at least two components.
Claim 64
Dependent Claim
21 words
The system of claim 48 wherein the component classifiers include classifiers for detecting object components of different sizes at multiple scales.
Claim 65
Dependent Claim
22 words
The system of claim 48 wherein component classifiers are defined by discriminant features and decision rules which are learned through boosted training.
Claim 66
Dependent Claim
11 words
The system of claim 63 wherein the discriminant features include corners.
Claim 67
Dependent Claim
12 words
The system of claim 63 wherein the discriminant features include horizontal edges.
Claim 68
Dependent Claim
12 words
The system of claim 63 wherein the discriminant features include vertical edges.
Claim 69
Dependent Claim
12 words
The system of claim 63 wherein the discriminant features include horizontal stripes.
Claim 70
Dependent Claim
12 words
The system of claim 63 wherein the discriminant features include vertical stripes.
Claim 71
Dependent Claim
12 words
The system of claim 63 wherein the discriminant features include diagonal stripes.
Claim 72
Dependent Claim
23 words
The system of claim 63 further comprising the step of: performing an online adaptation to adapt a classifier structure to an online pattern.
Claim 73
Dependent Claim
42 words
The system of claim 70 wherein the step of performing an online adaptation further comprises the step of: applying a dynamic switching strategy to direct the detector to take appropriate weak classifiers as discriminants according to auxiliary information about the online pattern.
Claim 74
Dependent Claim
15 words
The system of claim 48 wherein the one or more classifiers include overlapping component classifiers.
Claim 75
Dependent Claim
20 words
The system of claim 72 wherein the overlapping component classifiers comprises four corners representing a rear profile of a vehicle.
Claim 76
Dependent Claim
20 words
The system of claim 73 wherein one of the overlapping component classifiers detects the bottom left corner of a vehicle.
Claim 77
Dependent Claim
20 words
The system of claim 73 wherein one of the overlapping component classifiers detects the bottom right corner of a vehicle.
Claim 78
Dependent Claim
20 words
The system of claim 73 wherein one of the overlapping component classifiers detects the top left corner of a vehicle.
Claim 79
Dependent Claim
20 words
The system of claim 73 wherein one of the overlapping component classifiers detects the top right corner of a vehicle.
Claim 80
Dependent Claim
24 words
The system of claim 73 wherein positioning of the four corners of the rear profile for a vehicle differ for different classes of vehicles.
Claim 81
Dependent Claim
12 words
The system of claim 78 wherein a class of vehicle includes sedans.
Claim 82
Dependent Claim
14 words
The system of claim 78 wherein a class of vehicle includes sports utility vehicles.
Claim 83
Dependent Claim
12 words
The system of claim 78 wherein a class of vehicle includes vans.
Claim 84
Dependent Claim
13 words
The system of claim 78 wherein a class of vehicle includes tractor trailers.
Claim 85
Dependent Claim
17 words
The system of claim 73 wherein a distance between any two corners of the vehicle is constrained.
Claim 86
Dependent Claim
36 words
The system of claim 83 wherein the constraint between any two corners of the vehicle is scaled based on a distance between the vehicle and a camera capturing the video sequence as well as camera parameters.
Claim 87
Dependent Claim
31 words
The system of claim 84 wherein an image pyramid of multiple resolutions is used to detect objects of size 2*x, 4*x and so on with the classifier for the size x.
Claim 88
Dependent Claim
27 words
The system of claim 48 wherein when the accumulated confidence score is a linear combination of the confidence scores of multiple component classifiers and the whole-appearance classifiers.
Claim 89
Dependent Claim
32 words
The system of claim 48 wherein when the confidence score for a principal component classifier is sufficiently high, the confidence score of the remaining component classifiers and the whole-appearance classifier are computed.
Claim 90
Dependent Claim
19 words
The system of claim 48 wherein the processor comprises: means for tracking a detected object over subsequent image frames.
Claim 91
Dependent Claim
34 words
The system of claim 88 wherein tracking means further comprises: means for restricting an area of search in each subsequent image frame based on the location of the object in a current image frame.
Claim 92
Dependent Claim
34 words
The system of claim 88 wherein the tracking means further comprises: means for determining the optimal classifier scale based on a distance between the object and a camera detecting the object and camera parameters.
Claim 93
Dependent Claim
20 words
The system of claim 48 wherein the confidence scores of component classifiers are computed in a coarse to fine framework.
Claim 94
Dependent Claim
18 words
The system of claim 48 wherein detection and tracking is performed on an image pyramid of multiple resolutions.
Claim 95
Independent Claim
157 words
A method for detecting one or more objects belonging to the same object class comprising the steps of: a) receiving a video sequence from a video camera comprised of a plurality of image frames; b) applying one or more component classifiers to detect components of objects in an image frame in the video sequence, wherein the one or more component classifiers include overlapping component classifiers, and wherein component classifiers are defined by discriminant features and decision rules which are learned through boosting; c) computing a confidence score based in part on the response from the one or more component detectors; d) repeating steps b) and c) to detect components of objects belonging to the same object class in additional images frames in the video sequence; and e) accumulating confidence scores from the component detectors to determine if an object is detected, wherein said method is adapted for detecting moving and stationary objects from a moving video camera.
Claim 96
Independent Claim
157 words
A method for detecting one or more objects belonging to the same object class comprising the steps of: a) receiving a video sequence from a video camera comprised of a plurality of image frames; b) applying one or more component classifiers to detect components of objects in an image frame in the video sequence, wherein the one or more component classifiers include overlapping component classifiers, and wherein component classifiers are defined by discriminant features and decision rules which are learned through boosting; c) computing a confidence score based in part on the response from the one or more component detectors; d) repeating steps b) and c) to detect components of objects belonging to the same object class in additional images frames in the video sequence; and e) accumulating confidence scores from the component detectors to determine if an object is detected, wherein said method is adapted for detecting moving and stationary objects from a moving video camera.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2013-07-26 | 2014-01-25 | 2014-07-27 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2017-07-26 | 2018-01-25 | 2018-07-27 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2021-07-26 | 2022-01-25 | 2022-07-27 | $7700.00 | Window Closed / Lapsed |