Patent US6570608B1 successfully refreshed from Google Patents.
US6570608B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
System and method for detecting interactions of people and vehicles
Assignee: Texas Instruments Inc
•
Filing: 1999-08-24
•
Granted: 2003-05-27
•
Expires: 2019-08-24
Abstract
Abstract A video surveillance system that implements object detection and event recognition employing smart monitoring algorithms to analyze a video stream and recognize the interaction of people with cars. The system forms a reference image consisting of the background of the scene viewed by the video camera. The system forms a foreground difference image between the current image and an updated reference image including any stationary cars. The system detects any objects other than any stationary car in the scene. The updated reference image is formed by forming a background difference image between the current image and the reference image. Any detected object is examined to determine if it is a car object. Upon initial detection of a stationary car object, the system forms a reference car image of a portion of the current image corresponding to the position of the stationary car. The system then forms the updated reference image by substituting this reference car image for a corresponding portion of the reference image. A number of special techniques preserve object integrity when objects overlap cars. The video surveillance system notes events when objects interact with cars such as an ENTER CAR event and an EXIT CAR event.
Inventors
- 👤 Christopher Tserng
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 12:58
CPC Classifications
H; H04; H04N; H04N7/00; H04N7/18; H04N7/188; G; G06; G06V; G06V40/00; G06V40/20
Claims Analysis
Total Claims: 19
Claim 1
Independent Claim
172 words
A method for detecting interactions between people and vehicles from a video stream of a sequence of images of a scene comprising the steps of: forming a reference image consisting of a background of the scene; forming an updated reference image including any stationary car objects including forming a background difference image between a current image and said reference image, detecting objects in said difference image, determining if any detected object in said difference image is a car object, determining whether any car objects are stationary, upon initial determination that a detected car object becomes stationary forming a reference car image of a portion of said current image corresponding to a position of any stationary car object, and substituting said reference car image for a corresponding portion of said reference image, thereby forming said updated reference image; forming a foreground difference image between a current image and said updated reference image; and detecting any blobs in said foreground difference image thereby detecting objects other than any stationary car objects in the scene.
Claim 2
Independent Claim
172 words
A method for detecting interactions between people and vehicles from a video stream of a sequence of images of a scene comprising the steps of: forming a reference image consisting of a background of the scene; forming an updated reference image including any stationary car objects including forming a background difference image between a current image and said reference image, detecting objects in said difference image, determining if any detected object in said difference image is a car object, determining whether any car objects are stationary, upon initial determination that a detected car object becomes stationary forming a reference car image of a portion of said current image corresponding to a position of any stationary car object, and substituting said reference car image for a corresponding portion of said reference image, thereby forming said updated reference image; forming a foreground difference image between a current image and said updated reference image; and detecting any blobs in said foreground difference image thereby detecting objects other than any stationary car objects in the scene.
Claim 3
Dependent Claim
51 words
The method of claim 1 , wherein: said step of determining whether any detected object in said difference image is a car object includes determining a detected object is a car object if said detected object has a height greater than 3.5 feet and a width of greater than 11.0 feet.
Claim 4
Dependent Claim
54 words
The method of claim 1 , wherein: said step of determining whether any detected object in said difference image is a car object includes determining a detected object is a car object if said detected object has greater than a minimum size and a height/width aspect ratio of less than a predetermined aspect ratio.
Claim 5
Dependent Claim
33 words
The method of claim 3 , wherein: said minimum size includes a height greater than 3.5 feet and a width of greater than 11.0 feet; and said predetermined aspect ratio 1 s 0.7.
Claim 6
Dependent Claim
95 words
The method of claim 1 , further comprising: tracking a position of each detected object in an immediately prior image to a corresponding detected object of a current image; and if an object of said immediately prior image overlaps a car object of said immediately prior image for a first image when said car object is determined to be stationary, then said step of forming an updated reference image includes forming a fake background in said updated reference image having black pixels at all pixel locations of said overlapping object in said immediately prior image.
Claim 7
Dependent Claim
109 words
The method of claim 5 , further comprising: determining if a stationary car object in an immediately prior image begins to move in a current image; upon determination that a car object begins to move in said current image, checking for validity any detected object by determining if any pixels of each detected object includes black pixels corresponding to said fake background, and declaring said detected object as false and not tracking any object in an immediately prior image to said detected object of a current image if said determined number of black pixels corresponding to fake background is greater than 70% of all pixels of said detected object.
Claim 8
Dependent Claim
93 words
The method of claim 1 , further comprising the steps of: determining whether a previously stationary car object begins to move by comparing a stationary car object in a prior image with a corresponding car object in a current image, calculating a prior centroid of said previously stationary car object in the prior image, calculating a current centroid of said corresponding car object in the current image, and determining said previously stationary car object has begun to move if said prior centroid and said current centroid differ by more than a predetermined amount.
Claim 9
Dependent Claim
13 words
The method of claim 7 , wherein: said predetermined amount is 6.5 pixels.
Claim 10
Dependent Claim
90 words
The method of claim 1 , further comprising the steps of: upon detection of a stationary car object forming a car background image of a portion of said reference image corresponding to the position of said reference car image; determining whether a previously stationary car object begins to move; upon determining that a previously stationary car object has begun to move removing detected objects overlapping the now moving car object, and substituting said reference car image for a corresponding portion of said updated reference image, thereby restoring said reference image.
Claim 11
Dependent Claim
60 words
The method of claim 1 , further comprising the steps of: determining whether a previously stationary car object begins to move; upon determining that a previously stationary car object has begun to move determining if any detected object corresponds to an object overlapping a car object in an immediately prior image, and checking for validity any such detected overlapping object.
Claim 12
Dependent Claim
103 words
The method of claim 10 , further comprising: tracking a position of each detected object in an immediately prior image to a corresponding detected object of a current image; and said step of checking for validity includes determining whether a track of objects in prior images corresponding to any detected overlapping object is less than a predetermined number of images, declaring said detected overlapping object as false and not tracking any object in an immediately prior image to said detected overlapping object of a current image if said track of corresponding objects in prior images is less than said predetermined number of images.
Claim 13
Dependent Claim
123 words
The method of claim 11 , further comprising: if said track of objects in prior images corresponding to any detected overlapping object is greater than or equal to said predetermined number of images reviewing corresponding objects along said track of said object back track until a prior object is found having a width of less than 3 feet which is not changing substantially between images, storing said prior object, when said car object moves whereby said detected overlapping object no longer overlaps said car object comparing said prior object to said corresponding object in said current image, and confirming tracking of said prior object to said object in said current image if said prior object substantially matches said object in said current image.
Claim 14
Independent Claim
205 words
A method for detecting interactions between people and vehicles from a video stream of a sequence of images of a scene comprising the steps of: forming a reference image consisting of a background of the scene; forming a background difference image between a current image and said reference image; detecting objects in said background difference image; determining if any detected object in said background difference image is a car object; determining whether any car objects are stationary; forming an updated reference image including any stationary car objects; forming a foreground difference image between a current image and said updated reference image; detecting any blobs in said foreground difference image thereby detecting objects other than any stationary car objects in the scene; and checking for validity any objects newly detected in said current image by determining a number of pixels of any newly detected object in said foreground difference image matching corresponding pixels in said car difference image, and declaring said newly detected object as false and not tracking any object in an immediately prior image to said newly detected object of a current image if said determined number of matching pixels is less than half of all pixels of said detected newly moving overlapping object.
Claim 15
Independent Claim
205 words
A method for detecting interactions between people and vehicles from a video stream of a sequence of images of a scene comprising the steps of: forming a reference image consisting of a background of the scene; forming a background difference image between a current image and said reference image; detecting objects in said background difference image; determining if any detected object in said background difference image is a car object; determining whether any car objects are stationary; forming an updated reference image including any stationary car objects; forming a foreground difference image between a current image and said updated reference image; detecting any blobs in said foreground difference image thereby detecting objects other than any stationary car objects in the scene; and checking for validity any objects newly detected in said current image by determining a number of pixels of any newly detected object in said foreground difference image matching corresponding pixels in said car difference image, and declaring said newly detected object as false and not tracking any object in an immediately prior image to said newly detected object of a current image if said determined number of matching pixels is less than half of all pixels of said detected newly moving overlapping object.
Claim 16
Independent Claim
162 words
A method for detecting interactions between people and vehicles from a video stream of a sequence of images of a scene comprising the steps of: forming a reference image consisting of a background of the scene; forming a background difference image between a current image and said reference image; detecting objects in said background difference image; determining if any detected object in said background difference image is a car object; determining whether any car objects are stationary; forming an updated reference image including any stationary car objects; forming a foreground difference image between a current image and said updated reference image; detecting any blobs in said foreground difference image thereby detecting objects other than any stationary car objects in the scene; and determining an ENTER CAR event whenever any detected object overlaps a position of any stationary cars in one image in the video stream and a corresponding detected object is not present in a next following image in the video stream.
Claim 17
Independent Claim
162 words
A method for detecting interactions between people and vehicles from a video stream of a sequence of images of a scene comprising the steps of: forming a reference image consisting of a background of the scene; forming a background difference image between a current image and said reference image; detecting objects in said background difference image; determining if any detected object in said background difference image is a car object; determining whether any car objects are stationary; forming an updated reference image including any stationary car objects; forming a foreground difference image between a current image and said updated reference image; detecting any blobs in said foreground difference image thereby detecting objects other than any stationary car objects in the scene; and determining an ENTER CAR event whenever any detected object overlaps a position of any stationary cars in one image in the video stream and a corresponding detected object is not present in a next following image in the video stream.
Claim 18
Independent Claim
162 words
A method for detecting interactions between people and vehicles from a video stream of a sequence of images of a scene comprising the steps of: forming a reference image consisting of a background of the scene; forming a background difference image between a current image and said reference image; detecting objects in said background difference image; determining if any detected object in said background difference image is a car object; determining whether any car objects are stationary; forming an updated reference image including any stationary car objects; forming a foreground difference image between a current image and said updated reference image; detecting any blobs in said foreground difference image thereby detecting objects other than any stationary car objects in the scene; and determining an EXIT CAR event whenever a detected object overlaps a position of any stationary car in one image in the video stream and a corresponding detected object is not present in an immediately prior image in the video stream.
Claim 19
Independent Claim
162 words
A method for detecting interactions between people and vehicles from a video stream of a sequence of images of a scene comprising the steps of: forming a reference image consisting of a background of the scene; forming a background difference image between a current image and said reference image; detecting objects in said background difference image; determining if any detected object in said background difference image is a car object; determining whether any car objects are stationary; forming an updated reference image including any stationary car objects; forming a foreground difference image between a current image and said updated reference image; detecting any blobs in said foreground difference image thereby detecting objects other than any stationary car objects in the scene; and determining an EXIT CAR event whenever a detected object overlaps a position of any stationary car in one image in the video stream and a corresponding detected object is not present in an immediately prior image in the video stream.
References Cited (Backward Citations)
US5097328A
Cited
US5937077A
Cited
US6366316B1
Cited
US5956424A
Cited
US6061088A
Cited
US6198833B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2006-05-26 | 2006-11-25 | 2007-05-27 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2010-05-26 | 2010-11-25 | 2011-05-27 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2014-05-26 | 2014-11-25 | 2015-05-27 | $7700.00 | Window Closed / Lapsed |