Patent US5521633A successfully refreshed from Google Patents.
US5521633A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Motor vehicle obstacle monitoring system using optical flow processing
Assignee: Yazaki Corp
•
Filing: 1993-09-24
•
Granted: 1996-05-28
•
Expires: 2013-09-24
Abstract
Abstract A monitoring method for moving vehicles allows the presence of danger to be determined from the magnitude of an optical flow vector. The optical flow vector is detected as a movement of a single point on an object taken over two images, with one image taken at a preceding time and the other image taken at succeeding time in a series of images. A long and narrow window set in a radial direction from a focus of expansion (FOE) of the earlier image is moved in the same direction on the latter image. An optical flow vector of a target point is defined by a vector connecting the midpoint of a location of a subsequent window and the midpoint of a location at which the long and narrow window is set in the earlier image. The location of the subsequent window is determined by minimizing the sum of absolute values of differences in luminance between the long and narrow window and an area of the latter image overlapping such long and narrow window. The optical flow calculation area is limited to areas in which a difference in luminance between the earlier and latter images exceeds a predetermined threshold, and any influence from scenes outside the road boundaries and from lane-dividing lines or other symbols drawn on the road surface is removed in calculating the optical flow.
Inventors
- 👤 Masato Nakajima
- 👤 Takahiro Fujishiro
- 👤 Norio Kitamura
- 👤 Kazuyuki Sasaki
- 👤 Takahiro Oikawa
- 👤 Kouji Ishii
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-25 07:04
CPC Classifications
B; B60; B60Q; B60Q9/00; B60Q9/008; G; G06; G06T; G06T7/00; G06T7/20; G06T7/223; G06T7/246; G06V; G06V20/00; G06V20/50; G06V20/56; G06V20/58; B60G; B60G2401/00; B60G2401/14; B60G2401/142; B60W; B60W2554/00; B60W2554/80; B60W2554/801; G01; G01S; G01S11/00; G01S11/12; G06T2207/00; G06T2207/10; G06T2207/10016; G06T2207/30; G06T2207/30236; G06T2207/30248
Claims Analysis
Total Claims: 13
Claim 1
Independent Claim
234 words
A method for detecting obstacles in the path of a moving vehicle, comprising the steps of: imaging a view from the moving vehicle as at least two image frames, a first frame of the two image frames being taken at a first time and a second frame of the two image frames being taken at a succeeding time; targeting a point in the first frame; defining a location in the first frame for a long and narrow window encompassing the targeted point in a predetermined relationship and extending in a radial direction relative to a focus of expansion in the first frame; establishing a location in the second frame for the long and narrow window by (i) stepwise moving the window in the radial direction while calculating a sum of absolute values of differences in luminance between the window in the first frame and the window in the second frame, and (ii) determining where the calculated sum attains a minimum value; ascertaining a point in the second frame corresponding to the targeted point based on the location of the window in the second frame by utilizing the predetermined relationship; defining an optical flow vector as a displacement measured from the targeted point to the corresponding point; and utilizing a magnitude and a positioning of the optical flow vector relative to the focus of expansion to determine a degree of danger for the moving vehicle.
Claim 2
Independent Claim
234 words
A method for detecting obstacles in the path of a moving vehicle, comprising the steps of: imaging a view from the moving vehicle as at least two image frames, a first frame of the two image frames being taken at a first time and a second frame of the two image frames being taken at a succeeding time; targeting a point in the first frame; defining a location in the first frame for a long and narrow window encompassing the targeted point in a predetermined relationship and extending in a radial direction relative to a focus of expansion in the first frame; establishing a location in the second frame for the long and narrow window by (i) stepwise moving the window in the radial direction while calculating a sum of absolute values of differences in luminance between the window in the first frame and the window in the second frame, and (ii) determining where the calculated sum attains a minimum value; ascertaining a point in the second frame corresponding to the targeted point based on the location of the window in the second frame by utilizing the predetermined relationship; defining an optical flow vector as a displacement measured from the targeted point to the corresponding point; and utilizing a magnitude and a positioning of the optical flow vector relative to the focus of expansion to determine a degree of danger for the moving vehicle.
Claim 3
Independent Claim
167 words
A method for detecting obstacles in the path of a moving vehicle, comprising the steps of: using a video camera to capture a view from the moving vehicle as at least two image frames, a first frame of the two image frames being taken at a first time and a second frame of the two image frames being taken at a succeeding time; targeting points in the first frame; detecting displacements of the targeted points in the second frame as respective optical flow vectors; removing optical flow vectors resulting from a lane-dividing line or a character drawn on a road surface by: (i) determining a height measured from the line or the character to the video camera, and (ii) deleting those optical flow vectors for which the determined height coincides with a height of the video camera to the road surface; and utilizing magnitudes and positionings relative to a focus of expansion of remaining optical flow vectors to determine a degree of danger for the moving vehicle.
Claim 4
Independent Claim
167 words
A method for detecting obstacles in the path of a moving vehicle, comprising the steps of: using a video camera to capture a view from the moving vehicle as at least two image frames, a first frame of the two image frames being taken at a first time and a second frame of the two image frames being taken at a succeeding time; targeting points in the first frame; detecting displacements of the targeted points in the second frame as respective optical flow vectors; removing optical flow vectors resulting from a lane-dividing line or a character drawn on a road surface by: (i) determining a height measured from the line or the character to the video camera, and (ii) deleting those optical flow vectors for which the determined height coincides with a height of the video camera to the road surface; and utilizing magnitudes and positionings relative to a focus of expansion of remaining optical flow vectors to determine a degree of danger for the moving vehicle.
Claim 5
Independent Claim
142 words
A method for detecting obstacles in the path of a moving vehicle, comprising the steps of: using a video camera to capture a view from the moving vehicle as at least two image frames, a first frame of the two image frames being taken at a first time and a second frame of the two image frames being taken at a succeeding time; targeting points in a processing area of the first frame; detecting displacements of the targeted points in the second frame as respective optical flow vectors; dividing the processing area into a plurality of zones; for each of the zones, weighting a sum of magnitudes of the optical flow vectors present in each of the zones on a zone basis; and determining a degree of danger for the moving vehicle from the weighted sum calculated for each of the zones.
Claim 6
Independent Claim
142 words
A method for detecting obstacles in the path of a moving vehicle, comprising the steps of: using a video camera to capture a view from the moving vehicle as at least two image frames, a first frame of the two image frames being taken at a first time and a second frame of the two image frames being taken at a succeeding time; targeting points in a processing area of the first frame; detecting displacements of the targeted points in the second frame as respective optical flow vectors; dividing the processing area into a plurality of zones; for each of the zones, weighting a sum of magnitudes of the optical flow vectors present in each of the zones on a zone basis; and determining a degree of danger for the moving vehicle from the weighted sum calculated for each of the zones.
Claim 7
Dependent Claim
51 words
The method according to claim 3, wherein said determining step comprises: setting a respective, predetermined threshold for each of the zones; and if the weighted sum in at least one of the zones exceeds the predetermined threshold for that one zone, judging that the moving vehicle is in a dangerous situation.
Claim 8
Dependent Claim
49 words
The method according to claim 3, wherein said determining step comprises: for each of the zones, setting plural, predetermined thresholds of differing levels; and for each of the zones, determining the degree of danger based on which of the thresholds is exceeded by the weighted sum for that zone.
Claim 9
Dependent Claim
22 words
The method according to claim 5, further comprising the step of: outputting an alarm in accordance with the determined degree of danger.
Claim 10
Independent Claim
265 words
A method for detecting obstacles in a vicinity of a moving vehicle, comprising the steps of: imaging a view from the moving vehicle as at least two image frames, a first frame of the two image frames being taken at a first time and a second frame of the two image frames being taken at a succeeding time; targeting a point in the first frame; defining a location in the first frame for a long and narrow window encompassing the targeted point in a predetermined relationship and extending in a radial direction relative to a focus of expansion in the first frame; establishing a location in the second frame for the long and narrow window by (i) stepwise moving the window in the radial direction while calculating a sum of absolute values of differences in luminance between the window in the first frame and the window in the second frame, and (ii) determining where the calculated sum attains a minimum value; ascertaining a point in the second frame corresponding to the targeted point based on the location of the window in the second frame by utilizing the predetermined relationship; defining an optical flow vector as a displacement measured from the targeted point to the corresponding point; selecting the optical flow vector only if the optical flow vector defines a displacement away from the focus of expansion; and if said selecting step yields a selected optical flow vector, utilizing a magnitude and a positioning of the selected optical flow vector relative to the focus of expansion to determine a degree of danger for the moving vehicle.
Claim 11
Independent Claim
265 words
A method for detecting obstacles in a vicinity of a moving vehicle, comprising the steps of: imaging a view from the moving vehicle as at least two image frames, a first frame of the two image frames being taken at a first time and a second frame of the two image frames being taken at a succeeding time; targeting a point in the first frame; defining a location in the first frame for a long and narrow window encompassing the targeted point in a predetermined relationship and extending in a radial direction relative to a focus of expansion in the first frame; establishing a location in the second frame for the long and narrow window by (i) stepwise moving the window in the radial direction while calculating a sum of absolute values of differences in luminance between the window in the first frame and the window in the second frame, and (ii) determining where the calculated sum attains a minimum value; ascertaining a point in the second frame corresponding to the targeted point based on the location of the window in the second frame by utilizing the predetermined relationship; defining an optical flow vector as a displacement measured from the targeted point to the corresponding point; selecting the optical flow vector only if the optical flow vector defines a displacement away from the focus of expansion; and if said selecting step yields a selected optical flow vector, utilizing a magnitude and a positioning of the selected optical flow vector relative to the focus of expansion to determine a degree of danger for the moving vehicle.
Claim 12
Dependent Claim
18 words
The method according to claim 7, wherein the view imaged from the moving vehicle is a rear view.
Claim 13
Dependent Claim
18 words
The method according to claim 7, wherein the view imaged from the moving vehicle is a side view.
References Cited (Backward Citations)
US4926346A
Cited
DE3619824A1
Cited
DE3636946A1
Cited
DE3642196A1
Cited
US4985847A
Cited
JPH02241855A
Cited
US4970653A
Cited
JPH03118613A
Cited
US5265172A
Cited
US5128874A
Cited
DE4107177A1
Cited
US5257209A
Cited
US5139327A
Cited
US5249128A
Cited
US5291207A
Cited
US5410346A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1999-05-28 | 1999-11-27 | 2000-05-28 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2003-05-28 | 2003-11-27 | 2004-05-28 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2007-05-28 | 2007-11-27 | 2008-05-28 | $7700.00 | Window Closed / Lapsed |