Patent US4878183A successfully refreshed from Google Patents.
Patent US4635741A successfully refreshed from Google Patents.
Patent US5768404A successfully refreshed from Google Patents.
US5768404A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Motion and disparity estimation method, image synthesis method, and apparatus for implementing same methods
Assignee: Panasonic Holdings Corp
•
Filing: 1995-04-13
•
Granted: 1998-06-16
•
Expires: 2015-04-13
Abstract
Abstract To achieve this, the invention has a base image frame memory for storing a base image, a reference image frame memory for storing a reference image, a block correlation computing circuit for computing block correlations and confidence measure for estimation by using a plurality of block sizes, a representative pixel correlation computing circuit, and an estimation integrating computing circuit for evaluating the reliability of the result of estimation by block matching on the basis of a luminance gradient, image noise, a minimum value of an evaluation yardstick for differences between blocks, and block size, and for integrating the results of estimation obtained with the plurality of block sizes.
Inventors
- 👤 Atsushi Morimura
- 👤 Takeo Azuma
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-25 21:39
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/20; G06T7/223; G06T7/238; G06V; G06V10/00; G06V10/10; G06V10/20; G06V10/24; G06T2207/00; G06T2207/10; G06T2207/10004; G06T2207/10012; G06T2207/10016; G06T2207/10021; G06V2201/00; G06V2201/12; H; H04; H04N; H04N13/00; H04N13/10; H04N13/189; H04N13/20; H04N13/204; H04N13/239; H04N2013/0074; H04N2013/0081; H04N2013/0085
Claims Analysis
Total Claims: 48
Claim 1
Independent Claim
321 words
A disparity estimation apparatus, comprising: correspondence determining means, for determining correspondence between a first image and a second image; correspondence evaluation means for evaluating a confidence of correspondence determined by the correspondence determining means to produce evaluation data; disparity data computing means for computing disparity data for the first image and the second image based upon correspondence determined by the correspondence determining means; occlusion judging means for judging an occlusion portion of the second image which is not included in the first image, based upon the disparity data and the evaluation data; first edge extracting means, for extracting edges of the evaluation data; second edge extracting means, for extracting edges of the first and second images; first selecting means for selecting at least one edge of the evaluation data which corresponds to a non-occlusion portion of the first image; integrating means for integrating at least one edge of the first image with the selected at least one edge of the evaluation data to produce at least one non-occlusion disparity edge; correspondence converting means, for obtaining the vicinity in the second image corresponding to the non-occlusion disparity edge produced by the integrating means, based upon the disparity data; second selecting means, for selecting at least one edge of the second image from the occlusion portion of the second image; third selecting means, for selecting the selected at least one edge of the second image and the vicinity produced by the correspondence converting means, to produce at least one occlusion disparity edge; disparity supplementing means for supplementing disparity data for the occlusion portion of the second image, based upon the at least one occlusion disparity edge and the disparity data; disparity extrapolating means, for extrapolating disparity data for the non-occlusion portion of the first image based upon the non-occlusion disparity edge and the disparity data; and disparity integrating means, for integrating the disparity data, the supplemented disparity data, and the extrapolated disparity data.
Claim 2
Dependent Claim
321 words
A disparity estimation apparatus, comprising: correspondence determining means, for determining correspondence between a first image and a second image; correspondence evaluation means for evaluating a confidence of correspondence determined by the correspondence determining means to produce evaluation data; disparity data computing means for computing disparity data for the first image and the second image based upon correspondence determined by the correspondence determining means; occlusion judging means for judging an occlusion portion of the second image which is not included in the first image, based upon the disparity data and the evaluation data; first edge extracting means, for extracting edges of the evaluation data; second edge extracting means, for extracting edges of the first and second images; first selecting means for selecting at least one edge of the evaluation data which corresponds to a non-occlusion portion of the first image; integrating means for integrating at least one edge of the first image with the selected at least one edge of the evaluation data to produce at least one non-occlusion disparity edge; correspondence converting means, for obtaining the vicinity in the second image corresponding to the non-occlusion disparity edge produced by the integrating means, based upon the disparity data; second selecting means, for selecting at least one edge of the second image from the occlusion portion of the second image; third selecting means, for selecting the selected at least one edge of the second image and the vicinity produced by the correspondence converting means, to produce at least one occlusion disparity edge; disparity supplementing means for supplementing disparity data for the occlusion portion of the second image, based upon the at least one occlusion disparity edge and the disparity data; disparity extrapolating means, for extrapolating disparity data for the non-occlusion portion of the first image based upon the non-occlusion disparity edge and the disparity data; and disparity integrating means, for integrating the disparity data, the supplemented disparity data, and the extrapolated disparity data.
Claim 3
Dependent Claim
21 words
A disparity estimation apparatus as recited in claim 1, further including noise suppressing means for suppressing noise in the disparity data.
Claim 4
Dependent Claim
25 words
A disparity estimation apparatus as recited in claim 1, further including evaluation data storing means for storing evaluation data produced by the correspondence evaluation means.
Claim 5
Dependent Claim
29 words
A disparity estimation apparatus as recited in claim 1, further including occlusion data storing means for storing occlusion data corresponding to occlusion portions judged by the occlusion judging means.
Claim 6
Dependent Claim
104 words
A disparity estimation apparatus as recited in claim 1, wherein the correspondence determining means determines correspondence between at least one correspondence region of the second image and the first image; and further including region shifting means, for shifting the at least one correspondence region when the at least one correspondence region includes a portion of the second image for which corresponding evaluation data does not meet a second predetermined threshold, and for providing the shifted at least one correspondence region to the correspondence determining means for determining correspondence between the shifted at least one correspondence region of the second image and the first image.
Claim 7
Dependent Claim
74 words
An image synthesizing apparatus, comprising: the disparity estimation apparatus as recited in claim 1; disparity distance converting means, for converting the integrated disparity data into distance data; image generating means for generating hidden image data for a hidden portion of the second image based upon the occlusion portion, the integrated disparity data, and the image data; and image synthesizing means for synthesizing a third image from the image data and the hidden image data.
Claim 8
Dependent Claim
70 words
An image synthesizing apparatus as recited in claim 6, wherein the image generating means detects which side of the occlusion portion has a greater disparity; determines the hidden portion to be on the detected side of the occlusion portion; and employs image data for the occlusion portion and image data for a side of the occlusion portion in the second image opposite the selected side as the hidden image data.
Claim 9
Dependent Claim
63 words
An image synthesizing apparatus as recited in claim 6, wherein the image generating means detects which side of the occlusion portion has a greater disparity; determines the hidden portion to be on the detected side of the occlusion portion; and employs image data for a side of the occlusion portion in the second image opposite the selected side as the hidden image data.
Claim 10
Dependent Claim
21 words
An image synthesizing apparatus as recited in claim 6, further including noise suppressing means for suppressing noise in the disparity data.
Claim 11
Dependent Claim
25 words
An image synthesizing apparatus as recited in claim 6, further including evaluation data storing means for storing evaluation data produced by the correspondence evaluation means.
Claim 12
Dependent Claim
29 words
An image synthesizing apparatus as recited in claim 6, further including occlusion data storing means for storing occlusion data corresponding to occlusion portions judged by the occlusion judging means.
Claim 13
Dependent Claim
104 words
An image synthesizing apparatus as recited in claim 6, wherein the correspondence determining means determines correspondence between at least one correspondence region of the second image and the first image; and further including region shifting means, for shifting the at least one correspondence region when the at least one correspondence region includes a portion of the second image for which corresponding evaluation data does not meet a second predetermined threshold, and for providing the shifted at least one correspondence region to the correspondence determining means for determining correspondence between the shifted at least one correspondence region of the second image and the first image.
Claim 14
Dependent Claim
90 words
An image synthesizing apparatus, comprising: the disparity estimation apparatus as recited in claim 1; image shifting means for shifting at least the second image according to a designated viewing direction; image dropout detection means, for detecting no-image data regions which occur in at least the second image as a result of the image shift; image data extrapolating means, for extrapolating image data for the no-image data regions based upon the image data; and image synthesizing means for synthesizing a third image from the image data and the extrapolated image data.
Claim 15
Dependent Claim
70 words
An image synthesizing apparatus as recited in claim 13, wherein the image generating means detects which side of the occlusion portion has a greater disparity; determines the hidden portion to be on the detected side of the occlusion portion; and employs image data for the occlusion portion and image data for a side of the occlusion portion in the second image opposite the selected side as the hidden image data.
Claim 16
Dependent Claim
63 words
An image synthesizing apparatus as recited in claim 13, wherein the image generating means detects which side of the occlusion portion has a greater disparity; determines the hidden portion to be on the detected side of the occlusion portion; and employs image data for a side of the occlusion portion in the second image opposite the selected side as the hidden image data.
Claim 17
Dependent Claim
21 words
An image synthesizing apparatus as recited in claim 13, further including noise suppressing means for suppressing noise in the disparity data.
Claim 18
Dependent Claim
25 words
An image synthesizing apparatus as recited in claim 13, further including evaluation data storing means for storing evaluation data produced by the correspondence evaluation means.
Claim 19
Dependent Claim
29 words
An image synthesizing apparatus as recited in claim 13, further including occlusion data storing means for storing occlusion data corresponding to occlusion portions judged by the occlusion judging means.
Claim 20
Dependent Claim
104 words
An image synthesizing apparatus as recited in claim 13, wherein the correspondence determining means determines correspondence between at least one correspondence region of the second image and the first image; and further including region shifting means, for shifting the at least one correspondence region when the at least one correspondence region includes a portion of the second image for which corresponding evaluation data does not meet a second predetermined threshold, and for providing the shifted at least one correspondence region to the correspondence determining means for determining correspondence between the shifted at least one correspondence region of the second image and the first image.
Claim 21
Independent Claim
256 words
A method of estimating disparity, comprising: determining correspondence between a first image and a second image; evaluating a confidence of the determined correspondence to produce evaluation data; computing disparity data for the first image and the second image based upon the determined correspondence; judging an occlusion portion of the second image which is not included in the first image, based upon the disparity data and the evaluation data; extracting edges of the evaluation data; extracting edges of the first and second images; selecting at least one edge of the evaluation data which corresponds to a non-occlusion portion of the first image; integrating at least one edge of the first image with the selected at least one edge of the evaluation data to produce at least one non-occlusion disparity edge; obtaining the vicinity in the second image corresponding to the non-occlusion disparity edge produced by the integrating means, based upon the disparity data; selecting at least one edge of the second image from the occlusion portion of the second image; selecting the selected at least one edge of the second image and the vicinity produced by the correspondence converting means, to produce at least one occlusion disparity edge; supplementing disparity data for the occlusion portion of the second image, based upon the at least one occlusion disparity edge and the disparity data; extrapolating disparity data for the non-occlusion portion of the first image based upon the non-occlusion disparity edge and the disparity data; and integrating the disparity data, the supplemented disparity data, and the extrapolated disparity data.
Claim 22
Independent Claim
256 words
A method of estimating disparity, comprising: determining correspondence between a first image and a second image; evaluating a confidence of the determined correspondence to produce evaluation data; computing disparity data for the first image and the second image based upon the determined correspondence; judging an occlusion portion of the second image which is not included in the first image, based upon the disparity data and the evaluation data; extracting edges of the evaluation data; extracting edges of the first and second images; selecting at least one edge of the evaluation data which corresponds to a non-occlusion portion of the first image; integrating at least one edge of the first image with the selected at least one edge of the evaluation data to produce at least one non-occlusion disparity edge; obtaining the vicinity in the second image corresponding to the non-occlusion disparity edge produced by the integrating means, based upon the disparity data; selecting at least one edge of the second image from the occlusion portion of the second image; selecting the selected at least one edge of the second image and the vicinity produced by the correspondence converting means, to produce at least one occlusion disparity edge; supplementing disparity data for the occlusion portion of the second image, based upon the at least one occlusion disparity edge and the disparity data; extrapolating disparity data for the non-occlusion portion of the first image based upon the non-occlusion disparity edge and the disparity data; and integrating the disparity data, the supplemented disparity data, and the extrapolated disparity data.
Claim 23
Dependent Claim
117 words
A method of estimating disparity, as recited in claim 20, wherein the integration of at least one edge of the first image with the selected at least one edge of the evaluation data to produce at least one non-occlusion disparity edge includes: obtaining an absolute value of an edge component of the at least one edge of the first image; obtaining an absolute value of an edge component of the selected at least one edge of the evaluation data; and averaging the absolute value of the edge component of the at least one edge of the first image with the absolute value of the edge component of the selected at least one edge of the evaluation data.
Claim 24
Dependent Claim
90 words
A method of estimating disparity, as recited in claim 20, wherein the supplementing of disparity data for the occlusion portion of the second image includes: determining which portion of the second image adjacent the occlusion portion has the smallest disparity; and employing the disparity data for the determined portion of the second image adjacent the occlusion portion to supplement the disparity data for the occlusion portion, the disparity data being supplemented in a direction from the determined portion of the second image to the at least one occlusion disparity edge.
Claim 25
Dependent Claim
102 words
A method of estimating disparity, as recited in claim 20, wherein the extrapolation of disparity data for the non-occlusion portion of the first image includes: selecting a first point in the first image having a stable disparity and being positioned on a first side of the non-occlusion disparity edge; extrapolating disparity data from the first point to the non-occlusion disparity edge; selecting a second point in the first image having a stable disparity and being positioned on a second side of the non-occlusion disparity edge opposite the first side; and extrapolating disparity data from the second point to the non-occlusion disparity edge.
Claim 26
Dependent Claim
18 words
A method of estimating disparity, as recited in claim 20, further including filtering noise from the disparity data.
Claim 27
Dependent Claim
79 words
A method of estimating disparity, as recited in claim 20, wherein the integration of the disparity data, the supplemented disparity data, and the extrapolated disparity data includes the steps of: first, adopting disparity data that is neither for a region of the non-occlusion disparity edge nor in a region of the occlusion disparity edge; second, adopting disparity data for the region of the non-occlusion disparity edge; and third, adopting disparity data for the region of the occlusion disparity edge.
Claim 28
Dependent Claim
92 words
A method of estimating disparity as recited in claim 20, wherein the determination of correspondence between the first image and the second image includes: determining correspondence between at least one correspondence region of the second image and the first image; shifting the at least one correspondence region when the at least one correspondence region includes a portion of the second image for which corresponding evaluation data does not meet a second predetermined threshold, and determining correspondence between the shifted at least one correspondence region of the second image and the first image.
Claim 29
Dependent Claim
74 words
A method of synthesizing an image, comprising: estimating disparity for a first image and a second image as recited in claim 20, to obtain integrated disparity data; converting the integrated disparity data into distance data; generating hidden image data for a hidden portion of the second image based upon the occlusion portion, the integrated disparity data, and the image data; and synthesizing a third image from the image data and the hidden image data.
Claim 30
Dependent Claim
118 words
A method of synthesizing an image, as recited in claim 27, wherein the integration of at least one edge of the first image with the selected at least one edge of the evaluation data to produce at least one non-occlusion disparity edge includes: obtaining an absolute value of an edge component of the at least one edge of the first image; obtaining an absolute value of an edge component of the selected at least one edge of the evaluation data; and averaging the absolute value of the edge component of the at least one edge of the first image with the absolute value of the edge component of the selected at least one edge of the evaluation data.
Claim 31
Dependent Claim
91 words
A method of synthesizing an image, as recited in claim 27, wherein the supplementing of disparity data for the occlusion portion of the second image includes: determining which portion of the second image adjacent the occlusion portion has the smallest disparity; and employing the disparity data for the determined portion of the second image adjacent the occlusion portion to supplement the disparity data for the occlusion portion, the disparity data being supplemented in a direction from the determined portion of the second image to the at least one occlusion disparity edge.
Claim 32
Dependent Claim
103 words
A method of synthesizing an image, as recited in claim 27, wherein the extrapolation of disparity data for the non-occlusion portion of the first image includes: selecting a first point in the first image having a stable disparity and being positioned on a first side of the non-occlusion disparity edge; extrapolating disparity data from the first point to the non-occlusion disparity edge; selecting a second point in the first image having a stable disparity and being positioned on a second side of the non-occlusion disparity edge opposite the first side; and extrapolating disparity data from the second point to the non-occlusion disparity edge.
Claim 33
Dependent Claim
19 words
A method of synthesizing an image, as recited in claim 27, further including filtering noise from the disparity data.
Claim 34
Dependent Claim
80 words
A method of synthesizing an image, as recited in claim 27, wherein the integration of the disparity data, the supplemented disparity data, and the extrapolated disparity data includes the steps of: first, adopting disparity data that is neither for a region of the non-occlusion disparity edge nor in a region of the occlusion disparity edge; second, adopting disparity data for the region of the non-occlusion disparity edge; and third, adopting disparity data for the region of the occlusion disparity edge.
Claim 35
Dependent Claim
93 words
A method of synthesizing an image as recited in claim 27, wherein the determination of correspondence between the first image and the second image includes: determining correspondence between at least one correspondence region of the second image and the first image; shifting the at least one correspondence region when the at least one correspondence region includes a portion of the second image for which corresponding evaluation data does not meet a second predetermined threshold, and determining correspondence between the shifted at least one correspondence region of the second image and the first image.
Claim 36
Dependent Claim
26 words
A method of synthesizing an image as recited in claim 27, further including supplementing disparity data for the uncertain portions of the first and second images.
Claim 37
Dependent Claim
83 words
A method of synthesizing an image as recited in claim 27, wherein the generation of hidden image data for a hidden portion of the second image includes: detecting which side of the occlusion portion has a greater disparity; determining the hidden portion to be on the detected side of the occlusion portion; and employing image data for the occlusion portion and image data for a side of the occlusion portion in the second image opposite the selected side as the hidden image data.
Claim 38
Dependent Claim
76 words
A method of synthesizing an image as recited in claim 27, wherein the generation of hidden image data for a hidden portion of the second image includes: detecting which side of the occlusion portion has a greater disparity; determining the hidden portion to be on the detected side of the occlusion portion; and employing image data for a side of the occlusion portion in the second image opposite the selected side as the hidden image data.
Claim 39
Dependent Claim
85 words
A method of synthesizing an image, comprising: estimating disparity for a first image and a second image as recited in claim 20, to obtain integrated disparity data; shifting at least the second image according to a designated viewing direction; detecting no-image data regions which occur in at least the second image as a result of the image shift; extrapolating image data for the no-image data regions based upon the image data; and synthesizing a third image from the image data and the extrapolated image data.
Claim 40
Dependent Claim
34 words
A method of synthesizing an image as recited in claim 37, further including replacing suppressed disparity data with a median value of neighboring disparity data or a mean weighted value of neighboring disparity data.
Claim 41
Dependent Claim
63 words
A method of synthesizing an image as recited in claim 37, wherein the extrapolation of image data for the no-image data regions includes extrapolating data for a no-image data region in a first direction from a start of the no-image data region by employing image data obtained in a second direction, opposite the first direction, from the start of the no-image data region.
Claim 42
Dependent Claim
118 words
A method of synthesizing an image as recited in claim 37, wherein the integration of at least one edge of the first image with the selected at least one edge of the evaluation data to produce at least one non-occlusion disparity edge includes: obtaining an absolute value of an edge component of the at least one edge of the first image; obtaining an absolute value of an edge component of the selected at least one edge of the evaluation data; and averaging the absolute value of the edge component of the at least one edge of the first image with the absolute value of the edge component of the selected at least one edge of the evaluation data.
Claim 43
Dependent Claim
91 words
A method of synthesizing an image, as recited in claim 37, wherein the supplementing of disparity data for the occlusion portion of the second image includes: determining which portion of the second image adjacent the occlusion portion has the smallest disparity; and employing the disparity data for the determined portion of the second image adjacent the occlusion portion to supplement the disparity data for the occlusion portion, the disparity data being supplemented in a direction from the determined portion of the second image to the at least one occlusion disparity edge.
Claim 44
Dependent Claim
103 words
A method of synthesizing an image, as recited in claim 37, wherein the extrapolation of disparity data for the non-occlusion portion of the first image includes: selecting a first point in the first image having a stable disparity and being positioned on a first side of the non-occlusion disparity edge; extrapolating disparity data from the first point to the non-occlusion disparity edge; selecting a second point in the first image having a stable disparity and being positioned on a second side of the non-occlusion disparity edge opposite the first side; and extrapolating disparity data from the second point to the non-occlusion disparity edge.
Claim 45
Dependent Claim
19 words
A method of synthesizing an image, as recited in claim 37, further including filtering noise from the disparity data.
Claim 46
Dependent Claim
80 words
A method of synthesizing an image, as recited in claim 37, wherein the integration of the disparity data, the supplemented disparity data, and the extrapolated disparity data includes the steps of: first, adopting disparity data that is neither for a region of the non-occlusion disparity edge nor in a region of the occlusion disparity edge; second, adopting disparity data for the region of the non-occlusion disparity edge; and third, adopting disparity data for the region of the occlusion disparity edge.
Claim 47
Dependent Claim
93 words
A method of synthesizing an image as recited in claim 37, wherein the determination of correspondence between the first image and the second image includes: determining correspondence between at least one correspondence region of the second image and the first image; shifting the at least one correspondence region when the at least one correspondence region includes a portion of the second image for which corresponding evaluation data does not meet a second predetermined threshold, and determining correspondence between the shifted at least one correspondence region of the second image and the first image.
Claim 48
Dependent Claim
26 words
A method of synthesizing an image as recited in claim 37, further including supplementing disparity data for the uncertain portions of the first and second images.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2001-06-15 | 2001-12-15 | 2002-06-16 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2005-06-15 | 2005-12-15 | 2006-06-16 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2009-06-15 | 2009-12-15 | 2010-06-16 | $7700.00 | Window Closed / Lapsed |