US7343044B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Interpolation image generating method and apparatus
Assignee: Toshiba Visual Solutions Corp
•
Filing: 2005-01-11
•
Granted: 2008-03-11
•
Expires: 2025-01-11
Abstract
Abstract An interpolation image generating method includes dividing each of the first reference image and the second reference image into reference regions each including pixels, executing a correlation operation between the first reference image and first destination images located before and after the first reference image and a correlation operation between the second reference image and a second destination image to derive motion vectors for the first and second destination images every reference region, obtaining correlation values between the regions of the first and second destination images that are indicated by the motion vectors and the reference region to determine the reference region as a high or low correlation region, generating an interpolation image candidate between the first reference and second images using the reference region determined as the high correlation region, and mixing the interpolation image candidates using the motion vectors of the reference region to produce an interpolation image.
Inventors
- 👤 Masahiro Baba
- 👤 Nao Mishima
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 21:59
CPC Classifications
H; H04; H04N; H04N7/00; H04N7/01; H04N7/0135; H04N7/014; G; G06; G06T; G06T3/00; G06T3/40; G06T3/4007; G06T7/00; G06T7/20
Claims Analysis
Total Claims: 23
Claim 1
Independent Claim
232 words
A method of generating an interpolation image between a first reference image of an input video and a second reference image thereof, comprising: dividing each of the first reference image and the second reference image into a plurality of reference regions each including a plurality of pixels, executing a correlation operation between the first reference image and each of at least two first destination images located before and after the first reference image in terms of time and a correlation operation between the second reference image and at least one second destination image corresponding to the second reference image to derive a plurality of motion vectors between the first reference image and each of the first destination images, and a plurality of motion vectors between the second reference image and the second destination image, for each reference region of the reference regions; obtaining a correlation value between the reference region and a corresponding region of at least one of the first destination images and the second destination image that is indicated by each of the motion vectors to determine the reference region as a high correlation region or a low correlation region; generating a plurality of interpolation image candidates between the first reference image and the second reference image using the reference region determined as the high correlation region; and mixing the plurality of interpolation image candidates to produce an interpolation image.
Claim 2
Independent Claim
232 words
A method of generating an interpolation image between a first reference image of an input video and a second reference image thereof, comprising: dividing each of the first reference image and the second reference image into a plurality of reference regions each including a plurality of pixels, executing a correlation operation between the first reference image and each of at least two first destination images located before and after the first reference image in terms of time and a correlation operation between the second reference image and at least one second destination image corresponding to the second reference image to derive a plurality of motion vectors between the first reference image and each of the first destination images, and a plurality of motion vectors between the second reference image and the second destination image, for each reference region of the reference regions; obtaining a correlation value between the reference region and a corresponding region of at least one of the first destination images and the second destination image that is indicated by each of the motion vectors to determine the reference region as a high correlation region or a low correlation region; generating a plurality of interpolation image candidates between the first reference image and the second reference image using the reference region determined as the high correlation region; and mixing the plurality of interpolation image candidates to produce an interpolation image.
Claim 3
Dependent Claim
114 words
The method according to claim 1 , wherein executing the correlation operation includes executing a correlation operation between the first reference image and the two first destination images located before and after the first reference image in terms of time and a correlation operation between the second reference image and the two second destination images located before and after the second reference image in terms of time for each of the reference images to derive the plurality of motion vectors between the first reference image and each of the first destination images, and the plurality of motion vectors between the second reference image and the second destination images, for each of the reference regions.
Claim 4
Dependent Claim
28 words
The method according to claim 1 , wherein the reference regions includes a square block region formed of a plurality of pixels arranged in horizontal and vertical directions.
Claim 5
Dependent Claim
28 words
The method according to claim 1 , wherein executing the correlation operation includes deriving the motion vectors by block matching from the reference image to the destination image.
Claim 6
Dependent Claim
33 words
The method according to claim 1 , wherein the obtaining the correlation value includes determining an absolute value difference between the reference region and the corresponding region every pixel as the correlation value.
Claim 7
Dependent Claim
32 words
The method according to claim 1 , wherein the mixing includes obtaining a median of the interpolation image candidates for each of the pixels and pasting the median on the interpolation image.
Claim 8
Dependent Claim
42 words
The method according to claim 1 , wherein the obtaining the correlation value includes obtaining a correlation value between the reference region and the corresponding region and determining the whole of the reference region as low correlation when the correlation is low.
Claim 9
Dependent Claim
32 words
The method according to claim 1 further comprising executing recursively the steps of executing the correlation operation, obtaining the correlation value and generating the interpolation image candidate, using the low correlation region.
Claim 10
Dependent Claim
50 words
The method according to claim 1 , wherein the mixing includes obtaining a weighted median that weighted each pixel of the interpolation image candidate increases as a temporal distance between the interpolation image and the reference images or the destination images increases, the weighted median pasted on the interpolation image.
Claim 11
Independent Claim
227 words
An apparatus to generate an interpolation image between a first reference image of an input video and a second reference image thereof, the apparatus comprising: a divider to divide each of the first reference image and the second reference image into a plurality of reference regions each having a plurality of pixels; a motion estimator to execute a correlation operation between the first reference image and at least two first destination images located before and after the first reference image in terms of time and a correlation operation between the second reference image and at least one second destination image corresponding to the second reference image and derive a plurality of motion vectors between the first reference image and the first destination image, and the second reference image and the second destination image for each of the reference regions; a correlation determination unit configured to obtain a correlation value between the reference region and a corresponding region of the destination image indicated by each of the motion vectors to determine the reference region as a high correlation region or a low correlation region; a motion compensator to generate interpolation image candidates between the first reference images and the second reference image using the reference region determined as the high correlation region; and a mixer to mix the plurality of interpolation image candidates to produce an interpolation image.
Claim 12
Independent Claim
227 words
An apparatus to generate an interpolation image between a first reference image of an input video and a second reference image thereof, the apparatus comprising: a divider to divide each of the first reference image and the second reference image into a plurality of reference regions each having a plurality of pixels; a motion estimator to execute a correlation operation between the first reference image and at least two first destination images located before and after the first reference image in terms of time and a correlation operation between the second reference image and at least one second destination image corresponding to the second reference image and derive a plurality of motion vectors between the first reference image and the first destination image, and the second reference image and the second destination image for each of the reference regions; a correlation determination unit configured to obtain a correlation value between the reference region and a corresponding region of the destination image indicated by each of the motion vectors to determine the reference region as a high correlation region or a low correlation region; a motion compensator to generate interpolation image candidates between the first reference images and the second reference image using the reference region determined as the high correlation region; and a mixer to mix the plurality of interpolation image candidates to produce an interpolation image.
Claim 13
Dependent Claim
110 words
The apparatus according to claim 10 , wherein the motion estimator includes a correlation operation unit configured to execute a correlation operation between the first reference image and the two first destination images located before and after the first reference image in terms of time and a correlation operation between the second reference image and the second destination images located before and after the second reference image in terms of time for each of the reference images to derive the plurality of motion vector between the first reference image and the first destination image, and the second reference image and the second destination image for each of the reference regions.
Claim 14
Dependent Claim
28 words
The apparatus according to claim 10 , wherein the reference regions includes a square block region formed of a plurality of pixels arranged in horizontal and vertical directions.
Claim 15
Dependent Claim
34 words
The apparatus according to claim 10 , wherein the motion estimator includes a motion vector deriving unit configured to derive the motion vectors by block matching from the reference image to the destination image.
Claim 16
Dependent Claim
39 words
The apparatus according to claim 10 , wherein the correlation determination unit includes an absolute value determination unit configured to determine an absolute value difference between the reference region and the corresponding region every pixel as the correlation value.
Claim 17
Dependent Claim
38 words
The apparatus according to claim 10 , wherein the mixer includes a median deriving unit configured to derive a median of the interpolation image candidates for each of the pixels and pasting the median on the interpolation image.
Claim 18
Dependent Claim
51 words
The apparatus according to claim 10 , wherein the correlation determination unit includes a correlation value calculator to obtain a correlation value between the reference region and the corresponding region and a determination unit configured to determine the whole of the reference region as low correlation when the correlation is low.
Claim 19
Dependent Claim
30 words
The apparatus according to claim 10 , further comprising a controller to operate recursively the motion estimator, the correlation determination unit, and the motion compensator, using the low correlation region.
Claim 20
Dependent Claim
57 words
The apparatus according to claim 10 , wherein the mixer includes a weighted median deriving unit configured to derive a weighted median that weighted each pixel of the interpolation image candidate increases as a temporal distance between the interpolation image and the reference images or the destination images increases, the weighted median pasted on the interpolation image.
Claim 21
Independent Claim
153 words
A method of generating an interpolation image between a first image and a second image, comprising: dividing each of the first image and the second image into a plurality of regions; performing motion estimation from the first image to the second image in units of region to derive a motion vector; performing, in units of region, motion estimation from the first image to a third image opposite to the second image in terms of time using the first image as a reference; performing motion estimation from the second image to the first image in units of region; obtaining a correlation value with respect to a region indicated by the motion vector to determine a high correlation region or a low correlation region; generating an interpolation image candidate between the first image and the second image using the high correlation region; and mixing a plurality of interpolation image candidates to produce an interpolation image.
Claim 22
Independent Claim
153 words
A method of generating an interpolation image between a first image and a second image, comprising: dividing each of the first image and the second image into a plurality of regions; performing motion estimation from the first image to the second image in units of region to derive a motion vector; performing, in units of region, motion estimation from the first image to a third image opposite to the second image in terms of time using the first image as a reference; performing motion estimation from the second image to the first image in units of region; obtaining a correlation value with respect to a region indicated by the motion vector to determine a high correlation region or a low correlation region; generating an interpolation image candidate between the first image and the second image using the high correlation region; and mixing a plurality of interpolation image candidates to produce an interpolation image.
Claim 23
Dependent Claim
36 words
The method according to claim 19 , which includes executing motion estimation from the second image to a fourth image opposite in terms of time to the first image using the second image as a reference.
References Cited (Backward Citations)
US6744931B2
Cited
JPH0654302A
Cited
US6327391B1
Cited
JPH09214899A
Cited
JPH11112940A
Cited
JP2000224593A
Cited
US6625333B1
Cited
US6795577B2
Cited
US6628827B1
Cited
JP2003047011A
Cited
US20030103567A1
Cited
US20040086193A1
Cited
US20040240551A1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2011-03-11 | 2011-09-10 | 2012-03-11 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2015-03-11 | 2015-09-10 | 2016-03-11 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2019-03-11 | 2019-09-10 | 2020-03-11 | $7700.00 | Window Closed / Lapsed |