Patent US5745668A successfully refreshed from Google Patents.
US5745668A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Example-based image analysis and synthesis using pixelwise correspondence
Assignee: Massachusetts Institute of Technology
•
Filing: 1997-02-12
•
Granted: 1998-04-28
•
Expires: 2017-02-12
Abstract
Abstract Synthesis of novel images from example images is achieved by determining a pixelwise optical flow among example images, computing a parameter set for a new image, and synthesizing the new image based on the parameter vector and the example images. The parameter set may describe characteristics of the image, in which case the characteristics are applied to a neural network trained with the example images in order to synthesize the novel image. The parameter set may also be an estimate of relative contributions of each of the example images to the new image, in which case the new image may be synthesized by taking a linear combination of the example images, weighted by the parameter set. In one embodiment, both the example and the new images are of the same object. In another embodiment, the example image represents a first object while the new image represents a second object. In yet another embodiment, a set of target images is synthesized from a single target image and a set of source images. Analysis of existing images to determine image parameters is achieved by determining parameters for the existing image based on comparison of the existing image with an image set for which parameters are known.
Inventors
- 👤 Tomaso Poggio
- 👤 David J. Beymer
- 👤 Amnon Shashua
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 11:12
CPC Classifications
G; G06; G06T; G06T9/00; G06T9/001
Claims Analysis
Total Claims: 28
Claim 1
Independent Claim
94 words
A method of synthesizing a new image from a plurality of example images, the example images and the new image each being comprised of a plurality of pixels and characterized by a parameter set, the method comprising: analyzing said example images to determine pixelwise vector representations of said example images, the pixelwise vector representations providing pixel-by-pixel positions of said sample images with respect to a reference; determining a desired parameter set for said new image; and synthesizing said new image from said example images using said pixelwise vector representations and said desired parameter set.
Claim 2
Independent Claim
94 words
A method of synthesizing a new image from a plurality of example images, the example images and the new image each being comprised of a plurality of pixels and characterized by a parameter set, the method comprising: analyzing said example images to determine pixelwise vector representations of said example images, the pixelwise vector representations providing pixel-by-pixel positions of said sample images with respect to a reference; determining a desired parameter set for said new image; and synthesizing said new image from said example images using said pixelwise vector representations and said desired parameter set.
Claim 3
Dependent Claim
44 words
A method as in claim 1, wherein analyzing said example images to determine pixelwise vector representations includes determining pixelwise optical flows among each of the example images and the reference, and wherein the reference is derived from at least one of said example images.
Claim 4
Dependent Claim
45 words
A method as in claim 1, the method further comprising training a neural network with deformation information derived from optical flow applied to each of the example images such that subsequent application of the desired parameter set to the neural network results in said synthesizing.
Claim 5
Independent Claim
108 words
A method of synthesizing a new image from a first example image and a second example image, the example images and the new image each being comprised of a plurality of pixels, the method comprising: analyzing said first example image and said second example image to determine a pixelwise optical flow from said first example image to said second example image, the pixelwise optical flow providing a pixel-by-pixel relative position of said second example image with respect to said first example image; determining a desired parameter set for said new image; and synthesizing said new image from said example images using said optical flow and said parameter set.
Claim 6
Independent Claim
108 words
A method of synthesizing a new image from a first example image and a second example image, the example images and the new image each being comprised of a plurality of pixels, the method comprising: analyzing said first example image and said second example image to determine a pixelwise optical flow from said first example image to said second example image, the pixelwise optical flow providing a pixel-by-pixel relative position of said second example image with respect to said first example image; determining a desired parameter set for said new image; and synthesizing said new image from said example images using said optical flow and said parameter set.
Claim 7
Independent Claim
102 words
A method of transmitting images from a transmitting location to a receiving location, comprising: transmitting a subset of said images as training images from said transmitting location to said receiving location; determining a pixelwise optical flow for a subsequent image, the pixelwise optical flow providing a pixel-by-pixel position of said subsequent image with respect to a reference; determining a parameter set for said subsequent image using the pixel-by-pixel position; transmitting said parameter set from said transmitting location to said receiving location; and synthesizing a replica of said subsequent image at said receiving location using said training images and said transmitted parameter set.
Claim 8
Independent Claim
102 words
A method of transmitting images from a transmitting location to a receiving location, comprising: transmitting a subset of said images as training images from said transmitting location to said receiving location; determining a pixelwise optical flow for a subsequent image, the pixelwise optical flow providing a pixel-by-pixel position of said subsequent image with respect to a reference; determining a parameter set for said subsequent image using the pixel-by-pixel position; transmitting said parameter set from said transmitting location to said receiving location; and synthesizing a replica of said subsequent image at said receiving location using said training images and said transmitted parameter set.
Claim 9
Dependent Claim
16 words
A method as in claim 5, wherein said parameter set defines characteristics of said subsequent image.
Claim 10
Dependent Claim
41 words
A method as in claim 5, wherein said parameter set defines relative contributions of said training images, and wherein said replica of said subsequent images is synthesized by taking a linear combination of said training images weighted by said parameter set.
Claim 11
Independent Claim
144 words
A method of synthesizing a new target image by mapping image parameters from a source image set to a target image set, comprising: defining a first set of image parameters for said source image set by using a subset of said source image set as a source training subset; mapping said first set of image parameters to a second set of image parameters by using a subset of said target image set as a target training subset; determining image parameters of a first image in said source image set based on pixelwise optical flow among ones of said source image set, said pixelwise optical flow providing a pixel-by-pixel position of said first image with respect to a reference; and synthesizing said new target image from said target image set by applying said determined image parameters of said first image to said target image set.
Claim 12
Independent Claim
144 words
A method of synthesizing a new target image by mapping image parameters from a source image set to a target image set, comprising: defining a first set of image parameters for said source image set by using a subset of said source image set as a source training subset; mapping said first set of image parameters to a second set of image parameters by using a subset of said target image set as a target training subset; determining image parameters of a first image in said source image set based on pixelwise optical flow among ones of said source image set, said pixelwise optical flow providing a pixel-by-pixel position of said first image with respect to a reference; and synthesizing said new target image from said target image set by applying said determined image parameters of said first image to said target image set.
Claim 13
Independent Claim
69 words
A method of synthesizing a novel image from a single example image, comprising: determining a mapping for pixelwise image changes by comparing prototype images in a prototype image set; determining a pixelwise correspondence between said example image and one of said prototype images using pixelwise optical flow; and synthesizing said novel image responsive to the results of said determining a pixelwise correspondence and responsive to said determining a mapping.
Claim 14
Independent Claim
69 words
A method of synthesizing a novel image from a single example image, comprising: determining a mapping for pixelwise image changes by comparing prototype images in a prototype image set; determining a pixelwise correspondence between said example image and one of said prototype images using pixelwise optical flow; and synthesizing said novel image responsive to the results of said determining a pixelwise correspondence and responsive to said determining a mapping.
Claim 15
Dependent Claim
57 words
An image processing system, comprising: an optical flow processor adapted to determine pixelwise optical flows among a plurality of applied images, the pixelwise optical flows providing pixel-by-pixel positions of said applied images with respect to a reference; and an image synthesizer coupled to said optical flow processor and adapted to synthesize novel images using said pixel-by-pixel positions.
Claim 16
Dependent Claim
57 words
An image processing system, comprising: an optical flow processor adapted to determine pixelwise optical flows among a plurality of applied images, the pixelwise optical flows providing pixel-by-pixel positions of said applied images with respect to a reference; and an image synthesizer coupled to said optical flow processor and adapted to synthesize novel images using said pixel-by-pixel positions.
Claim 17
Dependent Claim
18 words
An image processing system as in claim 10, wherein said image synthesizer is implemented as a neural network.
Claim 18
Dependent Claim
26 words
An image processing system as in claim 10, wherein said novel images are defined by parameter vectors and wherein said image synthesizer uses said parameter vectors.
Claim 19
Dependent Claim
46 words
An image processing system, comprising: an optical flow processor adapted to determine pixelwise optical flows among a plurality of applied images; and an image analyzer coupled to said optical flow processor and adapted to estimate parameters of applied novel images responsive to said pixelwise optical flows.
Claim 20
Dependent Claim
46 words
An image processing system, comprising: an optical flow processor adapted to determine pixelwise optical flows among a plurality of applied images; and an image analyzer coupled to said optical flow processor and adapted to estimate parameters of applied novel images responsive to said pixelwise optical flows.
Claim 21
Dependent Claim
18 words
An image processing system as in claim 13, wherein said image analyzer is implemented as a neural network.
Claim 22
Dependent Claim
85 words
An image processing system, comprising: a first optical flow processor adapted to determine first pixelwise optical flows among a plurality of applied source images, the first pixelwise optical flows providing pixel-by-pixel positions of said applied source images with respect to a reference; an image analyzer coupled to said first optical flow processor and adapted to estimate parameters of applied novel images using said pixel-by-pixel positions; and an image synthesizer coupled to said second image analyzer and adapted to synthesize novel images using said estimated parameters.
Claim 23
Dependent Claim
85 words
An image processing system, comprising: a first optical flow processor adapted to determine first pixelwise optical flows among a plurality of applied source images, the first pixelwise optical flows providing pixel-by-pixel positions of said applied source images with respect to a reference; an image analyzer coupled to said first optical flow processor and adapted to estimate parameters of applied novel images using said pixel-by-pixel positions; and an image synthesizer coupled to said second image analyzer and adapted to synthesize novel images using said estimated parameters.
Claim 24
Dependent Claim
24 words
An image processing system as in claim 15, wherein said source images represent a first object and said novel images represent a second object.
Claim 25
Dependent Claim
19 words
An image processing system as in claim 15, wherein said image synthesizer is remotely located from said image analyzer.
Claim 26
Dependent Claim
47 words
An image processing system as in claim 15, further comprising a second optical flow processor adapted to determine second pixelwise optical flows among a plurality of applied target images, wherein said image synthesizer is also adapted to synthesize novel images responsive to said second pixelwise optical flows.
Claim 27
Dependent Claim
18 words
A system as in claim 18, wherein said source images and said target images represent a single object.
Claim 28
Dependent Claim
22 words
A system as in claim 18, wherein said source images represent a first object and said target images represent a second object.
References Cited (Backward Citations)
US4600919A
Cited
US4600919B1
Cited
US4672444A
Cited
US4663660A
Cited
US4797836A
Cited
US5041916A
Cited
US5025282A
Cited
US5247583A
Cited
US5029997A
Cited
US5067014A
Cited
US5187592A
Cited
US5261041A
Cited
US5184218A
Cited
US5416899A
Cited
US5245672A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2001-04-27 | 2001-10-27 | 2002-04-28 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2005-04-27 | 2005-10-27 | 2006-04-28 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2009-04-27 | 2009-10-27 | 2010-04-28 | $7700.00 | Window Closed / Lapsed |