US5694491A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Methods and apparatus for assessing the visibility of differences between two image sequences
Assignee: ASEV DISPLAY LABS
•
Filing: 1996-06-17
•
Granted: 1997-12-02
•
Expires: 2016-06-17
Abstract
Abstract A method and apparatus for assessing the visibility of differences between two input image sequences for improving image fidelity and visual task applications. The apparatus comprises a visual discrimination measure having a retinal sampling section, a plurality of temporal filters and a spatial discrimination section. The retinal sampling section applies a plurality of transformations to the input image sequences for simulating the image-processing properties of human vision. The temporal filters separate the sequences of retinal images into two temporal channels producing a lowpass temporal response and a bandpass temporal response. The spatial discrimination section applies spatial processing to the temporal responses to produce an image metric which is used to assess the visibility of differences between the two input image sequences.
Inventors
- 👤 Michael Henry Brill
- 👤 Jeffrey Lubin
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 12:59
CPC Classifications
G; G06; G06T; G06T5/00; G06T5/50
Claims Analysis
Total Claims: 27
Claim 1
Independent Claim
57 words
An apparatus for assessing visibility of differences between two input image sequences, said apparatus comprising: a temporal filter for filtering each of the input image sequence into a lowpass temporal response and a bandpass temporal response; and a spatial discriminator, coupled to said temporal filter, for generating an image metric from said lowpass and bandpass temporal responses.
Claim 2
Independent Claim
57 words
An apparatus for assessing visibility of differences between two input image sequences, said apparatus comprising: a temporal filter for filtering each of the input image sequence into a lowpass temporal response and a bandpass temporal response; and a spatial discriminator, coupled to said temporal filter, for generating an image metric from said lowpass and bandpass temporal responses.
Claim 3
Dependent Claim
57 words
The apparatus of claim 1, further comprises: a retinal sampler, coupled to said temporal filter, for sampling each of the input image sequence to produce a retinal sampled image sequence, where said retinal sampler simulates a sampling process of a human eye, where said input image sequence to said temporal filter is said retinal sampled image sequence.
Claim 4
Dependent Claim
146 words
The apparatus of claim 2, wherein said retinal sampler comprises: an approximator for approximating optics of a human eye by convolving said input image sequence with a function expressed as Q(ρ)=0.952exp (-2.59|ρ|.sup.1.36)+0.048exp (-2.43|ρ|.sup.1.74) where ρ is a distance in minutes of arc from a point of light, and Q(ρ) is an intensity of light at a distance ρ to produce a convolved output; a matcher, coupled to said approximator, for matching a fixation depth with an image depth of said convolved output to produce a matched output; and a sampler, coupled to said matcher, for sampling said matched output at a density of 120 pixels per degree of visual angle for foveal viewing and for sampling said matched output for non-foveal viewing at a density in accordance with ##EQU8## where d is a calculated density, e is an eccentricity in degrees, and k is a constant.
Claim 5
Dependent Claim
51 words
The apparatus of claim 1, wherein said temporal filter comprises: a first temporal filter, coupled to said spatial discriminator, for filtering said input image sequence into said lowpass temporal response; and a second temporal filter, coupled to said spatial discriminator, for filtering said input image sequence into said bandpass temporal response.
Claim 6
Dependent Claim
45 words
The apparatus of claim 4, wherein said first temporal filter is expressed as h.sub.1 (t)=λe.sup.-λt u(t) and said second temporal filter is expressed as ##EQU9## where u(t) is a unit step function, μ is a constant, λ is a consist and n is an integer.
Claim 7
Dependent Claim
28 words
The apparatus of claim 1, wherein said spatial discriminator comprises: a contrast transform for computing a local contrast for each image of said lowpass and bandpass temporal responses.
Claim 8
Dependent Claim
28 words
The apparatus of claim 6, wherein said contrast transform generates a contrast pyramid having a plurality of levels for each image of said lowpass and bandpass temporal responses.
Claim 9
Dependent Claim
64 words
The apparatus of claim 7, wherein said contrast pyramid is expressed as ##EQU10## where c k (x) is a contrast at pyramid level k, x is a two-dimensional position vector, I(x) is an image from one of said first and second temporal filters, and where G k (x) is a Gaussian convolution kernel expressed as ##EQU11## where σ k =2 k-1 σ 0 .
Claim 10
Dependent Claim
39 words
The apparatus of claim 7, wherein said spatial discriminator further comprises: a weigher, coupled to said contrast transform, for applying a weighing factor to each of said plurality of levels of said contrast pyramid to produce a normalized output.
Claim 11
Dependent Claim
91 words
The apparatus of claim 9, wherein said spatial discriminator further comprises: a transducer, coupled to said weigher, for applying a sigmoid non-linearity to said normalized output to produce a transducer output; a pooler, coupled to said transducer, for averaging said transducer output over a kernel of pixels to produce a pooled output; and a distancer, coupled to said pooler, for computing a distance between a plurality of m-dimensional vectors to produce said image metric, where each of said m-dimensional vectors represents a spatial position of each image of said pooled output.
Claim 12
Dependent Claim
157 words
The apparatus of claim 10, wherein said sigmoid non-linearity is expressed as ##EQU12## and wherein said kernel of pixels for eccentricities inside a fovea is a kernel of a foveal diameter (d 0 ) of 5 and wherein said kernel of pixels for stimulus eccentricities outside of said fovea, is a diameter d p of said kernel increasing as a linear function of eccentricity, according to the expression ##EQU13## where e is an eccentricity in degrees and k p is a scaling factor; and wherein said distance is calculated using a distance measure D according to the expression ##EQU14## where m is a number of said levels of said contrast pyramid times a number of orientations, P i (x) is a set of m arrays (where i indexes from 1 to m) for each input image x, where x 1 and x 2 are two input images of said input image sequences, and Q is a parameter.
Claim 13
Dependent Claim
122 words
The apparatus of claim 7, wherein said spatial discriminator further comprises: a spatially oriented filter, coupled to said contrast transform, for applying orientation tuning to each of said level of said contrast pyramid to produce a spatially oriented output; a transducer, coupled to said spatially oriented filter, for applying a sigmoid non-linearity to said spatially oriented output to produce a transducer output; a pooler, coupled to said transducer, for averaging said transducer output over a kernel of pixels to produce a pooled output; and a distancer, coupled to said pooler, for computing a distance between a plurality of m-dimensional vectors to produce said image metric, where each of said m-dimensional vectors represents a spatial position of each image of said pooled output.
Claim 14
Dependent Claim
14 words
The apparatus of claim 12, wherein said spatially oriented filter comprises a steerable filter.
Claim 15
Dependent Claim
139 words
The apparatus of claim 12, wherein said spatially oriented filter comprises: a convolver for convolving each level of said contrast pyramid with a set of four pairs of spatially oriented filters with each pair of spatially oriented filters comprising a directional second derivative of a Guassian and its Hilbert transform to produce a Hilbert filter output; a transformer, coupled to said convolver, for transforming said Hilbert filter output into an energy response in accordance with e.sub.k,n,θ (x.sub.t,t)=(o.sub.k,n,θ (x.sub.t,t)).sup.2 +(h.sub.k,n,θ (x,.sub.t,t)).sup.2 where θ indexes over four orientations of said four pairs of spatially oriented filters, n indexes over two channels of said temporal responses, k is a contrast pyramid level, and o and h are an oriented operator and its Hilbert transform; and a weigher, coupled to said transformer, for normalizing said energy response to produce said spatially oriented output.
Claim 16
Independent Claim
48 words
A method of assessing visibility of differences between two input image sequences, comprising the steps of: (a) filtering each of the input image sequence into a lowpass temporal response and a bandpass temporal response; and (b) discriminating said lowpass and bandpass temporal responses to generate an image metric.
Claim 17
Independent Claim
48 words
A method of assessing visibility of differences between two input image sequences, comprising the steps of: (a) filtering each of the input image sequence into a lowpass temporal response and a bandpass temporal response; and (b) discriminating said lowpass and bandpass temporal responses to generate an image metric.
Claim 18
Dependent Claim
52 words
The method of claim 15, further comprising the steps of: (a') sampling each of the input image sequence to produce a retinal sampled image sequence, where said sampling simulates a sampling process of a human eye, where said input image sequence to said filtering step (a) is said retinal sampled image sequence.
Claim 19
Dependent Claim
135 words
The method of claim 16, wherein said sampling step (a') comprises the steps of: (a'1) approximating optics of a human eye by convolving said input image sequence with a function expressed as Q(ρ)=0.952exp (-2.59|ρ|.sup.1.36)+0.048exp (-2.43|ρ|.sup.1.74) where ρ is a distance in minutes of arc from a point of light, and Q(ρ) is an intensity of light at a distance ρ to produce a convolved output; (a'2) matching a fixation depth with an image depth of said convolved output to produce a matched output; and (a'3) sampling said matched output at a density of 120 pixels per degree of visual angle for foveal viewing and sampling said matched output for non-foveal viewing at a density in accordance with ##EQU15## where d is a calculated density, e is an eccentricity in degrees, and k is a constant.
Claim 20
Dependent Claim
70 words
The method of claim 15, wherein said filtering step (a) comprises the steps of: (a1) filtering said input image sequence into said lowpass temporal response in accordance with h 1 (t)=λe - λt i(t); and (a2) filtering said input image sequence into said bandpass temporal response in accordance with ##EQU16## where u(t) is a unit step function, μ is a constant, λ is a constant and n is an integer.
Claim 21
Dependent Claim
30 words
The method of claim 15, wherein said discriminating step (b) comprises the step of: (b1 ) computing a local contrast for each image of said lowpass and bandpass temporal responses.
Claim 22
Dependent Claim
33 words
The method of claim 19, wherein said computing step (b1) comprises the step of: generating a contrast pyramid having a plurality of levels for each image of said lowpass and bandpass temporal responses.
Claim 23
Dependent Claim
64 words
The method of claim 20, wherein said generating step is in accordance with ##EQU17## where c k (x) is a contrast at pyramid level k, x is a two-dimensional position vector, I(x) is an image resulting from one of said temporal filtering steps, and where G k (x) is a Gaussian convolution kernel expressed as ##EQU18## where σ k =2 k-1 σ 0 .
Claim 24
Dependent Claim
98 words
The method of claim 20, wherein said discriminating step (b) further comprises the steps of: (b2) applying a weighing factor to each of said plurality of levels of said contrast pyramid to produce a normalized output; (b3) applying a sigmoid non-linearity to said normalized output to produce a transducer output; (b4) averaging said transducer output over a kernel of pixels to produce a pooled output; and (b5) computing a distance between a plurality of m-dimensional vectors to produce said image metric, where each of said m-dimensional vectors represents a spatial position of each image of said pooled output.
Claim 25
Dependent Claim
97 words
The method of claim 20, wherein said discriminating step (b) further comprises the steps of: (b2) applying orientation tuning to each of said level of said contrast pyramid to produce a spatially oriented output; (b3) applying a sigmoid non-linearity to said spatially oriented output to produce a transducer output; (b4) averaging said transducer output over a kernel of pixels to produce a pooled output; and (b5) computing a distance between a plurality of m-dimensional vectors to produce said image metric, where each of said m-dimensional vectors represents a spatial position of each image of said pooled output.
Claim 26
Dependent Claim
127 words
The method of claim 23, wherein said applying orientation tuning step (b2) comprises the steps of: convolving each level of said contrast pyramid with a set of four pairs of spatially oriented filters with each pair of spatially oriented filters comprising a directional second derivative of a Guassian and its Hilbert transform to produce a Hilbert filter output; transforming said Hilbert filter output into an energy response in accordance with e.sub.k,n,θ (x.sub.t,t)=(o.sub.k,n,θ (x.sub.t,t)).sup.2 +(h.sub.k,n,θ (x,.sub.t,t)).sup.2 where θ indexes over four orientations of said four pairs of spatially oriented filters, n indexes over two channels of said temporal responses, k is a contrast pyramid level, and o and h are an oriented operator and its Hilbert transform; and normalizing said energy response to produce said spatially oriented output.
Claim 27
Dependent Claim
70 words
The method of claim 20, further comprises the step of: (c) calculating a plurality of weighing factors for said plurality of levels of said contrast pyramid by adjusting a contrast of a grating pattern of a fixed spatial frequency until said grating is just detectable; and (d) repeating step (c) for a number of different spatial frequencies, where each of said spatial frequency represents one level of said contrast pyramid.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2000-12-01 | 2001-06-02 | 2001-12-02 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2004-12-01 | 2005-06-02 | 2005-12-02 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2008-12-01 | 2009-06-02 | 2009-12-02 | $7700.00 | Window Closed / Lapsed |