Blind signal processing system employing information maximization to recover unknown signals through unsupervised minimization of output redundancy

Claim 1

A method performed in a neural network having input means for receiving a plurality J of input signals (X j ) and output means for producing a plurality I of output signals (U i ) each said output signal U i representing a combination of said input signals (X j ) weighted by a plurality I of bias weights (W i0 ) and a plurality I 2 of scaling weights (W ij ) such that (U i )=(W ij )(X j )+(W i0 ), said method minimizing the information redundancy among said output signals (U j ), wherein 0<i≦I>1 and 0<j≦J>1 are integers, said method comprising: (a) selecting initial values for said bias weights (W i0 ) and said scaling weights (W ij ); (b) producing a plurality I of training signals (Y i ) responsive to a transformation of said input signals (X j ) such that Y i =g(U i ), wherein g(x) is a nonlinear function and the Jacobian of said transformation is J=det(∂Y i /∂X j ) when J=I; and (c) adjusting said bias weights (W i0 ) and said scaling weights (W ij ) responsive to one or more samples of said training signals (Y i ) such that each said bias weight Wi i0 is changed proportionately to a corresponding bias measure ΔW i0 accumulated over said one or more samples and each said scaling weight W ij is changed proportionately to a corresponding scaling measure ΔW ij =ε·∂(ln|J|)/∂W ij accumulated over said one or more samples, wherein ε>0 is a learning rate.

Claim 2

A method performed in a neural network having input means for receiving a plurality J of input signals (X j ) and output means for producing a plurality I of output signals (U i ) each said output signal U i representing a combination of said input signals (X j ) weighted by a plurality I of bias weights (W i0 ) and a plurality I 2 of scaling weights (W ij ) such that (U i )=(W ij )(X j )+(W i0 ), said method minimizing the information redundancy among said output signals (U j ), wherein 0<i≦I>1 and 0<j≦J>1 are integers, said method comprising: (a) selecting initial values for said bias weights (W i0 ) and said scaling weights (W ij ); (b) producing a plurality I of training signals (Y i ) responsive to a transformation of said input signals (X j ) such that Y i =g(U i ), wherein g(x) is a nonlinear function and the Jacobian of said transformation is J=det(∂Y i /∂X j ) when J=I; and (c) adjusting said bias weights (W i0 ) and said scaling weights (W ij ) responsive to one or more samples of said training signals (Y i ) such that each said bias weight Wi i0 is changed proportionately to a corresponding bias measure ΔW i0 accumulated over said one or more samples and each said scaling weight W ij is changed proportionately to a corresponding scaling measure ΔW ij =ε·∂(ln|J|)/∂W ij accumulated over said one or more samples, wherein ε>0 is a learning rate.

Claim 9

A method implemented in a transversal filter having an input for receiving a sensor signal X that includes a combination of multipath reverberations of a source signal S and having a plurality I of delay line tap output signals (T i ) distributed at intervals of one or more time delays τ, said source signal S and said sensor signal X varying with time over a plurality J≧I of said time delay intervals τ such that said sensor signal X has a value X j at time τ(j-1) and each said delay line tap output signal T i has a value X j+1-i representing said sensor signal value X j delayed by a time interval τ(i-1), wherein τ>0 is a predetermined constant and 0<i≦I>1 and 0<j≦J≧I are integers, said method recovering said source signal S from said sensor signal X and comprising: (a) selecting a plurality I of filter weights (W i ); (b) adjusting said filter weights (W i ) by repeatedly performing the steps of (b.1) producing a plurality K=I of weighted tap output signals (V k ) by combining said delay line tap output signals (T i ) such that (V k )=(F ki ) (T i ), wherein 0<k≦K=I>1 are integers, and wherein F ki =W k+1-i when 1≦k+1-i≦I and F ki =0 otherwise, (b.2) summing a plurality K=I of said weighted tap signals (V k ) to produce an estimation signal ##EQU24## wherein said estimation signal U has a value U j at time τ(j-1), (b.3) producing a plurality J of training signals (Y j ) responsive to a transformation of said sensor signal values (X j ) such that Y j =g(U j ) wherein g(x) is a nonlinear function and the Jacobian of said transformation is J=det(∂Y i /∂X j ) when J=I, and (b.4) adjusting each said filter weight W i responsive to one or more samples of said training signals (Y j ) such that said each filter weight W i is changed proportionately to a corresponding leading measure ΔW 1 accumulated over said one or more samples when i=1 and a corresponding scaling measure ΔW i =ε·∂(ln|J|)/∂W i accumulated over said one or more samples otherwise; and (c) producing said estimation signal U to represent said recovered source signal S.

Claim 10

A method implemented in a transversal filter having an input for receiving a sensor signal X that includes a combination of multipath reverberations of a source signal S and having a plurality I of delay line tap output signals (T i ) distributed at intervals of one or more time delays τ, said source signal S and said sensor signal X varying with time over a plurality J≧I of said time delay intervals τ such that said sensor signal X has a value X j at time τ(j-1) and each said delay line tap output signal T i has a value X j+1-i representing said sensor signal value X j delayed by a time interval τ(i-1), wherein τ>0 is a predetermined constant and 0<i≦I>1 and 0<j≦J≧I are integers, said method recovering said source signal S from said sensor signal X and comprising: (a) selecting a plurality I of filter weights (W i ); (b) adjusting said filter weights (W i ) by repeatedly performing the steps of (b.1) producing a plurality K=I of weighted tap output signals (V k ) by combining said delay line tap output signals (T i ) such that (V k )=(F ki ) (T i ), wherein 0<k≦K=I>1 are integers, and wherein F ki =W k+1-i when 1≦k+1-i≦I and F ki =0 otherwise, (b.2) summing a plurality K=I of said weighted tap signals (V k ) to produce an estimation signal ##EQU24## wherein said estimation signal U has a value U j at time τ(j-1), (b.3) producing a plurality J of training signals (Y j ) responsive to a transformation of said sensor signal values (X j ) such that Y j =g(U j ) wherein g(x) is a nonlinear function and the Jacobian of said transformation is J=det(∂Y i /∂X j ) when J=I, and (b.4) adjusting each said filter weight W i responsive to one or more samples of said training signals (Y j ) such that said each filter weight W i is changed proportionately to a corresponding leading measure ΔW 1 accumulated over said one or more samples when i=1 and a corresponding scaling measure ΔW i =ε·∂(ln|J|)/∂W i accumulated over said one or more samples otherwise; and (c) producing said estimation signal U to represent said recovered source signal S.

Claim 17

A system for adaptively cancelling one or more interferer signals (S n ) comprising: input means for receiving a plurality J of input signals (X j ) each including a combination of at least some of a plurality I of independent source signals (S i ) that includes said one or more interferer signals (S n ), wherein 0<i≦I>1, 0<j≦J≧I and 0<n≦N≧1 are integers; weight means coupled to said input means for storing a plurality I of bias weights (W i0 ) and a plurality I 2 of scaling weights (W ij ); output means coupled to said weight means for producing a plurality I of output signals (U i ) responsive to said input signals (X j ) such that (U i )=(W ij ) (X j )+(W i0 ); training means coupled to said output means for producing a plurality I of training signals (Y i ) responsive to a transformation of said input signals (X j ) such that Y i =g(U i ), wherein g(x) is a nonlinear function and the Jacobian of said transformation is J=det(∂Y i /∂X j ); adjusting means coupled to said training means and said weight means for adjusting said bias weights (W i0 ) and said scaling weights (W ij ) responsive to one or more samples of said training signals (Y i ) such that each said bias weight W i0 is changed proportionately to a corresponding bias measure ΔW i0 accumulated over said one or more samples and each said scaling weight W ij is changed proportionately to a corresponding scaling measure ΔW ij =ε·∂(ln|J|)/∂W ij accumulated over said one or more samples, wherein ε>0 is a learning rate; and feedback means coupled to said output means and said input means for selecting one or more said output signals (U n ) representing said one or more interferer signals (S n ) for combination with said input signals (X j ), thereby cancelling said interferer signals (S n ).

Claim 18

A system for adaptively cancelling one or more interferer signals (S n ) comprising: input means for receiving a plurality J of input signals (X j ) each including a combination of at least some of a plurality I of independent source signals (S i ) that includes said one or more interferer signals (S n ), wherein 0<i≦I>1, 0<j≦J≧I and 0<n≦N≧1 are integers; weight means coupled to said input means for storing a plurality I of bias weights (W i0 ) and a plurality I 2 of scaling weights (W ij ); output means coupled to said weight means for producing a plurality I of output signals (U i ) responsive to said input signals (X j ) such that (U i )=(W ij ) (X j )+(W i0 ); training means coupled to said output means for producing a plurality I of training signals (Y i ) responsive to a transformation of said input signals (X j ) such that Y i =g(U i ), wherein g(x) is a nonlinear function and the Jacobian of said transformation is J=det(∂Y i /∂X j ); adjusting means coupled to said training means and said weight means for adjusting said bias weights (W i0 ) and said scaling weights (W ij ) responsive to one or more samples of said training signals (Y i ) such that each said bias weight W i0 is changed proportionately to a corresponding bias measure ΔW i0 accumulated over said one or more samples and each said scaling weight W ij is changed proportionately to a corresponding scaling measure ΔW ij =ε·∂(ln|J|)/∂W ij accumulated over said one or more samples, wherein ε>0 is a learning rate; and feedback means coupled to said output means and said input means for selecting one or more said output signals (U n ) representing said one or more interferer signals (S n ) for combination with said input signals (X j ), thereby cancelling said interferer signals (S n ).