US5729694A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Speech coding, reconstruction and recognition using acoustics and electromagnetic waves
Assignee: University of California
•
Filing: 1996-02-06
•
Granted: 1998-03-17
•
Expires: 2016-02-06
Abstract
Abstract The use of EM radiation in conjunction with simultaneously recorded acoustic speech information enables a complete mathematical coding of acoustic speech. The methods include the forming of a feature vector for each pitch period of voiced speech and the forming of feature vectors for each time frame of unvoiced, as well as for combined voiced and unvoiced speech. The methods include how to deconvolve the speech excitation function from the acoustic speech output to describe the transfer function each time frame. The formation of feature vectors defining all acoustic speech units over well defined time frames can be used for purposes of speech coding, speech compression, speaker identification, language-of-speech identification, speech recognition, speech synthesis, speech translation, speech telephony, and speech teaching.
Inventors
- 👤 John F. Holzrichter
- 👤 Lawrence C. Ng
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-01 03:46
CPC Classifications
G; G10; G10L; G10L15/00; G10L15/24; G06; G06Q; G06Q20/00; G06Q20/08; G06Q20/20; G06Q20/204; G01; G01N; G01N2291/00; G01N2291/02; G01N2291/024; G01N2291/02491; G01N2291/028; G01N2291/02836; G01N2291/02872; G10L15/02; G10L19/00
Claims Analysis
Total Claims: 44
Claim 1
Independent Claim
67 words
A method for characterizing speech, comprising: directing EM radiation toward speech organs of a speaker; detecting EM radiation scattered from the speech organs to obtain speech organ information; detecting acoustic speech output from the speaker to obtain acoustic speech information; combining the EM speech organ information with the acoustic speech information using a speech coding algorithm to obtain the speaker's excitation function and speech tract transfer function.
Claim 2
Independent Claim
67 words
A method for characterizing speech, comprising: directing EM radiation toward speech organs of a speaker; detecting EM radiation scattered from the speech organs to obtain speech organ information; detecting acoustic speech output from the speaker to obtain acoustic speech information; combining the EM speech organ information with the acoustic speech information using a speech coding algorithm to obtain the speaker's excitation function and speech tract transfer function.
Claim 3
Dependent Claim
12 words
The method of claim 1 further comprising defining a speech time frame.
Claim 4
Dependent Claim
20 words
The method of claim 2 further comprising defining the time of start, stop, and duration of the speech time frame.
Claim 5
Dependent Claim
15 words
The method of claim 2 further comprising forming feature vectors for each speech time frame.
Claim 6
Dependent Claim
11 words
The method of claim 4 further comprising forming difference feature vectors.
Claim 7
Dependent Claim
20 words
The method of claim 4 further comprising comparing a feature vector to stored feature vector information to identify a speaker.
Claim 8
Dependent Claim
36 words
The method of claim 4 further comprising comparing a feature vector to stored feature vector information in many language codebooks to identify the language being used by the speaker for the formation of acoustic speech units.
Claim 9
Dependent Claim
22 words
The method of claim 4 further comprising normalizing the feature vector of a speaker to that of one or more reference speakers.
Claim 10
Dependent Claim
38 words
The method of claim 4 further comprising quantizing a continuous coefficient-value band of a feature vector to a small number of distinct coefficient values representing a small number of distinct user-discernible, application-related speech conditions defined by each coefficient.
Claim 11
Dependent Claim
50 words
The method of claim 4 further comprising defining acoustic speech unit feature vectors by combining one or more excitation function feature vectors, vocal tract transfer function feature vectors, prosody feature vectors, timing, algorithm control coefficients, neighboring frame connectivity coefficients, and acoustic feature vectors for all acoustic units in a language.
Claim 12
Dependent Claim
21 words
The method of claim 10 further comprising storing the acoustic speech unit feature vectors in a library, code book, or database.
Claim 13
Dependent Claim
128 words
The method of claim 10 where a speech segment is compressed by: forming a sequence of feature vectors for each sequential time frame in the speech segment; comparing sequential changes in the feature vector coefficients, for each feature vector in the sequence, against a predefined model describing change in one or more of the coefficients over the sequential time frames; forming a single representative feature vector for several time frames over which the coefficients meet the criteria of the predefined model; adding to the representative feature vector extra coefficients describing the predefined model and a parametric fit to the model; adding the total duration time of the several time frames to the representative, multi-time frame feature vector as an extra coefficient; storing or transmitting the compressed segment electronically.
Claim 14
Dependent Claim
34 words
The method of claim 10 further comprising generating said combined feature vectors with identifying symbols for all acoustic speech units used in a language and storing them in a library, codebook or data base.
Claim 15
Dependent Claim
41 words
The method of claim 13 further comprising associating a foreign language word or phrase symbol in a second language with each unit of a first language coded by a speaker or speakers and storing them in a codebook or data base.
Claim 16
Dependent Claim
44 words
The method of claim 10 further comprising averaging feature vector coefficients from the excitation, transfer, acoustic, prosody, and timing functions of one or more speakers to form a reference speaker acoustic sound unit feature vector and storing them in a codebook or data base.
Claim 17
Dependent Claim
49 words
The method of claim 10 further comprising modifying feature vector coefficients and functional representations of the excitation, transfer, acoustic, prosody, neighboring frame connectivity, and timing functions of one or more speakers to form a modified acoustic sound unit feature vector and storing them in a codebook or data base.
Claim 18
Dependent Claim
18 words
The method of claim 4 further comprising identifying all sound units in a language from the feature vectors.
Claim 19
Dependent Claim
63 words
The method of claim 17 further comprising identifying all acoustic speech units in a language by a method selected from the group consisting of template matching techniques, HMM techniques, neural network techniques, a method of joint probabilities of two or more identifying algorithms, and a method of exclusion to reject identified units in a sequence of tests by two or more identifying algorithms.
Claim 20
Dependent Claim
20 words
The method of claim 17 further comprising identifying each acoustic speech unit with a symbol of the language unit identified.
Claim 21
Dependent Claim
52 words
The method of claim 4 further comprising communicating back to the speaker or to others speech organ articulation qualities, which are coded in the feature vectors for the speech time frames, by using communication vehicles selected from the group consisting of visual images, printed information, acoustic messages, and tactile and/or electrical stimulus.
Claim 22
Dependent Claim
22 words
The method of claim 2 wherein the speech time frame is defined by measuring glottal opening and closing using reflected EM waves.
Claim 23
Dependent Claim
22 words
The method of claim 21 further comprising defining a composite time frame from two or more glottal opening and closing time frames.
Claim 24
Dependent Claim
29 words
The method of claim 21 further comprising precalibrating an EM sensor so that the EM signals can be converted to either pressure and/or volume air flow in real time.
Claim 25
Dependent Claim
30 words
The method of claim 21 wherein a voiced excitation function feature vector is described by numerical table values or by fitting a mathematical functional model to the numerical table values.
Claim 26
Dependent Claim
26 words
The method of claim 21 comprising defining an unvoiced speech time frame by the absence of EM detected glottal opening/closing and the presence of acoustic output.
Claim 27
Dependent Claim
18 words
The method of claim 21 comprising forming the feature vector for combined voiced and unvoiced speech time frames.
Claim 28
Dependent Claim
13 words
The method of claim 2 comprising obtaining the excitation function for unvoiced speech.
Claim 29
Dependent Claim
23 words
The method of claim 1 further comprising deconvolving the speech excitation function from the acoustic speech information to produce a deconvolved transfer function.
Claim 30
Dependent Claim
21 words
The method of claim 28 further comprising forming a feature vector by fitting the deconvolved transfer function to a mathematical model.
Claim 31
Dependent Claim
41 words
The method of claim 29 wherein the feature vector is formed by one of numerical table look-up, Fourier transform, an ARMA model technique, an electrical or mechanical analog model of the acoustic system, or an organ-dimension physiological/acoustic-model of the acoustic system.
Claim 32
Dependent Claim
25 words
The method of claim 29 further comprising choosing the transfer function mathematical model using EM sensor information describing the dimensions and locations of vocal organs.
Claim 33
Dependent Claim
15 words
The method of claim 31 further comprising obtaining the transfer function using real time measurements.
Claim 34
Dependent Claim
33 words
The method of claim 29 further comprising dividing the transfer function into "important" pole-zero terms describing major vocal tract configurations and "less-important" pole-zero terms describing idiosyncratic speaker's vocal organ physical and acoustical conditions.
Claim 35
Dependent Claim
35 words
The method of claim 1 wherein the EM radiation is directed to and reflected from the glottal region and is sensed in the near field mode, the intermediate field mode, or the far field mode.
Claim 36
Dependent Claim
17 words
The method of claim 1 further comprising synthesizing speech from the EM and acoustic speech organ information.
Claim 37
Dependent Claim
99 words
The method of claim 35 wherein speech is synthesized by: generating a code book of reference speaker feature vectors and identifying symbols; identifying speech units for synthesis using a text to speech translator; selecting the sound units and timing; providing selected sound feature vectors from a stored data base; concatenating the sound units in speech sound sequences; modifying feature vector coefficients or sequences of feature vector coefficients using prosody rules; modifying the time duration of individual sounds; and generating sound feature vectors by convolving the modified excitation functions with the modified transfer functions to produce an output sound function.
Claim 38
Dependent Claim
23 words
The method of claim 36 further comprising measuring positions on an excitation function amplitude versus time function to join speech interval segments together.
Claim 39
Dependent Claim
22 words
The method of claim 37 further comprising using a time during glottal closure as a timing marker for joining speech frame segments.
Claim 40
Dependent Claim
26 words
The method of claim 1 further comprising coding acoustic speech units, transmitting the codes to a receiver system, and reconstructing the transmitted codes to acoustic speech.
Claim 41
Dependent Claim
11 words
The method of claim 39 wherein the codes are symbolic codes.
Claim 42
Dependent Claim
29 words
The method of claim 39 further comprising modifying the codes to transmit minimal information, and reconstructing the codes to acoustic speech using locally stored code books of reference speakers.
Claim 43
Dependent Claim
34 words
The method of claim 39 further comprising obtaining an associated foreign language symbol or speech code, transmitting the foreign language code to the receiver system, and reconstructing to acoustic speech in the foreign language.
Claim 44
Dependent Claim
59 words
The method of claim 39 further coding the acoustic speech units in a first language, transmitting the coded information from the first language, recognizing the transmitted coded units, obtaining associated language symbols or speech codes in a second language from a system codebook at the receiver system, and reconstructing acoustic speech in the second language at the receiver system.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2001-03-16 | 2001-09-15 | 2002-03-17 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2005-03-16 | 2005-09-15 | 2006-03-17 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2009-03-16 | 2009-09-15 | 2010-03-17 | $7700.00 | Window Closed / Lapsed |