MADALINE Neural Networks (1962) › A Concrete Example: Voice Activity Detection

To make these abstractions more concrete, consider a MADALINE system designed specifically for voice activity detection in telephone systems. This application exemplified both the strengths and limitations of the approach, and following through the details helps build intuition for how the system operated.

The engineers designing this system started by identifying relevant features that could be extracted from the audio signal. They settled on measurements of signal energy in three frequency bands. The low-frequency band from 0 to 1 kilohertz captured the fundamental frequencies of voice and the energy of vowel sounds. The mid-frequency band from 1 to 3 kilohertz contained important information about vowel formants and some consonant sounds. The high-frequency band from 3 to 8 kilohertz picked up unvoiced consonants like "s," "t," and "th" that produced noise-like high-frequency energy. These three numbers, computed from each short window of the audio signal, formed the input vector to the MADALINE network.

The system employed three ADALINE units, each learning to recognize different aspects of the speech signal. The first ADALINE learned to detect voiced speech, the rich low-frequency energy characteristic of vowels and voiced consonants. Through training on examples of speech and noise, its weights adjusted so that it responded strongly when the low-frequency band showed the patterns typical of human vocalization. The second ADALINE specialized in detecting fricatives and other unvoiced consonants by learning the characteristic pattern of high-frequency energy they produced. The third ADALINE acted as an overall signal strength detector, learning to distinguish signals that exceeded the energy threshold typical of speech from quiet background noise.

The outputs of these three units fed into a logical combination that implemented the decision rule: classify as speech if there is evidence of either voiced speech OR unvoiced consonants, AND if the overall signal strength is adequate. This rule made linguistic sense. Speech typically contains either voiced sounds or unvoiced sounds at any given moment, but rarely neither, and distinguishing speech from noise required confirming that the signal had sufficient energy to represent actual vocalization rather than just background hiss.

During training, the system was presented with labeled examples: short segments of audio that engineers had manually classified as speech or noise. For each example, the system computed the three frequency band features, fed them through the ADALINE units, and combined the results through the logic gates to produce a classification. When this classification matched the correct label, the weights remained unchanged. They were already doing the right thing for this example. When the classification was wrong, the system needed to adjust.

The challenge in training a multi-layer system is determining which units to adjust and how. MADALINE used a simple heuristic approach. It looked at which ADALINE units had made incorrect individual predictions, and updated only those units using the LMS rule. If the final output should have been speech but the network said noise, the system examined which units had incorrectly predicted noise and adjusted their weights to make them more likely to predict speech for this pattern of inputs. This training procedure was not guaranteed to find the global optimum, but in practice it worked well for the relatively simple architectures and problems that MADALINE tackled.

The trained system could then be deployed in a telephone system where it would continuously process the incoming audio stream, extracting features from each time window and classifying whether speech was present. This information could drive various adaptive processes: noise suppression algorithms that should only operate during non-speech periods, echo cancellation systems that needed to know when the near-end speaker was talking, or voice-activated recording systems that should capture speech but ignore silent periods.