Quantization Error and Artifact Reduction (QEAR) algorithm

White Paper

Digital signals often have discernible quantization error as well as other error artifacts which are together considered static, or pseudo-random noise. If we consider the DFT profile of an otherwise perfectly quarantined and artifact-free signal s(n) with only one bit of error, and located at (n=0), the discrete difference of this error bit (represented by a upward spike of magnitude unity followed immediately by a downward spike of magnitude unity), has a Gaussian frequency profile, with its peak located at half the sampling frequency (fs/2).

We can further view the discrete difference of the net error signal function de(n)=e(n+1)-e(n) as being comprised of superposition of such centered Gaussian functions, only with the elements having magnitude and phase corrections to account for the size and time shifting of each discrete difference error bit. By inspection this net frequency domain function also has a pseudo-Gaussian DFT with its peak around half the sampling frequency.

Concerning the discrete difference error signal, de(n), being processed by the restorative up-sampling (RUP) algorithm, the Gaussian shape is preserved, but, to our advantage, the center-point of the Gaussian distribution is shifted upwards to half the newly increased sampling frequency. This concentrates the spectral weight of this noise around higher frequencies. Therefore, once a noise-laden signal is up-sampled using restorative up-sampling to the desired higher sampling frequency (fd), an interesting possibility presents itself.

Again, in the newly up-sampled signal, the net spectral weight of the error signal (de(n)) is concentrated around higher and higher frequencies. This fact combined with a look at the shape of Gaussian curves suggests that the frequency components of our discrete difference error at lower frequencies is reduced as the signal is up-sampled using restorative up-sampling.

Taking these facts in hand, we see the possibility of using RUP to over-sample the discrete difference of a signal containing error, and then applying a high-order low-pass digital filter with roll-off at (fd) to significantly reduce the quantization error and artifact noise, the bulk of whose spectral weight now lies well above the desired sampling frequency. Following this we can perform numerical integration to undo the discrete difference operation, and then down-sample the over-sampled signal to the desired sampling frequency (fd). A high-pass filter is included to handle low-frequency noise from the numerical integration.

We speculate that the RUP function treats pure signal differently than noise because noise seems to be built up piecemeal in the time-domain, whereas pure signal is composed of frequency-domain tones, but this just qualifies as a "guess." If our reasoning and logic for this and supporting pages proves correct, a MATLAB rendering of this proposed algorithm appears on the last page of the website.

As for test files, "E" contains an audio-clip of U2's "With or Without You" with an equal RMS amount of pseudo-random noise added. This input was passed through the QEAR noise reduction algorithm with three frequency doublings set (p=3) to yield the output File "F." We are listening for a marked reduction of the amount of noise mixed in with the song-signal in the output of the QEAR algorithm.