Effects of Finite Register Length in the Signal Processing Arithmetic Unit of the AN/UYK-17.
NAVAL RESEARCH LAB WASHINGTON DC
Pagination or Media Count:
Because of finite register length, several kinds of error are introduced in NRLs Signal Processing Arithmetic Unit SPAU. The SPAU has a 16-bit word length and uses twos-complement, fixed-point, truncated arithmetic. A statistical model for arithmetic roundoff is formulated and used in discussing sources of error in multiplication and addition. Simulation proves a useful tool for studying quantization effects in the two fundamental digital signal processing algorithms, the fast Fourier transform FFT and recursive filtering. Simulation results indicate that conditional block scaling should be used, that scaling other than scaling the input should be used for the 4-pole filter to utilize more completely the SPAUs 90-dB dynamic range 6 dB per bit, and that rounding arithmetic should be used in the next version of the SPAU. The possibility of a floating-point machine should be investigated. Author
- Computer Hardware