Cancellation and Rounding Errors.
Technical summary rept.,
WISCONSIN UNIV MADISON MATHEMATICS RESEARCH CENTER
Pagination or Media Count:
The fact that a calculating machine can usually hold only an approximation to the number that one is concerned with leads to cancellation errors and rounding errors. These concepts are defined precisely and examples are given. Suggestions are given for reducing when possible the size of errors arising from these two effects.
- Theoretical Mathematics
- Computer Hardware