roundoff error
roundoff error The error caused by truncating numbers in a calculation, usually necessitated because registers in a computer can only hold numbers of a fixed length, say t binary digits. Arithmetical operations on such numbers often give results requiring more than t digits for their representation, which must then be reduced to t digits for further calculation. The nearest t-digit approximation may be used (rounding) or digits after the tth may be dropped (chopping or truncation). The repeated reduction to t digits can cause systematic buildup of error in certain types of calculation.
More From encyclopedia.com
You Might Also Like
NEARBY TERMS
roundoff error