Decimal

Decimal is the base-10 positional number system that people use for everyday counting and money, writing every quantity with the ten digits 0 through 9 and giving each place ten times the value of the place to its right. Computers store numbers in binary, so every decimal figure typed into a program has to be converted, and for fractions that conversion is often inexact, which is the source of some of programming's most famous surprises.

Decimal's dominance is a matter of history, not mathematics. The place-value system with a zero came to Europe from Indian mathematics through Arabic sources, and Muhammad al-Khwarizmi's work around 825 helped spread it. Decimal fractions in their modern use were popularized by Simon Stevin's 1585 booklet De Thiende. The base of ten is usually attributed to counting on ten fingers, and it is deeply embedded in measurement and currency. Nothing about base ten is mathematically special: it is merely convenient for humans, which is why decimal is the system that input and output must translate to and from.

Converting a whole number from decimal to binary uses repeated division by two. To convert 13, divide by 2 and record each remainder: 13 gives 6 remainder 1, 6 gives 3 remainder 0, 3 gives 1 remainder 1, and 1 gives 0 remainder 1. Reading the remainders from last to first yields 1101. Going the other way, each digit of a binary number is multiplied by its power of two and added. Whole numbers always convert exactly in both directions, which is why integers are safe in computers until they exceed the available number of digits.

Fractions convert by repeated multiplication by two, and here the exactness breaks down. To convert 0.1, multiply by 2 and record the whole part each time: 0.2 gives 0, 0.4 gives 0, 0.8 gives 0, 1.6 gives 1 with 0.6 left, 1.2 gives 1 with 0.2 left, and the sequence returns to 0.2 and repeats forever. So 0.1 in decimal is 0.0001100110011... in binary, a repeating fraction with no end, exactly as one third is 0.333... in decimal. A fraction has a finite binary form only if its denominator, in lowest terms, is a power of two, so 0.5, 0.25, and 0.125 are exact while 0.1, 0.2, and 0.3 are not.

Typed in a program: 0.1 + 0.2
two decimal fractions
Each decimal fraction is rounded to the nearest binary double
0.1
0.000110011001100... repeating, cut at 53 significant bits
0.2
0.00110011001100... repeating, cut at 53 significant bits
Binary addition of two slightly inexact values
the sum is rounded again to the nearest double
Printed: 0.30000000000000004
Python and JavaScript both show this; 0.1 + 0.2 == 0.3 is false
Safe practices
Money as whole cents
store 1999 cents, not 19.99
Decimal types
Python Decimal, Java BigDecimal
Tolerant comparison
test whether values differ by less than a small epsilon

Floating-point formats make the rounding systematic, and the standard that governs them is IEEE 754. Published in 1985, it stores a number as a sign, an exponent, and a fixed-length fraction, in the way scientific notation does. The common 64-bit double format carries 53 significant bits, about 15 to 17 decimal digits of precision, so 0.1 is stored as a value slightly above one tenth, and 0.2 slightly above two tenths. Adding them and rounding gives a result just above 0.3, which prints as 0.30000000000000004 and is not equal to the stored nearest value to 0.3. No language is at fault: the result is the correct IEEE 754 answer to a question that the decimal notation only seemed to pose exactly.

The consequences range from harmless to serious, and the cure depends on what the numbers mean. Comparing floating-point results with an equality test is a classic bug; the remedy is to test whether two values are within a small tolerance. Summing many small values accumulates error, and subtracting nearly equal values can cancel away most of the precision. For money, where exact cents matter, programs use integers counting the smallest unit, or decimal arithmetic types that store base-10 digits exactly; IEEE 754 itself was revised in 2008 to add decimal floating-point formats for this reason. A real-world example is the 1991 Patriot missile failure at Dhahran, where a clock value in tenths of a second, truncated in binary, drifted by about a third of a second after 100 hours of continuous operation.

For data science and machine learning, the practical rule is to treat decimal values as approximations in binary. Spreadsheets and databases often display rounded numbers while holding the full binary value, so a column that appears to sum to exactly 100 can differ from 100 in the last digits. Reduced-precision formats used for neural networks have far fewer bits than doubles and so show larger rounding effects. Knowing that the trouble lies in the conversion from base ten, and not in addition itself, explains why the same decimal numbers behave perfectly when they are whole and strangely when they are fractional.

Read decimal as the human-facing notation that every computer must translate into binary — exact for whole numbers, but for many fractions only a rounded approximation, which is why 0.1 + 0.2 does not quite equal 0.3.

Take decimal further

Ask the copilot about this term, or have our engineers assess it against your process.