When Two Calculations Almost Agree

Botie converts a report, then reverses the conversion to check the original readings. The recovered numbers may differ by a tiny rounding amount. An exact comparison can reject a perfectly useful calculation, but accepting every small-looking difference would also be careless.

We need to choose what agreement means for the task. That includes the arrangement of the result, the kind of numbers it stores, whether its entries are finite, and how much numerical error we allow.

Reference: 20
Calculated: 20.0001
Absolute difference = |20.0001 − 20| = 0.0001

Absolute difference: 0.0001
Allowed difference: 0.001
0.0001 ≤ 0.001: passes

Two routes through a calculation may differ slightly. Change one recovered reading or the permitted error and find the point where the comparison stops passing.

1

The dtype controls the representation used for every entry in an ordinary numerical array. Assigning a fraction into integer storage discards its fractional part. Converting the array afterwards cannot recover information that has already been lost.

Fixed-width integer types also have limits, unlike Python's arbitrary-precision integers. Choose a type that can represent the required range. Narrowing a type merely to save memory can change the calculation.

import numpy as np

integer_report = np.array([12,18,24], dtype=int)
integer_report[0] = 12.5
float_report = np.array([12,18,24], dtype=float)
float_report[0] = 12.5
print(integer_report.tolist())
print(float_report.tolist())
print(np.array([12,18]).astype(float).tolist())
print(np.iinfo(np.int8).max)

[12, 18, 24]
[12.5, 18.0, 24.0]
[12.0, 18.0]
127

NaN means not a number. It can mark an absent numerical observation or result from invalid arithmetic. Infinity can result from an unbounded or invalid numerical calculation. Neither is an ordinary finite thermometer reading.

NaN does not compare equal to itself. Use isnan to identify it and isfinite to require an ordinary finite value. Replacing these values is a modelling decision, so detecting them must come before silently filling them.

import numpy as np

report = np.array([0., np.nan, 18., np.inf])
print(np.isnan(report).tolist())
print(np.isfinite(report).tolist())
print(report[np.isfinite(report)].tolist())
print(np.nanmean(np.array([0., np.nan, 18.])))

[False, True, False, False]
[True, False, True, False]
[0.0, 18.0]
9.0

An absolute tolerance permits a fixed error in the result's units. A relative tolerance permits an error proportional to the reference value. np.isclose compares the difference with their combined allowance, using the second argument as the reference.

Use an explicit absolute tolerance near zero, and choose it for the required precision. allclose asks whether every aligned comparison passes. array_equal instead checks exact shape and value equality.

import numpy as np

print(0.1 + 0.2 == 0.3)
print(np.isclose(0.1 + 0.2, 0.3, rtol=0, atol=1e-12))
actual = np.array([20.0001, 18.01])
expected = np.array([20., 18.])
print(np.isclose(actual, expected, rtol=0, atol=0.001).tolist())
print(np.allclose(actual, expected, rtol=0, atol=0.001))

False
True
[True, False]
False

Approximate comparisons can broadcast compatible inputs. That is useful for some numerical checks but dangerous if a wrong result shape is supposed to fail. Confirm shape and finiteness first, then compare values.

Python's assert raises an AssertionError when a condition is false. It is useful in these learning checks and tests. For validation that must run in a deployed program, use an explicit condition and raise an error, because Python can disable assertions when run with optimisation.

import numpy as np

actual = np.array([20., 20.])
wrong_shape = np.array([[20.], [20.]])
print(np.allclose(actual, wrong_shape))
print(actual.shape == wrong_shape.shape)
expected = np.array([20., 20.])
assert actual.shape == expected.shape
assert np.isfinite(actual).all()
np.testing.assert_allclose(actual, expected, rtol=0, atol=0.001)
print("Shape, finite values, and tolerance checked.")

True
False
Shape, finite values, and tolerance checked.

Image values often arrive as unsigned eight-bit integers. Those integers have a limited range and cannot represent a negative difference. If we subtract before changing the dtype, we may already have lost the answer we wanted. Converting the result afterwards cannot undo that overflow.

We need a signed or floating-point representation before the subtraction. When adding many values, we also need a type large enough for their total. Python finishing the calculation without an exception does not tell us whether the chosen representation was suitable.

Later, probabilities will be calculated from exponentials of model scores. Large positive scores can overflow even when the final probabilities are ordinary numbers. Subtracting the largest score leaves every probability ratio unchanged while making the largest exponential one.

This is an example of changing the numerical route while keeping the mathematical quantity. A tolerance check belongs after such a calculation, together with checks for finiteness and the expected shape.

import numpy as np
scores = np.array([1000., 1001.])
weights = np.exp(scores - scores.max())
probabilities = weights / weights.sum()
print(np.round(probabilities, 4).tolist())
print(bool(np.isfinite(probabilities).all()))

[0.2689, 0.7311]
True