Programming languages should expose a flexible variety of explicit types and operators for handling arithmetic overflow and related problems. As language design problems go, this might be relatively less difficult to achieve. Rust is closest to where programming languages need to be, but not all the way there yet.
I believe that software should be able to reliably compute arithmetic expressions. You may say I’m a dreamer, but I’m not the only one.
To represent an infinite set in finite space is problematic. This is not news to most software engineers. We always need a coherent policy for how to deal with our inability to represent some elements of infinite sets, or expressions that would evaluate to representable elements if only we had space.
We can consistently apply some policy (whatever it may be) using the language’s type system: by encoding policy in the set’s type, and operators on and functions of it, we can get (if nothing else) consistent representation of and behavior in error states. (I’ll call them representation errors generically.)
Surely, we can all agree that this is the minimum necessary for program correctness.
Surely, we can all agree that correctness is the minimum necessary for program safety.
Surely, we can all agree that providing people correct and safe software is our duty as engineers.
Even the most fundamental objects of computation, the reals and the integers and arithmetic on them, require some policy for unrepresentable values — even if it is simply to crash when no more memory is available to an arbitrary-precision arithemtic library (for example).
Machine words are more limited still, and have far less range to represent the reals and integers than do arbitrary-precision types. Even so, for practical efficiency, we typically use machine words to represent elements of these sets. For many circumstances, this is not a problem.
The machines we typically use implement reasonable error policies, at least for the integer types. For example, ARM’s and Intel’s policy is to make arithmetic modular, and to indicate carry, borrow, and overflow in a status register. (See e.g. carry flag, overflow flag, and conditional execution.)
Unfortunately, most of the programming languages that aim for machine-level
efficiency — e.g. C, C++, and Rust — do not provide a standard way to access the
machine status register directly, or otherwise indicate representation errors.
(Rust does provide indirect access, however; see below.) Programmers must switch
to machine language, or must use non-standard, implementation-specific features
such as __builtin_add_overflow.
It would be helpful if these languages’ developers defined standardized syntax and semantics for accessing the error state that the machine provides. (If the programmer uses such a mechanism, but compiles for a machine that doesn’t signal representation errors, the compiler could emulate such a signal, otherwise attempt to accommodate the programmer, or raise a compilation error.)
However, some languages often violate and obscure the clear and simple policy that most machines define.
| Language | Unrepresentable Signed Integer Behavior | Unrepresentable Unsigned Integer Behavior |
|---|---|---|
| C, C++ | Undefined behavior | Modular arithmetic |
| Rust (default Release configuration) | Modular arithmetic | Modular arithmetic |
| Rust (default Debug configuration) | Machine trap | Machine trap |
| Java, Go | Modular arithmetic | Modular arithmetic |
There are a number of problems with this state of affairs:
Undefined behavior is hard or impossible to test for, and leads to increased cognitive load for the programmer. The programmer has to divine the behavior that the compiler will choose to generate, including in different build configurations and at different optimization levels.
This can lead to situations where what programmers test (e.g. debug builds) is not the same as the code that runs in production, and the production behavior differs from what was tested.
Needless to say, when the ultimate behavior of the program is not predicted, intended, or tested by the programmer, bugs inevitably ensue — sometimes including exploitable vulnerabilities.
Having different policies for signed and unsigned integer types exposes the quirks of obsolete and special-use machines, and increases cognitive load on the programmer. In fact, many programmers are unaware of the policy difference. It is necessary and good to expose machine-specific behavior in machine languages, but it’s undesirable in higher-level languages.
Trapping is a rather rigid policy, and programmers may not expect it as the default, at least at first. It has the benefit of exposing the representation problem early in the development process, though. In a language that offers many policy choices, trapping might be the best default.
Rust has the problem that the tested behavior is not the production behavior, unless the programmer takes the somewhat unusual step of changing the default build configurations. This might be an unwelcome surprise after trapping behavior during testing.
I suppose the theory is that the trapping behavior will shake out all the bugs, but it can definitely be the case that program inputs seen during testing differ from those seen in production — especially in the case where the program must handle intentionally hostile input.
People often say that since Rust provides array bounds checking (in all builds), not trapping on integer overflow in production is OK. That is true as far as it goes, but invalid array access is not the only possible consequence of integer overflow.
The set of policy options is incomplete, and insufficient for the full
range of program requirements. For all languages, it would be useful to have
a suite of standardized policies available, including at least wrapping, trapping,
clamping/saturation, undefined behavior, and promotion to BigInt or
other arbitrary precision type.
Only Rust programmers have easy access to choice. If the language’s policy is not right for the program requirements, the programmer must go to extraordinary lengths to implement the right policy — typically without any help from standard mechanisms to observe the machine’s error state.
Rust is closest to where we need to be. If a program requires a different
policy, Rust provides a
Wrapping type and operators like wrapping_add,
overflowing_add,
saturating_add,
checked_add,
and unchecked_add.
The usual reason given for the use of undefined behavior is that it can
enable certain micro-optimizations. This is potentially useful and a legitimate
policy option. (Language designers hesitate to pay the micro-cost of overflow
checking even in special
cases. But to me it seems like unchecked_add is available for
the few people whose application is unacceptably affected by a check.)
In any case, and in any language, “potentially (but untestably) fast, hard to predict, possibly wrong, and possibly unsafe”, even if occasionally desirable, should not be the default policy, as it is for signed arithmetic in C and C++.
Fortunately, all of the languages in mainstream use have active standards
committees and change processes. They could define ways to access the machine
state, and could define standard types and operators implementing a wide range
of reasonable representation error policies. (In addition to the suite of
special policy operators that Rust provides, it’d help to have types whose
default operators implement each of those policies, as well — not just
Wrapping.)
In some cases, it may even be possible to change the default policies. I think language designers and standardizers should do so.
(Perhaps something similar could be done for the error modes of floating-point arithmetic, too.)