Idempotency is one of the fundamental ideas when managing systems: it’s both convenient and natural to demand that any management action has the same result whether it’s performed once or multiple times. For example, if the management action is ‘make sure that httpd is running’, we only want to say that once, and if, for some reason, that action gets performed multiple times, the result should stay the same. In this post, I’ll use ‘classical’ management actions on long-lived, stateful servers as examples, but the arguments apply in the same way to management actions that manipulate cloud services or kubernetes clusters, or really any other system that you’d want to manage.
It has always bothered me that it’s not obvious that stringing such idempotent actions together will always be idempotent, too.
Formally an action is a function
f that turns some system state
an updated state
f(x). Idempotent then means that
f(f(x)) = f(x), which
I’ll also write as
f f = f, dropping the argument
f. For two
g to be idempotent when we string them together then
f g f g = f g. Clearly, if
g commute, for example
f is ‘httpd must be running’ and
g is ‘crond must be running’,
the result of combining them is
f g f g = f f g g = f g because both
g are idempotent.
But what if they are not ? What if
f is ‘make sure httpd is running’ and
g is ‘make sure
httpd.conf has this specific content’ ? How can we
convince ourselves that combining these two actions is still idempotent ?
When we look at real-life management actions, they are actually more than
just idempotent: they are constant functions. No matter what state the
system is in, we want the result of
f to be that
httpd is running. That
f is not just idempotent, i.e. that
f f = f, but that for
any other management action
g, we have
f g = f. And if
f g = f and therefore
f g f g = f f = f, which
makes the combination
f g idempotent, too, but is much stronger than mere
In practice, there are of course other considerations. For example, the
action ‘make sure httpd is running’ will generally fail if
httpd is not
even installed, but that does not really affect the argument above, we’d
just have to get more technical and talk of management actions as partial
functions, where concatenating them only makes sense where they are both
defined. Similarly, we can work around order-dependency by getting a little
more formal about what we consider the system state
x and that management
actions should actually be constant on the ‘interesting’ part of the state,
and the identity everywhere else.
It therefore seems misleading to harp so much on idempotency when we talk about systems management. What we really want are constant functions, not just idempotent ones, a fact that the notion of ‘desired-state management’ nicely alludes to, but doesn’t make quite clear enough.