A model of computation

In Asymptotic analysis of algorithms, we saw how, given a certain sort of operation on a list of numbers, we could work out how many times that operation needs to be applied to sort the list. However, in general, we should like to be able to investigate the resource-intensiveness of computations for all sort of problems. In some cases, it is not obvious what the analogous operations should be. We shall now investigate a more general approach that allows us to deal with a considerably greater number of problems. We begin with a seemingly arbitrary model of computation, and then explain why it is not so arbitrary after all.

Arora and Barak (Complexity: § 1.1).

Let $𝑓$ be a function that takes a string of bits…and outputs either 0 or 1. An algorrthm for computing $𝑓$ is a set of mechanical rules, such that by following them we can compute $𝑓\left(𝑥\right)$ given any input $𝑥\in {\left[0,1\right]}^{\ast }$. The set of rules being followed is fixed (i.e. the same rule must work for all possible input) though each rule in this set may be applied arbitrarily many times. Each rule involves one or more of the following ‘elementary’ operations:

1. Read a bit of the input.
2. Read a bit…from the scratch pad or working space we allow the algorithm to use.

Based on the values read,

1. Write a bit/symbol to the scratch pad.
2. Either stop and output 0 or 1, or choose a new rule from the set that will be applied next.

Finally, the running time is the number of these basic operations performed.

This seems quite arbitrary—

1. why limit ourselves just to symbols 0 and 1?
2. why are the operations so limited? and
3. why not read multiple bits at the same time?

In general, we might therefore think that the putative runtime of an algorithm required to solve a problem varies too much with respect to the model of computation. However, one class of runtimes is highly robust to such changes.

We say that the runtime of an algorithm is polynomial if its runtime is $𝑓\left(𝑛\right)\in 𝑂\left({𝑛}^{𝑘}\right)$ for some fixed $𝑘$.

Note that this definition is relative to a model of computation. However, it turns out that the class of problems that admit polynomial-time algorithms is robust with respect to all the worries above, i.e., changing any of those features does not change the class of problems for which there are polynomial-time algorithms.

van Emde Boas (‘Models’: § 1) puts it thus.

There exists a standard class of machine models…[that] simulate each other with polynomially bounded overhead…

This standard class corresponds, it is thought, to all ‘reasonable’ (Dean, ‘Complexity’: § 2.2) or physically realisable models of computation. The evidence for this is in effect that no convincing counterexamples have yet been found; it is somewhat analogous, therefore, to the Church–Turing thesis.