Schroedinger from nothing

When learning Quantum Mechanics the rules can seem very abstract, and also very arbitrary.  The starting point is often taken to be the Schroedinger Equation, a deterministic equation describing the rate of change of an isolated system's state, and the Born Rule, which describes how that state gives rise to probabilities for each outcome when you interact with the system by making a measurement.  These two form the axioms of quantum mechanics from which everything else flows.

These axioms seem incredibly arbitrary, which makes accepting them harder.  In this post I will show that you can, in fact, start with much more basic axioms and then derive both the Schroedinger Equation and the Born Rule from them.  What's more, we'll be able to show how collapse of the wavefunction emerges from these simpler axioms as a subjective phenomenon.  This then deals with the measurement problem, which is that collapse appears to require us to arbitrarily divide the world into quantum systems which, left alone, evolve continuously and deterministically according to Schroedinger, and classical systems that act on quantum systems by taking measurements, forcing them to collapse discretely and probabilistically according to Born.

So, what do we need to assume?  It turns out that we need little more than that the state is a vector $\lvert \Psi \rangle$ in a complex vector space, and that the map $\lvert \Psi(0) \rangle \to \lvert \Psi(t)\rangle$ is linear.  But before we start deriving, let's find some motivation for those assumptions. 

Consider the following simple interferometer experiment

Time $t$ after a photon is released it can be in state $\lvert a \rangle$ on the top branch, and it can also be in state $\lvert b \rangle$ on the bottom branch.  We can guarantee a photon is in state $\lvert a \rangle$ by blocking the bottom path, and when we do this we find that, at $2t$ it is detected at $\lvert c\rangle$ 50% of the time, and at $\lvert d\rangle$ 50% of the time.  Similarly we can guarantee a photon in state $\lvert b\rangle$ by blocking the top path, and, at $2t$ we get $\lvert c\rangle$ and $\lvert d\rangle$ 50% of the time each. 

So you would assume that if we don't block either path then, at $2t$ we'd also have $\lvert c\rangle$ half the time and $\lvert d\rangle$ the other half.  But that is not what happens.  In fact we never get $\lvert c\rangle$ and we always get $\lvert d\rangle$.

The resolution of this conundrum - and many other experiments like it - is to make two assumptions

1. $\lvert a\rangle$,  $\lvert b\rangle$, $\lvert c\rangle$, and $\lvert d\rangle$ are vectors in some space representing states of the system, which also includes superpositions of these states
2. The map which takes the state $\lvert \Psi(0)\rangle \to \lvert \Psi(t)\rangle$ is a linear map $U(t)$, i.e. $\lvert \Psi(t)\rangle = U(t) \lvert \Psi(0)\rangle$

Additionally - and this is an addendum because it is specific to this experiment - each mirror causes a phase shift changing a state $\lvert \Psi\rangle$ to $i\lvert\Psi\rangle$.  That is interesting because it means we need a complex vector space.  Let's see how these assumptions help us out.

After time $t$ the new state is
$$
\begin{align}
\lvert \Psi(t) \rangle &= U(t) \lvert \Psi(0) \rangle \\
&\propto i^2\lvert a \rangle + \lvert b \rangle \\
&= -\lvert a \rangle + \lvert b \rangle \\
\end{align}
$$

... where the $i^2$ is because there were two reflections needed to get to $\lvert a \rangle$.

At $2t$ the new state is
$$
\begin{align}
\lvert \Psi(2t) \rangle &= U(t) \lvert \Psi(t) \rangle \\
&\propto U(t)(-\lvert a \rangle + \lvert b \rangle) \\
&= -U(t)\lvert a \rangle + U(t) \lvert b \rangle \\
&\propto -(i\lvert c \rangle + \lvert d \rangle) + (i \lvert c \rangle + i^2 \lvert d \rangle)\\
&\propto \lvert d \rangle
\end{align}
$$

The intermediate state at time $t$ proportional to $-\lvert a \rangle + \lvert b \rangle$ is a superposition.  The relevant thing about superpositions is that they combine two states each of which we'd recognise as a possible outcome of a classical measurement - in this case a measurement of the location of the photon.  Each superposition of classical states can, on measurement, "collapse" down to any one of its non-zero summands, but not to any other state.  This means a couple of things:

1. All the classical states ($\lvert a\rangle$, $\lvert b\rangle$ etc.) must be linearly independent
2. We can impose an inner product by taking the classical states as an orthonormal basis 

Notice that the classical states are all normalized by definition.  The final assumption we are going to make, which seems reasonable now, is that $U(t)$ takes normalized states to normalized states.

We now have all the assumptions we need, and everything else will follow from them.  But before we dive into the derivation of Schroedinger and Born let's state them clearly

1. The state of the system is a normalized vector $\lvert \Psi\rangle$ from a complex vector space with an inner product
2. The time evolution mapping is a linear map $\lvert \Psi\rangle \to U(t)\lvert \Psi \rangle$ that takes normalized vectors to normalized vectors.

Let's derive... 

Lemma 1

If a linear operator $A$ maps normalized vectors to normalized vectors then $A$ preserves overlaps, i.e. if
$$
\begin{align*}
\lvert u' \rangle &= A\lvert u \rangle \\
\lvert v' \rangle &= A\lvert v \rangle  \\
\end{align*}
$$
then
$$
\langle u' \lvert v' \rangle = \langle u
\lvert v \rangle
$$

Proof

It is clear from linearity that $A$ preserves the norm of all vectors and not just normalized ones. From this we can deduce the following for any $u, v$
$$
\begin{align*}
(\langle u'\vert + \langle v'\vert) (\vert u'\rangle + \vert v'\rangle) &= (\langle u\vert + \langle v\vert) (\vert u\rangle + \vert v\rangle)\\
\langle u'\vert u' \rangle + \langle v'\vert v' \rangle + \langle u'\vert v' \rangle + \langle v'\vert u' \rangle &= \langle u\vert u \rangle + \langle v\vert v \rangle + \langle u\vert v \rangle + \langle v\vert u \rangle\\
\langle u'\vert v' \rangle + \langle v'\vert u' \rangle &= \langle u\vert v \rangle + \langle v \vert u \rangle\\
Re(\langle u'\vert v' \rangle) &=  Re(\langle u\vert v \rangle)\\
\end{align*}
$$
Replacing the ‘$+$’s with ‘$-$’s on the first line results in $Im(\langle u'\vert v' \rangle) =Im(\langle u\vert v \rangle)$ which completes the proof.

Theorem 1 (Unitarity)

An operator $A$ is unitary if and only if $A$ is linear and maps normalized vectors to normalized vectors

Proof

The “only if” direction is trivial so we’ll focus on the “if” direction. Assume $A$ is a linear operator mapping normalized vectors to normalized vectors. Let $\{\vert e_i \rangle\}$ be any orthonormal basis, then, using lemma 1, we have
$$
\begin{align*}
\langle e_i \vert A^\dagger A \vert e_j \rangle &=\\
\langle e_i' \vert e_j' \rangle &= \delta_{ij}
\end{align*}
$$
which proves $A^\dagger A = I$ i.e. that A is unitary.

NB: this only covers the finite dimensional case - the infinite dimensional case is a bit longer and requires $A$ to be invertible.

Theorem 2 (The Schroedinger Equation)

If the time evolution operator is linear and maps normalized vectors to normalized vectors then for some Hermitian $H$
$$
i \frac{d\vert \Psi \rangle}{dt} = H \vert \Psi \rangle
$$

Proof

Expand the time operator $U(dt)$ to the first order in dt and write it as $I - i H
dt$. We know from theorem 1 this is unitary,
and so
$$
\begin{align*}
I &= U(dt)^\dagger U(dt) \\
&= (I+iH^\dagger dt)(I - iH dt) \\
&= I + i dt (H^\dagger - H) + O(dt^2)
\end{align*}
$$
Matching linear terms gives us that $H^\dagger = H$. This proves that $H$ is Hermitian; the Schroedinger equation then follows trivially from the definition of $H$.

Theorem 3 (Subjective collapse)

A composite system consisting of observer and object subsystems will, following time evolution, be represented as an entangled sum over all possible measurement outcomes, with each term a product of a different object state and an matching observer state.

Proof

Let $U$ represent the unitary time evolution operator covering a period in which an observable is measured. Let $\vert \Psi \rangle$ be the initial state of the observer and
$$
\vert \Phi \rangle = \sum_i \alpha_i \vert \Phi_i \rangle
$$
be the initial state of the object system, where the $\vert \Phi_i \rangle$ are eigenvectors of the observable in question. Initially the observer and object systems are not entangled and so the combined state is
$$
\vert \Psi \rangle \vert \Phi \rangle =
\sum_i \alpha_i \vert \Psi\rangle \vert \Phi_i
\rangle
$$
The state following the observation is, by linearity
$$
\sum_i \alpha_i U (\vert \Psi\rangle \vert \Phi_i \rangle)
$$
We know that an initial state of $\vert \Phi_i \rangle$ guarantees the result of the measurement (from the POV of the observer) to be $i$ and also guarantees that the subsystem state is unchanged. Therefore the above must be equal to
$$
\sum_i \alpha_i \vert \Psi_{,i}\rangle \vert \Phi_i \rangle
$$
where the “$,i$” indicates a state in which result $i$ was observed. This is the entangled sum we were looking for. Note that if the same observable is measured again the resulting state is obtained by another application of $U$. This gives
$$
\sum_i \alpha_i \vert \Psi_{,i,i}\rangle \vert \Phi_i \rangle
$$
This is a sum over states where in each one the observer got the same result twice. So although the observer had no way of knowing which $i$ they would get before the first observation, they can be certain that all subsequent observations will yield the same result. This looks subjectively like the first observation “collapsed” the wavefunction, but in fact we have assumed nothing beyond that the observer and object subsystems combine to form a tensor product which evolves in a way that maps normalized vectors to normalized vectors.

Lemma 2

Let $G : [0,1] \to [0,1]$ satisfy $G(y_1 + y_2) = G(y_1) + G(y_2)$ whenever $y_1, y_2 \ge 0$ and $y_1 + y_2 \le 1$.  Then $G(y) = ky$ with $k = G(1)$

Proof

(i) $G(0) = G(0+0) = 2G(0)$, so $G(0) = 0$.

(ii) By induction, $G(y_1 + \dots + y_n) = \sum_i G(y_i)$ whenever $\sum_i y_i \le 1$.

(iii) Put $k = G(1)$. Taking $y_1 = \dots = y_n = 1/n$ in (ii) gives $n\,G(1/n) = G(1)$, so $G(1/n) = k/n$; and then $G(m/n) = m\,G(1/n) = km/n$ for integers $0 \le m \le n$. Hence $G(y) = ky$ for every rational $y \in [0,1]$.

(iv) $G$ is non-decreasing: if $0 \le y \le y' \le 1$ then $G(y') = G(y) + G(y'-y) \ge G(y)$, since $G$ takes non-negative values.

(v) Let $y \in [0,1]$ be arbitrary and choose rationals $p, q \in [0,1]$ with $p \le y \le q$. By (iii) and (iv), $kp = G(p) \le G(y) \le G(q) = kq$

Letting $p \uparrow y$ and $q \downarrow y$ gives $G(y) = ky$.

Definition (Probability as fair odds)

This definition requires that we accept the concept of an ideal rational person, and imagine them involved in a betting game.  Rational players stake money on mutually exclusive outcomes, with the winner taking the pot; the stakes are fair if no player would prefer to swap places with another. Define the probability of outcome $i$, or $P(i)$, as the fraction of the pot staked on outcome $i$ at fair odds.

Note that this sidesteps the difficulty that in a branching universe every outcome occurs.  Note also that $\sum_i P(i) = 1$ by construction.

Theorem 4 (The Born Rule)

The probability of the observer finding themselves in state $\vert \Psi_{,i} \rangle$ post observation is $\lvert \alpha_i \rvert^2$ if the following (reasonable) assumptions hold

1. Supervenience: $P(|i\rangle)$ depends only on $\alpha_i$, so $P(|i\rangle) = g(\alpha_i)$ for some $g$
2. Branching indifference: if a later measurement splits outcome $i$, a rational player's total stake on $i$ is unchanged.
3. Phase gauge: multiplying the initial state by a phase $e^{i\theta}$ leaves probabilities unchanged

Proof

An overall phase $e^{i\theta}$ is physically inert and sends $\alpha_i \to e^{i\theta}\alpha_i$, so by (3) $g$ depends only on $|\alpha_i|$ and we may write $g(\alpha) = G(|\alpha|^2)$ with $G : [0,1] \to [0,1]$.

Splitting outcome $i$ yields amplitudes with $|\alpha_{i_1}|^2 + |\alpha_{i_2}|^2 = |\alpha_i|^2$, so (2) reads $G(y_1 + y_2) = G(y_1) + G(y_2)$, where $y_k = |\alpha_{i_k}|^2$.  Thus $G$ is additive, and bounded because it takes values in $[0,1]$.  We also know that $G(1) = 1$ because a state
with a single term has the whole pot staked on it.  By Lemma 2 $G$ must be the identity. Hence $P(|i\rangle) = |\alpha_i|^2$.

Conclusion

I find it pretty amazing that we can derive so much from so little. We started off with little more than an assumption about the state of the system belonging to a complex vector space and time evolution being a linear operation, both motivated by experiment.  That led us to show that time evolution is a Unitary operator, that the Schroedinger Equation holds, and that the Hamiltonian is Hermitian. Then by considering a composite subsystem consisting of an observer and an object and treating both parts quantum mechanically we showed that although the system is objectively a superposition of observer/object product states, in each of those the observer has the illusion of having collapsed the wavefunction.  Furthermore, the Born rule turned out to not require an additional postulate, but follow as a consequence of a handful of more fundamental postulates.

References

The derivation of the Schroedinger Equation came from Quantum Mechanics, The Theoretical Minimum, by Leonard Susskind and Art Friedman.

The emergence of collapse as a subjective phenomenon came from Relative State Formulation of Quantum Mechanics, by Hugh Everett III.

The fair odds definition of probability came from Interpretations of Probability, by A. Hájek.

The derivation of the Born Rule combines ideas from Deutsch, de Finetti, Wallace, and Everett.