It is believed that the non-commutation of operators is a characteristic property of quantum mechanics. So much so that axiomatic mathematical structures are developed specifically to represent this non-commuting nature for the purpose of being the ideal formalism in terms of which quantum physics can be modeled.
Is quantum physics the exclusive scenario in which non-commuting operators are found? Is the non-commutative nature of these operators in quantum mechanics a fundamental property of nature?
No, one can also define operators in classical theories and find that they are non-commuting. And, no, this non-commuting property is not fundamental. It is a consequence of more fundamental properties.
To illustrate these statements, I’ll use a well-known classical theory: Fourier optics. It is a linear theory in which the propagation of a beam of light is represented in terms of an angular spectrum of plane waves. The angular spectrum is obtained by computing the two-dimensional Fourier transform of the complex function representing the optical beam profile on some transverse plane.
The general propagation direction of such a beam of light, which is the same thing as the expectation value of its momentum, can be calculated with the aid of the angular spectrum as its first moment. An equivalent first moment of the optical beam profile gives us the expectation value of the beam’s position. Both these calculations can be represented formally as operators. And, these two operators do not commute. Therefore, the non-commutation of operators has nothing to do with quantum mechanics.
So what is going on here? It is an inevitable consequence that two operators associated with quantities which are Fourier conjugate variables would be non-commuting. Therefore, the non-commuting property is an inevitable result of Fourier theory. Quantum mechanics inherits this property because the Planck relationship converts the phase space variables, momentum and position, into Fourier conjugate variables.
So, is Fourier analysis then the fundamental property? Well, no. There is a more fundamental property. The reason why Fourier conjugate variables lead to non-commuting operators is because the bases associated with these conjugate variable are mutually unbiased.
We can again think of Fourier optics to understand this. The basis of the angular spectrum consists of the plane waves. The basis of the beam profile are the points on the transverse plane. Since plane waves have the same amplitude at all points in space, the overlap of a plane wave with any point on the transverse plane gives a result with the same magnitude. Hence, these two bases are mutually unbiased.
Although Fourier theory always leads to such mutually unbiased bases, not all mutually unbiased bases are produced by a Fourier relationship. Another example is found with Lie algebras. For example, consider the Lie algebra associated with three dimensional rotations. This algebra consists of three matrices called the Pauli matrices. We can determine the eigenbases of the three Pauli matrices and we’ll see that they are mutually unbiased. These three matrices do not commute. So, we can make a general statement.
Two operators are maximally non-commuting if and only if their eigenbases are mutually unbiased
The reason for the term “maximally” is to take care of those cases where some degree of non-commutation is seen even when the bases are not completely unbiased.
Although the Pauli matrices are ubiquitous in quantum theory, they are not only found in quantum physics. Since they represent three-dimensional rotations they are also found in purely classical scenarios. Therefore, their non-commutation has nothing to do with quantum physics per se. Of course, as we already showed, the same is true for Fourier analysis.
So, if we are looking for some fundamental principles that would describe quantum physics exclusively, then non-commutation would be a bad choice. The hype about non-commutation in quantum physics is misleading.