The prevailing “crisis in physics” has lead some people to suggest that physicists should only follow a specific path in their research. It creates the impression that one person is trying to tell the entire physics community what they are allowed to do and what not. Speculative ideas are not to be encouraged. The entire physics research methodology need to be reviewed.
Unfortunately, it does not work like that. One of the key underlying principles of the scientific method is the freedom that all people involved in it have to do whatever they like. It is the agreement between these ideas and what nature says that determines which ideas work and which do not. How one comes up with the ideas should not be restricted in any way.
This freedom is important, because nature is resourceful. From the history of science we learn that the ways people got those ideas that turned out to be right differ in all sorts of ways. If one starts to restrict the way these ideas are generated, one may end up empty handed.
Due to this diversity in the ways nature works, we need a diversity in perspectives to find the solutions. It is like a search algorithm in a vast energy landscape. One needs numerous diverse starting points to have any hope to find the global minimum.
Having said that, one does find that there are some guiding principles that have proven useful in selecting among various ideas. One is Occam’s razor. It suggests that one starts with the simplest explanation first. Nature seems to be minimalist. If we are trying to find an underlying system to explain a certain phenomenology, then the underlying system needs to be rich enough to be able to produce the level of complexity that one observes in the phenomenology. However, it should not be too rich, leading to too much complexity. As an example, conjuring up extra dimensions to explain what we see, we produce too much complexity. Therefore, chances are that we don’t need this.
Another principle, which is perhaps less well-known is the minimum disturbance principle. It suggests that when we find that something is wrong with our current understanding, it does not make sense to through everything away and build up the whole understanding from scratch. Just fix that which is wrong.
Now, there are examples in the history of science where the entire edifice of existing theory in a particular field is changed to solve a problem. However, this only happens when the observations that contradict the current theory start to accumulate. In other words, when there is a crisis.
Do we have such a kind of crisis at the moment? I don’t think so. The problem is not that the existing standard model of particle physics have all these predictions that contradict observations. The problem is precisely the opposite. It is very good at making predictions that agree with what we can observe. We don’t seem to see anything that can tell us what to do next. So, the effort to see what we can improve may well be beyond our capability.
The current crisis in physics may be because we are nearing the end of observable advances in our fundamental understanding. We may come up with new ideas, but we may be unable to get any more hints from experimental observation. In the end we not even be able to test these new ideas. This problem starts to enter the domain of what we see as the scientific method. Can we compromise it?
That is a topic for another day.