Argumentation about de Broglie-Bohm pilot wave theory

Guest blog by Ilja Schmelzer, a right-wing anarchist and independent scientist

A nice summary of standard arguments against de Broglie-Bohm theory can be found at R. F. Streater's "Lost Causes in Theoretical Physics" website. Ulrich Mohrhoff [broken link, sorry] also combines the presentation of his position with an interesting rejection of pilot wave theory. These arguments I consider in a different file. Here, I consider the arguments proposed in several articles of Luboš Motl's blog "The reference frame": David Bohm born 90 years ago and Bohmists & segregation of primitive and contextual observables, Anti-quantum zeal and in off-topic responses of "Nonsense of the day: click the ball to change its color". Below, we refer to Luboš Motl simply as lumo (his nick in his blog).




Another argument (also with lumo's participation), related to Lorentz-invariance, I have considered at another place.

If you know other interesting pages critical of de Broglie - Bohm pilot wave theory, Nelsonian stochastics, non-local hidden variable theories in general, as well as ether theories, please tell me about them.




The most important thing: Measurement theory

The most important part of physics are, of course, experiments. Moreover, this is also the point where lumo is simply wrong, so it is worth to start with it.:

... it is not true that the de Broglie-Bohm theory gives the same predictions in general. It can be arranged to do so in the case of one spinless particle. But in the real quantum theories we find relevant today, such as quantum field theory, de Broglie-Bohm theory cannot be constructed to match probabilistic QFT exactly, and one can see that its very framework contradicts observable facts.

At another place, we find some hint where his misunderstanding is located:

Your equations about \(X\) are completely irrelevant for the measurement of the spin. The problem is not when one wants to measure \(X\). Indeed, the measurement of \(X\) might occur analogously to its measurement in the spinless case. The problem occurs when one actually wants to measure the spin itself.

The projection of the spin \(j_z\) is an observable that can have two values, in the spin \(1/2\) case, either \(+1/2\) or \(-1/2\). It is a basic and completely well-established feature of QM that one of these values must be measured if we measure it.

How is your 17th century deterministic theory supposed to predict this discrete value? Like with \(X\), it must already have a classical value for this quantity. Except that in this case, it has to be discrete, so it can't be described by any continuous equation. ...

Preemptively: you might also argue that any actual measurement of the spin reduces to a measurement of \(X\). But it's not true. I can design gadgets that either absorb or not absorb the electron depending on its \(j_z\). So they measure \(j_z\) directly. deBB theories of all kinds will inevitably fail, not being able to predict that with some probability, the electron is absorbed, and with others, they're not. This has nothing to do with \(X\) or some driving ways. It is about the probability of having the spin itself.

The last paragraph gives the hint: lumo has interpreted the claim that all measurements reduce to position measurements as "all measurements of the electron reduce to position measurements of the electron". If that would be true, I would concede that lumo's polemics against pilot wave theorists are justified. This was, by the way, the state of the art before Bohm's measurement theory appeared 1952. Thus, lumo's arguments illustrate in a nice way why de Broglie had given up pilot wave theory.

Once the question has been asked how the 17th century deterministic theory manages to predict discrete values, let's explain this story. As a 17th century theory, with real aristocratic origin, it leaves the hard work to servants (quantum operators), reserving for itself the final (and most important) decisions ;-).

First, there is some interaction of the wave function of the electron with the wave function of the measurement device. (There is of course also an equation for the position of the electron \(q_{el}\) – the \(X\) in lumo's text – but it is completely irrelevant, not only at this stage, but in the whole process.) The result of the measurement is, as usual, a wave function of type\[

|\psi\rangle = \alpha_1|{\rm up}\rangle|q_1\rangle + \alpha_2|{\rm down}\rangle|q_2\rangle

\] This exploitation of standard QT is not enough – now decoherence will be exploited in an equally shameless way. We leave it to decoherence considerations to decide which observables of the measurement device become amplified or macroscopic. Assume the quantum states \(|q_1\rangle, |q_2\rangle\) are decoherence-preferred. In this case, decoherence amplifies the microscopic measurement results \(|q_1\rangle, |q_2\rangle\) into classical, macroscopically different states \(|c_1\rangle, |c_2\rangle\). After finishing this hard job, it presents the following state:\[

|\psi\rangle = \alpha_1|{\rm up}\rangle|c_1\rangle + \alpha_2|{\rm down}\rangle|c_2\rangle

\] Now, everything is prepared, it remains to make the really important decision which of the wave packets is the best one ;-). At this moment a hidden variable enters the scene. But, surprise, it is not the hidden variable of the electron \(q_{el}\) (lumo's X), but that of the classical measurement device \(q_c\).

The job of \(q_c\) is not a really hard one. After driving around (no, being driven around by quantum guides) in an almost unpredictable way, it simply takes the wave packet prepared for him by the quantum operators at the point of arrival ;-). In other words, we simply have to put the actual value of \(q_c(t)\) into the full wave function \(|\Psi\rangle\) to obtain the (unnormalized) effective wave function:\[

\psi(q_e) = \Psi(q_e, q_c(t))

\]What we need for this scheme to work as an ideal quantum measurement is not much. We need that the two states of the macroscopic device \(|c_1\rangle, |c_2\rangle\) do not (significantly) overlap as functions of the hidden variable \(q_c\). In this case, whatever the value of \(q_c\), the result \(\psi(q_e)\) will be a unique choice between two effective wave functions, namely between \(|{\rm up}\rangle\) if \(q_c\) is in the support of \(|c_1\rangle\), and \(|{\rm down}\rangle\) otherwise. And we need the quantum equilibrium assumption for \(q_c\) to obtain the probabilities for these two choices as \(|\alpha_1|^2\) resp. \(|\alpha_2|^2\).

Thus, everything works as in quantum theory – Born rule as well as state preparation by measurement (only without any ill-defined wave function collapse or subdivision of the world into a classical and quantum part, or the equally ill-defined "subdivision of the world into systems" used in many worlds or other decoherence-based approaches).

But maybe one of the two assumptions we have used used are wrong? Given Valentini's subquantum H-theorem, together with the numerical results of Valentini and Westman, which show a remarkable relaxation to equilibrium already in the two-dimensional case in a quite short period of time (arXiv:quant-ph/0403034), there is not much hope for observations of non-equilibrium in our universe.

One can, of course, also doubt that macroscopically different states do not have a significant overlap in the hidden variables. Such doubts have been, for example, expressed by Wallace and Struyve for pilot wave field theories. See my paper "Overlaps in pilot wave field theories" at arXiv:0904.0764 about the solution of this problem.

About the zeros of the wave function

There is a second point where experiment is involved, with an easy solution:

How do we know that \(m=l_z/\hbar\) must be an integer? Well, it is because the wave function \(\psi(x,y,z)\) of the m-eigenstates depends on \(\phi\), the longitude (one of the spherical or axial coordinates), via the factor \(\exp(i\cdot m\cdot\phi)\) which must be single-valued. Only in terms of the whole \(\psi\), we have an argument.

However, when you rewrite the complex function \(\psi(r,\theta,\phi)\) in the polar form, as \(R\exp(iS)\), the condition for the single-valuedness of \(\psi\) becomes another condition for the single-valuedness of S up to integer multiples of \(2\pi\). If you write the exponential as \(\exp(iS/\hbar)\), the "action" called S here must be well-defined everywhere up to jumps that are multiples of \(h = 2\pi\hbar\).

That's a nice argument, and, because of this argument, today the original form of de Broglie's "pilot wave theory" is preferred in comparison with the "Bohmian mechanics" version proposed 1952 by Bohm. In pilot wave theory, the pilot wave is really a wave, and you can apply the original argument to show that these observables are quantized. In Bohm's second order version, this is different, and the quantization of certain observables becomes, indeed, problematic. This has been another reason for me (beyond history, see arXiv:quant-ph/0609184) to prefer the name "pilot wave theory" in comparison with "Bohmian mechanics".

More generally, something very singular seems to be happening near the \(R=0\) strings in the Bohmian model of space.

The "model of space" in pilot wave theory is a trivial one, nothing strange happens there if R = 0. The singularity of the velocity at these points is harmless – a simple rotor localized in a string, moreover, there is nothing in the place where velocity becomes undefined.

So even though the Bohmian mechanics stole the Schrödinger equation from quantum mechanics, the superficially innocent step of rewriting it in the polar form was enough to destroy a key consequence of quantum mechanics - the discreteness of many physical observables.

If there would be property rights for equations or functions, one could argue as well that Schrödinger has stolen the wave function from de Broglie's pilot wave theory. Fortunately, such nonsense does not exist in science. But there is a point worth to be mentioned: Without pilot wave theory, there would be no Schrödinger picture, and we would have to use the Heisenberg formalism all the time. And if some Bohm would have found the Schrödinger equation later, it would have been named, as well, an unnecessary superconstruction and banned from physics, for almost the same reasons.

About relativistic symmetry and the preferred frame

Last but not least, there are some claims that pilot wave theories will be unable to recover QFT predictions in the relativistic domain. Unfortunately for his argumentation, the equivalence theorem remains to be a theorem even in the relativistic domain – nothing used in it has any connection to the particular choice of spacetime symmetry. Thus, if the quantum theory has relativistic symmetry for it's observable predictions, the same holds for the observable predictions of pilot wave theory.

More concretely, it is inconsistent with modern physics in many ways, as we will see.

Special relativity combined with the entanglement experiments is the most obvious example. Bell's theorem proves that if a similar deterministic theory reproduces the high correlations observed in Nature (and predicted by conventional quantum mechanics), namely the correlations that violate the so-called Bell's inequalities, the objects in the theory must actually send physical superluminal signals.

But superluminal signals would look like signals sent backward in time in other inertial frames. It follows that at most one reference frame is able to give us a causal description of reality where causes precede their effects. At the fundamental level, basic rules of special relativity are inevitably violated with such a preferred inertial frame.

I was already afraid that lumo does not even understand that in a preferred frame everything is fine with causality. The introduction was, at least, the highly dramatic one which is typical for such crank cases.

I like the formulation "at most". Sounds as if we would really like to have more reference frames and are, now, very disturbed that at most one preferred frame is available ;-).

You might think that the experiments that have been made to check relativity simply rule out a fundamentally privileged reference frame. Well, the Bohmists still try to wave their hands and argue that they can avoid the contradictions with the verified consequences of relativity.

Who is hand waving here? Lumo might, of course, think that experiments rule out a hidden preferred frame. But it's his job, in this case, to point out which observations rule out such a preferred frame. As long as he fails to do it, I don't even have contradictions with any verified consequence of relativity to wave my hands.

I wonder whether they actually believe that there always exists a preferred reference frame, at least in principle, because such a belief sounds crazy to me (what is the hypothetical preferred slicing near a black hole, for example?).

I'm happy to answer this question: The preferred coordinates are harmonic. Given, additionally, the global CMBR frame, with time after big bang as the time coordinate, this prescription is already unique. For a corresponding theory of gravity, mathematically almost exactly GR on flat background in harmonic gauge, physically with preferred frame and ether interpretation, see my generalization of the Lorentz ether to gravity.

But it is possible to see that one can't get relativistic predictions of a Bohmian framework for all statistically measurable quantities at the same moment, not even in principle. If a theory violates the invariance under boosts "in principle", it is always possible to "amplify" the violation and see it macroscopically, in a statistically significant ensemble. If such a violation existed, we would have already seen it: almost certainly.

I would be interested to learn more about this mystical way to amplify high energy violations of Lorentz symmetry into the low energy domain, without access to the necessary high energies. As far, it is lumo who is waving his hands.

I know that there are some nice observations, which use the extremely large distances light has to travel for some astronomical observations, to obtain boundaries for a frequency dependence of the velocity of light. Some of the boundaries obtained in this and different ways suggest even that these Lorentz-violating effects are absent for distances below Planck length. But Planck length is merely the distance where quantum gravity becomes important. The fundamental distance where our continuous field theories start to fail may be different.

In proper quantum mechanics, locality holds. If one considers a Hamiltonian that respects the Lorentz symmetry - such as a Hamiltonian of a relativistic quantum field theory - the Lorentz symmetry is simply exact and it guarantees that signals never propagate faster than light.

In proper quantum mechanics, one can define the operators that generate the Poincaré group and rigorously derive their expected commutators. Also, it is exactly true that operators in space-like-separated regions exactly commute with each other. This fact is sufficient to show that the outcome of a measurement in spacetime point B is never correlated with a decision made at a space-like-separated spacetime point A.

These facts allow us to say that quantum field theory respects relativity and locality. The actual measurements can never reveal a correlation that would contradict these principles. And it is the actual measurements that decide whether a statement in physics is true or not. Bohmian mechanics is different because these principles are directly violated. You may try to construct your mechanistic model in such a way that it will approximately look like a local relativistic theory but it won't be one. Consequently, you won't be able to use these principles to constrain the possible form of your theory. Moreover, tension with tests of Lorentz invariance may arise at some moment.

First, there is no reason not to use some symmetry principles for one part of the theory which do not hold for another part of it. For example, the symplectic structure in the classical Hamilton formalism has another symmetry group – the group of all canonical transformations – than the whole theory including the Hamiltonian.

Then, to postulate a fundamental Poincare symmetry is, of course, a technically easy way if one wants to obtain a theory with Poincare symmetry. But what is the purpose of a postulated global Poincare symmetry in a situation where the observable symmetry is different, depends on the physics, as in general relativity? Whatever the representation of the \(g_{\mu\nu}(x)\) on the Minkowski background – it will (except for simple conformally trivial cases) have a different light cone almost everywhere. If the Minkowski background lightcone is the smaller one, one has somewhere to violate the background Poincare symmetry. It may be always the other way. But in this case, the axioms of the theory give only restrictions for the background Minkowski light cone, not for the physical light cone. Thus, tensions with the physical Lorentz invariance may arise in the same way, because the theory only looks like one which, in the particular point \(x\), has the Lorentz invariance for the metric \(g_{\mu\nu}(x)\). But really it is a theory with Lorentz invariance for a different metric \(\eta_{\mu\nu}\), with a larger light cone, thus, allows for superluminal information transfer relative to \(g_{\mu\nu}(x)\).

String theory, as far as I understand, obtains gravity as a spin two field on Minkowski background. This requires, as far as I understand, that this problem is solved in string theory. Fine. Means, it is a solvable one.

The contradiction between relativity and semi-viable Bohmian models (that violate Bell's inequalities, and they have to in order not to be ruled out by experiments) is a very profound problem of these models. It can't really be fixed.

Again, nice formulation. Sounds like poor Bohmians have tried hard not to violate Bell's inequalities and finally given up. "Semi-viable" is also a nice word. But the "very profound problem" remains hidden. (A nice place for problems in a hidden variable theory.;-))

Instead, I prefer to follow the weak suggestions one can obtain based on mathematical equivalence proofs. When I construct a pilot wave theory based on a relativistic QFT, it seems really hard to avoid the consequences of this theorem to violate Lorentz invariance. At least, I don't know how to manage this. We obtain a pilot wave theory which does not violate observable relativistic symmetries. Simply because there is an equivalence proof for observables.

Today, we have some more concrete reasons to know that the hidden-variable theories are misguided. Via Bell's theorem, hidden-variable theories would have to be dramatically non-local and the apparent occurrence of nearly exact locality and Lorentz invariance in the world we observe would have to be explained as an infinite collection of shocking coincidences.

I'm impressed by the verbal power of "dramatically nonlocal", even more by the "infinite collection of shocking coincidences". Sounds really impressive. But I would not name a nonlocality, which, because of an equivalence theorem, cannot be used even for information transfer, and can be observed only indirectly, via violations of Bell's inequality, a dramatical one. Instead, it seems to me the most non-dramatical one. As well, I would distinguish the simple and straightforward consequences of an equivalence theorem from an "infinite collection of shocking coincidences". Instead, I would be more surprised if an quantum equilibrium large distance low energy limit would not change anything in the symmetry group of a theory.

Last but not least, the Lorentz group is simply the invariance group of a quite prosaic wave equation, an equation we find almost everywhere in nature. And such, a wave equation (or it's linearization) usually defines also an effective (and in general curved) Lorentz metric, so that the wave equation becomes the harmonic equation of this Lorentz metric. As a consequence, for everything which follows such a wave equation we obtain local Lorentz symmetry. (See arXiv:0711.4416, arXiv:gr-qc/0505065 for overviews.)
To assume that a symmetry, which so often and for very different materials appears as an effective symmetry in condensed matter theory, is fundamental, is a hypothesis which seems quite unnatural for me.

... and the ether ...

The similarity with the luminiferous aether seems manifest. ...

I just don't think that this is a rationally sustainable belief. It's just another repetition of the old story of the luminiferous aether.

About the similarity with the aether I fully agree with lumo ;-)))). But what is irrational in the belief that there is an ether? I would like to hear some details. I would be really interesting to hear which of the beliefs expressed in my ether model for particle physics are not rationally sustainable.

Now, it seems we have finished the claims of empirical inadequacy. It's time to consider the metaphysical arguments.

About signs of the heavens

It is not surprising in any way that the new, Bohmian equation for \(X(t)\) can be written down: it is clearly always possible to rewrite the Schrödinger equation as one real equation for the squared absolute value (probability density) and one for the phase (resembling the classical Hamilton-Jacobi equation). And it is always possible to interpret the first equation as a Liouville equation and derive the equation for \(X(t)\) that it would follow from. There's no "sign of the heavens" here.

I think there are "signs of the heavens" here. First, the guiding equation for the velocity is a nice, simple, and local (in configuration space) equation. The derivation mentioned by lumo could as well lead to a dirty nonlocal one.

Then, the equation for the phase resembles the classical Hamilton-Jacobi equation, and for constant density becomes simply identical with it. Now, the same guiding equation is, as well, part of the classical Hamilton-Jacobi theory – a theory which was in no way related to the conservation law of the first derivation.

Now, Hamilton-Jacobi theory is really beautiful mathematics, it has all properties of "signs of the heavens", even if taken only alone. See arXiv:quant-ph/0210140 for an introduction. That one and the same simple law for velocity gives, on one hand, Hamilton-Jacobi theory in the classical limit, and, on the other hand, a Liouville equation, is, at least for me, a sufficiently strong hint from the mathematical heaven. In many worlds I have not seen any comparable signs of beauty.

And there is, of course, the really beautiful derivation of the whole quantum measurement formalism.

How to distinguish useful improvements from unnecessary superconstructions

The mechanistic models add a new layer of quantities, concepts, and assumptions.

Indeed, every new, more fundamental theory adds a new layer of quantities, concepts, and assumptions. So what?

[Einstein] called the picture an unnecessary superconstruction.

Appeal to authority does not count. And there is no reason to expect that the father of relativity would like a theory which violates his child. But how to distinguish unnecessary superconstructions from interesting more fundamental theories? Above add something to the old theory. But useful more fundamental theories allow to explain something else from the old theory: Some postulates of the old theory can be derived now. So, one has to compare what one has to add with what can be derived now.

This relation is quite nice for pilot wave theory: The new layer is, essentially, the configuration together with a single additional equation – the guiding equation for the configuration. What can be derived from this equation is, instead, the whole measurement theory of quantum mechanics, including the Born rule and the state preparation by measurement. Compared with the Copenhagen interpretation, the additional layer also replaces the "classical part" of this interpretation and removes the collapse from the theory.

These last two points have been a major motivation of other reinterpretations as well. In particular, for many worlds it seems to be the only aim. The interpretation I prefer to name "inconsistent histories" is focussed on this aim too. Thus, two things which have been obtained in pilot wave theory first, have been widely recognized today as important contributions to the foundations of quantum theory. One can object that pilot wave theory does not get rid of the classical part, but even extends it into the quantum domain. This depends on what one considers as problematic with the classical part: If the problem is the imprecision of this notion, the absence of well-defined rules for this part, then it is clearly solved in pilot wave theory. Anyway, pilot wave theory was the first interpretation with completely unitary dynamics for the wave function, without a collapse.

One can perhaps create classical mechanistic models that mimic the internal workings of quantum mechanics in many situations. For example, one can write a computer simulation. But you can't say that the details of such a program or Bohmian picture is justified as soon as you confirm the predictions of conventional quantum mechanics.

There is no necessity to justify every detail. The important point of the pilot wave interpretation is that to explain the observable facts there is no necessity to reject classical logic, realism, or to introduce many worlds, inconsistent histories, correlations without correlata or other quantum strangeness and mysticism. We have at least one simple, realistic, even deterministic, explanation of all observable facts. That's enough to reject quantum mystery. Why should we justify every detail of some particular realistic model? There may be several realistic models compatible with observation. I would expect this anyway, given large distance universality.

The mechanistic models add a new layer of quantities, concepts, and assumptions. They are not unique and they are not inevitable. The similarity with the luminiferous aether seems manifest. If they only reproduce the statistical predictions of quantum mechanics, you could never know which mechanistic model is the right one: it could be a computer simulation written by Oracle for Windows Vista, after all.

But what's the problem with this? Is Nature obliged to work with theories which can be inevitably reconstructed by internal creatures? You could never know? Big problem. Anyway, our theories are only guesses about Nature, and we can never know if they are really true. If you doubt, I recommend to read Popper. (I ignore here, for simplicity, the modern ways to recognize the truth of theories, like counting the number of papers written about them, or getting inspirations about the language in which God wrote the world.)

Moreover, science has developed lot's of criteria which allow to compare theories which do not make different predictions: Internal consistency, simplicity, explanatory power, symmetry, mathematical beauty. Lumo uses such arguments himself, thus, he is aware of their power. They are usually sufficient to rule out most of the competing models. And if there remain a few different theories, all in agreement with observation, this is not problematic at all – it is even useful: It allows to see the difference between the empirically established parts of these theories – these parts will be shared by all viable theories – and the remaining, metaphysical parts, which may be very different in the different theories. Thus, they serve as a useful tool to show the boundaries of what science can tell at a given moment.

For example, today the existence of pilot wave theory shows that almost all of the quantum strangeness, in particular the rejection of realism, "quantum logic", and the esoterics of many worlds, are in no way forced on us by any empirical evidence, but purely metaphysical choices of some particular interpretations.

What are the fundamental beables?

I could make things even harder for the Bohmian framework by looking into quantum field theory. What are the real, "primitive" properties in that case?

In the simplest case of a scalar field, the natural candidate for the "primitive property" or the "beable" is simply the field \(\phi(x)\). This is a very old idea, proposed already by Bohm. But the effective fields of the standard model are also bad candidates for really fundamental beables. They are, last but not least, only effective fields, not fundamental fields. In my opinion, one needs a more fundamental theory to find the true beables.

My proposal for such more fundamental beables can be found in my paper about the cell lattice model arXiv:0908.0591. Even if pilot wave theory is not mentioned at all in this paper, it is quite obvious that the canonical quantization proposal for fermion fields I have made there allows to apply the standard formalism of pilot wave theory to obtain a pilot wave version of this theory.

Problems with spin and with particle ontology in quantum field theories

A large part of lumo's arguments is directed against two particular versions of pilot wave theory – strangely, I don't like them too. The first one is the idea to describe particles with spin using only wave functions of particles with spin, but leaving the configuration without spin. In this case, the wave function is no longer a complex function on configuration space, but a function with values in some higher-dimensional Hilbert space. But, as a consequence, the very nice pilot wave way to obtain the classical limit via the Hamilton-Jacobi theory no longer works, and one would have to use the dirty old way based on wave packets to obtain some classical limit.

There are other examples of such pilot wave theories. First, this trick was used by Bell, who has proposed a pilot-wave-like field theory with beables for fermions, but not for bosons. Now, one can argue that this is already sufficient, and leave the bosons without beables. The reverse situation was a theory from Struyve and Westman for the electromagnetic field. Again, it has been argued that this is sufficient. And, for the purpose to obtain a realistic theory which is able to recover QFT predictions, it is. But I think that such pilot wave theories are sufficient only for one purpose: To be used as a quick and dirty existence proof for realistic theories in situations where some parts of the theory cause problems. For this purpose, they are indeed sufficient, if the part of the theory represented in the beables is large enough to distinguish all macroscopic states – a quite weak requirement. If one doubts that a theory without fermions, or without bosons, is sufficient for this, one should think about renormalization: If we use these incomplete theories to describe one type of the bare fields (for some energy), then all types of the dressed fields already depend on this single type.

The second type of theories I don't want to defend are theories with particle ontology in the domain of field theory. One reason is that semiclassical gravity shows nicely that fields are more fundamental, and the pilot wave beables have to be, of course, fundamental. Then, to handle variable particle numbers is a dirty job. There should be something more beautiful. Particles which pretend for a status of beables should be at least conserved.

Therefore, the parts of the argumentation where lumo attacks particle theories I can leave unanswered. Let's note only that a short look at the particle-based approach to field theory in arXiv:quant-ph/0303156 suggests that lumo's arguments don't hit this target as well. This version introduces stochastic jumps into the theory (showing, by the way, that pilot wave theorists are not preoccupied with determinism). But I can leave the comparison to the reader.

About the "segregation" among observables

Because experiments eventually measure some well-defined quantities, the likes of Bohm think that there must exist preferred observables - and operators - that also exist classically. They are classical to start with, they think. Positions of objects are an important example.

But the quantum mechanical founding fathers have known from the very beginning that this was a misconception. All Hermitean operators acting on a Hilbert space may be identified with some real classical observables and none of them is preferred.

I think it is a misconception to interpret pilot wave theory as preferring some observables. It is not an accident that Bell has even proposed another word, beables, for the configuration space variables in pilot wave theory. In particular, measurements of the beables play no special role at all, nor in the classical limit, nor everywhere else in pilot wave theory. To derive the measurement theory, we don't need them (this would be circular anyway). What we need are the actual values of the beables, not some results of observations. Indeed, let's assume for simplicity we consist of atoms, which are the beables of some simplified pilot wave theory. Then, a theory about our observations does not need anything about our observations of atoms – if we "observe" them at all, then only in a quite indirect way, and most people do not observe atoms at all. Therefore, observations of atoms cannot play any role in an explanation of our everyday observations. Of course, in these explanations atoms have to play a role, at least indirectly – as constituent parts of our brain cells. But these atoms inside our brain cells are nothing we observe, if we observe something in everyday life. Thus, we use only the atoms themself, not the observations of atoms, in such explanations of our observations.

Thus, as observables the beables play no special role – in particular, the theory of their measurements can be derived in the same way, without danger of circularity. In particular, their measurements have to be described by self-adjoint operators or POVMs as those of every other observable too. In this sense, there are no preferred observables in pilot wave theory.

And this construction is actually very unnatural because it picks \(X\) as a preferred observable in whose basis the wave vector should be (artificially) separated into the probability densities and phases

Configurations (I prefer "q" instead of "X", because "X" is associated with usual space, while "q" is associated with configuration space) play indeed a special role. But this is the same special role they play in the Lagrange formalism as well as in Hamilton-Jacobi theory. Above are very beautiful, useful approaches. I don't remember to have heard any objections that the Lagrange formalism is unnatural, because it picks "q" as a preferred observable. Instead, the Lagrange formalism is an extremely important tool in modern physics, in quantum field theory as well as in general relativity. Moreover, this "segregation" is a very natural one: If nothing changes, the configuration remains the same, while the velocities have to be zero. Instead, I have found the symmetry between such different things as position and momentum in the Hamilton equations (and, similarly, in the canonical approach to quantum theory) always strange and unnatural, (even if, because of its symmetry, beautiful).

So why lumo does not fight against segregation in the Lagrange formalism? The segregation is the same, the poor momentum variables are degraded to the role of "derivatives". (Or maybe he does? I have not checked. Anyway, the important role of the Lagrange formalism in modern science, which is based on exactly the same "segregation", is a fact which shows that there is nothing wrong with this particular segregation.)

In order to celebrate the Martin Luther King Jr Day, I will dedicate the rest of the text to a fight against the segregation of observables. :-) So my statement is very modest – that observables can't be segregated into the "real" primitive ones and the "fictitious" contextual ones – a fact that trivially rules out all theories (such as the Bohmian ones) that are forced to do so.

... I guess that you must agree that the "philosophical democracy" between all observables is pleasing and natural.

I see no reason at all to find such a "democracy" pleasing. You can observe a honest guy telling us the truth. As well you can observe how a liar is telling us lies. Above are observable. There may be even more symmetry between them. They may even make the same claims: "I have seen that he has stolen the money". That means, without segregation among observables, without destroying observable symmetry, we have to give them equal status. I don't plan to follow this idea, and will always prefer a segregation between truth and lies, even if this destroys some observable symmetries.

The segregation between contextual and non-contextual observables is less important, but is part of our everyday life as well. You can ask somebody about things he has not decided yet. He will think about them, possibly argue with you, and, maybe, give you an answer. This answer does not exist before you have started to argue with him, it is, therefore, contextual. Arguing with somebody else, he could have made a different decision. (Last but not least, this is one purpose of communication – to modify our decisions, if we hear good arguments to do this.) In other words, this answer will be contextual. But in a different situation, he has already decided about this question, and the answer was already part of reality of his thoughts when you have asked him. In this case, the answer is not contextual. Above answers we observe as results of complex verbal interactions, and they are, in this sense, on equal foot. Nonetheless, a realistic theory about his thoughts has to segregate between them. Without segregation, he should be or almighty, able to think and decide about all imaginable questions before you ask him, or completely dependent, deciding about nothing before you ask him.

In all these cases, the same "formalism" is used to obtain the results – communication in human language. Thus, that the same formalism – that of self-adjoint operators, or, more general, of POVM's – is used to describe the results of interactions in quantum theory is in no way an argument against this particular segregation.

Clearly, some quantities in the real world look more classical than others. But what are the rules of the game that separates them? The Bohmists assume that everything that "smells" like \(X\) or \(P\) is classical while other things are not. ...

Clearly, they want some quantities that often behave classically in classical limits.

Clearly not. The "segregation" in pilot wave theory is between configuration and momentum variables, and it is in no way related with one of them being "more classical". In classical situations, above behave classically, and the same segregation exists in classical theory too, in the Lagrange formalism as well as in Hamilton-Jacobi theory. There is no place in pilot wave theory where one has to care that something in the behaviour of the configuration is "classical": In the classical limit, it follows automatically, from the classical Hamilton-Jacobi equation, that everything behaves classically. For other questions this is simply irrelevant.

It is the many worlds community which is focussed around the classical limit. That's reasonable – they have a very hard job to construct something which at least sounds plausible (at least if one uses words like "contains" for a linear relation between some points in a Hilbert space, talks about "evolution" of branches without defining any evolution law, and applies decoherence techniques without explaining how to obtain the decomposition into systems one needs to apply them).

In order to simplify their imagination, the Bohmists imagined the existence of additional classical objects – the classical positions.

Simplification has, it seems, been removed from the aims of science. Ockham's razor is out, simple theories have to be rejected. The higher the dimension, the better.

But the objects are in no way additional. They have been part of the Copenhagen interpretation: Its classical part contains, in particular, all the measurement results. And Schrödinger's cat proves that a unitary wave function alone is not sufficient, that we need something else. Or some non-unitary collapse, or some particular configuration as in pilot wave theory. Something – be it the collapsed wave function, or some different entity – has to describe the reality we see: or the dead, or the living cat. Many worlds claims something different, but introduces, for this purpose, the "branches" – some sort of collapsed wave functions without collapse, or configurations without a guiding equation, which is claimed to be "contained" in the wave function. (How a decomposition of some vector into a linear combination of others defines a containment relation remains unclear. A concept where a function like \(\psi(q) = 42\) "contains" all possible universes has it's appropriate place in the Hitchhiker's Guide to the Galaxy, not in scientific journals.) The approach named "consistent histories" leaves us with many inconsistent histories, subdivided into families.

Theories with physical collapse need dirty and artificial non-unitary modifications of the Schrödinger equation. The branches of many worlds are, it seems, left today without any equations at all. (A very scientific approach, indeed. Time to rename it into "many words"). Only pilot wave theory gives us a nice, simple, and beautiful equation for this "additional" entity. Moreover, it allows, just for nothing, to derive the whole measurement formalism of quantum theory.

Imagination is completely irrelevant for these questions. I see, of course, no reason to object if a theory allows to simplify our imaginations too. Instead, I would count it as one additional advantage of a theory. But I recognize that this attitude is not shared by other scientists. And there are, indeed, good reasons to prefer theories which are complex and mystical. Imagine you are in a company of nice girls (or boys, whatever you prefer), and they ask you what you are doing. Isn't it much more impressive if you can tell them about curved spacetimes, large dimensions, a strange new quantum realism, or even quantum logic, many worlds and other strange quantum things? Compare this with the poor 17th century scientist, the fighter against any form of mystery, the classical loser in every popular mystery film. The choice is quite obvious.

About history

Louis de Broglie wrote these equations for the position of one particle, David Bohm generalized them to N particles.

Not correct, the configuration space version of pilot wave theory was presented by de Broglie already at the Solvay conference. See de Broglie, L., in “Electrons et Photons: Rapports et Discussions du Cinquieme Conseil de Physique”, ed. J. Bordet, Gauthier-Villars, Paris, 105 (1928), English translation: G. Bacciagaluppi and A. Valentini, Quantum Theory at the Crossroads: Reconsidering the 1927 Solvay Conference”, Cambridge University Press, and arXiv:quant-ph/0609184 (2006)

I think that in analogous cases, we wouldn't be using the name of the "updater" for the final discovery.

After having read something about the history of this theory (I do not care that much about history), I use "pilot wave theory" instead of "Bohmian mechanics". But Bohm has a point too: de Broglie has broken his theory as not viable, being unable to develop the general measurement theory. This has been done by Bohm. Therefore, if I use names, I use now the combination "de Broglie-Bohm".

Of course that I have always known that Bell constructed his inequalities because he wanted to prove exactly the opposite than what he proved at the end. He was unhappy until the end of his life. Bad luck. Nature doesn't care if some people can't abandon their prejudices.

This sounds like lumo thinks that Bell has tried to prove, with his inequalities, that quantum mechanics is wrong. This does not sound very plausible. It is quite clear that he liked Bohmian mechanics, that he has seen it's nonlocality as an argument against it, and tried to remove this argument, by showing that this nonlocality is a necessary property of all hidden variable theories. About his bets before the experiments have been performed, there is the following quote: "In view of the general success of quantum mechanics, it is very hard for me to doubt the outcome of such experiments. However, I would prefer these experiments, in which the crucial concepts are very directly tested, to have been done and the results on record. Moreover, there is always the slim chance of an unexpected result, which would shake the world." (Freire, arXiv:quant-ph/0508180, p.20)

[arguing against "I've read that the Broglie-Bohm theory makes the same predictions that the normal quantum randomness theory makes but the latter was chosen because it was conceived first.":]

Concerning the first point, people can have various theories in the first run. But once they have all possible alternative theories, they can compare them.

Second, it is not true that the probabilistic interpretation was conceived "first". Quite on the contrary. Technically, it's true that de Broglie wrote his pilot wave theory in 1927, one year after Max Born proposed the probabilistic interpretation, but the very idea that the wave connected with the particle was "real" was studied for many years that preceded it. Both de Broglie (1924) and Schrödinger (1925) explicitly believed that the wave was real which is incorrect.

Given that de Broglie has given up pilot wave theory shortly after 1927, unable to find a viable measurement theory for other observables than position, one can say that pilot wave theory appeared in a viable form only 1952, with Bohm's measurement theory. At that time, the Copenhagen interpretation was already well-established (even if the label "Copenhagen interpretation" was coined only later). So there was an advantage of historical accident for the standard interpretation.

In 1952, Bohm wrote down a very straightforward multi-particle generalization of de Broglie's equations and added a very controversial version of "measurement theory". Is it a substantial improvement you expect from 25 years of progress?

Depends on how many people have worked on it during this time. In this case, most of these 25 years nobody has worked on it. In particular, de Broglie himself had broken it, because he was unable to find the "very controversial" measurement theory found later by Bohm. Bohm, who was 1927 only 10 years old, had not worked most of this time in this domain too. Thus, very few man-years have been sufficient to transform a theory broken by it's creator as not viable into a viable theory. I would name this a sufficiently efficient and substantial improvement.

The next important defender of this theory – again almost alone for a long time – was Bell. The results of his work in the foundations of quantum theory are also well-known. Despite their foundational character, they have caused a large experimental activity. Thus, also a quite efficient relation between man-years and results.

(Given that lumo has not understood the main point of Bohm's measurement theory, we can ignore the characterization of this theory as "very controversial").

About decoherence and the classical limit

Moreover, the question which of them will emerge as natural quantities in a classical limit cannot be answered a priori. Which observables like to behave classically? Well, it is those whose eigenstates decohere from each other.

The role of decoherence in the classical limit is largely overexaggerated, see the Hyperion discussion about this (Ballentine, Classicality without Decoherence: A Reply to Schlosshauer, Found Phys (2008) 38: 916-922, DOI 10.1007/s10701-008-9242-0, Schlosshauer, Classicality, the ensemble interpretation, and decoherence: Resolving the Hyperion dispute, Found Phys (2008) 38: 796-803, DOI 10.1007/s10701-008-9237-x, arXiv:quant-ph/0605249, Wiebe and Ballentine, Phys. Rev. A 72:022109, 2005, also arXiv:quant-ph/0503170).

Essentially, you can measure every operator, together with every other, if the accuracy of the common measurement is below the boundaries of the uncertainty relations. And in the classical \(\hbar \to 0\) limit they all like to behave classically.

Everything in this real world is quantum while the classical intuition can only be an approximation, and it is a good approximation only if decoherence is fast enough i.e. if the interference between the different eigenstates is eliminated. If it is so, the quantum probabilities may be imagined to be ordinary classical probabilities and Bell's inequalities are restored.

So if you want to know whether a particular quantity may be imagined to be classical, you need to know how quickly its eigenvectors decohere from each other. And the answer depends on the dynamics. Decoherence is fast if the different eigenvectors are quickly able to leave their distinct fingerprints in the environment with which they must interact.

A nice description of the decoherence paradigm. The little dirty secret of decoherence is that it depends on some decomposition of the world into systems. Such a decomposition can be found, without problems, if we have some classical context as in the Copenhagen interpretation, or some well-defined configuration of the universe as in pilot wave theory, by considering an environment of the actual state of the universe. But without such a background structure you have nothing to start these decoherence considerations. The different systems we see around us – cats, for example – cannot be used for this purpose, at least not if we want to avoid circular reasoning. arXiv:0901.3262 The Hamilton operator, taken alone, is not enough to derive a decoherence-preferred basis uniquely.

Mechanistic models of state-of-the-art quantum theories are not available: it is partly because it's not really possible and it's not natural but it is also partly because the champions of Bohmian mechanics are simply not good enough physicists to be able to study state-of-the-art quantum theories. They're typically people with philosophical preconceptions who simply believe that the world has to respect their rules of "realism" or even "determinism".

I have a quite nice "mechanistic model" for the standard model of particle physics. One which essentially allows to compute the SM gauge group (as a maximal group which fulfills a few simple "mechanistic" axioms). How many more years (and how many more man-years) string theory needs to reach something comparable?

The idea of "philosophical preconceptions" is quite funny. My concept is quite pragmatical: If there is a simple way to do the things, use it. Simplicity is a good thing, independent of the age or the popularity of the particular concept. About determinism I don't care even today, in particular I have certain sympathies for Nelson's stochastics. And I have as well looked at non-realistic interpretations of quantum theory, like the concept I prefer to name "inconsistent histories". But I think there should be really good evidence to justify the rejection of such simple, general, fundamental and beautiful principles like realism. But pilot wave theory would be preferable even without it, simply for the beauty of the guiding equation.

Last but not least, some funny but unimportant polemics

The attempts to return physics to the 17th century deterministic picture of the Universe are archaic traces of bigotry of some people who will simply never be persuaded by any overwhelming evidence – both of experimental and theoretical character – if the evidence contradicts their predetermined beliefs how the world should work.

Well formulated. I like such polemics. Especially replacing the standard 19th century in such flames by 17th century is nice. But there is room for enhancement. In philosophy of science, I follow Popper, who likes to identify the origin of some of his ideas in Ancient Greece. I also prefer the economic system based on ideas of Adam Smith in comparison with much more modern ones developed by Lenin and Mao, so one can identify this sympathy for old ideas as deeply rooted in my personality. Indeed, I think there is nothing wrong with old ideas.

To describe pilot wavers as "predetermined" sounds really nice, but is, unfortunately, wrong. There are, of course, people who follow predetermined ideas. But these are the ideas they have learned in their youth. Where are the proponents of pilot wave ideas supposed to have learned it? What I was teached was quantum theory and Marxism-Leninism, not pilot wave theory and Adam Smith. And I remember, in particular, some uncritical fascination learning von Neumann's proof of impossibility of a classical picture. I have had nor a prejudice for 17th century determinism, nor any of the "bourgeois prejudices" the communists have liked to argue against.

It was not predetermination, but the power of arguments (in particular, of Bell's "speakable and unspeakable in quantum mechanics"), which has persuaded me to switch to pilot wave theory. And an important part of this argumentative power was the simple proof of equivalence between pilot wave theory and quantum theory. There simply is no experimental evidence against pilot wave theory.

And, indeed, the "experimental evidence" presented by lumo was (in his polarizer argument, and similar ones about spins) based on the common error not to take into account the measurement device, or (in his quantization argument) not applicable to de Broglie's version of pilot wave theory. About the theoretical evidence judge yourself.

But the very fact that the Bohmists actually don't work on the cutting-edge physics of spins, fields, quarks, renormalization, dualities, and strings is enough to lead us to a very different conclusion: they're just playing with fundamentally wrong toy models and by keeping their focus on the 1-particle spinless case, they want to hide the fact that their obsolete theory contradicts pretty much everything we know about the real world.

It is always fun to compare the "very facts" of such claims with reality. The one-particle spinless case has never been in the focus of my interest, except if this appears sufficient to show some serious problems of other interpretations ( arXiv:0901.3262, arXiv:0903.4657). The results of my work with spins, fields, and quarks I have already mentioned. And even renormalization is on my todo list, even if some other problems have, yet, higher priority for me.

I'm not sure that naming strings and dualities "cutting-edge physics" is justified. This is clearly a domain of research I leave to lumo – it may have a value as a nice exercise in mathematics, which is an important part of human culture, even if it has nothing to do with physics. Of course, one never knows – results of pure mathematicians, who have been proud of doing things which will never find an application, are applied today in cryptography. It would be a really nice joke if some result found by lumo would find a physical application in some hidden variable ether theory ;-).

Read More

In defense of five standard deviations

Originally posted on August 12th. The second part was added at the end. The third part. Last, fourth part.

Five standard deviations are cute.



However, Tommaso Dorigo wrote the first part of his two-part "tirade against the five sigma",

Demistifying The Five-Sigma Criterion

I mostly disagree with his views. The disagreement begins with the first word of the title ;-) that I would personally write as "demystifying" because what we're removing is mystery rather than mist (although the two are related words for "fog").

He "regrets" that the popular science writers tried to explain the five-sigma criterion to the public – I think they should be praised for this particular thing because the very idea that the experimental data are uncertain and scientists must work hard and quantitatively to find out when the certainty is really sufficient is one of the most universal insights that people should know about the real-world science.




When I was a high school kid, I mostly disliked all this science about error margins, uncertainties, standard deviations, noise. This sentiment of mine must have been a rather general symptom of a theorist. Error margins are messy. They're the cup of tea of the sloppy experimenters while the pure and saint theorist only works with perfect theories making perfect predictions about the perfectly behaving Universe.

Of course, sometimes early in the college, I was forced to get dirty a little bit, too. You can't really do any empirical research without some attention paid to error margins and probabilities that the disagreements are coincidental. As far as I know, the calculations of standard deviations was one of the things that I did not learn from any self-studies – these topics just didn't previously look beautiful and important to me – and the official institutionalized education system had to improve my views. The introduction to error margins and probabilistic distributions in physics was a theoretical introduction to our experimental lab courses. It was taught by experimenters and I suppose that it was no accident because they were more competent in this business than most of the typical theorists.

At any rate, I found out that the manipulations with the probability distributions were a nice and exact piece of maths by themselves – even though they were developed to describe other, real things, that were not certain or sharp – and I enjoyed finding my own derivations of the formulae (the standard deviations for the coefficients resulting from linear regression were the most complex outcomes of this fun research).




At any rate, hypotheses predict that a quantity \(X\) should be equal to \(\bar X\pm \Delta X\) if I use simplified semi-laymen's conventions. The error margin – well, the standard deviation – \(\Delta X\) is never zero because our knowledge of the theory, its implications, or the values of the parameters we have to insert to the theory are never perfect.

Similarly, the experimenters measure the value to be \(X_{\rm obs}\) where the subscript stands for "observed". The measurement also has its error margin. The error margin has two main components, the "statistical error" and the "systematic error". The "total error" for a single experiment may always be calculated (using the Pythagorean theorem) as the hypotenuse of the triangle whose legs are the statistical error and the systematic error, respectively.

The difference between the statistical error and the systematic error is that the statistical error contains all the contributions to the error that tend to "average out" when you're repeating the measurement many times. They're averaging out because they're not correlated with each other so about one-half of the situations are higher than the mean and one-half of them are lower than the mean etc. and most of the errors cancel. In particular, if you repeat the same dose of experiments \(N\) times, the statistical error decreases \(\sqrt{N}\) times. For example, the LHC has to collect many collisions because the certainty of its conclusions and discoveries is usually limited by the "statistics" – by their having an insufficient number of events that can only draw a noisy caricature of the exact graphs – so it has to keep on collecting new data. If you want the relative accuracy (or the number of sigmas) to be improved \(K\) times, you have to collect \(K^2\) times more collisions. It's that simple.

On the other hand, the systematic error is an error that always stays the same if you repeat the experiment. If the CERN folks had incorrectly measured the circumference of the LHC to be 27 kilometers rather than 24.5 kilometers, this will influence most of the calculations and the 10% error doesn't go away even after you perform quadrillions of collisions. All of them are affected in the same way. Averaging over many collisions doesn't help you. Even the opinions of two independent teams – ATLAS and CMS – are incapable of fixing the bug because the teams aren't really independent in this respect as both of them use the wrong circumference of the LHC. (This example is a joke, of course: the circumference of the LHC is known much much more accurately; but the universal message holds.)

When you're adding error margins from two "independent" experiments, like from the ATLAS collisions and the CMS collisions, you may add the statistical errors for "extensive" quantities (e.g. the total number of all collisions or collisions of some kind by both detectors) by the Pythagorean theorem. It means that the statistical errors in "intensive quantities" (like fractions of the events that have a property) decreases as \(1/\sqrt{N}\) where \(N\) is the number of "equal detectors". However, the systematic errors have to be added linearly, so the systematic errors of "intensive" quantities don't really drop and stay constant when you add more detectors. Only once you calculate the total systematic and statistical errors in this non-uniform way, you may add them (total statistical and total systematic) via the Pythagorean theorem (physicists say "add them in quadrature").

So far, all the mean values and standard deviations are given by universal formulae that don't depend at all on the character or shape of the probabilistic distribution. For a distribution \(\rho(X)\), the normalization condition, the mean value, and the standard deviation are given by\[

\eq{
1 & = \int dX\,\rho(X) \\
\bar X &= \int dX\,X\cdot \rho(X) \\
(\Delta X)^2 &= \int dX\,(X-\bar X)^2\cdot\rho (X)
}

\] Note that the integral \(\int dX\,\rho(X)\) with the extra insertion of any quadratic function of \(X\) is a combination of these three quantities. The Pythagorean rules for the standard deviations may be shown to hold independently of the shape of \(\rho(X)\) – it doesn't have to be Gaussian.

However, we often want to calculate the probability that the difference between the theory and the experiment was "this high" (whether the probability is high enough so that it could appear by chance) – this is the ultimate reason why we talk about the standard deviations at all. And to translate the "number of sigmas" to "probabilities" or vice versa is something that requires us to know the shape of \(\rho(X)\) – e.g. whether it is Gaussian.



There's 32% risk that the deviation from the central value exceeds 1 standard deviation (in either direction), 5% risk that it exceeds 2 standard deviations, 0.27% that it exceeds 3 standard deviations, 0.0063% that it exceeds 4 standard deviations, and 0.000057% which is about 1 part in 1.7 million that it exceeds five standard deviation.

So far, Dorigo wrote roughly four criticisms against the five-sigma criterion:

• five sigma is a pure convention
• the systematic errors may be underestimated which results in a dramatic exaggeration of our certainty (we shouldn't be this sure!)
• the distributions are often non-Gaussian which also means that we should be less sure than we are
• systematic errors don't drop when datasets are combined and some people think that they do

You see that this set of complaints is a mixed bag, indeed.

Concerning the first one, yes, five sigma is a pure convention but an important point is that it is damn sensible to have a fixed convention. Particle physics and a few other hardcore hard disciplines of (usually) physical sciences require 5 sigma, i.e. the risk 1 in 1.7 million that we have a false positive, and that's a reasonably small risk that allows us to build on previous experimental insights.

The key point is that it's healthy to have the same standards for discoveries of anything (e.g. 1 in 1.7 million) so that we don't lower the requirements in the case of potential discoveries we would be happy about; the certainty can't be too small because the science would be flooded with wrong results obtained from noise and subsequent scientific work building on such wrong results would be ever more rotten; and the certainty can't ever be "quite" 100% because that would require an infinite number of infinitely large and accurate experiments and that's impossible in the Universe, too.

We're gradually getting certain that a claim is right but this "getting certain" is a vague subjective process. Science in the sociological or institutionalized sense has formalized it so that particle physics allows you to claim a discovery once your certainty surpasses a particular thresholds. It's a sensible choice. If the convention were six sigma, many experiments would have to run for a time longer by 35% or so before they would reach the discovery levels but the qualitative character of the scientific research wouldn't be too different. However, if the standard in high-energy physics were 30 sigma, we would still be waiting for the Higgs discovery today (even though almost everyone would know that we're waiting for a silly formality). If the standard were 2 sigma, particle physics would start to resemble soft sciences such as medical research or climatology and particle physicists would melt into stinky decaying jellyfish, too. (This isn't meant to be an insulting comparison of climatology to other scientific disciplines because this comparison can't be made at all; a more relevant comparison is the comparison of AGW to other religions and psychiatric diseases.)

Concerning Tommaso's second objection, namely that some people underestimate systematic errors, well, he is right and this blunder may shoot their certainty about a proposition through the roof even though the proposition is wrong. But you can't really blame this bad outcome – whenever it occurs – on the statistical methods and conventions themselves because you need some statistical methods and conventions. You must only blame it on the incorrect quantification of the systematic error.

The related point is the third one, namely that the systematic errors don't have to be normally distributed (i.e. with a distribution looking like the Gaussian). When the distribution have thick tails and you have several ways to calculate the standard deviations, you should better choose the largest one.

However, I need to say that Tommaso heavily underestimates the Gaussian, normal distribution. While he says that it has "some merit", he thinks that it is "just a guess". Well, this sentence of his is inconsistent and I will explain a part of the merit below – the central limit theorem that says that pretty much any sufficiently complicated quantity influenced by many factors will be normally distributed.

Concerning Tommaso's last point, well, yes, some people don't understand that the systematic errors don't become any better when many more events or datasets are merged. However, the right solution is to make them learn how to deal with the systematic errors; the right solution is not to abandon the essential statistical methods just because someone didn't learn them properly. Empirical science can't really be done without them. Moreover, while one may err on the side of hype – one may underestimate the error margins and overestimate his certainty – he may err on the opposite, cautious side, too. He may overstate the error margins and \(p\)-values and deny the evidence that is actually already available. Both errors may turn one into a bad scientist.

Now, let me return to the Gaussian, normal distribution. What I want to tell you about – if you haven't heard of it – is the central limit theorem. It says that if a quantity \(X\) is a sum of many (\(M\to\infty\)) terms whose distribution is arbitrary (the distributions for individual terms may actually differ but I will only demonstrate a weaker theorem that assumes that the distributions coincide), then the distribution of \(X\) is Gaussian i.e. normal i.e. \[

\rho(X) = C\exp\zav{ - \frac{(X-\bar X)^2}{2(\Delta X)^2} }

\] i.e. the exponential of a quadratic function of \(X\). If you need to know, the normalization factor is \(C=1/(\Delta X)\sqrt{2\pi}\). Why is this central limit theorem true?

Recall that we are assuming\[

X = \sum_{i=1}^M S_i.

\] You may just add some bars (i.e. integrate both sides of the equation over \(X\) with the measure \(dX\,\rho(X)\): the integration is a linear operation) to see that \[

\bar X = \sum_{i=1}^M \bar S_i.

\] It's almost equally straightforward (trivial manipulations with integrals whose measure is still \(dX\,\rho(X)\) or similarly for \(S_i\) and that have some extra insertions that are quadratic in \(S_i\) or \(X\)) to prove that\[

(\Delta X)^2 = \sum_{i=1}^M (\Delta S_i)^2

\] assuming that \(S_i,S_j\) are independent of each other for \(i\neq j\) i.e. that the probability distribution for all \(S_i\) factorizes to the product of probability distributions for individual \(S_i\) terms. Here we're assuming that the error included in \(S_i\) is a "statistical error" in character.

So the mean value and the standard deviation of \(X\), the sum, are easily determined from the mean values and the standard deviations of the terms \(S_i\). These identities don't require any distribution to be Gaussian, I have to emphasize again.

Without a loss of generality, we may linearly redefine all variables \(S_i\) and \(X\) so that their mean values are zero and the standard deviations of each \(S_i\) are one. Recall that we are assuming that all \(S_i\) have the same distribution that doesn't have to be Gaussian. We want to know the shape of the distribution of \(X\).

An important fact to realize is that the probabilistic distribution for a sum is given by the convolution of the probability distributions of individual terms. Imagine that \(X=S_1+S_2\); the arguments below hold for many terms, too. Then the probability that \(X\) is between \(X_K\) and \(X_K+dX\) is given by the integral over \(S_1\) of the probability that \(S_1\) is in an infinitesimal interval and \(S_2\) is in some other corresponding interval for which \(S_1+S_2\) belongs to the desired interval for \(X\). The overall probability distribution is given by \[

\rho(X_K) = \int dS_1 \rho_S(S_1) \rho_S(X_K-S_1).

\] You should think why it's the case. At any rate, the integral on the right hand side is called the convolution. If you know some maths, you must have heard that there's a nice identity involving convolutions and the Fourier transform: the Fourier transform of a convolution is the product of the Fourier transforms!

So instead of \(\rho(X_K)\), we may calculate its Fourier transform and it will be given by a simple product (we return to the general case of \(M\) terms immediately)\[

\tilde \rho(P) = \prod_{i=1}^M \tilde\rho(T_i).

\] Here, \(P\) and \(T_i\) are the Fourier momentum-like dual variables to \(X\) and \(S_i\). However, now we're almost finished because the products of many (\(M\)) equal factors may be rewritten in terms of an exponential. If \(\rho(T_i)=\exp(W_i)\), then the product of \(M\) equal factors is just \(\exp(MW_i)\) and the funny thing is that this becomes totally negligible if \(MW_i\gg 1\). So we only need to know how the right hand side behaves in the vicinity of the maximum of \(T_i\) or \(W_i\). A generic function \(W_i\) may be approximated by a quadratic function over there which means that both sides of the equation above will be well approximated by \(C_1\exp(-MC_2 T_i^2)\) for \(M\to\infty\).

It's the Gaussian and if you make the full calculation, the Gaussian will inevitably come out as shifted, stretched or shrunk, and renormalized so that \(X\), the sum, has the previously determined mean value, the standard deviation, and the probability distribution for \(X\) is normalized. Just to be sure, the Fourier transform of a Gaussian is another Gaussian so the Gaussian shape rules regardless of the variables (or dual variables) we use.

So there's a very important – especially in the real world – class of situations in which the quantity \(X\) may be assumed to be normally distributed. The normal distribution isn't just a random distribution chosen by some people who liked its bell-like shape or wanted to praise Gauss. It's the result of normal operations we experience in the real world – that's why it's called normal. The more complicated factors influencing \(X\) you consider, and they may be theoretical or experimental factors of many kinds, the more likely it is that the Gaussian distribution becomes a rather accurate approximation for the distribution for \(X\).

Whenever \(X\) may be written as the sum of many terms with their error margin (even though the inner structure of these terms may have a different, nonlinear character etc.; and the sum itself may be replaced by a more general function because if it has many variables and the relevant vicinity of \(X\) is narrow, the linearization becomes OK and the function may be well approximated by a linear combination i.e. effectively a sum, anyway), the normal distribution is probably legitimate. Only if the last operation to get \(X\) is "nonlinear" – if \(X\) is a nonlinear function of a sum of many terms etc. or if you have another specific reason to think that \(X\) is not normally distributed, you should point this fact out and take it into account.

But Tommaso's fight against the normal distribution as the "default reaction" is completely misguided because pretty much no confidence levels in science could be calculated without the – mostly justifiable – assumption that the distribution is normal. Tommaso decided to throw the baby out with the bathwater. He doesn't want an essential technique to be used. He pretty much wants to discard some key methodology but as a typical whining leftist, he has nothing constructive or sensible to offer for the science he wants to ban.

Second part

Dorigo's second part of the article is insightful and less controversial.

He reviews a nice 1968 Arthur Rosenfeld paper showing that the number of fake discoveries pretty much agrees with the expectations – some false positives are bound to happen due to the number of histograms that people are looking at. Sometimes experimenters tend to improve their evidence by ad hoc cuts if they get excited by the idea that they have made a discovery. Too bad.

Dorigo argues that the five-sigma criterion should be replaced by a floating requirement. This has various arguments backing it. One of them is that people have differing prior subjective probabilities quantifying how much plausible or almost inevitable they consider a possible result. Of course that extraordinary claims require extraordinary evidence while almost robustly known and predicted ones require a weaker one. It's clear that people get convinced by some experimental claims at a lower number of sigmas than for other claims. But I wouldn't institutionalize this variability because due to the priors' intrinsically subjective character, it's extremely hard to agree on the "right priors".

He also mentions OPERA that made the ludicrous claim about the superluminal neutrinos that was called bogus on this blog from the very beginning. It was a six-sigma result (60 nanoseconds with a 10 nanoseconds error), we were told. Dorigo blames it on the five-sigma standards. But this is just silly. Whatever statistical criterion you will introduce for a "discovery", you will never fully protect physics against a silly human error that may introduce an arbitrary large discrepancy to the results – against stupid errors such as the loosened cable. I wouldn't even count it as a systematic error; it's just a stupid human error that can't really be quantified because nothing guarantees that it remains smaller than a bound. So I think it's irrational to mix the debate about the statistical standards with the debate about loosened cables and similar blunders that cripple the quality of an experimental work "qualitatively" – they have nothing to do with one another!

Third part

In the third part, Dorigo discusses three extra classes of effects and tricks that may lead to fake discoveries. I agree with everything he writes but none of these things implies that the 5-sigma standard is bad or that it could be replaced by something better.

The first effect is mismodeling (well, a systematic error on the theoretical side); the second effect is aposterioriness, the search for bumps in places where we originally didn't want to look (which is OK for discovering new physics but such unplanned situations may heavily and uncontrollably increase the number of discrepancies i.e. false positives we observe, and we shouldn't forget about it when we're getting excited about such a discrepancy); and dishonest manipulation of the data (there's no protection here, except to shoot the culprit; if someone wants to spit on the rules, he will be willing to spit on any rules).

Fourth, last part

In this fourth and final part, Dorigo continues in a discussion of a bump near \(150\GeV\). At the very end, he proposes ad hoc modifications of the five-sigma rule – from 3 sigmas for the B decay to two muons to 8 sigmas for gravitational waves. One could assign ad hoc requirements that differ for different phenomena but it's not clear how it would be determined for many other phenomena for which the holy oracle Dorigo hasn't specified his miraculous numbers. Moreover, the patterns in his numbers don't seem to make any sense. It is very bizarre why a certain exotic, not-guaranteed-to-exist decay of the B-mesons is OK with 3 sigmas while the gravitational waves that have to exist must pass 8 sigmas. Moreover, some other observed signatures, like "SUSY", aren't really signatures but whole frameworks that may manifest themselves in many experiments and each of them clearly requires a different assignment of the confidence levels if we decide that confidence levels should be variable. There would be some path from being sure that there's new physics to being reasonably sure that it's a SUSY effect – Dorigo seems to confuse these totally different levels of knowledge.

If this table with the variable confidence levels were the goal of Dorigo's series, then I must say that the key thesis of his soap opera is crap.

Read More

Some physics links

Carroll's QM, NYT's firewalls, Jester's whining on scales

Sean Carroll has unlocked the quantum chapter from his "Eternity" book,

Quantum Mechanics Made Easy,

which I found better than expected despite its misleading comments about the "collapse", "its" relationships with the "arrow of time", the meaning of the "Copenhagen Interpretation", the "many worlds" as the "leading alternative contender", and many other things (he omits Bohm etc.). Carroll's text is flawed in different ways than e.g. Brian Greene's musings about the interpretation of quantum mechanics but I wouldn't say it's "more flawed". I am still not aware of any popular presentation of the foundations of quantum mechanics that is done right.




Now, Dennis Overbye wrote quite an informative and long The New York Times article about the black hole firewalls and the ER-EPR correspondence,

A Black Hole Mystery Wrapped in a Firewall Paradox.

Overbye confirms his status as one of the world's best popular science writers who focuses on theoretical physics. As a reader, you are reminded about some of the intellectual stories of Einstein and Hawking as well as the interactions that made various people confuse each other about the firewall fallacy on the West Coast. Susskind whose opinions resembled a yo-yo hasn't changed his views for several months, we're told.

Incidentally, Bousso has a new paper about the firewalls and if I judged the situation according to everything that is written in these still equally confusing if not crazy papers (Bousso claims that there's a firewall even when the black hole is very young – a brutal violation of locality), I would have to agree that they're as confused as 40 years ago – something written in Overbye's article. In the real world, the claim that things haven't become much clearer in the last 40 years is an outrageously crazy lie. At least my understanding of all these questions is much better than even 15 years ago – perhaps because I don't change my mind whenever someone writes some new bullshit.



Most frequent surnames. Click for all 40 informative and funny maps that you will ever need. Via the Pirate of Prague.

There will be a workshop in Santa Barbara about these matters. It's organized by Don Marolf. He also posted his own new paper arguing that holography doesn't depend on strings. As far as I can see, it's a sloppy tirade full of unsubstantiated claims – well, the claims are supported at most by a vague analysis of something that has been analyzed very accurately since late 1997. We know pretty much exactly what the bulk stringy objects – strings and branes and black holes etc. – mean in the boundary CFT, when they matter, when they can be neglected, which properties of string theory are important and unimportant for various derivations (e.g. of black hole entropy), and so on. It's hard to imagine how a superficial paper such as Marolf's could be considered as anything else than a redundant, outdated, not quite competent, low-quality addition to the literature.




Finally, Jester at Resonaances published a new blog entry after 4 months or so (delay caused by "work, travel, frustration, depression, and sloth").

I agree with all of his "matter of fact" claims but I don't share his emotional reaction at all.

Jester says that since the 1930s, physicists – we would say phenomenologists – would be looking towards a holy grail, a high-energy scale, which always happened to be the electroweak scale. New physics had to be added to make the theory work. Now, the theory at the electroweak scale (the Standard Model) seems to be completed and consistent as a renormalizable quantum field theory so there's nothing else to "inevitably expect" at that scale – and the LHC confirms that except for the Higgs, this scale isn't "overflowing" with tons of new physics. So even though it's still likely but not certain that there's new physics such as SUSY at a nearby scale, the physicists have to look at another scale to achieve a certainty about new physics but he doesn't know which one.



So this is how the situation looks to Jester – and Savas. No clear direction for progress, they think.

Jester's claims about the physics are true but his reasons for frustration are inappropriate. What he dislikes is that regardless of the energy of the next (realistic) collider, \(30\TeV\), \(100\TeV\), or \(1,000\TeV\), we can't be quite sure that the collider finds new physics. I agree with that but I am not shocked by it in any way.

It has been clear to me that the Standard Model was a complete renormalizable theory – a theory that can be extrapolated to really, really high energy scales – since I was a college sophomore. (In the following year, I began to attend QFT courses: buy the book on the electroweak theory by my major undergraduate QFT instructor.) So of course that once all the necessary pieces of this Standard Model are established, there is no other new physics testable at the accelerators that is "inevitable" in an energy range that is "guaranteed".

One could say that I was solving similar questions as Jester but it was 20 years ago. Is that the first time when Jester realizes that the Standard Model is a complete renormalizable QFT that doesn't need extra additions? We know about some likely additions linked to the Higgs stability, hierarchy problem, dark matter, inflation etc. (Jester discusses them as well) but we're not guaranteed that they're accessible by any particular realistic collider.

Well, I have always even shared Jester's strategy to find an energy scale below which new physics is guaranteed. And I found it (so did Max Planck 100 years earlier although he didn't quite understand the physics of the scale). It's the Planck scale. New phenomena have to occur at the scale \(10^{19}\GeV\) or lower (it may be much lower in models with large or warped extra dimensions but the true fundamental scale can't be higher).

It would be fun if the scale could be accessed by direct experiments except that it seems unlikely and we have always thought it was unlikely. So why should we be "negatively surprised" now? I, for one, don't care much (and have never cared much) that Nature has made the scale so high that it's not accessible by our particle physics experiments. (Of course, the gap between the scales has to be this high because it's ultimately needed for the existence of life. Evolution of life requires lots of sunny days. Stars are long-lived because they contain many nuclei that may be burned and the number of nuclei ends up large because gravity is so much weaker than other forces and pressures so lots of nuclei are needed for the gravity to become significant and compensate the repulsive pressures.)

The Planck scale is the scale where my thinking about fundamental physics – and the thinking of any top-down theorist – begins. From this inaccessible scale, one may dig deeper perhaps down to the experimenters' scales but it has always been clear that the true fundamental scale where important things have to exist is the Planck scale. (Stringy and grand unified theories make it likely but not guaranteed that there are some interesting additional phenomena at scales slightly, perhaps 10-100 times, lower than the Planck scale, too.) Everything else are just optional islands alternating with optional deserts. You may demand that there is some land in the ocean at each 100 miles (and even claim that gaps in the ocean that are longer than 100 miles prove that geography is no science because you can't swim there) except that the rules of geography don't have to respect your demands. The existence and inner workings of Eurasia is mostly independent of America so – even though Jester may find it surprising – no one is guaranteed that there has to be an America at most 100 miles from Spain.

So what the confused headless hens show is just a minor thing – that the dogmatic bottom-up, direct-experiments-based view on fundamental physics isn't a sustainable strategy to keep the research going. All the people claiming that physicists are only allowed to think about things that must have consequences for particular near-future experiments have always been as deeply misguided as the Spanish Catholics who wanted to prevent Columbus from trying to sail to another continent around the round globe.

One has to think about topics that are behind the currently observable horizons if he wants to make real progress! We could say that the bottom-up paradigm is complete and dead and it's time for every single phenomenologist to learn some string/M-theory because this is the new physics that is guaranteed at a certain well-defined scale. Otherwise one will be stuck with increasingly bizarre speculations based on a wishful thinking. The hen cartoon is right: if you're chasing a new visible island of new physics that has to be visible from your place, you will lose the sense of direction and your search will be chaotic and unguided. Look at Jester's words:

So, while pushing up the energy frontier in accelerators will continue, I think that currently searching high and low for a new scale is the top priority. ...

He wants to guess the kingdom of the new scale and its princess from some higher-dimension operators such as\[

\frac{1}{\Lambda^2}(\bar s\gamma_\rho b) (\bar \mu \gamma^\rho \mu).

\] It's a fact that you're not guaranteed to find anything at any particular scale. It's like looking at the Atlantic Ocean with your binoculars. Are you guaranteed to see hints of a new continent, America? Isn't it better to be more courageous and abandon the totally unjustified assumption that your left leg should permanently stand on the Iberian peninsula? Columbus did so. His group was sailing for a few thousand kilometers – he didn't exactly know how much was needed – and he discovered a new continent. Chances are that you can't discover a new important land without getting wet!

Just to be sure, I think it's slightly more likely than not that SUSY will be discovered at the LHC in 2015 and the probability is comparable to one-half that the dark matter underground experiments will agree about a dark matter particle. But none of these probabilities is "well above 99 percent", for example. Indeed, we're not guaranteed that any particular experiment will discover these new things. There's no scientific or rational reason why we should be guaranteed that. And there's no reason for science to stop just because we're not guaranteed such things.

Again, it's time for HEP non-experimenters to (at least partially) switch to the top-down thinking unless they want to increasingly resemble the headless hens.

Read More

Steve Pinker is right to defend "scientism"

A week ago, Harvard's top evolutionary psychologist Steven Pinker wrote an essay for The New Republic,

Science Is Not Your Enemy: An impassioned plea to neglected novelists, embattled professors, and tenure-less historians,

that defends the application of the scientific method to various fields, including those that used to be monopolized by the tools of humanities and other methods and non-methods. I think that both Pinker and your humble correspondent think that the would-be expletive "scientism" is being mostly used for the idea that scientific reasoning shouldn't be confined just to the traditional places but it should be extended to new realms.

If that's so, count me as a scientist! Or what's the word for the champion of scientism? ;-)

To be sure, I have met people who were applying naive, science-inspired models to very complex systems and they would deserve to be criticized or told why they were wrong. But in my experience, these were not the primary recipients of the label "scientism".

Pinker starts by saying that the great folks of the Enlightenment were scientists, science has improved our lives in many ways, the understanding itself is extremely valuable (in contrast with a despicable statement in the 2006-2007 Harvard general education requirement that offended me as much as it offended Pinker).




He says that science is facing a coalition of the religious fundamentalists and postmodernists – I agree with that – and that the defenders of science don't claim that real-world scientists are infallible or the wisest ones. On the contrary, science is based on two ideals – that the world may be studied and get increasingly familiar; and that this process of learning is hard and should be hard.




My degree of agreement with Pinker is so high – it's not the first time – that reviewing his ideas could be boring. I guess that you expect some polarization so I chose to criticize a critic of Pinker called Massimo Pigliucci, a department head in an NY philosophy department, a self-described non-postmodernist, and a postmodernist. In Rationally Speaking, he wrote a critique with the title

Steven Pinker Embraces Scientism. Bad Move, I Think (Science 2.0 copy)

I find his surname too complicated so I will refer to him as the Liberian because this man educated in Rome, Italy was born in Liberia.

Pinker begins awfully, waxing poetic about how the Great Thinkers of the Enlightenment were all scientists, and in particular, cognitive neuroscientists, evolutionary psychologists (!!), and social psychologists. Such thinkers include Descartes, Spinoza, Hobbes, Locke, Hume, Rousseau, Leibniz, Kant, and Smith. All, obviously, philosophers. Yeah, I get it, it was a rhetorical opening gamble. But it is precisely the sort of rhetoric that justly pisses off people in the humanities, so why start an essay that way which ostensibly attempts to reconcile the so-called two cultures?

OK, these men were (also/primarily) "philosophers" but all of them also practiced natural science and held dear its principles. Pinker calls them "thinkers of the Age of Reason and Enlightenment" which is a neutral label, I guess, and the claim that they "were scientists" doesn't mean that they were "nothing else"; this description of the men is not mutually exclusive with other propositions about them. That's why the Liberian's (and other philosophers'?) irritated reaction seems irrational to me.

Pinker clearly wants to suggest that the big men of the Enlightenment were much more into science than the contemporary philosophers and it's a pity! Those men of the Enlightenment really did symbolize a peaceful co-existence of the two "cultures"; it's no cliché.

This – the very suggestion that the folks in the humanities should be interested in science because it may enrich them – is already too much for the Liberian so he makes a lousy joke about the "Prime Directive" when Pinker says (a serious and important thing) that the men of the Enlightenment would be happy to see some modern scientific findings. The Liberian is quickly eager to say that Pinker is not worth reading. Why?

Cue the predictable "scientism is an arbitrary label thrown at things one doesn't like" complaint and you don't need to bother reading the rest of the essay.

Wow, that was fast. It's a fact that the word "scientism" is mostly used exactly in the way that Pinker describes. It is an arbitrary label that people use to reject scientific arguments, findings, facts, and methods whenever they don't like where they would lead. It's an enchantment designed stop a serious discussion before it starts. Does the Liberian have some arguments against Pinker's claim? Perhaps it's the following paragraph:

So, once again, let's revisit the issue of scientism, this time using a different take, which I hope will help us make some progress. I have begun to think of scientism as in a sense the opposite extreme of pseudoscience: while pseudoscientific notions arise from science badly done (or non-science masquerading as science), scientism is about science overreaching (or science trying to expand into non scientific domains).

That may sound nice to him except that as far as I could see, the rest of his essay doesn't contain a glimpse of evidence that science is overreaching, examples when it is overreaching, and evidence that it is a bad thing. So yes, the Liberian's words are just another proof of Pinker's definition of "scientism". It's a label meant to stop a discussion about something before any sensible arguments or thoughts may be raised at all. The word is a weapon meant to protect irrationality's control over whole domains of human activity "because we just want it to be this way forever". The Liberian has a word, "scientism", and this word seems enough to him to identify Pinker as a counterpart of pseudoscientists. The only problem is that Pinker isn't analogous to pseudoscientists at all.

Interestingly, the word pseudoscience can also be used to deflect genuine criticism: oh, you are just throwing pseudoscience at me in order to dismiss what I do without argument, says the ufologist (or astrologist, or homeopath, or...). And of course it is perfectly true that both scientism and pseudoscience can indeed be used inappropriately, just like the term science itself can and has been invoked to prop up all sorts of bad doctrines (scientific psychoanalysis, scientific Marxism, phrenology, eugenics, and so forth).

Well, the objection by the ufologist might even be considered legitimate. If someone just screams expletives such as "pseudoscience" at him, he isn't providing the ufologist with counter-evidence. The difference between "pseudosciences" and "scientism" is that one may also present perfectly scientific, rational, nearly rock-solid arguments supporting the conclusion that pseudosciences are wrong (better arguments than just the screaming of the word "pseudoscience"). One may show how easy it is to create UFOs terrestrially in many ways, how it is impossible that planets or constellations affect the human lives, and so on. On the other hand, one can't present any sensible evidence that the scientific method is bound to fail when applied to human affairs and fields previously dominated by the humanities.

So unlike the opponents of pseudosciences who are backed by pretty much all the content of science, the opponents of "scientism" are only supported by the childish screaming of the would-be insult "scientism". That's the reason why the "scientism" isn't analogous to "pseudosciences" and their opponents aren't analogous, either.

But Jackson Lears' target [JL was criticized by Pinker; he angrily responds beneath Pinker's TNR article] are the writings of Sam Harris, a textbook example of the excesses of scientism if there is any to be found out there! And therein lies the problem: just as in the case of pseudoscience, the devil, so to speak, is in the details. Generic cries of "scientism!" or "pseudoscience!" won't stick, nor should they. But generic dismissals of criticisms of either pseudoscience or scientism shouldn't either. It's just not that simple.

The comment about Sam Harris is off-topic because Pinker has only mentioned his name – but none of his views – above the segment about Lears (which is started by a quote from Lears' review that seems independent of Harris: it's a far-left tirade equating 20th century science with eugenics and imperialism). On the other hand, even though the Liberian opened this new topic, he hasn't written an epsilon of a sensible criticism of Sam Harris, either. Are people supposed to be in the "consensus" that Sam Harris represents "excesses of scientism" without any evidence it's so? Sorry, I haven't read Sam Harris' work but (or: and therefore?) I won't do that. Philosophers may be in the "consensus" that they dislike Sam Harris and his writing but they can still be wrong.

If one demonstrates that the criticisms of "scientism" are nothing else than vacuous anti-scientific babbling, then generic dismissals of the criticism of "scientism" should stick.

Pinker claims that science couldn't possibly indulge in the excesses that its critics level at it because, you know, the whole process employs a series of safeguards, including open debate, peer review, and double blind experiments. Yes, and when the system works, it works really well. But Pinker seems to ignore much research in the history and sociology of science that shows that sometimes that system goes wrong, occasionally worrisomely wrong (e.g., a lot of medical research on drugs is seriously flawed, particularly - but not only - when the funding for it comes from the pharmaceutical industry).

Looking at the comment about the "Big Pharma", you see that the Liberian is just a far left activist. People like him are doing no science or impartial analyses of anything. They are on a crusade to hurt the big corporations because they're unhinged fanatical dirty commies. I am not 100% sure that Steven Pinker would use exactly the same wording ;-) but Massimo Pigliucci is a shitty dishonest jerk.

He is on a crusade not only against the corporations but against the natural sciences, too. That's why he's so obsessed with the – mostly worthless – research trying to hurt the (surely not only) medical research. Much of the medical research is flawed. Some scientific disciplines are more successful or follow the scientific standards more carefully than others. But those that are more sloppy or distorted also enjoy a smaller percentage of Pinker's defense. When science doesn't work properly, it doesn't work properly. It's not really proper science and it's not quite what Pinker is defending. So why shouldn't Pinker "ignore" that?

Of course that he may also write an essay against bad science – and he has written such essays, too. But this essay wasn't about bad science; it was about the bogus label "scientism" which is why different topics were ignored by Pinker. Do you understand it, the Liberian? On the contrary, it's a case of demagogy when the Liberian tries to connect Pinker with the bad science. There isn't any evidence that the bad medical research should be associated with Pinker.

Not to mention that he entirely misses the point of the most frequent cases of scientism: they are not to be found in the technical scientific literature, but rather in popular science writings, when scientists (or people who claim to be interpreting science on behalf of the public) make claims that are simply disproportionate to the evidence (as in many recent instances of neurobabbling).

The popular literature may often print many hypotheses or claims that aren't established and that are sometimes wrong. Sometimes they're shockingly and horribly wrong. On the other hand, it's completely normal that ideas about portions or disciplines of science that aren't quite established or born appear in the popular literature. So if there are no scientific departments dedicated to the scientific research of traditionally humanities-based topics, of course that the discussion about these potential future disciplines has to occur elsewhere.

Science, says Pinker, is committed to two ideals: that the universe is intelligible, and that acquisition of knowledge is hard. Well, I'm not sure why these are "ideals" rather than, say, working assumptions (the first) and acknowledgement of fact (the second). But this is a red herring, of course. Nobody in his right mind is arguing that the universe isn't (to a point, no guarantees!) understandable by us, and certainly nobody is accusing scientists of being lazy. So why bring that up to begin with?

Why bring that up? Because the critics of "scientism" don't have any respect for these basic ideals of science – even though they are critically important. The Liberian has no respect for them, either. You can see this fact on his very attack on the word "ideals". They're just not ideals in his opinion! He explicitly says so.

To a scientifically inclined mind, the two propositions are ideals. The scientists like the apparent fact that the Universe is intelligible and they're decided not to give up the attempts to understand the world ever more intimately. It's their mission so the justification of the mission is an ideal. Similarly, the insight that the scientific method to get familiar with the world is hard is also an ideal because it means that the scientists are ready to avoid the temptation of easy solutions and shortcuts and they're ready for hard work and work that requires patience. They're not scared by this vision – on the contrary, they think it's a part of their method's superiority. That's also why the description of this superiority deserves to be called an ideal.

But bashing once again Stephen Jay Gould's (in)famous idea of two separate magisteria for science and religion, he commits the very same mistake that Gould made: (rational) morality isn't the province of religion, it is a branch of philosophy, and it is philosophers such as myself that have taken to task the scientistic excesses of Harris, Shermer, and co. See?

No, I don't see anything of the sort.

Science and religion are sometimes about different questions but sometimes they're not. Sometimes conflicts arose and still arise. One shouldn't deny the fact that the religion sometimes wants to protect its territory against science even though the scientific evidence is getting strong. Gould has arguably argued that science and religion are guaranteed not to overlap and Pinker criticized him for this naive wishful thinking.

Also, Pinker hasn't used the term "rational morality" or "(rational) morality". The word "rational" was inserted by the Liberian. Clearly, morality has been and still often is under the directorate of religions. Pinker suggests that science influences morality as well – the science-inspired morality is the search for a better happiness of individuals and flourishing of the mankind. Well, I have argued that it's impossible to "scientifically prove" that one principle is moral and another one is not. On the other hand, it would be foolish to deny that science influences our opinions about morality. If we learn that someone has certain preferences since the birth, perhaps hardwired in his DNA, it's silly to try to reeducate him – and perhaps even silly to chastise him for that. Science, when it tells us what are the actual causes of various things and whether they can be changed and what someone may really feel and so on is surely influencing our opinions about morality. Most generally, science has really established that the animals and humans weren't created for a purpose and that the bulk of the religious moral values linked to the worshiping of deity are indefensible (along with the murders committed in the name of deity). All these insights surely do affect a pro-scientific person's morality. So the moral questions can't really be completely "defended" against science; science can't be confined to another "magisterium".

A branch of philosophers also studies morality but it's questionable whether they're much more rational about it than the religious defenders of morality. And I think that Pinker is right when he's trying to convince these philosophers to adopt some scientific approaches as well.

Once again things are more complicated: I am a staunch ally of Pinker when it comes to defending science from religion, but that doesn't mean I cannot raise the issue of scientism when my allies themselves say silly or unsubstantiated things.

It doesn't mean that. It's also true that when you "raise some issues", it doesn't mean that your babbling makes any sense.

Pinker, again predictably, and largely off the topic, goes on to claim that science has contributed enormously to the welfare of humanity, which of course nobody is denying.

Well, the critics of "scientism" are denying a related (although not equally unquestionable) thing, namely that based on the previous experience, it is likely that the propagation of the scientific method to new realms is likely to contribute to the welfare of humanity in the future. This is what Pinker is effectively suggesting and the Liberian doesn't like it.

He also conveniently dismisses or minimizes the problems that science and technology have brought to us: it's ok for science to take credit for vaccines (as it should), but not ok for critics to point out nasty stuff like atomic bombs and biological warfare. See, those aren't really the results of "science," but of bad politicians misusing science. This is such a naive understanding of human power relations, not to mention of the complex social role of science, that it is downright laughable. I keep wondering why serious thinkers like Pinker cannot simply admit science's blunders, graciously acknowledge the criticisms, and genuinely try to forge a better way forward. One would almost suspect that these people are feeling guilty of something.

Pinker criticized the awful education requirements at Harvard from 2006-2007, the early post-Summers era, that (among other bad things) singled out science because science became the only field of the human activity that was criticized for something like nuclear weapons.

Science gives us tools that may be used for good goals as well as bad goals. It amplifies the people's power. The good things have arguably prevailed because people ultimately have more good inside them them the bad.

But the negative description could have been added to the humanities as well. It's really the humanities that have manipulated the societies into the thinking supporting the Inquisition, NSDAP, and others. No philosopher is being forced to apologize for Nietzsche whose philosophy inspired the Nazis, to a significant extent. No philosopher is being routinely asked to apologize for Marx and Lenin whose musings led to the crippling of 1/2 of Europe for much of the 20th century, not to mention tens of millions of murders committed by Stalin (they're not asked to do so even though their philosophy is often nearly identical to that of Marx or Lenin or Stalin or others from this clique). It's only science (and scientists) who are supposed to be this submissive. The education requirement at Harvard was undoubtedly written by science haters with the clear goal to suppress the students' love for science and their idealistic ideas that science is a pure thing. These bastards dreamed about convicting science as a principle.

It makes absolutely no sense to hold Pinker – or any other scientists today – more responsible for the casualties in Hiroshima and Nagasaki than it is to hold professors in humanities responsible for the Auschwitz. The Liberian likes the former but he doesn't like the latter. He has double standards.

I will personally not "graciously accept the criticisms" of this kind (that science should be ashamed for Hiroshima etc.) because the authors of this criticism are dishonest scum. Moreover, I think that it was a good idea to throw the bombs on Japan – it was a decision that has saved millions of people, too. Scientific results may help both the good and the bad and because I think that the U.S. was on the better side of the war than Japan, it shouldn't be unexpected that I also believe that the atomic bomb played an overall positive role because it was developed by the better side.

Moreover, for some reason the accomplishments of science need to be highlighted while at the same time those not attributable to science go acknowledged only parenthetically: "If one were to list the proudest accomplishments of our species (setting aside the removal of obstacles we set in our own path, such as the abolition of slavery and the defeat of fascism), many would be gifts bestowed by science." Yes, let's not count little things like the abolition of slavery and the defeat of fascism, or perhaps the general improvement in human rights, women rights, gay rights, general education, access to health care (as distinct from the science-based quality of that care), and countless other improvements the human race has managed to make without science.

It's disingenuous to claim that the abolition of slavery had nothing to do with science (and technology, which is related). Slavery could only be abolished without brutal consequences for the economy because some slaves could have been gradually replaced by machines.

It's equally crazy to claim that the defeat of fascism was made "without science". It's not nust the military technology that mattered. The strategic planning etc. was also done in a rather "scientific" way.

The human rights were also improved because the society could afford it and this couldn't quite work without science. And it's not quite a coincidence that the modern human rights are linked to the Enlightenment that was started by the pro-science men from the beginning of this whole story. Women rights and gay rights, to the extent that they are justifiable, are justified by science, too. It's about the scientific findings on where gayness comes from and what women may or may not actually do and whether allowing this or that may seriously harm the society.

Without science, the general education would be pretty much general brainwashing. The access to health care couldn't be universal because the society couldn't even afford a sufficient number of people employed as doctors and nurses. Science (and technology) is not just about the quality of healthcare. It's also about the ability to reserve the people who can do this job which is a "luxury" for the society.

But don't get me wrong. I agree that those historical developments are not "primarily" stories about science. But what the Liberian completely neglects is Pinker's important point that these victories just removed some obstacles that other people had previously placed in the mankind's path. So when it comes to the "non-scientific" advances that are worth celebrating, the total progress equals zero: bad policies were introduced and they were later abolished. In this sense, the purely non-scientific progress represents a random walk of a sort. The change is equally likely to be positive and negative. But science is different because it systematically brings new benefits. It isn't just undoing our sins from the past.

Again, this isn't an attack on science, it's simply a matter of pointing out that science has done great goods as well as the more than an occasional evil, and moreover, that much has been accomplished without a lot of help from science. Nuance, people, nuance.

The Liberian is trying to suggest that societies "with science" and "without science" have achieved the same things so it doesn't matter etc. But it greatly matters. The human race has lived without science for a few million years and it has achieved almost nothing. Pretty much everything we associate with the civilization of the last 6,000 years depends on science in one way or another and one should sensibly compare how the mankind looked in those recent millennia with our ideas about the lives of the Homo Habilis.

Of course, the far left activist can't omit this topic:

One more example of the oddly slanted view that Pinker presents: "contrary to the widespread canard that technology has created a dystopia of deprivation and violence, every global measure of human flourishing is on the rise." Well, yes, so is the temperature of the planet, just to mention one example, which may very well put a rather abrupt and unpleasant end to that satisfying rise in human flourishing. And climate change is the result of technology, unless you are a denier of the obvious. (Nuance, people, nuance...)

Pinker correctly said that every global measure of human flourishing is on the rise. This is a comment that contradicts the doomsayers of various kinds – and most of the doomsayers are anti-scientific Luddites of one way or another. They may be motivated religiously or by analogous far left-wing ideologies but the result is pretty much the same.

Climate change isn't the result of "technology"; it's a nearly tautological result of basic laws of astronomy, heliophysics, hydrodynamics, thermodynamics, and especially atmospheric physics in general, laws that have been operating for billions of years. The denial of this trivial observation is also a denial of science as a principle. The recent contamination of the climatology departments by environmentalists, Marxists, carbon regulators, doomsayers, Luddites, and deniers of the Ancient Earth who claim that the Earth was only changing in recent 100-200 years is an example of the opposite trend to scientism – the propagation of dishonest and unscientific jerks, ideologically motivated activists, and imbeciles to disciplines that used to belong to science.

The global mean temperature isn't a standard measure of human flourishing. Moreover, it's been constant for 16 years or so. However, if we strangely decided that the global mean temperature is a standard of human flourishing and took a longer time scale, 30 years or 100 years, it would still confirm Pinker's general thesis because the global mean temperature – our new measure of human flourishing – rose in the last 30 or 100 years. Unless you are a complete idiot, you know that the warmer, the better, and this relationship would hold even if the Earth were 10 °C warmer than it is today. The same comment would apply to CO2 concentrations that have increased by 40 percent since 1700 which improved the life for all plants and, directly or indirectly, to all life forms on Earth. So you may count the CO2 concentration as another measure of human flourishing.

More: "A demonization campaign anachronistically impugns science for crimes that are as old as civilization, including racism, slavery, conquest and genocide." True, science per se is certainly not to blame for those human moral failings. But (some) scientists have actively contributed to the design and production of technological instruments that have made possible the raising of those crimes to never before seen levels. No blame at all? Not even a little tiny bit?

No. This paragraph is another clear indication that the Liberian doesn't understand what science is. Science isn't just another example of the worshiping of some particular people. Science is something else than the the union of acts of people who are called scientists. Only some particular activity of the people – usually people who are called scientists but not necessarily so – is science. So if someone also helped to commit some big crimes – and I have already said that I don't count Hiroshima and Nagasaki to this group – it's not something that science as the process of systematic rational learning of the Nature should be blamed for. Not even a little tiny bit.

You see, the humanities have not yet recovered from the self-inflicted wound of postmodernism, and their insistence in rejecting science is just downright suicidal. I am no defender of postmodernism, as my readers will hopefully well know, but some postmodernists (Foucault, for instance, and before him the pre-postmodernist Feyerabend) have raised serious questions about the social role of science, the unchecked power of scientific institutions, and so forth.

Holy crap. I am no postmodernist, well, except for Feyerabend and occasional Foucault: I just love rants by Foucault, especially his rants against science! Is this guy joking? Foucault represents some of the most typical and most outrageous delusions ever written against science from the postmodern viewpoint.

Whenever such critiques degenerate into a wholesale rejection of science, the critics themselves need to be called out. But it is foolish to throw out the bathwater without checking whether there is a baby still inside the bathtub (to use one of Pinker's own metaphors).

Except that Foucault's – and similar critics' – bathtub only contains bathwater and no baby.

"Several university presidents and provosts have lamented to me that when a scientist comes into their office, it's to announce some exciting research opportunity and demand the resources to pursue it. When a humanities scholar drops by it's to plead for respect for the way things have always been done."

This comment by Pinker may have been a shortcut but it surely reflects a general difference between scientific fields and the non-scientific ones. The cold, objective, meritocratic results are much more important in the former; people's sometimes excessively special relationships are more important in the latter.

Seriously? I am a Department Chair, and regularly talk to Deans, Provosts and Presidents. And I have been on both sides of the divide, beginning my career as a scientist and continuing it as a philosopher. And I say, bullshit. To begin with, administrators don't get excited at the prospect of new scientific discoveries. They get excited at the prospect of the millions of dollars that new research grants will bring into the coffers of the university (see Pinker's own comment above about the deplorable commercialization of universities).

Interesting. Just a few paragraphs above, he wrote that no one is denying science has made wonderful contributions to the well-being of the human society. Now we learn that no administrators actually care about it! Needless to say, this comment that "no one cares" is nothing else than the Liberian's attempt to spit on science.

Moreover, in the real world, scientific advances are often – although surely not perfectly – correlated with the financial gain. There is nothing wrong about it. On the contrary, the world would be a better place if the correlation were tighter than it is. If the administrators get excited about the gain, it's a proxy of their being excited about the science itself. If they don't understand the inner workings of the science, the material benefits that the science brings is their way of seeing a glimpse of the science's power.

Second, I certainly don't go to administrators to plead for respect and tradition. I go to point out that universities are supposed to create the next generation of citizens, voters, and critical thinkers, not just cheap and flexible labor for big corporations.

It's amazing that a chap may openly say that he thinks that his mission is to hurt big corporations and a university in the New York state may appoint this asshole a department chair of a philosophy department. What's your problem with big corporations, you communist shithead? They have surely done better things to improve the life of the mankind than assholes like you. Where's your facility where I can buy a cheeseburger, tank gasoline, or where is your new operating system or online payment system?

I go to remind them that the humanities are crucial for the understanding of vital social debates about the nature of our democratic system, the rights of various groups of people, the concept and implementation of justice, and so forth. And I also go to remind them that philosophy students consistently score higher than pretty much anyone else on a number of tests that are used as gateways for graduate school, medical school, business school, and law school. So there.

We hear that the humanities are crucial but how do they really influence these topics? Is there any reason to think that the net impact is positive rather than negative? As Pinker pointed out, the humanities-driven progress is mostly about the undoing of some societal changes that were done previously. How do we know whether the current changes are "positive" or "negative"?

Philosophy students' scores are close to the average in the table that is led by physics, mathematics, computer science, economics, and four engineering specializations.

Pinker wraps things up by highlighting some areas where the sciences and the humanities should collaborate, rather than fight. Again, some of these are good suggestions, and if scholars in the humanities reject them then they are science-phobic to their own detriment. Other of the suggestions, frankly, leave me quite cold, and again bring to mind the scientistic attitude of wanting to get science's nose sniffing everywhere, regardless of the utility of doing so.

I would guess that he likes the suggestions that would want something from his enemies in the humanities but hates the suggestions that want himself and his friends to do something. But in both cases, the humanities chaps have no arguments to reject science.

The purpose of science isn't just "utility"; the purpose of science is finding the truth. And indeed, "sniffing" is needed for that. If someone wants to invite the scientific method only if he sees the utility, then he is a utilitarian, not a friend of science. If someone actually likes science, he invites science for the intrinsic purpose of science that doesn't need to be "reduced" to any other purpose – for learning the truth. The Liberian is obviously not a friend of science in this sense.

Yes, quantitative methods can (and should) be used by historians, though this will always likely be complementary to, rather than substitutive of, classical historical methods. And yes, the fruitful collaboration between philosophers of mind and cognitive scientists is a shining example of how to bridge the divide between the two cultures. But no, quantitative analyses of Jane Austin novels interpreted in evolutionary psychological key are frankly ridiculous (I've seen it done), and while clearly the study of the physiology of visual or auditory perception are fascinating fields in their own right, they are far less useful to my enjoyment of a Picasso or a Beethoven sonata than knowledge of the history of art or of music.

Quantitative analyses of Jane Austin novels depend on Jane Austin novels and this "not quite essential" topic automatically restricts the possible value of the quantitative research, too. (I am taking no stance on the quantitative Jane Austin novel analyses; they may be silly, they may be interesting, I am not familiar with them.) However, there are many other topics that are more important than Jane Austin novels for which the quantitative analyses are both sensible and valuable.

Pinker really wasted a good chance here. He has the intellectual stature and public visibility to nudge the debate forward in a positive direction. Instead of embracing scientism as a positive label, he should have acknowledged that some criticism of science is well founded and sorely needed.

Does the Liberian have any evidence that Pinker should have done so? Pinker clearly shouldn't have defended this opinion because he doesn't believe it and scholars shouldn't defend opinions they believe to be incorrect. And I don't believe these things, either. The Liberian and people in the humanities are probably personifying science in some way. Because no people they know are "perfect", they believe that science can't be "perfect", either. But the reality just doesn't work in this way. When science is really what it should be, it is perfect so any criticism of the scientific method as a principle is bound to be a symptom of the critic's idiocy. Pinker isn't defending any particular research project or claim about science that may be right, wrong, serious, or silly. He is defending the scientific method as such and he can't be wrong about that.

Instead of telling us again platitudes about the benefits of science (while ignoring its darker side) and chastising the humanities for not embracing it whole heartedly, he could have presented a nuanced examination of where science really is useful to the humanities and where the latter are useful to the sciences - not to mention those several areas where the two can safely ignore each other in pursuit of different goals. Oh well, next time, perhaps.

Sorry but the humanities aren't useful in science. It's a defining feature of science that it is independent of all social conventions and human idiosyncrasies and the humanities are pretty much only about them, to the extent that they're pure humanities. It's fun for a scientist to be a human and a part of a culture, an individual influenced by many things, perhaps even when it comes to science, but the more we talk about the human culture-dependent influences, the less scientific the chatter is.

Pinker's goal wasn't to rate hundreds of particular scholarly attempts to cross the border, so to say. His goal was much more general or grander, if you wish. It was about the fallacies that pretty much all criticisms of science as a principle share. Too bad that the Liberian – a representative of the far left in the anti-science coalition with some religious fundamentalists – hasn't understood the point or he considers it a heresy. I am afraid that next time, he won't be any better.

Read More