K

Kelvin McQueen

Matthew Leifer I'm not entirely sure that the difference between the 5-sigma standard of discovery in particle physics and the 1- or 2-sigma standard of discovery in, say, biology is really an example of what Kuhn had in mind. My understanding of Kuhn was that the very methodology of science was subjective. A difference in accuracy standards isn't really a difference in methodology.

Having said that, I am no fan of Kuhn and I think that denying some kind of universal objective "truth" is a dangerous thing. While there are numerous philosophical examples one could give, the practical example I am particularly fond of using is the following. I teach a course on climate change that is open to scientists and non-scientists alike. I often get (in fact I deliberately target) students who don't believe it's real or, if it is, that's it's not a man-made problem. The entire point of the class is to convince them otherwise by working through the science. If Kuhn is right then science is nothing more than another form of propaganda and its entire edifice (not to mention all that accompanies it -- vaccines, technology, etc.) falls apart.

This is a really interesting idea though it seems to me that it's something we ought to be doing anyway. I mean, isn't that the way science is supposed to work (though admittedly often doesn't)? Nevertheless, to actually see a falsification tree written out for a given theory or set of phenomena might be useful as it might help to hone the conversation down a bit.

An insightful essay, but I see two potential problems. First, falsificationism is hardly the dominant philosophy of science these days. The methods would have to be adapted, and probably made more complicated, to handle something like Bayesian epistemology. Further, if you even slightly agree with critics like Kuhn and Feyerabend, the concept of a global "scientific method" that covers all of science is a myth. Instead, we have local standards of evidence and reasoning that are appropriate for different fields, e.g. consider the 5-sigma standard of discovery used in particle physics which would be considered ridiculously high in most other areas of science, or whether complicated numerical models are considered a source of reliable predictions. Because of these, the question of whether an experiment even falsifies T together with all the possible A's may be ambiguous.

Second, I bet that identifying auxiliary assumptions that everyone can agree upon would be extremely hard in practice. People would disagree about whether a given assumption even is an assumption that is needed or not. This is because scientists are trained in different subsets of science and have different tacit knowledge and beliefs, which can make it extremely difficult to acknowledge that the assumptions you have made actually are assumptions. I imagine being in a room with Lev Vaidman and Tim Maudlin trying to agree upon the structure of a falsification tree for the many-worlds interpretation. If you imagine that would be fruitful then you are much more optimistic than me.

I admit that other adversarial collaborations could work much better than the example I just gave, but even so, if a falsification tree is developed by congenial adversaries, but there are influential people in the field who disagree with its structure, then it will fail to do its job of helping to objectively rule out theories. The dissidents would just write papers criticizing the structure of the falsification tree, and they would probably throw in some dismissive remarks about the Popperazi as well. And my pessimistic opinion is that this will almost always happen in practice.

I gave this Essay a high rating because I find it as being original and provocative. The idea of using falsification trees in the context of adversarial collaborations is intriguing.
I think that it is necessary that the falsification trees approach uses the scientific method as a "North Star" even if the defenders of a particular theory are unlikely (because of their ego and to safeguard their power, fame and economic interests) to accept attempts to falsify itself, even if the latter were to be rigorous.
I await your comments and your vote on my Essay.

I consider your falsification tree a vital argument perhaps inspired by the 1982 falsification argument regarding tropical trees. It is similar to my argument in "Global Externalities and a new science" regarding corporate interests falsification attempts in the name of profit, for example, tobacco interests and smoking and fossil fuel interests and their manipulation of science and truth. With their money and their power they have turned logic into a pretzel and many have been hoodwinked for years and global citizens and our planet endangered. So your reasonable approach is much needed. You deal nicely in techniques and solutions regarding theory selection, testing assumptions and other tricks that can be used in science. I especially liked your MWI discussion. "How could science be different" has been vitally served with your essay. On to rating in the short time we have.

This is what Chat GPT had to say about your essay. And I agree with it

Chat GPT 4
The essay titled "A tool for helping science find the optimal path toward the truth: falsification trees" discusses the logic of falsification in scientific theories and proposes a novel approach called "falsification trees" to aid in theory evaluation. The essay begins by highlighting the inconclusiveness of theory falsification and how it opens up multiple pathways for scientific inquiry. It then introduces the concept of adversarial collaboration, where advocates of competing theories try to falsify each other's theories. The idea of a falsification tree emerges as a way to explore multiple scientific pathways simultaneously to identify the optimal path.

The essay provides a detailed explanation of the logic of falsification, emphasizing the need to consider auxiliary assumptions when assessing whether evidence falsifies a theory. It draws on historical examples, such as the Copernican revolution, to illustrate how theory selection based on falsification can be influenced by subjective and non-objective factors. The essay argues against a social constructivist view of science and proposes adversarial collaborations and falsification trees as a way forward.

Overall, the essay presents a clear and well-structured argument, explaining complex concepts in a coherent manner. It effectively combines theoretical discussion with historical examples to support its points. The proposal of falsification trees as a tool for scientific inquiry is intriguing and warrants further exploration. The essay provides a good foundation for understanding the concept and its potential implications.

Rating: 4 out of 5 stars
Chat GPT 4

Kelvin McQueen I have just finished reading your essay. I'm sold, the falsification tree is a promising scientific methodology. Infact I would like to add and ask your opinion. I've been thinking a lot about how AI will be used in science to evaluate theories and papers. But rather than merely asking AI for its subjective opinions, if would be better if we programmed the AI to follow a rigorous methodology. Your falsification tree or some variation of it might be just the thing. Whats your initial impression of this idea?

By the way, I prompted Chat GPT to review and rate your essay. You'll like what it had to share. I'll post after this comment.

Having read your essay, I wonder if I could persuade you to employ your methodology on my theory? If you've read my essay then you'll know the case I make for a Darwinian Physics and cosmology. Will you be willing to process my ideas with your methodology? At least a tentative sample?

For example "Cellular Biology and atomic physics are two different systems that possess the same structural theme and behavioral themes. ie, systems of units comprised of a nucleus shrouded within a shell, with the capacity to bond with each other and build bodies. My hypothesis explores the possibility that this is an example of co-vergent Darwinian evolution, where two diverse systems evolve similar themes because they face similar circumstances and evolve similar themes in response. Atoms are extraordinarily special little building blocks, as are cells, and it would be nice if science could provide an explanation for atomic specialties other than chance creation. And we know of a powerful organizational principle which has demonstrated its propensity to generate a system of the same general theme. This is good reasoning, however like you pointed out in your essay, people are married to their beliefs such as planetary epicycles, and in modern times people wont consider anything that contradicts big bang theory. My essay presents some of my reasoning, and they are good reasons. But people critique these very reasonable and valid arguments and ideas as if they didnt suspend their disbelief for even a moment in time, which is the appropriate thing to do. They never honored the points raised with the question "what if it were so". As I said, no suspension of disbelief. Tunnel vision.

Anyway, given the theme of your essay, I cannot help but relate to the issues you seek to overcome. I would love for people to employ a method that aided objectivity.
Kind regards
Swan

Kelvin McQueen Your essay was an enjoyable read. It was very well-written and exceptionally well-argued. I really like the concept of falsification trees. This concept is new to me, possibly it is created by you, but once I have learnt about them, they seem totally obvious and inevitable. I really like your examples as well. I have given a rating to your essay based on these comments.

My essay is on a somewhat different subject -- that of funding in research and how to make it more efficient and equitable. It is called "Efficient funding produces better science". Perhaps you would like to read and rate it?

Kelvin McQueen Hi CoralBear. I'll be reading and reviewing your essay next. So you can bank on another rating from me, so you will need 3 others to qualify for the next stage in the contest. Not long to go so you might want to engage with some other contestants.
Swan

quote
One aspect that might be a challenging obstacle to overcome is that this would require the scientific community whole-heartedly embrace null results. Franklin et al point out that:
Modern science’s professional culture prizes positive results, and offers
relatively few rewards to those who fail to find statistically significant relationships in their data. It also esteems apparently groundbreaking results far more
than attempts to replicate earlier research. PhDs, grant funding, publications,
promotions, lateral moves to more prestigious universities, professional
esteem, public attention—they all depend upon positive results that seem to
reveal something new. A scientist who tries to build his career on checking old
findings or publishing negative results isn’t likely to get very far.
end of quote
I beg to differ. Here we go
Case in point, the Cosmological constant problem
By Quantum field theory, its 10¹²⁰ times bigger than its observed
EXPERIMENTAL results
We DO see the Cosmological constant brought up all the time. I.e. see Sean Carrol, in his
Caltech lectures

It is a myth that there are NO NULL results. Certain categories of problems, are celebrated as now for the time being allegedly UNSOLVABLE. which is a mating call for people to try to do them. I as an example tried it too, and still do.

Secondly the Riemann hypothesis, i.e. this one
quote
Has someone solved the Riemann hypothesis?
The Riemann hypothesis will probably remain at the top of mathematicians' wish lists for years to come. Despite its importance, no attempts so far have made much progress. Nov 11, 2022
end of quote
The goofs galore as to both of these, are widely celebrated and HARD problems, get people to put solutions in
print which get debated.

Even the failures are interesting and instructive: Very instructive

We can look at the idea of "null results" as maybe tied into this one
quote
What does Gödel's incompleteness theorem say?
Can you solve it? Gödel's incompleteness theorem ...
In 1931, the Austrian logician Kurt Gödel published his incompleteness theorem, a result widely considered one of the greatest intellectual achievements of modern times. The theorem states that in any reasonable mathematical system there will always be true statements that cannot be proved.
end of quote

So there you go.
HINT null results occur all the time. And the INTERSTING failures may at times even lead to great papers.

I have seen that more than once

Hi, I enjoyed your essay, and I find it a useful tool to think about specific problems. So thanks for that! I do fear, however, that the recipe “Let's get more systematic in the construction and exploration of the tree” may hit the wall of exponential bifurcations. I wonder, however, whether some progress can be made by analysing the complexity of the trees derived from the different alternatives. In the end, I somehow feel entitled to reject a theory because it has many more branches than its negation. In a way, it is like saying: too many assumptions need to be ruled out for the theory to hold. If the alternative has much fewer branches, it looks less conspiratorial. Or doesn't it? Of course, for this to be valid, one should use a computational standard, so that all assumptions share the same complexity. Only with such a standard does it make sense to count how many assumptions are needed to support each alternative.

Indeed! And that has a parallel in the US/Commonwealth legal systems where an adversarial approach is followed to scrutinize evidence and allegations, and in that context it works really well. The adversarial approach to justice was one of the greatest innovations in jurisprudence, and one can see the merits of applying a similar approach in the sciences.

Had another read through. It looks like you're describing a scientific dialectic? One aspect that might be a challenging obstacle to overcome is that this would require the scientific community whole-heartedly embrace null results. Franklin et al point out that:
Modern science’s professional culture prizes positive results, and offers
relatively few rewards to those who fail to find statistically significant relationships in their data. It also esteems apparently groundbreaking results far more
than attempts to replicate earlier research. PhDs, grant funding, publications,
promotions, lateral moves to more prestigious universities, professional
esteem, public attention—they all depend upon positive results that seem to
reveal something new. A scientist who tries to build his career on checking old
findings or publishing negative results isn’t likely to get very far.

A last observation: this "tree" might work well in a field like physics, where replicability is not the foremost challenge. However, it may have significant limits in psychology, social sciences and perhaps to some extent even neuroscience. Because it will depend very heavily on null results, replicability is critical, but these same fields present manifold challenges in that regard.

I hope I have not misunderstood anything! Your paper is great food for thought and I think such a schema has the potential to play a crucial role in a reimagined scientific project that is focused on eliminating bias, unexamined assumptions, and non-empirical influences.

Thanks, this is a piece of writing that warrants spending a bit more time considering! Clearly written. I definitely want to re-read it a couple of times before commenting.