Open Peer Review to Save the World by Philip Gibbs

Hi Philip,

Thanks for your essay. I agree that bias is a persistent and important problem, and, as an academic author myself, have seen how it plays out.

I think you might find some connections with my essay on computationally intelligent personal dialogic agents. I developed a prototype with a grant from the National Science Foundation as part of research on approaches to team training. One of the goals of the dialogic system is to reduce bias, and I have shown some positive results in terms of group outcomes in problem solving.

I'd appreciate a rating, if you can do that, since I am a bit short on ratings.

Thanks,

Ray

Dear Peter,

I'm doubtful that one could effectively make an author anonymous to the referees: I'm afraid that in the vast majority of the cases the identity of most authors would be completely apparent to the referees from what they write. However, young or first-time authors would be guaranteed fair treatment by such a system, and also it would guarantee that, if the editors decides to reject your paper before sending it through peer-review, they had to read the paper (or at least the abstract!) and couldn't reject a paper just on the basis of the author's reputation. What do you think?

best,

Flavio

Hello Philip, May I post a short, but sincere critique of your essay? I'd ask you to return the favour. Here's my policy on that. - Mike

Philip,

Having had rating problems with my Firefox browser and with some 5 days remaining, I am revisiting essays I've read to see if rated. I find that I rated yours on 4/30.

Glad to see your essay is doing well. Given time, I would like to see your thoughts on my essay: http://www.fqxi.org/community/forum/topic/2008

Jim

Hello Dr. Gibbs,

Thank you for your essay proposal based on open peer review. I concur with you that the human mind is fraught with biases which cloud our judgement. While the wisdom of the crowd can sometimes be shown to be equivalent in predictive power to an expert, I am not sure how well it can be applied to the scientific endeavor. Even if we had a broader scientific consensus than we do currently, I'm not sure we will ever achieve a complete consensus, nor does it imply that a correct course of action would be chosen in regards to that consensus. I would like to believe otherwise, but I don't have that much faith in the rationality of my fellow humans.

Regardless of my criticisms, though, I do believe that the academic and scientific endeavors require updates to their operational mechanisms, especially where elimination of bias is concerned and verification and accuracy can be improved. These improvements certainly wouldn't hurt any future I would choose for our species.

Thanks,

Mark

Mark,

I believe open peer review is an important part of business incubation. Much more than just on-subject discussion results, leading to moments of inspiration being produced in other diverse subject areas. Types of inspirations resulting might involve: diverse related products to develop, marketing insights, ethical concerns, synergy with other research ...

Open peer review is very much a part of Staged peer review with business incubation tools.

Staged Peer Review & Business Incubator

But unless something actively and physically is pursued, of what use is this discussion; or any discussion ...

Contributors here have diverse perspectives, and most, obviously thoughtful people. Therefore, everything presented in these essays has the potential to contribute productively toward any overall goal; within staged constraints. A person with a passion to provoke anarchy is most likely going to disrupt product development and make it impossible to produce a product to fund further efforts. However, that same anarchist is an important controls feedback function for philosophical discussions about ethics related to what should be considered about the uses and abuses of the product when developed.

Most business people want to jump onto a path that most directly provides them with the greatest return on investment. The problem is that without negative feedback the initial business resources are used up before sustainable systems can be put in place to establish a stable AND sustainable cash flow. As an example, the lowest risk investment on average is a franchise, but only one in 20 franchises become sustainable. The diverse types of influences on a business start-up takes time and a slow trickling of resources so that unforeseen influences can be characterized and incorporated into the business control systems. But meetings and efforts without useful outcomes is wasted resources and undeveloped opportunities.

The following was a national headline for the James-Rivers Paper Company journal. Related to unproductive meetings without an Action-Item Log and related Action-Item Worksheets to capture and track meeting productivity:

"If you are going to listen, do something" ~Thomas Dunn

Example of Action Item Log

Philip,

I hope the new deadline allows you to get to mine. I do understand if QM turns you off, it would do anybody, but unification is important enough to our future to peek at a logical version. I've condensed the solution in the last few posts of my blog. The experiment (end notes) allowed comprehension by all ages down to 11!

As you're aware I feel both our essays should be a bit higher, but as we also know, the judges take no account of peer score places. I'm interested in all viewpoints in any case and would respect yours.

Best of luck in the final judging anyway.

Peter

Mark,

Vladimir,

I believe open peer review is an important part of business incubation. Much more than just on-subject discussion results, leading to moments of inspiration being produced in other diverse subject areas. Types of inspirations resulting might involve: diverse related products to develop, marketing insights, ethical concerns, synergy with other research ...

Open peer review is very much a part of Staged peer review with business incubation tools.

Staged peer Review & Business Incubator

But unless something actively and physically is pursued, of what use is this discussion; or any discussion ...

Meetings: Pandering waste of time, or productive asset

Contributors here have diverse perspectives, and most, obviously thoughtful people. Therefore, everything presented in these essays has the potential to contribute productively toward any overall goal; within staged constraints. A person with a passion to provoke anarchy is most likely going to disrupt product development and make it impossible to produce a product to fund further efforts. However, that same anarchist is an important controls feedback function for philosophical discussions about ethics related to what should be considered about the uses and abuses of the product when developed.

Most business people want to jump onto a path that most directly provides them with the greatest return on investment. The problem is that without negative feedback the initial business resources are used up before sustainable systems can be put in place to establish a stable AND sustainable cash flow. As an example, the lowest risk investment on average is a franchise, but only one in 20 franchises become sustainable. The diverse types of influences on a business start-up takes time and a slow trickling of resources so that unforeseen influences can be characterized and incorporated into the business control systems. But meetings and efforts without useful outcomes is wasted resources and undeveloped opportunities.

The following was a national headline for the James-Rivers Paper Company journal. Related to unproductive meetings without an Action-Item Log and related Action-Item Worksheets to capture and track meeting productivity:

"If you are going to listen, do something" ~Thomas Dunn

Action Item Log and Action Item Worksheet with summary of utilization

Scroll down to Project Management section

Hi,

I've enjoyed your very interesting essay.

While I agree that current closed-peer review has a lot of issues (my pet peeve is single blind refereeing), I am not sure open peer-review -- at least the way you suggested it to be -- is the solution. You cited BICEP2 "open peer review", and being a participant of the whole social-media dissection of the paper, the whole process was messy and more often than not, all the 9 biases (especially bandwagon and confirmation) reared their ugly heads up. It led to misinformation (e.g. was BICEP2 keck array cross correlation with 100 or 150 Ghz? ). One of the main goal of open peer review -- everybody should have a voice -- led to basically more noise than signal in the ensuing "discussion".

Even worse, one of the ills of "closed expert peer review" -- loud voices of authority from august institutions, instead of being silenced by the noisy (in the signal-to-noise sense) crowd, grew even louder. Now everybody cites the words of Prof XX from Ivy Institution YY, not because she is right, but because she is who she is. Of course, Prof XX had tried to disentangled stuff signal from the noise, and actually got wrong information (because it's so noisy), and ended up amplifying misinformation. Harnessing the wisdom of the masses also harnessed the noise that comes with it.

The BICEP 2 "open peer review" via social media was conducted among basically highly qualified cosmologists -- not the general public. And the BICEP experience at least taught me that "wow that's messy". The whole experience has an extremely useful side-effect -- it provide rapid dissemination of knowledge and data, and personally it forced me to learned a lot of stuff to follow up and understand the discussion. But it provided no proper "review". Personally, even though I gave several technical talks on the BICEP2 results to several groups of colleagues, I provided zero input into the "review" part, i.e. I was not acting as an expert, because frankly I do not have the expertise There are very few experts in this world who has the ability to peer review a technical work like BICEP2, and I am happy to let them peer review it on my behalf.

And then I will use my own judgement to decide whether I believe their work -- because peer review does not automatically imply "correctness". We, as consumers of the whole scientific peer-review process, have the responsibility to use our heads to decide for ourselves and not follow the loudest voices of the authorities. Or the silent voice of the secret peer reviewer.

Good luck with the contest!

Best,

Eugene

Dear Philip,

With respect to your question, "Could a deadly virus sweep round the world in days wiping out billions of people?", I would like share some of my perspectives:

Virus always depends on the genome of living cells for their replications and causal for Horizontal gene transfer that effects the propagation of tagged radioactive isotopes through gene transfer.

As per ECSU paradigm, intrinsic ionising radiation by radioactive isotopes is causal for the modifications in the genomes.

This implies that the increase of radioactive isotopes in the food web by Humanity is causal for the massive alterations in the genomes of the biosphere, causing degradation and collapse of the bio-systems; if not regulated.

Thus 'Open Peer Review' is to be with more on healthcare rather than with other disciplines of science, that is already existing and mounding; as healthcare is the core of Humanity.

With best wishes,

Jayakar

Hi Phil,

Congrats for this fantastic Essay which discusses a issue that I think to be fundamental. Here are my comments/questions:

1) The problem of the peer review process is serious and it is directly connected with the issue of freedom in science. This is a controversial issue. Although I work essentially within mainstream physics, I criticize some points of the same mainstream physics and I am all in favour of being open minded about alternatives, but they must be properly formulated and plausible scientific proposals. Freedom in science is a good think, but endorsing crackpot nonsenses is evil instead.

2) The issue that "the ones who decide the truth have self-interests driven by the funding that supports them" is sadly what govern "scientific politics".

3) I think that various "Bias" are consequence of economic interests.

4) I am not sure that Higgs boson has been really detected.

5) I agree with your point on energy conservation in general relativity but I think a clarification is needed. Based on Einstein Equivalence Principle, energy is coordinate dependent in general relativity. But, when we fix the coordinate system, energy conservation works.

6) I think that the importance of citations in determining the ultimate importance of a research work is largely overestimated.

7) The issue that "it is common for good papers to be rejected by top journals" sometimes is extremely ridiculous. I know of papers rejected by a top journal for which, after published by another journal, the authors obtained the Nobel Prize in physics...

8) The issue that "the UK government spent £580 million on the drug Tamiflu to combat flu epidemics, only to learn that the research was flawed and not properly checked" is only one of a big number. In fact, the problem of "scientific politics" becomes dramatic when it concerns medicine because it interacts with people's health. I think that "mainstream medicine" is completely governed by pharmaceutical companies which have interest in nursing people rather than in recovering. This is terrible and shameful.

9) I strongly endorse your 6 basic principles for a new peer-review.

10) I appreciate your viXra website and sometimes I submit my papers to it. On the other hand, in open archives there is the opposite problem to see published crackpot papers together with good ones.

As I previously wrote, I find this Essay fantastic. Thus, I am going to give you an high rate accordingly.

I hope you will find the time to read, comment and rate my Essay (Maybe you remember my pioneering work on this issue was originally rejected by arXiv. I discussed this issue with you 3 years ago. Thus, I originally submitted it in viXra. Arxiv accepted it only after is acceptance by JHEP).

I wish you best luck in the Contest.

Cheers,

Ch.

Dear Phil,

Thanks for clarifying your point concerning viXra. I completely understand your point of view and I also agree with it. On the other hand, I was not attacking vixra (I told you that I appreciate it). My using of the word "problem" could generate misunderstanding. Yes, the mistake was mine. Sorry.

Cheers,

Ch.

Hi Phil,

Am wondering might you still have the chance to read, rate and coment on my perhaps unconventional thesis. I always appreciate your FRANK input.

Regards,

Chidi

The link above didn't work so I repeat hoping: here the correct link

chidi

Phil,

Open peer review is a good idea, but it will not save the world. I know that there has been and will be situations that are unfair, careers sadly ended and work wrongly credited. That is pain on the level of the individual, but good science in the long view of history will win. If we are talking about the future of humanity, science as a slow process is what is important.

My name was one of several authors on a paper in experimental physics written about 20 years ago. That was my only experience with a peer review process.

Now, I teach mostly at community colleges to mostly non-science majors and I see my job as a tour guide to the world of science and not part of the process of science. Teachers are at the tail end of the process of science (the old dusty textbook part). Teachers decide if Tycho Brahe was a rich, nose-less, fool or a careful researcher who followed the best data he had to a conclusion that proved to be wrong with better data.

All the Best,

Jeff

Jeffrey,

In my opinion, teachers train the minds of their students to identify shortcomings in relating information to outcomes, and then efficiently teach themselves the relationships and information they need to know to predictably manipulate their desired outcomes.

All of this is implicit to problem solving.

However, related to Open Peer Review and being a steerable pathway to guide future human outcomes, I strongly believe Open Peer Review is part of that process. This forum of essay review is a form of Open Peer Review. I have generated at least two activities for business generation resulting from the FQXi Open Peer Review process.

Open peer review is very much a part of Staged peer review with business incubation tools.

Staged peer Review & Business Incubator

Open Peer Review is ANY open forum.

The term "Crackpot" in science is a derogatory statement to provoke political influence in dismissing competing pathways for investigation. Mainstream physics is currently at a apparent dead-end related to space and time experimentation. So the crackpot is going to be the future inspiration for new investigations to steer the future away from dead-ends.

The Crackpot is a great source for inspirations in diverse areas of business. Who would buy a mop with a spray bottle attached to it? My wife.

The Crackpot simply has a set of related but not well defined, nor broadly consistent relationships that are important enough for them to stick their neck out for ridicule.

Every scientist I know falls into the above classification of Crackpot because they are all dealing with an incomplete set of information. Scientists however are able to tie their works to useful Applications; except of course theoretical physcists.

In almost every system proposed by scientists, their works must be stated within exactly defined constraints or their outcomes are not repeatable broadly. Very few scientists take the time to identify the broad limiting constraints of their assertions. This makes these mainstream scientists Crackpots.

I reviewed a concept for time characterized as pages in a book. The mathematics was quite impressive, until constraints in relationships related to set theory were considered. The mathematics was disjoint. This does not mean their work may not become important one day. It just means the work is incomplete; a partial perspective and an illusion of being a completely valid perspective.

The are hundreds of thousands of physicists in the world; how many of them contribute toward a final solution? Very few. So aiming a an arrow at open peer review as a target of dismissive condemnation is like saying no one should speak until they are fully knowledgeable in what they are about to say.

Every conversation is Open Peer Review.

By educating people related to the biases that misdirect useful outcomes and relating these biases infused with common sense, is a strong basis for "supporting" broad systems of development; in very diverse subject areas.

However, Corruption derails the effort not to be unethically biased.

Corruption = unethical/illegal allocation of resources and/or opportunities

But unless something actively and physically is pursued, of what use is this discussion; or any discussion ...

Meetings: Pandering waste of time, or productive asset

As a result of yours or anyone's efforts to read essays and write comments, what productively are you and other's "doing" to convert rhetoric into outcomes?

Contributors here have diverse perspectives, and most, obviously thoughtful people. Therefore, everything presented in these essays has the potential to contribute productively toward any overall goal; within staged constraints. A person with a passion to provoke anarchy is most likely going to disrupt product development and make it impossible to produce a product to fund further efforts. However, that same anarchist is an important controls feedback function for philosophical discussions about ethics related to what should be considered about the uses and abuses of the product when developed.

Most business people want to jump onto a path that most directly provides them with the greatest return on investment. The problem is that without negative feedback the initial business resources are used up before sustainable systems can be put in place to establish a stable AND sustainable cash flow. As an example, the lowest risk investment on average is a franchise, but only one in 20 franchises become sustainable. The diverse types of influences on a business start-up takes time and a slow trickling of resources so that unforeseen influences can be characterized and incorporated into the business control systems. But meetings and efforts without useful outcomes is wasted resources and undeveloped opportunities.

The following was a national headline for the James-Rivers Paper Company journal. Related to unproductive meetings without an Action-Item Log and related Action-Item Worksheets to capture and track meeting productivity:

"If you are going to listen, do something" ~Thomas Dunn

Action Item Log and Action Item Worksheet

So with all my rhetoric, what will I "do"?

http://www.ua-kits.com

http://jamesbdunn2.blogspot.com

I'm currently involved with educational grants and am using the principles I presented.

Everyone commonly dismisses other's works as a means of covering their own lack of self-esteem in related areas. This isn't appropriate for the benefit of society. Use all information for the "greatest" useful benefit without harmful contradictions.

What will be your next "do"? Then tell us about it so we can learn from your efforts.

Respect is based in the communication process of sharing useful information.

Disdain is based in sharing information that is not useful, or that is destructive.

Self-esteem is the additive accumulation of both.

I am defending Phil's essay because it CONTRIBUTES toward a total solution for the Steering of Humanity more strongly action-oriented toward useful outcomes than many of the other essays presented.

I played around with Excel last night and came up with a way to predict the contest winner. Basically, by downloading all the data pertinent to this contest such as the title of the essay, how many posts, the community rating, the public rating, how many community ratings and how many public ratings, and one more column for a combination of all the ratings and how the essay judges are likely to weight all the columns with respect to eachother, it spits out an answer.

With all those numbers, I sorted on each column and changed the color of the top 10 essays in each column. Then when it was all done I just looked for the "most colorful essay".

And the winner (will likely be)...

Open Peer Review to Save the World by Philip Gibbs

#2: Recognizing the Value of Play by Jonathan J. Dickau

#3: Bohr-like model for black holes: the route for quantum gravity by Christian Corda

#3 wins the slot because the contest judges will want to be science-minded. That's why Corda will likely win out over the Honorable Mention

How to save the world by Sabine Hossenfelder

because #3 is very science-y and #4 is a bit more of a preachy title without as much of a hint towards what the essay is about.

Well, there's my prediction. It was enjoyable to participate in this contest. By my own criteria, my essay wasn't "colorful" at all. Maybe the judges will score highly on ease of understanding and practicality? Nahh, the guys who are at the top of this list still do very well in such categories.

Good luck to you all.

Kevin O