Open Peer Review to Save the World by Philip Gibbs

Israel, there are a lot of people who say the same things as you but it is because they have failed to see how the now paradigm for publications works. Anyone can publish now, on a blog or an e-book or whatever. It is not possible to filter out low quality material as journals used to do in the past. I see this as a good thing because in the past a lot of good papers were delayed or hidden by the old system. It does not do harm that bad papers can be read by the public. The harm comes when people wrongly judge a good paper to be bad or a bad paper to be good. That is the business for peer-review to sort out.

viXra does not try to build a reputation for quality. It never has and it never will. This is written on the web site in several places and I have lost count of the number of times I have said it in response to this kind of criticism. You cannot judge any paper on viXra merely by the fact that that is where it is, because viXra is open to anything (except documents that are off-topic and where legal issues intervene) People are gradually starting to understand this and the quantity of papers we receive is rising at about 40% per year. I do not monitor quality but my general impression is that the ratio of useful science to junk on viXra is also increasing. People still sometimes try to mock us by pointing to low quality papers they find there but that is because they are behind the wave and have not yet got the idea of how to surf it.

viXra works on the principle that publication is completely separate from peer-review. The traditional system says otherwise but that is the old dogma and the new publication paradigm usurps it. Some people will never get the new way but more are waking up to it. viXra is just one small part of the change that is happening. The bigger picture is open peer-review which is now following on the tail of open access publication.

It is difficult to gauge how many people are coming over to the new concept but there are signs that lots of people are. Your old idea that people need to have qualifications to have a say is a dying one. The UK government has offered a £10 million prize to anyone who can make progress on a problem that threatens humanity (climate change, resistant bugs etc) They have asked the public to decide what issue to tackle and are encouraging anyone to compete for the prize, whether academic, corporation or just independent scientist/inventor, it does not matter. Some people say that this is the wrong approach but they are doing it because other public prizes have already worked and they want to see how far the idea can be taken.

Of course expertise and qualifications will always be important for research but you also have to count the paths that do not follow the classical route, as in Douglas Singleton's path integral metaphor.

Some new experiments in publication do try to restrict their input to academics. I think those are the systems that will ultimately fail, not the open repositories like viXra. Philica is already sinking because it tried to set a minimum quality standard for submission and failed. I think arXiv will ultimately find that its filter is its biggest limitation. Microsoft's Academic search also failed miserably because they restricted its scope too much. Google scholar does better because they accept papers from almost anywhere. They bowed to pressure from academics to filter out viXra but as our scope grows they will either have to change that or they will suffer for the omission. Figshare has no filter and is doing very well. viXra is boomimg despite actively setting itself up as the place for arXiv rejects and encouraging anyone to submit. A filter is not a prerequisite for success. Journals used to be open to anyone but now they are quietly starting to filter out submissions from academic outsiders without even reading the papers. They are part of the old paradigm and if they cant find a new business model based on the new one they will die. Open access, Open publication and Open peer review are the future.

I wish you good luck in the contest too.

Philip,

Thank you for a very interesting essay. I found your list of biases particularly to the point, because I agree with you that finding ways to minimize the effect of biases (by improving peer review and by other means) is crucial if we want humanity to optimally steer the future. Being aware of biases and actively fighting them is certainly one of the main goals that education should aim at, especially in the futurocentric perspective that I propose in my essay.

Good luck in the contest!

Marc

Dear Phillip

It is said that nowadays we are in the age of information just because by clicking we have access to whatever we like, good or bad. The fact that we have access won't change the perception of quality in research. There is a list of more than 200 new open-access journals in physics with a terrible reputation. People who seek quality in research will go to places were good quality is published, the others will be ignored, simply because there is too much information that one cannot handle it all and visit all websites. I'm not saying that all publications in vixra are not good, I'm just saying that vixra does not guarantee a minimum of quality. So, there is no big difference between vixra and any other blog find in the internet.

Please take a look at this manuscript: http://vixra.org/pdf/1405.0315v1.pdf. Look at the graphs, presentation, etc. Perhaps this manuscript has something interesting to say but the quality speaks for itself. This is not science. A professional will never publish a paper like this one. When arxiv started to receive this kind of manuscripts, its quality and reputation started to reduce, that's why arxiv implemented filters. Arxiv maintains good quality standards as a repository. This is the great difference between vixra and arxiv.

You: Your old idea that people need to have qualifications to have a say is a dying one.

I don't think so. One cannot expect that a layman who doesn't know basic calculus solve a problem that requires knowledge in differential geometry or higher mathematics. I'm afraid your example does not apply to physics or any other science.

You: The UK government... ...it does not matter.

I'm sure a layperson will not win the prize.

You: They are part of the old paradigm and if they cant find a new business model based on the new one they will die. Open access, Open publication and Open peer review are the future.

I think, there is no "new one" at all. Open access is not well established yet and I don't think it will. It has shown to be worse than traditional publication and it is jeopardizing science reputation by publishing bad quality science. Lets see if you are right in the following years!

Best Regards

Israel

Dear Tihamer,

The group Nicolas Bourbaki claimed having made set theory the basis of all mathematics; and they were accepted in textbooks and by teachers of mathematics.

Claude Shannon has been known and respected for information theory. However, his view on past and future as well as Alfred Nobel's view on mathematics were at odds with current tenets. See my previous essay too.

When I wrote outsider, I didn't mean the opposite of insider but someone outside of the mainstream.

What about your three suggested crucial technologies, I think your optimistic approach is better than pessimism. You might just be a bit too optimistic. Anyway, we both will perhaps agree on that technology is not our fate, not our destiny but rather a challenge to everybody everywhere: Let's contribute to the preconditions for intelligent and satisfying life in peace as intended by Alfred Nobel!

Regards,

Eckard

Dear Phillip,

You discuss the topic of open peer review thoroughly, and while important, I do not think it is enough. This is because many policy makers suffer from one or more of your listed (and other) biases, and they make the decisions that steer humanity, for better or worse. Unless you address their biases, merely creating a pool of 'objectively true' information is of limited use. Consider the matter of climate change.

I suggest three further biases to be important: Attention bias: There is just so much data to attend to that critical information is often overlooked. We tend to attend to things we attend to. Deciding what to attend to, deciding what is important, is often the most difficult decision. Policy makers often attend to the concerns of their supporters, rather than the objective demands of reality, yet which is more important, and critical?

This suggests another bias: Investment bias: You tend to believe what you are paid to believe.

And a third is locality bias: You tend to believe what is near and immediate is more important than what is distant in space and/or time. This bias must be overcome if we are to develop truly global solutions.

Hmm: Checking the WEB shows a more complete list of biases to be quite long: http://en.wikipedia.org/wiki/List_of_biases_in_judgment_and_decision_making

but this merely emphasizes your main point.

In a democracy, we are supposedly all peers, but a democratic system of review has its own failings. Some sort of filtration and transformation of data and information is necessary. Imperfections in the evaluation system should be sought out and corrected, and complaints addressed. And we should all do our part to help.

A very readable and coherent essay, though.

Good luck in the contest.

Sincerely,

Charles

Hi Philip,

I just wanted to reach out and say thank you for the consideration you put into your work. I, too, am an advocate for transparency in the sciences (whether institutional or otherwise), as anything else is obstructive to the sort of progress humanity may need, in order to evade the many existential threats you were attentive enough to mention. Thank you for that, and for an otherwise very well-written, and well-thought-out discussion (both in your essay and throughout the comments that followed it).

Wishing you well in the competition, and hoping to hear from you soon.

Best regards.

=)

Dear Philip,

I've commented on your fine essay above, but as you were favorably inclined to Douglas Singleton's application of a physics metaphor to the problem, I'd like to again invite you to read my essay, which also applies physics ideas to the topic.

Congratulations for continuing to stay near the top of the rankings.

Best,

Edwin Eugene Klingman

Hi Philip,

Thanks for your essay. I agree that bias is a persistent and important problem, and, as an academic author myself, have seen how it plays out.

I think you might find some connections with my essay on computationally intelligent personal dialogic agents. I developed a prototype with a grant from the National Science Foundation as part of research on approaches to team training. One of the goals of the dialogic system is to reduce bias, and I have shown some positive results in terms of group outcomes in problem solving.

I'd appreciate a rating, if you can do that, since I am a bit short on ratings.

Thanks,

Ray

Hello Philip, May I post a short, but sincere critique of your essay? I'd ask you to return the favour. Here's my policy on that. - Mike

Philip,

Having had rating problems with my Firefox browser and with some 5 days remaining, I am revisiting essays I've read to see if rated. I find that I rated yours on 4/30.

Glad to see your essay is doing well. Given time, I would like to see your thoughts on my essay: http://www.fqxi.org/community/forum/topic/2008

Jim

Hello Dr. Gibbs,

Thank you for your essay proposal based on open peer review. I concur with you that the human mind is fraught with biases which cloud our judgement. While the wisdom of the crowd can sometimes be shown to be equivalent in predictive power to an expert, I am not sure how well it can be applied to the scientific endeavor. Even if we had a broader scientific consensus than we do currently, I'm not sure we will ever achieve a complete consensus, nor does it imply that a correct course of action would be chosen in regards to that consensus. I would like to believe otherwise, but I don't have that much faith in the rationality of my fellow humans.

Regardless of my criticisms, though, I do believe that the academic and scientific endeavors require updates to their operational mechanisms, especially where elimination of bias is concerned and verification and accuracy can be improved. These improvements certainly wouldn't hurt any future I would choose for our species.

Thanks,

Mark

Mark,

I believe open peer review is an important part of business incubation. Much more than just on-subject discussion results, leading to moments of inspiration being produced in other diverse subject areas. Types of inspirations resulting might involve: diverse related products to develop, marketing insights, ethical concerns, synergy with other research ...

Open peer review is very much a part of Staged peer review with business incubation tools.

Staged Peer Review & Business Incubator

But unless something actively and physically is pursued, of what use is this discussion; or any discussion ...

Contributors here have diverse perspectives, and most, obviously thoughtful people. Therefore, everything presented in these essays has the potential to contribute productively toward any overall goal; within staged constraints. A person with a passion to provoke anarchy is most likely going to disrupt product development and make it impossible to produce a product to fund further efforts. However, that same anarchist is an important controls feedback function for philosophical discussions about ethics related to what should be considered about the uses and abuses of the product when developed.

Most business people want to jump onto a path that most directly provides them with the greatest return on investment. The problem is that without negative feedback the initial business resources are used up before sustainable systems can be put in place to establish a stable AND sustainable cash flow. As an example, the lowest risk investment on average is a franchise, but only one in 20 franchises become sustainable. The diverse types of influences on a business start-up takes time and a slow trickling of resources so that unforeseen influences can be characterized and incorporated into the business control systems. But meetings and efforts without useful outcomes is wasted resources and undeveloped opportunities.

The following was a national headline for the James-Rivers Paper Company journal. Related to unproductive meetings without an Action-Item Log and related Action-Item Worksheets to capture and track meeting productivity:

"If you are going to listen, do something" ~Thomas Dunn

Example of Action Item Log

Philip,

I hope the new deadline allows you to get to mine. I do understand if QM turns you off, it would do anybody, but unification is important enough to our future to peek at a logical version. I've condensed the solution in the last few posts of my blog. The experiment (end notes) allowed comprehension by all ages down to 11!

As you're aware I feel both our essays should be a bit higher, but as we also know, the judges take no account of peer score places. I'm interested in all viewpoints in any case and would respect yours.

Best of luck in the final judging anyway.

Peter

Mark,

Vladimir,

I believe open peer review is an important part of business incubation. Much more than just on-subject discussion results, leading to moments of inspiration being produced in other diverse subject areas. Types of inspirations resulting might involve: diverse related products to develop, marketing insights, ethical concerns, synergy with other research ...

Open peer review is very much a part of Staged peer review with business incubation tools.

Staged peer Review & Business Incubator

But unless something actively and physically is pursued, of what use is this discussion; or any discussion ...

Meetings: Pandering waste of time, or productive asset

Contributors here have diverse perspectives, and most, obviously thoughtful people. Therefore, everything presented in these essays has the potential to contribute productively toward any overall goal; within staged constraints. A person with a passion to provoke anarchy is most likely going to disrupt product development and make it impossible to produce a product to fund further efforts. However, that same anarchist is an important controls feedback function for philosophical discussions about ethics related to what should be considered about the uses and abuses of the product when developed.

Most business people want to jump onto a path that most directly provides them with the greatest return on investment. The problem is that without negative feedback the initial business resources are used up before sustainable systems can be put in place to establish a stable AND sustainable cash flow. As an example, the lowest risk investment on average is a franchise, but only one in 20 franchises become sustainable. The diverse types of influences on a business start-up takes time and a slow trickling of resources so that unforeseen influences can be characterized and incorporated into the business control systems. But meetings and efforts without useful outcomes is wasted resources and undeveloped opportunities.

The following was a national headline for the James-Rivers Paper Company journal. Related to unproductive meetings without an Action-Item Log and related Action-Item Worksheets to capture and track meeting productivity:

"If you are going to listen, do something" ~Thomas Dunn

Action Item Log and Action Item Worksheet with summary of utilization

Scroll down to Project Management section

Hi,

I've enjoyed your very interesting essay.

While I agree that current closed-peer review has a lot of issues (my pet peeve is single blind refereeing), I am not sure open peer-review -- at least the way you suggested it to be -- is the solution. You cited BICEP2 "open peer review", and being a participant of the whole social-media dissection of the paper, the whole process was messy and more often than not, all the 9 biases (especially bandwagon and confirmation) reared their ugly heads up. It led to misinformation (e.g. was BICEP2 keck array cross correlation with 100 or 150 Ghz? ). One of the main goal of open peer review -- everybody should have a voice -- led to basically more noise than signal in the ensuing "discussion".

Even worse, one of the ills of "closed expert peer review" -- loud voices of authority from august institutions, instead of being silenced by the noisy (in the signal-to-noise sense) crowd, grew even louder. Now everybody cites the words of Prof XX from Ivy Institution YY, not because she is right, but because she is who she is. Of course, Prof XX had tried to disentangled stuff signal from the noise, and actually got wrong information (because it's so noisy), and ended up amplifying misinformation. Harnessing the wisdom of the masses also harnessed the noise that comes with it.

The BICEP 2 "open peer review" via social media was conducted among basically highly qualified cosmologists -- not the general public. And the BICEP experience at least taught me that "wow that's messy". The whole experience has an extremely useful side-effect -- it provide rapid dissemination of knowledge and data, and personally it forced me to learned a lot of stuff to follow up and understand the discussion. But it provided no proper "review". Personally, even though I gave several technical talks on the BICEP2 results to several groups of colleagues, I provided zero input into the "review" part, i.e. I was not acting as an expert, because frankly I do not have the expertise There are very few experts in this world who has the ability to peer review a technical work like BICEP2, and I am happy to let them peer review it on my behalf.

And then I will use my own judgement to decide whether I believe their work -- because peer review does not automatically imply "correctness". We, as consumers of the whole scientific peer-review process, have the responsibility to use our heads to decide for ourselves and not follow the loudest voices of the authorities. Or the silent voice of the secret peer reviewer.

Good luck with the contest!

Best,

Eugene

Dear Philip,

With respect to your question, "Could a deadly virus sweep round the world in days wiping out billions of people?", I would like share some of my perspectives:

Virus always depends on the genome of living cells for their replications and causal for Horizontal gene transfer that effects the propagation of tagged radioactive isotopes through gene transfer.

As per ECSU paradigm, intrinsic ionising radiation by radioactive isotopes is causal for the modifications in the genomes.

This implies that the increase of radioactive isotopes in the food web by Humanity is causal for the massive alterations in the genomes of the biosphere, causing degradation and collapse of the bio-systems; if not regulated.

Thus 'Open Peer Review' is to be with more on healthcare rather than with other disciplines of science, that is already existing and mounding; as healthcare is the core of Humanity.

With best wishes,

Jayakar

This is posted by me only, regretting for the technical error.

Jayakar