George, thanks for the comment and questions

I think systems of government are hard to change for the simple reason that the people who get into power like the way the system worked, because it got them into power.

What we can hope to do in a democracy is influence public opinion which then indirectly influences government. This is something that is beginning to happen on the internet where public debate on important issues is fast and powerful. The need then is to ensure that systems of public debate emphasise logic and rational thinking over more biased illogical ideals. This does not always happen.

I like that when people comment on some news reports people can vote comments up and down. This really shows where opinion lies. I'd like to see a little more complexity in such systems to help the rational side. There is also a danger that some news media bias the results by deleting comments that are against their political views so I'd like to see independent commenting systems that cant be cheated. As an example of what I like try out http://rbutr.com/

Dear Dr. Gibbs,

I enjoyed your essay and agreed with many of the points: peer review as currently practiced could be greatly improved; the academic "game" is rigged against those without academic affiliations or with academic affiliations that are not from R1 universities; the idea of open access is good, but the exact way in which it will be funded is still unclear (this I think was the point of your Elsevier story on page 5 where Elsevier's stock price went up *after* open access since now they were getting paid by the government plus still getting library subscription fees (it was the government that paid right? not the individual authors).

My only question/quibble is in the larger picture will having a better peer review system really have that big an impact on the progress of science, steering humanity to a "good" state, etc.? Aren't there more important factors to focus on? I just say this to play devil's advocate since as a theoretical physicist my main output in science is through what I publish so this is an extremely important point to me. But I am also reminded of a student-athlete at my university who once told me that the best thing to advance the goals of the university would be if there was *more* support for athletics on campus. This kind of statement may seem strange unless you spent some time at an American university which not only are academic centers, but also amateur athletic training institutions.

Anyway I like the idea you mention of having peer review be public. Something like post the paper to some electronic archive and then people would post comments/critics of a given work. You might still need to keep the names of the people leaving critics anonymous since people, even those trained in the scientific method, often fall in love with their own ideas and have a hard time hearing even well intended negative things about a given piece of work. It doesn't help that often the criticism is in a tone that can be unnecessarily negative. Also the more people that give some comment on a piece of work the better. Something some journals have done is to list the editor in charge of the particular article along with the final published version. The idea is this then forces the editor to take some responsibility for the quality of those articles accepted. It give another layer of oversight into the quality of the paper since both the referee(s) and the editor have given the paper the OK and the editor even puts their name with the paper. Of course the reverse does not happen -- no one publishes the rejected articles with the name of the editor that rejected them. So there is some asymmetry on this point.

Finally, you might find interesting a recent study of NIH R01 grants published in

N. Danthi, et al. "Percentile Ranking and Citation Impact of a Large Cohort of NHLBI-Funded Cardiovascular R01 Grants", Circulation Research, 114, 302656 (2014)

this study found that there was virtually no difference in the scientific impact of those grants which were ranked Category 1 (excellent grants according to the grant reviewers and which therefore got funded), Category 2 (very good grants according to the grant reviewers but which just missed the funding mark) and Category 3 (fair or marginal grants as judged by the reviewers which also did not get funded). Grants in all three categories published about the same number of articles and received the same number of citations. This is not necessarily the best measure of scientific worth, but it is the metric by which NIH says it judges the grants.

Anyway and interesting and thought provoking essay.

Best,

Doug

    Philip,

    I overlooked in your reply "the biases of public opinion". Well, public opinion tends to be manipulated. That's why I maintain my objection to your claim "saving the world".

    Doesn't already your exemplary selection of expected catastrophes reflect the manipulated public opinion?

    Does global warming really deserve much more attention than poisoning the environment with radioactive and other waste?

    Can we expect the public opinion to abandon in time the logically untenable slogan "Earth has room and food for all"? While this naive belief might be true for now and the near future but not beyond, it protects the interests of those who are irresponsible and could misuse an open peer-review accordingly because they are strongly interested in their profit.

    I would rather appreciate if you could point to a list of viXra articles that proved at least as valuable as arXive articles. I agree on the necessity of scientific discussion on high level. To me in science, nothing must be taboo. Read my essays. Agreeing with outsiders like Nobel and Shannon rather than Einstein and Bourbaki, I do not even expect much ercy by open peers.

    Eckard

    Phil,

    Excellent job.

    Probably not a snowballs chance in a Bessemer converter of having any effect, but then a billion snowballs is an avalanche so we must start somewhere. To prove I really mean that I had this published in the 'Skeptical intelligencer' a few years ago;

    Subjugation of Skepticism in Science. (I spose I should have logged that into viXra too, whadda'you think?)

    Of course there are twofold problems; that one man's genius is another man's crackpot as all think differently, and we already have 'information overload' with the number of papers that DO get through (I have a growing pile of unread journals building up despite reading 20 papers a week). Overcome those problems and the ruling troglodyte hordes who have it all sewn up may start to worry.

    I note 'Nature' has recently be lauded for heading the other way and insisting all references are from only the landed gentry of peers. Only marriages to sisters are allowed! You may be right, the problem may now be starting to be one of mankind's very survival.

    I've had Vladimirs problems too. Some papers on arXiv, then suddenly a brick wall. Do they not also have any code of ethics? Unifying QM and SR mean nothing if nobody will publish it! (I hope you'll check out my essay).

    I greatly commend your efforts to beat Christian's record as the highest scoring community peer score to be passed over in the judging. Well done and best of luck.

    Peter

      Jonathan, Hi,

      I think the attitude of professional academics to independent researchers is very variable. Many of them are well aware of the way arXiv filters its papers and they regard that as acceptable. happily there are others who are much more open. FQXi would not be so open to outsiders if that was not the case.

      Its good to see you in the contest again with another strong contribution.

      Philip: better peer-review would surely help. I think a key element is expansion of "blind peer review" so reviewers don't know author names. The power of ad-hom prejudice is great. Meanwhile, viXra helps by providing an open forum for challenging ideas.

      Heh, clever coincidance, this was my legitimacy test:

      Important: In order to combat spam, please select the letter in this menu between 'V' and 'X':

        I agree that it would be a useful ideal if peer-review could be anonymous on the part of the author. In the present journal system papers submitted by independent authors are often rejected by editors who know their contact details.

        Perhaps a future open peer-review system could have an option for the authors to remain anonymous until initial reviews have been posted. However, for the most part authors like artists can be recognised by their work even if they dont sign it, so I dont think this ideal can be met in practice.

        The hope is that if peer-review is more open then any prejudices would be more obvious and counterbalanced.

        The letter between V and X is i :-)

        I am not the only one throwing snowballs but the avalanche has not yet begun. The traditional peer-review system is very well protected on all sides, but there are some signs that things will change eventually.

        I dont know what you are referring to about Nature. Is there a link for that?

        Good luck with your essay. I will get to it soon.

        When Wikipedia first started I thought it was hopelessly ambitious and had no chance of working. I was editor of the Physics FAQ so I knew how difficult it is to get people to write good content for free. The line between success and failure on the interwebs is a fine one but if you get the right formula your project can work.

        Wikipedia works because it has the right combination of rules and admin. It is not perfect and has its biases but smart people look at the talk pages as much as the main article if they want a more complete picture. The admin do not always get their way. They tried to delete the viXra article but failed because it fits the rules.

        Many systems for open peer-review have already failed. Look at scirate.com for example. It has been around for ages and is well known but it still gets little traction. One day someone will implement the right system, perhaps by luck. More likely they look at what works for Wikipedia, Stackexchange, Reddit etc, and adapt those features to a full-blown pee-review system. Perhaps the formula that works will look nothing like the way we imagine peer-review to work now and it may reach far beyond just academic poapers.

        Phil,

        An interesting paper here, not only showing a far higher proportion of experiments than we realise are falsely based, but then applauding treating the symptoms not the cause;

        "In 2005 John Ioannidis, an epidemiologist from Stanford University, caused a stir with a paper showing why, as a matter of statistical logic, the idea that only one such paper in 20 gives a false-positive result was hugely optimistic. Instead, he argued, "most published research findings are probably false."

        As he told the quadrennial International Congress on Peer Review and Biomedical Publication, held this September in Chicago, the problem has not gone away."

        http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble">2013 Economist article.](https://

        http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble)

        It also refers to the Nature policy.

        Peter

        Philip,

        I would like to see your essay win this year. I am among the many independent researchers, like yourself, that experienced many years of bias. Your essay explained why for 30 years I either received curt rejections from an editorial staff or worse no response from my many attempts to contribute or even dialog. I spent a very enjoyable day reviewing your copious contributions to viXra. Thank you for starting the website. I put some thought into how to set up a peer review process and started to appreciate the difficulties. My motivation has been developing a consistent view of nature and I am concerned about where our valuable future mental resources will be spent. My grandkids will probably love the challenge of understanding as much as I did but I don't want them exposed to processes that limit their potential. We have a lot of challenges ahead and we need a world that stops fighting and wasting resources over trivial differences. I am struck by statements from authors saying that only a small fraction of people even believe in evolution of other basic science. Your statements that a good peer review process might change the future are right on.

        Philip,

        Wonderful to read your views on the need for Open Peer Review. I have felt this way for a very long time.

        We are on the same page.

        While you are opening up the world to grow its knowledge, I am working to make this knowledge available to the global public.

        I eagerly await your reaction to my essay.

        Thank you for your essay

        -Ajay

        Philip,

        "Humanity faces many dangers from climate change and wars to asteroid impacts that could harm our future. Often logical reasoning does not seem to play a strong part in discussions on such subjects and even peer-review is flawed. I contend that the solution is a better system of open peer-review."

        Certainly "open peer-review" is a step in the right direction in giving hearings to outside-the-moneyed-box ideas, but currently the problem seems to be money, power and access controls who is heard and who is not heard, and this is becoming global, not just a US characteristic. Without celebrity, power and access and without support by a corporate media, without representation by leaders, it is very difficult to be heard.

        Climate change and wars too often fulfill agendas of the most powerful in our world. Their focus is not asteroid dangers, though open discussion in an open-peer-review forum would help. We do need to have the best ideas heard, but they are often drowned out by the oligarchy.

        A repository like arXiv can be one part of change but the other part is a common effort by the growing legions of the oppressed whose clarion call need to drown out the monolithic establishment that a relentless conservative effort has already brought about. An equally relentless effort can make reason be heard and reasonable ideas applied.

        Unfortunately money and power now rule not ideas or their practical application.

        Repositories put me in mind of an academic setting which is becoming more influenced by the privatization movement, too often rendering some academicians compromised as well.

        Good ideas and good points but I wonder what catastrophe must befall us before rationality can return. The Great Recession didn't do it. In some ways Wall Street is worse -- We needed WWII and the Depression in the 20th Century to bring the upsurge of the middle class.

        What do you think, Phil?

        Jim

          Douglas, thank you for these thoughtful comments.

          On the subject of Elsevier you may have seen the latest from Timothy Gowers on his blog http://gowers.wordpress.com/2014/04/24/elsevier-journals-some-facts/ As well as exposing the high cost of journal subscriptions he makes some comments which suggest that Elsevier is indeed taking the open access charges in addition to the subscription charges. They justify this by pointing out that the total number of non-open access papers in their journals is still increasing.

          Of course you are right that peer-review is not the only thing or even the most important thing that needs to be addressed to save humanity. My point is rather that its importance is greatly underestimated compared to other things that people may highlight.

          I read your good article.

          I am thinking a new h-index for an article; for example if you ask to some great scientists to give a vote to viXra.org articles (something as "I like" of facebook) in some different field (expert in quantum mechanics, relativity, etc), and there is the possibility to search for good articles in some fields, then the contents of the article is important for some great scientists, or for some expert readers; it is not necessary that the expert work on a new research to vote it, whereas now the citation is made by the same research field.

          There may be a new statistical vote (index) that give the quality of the article, with a weighted average from the academic value, and the quality of the publication of the voters.

          An acceleration of the diffusion of ideas, in the academic world, can change the world faster.

          Dear Dr. Gibbs,

          I found reading your essay truly fascinating. I hope you do not mind me making a comment about it.

          As you will see if you read my essay REALITY, ONCE, I proved that although Bertrand Russell's perfect abstract proof of 1+1=2 is perfectly abstractly correct, it is a pragmatic impossible actual construct because identical states cannot exist. Everything real and imagined in the real Universe is unique, once. If you read my Theory of Inert Light that I have posted in the Comments section of some of the other essayists at this site, you will note that I have comprehensively refuted Einstein's perfect abstract Special and General Theory of Relativity.

          I have submitted papers and book proposals on these two subjects to Science Journals and Science Book Publishers. I have either received no answer, or I have received a snotty answer that that particular Science Journal or Science Book Publisher does not deal in the sort of science my paper or book proposal was supposedly about.

          It is only by the grace of The Foundational Questions Institute that my essay has ever been published at all. It is now being pre-judged fair and square by my peers,

          Regards,

          Joe Fisher

          Dear Philip,

          You are right on the spot with this essay. I must agree probably with everything you said, about how the peer review is done, and how should it be done, and how this can make the difference in the fate of humanity. I think I understand better the reasons why you are investing so much care in supporting open publishing and open peer review, when you could comfortably have a more "mainstream" trajectory, and ignore the struggling independent researchers. In relation to open peer review, you may be interested in this blog, from a very involved mathematician I know. I also like the thorough classification of various types of bias. I think your essay is a very down-to-earth example of applied critical thinking and freedom, about which I merely theorize in my essay.

          Best regards,

          Cristi

          Dear Dr Gibbs,

          Interesting essay.

          You say: "The hardest part of open peer-review will be to design a system of unbiased evaluation."

          Given your list of biases it does follow that the best test of non-bias is when ALL THOSE to whom ultimately a communication is addressed judge it to be non-biased. Such judgement must be ever a work in progress. But it is in human nature that once it finds you DO NEED its endorsement to survive then naturally it wants to dissent (even as a lobby group) simply to assert influence. And then you (the gate keeper) are in danger of seeking to placate in other to remain relevant.

          So then your question remains THE question:

          "How do you let anyone have their say while still maintaining an orderly process and arriving at an unbiased conclusion?" and more importantly, I say, how do you manage not to enthrone some form of mediocrity.

          It seems to me the success of Wikipedia is a very good example of the relevance of your argument. Personally, I still wonder how come Wikipedia works so well. It is probably because it tries to MAKE NO ASSUMPTIONS but leaves its OWN positions as ever a work-in-progress; same tenet the scientific method is aimed to be. That said, it still is important to study why or how Wikipedia actually comes to work.

          Given the significance of science presently in steering humanity your angle of essay is a most crucial and practical one. By extension if we can find a way of "wiki-editing" our influence/votes on governance then we have a new improved form of democracy! This is how constructive revolution should happen; one practice at a time!

          Meanwhile, how about some open peer-review here . Strictly a review; no being nice!

          Thanks again for this essay.

          Bests,

          Chidi

            Hi Philip,

            your essay seems to be very popular. It is well written and a pleasure to read and I think you are supporting a good cause.I can't help thinking, since you raise the issue of bias, that being founder of the viXra.org e-print archive has greatly influenced your interpretation of the essay question. As you point out though Daniel Kahnman reminds us that we are all prone to many different kinds of bias. Allan Savory:How to green the worlds deserts and reverse climate change This guy is someone already attempting to steer the future, having learned from his own mistake. I'm sorry'in my book' there are bigger issues than peer review. Good essay nontheless. Good luck, Georgina

              Georgina, yes, there are many big issues that concern the future of humanity. Climate change, overpopulation, energy, epidemics etc. The point is that all of these things require research and funding to steer the right course, and that requires good peer-review.