Open Peer Review to Save the World by Philip Gibbs

Author's Quote:

"An example I gave in the essay was the drug Tamiflu on which the UK and US governments have spent a fortune to build a stockpile in case of a deadly flu epedemic. We now learn that according to some the research was flawed and the data was hidden. Should we allow them to keep it hidden to preserve their trade secrets? I think not."

Third-Party Research is not made public unless the organization paying for related research authorizes disclosure.

An individual researcher selectively chooses the studies to be considered relevant for publication. So how does "hidden" research become available for review if the researcher chooses before-hand to avoid certain investigations that would otherwise reveal weaknesses to proposals or undesirable consequences?

Publicly accessible research that would require-by-law openly accessible support reveals trade secrets; unless a trusted organization that protects such trade secrets reviews supporting evidence (FDA).

What you seem to be inferring is that the FDA is not a trusted organization and is internally biased, or worse, corrupt.

Without an agency that has the facilities and authority to monitor everything (National Security Agency) the selective access provided by any other means may be biased by the funding entity.

Open access to the fundamental supporting information can be part of the trade secrets protected by business protocols.

The focus instead needs to be on better developing the trusted monitoring agencies to be sure they are performing without negligence.

I took the time to create a real-world application where you can implement your concept, so I am in support of your concepts, but ONLY if real-world respected oversight (FDA, NSA ...) provides protections of trade secrets.

Login to Google at http://www.google.com

https://groups.google.com/forum/?hl=en#!forum/open-peer-review

I think your essay is good but idealistic, because it is missing a key feature that makes it economically dangerous; but if the key feature is included it makes your essay content broadly implementable.

People generally do not like to discuss corruption as part of their solutions; because they do not understand the mechanisms of corruption. But in the real world corruption must be a consideration in everything we propose.

Your Open Peer Review process is presented as an effort to fight existing corruption (i.e. unethical allocation of resources and/or opportunities). I am only trying to shore up a weak point that would make your essay content realistically implementable. I am not in opposition to your essay, I am trying to realistically support it.

When people leave comments that broadly support your essay in its present configuration I see the potential danger of adoption into a paid effort that results in dismal outcomes.

What I see is missing in our existing FDA review is a listing of all efforts in review by the FDA, and related materials the companies "CAN" make available to the public for review without exposing trade secrets or business strategies. There should be a "Professionals" questions and answers forum for each FDA effort that allows licensed/certified Public Experts (diverse specialties) to ask questions related to proof of supporting information. Persons engaging in espionage would be assumed to always be present. The FDA would provide non-specific support with margins of error to forum questions. In this way the FDA can obtain reasonable focused feedback from the public about broad business related concerns.

In almost all open peer review an author's name is cited, and very often related contact information. Supporting data can be requested of the Author, this is nothing new, but who is going to pay for their labor to provide the related information? For an author to provide related information while being paid to produce other efforts is unethical. Who is going to pay for online access for all eternity? Who is going to deal with data obsolescence? Who is going to be liable for misuse of the information disclosed? I used to deal with UXO and much of the related information is confidential. Raw data is interpreted within margins of error. Open Peer Review and requiring detailed supporting data is very complex unless corruption can be eliminated for all related efforts. Providing details costs.

In a Federal Program of research, over 70% of the cost of development is often related to documentation. Yet the details are limited to help keep the costs down.

Idealistically, the NSA would ensure everyone remains honest globally. Then the most sensitive of data can be posted openly for public review.

Presently Open Peer Review cannot be idealistically open; the review process requires intervening trusted authority oversight to protect economic security and provide for public safety.

Dear Philip,

Many thanks for the actual essay, the deep analysis of the Science at the beginning of the Information age, relevant ideas, best offers and Open VixRa. Open Society and Open Science - a reliable way to the future of Humanity.

Let's keep The Peace, save The World and The Earth together! New Generation says all inhabitants spacecraft named "Earth": Time to start the path.

Thank FQXi that brings together people for "brainstorming" on very important topics of modern Humanity and modern Science!

I invite you to comment on and appreciate my ideas.

High regard,

Vladimir

Philip,

This was a good and needed essay. Clearly it tapped into much and well-warranted frustration in our community, lifting it to the top of the ratings. I agree with your well-structured, basic critique of the existing system. For example, the bias against independent researchers impedes their expression and careers and prevents others from benefiting from their work (even if some smaller portion of it would be valuable, compared to that of traditional academics. If ...) ViXra has commendably helped fill a gap in access.

However, I don't think that traditional peer review needs to be so radically overhauled. There is a place for it. I think one good way to at least reduce the power of bias is to promote and enlarge blind peer review, where the name and the affiliation of the author is hidden from the reviewers. (Well, maybe "peer" isn't the best word since the contributor may not really be a "peer", but this is the context we normally refer to.) Sure, reviewers have some clues to guess with, but at least they will not be sure and can more easily direct to the merits of the piece (writers will of course need subtle ways to avoid implying identity and background.) Yes, reviewers will still perhaps have some bias against the very act of having submitted as BPR. Yes, the Editor/s will still know, and so on. Yet I think this is helpful. Some journals already offer BPR, and I'd like to see more of it. Your thoughts would be appreciated. Thanks again for a great essay, with an impact enhanced by your own unfortunate experiences tilting against academic windmills. If enough of us ride together with purpose and direction, the result will not be quixotic.

Dear Phillip

Your clear and distinctive proposition addressed one of the key problems and pointed to the simple solution of the self-similar matrix of the age of problems... But as you may witness, in order to justify and preserve themselves, non-living systems - institutions - are not about solving but rather maintaining and creating problems... Small patches are sometime accepted, especially when those patches make things even more complicated or more expensive, but simplicity, logic and mathematical rigour become a part of some romantic past. As written by the Nobel Prize winner, Leon Lederman, the age of reaching for the truth purely by thinking of the phenomenon, drawing geometry on sand, the time of Tales, Pythagoras or Archimedes is an issue of some distant and in scientific terms more primitive past... Nowadays, says Lederman the truth is revealed by machines... In other words, exclude the creative thinking and switch on the accelerator... and if you cannot afford it, tuff luck, you are too primitive to be a scientist...

You may have been noticed that non-living systems - institutions (states, religions, corporations, institutionalised science...) - favour all which is unnecessary, artificial, unhealthy, uniformed, obedient, destructive, lethal, non-reproductive, and non-alive. On the other hand, living necessities like reasoning, individual intelligence, sexuality, reproductive organs, natural food, healing plants, freedom, life as such... are "legitimately" stigmatized, neglected, rejected, forbidden, tortured or executed... On the universal scale this self-similar matrix can be recognized as Tanatos versus Eros... On a scale of a body, it is virus versus cell.

I am sure you remember the OPERA neutrino experiment anomaly. Using very precise methods, during the time between 2009 and 2011, the OPERA team performed successive measurements in which the result remained unchanged. As specifically noted in the conclusion of their article published on http://arxiv.org/abs/1109.4897 they felt obliged to inform the scientific community of their unexplainable results. Shortly after the publication, CERN sends the division, significantly named ICARUS, which announces the discovery of insufficiently pressed cable :)... Entirely uncritically, the "argument" was immediately accepted by "scientific community", the anomaly was dead, the Standard model survived and CERN kept its throne... Consequently, the "rebellious" heads of the leaders of the OPERA team have silently fallen...

Then I read that the similar, though less significant results were measured and overlooked by some other, earlier experiments, performed by less powerful accelerators...

By putting my trust in the equality symbol of mathematics and the universal validity of its principles, I took a coffee break, a piece of paper and the pen, and calculate the implicit result obtained by OPERA team. Therefore, I consider the self-proclaimed "Temple of the science" not only irrelevant but intentionally false. At full conscience and responsibility I declare the OPERA experiment anomaly denial as being one of the biggest scientific frauds of our modern history. The only accepted argument which can refute the stated is proving that the speed of light is not equal to EM wavelength over its wave-time or that mathematic and geometry, being considered as primitive, aren't tools for the truth.

And of course, it is published on your "insignificant" viXra. Take its marginalisation, neglecting, ignoring and rejection as a compliment :)

I invite you to open review my nonaligned essay The Arrow of Time.

Regards,

andrej

Dear Gibbs,

I found your essay much absorbing. It held my interest through out.

I have attempted to rate you accordingly to keep the pace of your leadership but the system has not justified that. I am contacting the FQXI to resolve.

I want you to read my article STRIKING A BALANCE BETWEEN TECHNOLOGY AND ECOSYSTEM on this link http://fqxi.org/community/forum/topic/2020

I have a model that is similar with your ideology. I expect your comment and rating at the end.

Wishing you the very best in this competition.

Regards

Gbenga

Philip,

Open Peer Review with Business Incubator

I support your essay strongly, with the additional requirement of "qualified" peer reviewers in diverse fields of expertise.

I have written to an NSF program called Synergy and provided a whitepaper to test their interest in funding a cloud based tool. The social networking tool would be available to everyone to create broad enterprise based upon individual concepts being developed in a public forum.

"If you are going to listen, do something." ~Thomas Dunn

Hi Philip,

At first I though, is this really that important for steering the future.

On second thought ... Yes it is very important. Open peer review can help with the problem of being locked into what we know for certain (peer reviewed) that just isn't so.

I am reminded of the recent Nobel Prize speech in physics where the recipient encouraged that physicists should publicly agree on string theory when in private they are not sure about string theory. This strikes me as an unethical way to enhance the public image of physics and physicists and enhances the propagation of what isn't so.

Thanks for your essay.

Don Limuti

Hi Philip,

Your position is clearly stated, and you argue for it in detail. One concern, however, is

whether implementation of knowledge isn't a bigger problem than defects in the process of acquisition of new knowledge. At the end of the third paragraph, you state, "We have the intellectual capacity to figure these things out and steer the right course, yet we fail." Very true. But isn't the greater problem at the stage of steering rather than at the stage of figuring things out? Georgina Parry suggests this in the exchange with you on May 1. I think Sabine Hossenfelder in her essay especially emphasizes that deficiencies in implementation are at the present time more serious than problems with the acquisition of knowledge. Surely you have identified an important problem, but there might be even more serious problems at other places in the many processes of acquiring, distributing, and utilizing knowledge.

Laurence Hitterdale

Dear Philip,

Thanks for submitting your well-written, and well-thought-out essay. I especially appreciated your list of biases. Have you ever considered coding any of these biases or other logical fallacies into RDF/OWL? My hope is that such a system would detect articles with excessive bias and logical error (and save me the trouble of reading them). The Semantic Web will help enhance our thinking (and reduce our biases) in other areas (e.g. law, medicine), not just scientific discovery, is one of the Three Crucial Technologies ) that I mention in my essay (feel free to critique it).

I do wonder if the "IKEA effect" has more to it than just bias. As Richard Feynman said, "What I cannot create, I do not understand." OK, IKEA uses low-strength particle board instead of wood, but their design engineering and supply chain methods are fantastic.

You say that we are illogical because of bias, but then you don't ask the next question. What is the evolutionary advantage of bias in thought? Maybe I should leave it as an exercise for the reader, but the short answer is, "Because for a large enough percentage of the time, it works." Those of our ancestors who did not employ bias in their reason died before they could become our ancestors.

I understand that the current publishing system could use some improvement, and I appreciate your hard work in establishing a more open publishing venue. But, not being an academic (and most of the world are not academics) I'm not sure that it really matters. People who reify scientific ideas into products (like Steve Jobs), or people who incentivize (or disincentivize) socially beneficial work (like the governments who make the laws of a democratic capitalist country) matter a lot more than we do. And while it is humbling for us scientists and engineers, it turns out that most discoveries usually pop up in more than once place near the same time. We really don't matter that much. :-(

You mentioned the importance and desirability of "carefully considered logic." Well, yes, and I do call for it in my essay, but we must remember that logic has it's limits (as proven by Godel, and recognized by anyone who has done any work with Descriptive Logics that do inferences over Semantic Web data. The upshot is that knowledge is not enough. Even wisdom may not be enough, if you don't have the advanced nanotechnology needed to feed the hungry (and heal the obese) without overloading our ecosystem. And even advanced nanotechnology can't save us from large K/T asteroids, nuclear war, and the like--for that we need a space-faring civilization.

Fixing the peer review of today's journal publishing system would be a good thing, but it won't save the world.

@Laurence

"...defects in the process of acquisition of new knowledge" "ephasizes that deficiencies in implementation are at the present time more serious than problems with the acquisition of knowledge"

So perhaps broad development in skills related to Common Sense coupled with an Open Peer Review forum with related Business Incubation may be a stronger solution.

Common Sense = Self-esteem (shared social group skills) logic predicting consequences

The skills of predicting consequences within the complexities of skills shared by a social group (physicists for instance) seems to be an act of developing new relationships to support new applications.

Open Peer Review with Business Incubation has the potential of creating new sustainable outcomes.

I completely agree that the time has come for open peer review, Philip. The system we have is ridiculous and counterproductive. I do wonder whether some expert curation is important to make sure the best papers emerge from the general noise, but I largely agree with everything you say.

While I think open peer review is an excellent idea, I'm not sure it's enough to save the world. It would certainly help, for example, if we understood the global climate better than we do. But I don't think better science will convince climate change deniers that we need to take more action. I'm afraid that our hardest problems may not be scientific, but political; that they may be driven by conflict over ends and interests that scientific research alone cannot resolve.

Best,

Rober de Neufville

Robert de Neufville

@Tihamer @Robert @Philip

Would you like to collaborate on implementing an Actual system, and not just engage in mental exercise? These essay submissions are unimportant unless acted upon.

From my perspective, you each are talking about different parts of the same system.

1) broad and diverse open forum limited to unrestricted research

2) software automation to make phases of development obvious and provide supporting tools to grow each phase of development

3) motivational structures built into the system to drive development into implementation

4) crowd-sourcing tools made available to fund phases of development. This is especially interesting because then after seeding an NSF proposal it can evolve in parallel diverse directions to seed other developments not ever first considered in the original NSF proposal. So NSF would become a one of the many crowd-sourcing funding agencies.

5) Training tools for skills can be made available to everyone to acquire social group related skills. Where each development project that evolves within the open peer review forum involves diverse specialties. To share communication skills with the group each person will need to familiarize themselves with the jargon and rudimentary relationships that are common to each participant. These related skills can be made available via online courses (many of which are currently free).

6) A system of equitable participation is developed to track the total involvement and contributions of each peer development group that evolves from the open peer review group. In this way when monetary gains occur, each person can be equitably be compensated.

7) .........

http://jamesbdunn2.blogspot.com/2014/05/open-peer-review-to-support-synergy-of.html

I can go on, but I hope this is enough information to create a perspective of an open online tool where open peer review transitions into open peer development.

Tihamer's concepts for supporting his three points of steering the future involved rhetoric (the expert use of language) to define perspectives of commonly relatable situations that resulted in:

1. Develop physical tools that give atomically precise control over bulk matter,

2. Multiply the size, diversity, and hardiness of Earth's biosphere (by transplanting it elsewhere),

3. Develop tools that help us think better.

I myself have attempted to do mind mappings for all of these. I'm guessing I'm not unique. So why aren't we doing something about it together?

Robert, you seem to be relating how a person's political connections and influences (i.e. perspectives and relationships) drive the support systems for any development. Political tools are built in applying the same structural foundations but different environment of applications for common sense.

Common Sense = Self-esteem (sharing of social group skills) logic predicting consequences

So politics is based in understanding and applying emotional skills logically connected to social skills, to manipulate broad systems of human efforts toward a desired set of outcomes.

Nothing we know is absolute, except for one thing; "Something Exists". Everything else is built from relative systems of perspectives and supporting relationships. We cannot see or perceive everything relative to a perspective, so sometimes other unseen influences become dominant and our perspective no longer is completely valid.

I challenge you to find one perspective other than "Something Exists" that is Absolutely True always in every time frame and every dimensional consideration. If you do, then you have found a characteristic of quantum causality.

Our brains live in what others outside this forum would consider an illusion. We use repeatable observations that result from identified consequences that provide a higher probability of success than random. Bias is based in broad perceived likelihood for success relative to the total systems of involved with a consideration.

So no two people are going to share the same biases.

The Open Peer Review process inclusive of other areas is important.

Expertise in one area of consideration can be almost completely unrelated to another perspective being considered.

Open Peer Review that causes people of like interests to mutually develop common skills to share in a development has the potential to drive broad economic developments, that fund ever growing numbers of significant developments that we currently cannot fund with our present system of peer review.

What do you think? Productive collaboration?

You can contact me at the below url so you do not expose your email address to spam harvesters.

http://www.ua-kits.com/frontpage/index.php/contact-us/

Oh, I inadvertently repeated myself - and thanks for answering my original statement of the same issues (more elaborately here, but no big deal.) Keep up the good work.

Hi Philip,

I love your article and gave it the "10" it deserves. You've brought a lot of vital concepts to the fore, and I hope that you win the contest because of it. I hope that open peer review becomes standard practice someday, for so many reasons, and because it really could save the world. All the best!

Warmly,

Aaron

Dear Philip

Your essay is well written and organized. Your topic is well developed and controversial. Overall, I agree with some of parts of your essay, I just would like to express a couple of disagreements.

First, I do not think changing the peer-review process is an important factor that will steer the future. High quality research will always prevail over bad quality research no matter what judging process we adopt. Bias and subjective opinions are intrinsic human characteristics that cannot be eliminated. We see this everywhere.

Second, I think that the present peer-review system is the most solid. Science has a reputation gained over some centuries and it will not jeopardize it by changing its peer-review process. Your proposal, specially principle 3) will reduce the quality of research. And we see this in your repository. On average the quality of research published in vixra is inferior when compared to that one published in arxiv. Arxiv's policies (such as endorsement) guarantee a minimum of quality in research because most people are experts in their fields. Manuscripts that do not meet the minimum quality standards should be rejected. By contrast, in vixra, one can find many articles with very poor quality. In my opinion this is the reason vixra is not well seen in the academy. This is something I regret you do not discuss in your essay.

Likewise, traditional journals vs open-access journals. The average quality of research published in traditional journals is far superior than that one published in open-access journals where as long as you pay the publication fee, your manuscript will be published even without a peer-review process. In my opinion open-access journals has gained a negative reputation due to their unethical practices. If these practices continue, my prediction is that they will all disappear by the end of the decade.

Also regarding your third principle, if I do research in a field, I would like to be criticized by someone who knows about that topic but not by a layman who doesn't know the rudiments. Just in the same way, I would not criticize a musician for playing an instrument that I do not even know how to play. One can only criticize something if one has enough knowledge about it.

Exams are filters to select the best for a given activity. It is not a method of suppression. What you call suppression, I would call it filtering in favor of higher quality in research.

You: Wikipedia is a good example of a system that allows anyone to contribute.

I wish this were true...

Anyway. I would like to take this opportunity to invite you to read and comment on my thread.

Best Regards

Israel