In this Essay I would like to argue that if science is to keep up with the existing format of peer-review as a quality control, it is perhaps essential that peer review should be promoted as a permanent academic occupation in its own right. This in turn requires a shift in perspective on what it means to do science and what it means to be a scientist. We explore what a hypothetical universe where the peer review and academic research form a "dual track" in the scientific enterprise and discuss how it may look like. Interestingly we will be able to derive a few basic consequences of the dual track models, some of which are relevant in the realistic universe we live in. We conjecture that scientific progress may look different in such a dual-track system between research and peer review.
The Spear and The Shield: Science and Peer Review
This is a good argument for such a system.
Erickson Tjoa May be that is something we could give to chatGPT or another AI to keep it occupied. My approach to science is more fluid where I prefer to utilize social media and video to share new ideas and results through channels like LinkedIn. I know it is not peer reviewed and probably dismissed as something not following up with but it appears that an online community is useful to bounce ideas around in a public forum. You get to meet people with different views which is also helpful as a sounding board.
Roger Schlafly Thank you very much, though admittedly with page limits I am leaving a lot of things, unfortunately many of which are equally important, unaddressed.
Keryn Johnson Thanks for your thoughts. What you said to be your approach where you use social media to share new ideas and results are fine - in fact, they are also used by many scientists today (Twitter, Mastodon, etc.) to advertise their work. You can always advertise things before it is peer-reviewed, and possibly gain feedback from it. So "online community to bounce ideas around" is itself a nice idea (subject to some complications, if properly handled). However, to be part of an established knowledge, the argument is that one does need certain degrees of gatekeeping, since scientific work must go above being merely an opinion shared by many.
I do foresee ChatGPT and AI being useful for such a gatekeeping work, but at the moment I need to see more development in those directions in order to understand how they would be useful. For example, I think they would be very good at analysing potential mistakes or summarizing the core arguments, especially for extensive/long papers. What needs to be avoided is to simply leave them completely to ChatGPT or AI, unless, of course, that they would look very different from what they are today. Either way, I agree with you that I foresee their use in peer-review in possibly near future, and there is a need to ensure that they are used appropriately.
This was a really great piece, and given that the pressures on the system of scientific publication are likely to even further increase with the advances in chatbots, I don't think it could have been any more timely.
I particularly loved this:
"The first step is to make sure that at least undergraduate education (or earlier) in science reflects this ideal. That
is, any undergraduate program should, at least, incorporate in their curriculum how Science works."
I dealt with the same question in my own essay- "Schisms Beyond Arithmetick"-, although from a completely different angle. Please check it out it if you have the time.