I was in the middle of a twitter conversation about publishing when it again hit me that everything could be so much better if publications would just be published by publicly funded journals.
The currently hot idea for changing science publishing for the better: ‘pre-reg’ (article in The Guardian). This concept is tailored to give less incentives for overselling your studies. You submit your unfinished project as a kind of preliminary manuscript and it can be registered for future publication, regardless of the outcome. The idea is to base the publication decision on scientific soundness rather than outcome spectacularity. Awesome! But! By reducing one problem we increase another one. And that is the problem of reviewers being asked to make prophecies on whether your work will or will not be of interest. We see this problem already in the infamous grant application process. Funding agencies and publishers alike want to have ‘only the best work’ funded/published, so they would want reviewers to only allow the most predictable work for ‘pre-reg’/funding. This is or could be a big cockblock for explorational studies that aren’t really based on rock solid hypotheses but on a rather general question and assumptions. When competing for the same ressources, then strictly hypothesis driven studies win. To make things worse, reviewers forced to make predictions start taking the applicant’s scientific heritage into account (‘if he was a postdoc in the lab Prof X then his projects can only be excellent’). And the problem I have with this is that if new studies can only be based on older studies and the ‘offspring’ of certain silverbacks is selected for, we facilitate inbreeding of scientific ideas and we miss opportunities for diversity. Michael Eisen once proposed a system that would adress the problem of premature crystal-ball judgement by turning around the funding system from ‘pre-reg’ to funding based on success rate. In other words: publications, citations and progress reports are used for post-hoc evaluation of your work and estimation of how worthy you are of funding.
So, we have two systems: publishing and funding. Both evaluate our work. The one system is doing it before we actually have results, the other one afterwards. Both pose problems and both were proposed to be adressed by turning the process around. I agree in both cases that the existing system is neither good for our reputation in public, nor for the scientific process itself.
In both systems yet another critique of the review process is that reviewers might simply not like the authors. Bands of scientists stick together and dismiss all the work of other scientists in the field, or stall competitors in order to publish their own studies earlier. This is competition at it’s worst: when it leads to fights outside the framework of the scientific method. So there is a demand for transparency in the review process to make sure only scientific and not political aspects are considered and an idea to solve this would be post-publication review. Meaning that one would publish a paper and while it is already visible to the world, reviewers would comment on it in public and the interested reader can reproduce the development of the article.
Finally, there is the pressure to open science to the public. Open access, open data – some researchers want to move away from journals completely and instead go forward to self-publishing to prevent any sort of copyright problems. And there would be neither publication fees nor paywalls: no sinking of public money into private publishers’ pockets anymore. This has become a possibility since the rise of the internet. Everybody can produce professionally edited documents and host them on their webspace. So everybody can be their own publisher. Which other problem would that solve? Well, you can just publish whatever you want, garanteed!
A lot of good things so far, right? So why isn’t my title ‘Go start your open access, open data, post-publication review self-publishing website! Pre-reg totally unnecessary!’? Because there are good things about centralized publishing and institutionalized editing and reviewing of papers. The recent boom of for-profit open access journals shows what happens when the threshold for publishing is lowered to the level of the author’s willingness to publish. I am hinting towards that article on how dinosaurs didn’t go extinct but were ‘traumatized’ by god… in a ‘peer reviewed’ journal on head traumata. I still don’t think the word ‘traumatized’ means what the author thinks it means, unless he pictured god striking the dinosaurs with a big hammer… Thor style. Kinda cool, but also kinda coocoo crazy. I don’t know what went wrong in the process of that journal, I don’t know if they usually do better work or not. But it made me aware of what kind of bullshit people are trying to publish in scientific journals. If we all can put up our own websites, publish our own papers and call it ‘science’ it means that all those crazy people who have no idea what science is, can do that, too. If it is being positively reviewed later, or not, doesn’t change the fact that it is already public and has the science-stamp on it.
The real advantage of journals today, is centralized archiving and quality control. I am not happy about the for-profit nature and the way science is evaluated in that context, but some process needs to be in place to ensure that only science gets labeled ‘science’ and that there is one single place (or a few places) you can go to and you can trust in. This also has the advantage that everybody gets the same chance to be found and read… and what would be even more important in a post-publication review system: being critisized.
So here my not-thought-through idea:
Why don’t the public funding agencies found and sustain independent not-for-profit journals? No publication fees or paywalls would be necessary. Since the public is pretty much the only customer of scientific journals, taxes already pay for all the costs. But we wouldn’t be paying the profit margins of for-profit journals and it would free scientific publishing from economic competition. The author’s willingness to pay the publication fee is no entry criterium anymore. So we have open access down. Obviously, a journal is a central archive, so another point down. What about post-publication peer-review and quality control? In the controlled environment of a journal we can install mechanisms for a a public, post-publication peer review, ensuring for instance that it is appropriately moderated by an editor who then decides to label an article as ‘not reviewed’, ‘under review’, ‘peer accepted’ or ‘peer rejected’, based on the discussion (I always picture traffic lights ;)). What about the whole pre-reg thing? The post-publication review makes pre-reg obsolete. If you publish everything and let the scientists sort it out, everything is pre-reg. Public, transparent review will make non-scientific considerations on what should be published much harder, and noone can stall publication – because it is out there immediately – and no commercial decisions would bias for novel and spectacular findings, so less incentive to rig your data to make it look nicer.
There is definitely more to say and discuss. So, please make yourself a home in the comment area 😉
P.S. I want to recognize Björn Brembs blogs on the topic, who argued pro public publishing before me (and others did before him):
— Björn Brembs (@brembs) July 28, 2013
— Björn Brembs (@brembs) July 28, 2013