Scientists and Peer Review
One thing that has really started to puzzle me is how
un-scientific scientists are about the practice of science. In
their own domain researchers will tear arguments to pieces,
critically analyse each piece for flaws, and argue incessantly
over the data, the methodology, the analysis, and the
conclusions that are being put forward, usually with an open
mind and a positive attitude.
But shift their attention onto the process of research and all
that goes out the window. Personal anecdote, gut feelings,
half-baked calculations and sweeping statements suddenly become
de rigueur.
Very briefly my view is that Peer Review doesn't stand a chance
because it is being asked to do too many things. People want it
to:
1. Stop incorrect science being disseminated. 2. Detect fraud
3. Assess what people will find interesting now 4. Decide what
will have been important when we look back in 20 years 5. Pass
judgement on a researchers quality 6. Work with authors to
improve their publications 7. Act as a proxy for grant awarding
panels
That's too much to ask of any system. I could probably devise a
very efficient way to achieve one or maybe two of the above,
but all of them at the same time is an impossibility.