One thing that has really started to puzzle me is how un-scientific scientists are about the practice of science. In their own domain researchers will tear arguments to pieces, critically analyse each piece for flaws, and argue incessantly over the data, the methodology, the analysis, and the conclusions that are being put forward, usually with an open mind and a positive attitude.

But shift their attention onto the process of research and all that goes out the window. personal anecdote, gut feelings, half-baked calculations and sweeping statements suddenly become de rigueur.

Very briefly my view is that Peer Review doesn't stand a chance because it is being asked to do too many things. People want it to:

1. Stop incorrect science being disseminated.
2. Detect fraud
3. Assess what people will find interesting now
4. Decide what will have been important when we look back in 20 years
5. Pass judgement on a researchers quality
6. Work with authors to improve their publications
7. Act as a proxy for grant awarding panels

That's too much to ask of any system. I could probably devise a very efficient way to achieve one or maybe two of the above, but all of them at the same time is an impossibility.

Source: Science in the Open