When Does Peer Review End?
Post Reply
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
24-06-2015, 01:35 AM
When Does Peer Review End?
So, over the past couple years, there have been "Peer Review Scadals." It has come to the attention of the scientific community that Publishers arn't reviewing papers as well as they should. Not just the little journals, the big ones, too.

This is discovered via experiment. Scientists make a bunch of gibberish papers, and see if they make it through... the data is scary.

But what *is* peer review? Well, it's exactly what you'd expect. Peers reviewing your work. And do you know when that ends? Never. If your work is at all important, other scientists will use it until they find it doesn't work. Every time that happens, it is being reviewed by peers.

So, you may need to be careful about fresh off the press articles. Check how many papers cite the paper you're looking at, so you know it's safe. That is the true measure of peer review.
Find all posts by this user
Like Post Quote this message in a reply
[+] 1 user Likes MPythonGirl's post
24-06-2015, 03:16 AM
RE: When Does Peer Review End?
I work on the admin of five academic journals and I can say that all of the papers submitted to ours are processed correctly and fairly (many being rejected for various reasons and gibberish would certainly be a reason for rejection). But obviously I cannot speak for the others and how they operate.

"The person who is certain, and who claims divine warrant for his certainty, belongs now to the infancy of our species." - Christopher Hitchens

"Remember kids, if you don't sin, then Jesus died for nothing. Have a great day!" - Ricky Gervais
Find all posts by this user
Like Post Quote this message in a reply
[+] 2 users Like Eva's post
24-06-2015, 03:34 AM
RE: When Does Peer Review End?
So, here are the articles I'm getting my information from. The first was actually helpful in explaining how we find journals without a good peer review system. The second is a more reputable source about the actual scandal.

If your Journals are awesome, please tell me what they are called so I can relax when reading them. The only information you need is, sting papers exist, keep an eye out for them... and maybe post how many you catch? That would be cool. "We catch 100% of sting papers!" Would be a definite bonus for people trying to find a reputable journal to submit their articles to.

This post was mainly for the Layman atheist. A) to be careful about all information, especially when you are not an expert, and B) this has been mentioned in debates before, here is the response.

Not that we arn't all Laymen at something. I have a bachelor's in Physics, I barely know anything about Medicine.

http://www.slate.com/articles/health_and...sense.html

http://www.washingtonpost.com/news/morni...-journals/
Find all posts by this user
Like Post Quote this message in a reply
[+] 1 user Likes MPythonGirl's post
24-06-2015, 03:47 AM
RE: When Does Peer Review End?
Well I just bought a book reviewing Thucydides by Donald Kagan. Thucydides lived circa to 460 circa 400 BC and we do not know the exact date of the work.

When does peer review end? Depends I suppose on how important the work is.

NOTE: Member, Tomasia uses this site to slander other individuals. He then later proclaims it a joke, but not in public.
I will call him a liar and a dog here and now.
Banjo.
Find all posts by this user
Like Post Quote this message in a reply
24-06-2015, 08:08 AM
RE: When Does Peer Review End?
(24-06-2015 01:35 AM)MPythonGirl Wrote:  So, over the past couple years, there have been "Peer Review Scadals." It has come to the attention of the scientific community that Publishers arn't reviewing papers as well as they should. Not just the little journals, the big ones, too.

This is discovered via experiment. Scientists make a bunch of gibberish papers, and see if they make it through... the data is scary.

But what *is* peer review? Well, it's exactly what you'd expect. Peers reviewing your work. And do you know when that ends? Never. If your work is at all important, other scientists will use it until they find it doesn't work. Every time that happens, it is being reviewed by peers.

So, you may need to be careful about fresh off the press articles. Check how many papers cite the paper you're looking at, so you know it's safe. That is the true measure of peer review.

There are several things that I do when looking at any journal. If it is a new journal and they present something that is a new method, I look at their submission criteria. Some journals (the less reputable ones) actually allow you to chose your reviewers as long as they are in the field. Some pick their friends. I look at these journals more skeptically than the ones where the reviewers are assigned. That does not mean that the science is bad per se, but it also may mean that it may not be as solid as it should be.

You mentioned citations, which is good, but I would also add who is citing them. I have seen books and papers where the author references themselves more than others. That is a red flag.

Something to keep in mind is that the peer review process is more to determine whether the data/scholarship supports the conclusions reached by the authors and not to vet whether something is possible (within reason). They are not there to say "well this is impossible because it hasn't been published before." If they did that, there would be very little progress. Reviewers are there to ensure that the article is supported and argued properly based on the presented evidence. If that evidence supports the conclusion even though it may fly in the face of conventional wisdom, it needs to be accepted and thrown into the colosseum to face scrutiny and if it is indeed true, it will stand on it's own two feet.

"If we are honest—and scientists have to be—we must admit that religion is a jumble of false assertions, with no basis in reality.
The very idea of God is a product of the human imagination."
- Paul Dirac
Find all posts by this user
Like Post Quote this message in a reply
[+] 6 users Like The Organic Chemist's post
24-06-2015, 11:29 AM
RE: When Does Peer Review End?
You would also have to consider how "specialised" a journal is. I used to publish in and be reviewed in journals related to Assisted Reproduction Techniques. I was also a statistical reviewer (I looked at the stats methods etc used). That's a fairly small world. We all know (of) each other. I used to get my medical colleague to review the medical aspects - so, they got two reviews for the price of none (you don't get paid). He would do the same with me when he was asked to do a medical review.

The chance that someone could pull off a sting in that context is very small.

When you talk about more general journals, then you may have a problem. I would do an honest review of the stats and try to get an opinion on the bits I don't know about from someone who does know at least a bit about the subject. Not always easy to find! The journals for which I reviewed were medical journals and I always had access to doctors but often not to ones specialised in that particular area.

The other factor that needs to be considered is the choice of referee. I've had papers rejected because the referee was completely out of his/her depth when reviewing a paper about (say) Negative Binomial dispersion. They'd give a one sentence reply and would be unable to specify any particular problem. But that's the other side of the problem to which you're referring.
Find all posts by this user
Like Post Quote this message in a reply
24-06-2015, 05:14 PM
RE: When Does Peer Review End?
(24-06-2015 01:35 AM)MPythonGirl Wrote:  So, you may need to be careful about fresh off the press articles. Check how many papers cite the paper you're looking at, so you know it's safe. That is the true measure of peer review.

Impact factor is a metric which tries to capture that at the journal level instead of individual papers.

#sigh
Find all posts by this user
Like Post Quote this message in a reply
25-06-2015, 07:55 AM
RE: When Does Peer Review End?
It seems from the article, that the problems are 1) having friends review a manuscript, 2) agreement between researchers to positively review each others work, 3) fake reviews/reviewers.

I often get manuscripts to review from people I know. Some of them were former group members, others are acquaintances and every now and then I get one to review from what I would call a good friend. I have always felt I could write an impartial review. If I couldn't, I would decline to review the manuscript. I have never had an email from someone telling me they had put me down as a referee and to please give them a good review. However, according to the article, this does happen and if it does, then proper disciplinary action should be taken.

The fake reviewers issue is bizarre. So when you submit a manuscript, you can enter suggested referees to review the manuscript. The most important contact info is their e-mail, so I guess people figured out they could enter bogus e-mails of suggested referees so the manuscript was sent to someone who would approve it no matter what. If researchers are doing this, then they should have the book thrown at them.

Whichever of the above review scenarios, it is hard for me to believe that scientists would submit false results. Your publications and the quality of your work is your reputation. When someone inevitably tries to use your results and they find out they are wrong, then that just ends up blowing up in your face. That is not to say that scientific misconduct doesn't happen, it does, but it is rare because it never ends up well. I remember the MIT chemistry student who fabricated results... Such was the furor from the community, that she literally went dark and disappeared in disgrace.
Find all posts by this user
Like Post Quote this message in a reply
25-06-2015, 11:44 AM
RE: When Does Peer Review End?
(25-06-2015 07:55 AM)Iñigo Wrote:  I often get manuscripts to review from people I know.

I've had this as well. Former co-authors. Not surprising as (as I indicated in another post) I used to work in a pretty specialised field. I simply let the journal know about the possible conflict of interest and about the fact that I was sure I could provide an unbiased review.

It was never a problem. I was just honest about it.
Find all posts by this user
Like Post Quote this message in a reply
25-06-2015, 01:42 PM
RE: When Does Peer Review End?
I'm not overly concerned about someone you know reviewing your work. Scientists (as a whole) are generally pretty honost. I mean... some experiments require lying during, but that's a different kettle of fish. It is, of course, best to find a reviewer who has a different perspective than you.

I mean, the only Scientist I can think of who is lying is Campbell of The China Study and Knives over Forks. Mainly because his published articles say the opposite of what he claims they say in the documentaries. And those Cold Fusion guys, but they are the go-to example.
Find all posts by this user
Like Post Quote this message in a reply
Post Reply
Forum Jump: