Q & A: Registered Reports from Journal of Child Language

Beginning in summer 2018, Journal of Child Language will publish a new article format: Registered Reports. We asked two of the journal’s associate editors, Melanie Soderstrom and Elizabeth Wonnacott, a few questions about the introduction of this format.

 

What inspired the introduction of the Registered Reports?

MELANIE: Registered reports are a relatively new phenomenon in our research community, although to my understanding they come from a similar approach in the medical research community that has been around for many years for clinical trials. They are one part of the research community’s broad-based response to the so-called “Replication Crisis”. In early 2016, we were approached by the Center for Open Science requesting that we consider bringing this format to Journal of Child Language, and the idea received strong support.

LIZ: Yes – I don’t recall any dissenting voices when this idea was raised. I think many researchers in language development have been increasingly concerned about issues of replicability. Also, as editors, we see first-hand the benefit of getting feedback before the work is carried out. It is extremely frustrating to handle a paper where reviewers identify critical flaws, but a large amount of work has taken already place.

 

Can you describe the replicability crisis, for those who may not be aware?

MELANIE: The “Replication Crisis” is a term being used to describe the growing awareness of a cluster of phenomena centered around the idea that research findings may not be as robust as we like to think. On the empirical side, some very high profile studies have now shown that studies don’t replicate as well as we would like – although there is also now important discussion around what it means to replicate, and what IS a reasonable replication rate. Methodologically, it refers to widespread analytic practices like p-hacking, HARKing, and underpowered studies that inflate significant findings and lead to both Type I and Type II errors. On the practical side, there is little incentive among traditional publication/granting models to encourage replication of research findings. The Registered Report is a partial answer to these concerns.

LIZ: I think it is important to recognize that there are a lot of pressures on academics to publish papers with tidy results which tell a clear story. This might lead to practices such as exploring different ways of analysing a dataset until you find an analysis that “works out” (while reporting the results as though this was the only analysis attempted), or simply not attempting to publish studies which have null (or messy) results. These practices distort our literature. Registered Reports require researchers to publish regardless of the outcomes, and to be upfront about which aspects of the analyses are more exploratory.

 

How do these reports reflect wider trends in linguistics research?

MELANIE: Registered Reports are best suited for high-powered experimental work. As a linguistics journal, our mandate also includes analytic and observational work, and work on small populations, for which high power may be impractical, so not every study is suited for the Registered Report format. That said, there are benefits of this format that go beyond encouraging replication and reducing bad analytic practices. For example, authors receive valuable feedback from reviewers on their study prior to investing those crucial grant funds in an endeavor. Moreover, Registered Reports are part of a much larger movement toward Open Science (the idea that we have an obligation to share our data, methods and analyses for scrutiny and use by other researchers), something that is of importance to all sub-disciplines of experimental psychology and linguistics.

LIZ: It feels like a time of change in experimental research more broadly. For example, fifteen years ago, it was extremely difficult to publish null results in any journal, and this is certainly no longer the case. It was also rare for labs to make their data openly available, publish their analyses scripts etc. Attitudes are gradually changing so that it is now relatively common to see these types of Open Science practices, which are increasingly valued in researchers.

 

Read more about the format at this link.

 

Melanie Soderstrom is Associate Professor and Associate Head (Graduate) in the Department of Psychology at University of Manitoba, Canada.

Elizabeth Wonnacott is Senior Lecturer in Language and Cognition at University College London, UK.

Leave a Reply

 

 

 

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>