21st Aug 2019
As part of our ongoing credibility in neuroscience programme, we introduced preregistration posters at this year’s Festival of Neuroscience, BNA2019, in Dublin, Ireland.
Because preregistration posters are a very new concept to the biosciences, we would like to share some things we learnt along the way, which we hope will be useful if you are considering having preregistration posters at your own conference.
See below for some early insights about preregistration posters, and quick tips if you're thinking of having them at your own meetings or conferences - and look out for more guidance and information in months to come.
|
TL:DR Preregistration posters present plans for research, rather than a completed study. They allow for feedback at the most useful stage, i.e. the point where plans can still be changed! Nearly 20% of posters at BNA2019 were preregistration. Preregistration poster presenters at BNA2019 recieved valuable feedback & would consider doing again. Tips for organisers Give clear instructions & guidance to poster abstract submitters Ensure your poster abstract reviewers understand the principles of preregistration Make time to pass on feedback from reviewers to submitters - and for submitters to incorporate feedback into their abstracts Anticipate a wide range in the format and content; preregistration posters are still very new Highlight and celebrate your preregistration posters And celebrate all other posters too! |
What are preregistration posters?
A relatively new concept, the pre-registration poster format builds on the commentary piece by Tibon, CBU Open Science Committee C and Henson1, and the existing study preregistration service offered by the Center for Open Science.
In essence, preregistration posters give the opportunity to submit plans for studies yet to be undertaken, rather than work that has already been completed.
In the context of BNA2019, a preregistration poster had to include the following elements:
Importantly, preregistration posters have little/no data (apart from any preliminary work that helped to generate the research question).
What was the purpose of preregistration posters?
It was hoped that this new poster format would allow neuroscientists to obtain feedback from the international neuroscience community when it is most valuable (when there is still chance to revise the plan) and form collaborations that could help with the proposed study.
Having preregistration posters also helps further the principle of preregistering research and avoiding biases in reporting results2.
How was the introduction of preregistration posters recieved?
We can reveal that preregistration posters at BNA2019 were enthusiastically welcomed by the neuroscience community!
Nearly a fifth of all presented posters (98/525) were in the new preregistration format, and they covered a diverse range of neuroscience topics and disciplines.
Overall, the presenters of the preregistration posters found the experience valuable and received fantastic feedback from the neuroscience community - and would consider presenting a preregistration poster again in the future.
And the presence of preregistration posters at the festival helped to increase awareness amongst delegates of the concept of study preregistration process (see the COS website for more information about preregistration).
Did any preregistration posters contain data?
Interestingly, even though the criteria for preregistration abstracts specified that they should not include results, some of the preregistered posters presented at the conference did contain data.
Whilst we recommended that delegates present their research at the design stage, before collecting data, we appreciated that some studies needed to commence in the interval between submitting their poster abstract (autumn 2018) and presenting their actual poster at BNA2019 (in April 2019).
We therefore encouraged presenters to share their data so far, but not rush to premature conclusions simply to have some results to present at the Festival.
We also believe the presence of results on preregistration posters was because presenters felt an internal pressure to produce some form of data for the festival. As the neuroscience community becomes more familiar with the preregistration concept, we hope that delegates will feel more at ease with presenting their study design at a scientific meeting, without a percieved or actual expectation that - in order to be acceptable - it needs to contain results.
Any other findings?
Given that preregistration posters are a new format for conferences, we took the opportunity to undertake some evaluation to measure their effectiveness - initial findings of which are given above.
We are continuing to process data from this evaluation, and plan to publish a full analysis soon. Watch this space!
[Update: the findings from this evaluation have now been published in Nature Human Behaviour: Full paper: Brouwers, K., Cooke, A., Chambers, C.D., Henson R.N., Tibon, R. Evidence for prereg posters as a platform for preregistration. Nat Hum Behav (2020). DOI https://doi.org/10.1038/s41562-020-0868-z
Published 27 April 2020]
Read around
We recommend reading Title TBA: Revising the Abstract Submission Process. The Center for Open Science (COS) is a treasure-trove of useful information about Open Science in general.
Prepare your fellow organisers, delegates, and potential prereg poster submitters
Give your delegates and future presenters clear instructions on what to include in a preregistration poster. It's highly likely that they will be the first person in their lab group (or university) to submit a preregistration poster abstract, and so they won't be able to simply look at what someone has done before! If you are thinking to introduce preregistration posters at your conference, please feel free to use the explanatory text and/or abstract submission guidelines that we used at BNA2019 (in whole or part).
Make sure your reviewers are familiar with the concept of pre-registration
Despite sending information to abstract reviewers beforehand, we still had quite a few reviewers rejecting preregistration abstracts on the basis that they lacked results. In future, we will spend more time making sure that reviewers know what’s required of preregistration posters, and how to review them, and to allow time to answer any questions reviewers may have.
Pass on constructive feedback from reviewers - and give presenters the opportunity to improve their abstract before acceptance/publication
After all, one of the main aims of preregistration is to get useful input which will ultimately improve science! If you, as an organiser, recieve comments from your abstract reviewers which will improve the abstract and/or presentation, then take the time to pass on to the submitter so that they can act on the feedback. Ideally give them the chance to incorporate improvements into their abstract before it is accepted and used in the meeting programme and /or published.
Be prepared to be flexible about what’s actually presented
Preregistration posters are likely to vary widely in how much data they actually contain, from no data to a set or sets of results. Although the ideal is to present work that hasn't yet been carried out (so that plans can still be changed), the format of preregistration posters is still at a very early stage of develpment and it's likely you'll recieve a wide range of posters.
Highlight preregistration posters at your event
They are a great way to prompt discussion about new ways of doing and sharing research. Find ways to highlight the preregistration posters on the stands themselves, in the programme, or in the abstract book.
Use TOP Open Science stickers
The Open Science TOP badges were introduced for journals to acknowledge papers which met criteria for open science practices of sharing data, sharing materials, or that were preregistered. You can do the same by having stickers ready to put on posters which have used these practices, further highlighting preregistration. (We found that getting TOP stickers on your poster was very popular!)
[Update: We now have a badge specifically for Preregistraiton posters! See image above / right, and download hi-res version from here.}
Celebrate your preregistration posters - and your standard posters too!
If you run a poster competition, introduce separate categories for preregistration posters. It's important to celebrate those who are venturing into these new ways of doing research, but it's also important to recognise that they are different from 'traditional' posters which include restuls, and it wouldn't be fair to include both types in the same category. Celebrate the new, and celebrate the not-new too!
The BNA's campaign, ‘Credibility in Neuroscience’, is one of our most important programmes to date, to drive open, transparent and reproducible research. If you’d like to help shape the future of neuroscience, take a few minutes to take part in our Credibility Survey. We’d love to hear what you know, or don’t know, about reproducible, credible & open neuroscience.
1. Trends Cogn. Sci., 2018: Title TBA: Revising the Abstract Submission Process.
2. PNAS March 13, 2018 115 (11) 2600-2606; first published March 12, 2018: The preregistration revolution
