Discussion: On Using Estimation Statistics

In summer 2019, eNeuro began an initiative to encourage authors to use estimation statistics in their analysis. During review, Reviewing Editors would flag articles where estimation statistics may be appropriate, or where estimation statistics may enhance the analysis; authors were invited to convert their analysis during revision. Since then, we have reached out to some of these authors to ask what additional value did estimation stats offer? A collection of their responses is below.

To learn more about this initiative, read the eNeuro commentary Estimation for Better Inference in Neuroscience by Robert J Calin-Jageman and Geoff Cumming, and Come Discuss the "New Statistics"  on the blog.

Circuit-specific dendritic development in the piriform cortex
Laura Moreno-Velasquez, Hung Lo, Stephen Lenzi, Malte Kaehne, Jörg Breustedt, Dietmar Schmitz, Sten Rüdiger, and Friedrich W. Johenning
"It was satisfying to see how switching to estimation stats based data display increased the transparency of our data display in a reader-friendly format. By openly displaying the mean effect sizes and confidence intervals, we felt relieved from the pressure to solely rely on p-value based true/false statements about the data. Estimation statistics also makes it obvious to us and our readers where replications and further experiments are most needed in the future." -- Friedrich Johenning

HIV Tat protein selectively impairs CB1 receptor-mediated presynaptic inhibition at excitatory but not inhibitory synapses
Mariah M. Wu and Stanley A. Thayer
“The addition of estimation statistics to our paper nicely complemented hypothesis testing, which is typically the focus of our presentation of results. While it did not change our conclusions, estimation statistics emphasized the size of effects, which provided a more nuanced analysis. Going forward we will include confidence intervals when reporting effects.” -- Stanley Thayer

A novel mode of sympathetic reflex activation mediated by the enteric nervous system
T. J. Hibberd, W. P. Yew, B. N. Chen, M. Costa, S. J. Brookes, and N. J. Spencer
“Estimation statistics was not difficult to perform and added another factor for data visualization. The statistics and graphs for paired and unpaired data were generated using tools on the website made by Adam Claridge-Chang and Joses Ho (estimationstats.com). The process importing data and performing analysis was more intuitive than some of our standard statistical software packages. For this reason, and because the estimation of mean differences adds a readily understood component to data visualization, we will use estimation statistics where possible in future studies. It may be of interest that Altman-Gardner estimation plots are not currently automated in GraphPad Prism, a statistical package used by many in the biological sciences, but instructions on how to do it are shown here: https://www.graphpad.com/support/faq/creating-an-estimation-plot-of-the-results-of-an-unpaired-t-test/” -- Nick Spencer

JNK signaling regulates cellular mechanics of cortical interneuron migration
Skye E. Smith, Nicholas K. Coker, and Eric S. Tucker
“Converting to estimation statistics was relatively simple once we familiarized ourselves with the process. There was remarkable consistency between the p values that we obtained from standard T tests and estimation statistical analyses. Graphical representation of the data with Gardner-Altman plots allowed us to visualize individual data points, mean differences, and confidence intervals between our comparisons, which was far superior to the use of histograms. Altogether, converting to estimation stats gave us greater confidence in our data.” -- Eric Tucker

An Electrophysiological Abstractness Effect for Metaphorical Meaning Making
Bálint Forgács
“Confidence intervals were not difficult to add at all using R. They provided a quick and easy way to illustrate the magnitude of the difference of psycholinguistic variables between conditions beyond p-values, even with no figure. Confidence intervals are a great way to show details of the data not only in figures—and sometimes even without them.” -- Bálint Forgács

Pannexin 1 regulates dendritic protrusion dynamics in immature cortical neurons
Juan C. Sanchez-Arias, Rebecca C. Candlish, Emma van der Slagt, and Leigh Anne Swayne
“We were inspired by eNeuro's initiative to encourage the use of estimation statistics. We found that it critically shaped our interpretation of the data for this paper and also has changed the way we are approaching new projects. The availability of well-developed, relatively easy-to-use open source packages (https://github.com/ACCLAB/dabestr) and online tools (https://www.estimationstats.com/#/) greatly facilitated this analysis. We believe that more widespread adoption of estimation statistics will lead to better experimental design, reproducibility, and more appropriate interpretation and applications of experimental neuroscience.” -- Juan Sanchez-Arias and Leigh Anne Swayne

Effects of Inactivation of the Periaqueductal Gray on Song Production in Testosterone-Treated Male Canaries (Serinus canaria)
Chelsea M. Haakenson, Jacques Balthazart, and Gregory F. Ball
"Since this study had a relatively small sample size, estimation statistics were invaluable for increasing our confidence that the effects we saw were robust and not likely to be spurious. The Data Analysis using Bootstrap-Coupled ESTimation (dabestr) package in R was easy to incorporate into our typical data analysis pipeline. After installing the package, we were able to run the analysis by modifying the code found in the following tutorial: https://cran.r-project.org/web/packages/dabestr/vignettes/using-dabestr.html.” -- Chelsea Haakenson

Distinct age dependent C fibre driven oscillatory activity in the rat somatosensory cortex
Pishan Chang, Lorenzo Fabrizi, and Maria Fitzgerald
“Our data consists of comparisons before and after treatment across six different oscillation frequencies within an EEG trace. Using estimation statistics allowed a much better assessment of the distribution of data within each frequency and the confidence intervals in our comparisons than conventional t-test statistics. We directly entered the raw data into an open source estimation programme available on https://www.estimationstats.com and downloaded the results and graphs. Highly recommended.” -- Maria Fitzgerald

Frontal, parietal and temporal brain areas are differentially activated when disambiguating potential objects of joint attention
P.M. Kraemer, M. Görner, H. Ramezanpour, P.W. Dicke, and P. Their
“Using estimation statistics along with null-hypothesis significance testing (NHST) allowed us to test whether the specific results we found depended on the type of analysis and associated assumptions. We found that the different analysis methods converged allowing us to conclude that model assumptions were not the critical factor producing the effects we found. Furthermore, computing the Bayesian Credible Intervals allows us to make a statement on the range of possible parameter values and the effect size with a certain confidence, which would not be possible based on NHST alone.” -- Marius Görner

Doublecortin-like is implicated in adult hippocampal neurogenesis and in motivational aspects to escape from an aversive environment in male mice
Dirk-Jan Saaltink, Erik W. van Zwet, and Erno Vreugdenhil
“Statistics should not be limited to doing hypothesis tests, but it should also not be limited to estimation. Statistics is about learning from data using many different methods, including description, estimation, testing and visualization.” -- Erik van Zwet

Leave a comment below.

Do you have experiences or resources to share regarding using estimation statistics?
Do you have thoughts regarding when using other analysis tools are better suited in specific areas?