Is there a way to cope with the reproducibility crisis in science?

In the heart of scientific work is the strive to challenge and falsify existing hypothesis. Only by disproving old ideas new ways of thinking can be developed. To do so, scientists need in first step to be able to reproduce the results of other researchers: Same setups should give same results. However, many scientific disciplines are currently experiencing a “reproducibility crisis” because numerous scientific findings cannot be repeated consistently. Everybody certainly understands that reproducing results under real world settings is difficult. Imagine a forest ecosystem with dozens of tree species, hundreds of understory plant species and uncountable microbes; you do a measurement and the system and the environmental conditions are unique - you will never catch a point in time or space when the complex interplay is the same. That's why ecologists complement field work with experiments under controlled conditions. This is to reduce the complexity of the symphony of different biotic and abiotic players in order to make the system manageable. We like pots with defined soil, defined soil water availability and plants grown with defined distance among each others. We call these systems imitating parts of the real world "microcosms" let them grow in a climate controlled chamber and just manipulate one of multiple environmental conditions: Light or temperature or competition or…
This defined systems are claimed to produce results that are reproducible, but they are often not. They might be valid under given conditions (in Lab y with a climate chamber of company z, with an LED lighting system from supplier a) thus being local truths but not generalizable. Stringent levels of environmental and biotic standardization in experimental studies under controlled conditions might even reduce reproducibility among labs by amplifying impacts of location-specific environmental factors that are not (and often cannot be) accounted for in study designs.
We have now published a paper (Milcu et al. 2018) that aims at (at least partially) overcoming this problem by deliberately including variability in plant microcosm experiments in an ecological study. Such inclusion of controlled systematic variability has already been successfully adopted when assessing animal behavior.
Our results show that introducing genotypic controlled systematic variability (e.g. using not only one but different genotypes of a plant species) can increase reproducibility of results among different labs - the different genotypes of the grass species we used in our experiments might buffer the effects of the partially different and often unaccounted environmental conditions in different labs - i.e. genotype I might react to the slightly different light conditions between two labs intensively in one direction, the second genotype II in the other direction and genotype III might not react at all. On average, all three together won't be strongly affected. If you used only genotype I or II, inter-lab reproducibility would be lower. Even though counterintuitive at the first glimpse, deliberately including genetic variation may be a simple solution for increasing the reproducibility of ecological studies performed in controlled environments.

Milcu et al. 2018. Genotypic variability enhances the reproducibility of an ecological study. Nature Ecology & Evolution doi: 10.1038/s41559-017-0434-x Pasted Graphic

Preprint at bioRxiv Pasted Graphic

featured in Nature News: "Why 14 ecology labs teamed up to watch grass grow" Nature 548, 271 (17 August 2017) doi:10.1038/548271a  Pasted Graphic

Whole tree 13CO2 labelling

Since a few days we have now labelled in total 10 100-year old Scots pine trees with 13C enriched CO2 at the field site Pfynwald to trace the fate of carbon within the tree and ecosystem. The Pfynwald precipitation manipulation experiment that started in 2003 aims at releasing drought stress from adult pine trees growing at this stand in the Valais close to its area of distribution. On the one hand non-irrigated trees are more and more prone to mortality due to the increasing frequency of drought event, on the other hand the irrigation treatment (doubling the amount of the natural precipitation) resulted in a clear increase in growth and a reduction of mortality.
The 13C labelling of the trees of both treatment should now show if and how the allocation and distribution of newly assimilated carbon is affected by this long-term difference in water availability. With an international team with researchers from Switzerland, Germany, China and Finland we track the 13C label in different plant and ecosystem compartments (leaves, phloem, fine roots, soil microbes) and chemical compounds (resin, respired CO2, storage compounds) in order to understand if switches in C allocation under drought might lead to lack of reserves over longer times or predispose trees to insect attacks.


related film in the Swiss Television (in German)