Reproducing the computational environments of experiments
Reproducing experiments and results is a cornerstone of science, but researchers acknowledge that actually achieving this feat can be tricky. Specific experimental setups are usually the result of a lab's painstaking work and, in today's environment of high-throughput methods, are increasingly expensive. The fact that complex, customized sets of software are frequently involved in the analysis and interpretation of data makes it even more difficult to achieve true reproducibility.
Guix – a free software that is used to fully reproduce computational environments – might be part of the solution, says Ludovic Courtès of Inria, the French National Institute for computer science and applied mathematics in Bordeaux. To implement it he has joined forces with Ricardo Wurmus of the platform for bioinformatics and modeling at the the MDC's Berlin Institute of Medical Systems Biology (BIMSB), scientists from the Utrecht University Medical Center and a growing group of international colleagues.
Capturing complete computational environments
The National Science Foundation in the US and journals such as Nature are insisting that researchers share source code and support reproducibility. "The ability to reproduce an experiment depends – among other things – on the ability to reproduce the software environment," Courtès says. "That poses particular difficulties in the many cases which require high-performance computing (HPC) environments.”
Guix is an outgrowth of a project called GNU launched almost 40 years ago at MIT in the USA. It makes up for some deficits of earlier efforts and is addressing several challenges: Users are no longer dependent on software package management by system administrators, empowering them to fully customize the environment to their needs. It also solves problems that arise when scientists draw on "container solutions," which Courtès compares to receiving a brand-new computer where everything has already been installed. "That works until you make a small modification in the experiment to test a new hypothesis – which often happens in the world of research!"
The advantage of Guix is how it characterizes software environments in unambiguous terms, similar to a mathematical function. It completely describes all its relations and thus can reproduce them bit-for-bit. This way, Guix facilitates both reproducibility and customizability.
Adapting Guix to scientists' needs
Guix was not originally designed for the high-performance computing environments required by today's experiments. So scientists at the MDC, Inria and the partner institutes are building functions that permit Guix to be used on a computing cluster, to implement reproducible workflows. They are also adding packages that were developed at each site.
"Before Guix, the installation of scientific software was necessarily ad-hoc," Wurmus says. "Groups would build their own software, statically link it into existing systems, and hope that it would never have to change – because managing software environments was virtually impossible. Now not only can we manage a single environment per group in a reliable fashion, but we use Guix at all levels: of the group, user, workflow and so on."
The project is scheduled to last two years, at which time its initiators hope to have met the software reproducibility needs of their institutions. "The wider objective," Courtès says, "is to convince others who rely on high-performance computing that Guix represents a major advance toward a fundamental goal in science."
The Max Delbrück Center for Molecular Medicine (MDC)
The Max Delbrück Center for Molecular Medicine in the Helmholtz Association (MDC) is one of the world’s leading biomedical research institutions. Max Delbrück, a Berlin native, was a Nobel laureate and one of the founders of molecular biology. At the MDC’s locations in Berlin-Buch and Mitte, researchers from some 60 countries analyze the human system – investigating the biological foundations of life from its most elementary building blocks to systems-wide mechanisms. By understanding what regulates or disrupts the dynamic equilibrium in a cell, an organ, or the entire body, we can prevent diseases, diagnose them earlier, and stop their progression with tailored therapies. Patients should benefit as soon as possible from basic research discoveries. The MDC therefore supports spin-off creation and participates in collaborative networks. It works in close partnership with Charité – Universitätsmedizin Berlin in the jointly run Experimental and Clinical Research Center (ECRC), the Berlin Institute of Health (BIH) at Charité, and the German Center for Cardiovascular Research (DZHK). Founded in 1992, the MDC today employs 1,600 people and is funded 90 percent by the German federal government and 10 percent by the State of Berlin.
