Why do big science? Wouldn’t it be better to just distribute the money to individual labs?
Neuroscience is at a critical juncture. We are fortunate to live in an era of powerful new approaches for collection and analysis of neural data. These tools will allow us to solve problems previous generations could only dream of. But the complexity of the problems far exceeds the ability of individual labs to address them. Fully harnessing the power of modern tools requires expertise across a range of domains, and it is rare for neuroscientists in a single laboratory — let alone for a single scientist — to hold all the requisite skills. Moreover, because labs do not normally share raw data, the fruits of difficult experiments cannot be fully exploited by groups with complementary expertise.
In short, whereas a generation ago neuroscientists were largely limited by theory and tools, today a major bottleneck is how we as a community can effectively harness what is already available. This is the goal of IBL; we feel the time is right to launch a new experiment in collaborative neuroscience to address this challenge.
How did you decide on the size of IBL? I.e. why do you need 22 labs - wouldn’t it be better to start with say 5 labs and then enlarge if things work?
If a typical systems neuroscience project records from 1 or 2 brain areas in a given task, recording from ~100 areas in order to cover the entire brain — our scientific objective — in principle would take 50-100x the number of labs. However, we are using new technology that will make recordings more efficient and we will benefit from standardization and sharing of resources.
Beyond delivering a never-before-available data set from the entire brain, we aim to establish processes for merging experimental design, execution, and analysis across multiple laboratories spread throughout the world. These processes range from governance, decision-making and communication to data architectures and experimental standardization. This is a potentially game-changing deliverable in itself and has ramifications for critical issues such as reproducibility that has provoked crises in fields not so far from neuroscience.
Why not start even smaller and scale up?
Typical neuroscience collaborations of 2-5 labs either don’t encounter the issues of large-scale collaboration, or can split tasks in a way that avoids the need to tackle these issues. Twenty-one is a good compromise as it allows us to include experimental labs which collect data, labs which focus on analyzing data and integrating datasets, and theoretical labs — while being small enough to foster a community. Furthermore, we believe that if we succeed at 22, we will be able to grow to 50 or 100 using the same principles. We have planned for the possibility to expand potentially to much larger number of laboratories. Successful examples in science include the ATLAS project at CERN, where the number of labs involved is in the hundreds.
What is the incentive of a lab for joining IBL?
IBL was started from the ground up, by a group of scientists who were interested in doing neuroscience differently. Other labs have joined because they want to be part of this social and scientific experiment. Many of us have been frustrated by the lack of standardization in neuroscience, which makes comparisons of our results even to labs doing quite similar work very difficult. To be part of a team generating large scale data will advance each of our own scientific ideas at a far more rapid rate than we could do alone. Putting our heads together to tackle a big problem will not only be effective, it will be fun.
When is the next step in scaling up IBL? I.e. when are you adding new members?
In the long term we aim for IBL to become an open standard. Our behavior, experimental hardware, software, and data architecture will all be open source and we will encourage any and all to use them and contribute to the project.
In the short term, our ability to help new labs join the project is limited. We are still establishing our technical protocols and learning how to coordinate our current group. We have limited funding for the initial project phase, and although we may add one or two associated laboratories to help with specific scientific needs, we are unlikely to expand beyond the core group of 22 labs until we secure additional funding which we will seek toward the end of year two.
You currently have members from 5 different countries. Is IBL limited to these 5 countries already participating?
No, we anticipate bringing in people from many other countries.
WHY STUDY JUST ONE BEHAVIOR? AREN’T YOU CONCERNED THAT YOUR RESULTS WON’T GENERALIZE?
Large scale collaborations are more likely to succeed when focused on one common goal (e.g. the human genome, the Higgs boson). In our case, having multiple behaviors would diffuse the group’s efforts and would not allow us to fully benefit from standardizing the task, the recordings tools and the theoretical analysis.
We settled on decision making because making decisions is one of the core functions of the brain, which recruits a large fraction of the nervous system and which is remarkably adaptive. Moreover, there is a large theoretical background on decision making, including models that successfully fit data from humans, primates and rodents.
WHY STUDY MICE?
Mice have become a powerful genetic model system for neuroscience, and allow a wide range of experimental manipulations that are currently out of reach in most mammals (e.g. wide field calcium imaging, optogenetics), including rats. Moreover, their brains are small enough to allow relatively dense sampling of activity throughout the entire brain with Neuropixel probes.
YOUR TASK INVOLVES VISUAL CUES. DON’T YOU KNOW THAT MICE ARE NOT VISUAL ANIMALS?
Mice can see just fine. Although they lack a fovea, their vision is perfectly sufficient to let them perform our task. The human visual system is highly specialized compared to non-primates, but our goal is not to study vision per se but to study the neural basis of decision making. Any sensory stimulus would work for this purpose and using a visual stimulus has the further advantage that much is already known about how the mouse brain encodes stimuli of the type we will use in our experiment.
AREN’T MICE HARD TO TRAIN, WITH BEHAVIOUR THAT’S HARD TO REPRODUCE?
Progress in neuroscience has previously been hindered by poor reproducibility of mouse behavior. In our standardized behaviour paper we show that in a visual decision making task, reproducibility can be achieved by automating the training protocol and by standardizing experimental hardware, software, and procedures. We trained 101 mice in this task across seven laboratories at six different research institutions in three countries, and obtained 3 million mouse choices. In trained mice, variability in behavior between labs was indistinguishable from variability within labs. Psychometric curves showed no significant differences in visual threshold, bias, or lapse rates across labs. Moreover, mice across laboratories learned the task on average in 14 days, reaching strong psychometric performance with low lapse rates.
How is the IBL ORGANISED?
The vision of the IBL is a collective working democratically toward a common scientific goal. It aims to operate as a much as possible as a peer-to-peer collaboration, with minimal top-down control and bureaucracy. The IBL organization consists of the following bodies:
Scientific Advisory Board (SAB) is the external reviewing body.
The General Assembly (GA) is the IBL wide policy- and decision-making body, consisting of all Principal Investigators (PIs) plus a representative of the postdoctoral fellows.
Executive Board (EB) is the day-to-day executive body that responsible for successfully executing the objectives determined by the GA. The EB is comprised of the Facilitator, who acts as chair of the EB and GA meetings, a Spokesperson, a Resource Strategist and the Working Group chairs.
The Working Groups (WG)s are the organisation bodies that do the work of the IBL, each governed by a WG chair.
How are decisions made in the IBL?
Decisions are made by a consent decision-making process. Consent decision-making adheres to the principle of reasoned argument and equality amongst members. All members have an opportunity to provide comments in the form of concerns or objections. An objection is a well-reasoned argument why the proposal would impede progress toward the goals of the collaboration. A proposal is modified using this input and is accepted once objections are resolved, aiming to develop “good enough for now” solutions. In the Working Groups this process is the operating practice, but informal, however, in the General Assembly (GA) it is the formal decision making process. Proposals can be put forward to the GA from any member of the IBL, and accepted proposals govern the IBL.
How is daily work organized in the IBL?
The work of the IBL is done in Working Groups (WGs), which may be created temporarily or last for the life of the project. These groups may be spread across locations, but have a simple, common goal. Each work group is lead by a Working Group Chair and the members conduct the work of the group, meeting regularly as a group and using collaboration tools such as Slack and Google Apps for day-to-day communication and collaboration. The WG Chair acts as the WG facilitator, forging consensus and communicating with the Executive Board.
Who oversees the IBL from the outside?
The Scientific Advisory Board (SAB) is an independent body responsible for the reviewing the overall progress of the IBL toward its objectives. It is also responsible for the regular evaluation of the contribution of individual members and other programs to the collaboration goals.
What are the core Working Groups?
The permanent working groups in the IBL are:
Governance: interpretation of IBL policies and governance structures
Publications & Postdocs: policies regarding publications and authorship of IBL research papers; postdoc issues
Data architecture: data sharing and storage infrastructure
Behavior: common behavior rig, standardization of animal handling, and full automation of behavioral task
Physiology: recording rig and matrix for the large-scale Neuropixels map; video-based behavioral analysis
Histology: histological identification of recording sites
Behavioural Analysis: pipeline for preprocessing and shared analysis of behavioral data
Neural Analysis: pipeline for preprocessing and shared analysis of neural data
Theory: theoretical analyses and modeling
Outreach: coordinating communication of our resources and findings with the scientific and lay communities
How does the work of the IBL get shared internally?
The IBL follows three key internal policies:
All experimental data is automatically shared amongst the collaboration.
Planned experiments and analyses are shared within the collaboration through a registration process.
All tools and reagents are freely shared throughout the collaboration, fostering replicability and best practices throughout the collaboration.