The influence of cognitive biases on researchers and scientific production: the inner practice of science for science.

In Wikipedia, a cognitive bias is defined as “a systematic pattern of deviation from rationality in judgment”. As any human being, researchers are prone to cognitive biases, which is a critical matter as these biases can lead to an “unsustainable” science. In this talk, and after a brief introduction on cognitive biases, we will see how they can influence our research for instance through perceptual distortion, illogical choice or misinterpretation. Next, we will envisage some remedies to counteract our inner trend for irrationality. Finally, together with the M.E.EG community we will discuss what can be done at a larger scale to reduce the impact of cognitive biases on scientific practices.

Preregistered reports in M/EEG research – a road map through the garden of forking paths?

M/EEG analyses pipelines combine a multitude of processing steps, such as filters, artifact removal, time-frequency transforms, etc., to be chosen by the experimenter. Given that it is impossible to test the independent contribution of each step to the results, novice and even expert neuroscientists are often left with the frustration to not know how strongly a given effect (or the absence thereof) depends on the choices made in their pipeline. Preregistration provides a potential remedy to this problem in that the pre-and post-processing steps are fixed before the study is conducted, and, importantly, expert feedback can be obtained early.

Based on recently obtained ’in-principle-acceptance’ for an EEG replication study, assessing the seminal finding of enhanced delta phase coherence by temporal predictions (Stefanics et al. 2010), I would like to discuss to which extent preregistration can foster replicable and robust EEG/MEG research, and help the community to devise less user-dependent pipelines.

BIDS: a data standard to support good scientific practices in neuroimaging

The Brain Imaging Data Structure (BIDS) is a community-led standard for organizing, describing and sharing neuroimaging data. Currently, it supports many neuroimaging modalities, such as MRI, MEG, EEG, iEEG and more to come. Multiple applications and tools have been released to make it easy for researchers to incorporate BIDS into their current workflows, maximising reproducibility, data sharing opportunities and supporting good scientific practices. This talk will share an overview of the BIDS current status and related tools to facilitate this endeavour.

Open science for better AI

In the last ten years, publications on artificial intelligence (AI) have nearly doubled, opening promising avenue for an increased use of AI in our daily life. One key feature and challenge of AI is the generalizability of algorithms, notably for (inverse-)reinforcement learning or transfer learning (Arora and Doshi, 2019). Alike many AI research, these areas have a common root in neuroscience and psychology (Hassabis et al., 2020). Although working on freely available databases is becoming practice in computer science (e.g. ImageNet), it is not the case for neuroscience and psychology (except for the brain-computer interface community; e.g. BNCI Horizon 2020 project). Yet, a recent brain imaging study (Botvinik-Nezer et al., 2020) revealed the limits of reproducibility by sharing the same fMRI datasets with 70 research teams who analyzed them and identified consistent results for only 4 out of the 9 hypotheses tested. In the case of electroencephalography (EEG), generalizing data recording, processing and analysis is prevented from by: i) a lack of experimental design sharing; ii) recording format and parameters are not standardized (EEG systems have their own data format, compatibility and amplifier parameters); iii) processing pipelines are variable and not shared; iv) difficulties to store and share large datasets (Martinez-Cancino et al., 2020). We propose that following a general pipeline for setting-up experiments, collecting data, processing them and sharing files can increase the number of standardized datasets thus facilitating AI algorithms development for generalizability. This pipeline should be based on open access tools (LSL, BIDS, MNE, etc.). Sharing data allows AI researchers to improve modelling and machine learning algorithms quality, standardize algorithm performances comparison and allows laboratories with less means to perform quality research; but also it generalizes the possibility to get feedback from the whole community in an easier manner.

Best Practices in Clinical MEG – Patient Preparation and Data Acquisition

We present best practices for the MEG recording for patients with epilepsy, from our years of experience in conducting over 3,000 patient exams at Cleveland Clinic and UT Houston. We emphasize the preparation of the patient and the setup of MEG instrument to ensure a quality clinical recording. Several practices are quite general for any MEG instrument, such as generally insisting on MRI compatible gowns, careful acquisition of landmarks on the patient’s scalp, and the daily recording of empty room data. The techniques address a gap in the clinical literature addressing the multitude of potential sources of error during patient preparation and data acquisition, and how to prevent, recognize, or correct those.

ERP CORE: An Open Resource for Human Event-Related Potential Research

Event-related potentials (ERPs) have broad applications across basic and clinical research, and yet there is little standardization of ERP paradigms and analysis protocols across studies. To address this, we created ERP CORE (Compendium of Open Resources and Experiments), a set of optimized paradigms, experiment control scripts, data processing pipelines, and sample data (N = 40 neurotypical young adults) for seven widely used ERP components: N170, mismatch negativity (MMN), N2pc, N400, P3, lateralized readiness potential (LRP), and error-related negativity (ERN). This resource makes it possible for researchers to 1) employ standardized ERP paradigms in their research, 2) apply carefully designed analysis pipelines and use a priori selected parameters for data processing, 3) rigorously assess the quality of their data, and 4) test new analytic techniques with standardized data from a wide range of paradigms.

Designing analysis code that scales

Small projects grow up to be large projects. That simple analysis script you started out with was so innocent, so uncomplicated, so pure. Where have those days gone? Staring at your code folder now is like staring into the Abyss…and it stares back! In this talk, we’ll be thinking about how to organize code in such a way that things scale as we add more analysis steps, collect more data, and explore more avenues. Complexity is our biggest enemy. It must be fought every minute of every day. Lose sight of this for only an instant and it will grow out of your control… We will use an EEG analysis pipeline as an example to see how we can compartmentalize complexity to contain the monster. I’m going to show you how to be faster, more accurate and generally more happy with your analysis code. Many of the ideas in this talk are outlined in the paper: https://doi.org/10.1371/journal.pcbi.1007358

What to do and **not** to do with MNE-Python?

Magnetoencephalography and electroencephalography (M/EEG) measure the weak electromagnetic signals induced by brain electrical activity. Using these signals to characterize and locate brain activations is a challenging task as confirmed by three decades of methodological contributions. MNE, which holds its name from its historical ability to compute minimum norm estimates, is a
software package that addresses this particular challenge by providing a state-of-the-art analysis workflow spanning preprocessing, various source localization methods, statistical analysis, and estimation of functional connectivity between distributed brain regions. This talk aims to present good practices when working in Python and in particular using MNE. I will mention suggestions for development environments, how to efficiently organize, profile or debug your code etc. Full documentation, including many
examples and tutorials, is available at https://mne.tools/.

MEG and EEG in the age of machine learning: Data-driven versus Hypothesis-driven science

The recent proliferation of studies using brain decoding and data-driven methods has brought a lot of interest and excitement to the field of MEG and EEG. Amid strong claims about the endless power of machine learning (ML) techniques, including data representation learning, legitimate questions arise: Is the age of good-old hypothesis-driven neuroscience coming to an end ? What is the added value of ML for MEG/EEG research ? What are the good practices and pitfalls of ML analytics that we need to be aware of? And finally, how will I be able to discuss all this in twenty minutes ?

Statistical power in MEG data: spatial constraints for a reliable study

Statistical power is key to robust, replicable science. Here I will first remind everyone of the definition of statistical power, why it is important in data analysis, and in particular in MEEG. I will then highlight what increasing power often boils down to when planning an experiment, namely increasing trials and subjects number. We recently manipulated these variables in simulated experiments to find out the ideal number of trials and subjects. The answer is – as often – more complex than the question, and a step back is required here because many parameters come into play. In this study, we focus on a specific set of parameters: the spatial properties of the expected neural sources. I will illustrate the well-known effects of distance and orientation, as well as the less well documented effects of inter-subject variability on statistical power. Finally, I report on the effect of simple and widely used manipulations such as squaring the data before testing on statistical power. Generally, I want to advocate here for critical scrutiny and attention to expected sources of variability while planning an M/EEG experiment.

Does reporting of ERP methods support replicable research? The current state of publication for N400s and beyond

For decades, researchers in the fields of EEG and ERP have made calls for better transparency in the arena of reporting EEG/ERP data (e.g. Donchin et al., 1977, Picton et al., 2000; Keil et al., 2014; Duncan et al., 2009; Kappenman & Luck, 2016, Taylor & Baldeweg, 2002; Luck, 2014; Luck & Gaspelin, 2017; Gelman & Loken, 2013). Despite the availability of guidelines for transparent and accurate reporting, the state of the published literature suggests that authors are either unable or unwilling to follow current reporting guidelines. We propose that by leveraging the collective expertise of stakeholders in the ERP community, we can create an agreed reporting framework that is both easier to use and more transparent than current reporting models.

Background. One recent systematic review (Šoškić, Jovanović, Styles, Kappenman & Ković, 2019: Preprint https://psyarxiv.com/jp6wy/) has demonstrated that published journal articles rarely contain all of the information that would be required to replicate an N400 study, and that the reporting was inconsistent and often ambiguous, making it hard to critically assess, metaanalyse and replicate work. This project and its sister projects (Šoškić, Kappenman, Styles & Ković, Preregistration 2019: https://osf.io/6qbjs/; Ke, Kovic, Šoškić, & Styles, Preregistration, 2020: osf.io/5evc8) show that improvements to reporting standards in the ERP field will be necessary to increase replicability and reduce questionable research practices.

Current Presentation. On the basis of our metanalytic work, we are drafting a metadata template (i.e., a digital form or template to be filled), containing all of the metadata critical to an ERP recording and the subsequent processing and analysis chain. In contrast to the valuable work of Keil et al. (2014) and Pernet et al. (2018), who developed checklists for authors to indicate whether they have reported appropriate methodology details in the body of their article, the metadata template requires precise numerical/categorical data to be filled, thereby ensuring searchability and simplicity in future metascience. To facilitate transparency in this process, we want to engage the community of stakeholders in EEG/ERP research, in a consultative process to refine the template, and move towards an agreed reporting framework – we envisage a consultative process that involves frontline researchers, software developers, data archivists, journal editors and researchers with experience in open, replicable science.

Changing scales for a changing science: philosophical, sociological and ethical considerations

In the last decades we have assisted to an impressive increase of numbers across most scientific communities. More electrodes, more participants, more data, more methods, more publications, more scientists. To this extent the scales of science are changing. But how do they evolve? What are the factors that exert selection pressure? And what is the added value?

I will possibly give two short examples from art and literature to show that large scales and large numbers are not necessarily more informative than small scales and small numbers. I will then show some quantitative analyses of how long it takes us to publish, how much we publish, how science communication media change, how many people we are, how much science costs, how much we pollute etc…

While I am not an expert in the field, I will be happy to share with an informed scientific audience my modest knowledge and the many questions that emerge from such a reflection. Here is one, quoting Marcel Proust: “The real voyage of discovery consists, not in seeking new methods, but in having new eyes” (the word in italic is mine, the original is “landscape”).