Open science and a collaborative ethic in research: Motivations, barriers, and benefits

Supervisor: Professor Emma Marsden
(with a co-supervisor if the chosen subdomains are beyond applied linguistics – see below)

A) Rationale for the project

Open science practices involve making the processes and products of research freely available for scrutiny by all. Open science can include making available the materials and procedures used to collect, code, and analyse data, as well as the data and final reports themselves. Across many disciplines, including within social sciences, such practices are increasingly encouraged via incentives from governments, funders, universities, journals and publishers. Open science is considered desirable for reasons relating to (a) social equity (publicly paid research should be available to the public), (b) the quality of research (rigour, validity and reliability), and (c) the rate of progress (allowing more and better replication). However, despite many calls over several decades, research communities are slow to react, in part due to a lack of data. This PhD would be among the first studies to provide hard data about attitudes towards and benefits of open science.

B) References that should be read

Derrick, D. J. (2016). Instrument reporting practices in second language research. TESOL Quarterly, 50, 132–153. https://doi.org/10.1002/tesq.217

Marsden, E. J., and Kasprowicz, R. E. (2017). Foreign Language Educators’ Exposure to Research: Reported Experiences, Exposure Via Citations, and a Proposal for Action. The Modern Language Journal, 101(4), 613–642. DOI: 10.1111/modl.12426

Marsden, E., Mackey A., and Plonsky, L. (2016). The IRIS Repository: Advancing research practice and methodology. In A. Mackey and E. Marsden (Eds.), Advancing methodology and practice: The IRIS Repository of Instruments for Research into Second Languages (pp. 1–21). New York, NY: Routledge.

Marsden, E. J., Morgan-Short, K., Trofimovich, P., Ellis, N. (2018) Introducing Registered Reports at Language Learning: Promoting Transparency, Replication, and a Synthetic Ethic in the Language Sciences [Editorial]. Language Learning 68 (2)

Marsden, E. J., Morgan-Short, K., Thompson, S., and Abugaber, D. (2017). Replication in second language research: Narrative and systematic reviews, and recommendations for the field. Language Learning, 68(2).

Marsden, E. J., Thompson, S., and Plonsky, L. (2017). A methodological synthesis of self-paced reading in second language research: Methodological synthesis of SPR tests. Applied Psycholinguistics.

Morgan-Short, K. Marsden, E., Heil, J., Issa. B., Leow, R., Mikhaylova, A.,… Szudarski, P. (2018). Effects of attending to form while comprehending: A multi-site replication study. Language Learning, 68 (2).

Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35, 655–687. https://doi.org/10.1017/S0272263113000399

Wicherts, J., Bakker, M., and Molenaar, D. (2011). Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLoS One, 6, e26828. https://doi.org/10.1371/journal.pone.0026828

C) Research aims and questions

The project will select one (or a small number) of subdomains of education-related discipline(s), such as applied linguistics, cognitive or social psychology, sociology, or of education research (eg language or science education), and investigate:

why researchers do and do not adopt a range of open science practices
journal editors’ and reviewers’ views and behaviours in terms of promoting open science practices
the benefits of methodological transparency, in terms of increased quality, trustworthiness, and impact

D) Methods

Aims 1 and 2). To investigate the attitudes and behaviours of researchers, editors and reviewers, questionnaires will be developed and administered to approx. 200 researchers, 200 reviewers, and 30 journal editors.

Aim 3). To investigate benefits of open science, we would develop indices of ‘methodological transparency’ (eg instrument and data availability, reporting) and correlate these with indices of ‘quality’ (eg instrument or rater reliability, statistical rigour), ‘trustworthiness’ (readers’ perceptions), and ‘impact’ (eg citation, journal impact factor, extent of replication).

E) Skills and opportunities you could gain

This project is closely linked to the British Academy funded IRIS project, including opportunities for research assistant roles working with large established networks of researchers, editors and professional associations. See www.iris-database.org. The project will also draw on Marsden’s role as associate editor at Language Learning and The Department of Education’s working group on open science.