What is Meta-Research?
Meta-research is the study of research. Why study research? The last decade has seen an increasing awareness that much academic research may be unreliable. Incentives that reward researchers to produce novel and “important” results can distort the findings that are reported in journals. They can cause researchers to magnify findings that generate rewards, and downplay or not report findings that are not rewarded.
Even when researchers themselves are not influenced by these incentives, the academic publishing business has an incentive to disproportionately report headline-grabbing, “important” results in order to maximize impact and influence. The end result is that research reported in academic journals can become disconnected to the real world it purports to represent. When that happens, the production of academic knowledge is unable to achieve its goal of improving people’s lives.
How prevalent is the problem of unreliable science? This question has attracted much attention since 2005 when John Ioannidis published his seminal article, “Why Most Published Research Findings Are False.” It is now known that the results of many influential, highly-cited studies cannot be reproduced. This has come to be known as “The Replication Crisis”. Evidence of the irreproducibility of published research has been found in psychology, medicine, education, neuroscience, economics, and marketing and management.
Meta-research studies the extent of the problem, the underlying sources of irreproducibility, and ways in which science and academic publishing can be improved to produce more reliable results.
UCMeta is a network of researchers interested in meta-research. Our research approach is characterized by:
- the use of state-of-the-art techniques to synthesize knowledge within and across disciplines
- conducting replications of influential research
- encouraging the practice of open science in both research and teaching
- investigating the extent of science’s irreproducibility problem
- proposing solutions to improve scientific reliability
At the local level, we aim to work with stakeholders in Ōtautahi Christchurch and Waitaha Canterbury, including Ngāi Tūāhuriri and Ngāi Tahu, to identify quantitative and qualitative research that supports evidence-based decision-making. We will apply advanced research synthesis methods, including meta-analysis, to analyse existing research that is focused on identifying successful policy initiatives. We will supplement this with replication of existing research to confirm its soundness for evidence-based policy-making.
Further, we will partner with stakeholders to make data and supporting research materials transparent and open access. One of the great hindrances to research integrity is that researchers do not commonly make their data and supporting materials openly available and transparent. We will work with our partners to set up processes that make this normative. This will multiply the impact of the research by allowing other researchers to re-analyse and extend the data.
By pooling resources across departments and colleges, we will facilitate transdisciplinary research. For example, we will combine meta-analysis skills from researchers in the Department of Economics and Finance with area-specific knowledge from researchers in the Department of Management, Marketing and Entrepreneurship (Business School); School of Psychology, Speech and Hearing (College of Science); the School of Health Sciences (College of Education, Health & Human Development) and Te Rū Rangahau (Māori Research Lab) and elsewhere. The resulting, cooperative efforts will generate new collaborations across disciplines that support a meta-disciplinary approach to solving local and global challenges.
(All meetings are from 12-1pm on Tuesdays, Meremere 236 unless otherwise noted)
- 16 February 2023 (Thursday, Rehua 528, 12-1:30) | What is the Replication Crisis in Science and What Can We Do About It? | Speaker: Andy Vonasch (UC)
- 23 May 2023 | Can More Credible Standard Errors Exacerbate Bias in the Presence of Selective Publication? | Speaker: Patrick Vu (Brown University, USA)
- 29 August 2023 | Results from the Many Analysts Project in Ecology and Evolutionary Biology | Speaker: Hannah Fraser (University of Melbourne, Australia)
- 10 October 2023 | Title TBC | Speaker: Abel Brodeur (University of Ottawa, Canada)
- 14- 16 November 2023 UC Meta Colloquium
- 15 November 2023* | The REPRISE project: An Evaluation of Reproducibility and Replicability in Syntheses of Evidence | Matt Page (Cochrane, Monash University, Australia)
- 20-24 November 2023 | Scholar-in-residence | Tomas Havranek (Charles University, Czechia)
*Date yet to be confirmed
Funders and Partners
- UC Business School Research Committee
- Center for Open Science, SCORE Project