Open Research (PY0663)… .

class: center, middle, inverse, title-slide

.title[
# Open Research (PY0663)… .
]
.author[
### Prof. Thomas Pollet, Northumbria University (<a href="mailto:thomas.pollet@northumbria.ac.uk" class="email">thomas.pollet@northumbria.ac.uk</a>)
]
.date[
### 2025-11-03 | <a href="http://tvpollet.github.io/disclaimer">disclaimer</a>
]

---

<div>
<style type="text/css">.xaringan-extra-logo {
width: 110px;
height: 128px;
z-index: 0;
background-image: url(NU2.png);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
top:1em;right:1em;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.title-slide):not(.inverse):not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('div')
          logo.classList = 'xaringan-extra-logo'
          logo.href = null
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

## Outline

* Why open research?

* OSF.io - platform

* Escape room!

---
## Who am I?

* trained in Sociology (Ghent) --> Evolutionary Psychology (Liverpool, NCL).

* worked in Social Psychology depts. in Netherlands (Groningen, VU Amsterdam, Leiden)

* At Northumbria University (dept. of Psychology) since 2017. I teach statistics at MRes. level - Open Research advocate.

???
IL: 'Top Down': 'Senior management'; I meet with and develop policy with UKRN, promote UKRN, liaise with PVC on UKRN goals, and deliver training - I am independent from Local Network Lead (and technically this role is also independent from the University)

---
## Recent history

* Why care about reproducibility?

* I was there when it happened -- sort of ... .

* Similar to [Erdős](https://en.wikipedia.org/wiki/Paul_Erd%C5%91s) number (how many links away from the mathematician):

--> Stapel = 2; Förster = 2; Smeesters = 2; Wansink = 2; Gino = 2;...

---
## Diederik Stapel

* Stapel was in Groningen and then moved to Tilburg.

* In 2011, case came to light via student whistleblowers.

* Stapel generated data from his PhD students

* 58 retractions, professorship + PhD rescinded.

---
## Some further background reading

* <a name=cite-Goldacre2009></a><a name=cite-Goldacre2014></a>[Goldacre and Farley (2009)](#bib-Goldacre2009); [Goldacre (2014)](#bib-Goldacre2014) : Bad Science / Bad Pharma

* <a name=cite-Chambers2017></a>[Chambers (2017)](#bib-Chambers2017) : Seven deadly sins

* <a name=cite-Ritchie2020></a>[Ritchie (2020)](#bib-Ritchie2020) : Science fictions

* [Daniel Nettle's blog](https://www.danielnettle.org.uk/)

---
## Broader phenomenon

Publication pressure... .

---
## Key findings: Replication project

**Key Findings**:

- **97%** of original studies had statistically significant results

- Only **36%** of replications had statistically significant results

- Replication effect sizes were **half** the magnitude of original effects

- **47%** of original effect sizes were in the 95% CI of replication effect size

- **39%** of effects were subjectively rated to have replicated

---
## Many labs 2

Northumbria staff helped (e.g., Matthew Haigh, Nick Neave, Michael Smith)

---
## So, what is open research.

From Munafò et al. (2017) _Nat Hum Behav_:

> "The process of making the content and process of producing evidence and claims **transparent and accessible to others**" to enable the reproducibility and replicability to a research claim to be evaluated

**Reproducibility:** Given the same evidence (i.e., data) and the same analysis, can the same result be obtained?

**Replicability:** If a study is repeated with the same procedures, therefore generating new data, will the same finding be obtained?

---
## Open research

As psychological scientists, it is important that we uphold some key principles of scientific endeavour: openness, transparency, rigour, reproducibility, replicability and accumulation of knowledge – these principles capture the ideals of ‘Open Science’ (/Open Research)

---
## Threats to reproducibility and replicability

---
## YOU!

* **Pre-registration**: 
  - Specify your research plan and analyses upfront 
  - Specify whether research is exploratory or confirmatory

* **Open data**: Make data and analysis code openly available on a repository in accordance with FAIR principles (Findable, Accessible, Interoperable, Reusable)

---
## How???

---
## Set up a project

---
## Registrations

---
## Registrations (cont'd)

---
## Registrations (cont'd)

---
## Store files

---
## Store files (cont'd)

---
## Make public!

---
## Some myth busting...

---
##  Myth: If I preregister, I'll get 'scooped'.

Actually the **opposite** is true:

- Preregistration is **date stamped**

- You can stake your claim to your ideas **earlier**

- Preprints establish **first publication** of a result/idea

--> If you are really, really, really worried, you can embargo your pre-registration.

---
## Myth: Open science prevents exploratory research

Open research principles merely require you to:

- Establish **upfront** whether research is exploratory or confirmatory

- Prevents falsely claiming work is confirmatory

- Prevents HARKing (Hypothesizing After Results are Known)

---
## Myth: Preregistration prevents me from changing plans

* **Changes can be documented**: Update button on OSF. Document the rationale for any deviations from your original plan. Or make a separate document and upload this.

* But it is important to be transparent about changes.

* Your preregistration is a plan, not a prison!

---
## Myth: I have to share ALL my data

Sometimes not possible to share everything, for example:

- Commercial sensitivity

- Preservation of anonymity

- Ethical constraints

-->  **"As open as possible, as closed as necessary"**

---
## Myth: Open research not relevant for qualitative data

* There is a qualitative research template.

* Speak to your supervisor. "_As open as possible, as closed as necessary._"

---
## Open science escape room?

Can you escape?

This works best on a larger screen (so not a phone). Loading time is determined by your browser,... .

[https://tvpollet.github.io/shiny_live_open_science_escape_room](https://tvpollet.github.io/shiny_live_open_science_escape_room/)

---
## Any Questions?

[http://tvpollet.github.io](http://tvpollet.github.io)

Bluesky: [https://bsky.app/profile/tvpollet.bsky.social](https://bsky.app/profile/tvpollet.bsky.social)

---
## Acknowledgments

I am greatly indebted to my colleagues who helped me along the way. This talk is (in part) based on slides from [Dr. Michael Smith](https://www.northumbria.ac.uk/about-us/our-staff/s/michael-smith/).

Slides were built with [Xaringan](https://github.com/yihui/xaringan) and [XaringanExtra](https://pkg.garrickadenbuie.com/xaringanExtra/). Many thanks to those building packages and supporting the [R](https://www.r-project.org/) eco-system.

You for listening!

---
## References and further reading (errors = blame RefManageR)

<a name=bib-Chambers2017></a>[Chambers, C.](#cite-Chambers2017) (2017).
_The Seven Deadly Sins of Psychology A Manifesto for Reforming the
Culture of Scientific Practice_. Princeton, NJ: Princeton University
Press.

<a name=bib-Goldacre2014></a>[Goldacre, B.](#cite-Goldacre2014) (2014).
_Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients_.
Macmillan. ISBN: 0-86547-806-6.

<a name=bib-Goldacre2009></a>[Goldacre, B. and R.
Farley](#cite-Goldacre2009) (2009). _Bad Science_. Fourth Estate
London. ISBN: 0-00-728487-X.

<a name=bib-Munafo2014></a>[Munafò, M., S. Noble, W. J. Browne, et
al.](#cite-Munafo2014) (2014). "Scientific Rigor and the Art of
Motorcycle Maintenance". In: _Nature Biotechnology_ 32.9, pp. 871-873.
ISSN: 1546-1696. DOI:
[10.1038/nbt.3004](https://doi.org/10.1038%2Fnbt.3004).

<a name=bib-Ritchie2020></a>[Ritchie, S.](#cite-Ritchie2020) (2020).
_Science Fictions: Exposing Fraud, Bias, Negligence and Hype in
Science_. New York, NY: Random House. ISBN: 1-4735-6425-5.