Developing a scientific rationale

class: center, middle, inverse, title-slide

# Developing a scientific rationale
### Prof. Thomas V. Pollet (@tvpollet), Northumbria University
### 2021-10-05 | <a href="http://tvpollet.github.io/disclaimer">disclaimer</a>

---

## Outline of today.

* Intro.

* Identifying a topic.

* Formulating a research question.

* Summary.

We'll use 'Blackboard collaborate polling' to make it a bit more interactive. Keep an eye out for questions popping up.

--> Goal for today: give you some strategies for developing a rationale.

???
No need to extensive notes. References at end of slides.

---
## Who am I... ?

* I teach [Advanced Quantitative methods](https://tvpollet.github.io/PY0794). Also, I used to be the 'Director of Learning & Teaching: Experience' and taught the option course on human nature. I am our university's open science lead.

* I am an interdisciplinary behavioural scientist and I have published [too many papers](https://tvpollet.github.io/papers). I work mostly on social relationships (friendships, loneliness, social networks, jealousy, attraction in romantic relationships), but have ventured out (e.g., research methodology, sex ratios, left-handedness, body image (height), personality, PTSD, 'cyberpsychology',...)

* Some of the things published are Bachelor projects (!) (needless to say that those students did fairly well.)

---
## Who are you?

So which of the following best describes your proposed interest (pick a word! e.g., addiction, cognition, personality).

**Post in BB Collaborate chat**

---
## Why should this session matter to you?

* You have developed your skills in Year 2: Lab Skills Experience, Research Design, Data Collection/ Collation, Data Analyses, Report Writing, Lab Report Assessment.

* For the next 6 months, you will be applying all those skills in your final-year project.

* Some practical tips and some things from my own experience to help you.

---
## Meeting your supervisor.

* Have you met your supervisor? **Thomas Posts poll question**

a) Yes, have met

b) No, will meet

c) No, not yet contacted them

* Arrange an appointment to discuss your project proposal.

---
## First meeting!

Aim to design a manageable, empirical study:

To discuss: 
  1. Still what you want to do? 
  2. Read the literature?
  3. Theory - hypothesis
  4. What is your research question?
  5. How will your method answer the question?

---
## Identifying a topic (Kinmond, 2012: 26-ff).

* Derive research questions from the literature, instead of: 'armchair speculation' or anecdotal musing.

* Important to take into account the research that has already been conducted in the area. Do a thorough literature search!

* Avoid running the risk of investigating a research question which has been addressed and answered already! --> Some things are 'solved'!

---
## Read and synthesise the literature.

* Start with a paper you found interesting and work your way back. Who do they cite? Who cites them?

* Google Scholar. Use other database such as 'Web of Science', 'EBSCO', 'Pubmed' (medical).

* Good keywords.

* Ask a librarian!

---
## Help!

You have looked and nothing 'relevant' is there.

* Are you super-lucky and did nobody ever look at your research question?

* Did you use a good combination of search terms?

* Discuss with your prospective supervisor, if you find a dearth of research in that area? (Is it... 'solved'? 'out of fashion'? 'bad(?) research'?)

---
## Evaluating the literature.

* Triage: useful, not useful, maybe useful.

* Look out for review papers: e.g., _Annual Review of 'X'_ , _Psychological Bulletin_, _Psychological Review_, _Clinical Psychology Review_ .  --> (Systematic) reviews and meta-analyses.

* Use a reference manager to order your chaos. (Endnote; [Mendeley](www.mendeley.com); [Zotero](www.zotero.org))

* Make a [board](https://www.literatureandlatte.com/scrivener/overview)

---
## The importance of replication.

* You found a study but it's not particularly good! (clues: smallish sample size, incredibly big effects,....)

* It is extremely valuable to replicate earlier published studies. Discuss with your supervisor - Do you think the findings of the original paper hold?

* So your question could just be: does this hold up?

---
## Example 1: Red laptops... .

* Social Psychological research: 'Red' is suggested to enhance attractiveness (e.g., [Elliot & Niesta, 2008](https://www.bryanburnham.net/wp-content/uploads/2014/01/Elliot-Niesta-2008-JPSP-.pdf))

* In an extension, Lin (2014) found: _"Women carrying products in red only affected how men, but not women, perceived them."_

* Photo with woman holding a red laptop (vs. black, blue, or grey laptop) rated as significantly more attractive by the opposite gender.

---
## Replication.

Study 1 carried out by 2 MSc. students in the Netherlands (N = 273).

To the best degree possible, we followed the original study,... .

---
## Extending previous research: Finding clues... .

* Discussion: Often has useful things in it: look for the limitations / future directions section in a paper

* What has been missed?

---
## Easy: Improved measurement.

* Often there is substantial mileage to be gained from testing the same thing but with a better measure. So, close to replication but with a twist.

* For example, you found a study but it is using an unreliable measure. A clue could be the low reliability ( `\(\alpha\)` ) of the measure.

---
## Easy: Different population(s) / differences according to 'X'?

* In the example above, Dutch students instead of Chinese students,... .

* 'Moderators':  is the effect stronger for certain subgroups (e.g., gender, culture, sexual orientation, marital status, ... .)

* Could the effect be explained by a 'third variable'?

* Example: [Regulatory focus and every day risk taking](https://sbe.vu.nl/nl/Images/reg_focus_jw_tcm257-240967.pdf). (More advanced read [this](https://www.amazon.com/Book-Why-Science-Cause-Effect/dp/046509760X))

**Post in chat**

???
Mention 'prevention vs. promotion focus.'

---
## Easy - moderate: A different setting.

* Transpose to a different setting. Let's stick with risk taking for now,... .

* An example from a (successful) Bachelor thesis project I supervised.

---
## An example: Gender differences in every day risk taking behaviour in traffic.

We had previously done studies in the Netherlands: we found men are more likely to take risk in traffic than women do ([Cobey et al., 2013](https://journals.sagepub.com/doi/pdf/10.1177/147470491301100206)).

For example, we observed using lights on one's bike and gender (N = 555 cyclists).

---
## Bachelor project by O'Dowd.

Looking at unsafe crossing towards the metro station ('Jaywalking'), gender, and time of day (presence of daylight).

---
## Bachelor project by O'Dowd.

What do you think we found? **Thomas Posts Question.**

A) A main effect of gender

B) A main effect of time of day

C) Two main effects

D) An interaction effect

Check paper [here](https://lebs.hbesj.org/index.php/lebs/article/view/lebs.2018.65/220).

---
## More difficult: Testing competing predictions! I

* A critical test (Dunbar, 1995). 'Theory A' says 'X', but 'Theory B' says 'Y'. Can you design a study which will allow us to test between A and B? Or what type of tests allow you to find evidence

* An example by [Dunbar et al. (2002)](https://brill.com/view/journals/beh/139/5/article-p695_8.xml): Vigilance in human groups.

* People continuously evaluate their environment ('Vigilance'). The authors tested competing hypotheses ('predation', 'friend searching', 'mate search', 'mate guard').

---
## More difficult: Testing competing predictions! II

--> Want to read more about scientific process and 'strong inference' , see [Platt, 1964](http://houdekpetr.cz/!data/papers/Platt%201964.pdf).

---
## Summary

* Do a thorough literature search!

* Do you 'trust' the finding? How can you improve the 'original' (measurement)?

* What is missing? Are there any potential confounds?

* Can you transpose the study to a different setting or different population?

* Does a competing theory suggest something else?

---
## Any Questions?

[http://tvpollet.github.io](http://tvpollet.github.io)

Twitter: @tvpollet

---
## References and further reading (errors = blame RefManageR)

<cite>Amir, Y. and I. Sharon
(1990).
&ldquo;Replication Research: A &quot;Must&quot; for the Scientific Advancement of Psychology.&rdquo;
In: Journal of Social Behavior &amp; Personality 5.4, pp. 51-69.
ISSN: 0886-1641.</cite>

<cite>Bakker, M., A. van
Dijk, and J. M. Wicherts
(2012).
&ldquo;The Rules of the Game Called Psychological Science&rdquo;.
In: Perspectives on Psychological Science 7.6, pp. 543-554.
DOI: <a href="https://doi.org/10.1177/1745691612459060">10.1177/1745691612459060</a>.</cite>

<cite>Barrett, L. and G. Stulp
(2013).
&ldquo;Four More Reasons Why Ethology Matters: Tinbergen and the Human Sciences&rdquo;.
In: Human Ethology Bulletin 28.4, pp. 39-49.
ISSN: 2224-4476.</cite>

<cite>Baumeister, R. F., K. D. Vohs, and D. C. Funder
(2007).
&ldquo;Psychology as the Science of Self-Reports and Finger Movements: Whatever Happened to Actual Behavior?&rdquo;
In: Perspectives on Psychological Science 2.4, pp. 396-403.
ISSN: 1745-6916.
DOI: <a href="https://doi.org/10.1111/j.1467-8721.2007.00534.x">10.1111/j.1467-8721.2007.00534.x</a>.</cite>

<cite>Chambers, C.
(2017).
The Seven Deadly Sins of Psychology A Manifesto for Reforming the Culture of Scientific Practice.
Princeton, NJ: Princeton University Press.</cite>
---
## More refs 1.

<cite>Cobey, K. D., F. Laan, G. Stulp, et al.
(2013).
&ldquo;Sex Differences in Risk Taking Behavior among Dutch Cyclists&rdquo;.
In: Evolutionary Psychology 11.2, p. 147470491301100.
ISSN: 1474-7049.
DOI: <a href="https://doi.org/10.1177/147470491301100206">10.1177/147470491301100206</a>.</cite>

<cite>Crump, M. J., P. C. Price, R. S. Jhangiani, et al.
(2018).
Research Methods for Psychology: Brooklyn College Edition.
New York, NY: Brooklyn college.</cite>

<cite>Dunbar, K. N. and D. Klahr
(2012).
&ldquo;Scientific Thinking&rdquo;.
In: 
The Oxford Handbook of Thinking and Reasoning.
Ed. by K. J. Holyoak and R. G. Morrisson.
Oxford, UK: Oxford University Press, pp. 611-628.</cite>

<cite>Dunbar, R. I. M.
(1995).
The Trouble with Science.
Harvard, Ma.: Harvard University Press.
ISBN: 0-674-91019-2.</cite>

<cite>Dunbar, R. I. M., L. Cornah, F. Daly, et al.
(2002).
&ldquo;Vigilance in Human Groups: A Test of Alternative Hypotheses&rdquo;.
In: Behaviour 139.5, pp. 695-711.
ISSN: 0005-7959.
DOI: <a href="https://doi.org/10.1163/15685390260136771">10.1163/15685390260136771</a>.</cite>

---
## More refs 2.

<cite>Elliot, A. J. and D. Niesta
(2008).
&ldquo;Romantic Red: Red Enhances Men's Attraction to Women.&rdquo;
In: Journal of Personality and Social Psychology 95.5, pp. 1150-1164.
ISSN: 1939-1315.
DOI: <a href="https://doi.org/10.1037/0022-3514.95.5.1150">10.1037/0022-3514.95.5.1150</a>.</cite>

<cite>Evans, J.
(2007).
Your Psychology Project: The Essential Guide.
London, UK: Sage.
ISBN: 1-84920-832-8.</cite>

<cite>Forsyth, D. R.
(1976).
&ldquo;Crucial Experiments and Social Psychological Inquiry&rdquo;.
In: Personality and Social Psychology Bulletin 2.4, pp. 454-459.
DOI: <a href="https://doi.org/10.1177/014616727600200420">10.1177/014616727600200420</a>.
eprint: https://doi.org/10.1177/014616727600200420.</cite>

<cite>Fudge, D. S.
(2014).
&ldquo;Fifty Years of JR Platt's Strong Inference&rdquo;.
In: Journal of Experimental Biology 217.8, pp. 1202-1204.
ISSN: 0022-0949.
DOI: <a href="https://doi.org/10.1242/jeb.104976">10.1242/jeb.104976</a>.</cite>

<cite>Hamstra, M. R. W., J. W. Bolderdijk, and J. L. Veldstra
(2011).
&ldquo;Everyday Risk Taking as a Function of Regulatory Focus&rdquo;.
In: Journal of Research in Personality 45.1, pp. 134 - 137.
ISSN: 0092-6566.
DOI: <a href="https://doi.org/10.1016/j.jrp.2010.11.017">10.1016/j.jrp.2010.11.017</a>.</cite>

---
## More refs 3.

<cite>Kinmond, K.
(2012).
&ldquo;Coming up with a Research Question&rdquo;.
In: 
Doing Your Qualitative Psychology Project.
Ed. by C. Sullivan, S. Gibson and S. Riley.
London, UK: Sage, pp. 23-36.</cite>

<cite>Kinraide, T. B. and R. F. Denison
(2003).
&ldquo;Strong Inference: The Way of Science&rdquo;.
In: The American Biology Teacher 65.6, pp. 419-424.
ISSN: 0002-7685.
DOI: <a href="https://doi.org/10.2307/4451529">10.2307/4451529</a>.</cite>

<cite>Kuhn, T. S.
(1962).
The Structure of Scientific Revolutions.
Chicago, IL: University of Chicago press.
ISBN: 0-226-45814-8.</cite>

<cite>Lakatos, I.
(1974).
&ldquo;The Role of Crucial Experiments in Science&rdquo;.
In: Studies in History and Philosophy of Science Part A 4.4, pp. 309 - 325.
ISSN: 0039-3681.
DOI: <a href="https://doi.org/10.1016/0039-3681(74)90007-7">10.1016/0039-3681(74)90007-7</a>.</cite>

<cite>Lin, H.
(2014).
&ldquo;Red-Colored Products Enhance the Attractiveness of Women&rdquo;.
In: Displays 35.4, pp. 202-205.
ISSN: 0141-9382.
DOI: <a href="https://doi.org/10.1016/j.displa.2014.05.009">10.1016/j.displa.2014.05.009</a>.</cite>

---
## More refs 4.

<cite>Meehl, P. E.
(1967).
&ldquo;Theory-Testing in Psychology and Physics: A Methodological Paradox&rdquo;.
In: Philosophy of science 34.2, pp. 103-115.
ISSN: 0031-8248.</cite>

<cite>Meehl, P. E.
(1990).
&ldquo;Why Summaries of Research on Psychological Theories Are Often Uninterpretable&rdquo;.
In: Psychological reports 66.1, pp. 195-244.
ISSN: 0033-2941.
DOI: <a href="https://doi.org/10.2466/pr0.1990.66.1.195">10.2466/pr0.1990.66.1.195</a>.</cite>

<cite>Mertens, D. M.
(2019).
Research and Evaluation in Education and Psychology.
London, UK: SAGE Publications.</cite>

<cite>Neuman, L. W.
(1997).
Social Research Methods.
3rd.
Boston, MA: Allyn and Bacon.</cite>

<cite>Pearl, J.
(2009).
Causality.
Cambridge: Cambridge university press.
ISBN: 0-521-89560-X.</cite>

---
## More refs 5.

<cite>Pinker, S.
(2015).
The Sense of Style: The Thinking Person's Guide to Writing in the 21st Century.
London, UK: Penguin Books.
ISBN: 0-14-312779-9.</cite>

<cite>Platt, J. R.
(1964).
&ldquo;Strong Inference&rdquo;.
In: science 146.3642, pp. 347-353.
ISSN: 0036-8075.</cite>

<cite>Pollet, T. V., J. Costello, L. Groeneboom, et al.
(2019).
&ldquo;Do Red Objects Enhance Sexual Attractiveness? No Evidence from Two Large Replications&rdquo;.
In: Displays 56, pp. 23 - 29.
ISSN: 0141-9382.
DOI: <a href="https://doi.org/10.1016/j.displa.2018.10.008">10.1016/j.displa.2018.10.008</a>.</cite>

<cite>Pollet, T. V. and E. O'Dowd
(2018).
&ldquo;Gender Differences in Everyday Risk Taking: An Observational Study of Pedestrians in Newcastle upon Tyne&rdquo;.
In: Letters on Evolutionary Behavioral Science 9.1, pp. 1-4.
ISSN: 1884-927X.
DOI: <a href="https://doi.org/10.5178/lebs.2018.65">10.5178/lebs.2018.65</a>.</cite>

<cite>Pollet, T. V. and T. Saxton
(2019).
&ldquo;Jealousy as a Function of Rival Characteristics: Two Large Replication Studies and Meta-Analyses Support Gender Differences in Reactions to Rival Attractiveness but Not Dominance.&rdquo;
In: PsyArXiv.
DOI: <a href="https://doi.org/10.31234/osf.io/28yvp">10.31234/osf.io/28yvp</a>.</cite>

---
## More refs 6.

<cite>Popper, K.
(1959).
The Logic of Scientific Discovery.
London, UK: Hutchinson.
ISBN: 1-134-47002-9.</cite>

<cite>Punch, K.
(2000).
Developing Effective Research Proposals.
London, UK: Sage.
ISBN: 0-7619-6356-1.</cite>

<cite>Simmons, J. P., L. D. Nelson, and U. Simonsohn
(2011).
&ldquo;False-Positive Psychology&rdquo;.
In: Psychological Science 22.11, pp. 1359-1366.
ISSN: 0956-7976.
DOI: <a href="https://doi.org/10.1177/0956797611417632">10.1177/0956797611417632</a>.</cite>

<cite>Tinbergen, N.
(1963).
&ldquo;On Aims and Methods of Ethology&rdquo;.
In: Zeitschrift für Tierpsychologie 20.4, pp. 410-433.
ISSN: 1439-0310.
DOI: <a href="https://doi.org/10.1111/j.1439-0310.1963.tb01161.x">10.1111/j.1439-0310.1963.tb01161.x</a>.</cite>

<cite>Zwaan, R. A., A. Etz, R. E. Lucas, et al.
(2018).
&ldquo;Making Replication Mainstream&rdquo;.
In: Behavioral and Brain Sciences 41, p. e120.
ISSN: 0140-525X.
DOI: <a href="https://doi.org/10.1017/S0140525X17001972">10.1017/S0140525X17001972</a>.</cite>