Contents

Thesis, Introduction.

Hurray! You have embarked on a thesis journey with me. On this page I will provide some links and tips. This is still in progress, all feedback welcome. Do also alert me to dead links, which I will fix. Specifically, I cover the Open Science Framework, Qualtrics, SPSS, and writing. I even explain how you can get a meeting.

Help, I need a meeting.

Image help

Below applies to Masters students - for Bachelor students, you will have prebooked sessions

You can find my availability here. Before the meeting make sure you have uploaded all the relevant documents to the OSF. Please do so around two working days, beforehand. That will give me time to review your documents. I might reply outside of office hours but I do not expect you to.

Some other tips for our meetings:

  • Do you even need a meeting? Before booking a meeting, ask yourself whether you have exhausted all the possible options of fixing an issue yourself. In some cases, it is sufficient to upload a document to OSF, with some specific highlights of what you want me to look at. Ask yourself: What have you done to fix the problem? This is typically what I will also ask during our meeting. If you are stuck with a particular problem (e.g., how to run a certain type of analysis in SPSS), detail what you have done to resolve that problem (e.g., I have looked at chapter X in book Y and while their analysis is similar, I still struggle with Z). Bring with you the materials you have looked at. List specific obstacles. For example: ‘I don’t know if I have run this analysis correctly’ is not a very clear obstacle. Whereas, this statistical assumption seems violated but I am uncertain as to whether that invalidates my conclusion. is a much more specific obstacle.
  • Prepare your meeting. Make sure you have a list of things you want to discuss. You are in charge of your project and as I am handling many projects simultaneously, it is your job to keep oversight of your project.
  • Make notes / summarise. Often at the end of the meeting I will tell you what should be your target goals for the next one, make sure you write those down and remember those.
  • Expectations for statistical analysis. Some students expect that I will analyse their data for them, read all of their output, or run their code. As these are (typically) assessed components, I will not do so. With some exceptions, all the techniques you will employ are techniques covered in your previous courses. If it is something out of the ordinary, such as Ordinal Logistic Regression (Ba dum Tss), I will point you to a resource. I can always look at your data together with you but you would be running the analysis and point out specific problems. In short, you will still have to run your own analysis. Alternatively, if that does not work for you, I will show how a novel technique works with a different dataset or a mock dataset. For techniques which are part of your degree programme (e.g., correlations), I expect you to run your own analysis.

Open Science Framework.

Image science

If you choose to complete your thesis with me, we will use the Open Science Framework. Many reasons exist for using the open science framework but one reason is that we have a record of what we did. It also avoids the issue of having lots of files labelled ‘proposal’ and finding out which one is the final one. Some further background on why we should use the OSF can be found here.

In time, I will post more example projects, but here are already some examples to look at. Some of these projects have been reworked into papers and have been published(!). All my OSF projects are here.

How to set up your account.

You can sign up here. Have a look here or here if you get stuck.

Important: Do not choose institutional login.. Please use your personal email address (you will no longer have access to IT when you leave our institution).

Creating a project.

Have a look here. Ensure that the location it is hosted is Europe rather than U.S., if not we will have to amend the consent form, so that participants are aware where their data are stored.

Setting up a registration.

Have a look here. You can see some examples by browsing the Open Science Framework. Multiple options for registrations exist. If your project is a replication project, then we would follow the ‘replication recipe’. Follow the details in this paper.

Sharing a project

At the end of the project we might want to make it public. Information on how to do that here.

Some tips for Qualtrics.

Image Qualtrics

Qualtrics has an extensive help file for your projects. I list some tutorials, tips, and tricks here.

IMPORTANT: Do not setup or use a Qualtrics trial account, set up your own account as discussed here

Qualtrics: Tutorials

If you have never worked with Qualtrics, set aside some time.

If you click here , you can also download a tutorial (in Powerpoint) by Dale Metcalfe. I suggest you work through this tutorial, you will need these spreadsheets also Link 1 and Link 2.

If you have never made a Qualtrics survey, have a look at this video, this one or this one. Kent state also has an excellent tutorial, if you don’t want to watch videos. This tutorial by Barnard College (Columbia) is also useful.

Also make sure, you have seen this video tutorial : It walks through all the things you need for your thesis, except for randomisation.

Qualtrics: Tips and tricks.

Before starting check out these writing mistakes when making your survey.

  • Unless there are reasons not to: force participants to respond. If you want to still have the option for sensitive questions that participants are able to skip it, add “Prefer Not to Answer” as an option. Here you can see how to do that. Also look at this video.
  • Avoid open questions. For questions like age, nationality, etc. You would end up having to do a lot of post-hoc recoding otherwise. For example, some participants will put in ‘30’, while others will put in ‘30 years’. Have a long think, does that question really have to be open and how will you deal with the variety of responses which you will inevitably get? In most cases, you do not want an open question but sensible categories and perhaps a single free text option which captures any remaining option.
  • Name your questions. Qualtrics will name your survey quesions like Q1, Q2, Q3 etc. Please rename them with something meaningful (e.g., Gender, Age, etc). Qualtrics does not like space or symbols in these names. Name them so you can identify them in your survey.
  • Use page breaks. Make sure participants do not have to scroll endlessly. Use page breaks. Check this video
  • Randomization. Randomize your questions. Check this video. Note that it might disrupt your page breaks. When in doubt make separate blocks.
  • Templates. Rather than starting a survey from scratch, I might have some templates already for complex designs (e.g., social network surveys). Contact me before you start designing your survey.
  • Access. If you are making your own survey, I will require full access to it. Here you find explained how to share it. Make sure I have full rights!
  • Preview. Make sure that you have previewed your survey, does it look OK (also when on a phone? tablet? - if not instruct participants to only complete on a PC/Laptop). Make sure the survey flow works.

Attention checks.

You might want to include some measures to check data quality, such as attention checks, example here.

How many participants should I get? - Power Analysis.

How many participants? Original study

In most cases, you will want to conduct a prospective power analysis to determine your sample size. Ideally, in your ethics application, you will also provide a rationale for the sample size and a power analysis is an excellent way to provide such a rationale. Unfortunately, SPSS was historically not really geared up to easily do the power analyses for most designs you will come across. However, you can now also do power analyses in SPSS.

For your project, I would, however, strongly recommend you would use G*Power, which is freely available here. If you make use of the programme, you should cite it - this is the paper to cite. This page also has information on how to install this software and do some basic power analyses.

There is a detailed guide on power analysis for various designs here. This guide also explains why we do power analysis in the first place. You can also check out this youtube tutorial explaining how to conduct a power analysis. There might be occasions where your design is different to what is in the guide, such as a multilevel design (see here), in that case please do discuss this with me (or your supervisor).

How many participants? Replication study

If you are running a replication project, we follow the recommendation by Simonsohn (2015) and we will take 2.5 times the original study’s sample size. You can cite this paper in your ethics application to justify your sample size.

Experimental Design: checklist.

Some of you will use experimental designs for your thesis, I would highly recommend using this checklist.

Are you measuring the right things: a checklist.

One of the challenges for most studies is defining how a construct will be measured. I would highly recommend using this checklist.

Some tips for SPSS.

There are a lot of useful SPSS tutorials. For example, look at the one from Kent State University or this one. If you get stuck, do not forget that google is your friend. If you run into problems, I expect that you have at minimum googled the problem. When you ask me for help, clearly state what you have already done. In the future, I’ll post common issues listed here.

And if you want a free alternative to SPSS, why not use JASP. A bonus is that the tables are formatted appropriately for your report!

Graph and table misery.

As discussed in my post, I often see terribly formatted graphs (and actually I have been guilty of this myself). However, it is actually fairly easy to improve figures. There thus are no longer any excuses for terribly formatted figures and tables.

Recoding variables.

Have a look here.

Recoding missing with value.

If you did not manage to put a ‘forced’ choice in Qualtrics, you will end up with the rare missings. Often you will want to replace a few missing cases with the means. Have a look here.

Missings.

Have a look here. Note that summing across missings will create problems, look at this.

Conduct a one-way ANOVA.

Check this video.

Conduct a MANOVA

Check this video. Also look at this video and the videos following that. This could also be of use

More advanced SPSS madness.

Logistic regression.

If you are unlucky enough to have to run a logistic regression, then we often want information criteria and/or likelihood ratio tests. I had to find out how to get those. It turns out that the binomial logistic regression module in SPSS does not provide this, but the multinomial logistic regression module does (see this).

Mediation.

I have strong reservations against some of these procedures, that’s for a separate rant. It’s possible that you have to do a mediation, however, as for example, the paper you tried to replicate did one.

There are multiple resources available online on how to do a mediation analyses. For example check out this video and also this video. Here is another example in .pdf using the ‘Sobel Macro’ in SPSS.

Writing.

Image Bojack

Scientific writing.

I will list some resources on how to become a better writer. To be honest, I am a terrible writer and thus not the best placed to offer you advice. The best advice is to write often and rewrite. In the beginning do not waste a lot of time on formatting, save that for the end. Set yourself targets (e.g., 200-500 words for a day) and once you have a terrible first draft, flesh it out. Especially at the start of your project force yourself to write chunks often and then rewrite.

Some further tips:

If you are struggling with writing and starting your essay/thesis also consider booking a session in the Northumbria University library.

Reporting statistics.

APA style is the standard for your report, for most statistics: standardised formats exist. Go back to your statistics notes. Still confused, look at these resources:

As always google is your friend, and you are not the first one wondering how to write up your results in APA style.

Software.

I have also moved away from MS word and towards writing first draft in Markdown, I am still oscillating between RStudio, Macdown and Atom. I also have looked at Scrivener but that is perhaps more suitable for much longer pieces. If I ever find the optimal solution I will let you know.

References.

Use referencing software! Do not waste your time on editing references manually. This is error prone and takes up a lot of your time - which you could spend writing! No excuse not to use one!

Northumbria university offers Endnote. But do spend some time looking into Mendeley or zotero. Papers, although it comes at a cost after a 30 day trial, is also worth having a look at.

If you are using RStudio, it is easy to cite or have a look at papaja.

Secondary Data.

Perhaps inspired by COVID-19 - I decided to list some resources for (student) projects relying on secondary data analysis. You can find these resources here.

There is a wealth of available data, so you might do something more interesting by looking at these data - instead of setting up your own study.

Key resources used for this page.

This is an incomplete list. More to be linked in the future. All the sources are also linked in text.

Kent State University Libraries. (2017, May 15). Qualtrics tutorials Retrieved March 27, 2018, from https://libguides.library.kent.edu/qualtrics.