With a data analysis plan, you know what you’re going to do when you actually sit down to do the analysis of the data you’ve gathered. It’s a vitally important thing for you to have, as it will guide how you’re going to collect your data. After all, it’s very difficult to add in new variables afterward.
For that reason, you want to make sure you’ve created your plan beforehand so that you can be sure that you’re asking all the questions you need to and you know what you’re going to do. Sure, as they say, a plan only lasts until the first shot is fired. And yes, that’s also true in data analysis. Nonetheless, having a good plan can save you a great deal of time, while having a bad one (or even worse, none at all) at best means you’ll be struggling to make sense of the data and at worst will make you realize your data is worthless as you forgot to collect a crucial variable.
To make sure your plan rocks, follow these hints and tips.
As they say, you need a minimum of about 20 participants per cell to register any kind of effect. So if you’re doing a 2 X 2 design (Which is really quite common) you’ll need at least 80 participants.
So if you’re looking for a gender effect (which is the first ‘2’) and you expect it to be moderated by whether they went to college or not (the other ‘2’) you’ve got yourself a good old 2 X 2 design and will need at least 80 participants.
And, truth be told, that still will mean you’ll miss most of the time.
A much better bet is to go with double that number per cell. More if you’ve got the time and the inclination. In that way, you’ll be far more likely to actually find some kind of an effect. And that’s a good thing, as it’s far more fun to find something as that will give you something to write about (and possibly might give you a reason to publish).Â
Some people think it’s silly to draw up dummy tables, seeing as they don’t have any statistics. I disagree. The tables and the figures can be immensely helpful in that they can unearth assumptions that you may be making in your model that you weren’t aware of.
And that’s vital, as these assumptions might lead you down the garden path if not addressed, leading to your data collection not creating any significant results, because you forgot to measure some dimension or because you didn’t think carefully enough about what was going on.
So draw up the figures and don’t just put nonsense into them. Instead, try to draw them up in a realistic manner and work out what you would need for them to work out in that way. Chances are, that will help a lot.
Go crazy. Write down everything that might in some way be related to the variables you want to collect. You think hair color might affect IQ? Then write it down. You think pet ownership might play a role in how creative people are? Write it down. Go a little bit crazy if you have to. Think laterally (something that will help you improve your writing and be more creative besides)
Now, the best thing you can do is write down the connection, the direction and the role of these variables. If you want to do it really well, make sure you quote sources where such a connection has been previously established.
