Saturday, December 6, 2014

Who's studying who? Demand characteristics in psychological research


Trolls are psychopaths! A recent study understandably gathered a lot of attention by showing that those who enjoyed trolling were more likely to report higher levels of psychopathic traits. (I'm sure much of print media was keen to use these findings to take a dig at the online world which has cost them so much income). However, if asked to complete questions about enjoyment of trolling, followed by questions of how much of a psychopath one is, one does not need a Bachelor's degree in psych to guess that the researchers might be trying to see if online trolls are, in fact, psychopaths. So, being the poo-stirring troll that you are, might it not be fun to exaggerate your level of antisocial behaviour out of spite, just to enjoying the resultant media poo-storm?

Q 13. Do you have any further comments on your experience of trolling? (Please write these into the space below).

A 13. uruururur how i enjoin throwing kats in de bornfire with ma knuckly-dragging companions, den boastin bout it online to liberaln00bs wuh wuh wuh (Written by someone who wouldn't say boo to a cat in real life)

Perhaps a little unfair to pick on this one study, as this is a pervasive problem in psychological research. Demand characteristics are changes in behaviour which are due to the person taking part in the study seeing what the researcher is trying to show, and then changing their behaviour accordingly.

There are various ways to get around or minimise demand characteristics in human research. One handy method is to use a placebo condition. Say if you were studying caffeine it would be relatively easy to have two pills which look identical, but only one contains caffeine. Better yet, go double blind, so that the researcher (whichever one interacts with participants) doesn't know which condition they're in. However, for a lot of psychological research this is not so easy to pull off. If you're looking at the effects on depression of a type of psychotherapy versus a waiting list control, people can see whether or not they're getting the treatment. One way to address this is to have an active control, where people do engage in some kind of intervention (say a group meeting with other people with depression), so that there will still be demand characteristics, but the key ingredient of your new psychotherapy is missing.

It was cool to see an article taking this issue a bit further. Boot et al. suggested that in addition to having an active control condition, one should also look at the extent to which the active control and the actual intervention under study are associated with an expectation of an effect. In this way one can get a better sense of whether any change is due to a positive effect of the intervention or just an expectation that it will do as much. At the same time, one needs to watch out if the more novel therapy is perceived as what the researcher is really interested in, as even with equivalent expectations participants may still display more positive effects if they correctly perceive which one is the experimental  condition.

When doing studies about chewing gum it occurred to me that there was no placebo condition.  Unlike the caffeine pill scenario, there isn't really a strong equivalent with chewing gum; people can tell if they're chewing or not. I was surprised at the relative lack of research on demand characteristics in psychology more generally (perhaps it's a Pandora's Box), so I conducted a study of my own with Prof Andy Smith. People taking part were given chewing gum, rated their level of alertness and completed attention tasks. For the "positive" condition, I told them at the beginning that previous research had shown a positive effect of chewing gum on alertness and attention (I also chewed during this condition-always the method actor). In the "negative" condition I told them previous research had shown a negative effect. There was also a neutral condition where no explicit prompting was given.

We found that people in the negative condition reported lower alertness in response to chewing gum than those in the positive condition or the neutral. The lack of a serious difference between the neutral and positive conditions was interesting, although I'm still undecided if that's due to a genuine effect or the mere presence of some manipulation being enough to induce demand characteristics. In contrast to the observed effects on reported alertness, there was no performance difference in terms of attention. If people weren't showing better sustained attention when told that gum was good, this suggests they may have only been reporting greater alertness rather than actually experiencing it. I suppose the lesson is that if you actually have some kind of overt behaviour in your study this will be less vulnerable to demand characteristics.

While designing the study I recall watching the film "Inception". The plot concerns people who can enter other people's dreams. Furthermore, they can enter dreams within these dreams. I sometimes have trouble finding films escapist, as I can end up thinking about how some aspect of the film could say something about my own life. In this case, I started to think about demand characteristics within demand characteristics-what if people perceived that the study was about demand characteristics, and started responding so as to confirm my hypothesis that demand characteristics impact upon performance? A senior academic at Cardiff brought up this point as well. Unfortunately, I have no easy answer. I did ask people at the end of the study if they could guess what the hypothesis was, and only one person guessed correctly, although maybe some people did and decided that they'd keep quiet, and let me hear again what I wanted to hear.....

Allen, A. P., and A. P. Smith. (2012). Demand characteristics, pre-test attitudes and time-on-task trends in the effects of chewing gum on attention and reported mood in healthy volunteers. Appetite, 59(2), 349-356.

No comments:

Post a Comment