Quantifying the qualitative

Cole Davis

Cole Davis

Cole Davis is the author of SPSS step by step: Essentials for social and political science, published by The Policy Press on 13 February 2013, the unique and comprehensive statistics guide for social science users. In this blogpost, the first of two, he explains his approach to statistics and what motivated him to write the book.

Before using statistics in everyday life, I considered it a dry subject, perhaps for people in the money markets or for psychology students wanting to prove that a left-handed view of a house plan memorised on a dark night would be more successfully recalled by a right-handed burglar. There would be no impassioned arguments, nobody would be in denial about fundamentals, and everything would be drily abstruse.

I was totally wrong. The opinions of practitioners, teachers and students are quite diverse. They range between an insistence that only mathematical formulae can adequately describe phenomena, to a denial that statistical explanations could ever apply to data obtained in a qualitative manner. Those writers who have tried to find the middle ground have fought bravely but not always with clarity. While I know about and am influenced by the underlying mathematics, almost all of the explanations I use in the book are couched non-mathematically.

Some uses of statistical tests are fairly straightforward. Let us say, for example, that the supporters of different political parties are asked for their opinions on a proposed reform using a five point rating scale. We may find that the supporters of one political party are more inclined to be positive than the others, but we do not know if the result is likely to be merely the result of random fluctuations. A statistical analysis of the differences between the sets of supporters tells us whether or not the differences are likely to be significant or not.

Similarly, we may want to know if one psychological therapy is more effective than another at reducing individuals’ symptoms. So, smoothing out order effects by having some people starting with Therapy A and moving to Therapy B, with others being treated in the opposite order, we use statistical tests to find out if differences in therapeutic effects are a matter of chance or not.

Slightly more controversially, we can move away from the analysis of differences to the examination of relationships: correlations. Imagine the researcher having a hunch that the longer a person belongs to a real ale society, the shorter the duration of their personal relationships. Significant results from tests of correlation may tell us that we are on to something, but correlations do not distinguish cause from effect. Excessive drinking may have a negative effect on relationships, but it is also possible that relationship problems have spurred individuals to consolation at the bar. “My husband doesn’t understand me” may be the cry of the forlorn.

For insights into cause and effect the researcher will probably use qualitative methodology, perhaps interviewing spouses, running focus groups of marital therapists or even ascertaining the views of long-suffering bar staff. This is where many people who refer to themselves as qualitative researchers deny the possibility of using quantitative methods. The core claim is that the material can not be compared or contrasted in any way without detracting from its meaning.

To which my initial response is to ask how the researcher knows that a particular viewpoint is truly predominant within a sample, rather than reflecting the most vehement or articulate respondents. The special insights of a select few may indeed be important, but it should be recognised that they are a minority.

Not only can we count simple observations – yes and no, or preferences – but we can also create our own categories from the data. Comments such as “comes to bed stinking like a horse” and “gaining a beer belly” could be classified as ‘physical distaste’; “irritable” and “doesn’t understand me” could perhaps be categorised as ‘claims of incompatibility’.  The researcher decides, based upon the insights from their interviews and other qualitative sources.

There are special usage rules about the exclusivity and exhaustiveness of such data, but these are unlikely to create an insurmountable barrier. The richness of the data does not render quantification impossible. It merely challenges the researcher’s resourcefulness by requiring the creation of meaningful classifications. When it makes sense to do so, the classifications can be tested for significance; does one classification occur more frequently than would be expected by chance? Is there a relationship between two cross-tabulated factors? Crucially, when such insights provoke further questions, the quantification can feed into further qualitative research.  The potential richness of the relationship between quantitative and qualitative research led me to write about a range of qualitative tests, not just the Chi Square test usually cited in textbooks when dealing with cross-tabulated categorical data.

Another stimulus for further research, which has until now been ignored in introductory books on statistical testing, is the study of the time until events. This provides a graphical view of the period of time during which a series of events take place. The breadth of potential applications for this group of techniques is suggested by its other names, survival analysis and reliability analysis. The events can be negative, such as deaths, withdrawal from courses, re-offending, divorces and organisational meltdowns; or positive ones such as promotions and successful rehabilitations. In either case, the rate at which events occur provides insights into what may be happening in any given process. Once again, this is then likely to fuel qualitative research; for example, if re-offending tends to become more frequent after a particular period of time, or one type of offender tends to have a different rehabilitation pattern from others, then researchers may be spurred into interviewing offenders or convening focus groups of probation officers to gain an insight into these patterns.

SPSS step by step is available to order from www.policypress.co.uk

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Enter your email address to follow this blog and receive notifications of new posts by email.

Twitter Updates


Creative Commons License

Republish our articles for free, online or in print.

The work on the Policy Press blog is licensed under a Creative Commons licence.