Posted by: rayseghers | December 15, 2009

Comparing Survey Questions Measured on Different Scales

On several occasions I have been asked to compare current survey data with previous data or with some sort of benchmark data.  The request is a quite reasonable one.   In order to evaluate the current results it is useful to have some reference point or standard of comparison.

Often the problem is that the current questions are not measured on the same response scale as the other questions.  So, what to do?

Now you know the text book answer – to compare results on two questions, the questions must be worded identically and must be measured on the exact same response scale.  OK, but suppose your boss’s boss’s boss asks you to do it anyway.  He said to go ahead and state the limitations of the analysis and to do the best that you can.  Well, surprise, surprise, you do it anyway.

In working with various question response scales, you notice that the pattern of responses across the response categories is certainly very different.  However, when using a 5-point Likert Scale, you also notice that the pattern for the two lowest response categories is fairly similar regardless of the particular scale.  For example, the responses across 1=Strongly Disagree and 2=Disagree are comparable to those across 1=To a Very Little Extent and 2=To a Little Extent.

So, while the comparison across different response scales is not perfect, one may still draw a reasonably accurate conclusion.  Hey, my boss’s boss’s boss was pleased, so all was right with the world.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: