Exploring alternatives to Likert scale survey questions

Does survey takers' comfort with Likert scales lead to lower accuracy data? This methodology experiment - part of our ongoing Methodology matters series - pits a ranking-based question design with the traditional Likert scale.

Introduction and background

Likert-type questions are very common in survey research and they offer a variety of advantages. One of the primary advantages lies in their familiarity for respondents who have likely encountered similar items in the past. The survey research field is also very accustomed to these kinds of questions.

However, it is possible that this comfort with Likert scales leads to undesirable respondent behavior such as speeding through the survey or not paying much attention to the questions. The insights industry has leaned heavily on the Likert scale in most survey methodologies for almost 100 years, which has caused respondents to become inured to the approach. Survey designs that used to work relatively well in the past no longer work because the scale has become relatively flat and does a poor job of differentiating between brands, attributes or answers in a question set.

The basis of this experimental design study is the theory that due to 100 years of asking questions in the Likert format and the oversaturation of customer satisfaction research, Net Promoter Score (NPS) programs, and more, respondents have become accustomed to using a scale point at the top of the scale to represent their relatively positive outlook on life while also not taking the time to read or consider the question being asked.

We are also motivated by the need to redesign survey questions so that we could produce more meaningful results as Likert scale questions became less differentiated. There are two important considerations. First, people do not naturally consider their feelings or attitudes on a numeric scale. Second, people more readily weigh attributes, items, decisions, or companies in a relative context. Non-relative decisions are out of context and, therefore, difficult to measure accurately or meaningfully. Even impulse purchases are subject to an invisible competitive context. Survey researchers can and should be exploring alternative question types beyond Likert scales that can be placed in a relative context for a more accurate measurement.

>View data collection methodology statement

Experimental design

The goal of this experiment was to compare the Likert scale-based approach to understanding the relative value of a set of attributes (Control Cell) to an alternative approach in which all attributes are presented together in a competitive context where respondents first rank a set of attributes and then allocate a fixed number of points across each attribute (Test Cell).

Two unique questions are asked, one that mimics the traditional Net Promoter Score approach and one similar to evaluating a series of attitudinal attributes. Both sets of questions are very common uses of Likert Scale questions in today’s marketing research environment.