As we explore a variety of topics around customer satisfaction, we thought we should back up and answer the question: What is a rating scale?
According to Wikipedia: “A rating scale is a set of categories designed to elicit information about a quantitative or a qualitative attribute. In the social sciences, common examples are the Likert scale and 1-10 rating scales in which a person selects the number which is considered to reflect the perceived quality of a product.”
In most cases today, a rating scale is easy to construct with an online or web-based survey tool. But what often happens is you can make up your own rating scale variables instead of using more standard or accepted terms for specific behaviors you are trying to measure. We are putting together a download to help you use standard terms that can help you avoid bias in a simple Likert scale type survey.
Rating the Rating Scales
We often refer to this academic paper from 1999 written by two professors at the Brooklyn College of the City University of New York: Hershey H. Friedman, Ph.D. and Taiwo Amoo, Ph.D. Both of these professors, not surprisingly, are in the Department of Economics at the University. Their post/paper is entitled: Rating the Rating Scales and does an excellent job of explaining how to avoid bias in constructing your rating scale.
“Rating scales are used quite frequently in survey research and there are many different kinds of rating scales. A typical rating scale asks subjects to choose one response category from several arranged in hierarchical order. Either each response category is labeled or else only the two endpoints of the scale are “anchored.
“Unfortunately, there are many ways that a rating scale can be biased. Dishonest researchers can, of course, manipulate the outcome of their research, if they wish, but such biasing may also be totally unintentional. This paper will describe some of the problems involved in creating a relatively unbiased rating scale.“
Their paper goes on to give a number of examples for building a balanced rating scale — and points out how to word these based on your objective, for example:
As a “satisfaction” scale (“How satisfied are you with ___?” ) with the response choices being: “very satisfied,” “satisfied,” “neither satisfied nor dissatisfied,” “dissatisfied,” and “very dissatisfied.”
As a requirements scale (“How often does using ___ meet your requirements?”) with the response choices being: “always meets my requirements,” “usually meets my requirements,” “occasionally meets my requirements,” “rarely meets my requirements,” and “never meets my requirements.”
Some Practical Ways to Create a Rating Scale: Use a Word/Number 1-to-5 Rating Scale
The 5-point scale creates a more balanced approach because it allows you to have one variable in the neutral position. This achieves a superior focus over the popular 1-to-10 scale. Some common rating scales follow the work of Jon A. Krosnick, cited and linked below.
For example, if you were asking if the customer service rep’s effort was “acceptable” you would ask with these five variables:
- Not at all acceptable
- Slightly acceptable
- Moderately acceptable
- Very acceptable
- Completely acceptable
To determine “Agreement” one would build a scale that left Neither as the middle variable.
- Completely disagree
- Somewhat disagree
- Neither agree nor disagree
- Somewhat agree
- Completely agree
Both of these ideas stems from another academic paper from Stanford University professor, Jon A. Krosnick, with his paper: The Optimal Length of Rating Scales to Maximize Reliability and Validity (with Alex Tahk). In this brief abstract, he writes:
“Survey research frequently uses multi-point scales to assess respondents’ views. These scales vary from two points (e.g., agree or disagree) to 101 points (e.g., the American National Election Study’s thermometer-style ratings). Scales can also vary in another regard: being bipolar (meaning the zero point is in the middle and the end points are opposites, such as extremely positive and extremely negative) or unipolar (meaning the zero point is at one end, as in “not at all important”). However, different scale lengths may differ in reliability, so it is important to understand how the length of the scales affects the reliability of the responses.
They conducted a study to look at the relationship between scale length and its reliability with 706 tests from thirty different studies. They concluded that:
“In general, we found that five- or seven-point scales produced the most reliable results. Bipolar scales performed best with seven points, whereas unipolar scales performed best with five. We also found that offering a midpoint on a bipolar scale, indicating a neutral position, increased reliability.”
That is quite possibly more dense than you might like for creating a simple rating scale for your next online customer survey, however, it may lead you to keep to a five or seven point rating system.
We hope we’ve answered the question: What is a Rating Scale? and given you some ideas on how to build one. We will be sharing some of the survey tools we like and use in a future post.