What Is A Rating Scale

by admin on May 1, 2013

As we explore a variety of topics around customer satisfaction, we thought we should back up and answer the question: What is a rating scale?

According to Wikipedia: “A rating scale is a set of categories designed to elicit information about a quantitative or a qualitative attribute. In the social sciences, common examples are the Likert scale and 1-10 rating scales in which a person selects the number which is considered to reflect the perceived quality of a product.”

In most cases today,  a rating scale is easy to construct with an online or web-based survey tool. But what often happens is you can make up your own rating scale variables instead of using more standard or accepted terms for specific behaviors you are trying to measure. We are putting together a download to help you use standard terms that can help you avoid bias in a simple Likert scale type survey.

Rating the Rating Scales

We often refer to this academic paper from 1999 written by two professors at the Brooklyn College of the City University of New York: Hershey H. Friedman, Ph.D. and Taiwo Amoo, Ph.D. Both of these professors, not surprisingly, are in the Department of Economics at the University.  Their post/paper is entitled: Rating the Rating Scales and does an excellent job of explaining how to avoid bias in constructing your rating scale.

Rating scales are used quite frequently in survey research and there are many different kinds of rating scales. A typical rating scale asks subjects to choose one response category from several arranged in hierarchical order. Either each response category is labeled or else only the two endpoints of the scale are “anchored.

“Unfortunately, there are many ways that a rating scale can be biased. Dishonest researchers can, of course, manipulate the outcome of their research, if they wish, but such biasing may also be totally unintentional. This paper will describe some of the problems involved in creating a relatively unbiased rating scale.

Their paper goes on to give a number of examples for building a balanced rating scale — and points out how to word these based on your objective, for example:

As a “satisfaction” scale (“How satisfied are you with ___?” ) with the response choices being: “very satisfied,” “satisfied,” “neither satisfied nor dissatisfied,” “dissatisfied,” and “very dissatisfied.”

As a requirements scale (“How often does using ___ meet your requirements?”) with the response choices being: “always meets my requirements,” “usually meets my requirements,” “occasionally meets my requirements,” “rarely meets my requirements,” and “never meets my requirements.”

Some Practical Ways to Create a Rating Scale: Use a Word/Number 1-to-5 Rating Scale

The 5-point scale creates a more balanced approach because it allows you to have one variable in the neutral position. This achieves a superior focus over the popular 1-to-10 scale. Some common rating scales follow the work of Jon A. Krosnick, cited and linked below.

For example, if you were asking if the customer service rep’s effort was “acceptable” you would ask with these five variables:

  1. Not at all acceptable
  2. Slightly acceptable
  3. Moderately acceptable
  4. Very acceptable
  5. Completely acceptable

To determine “Agreement” one would build a scale that left Neither as the middle variable.

  1. Completely disagree
  2. Disagree
  3. Somewhat disagree
  4. Neither agree nor disagree
  5. Somewhat agree
  6. Agree
  7. Completely agree

Both of these ideas stems from another academic paper from Stanford University professor, Jon A. Krosnick, with his paper: The Optimal Length of Rating Scales to Maximize Reliability and Validity (with Alex Tahk). In this brief abstract, he writes:

“Survey research frequently uses multi-point scales to assess respondents’ views. These scales vary from two points (e.g., agree or disagree) to 101 points (e.g., the American National Election Study’s thermometer-style ratings). Scales can also vary in another regard: being bipolar (meaning the zero point is in the middle and the end points are opposites, such as extremely positive and extremely negative) or unipolar (meaning the zero point is at one end, as in “not at all important”). However, different scale lengths may differ in reliability, so it is important to understand how the length of the scales affects the reliability of the responses.

They conducted a study to look at the relationship between scale length and its reliability with 706 tests from thirty different studies. They concluded that:

“In general, we found that five- or seven-point scales produced the most reliable results. Bipolar scales performed best with seven points, whereas unipolar scales performed best with five. We also found that offering a midpoint on a bipolar scale, indicating a neutral position, increased reliability.”

That is quite possibly more dense than you might like for creating a simple rating scale for your next online customer survey, however, it may lead you to keep to a five or seven point rating system.

We hope we’ve answered the question: What is a Rating Scale? and given you some ideas on how to build one. We will be sharing some of the survey tools we like and use in a future post.

 

{ 0 comments }

Every company and every business needs to know, how satisfied their consumers are with their services and performance. How good or how bad was the customer experience with your brand or company? This is the key information to know and one can use different consumer satisfaction rating scales to get the right answer to the big question. Let’s take a brief look at the different options to choose from.

First of all, let’s understand what type of satisfaction scale ratings are available. There are several options to choose from and each of them has certain pros and cons to offer. Mostly, the rating scales differ by the number of answer options they include. The shortest one is three-item scale and the longest is 11-point scale. So, basically, you can break all the types of rating scales into three main categories: short, medium, and long. Let’s take a look at the basic overview of each of the options you have.

The Shortest Satisfaction Rating Scale

Pros:

  • It has only three answers to choose from. So, consumers generally do not feel overwhelmed by the size and look of the scale. Because it is short, they can grab the meaning quickly and answer accurately.
  • It provides a more accurate data on the consumer satisfaction rate. When choosing from just three answers, people are less inclined to pick the “average” or “good” answer. They can report a bad service more easily, and then it is with the other types of rating scales.

Cons:

  • Some people think it to be too short to provide relevant results.

The Medium Satisfaction Rating Scale

Pros:

  • This rating scale may include 5 to 7 items or answer variations in it. So, it provides you with just a bit more detail that helps you to analyze what your customer is trying to tell you. With more answer choices, you give people a chance to share how their experience is relevant to your company or brand.
  • This satisfaction rating scale is considered one of the most reliable ones among all the rest of the scale sizes. Research studies on various rating scales often conclude that this mid-size scale performs best.

Cons:

  • The 5 to 7 rating scale may “scare off” some consumers because it has too many answers to review and it may take too long to get through it.
  • Not sure yet why this is so, but some of the research posts we read showed that the 5-item satisfaction scale tends to show high satisfaction rate of the consumers. However, in some cases, this information is not reliable, as some customers avoid picking the top or the bottom points of the scales. So, even if they have had a bad experience with a company, they would still pick options 3 or 4 (which is neutral to satisfied on a five sat scale) instead of 2.

The Long Satisfaction Rating Scale

This satisfaction scale is obviously the longest and may consist of 10 or sometimes 11 points.

Pros:

  • People can rate your company and their experience with it more precisely. They have a wider range of points to express their satisfaction or dissatisfaction rate.
  • You can evaluate your consumers’ answers more easily. For instance, anything below 6 is negative. 9 to 10 or 11 is positive. The rest is just neutral, which stands for passive consumers.

Cons:

  • Consumers need to pay more attention and time to this type of rating scale, which may result in more accurate results because one would believe that they are spending more time thinking about their answer.

 

{ 0 comments }

Satisfaction Scale Is About Understanding Your Customer

January 24, 2013

Customer Satisfaction Scale Satisfaction scales are used to measure customer approval on written or Internet surveys. There are several different ways to set up scales, each with their own pros and cons. It is important to choose an appropriate satisfaction scale for the measurement that is being made. What is a Satisfaction Scale? Psychologist Rensis [...]

Read the full article →