Paper Title: Decoding The Rating Scale

1. Paper Title: Decoding The Rating Scale

2. Objective: Showcase a Technique to Extract the True Insights from Rating Scale Data

3. There shall be one measure of wine throughout our kingdom, and one of ale, and one measure of corn and one breadth of cloth. As with measures so shall it be with weights - Magna Carta, 1215

4. Defining a Measure • Long before science or mathematics emerged as professions, the commercial, architectural, political and moral necessities for abstract, exchangeable units of unchanging value were recognised and pursued. • All physical sciences have abstract units of measurement such as length, price, volume, weight, age,… and they are of unchanging value.

5. How do We Measure? • The Likert Rating Scale is the most often used measurement instrument in quantitative research. • Usually deployed in two context’s: • In isolation to understand satisfaction, product/ concept ratings. Interpretation is usually done basis norms to understand what the response means. • Within a battery of statements. Frequently multivariate techniques are used to extract the relationships in the data. • This is the area where a large gap exists in understanding the true response patterns.

6. Drawbacks of Rating Scale as a Measure • Respondent scale use bias; every respondent has his/her own way of responding to a scale. However we assume that all respondents have reacted to the scale in the same manner. In reality each respondent’s perception of the scale has been different. • Difference between scale points; the difference between scale points is assumed to be the same. We assume the data to be continuous when in reality it is not so. • Attribute Ease Bias (Floor & ceiling effects); a response on a hard to rate attribute is treated to be the same as on an easy to rate attribute. Further by constraining respondents to react to a fixed scale, we lose out on truly enthusiastic/ deeply negative responses.

7. Overcoming the Drawbacks • The Challenge: To construct an abstract linear measurement scale of unchanging value out of raw data that is bias free. • The goal is to produce a reference standard common scale for the exchange of quantitative value, so that all research and practice relevant to a particular variable can be conducted in uniform terms. • Thus from observed responses, a measurement scale is created– observations may be ordinal, measurements however must be interval.

8. Estimation of Respondent's True Rating Estimation of Respondent's True Rating Ease of Ease of Scale Level Scale Level + + Raw Raw Respondent Respondent + + + + rating rating = = Difference Difference rating rating bias bias attribute attribute Perception Perception Adjust Adjust Remove Remove Adjust Adjust Leading to Estimation of Leading to Estimation of True Rating True Rating for Every Respondent for Every Respondent Decomposing the Rating Scale • The above analysis is achieved through a variation of a technique called as ‘Rasch’ which uses a joint maximum likelihood estimation. Rasch predicts the score for every respondent for every attribute with the biases being factored for.

9. Rasch Transformed Rating vs. Raw Rating 3.80 3.71 3.70 3.69 3.67 3.62 3.60 3.55 3.53 3.46 3.45 3.40 3.38 3.37 3.30 3.30 3.28 3.24 3.20 3.20 3.19 3.16 3.09 3.09 3.02 3.00 2.98 2.98 2.98 2.97 2.97 2.93 True Rating 2.90 2.87 2.80 2.78 2.77 2.74 2.72 2.65 2.65 2.65 2.64 2.60 2.57 2.57 2.50 2.42 2.40 2.40 2.39 2.35 2.34 2.26 2.24 2.20 2.00 Resp Resp Resp Resp Resp Resp Resp Resp Resp Resp Resp Resp 1 2 3 4 5 6 7 8 9 10 11 12 Respondents RAW RATING 1 2 3 4 5 Impact of the Rasch Transformation

10. Impact of the Rasch Transformation • Change in distribution – post the transformation data is normally distributed. Post Rasch transformation Raw Data

11. Correlation Correction

12. Distance Between Each Scale Point 1.54 1.36 0.81 Toughest 0.28 Ease of Attribute Rating 1.41 1.20 0.73 Easiest 0.53 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Distance Very Poor to Poor Poor to Neither poor nor good Neither poor nor good to Good Good to Excellent Correcting for Floor/Ceiling Bias

13. Rasch Transformation: A summary • Puts every response on the same linear continuum, • Negating the effects of respondent scale heterogeneity bias. • Equating the distance perception between scale points across respondents. • Adjusting for ease of attribute rating. • Making the data continuous and normally distributed. • Removes the artificial correlation bias as a result of scale range used. • Data that is ideally suited for multivariate techniques.

14. Rasch Transformation Implication on Multivariate Analysis

15. Key Challenges Faced in Multivariate Techniques • Multivariate MR workhorses are either correlation based (factor analysis, regression, SEM,…) or distance based (clustering algorithms). • All of them assume data is continuous and normally distributed; an assumption that the raw rating scale data does not satisfy. • Correlation based techniques are plagued by multi-collinearity (a large part of which is caused by inconsistency in the manner in which respondents use scales). • Distance based techniques tend to cluster respondents basis the manner in which they have responded to the scale rather than their responses to the attributes.

16. Case Study 1 Driver Analysis

17. Case Study 1: Driver Analysis • FMCG client wished to understand drivers of brand image in the detergent category to assess their current marketing strategy and develop their marketing strategy for future. • 5 key brands rated on a 5 point (very poor to excellent) scale on 72 statements.

18. WHITENING PROMOTIONS / OFFERS Bright whiteness for white clothes Reduces physical effort/less scrubbing is required Offers variety of promotions COLOR - BRIGHT / PROTECT EMOTIONAL BENEFITS Offers attractive Promotions AVAILABILITY Keeps coloured clothes looking bright Heritage, i.e. used by my mother etc…. Protect colour of clothes Recommended by friends/family Available in the pack sizes I want STAIN REMOVAL /CLEANING Is a brand I trust Easy to find in the shop when I am buying detergents SIZE / PACKAGING FORMATS Removal of tough stains Understand my requirements Gives deep cleaning Capable of overcoming all my cleaning worries Available in different pack sizes So strong in cleaning that rewash is not required It makes clothes look new for long Offers convenient sizes Can wash more clothes with less quantity Feel confident wearing the clothes after washing Brand should come in a poly pack FRAGRANCE PRICE ELEMENTS Sure of all the stains would go away after washing Having Pleasant fragrance Gives fabric a fresh feel Affordable price Long lasting fragrance impact Feel refreshed wearing the clothes after washing Good price for the Quality I want Offers variety of fragrances Make me feel that I have fulfilled my duties as housewife Cleanliness at an affordable price LATHERING / FOAM Makes me feel like a smart housewife Gives good value for money MULTIPLICITY VS EXCLUSIVITY Generates thick lather Ultimate well being Rinses well / Rinse out easily Makes me feel intelligent Suitable for all types of clothes ADDITIVE POWER A socially aware/response brand Suitable for expensive clothes Has the power of natural ingredients Makes me feel confident Ideal detergent for black clothes Has the power of bleach Technologically superior brand Ideal detergent for Abayas Has the power of blue A prestigious brand Ideal for under garments SENSITIVE A hard working brand Ideal detergent for coloured clothes Gives the clothes soft texture A knowledgeable brand Ideal detergent for white clothes No need to add other products like fabric softeners A elegant brand Is ideal for special needs Gentle on hands A modern brand Gives multiple benefits Skin friendly ingredients An innovative brand which is meaningful to me Ideal for washing the bathroom BLACK/DARK WASH Makes me look nice Best detergent for washing tiles Makes darks soft & smooth Is a smart brand Best detergent for laundry & HHC OVERALL PARAMETERS Leaves no white spot on black clothes Ease of storing / carrying / using Does not fade the black clothes Is a sophisticated / premium brand Overall , a excellent product Tangible Intangible Packaging/Availability The Statements

19. Key Problems Faced • Huge Multi-collinearity issues – a factor analysis combined nearly half of the statements into the first factor and revealed only 7 factors. Despite extraction of larger number of factors, the first factor did not split. • Large cross-loadings of attributes across factors. • Finally researcher judgment had to be imposed to develop the latent variables. • However given that latent variables were extracted independently of each other, multi-collinearity remained prevalent in the data. • The latent variables were subsequently regressed against the overall measure.

20. Whiteness 15.7 8.6 Fragrance Tangible Stain removal / Deep cleaning 7.8 Benefits 50.8% (Functional) Coloured Clothes 7.2 5.7 Additive Power Sensitive 5.7 Makes me feel like a smart housewife 1.9 Intangible Sure of all the stains going away (reassurance) 1.8 Benefits Meaningful innovation 1.7 7.7% (Emotional) Makes clothes look new for long 1.2 1.0 Heritage/Recommendation Price Parameter Price 8.6 Other Promotions 12.9 Marketing Mix 18.7% Elements Availability 5.7 Multi-purpose Multiple Usage Benefits 14.3 Usage Findings (as presented to the Client)

21. Key Takeouts (as presented to the Client) • Tangible (functional) parameters clearly override the other needs detergent category with Whiteness being the key driver. • Stain Removal & Additive Power seen to be different (potential to leverage these separately). • Fragrance also plays a key role. • Promotions also is a key need in the detergent market – Innovative promotions a key way forward. • Product offering multi-purpose use seen as beneficial. • Emotional parameters have less influence on choice.

22. Impact of Rasch Transformation • Given that the biases were negated, a factor analysis on the transformed data gave a 16 factor solution that made intuitive sense. • Marginal cross-loadings of attributes observed across factors.

23. 12.1 Powerful product 8.7 Preserving brightness 8.4 Suitability for all clothes Tangible Benefits 5.0 Fragrance related (Functional) 3.8 44.7% Suitable for black clothes 2.5 Suitability for dark clothes 2.3 Mixing with water 1.8 Gentle & soft Makes me fulfill the role of a smart hw by understanding my requirements 14.4 Intangible Superior brand 11.5 39.7% Benefits 8.3 (Emotional) Confidence & fresh feeling 5.6 Heritage/recommendation Price Parameter 7.7 Price Other 1.3 Promotions Marketing Mix 5% 3.7 Elements Availability Multi-purpose 2.8 Multiple Use Benefits Usage Findings post transformation

24. Findings post transformation • Both emotional as well as tangible needs need to be communicated in conjunction. • The largest driver is the emotional need of making the housewife feel intelligent/smart. • The largest functional need is for a powerful product (power of additives + stain removal) and both of these go hand in hand. • Communication must be tailored to meet the above needs (both functional as well as emotional). • Promotions not the way forward. • Product offering multiuse benefits not seen to be beneficial.

25. Key Benefits of Transformation • Large (diametrically opposite) variation in the recommendation to the client. • The transformation enables to indentify the true relationship between attributes and extract the real insights.

26. Case Study 2 Segmentation

27. Case Study 2: Segmentation • Segmentation based on attitude battery in the hair care category. • Study was done across 2 ethnic groups and 3 countries and a common segmentation scheme was sought.

28. Key problem faced • Both the ethnic groups have different ways of reacting to scales. • One of the ethnic groups has a large gratuity bias implying that they broadly stuck to the top end of the scale.

29. The Segmentation Process • A consistent way of segmentation was carried out across 3 sets of data • Original raw data • Respondent centered data • Rasch transformed data • A 4 segment solution was explored in all the above 3 cases. • Segment solutions were arrived at by indentifying the initial starting points using hierarchical clustering using Ward’s method and then subsequently using these for a K-means clustering.

30. Segmentation Scheme on Raw Data

31. Segmentation Scheme on Rasch Transformed Data

32. Segmentation Scheme on Respondent Centered Data

33. Result Comparison • Segmentation on the raw data proved futile as it predominantly segregated the respondents basis ethnicity. • Segmentation on respondent centered data provided an improvement but still a clear segmentation scheme could not be arrived at. • Rasch Transformed data gave 4 segments with clearly differentiated needs from the category.

34. Statistical Validation • In order to understand statistically which of the 2 solutions between respondent centered data and Rasch Transformed data was better, a few tests were carried out • To understand which solution is the most homogeneous within and heterogeneous across • Discriminant Map, Calinsky & Harabasz Index • To understand predictive ability • Discriminant Analysis • To understand which solution discriminates more across the attributes • Manova

35. Respondent Centered Data Rasch Transformed Data Respondent Centered Data - Discriminant map Canonical Discriminant Functions Canonical Discriminant Functions Cluster Number of Cluster Number of 6 6 Case Case 1 1 2 2 4 4 3 3 4 4 Group Centroid Group Centroid 2 2 1 1 3 3 4 0 Function 2 0 Function 2 4 2 2 -2 -2 -4 -4 -6 -6 -8 -6 -4 -2 0 2 4 6 -4 -2 0 2 4 Function 1 Function 1 Statistical Validation

36. Key Benefits of Transformation • Segmentation on transformed data led to both • a more coherent segmentation scheme • segments which are more homogenous within and heterogeneous across • a more statistically valid scheme

37. In Conclusion • So far the way we have been interpreting Scale data for multivariate techniques has been sub-optimal. • A pre-processing technique to remove the biases associated with scale data is a necessity if we are to extract true relationships (insights) & deliver accurate recommendations. • It also saves tremendous amount of researcher time (lesser number of iterations required).

38. Let’s Rasch & Roll