A journey into Text Analytics

1. A journey into Text Analytics John McConnell Analytical People ASC Winchester 7th September 2013

2. Contents • Background & Objectives • Our current view on Text Analytics • Value • Process • An example application • Conclusions

3. Background • Text Analytics and Text Mining are largely synonymous • Interest and execution of Text Analytics is growing • Social Media sources are largely responsible for this • And that often means “Big Data” • This should lead to further improvements in technology and methodology which will benefit survey practitioners

4. Objectives • We’ve been involved in more Text Analytics work in the last 2 years than in all previous years • Our objective in this presentation is to share some of our experience and thoughts around some of the technology we have used

5. The Value Propositions • Reduce cost (and time) • Generating actionable insights • Improve public and commercial processes *http://wp.eaagle.com/

6. Using Text Analytics to find Text Analytics software http://www.isvworld.com/

7. 3 Software tools

8. The Process – Highest Level

9. Process – Level 2

10. How can we tell if we are using the right tool(s)? How well do the tools/methods deliver on the value propositions?

11. 1. Extract Algorithms and Dictionaries

12. Example Data • The American Physical Society (APS) • Student Survey Comments from 2009 (Base=1304) • Q4.2 Comments about the best features of and what could be added or improved to the special programsesfor Student Members* *http://www.aps.org/about/governance/committees/commemb/upload/2009-student-comments.pdf

13. The first extraction with R library("tm", lib.loc="C:/Users/jmcconnell/Documents/R/win-library/3.0") APS2009df = read.csv("C:/AP/ASC/APS/APS2009Verbatims.csv", header = TRUE) text_corpus <- Corpus(VectorSource(APS2009df), readerControl = list(language = "en")) summary(text_corpus) #check what went in text_corpus <- tm_map(text_corpus, removeNumbers) text_corpus <- tm_map(text_corpus, removePunctuation) text_corpus <- tm_map(text_corpus , stripWhitespace) text_corpus <- tm_map(text_corpus, tolower) We apply a basic set of text handling methods (simple NLP) e.g. removePunctuation We also apply a small dictionary of known “Stopwords” (not shown)

14. R Extraction Results – Top 20 Terms

15. The first extraction with Rapid Miner We visually construct a similar set of steps

16. Rapid Miner Extraction Results – Top 20 Terms

17. 2. Refine Improving and creating new data In second and subsequent waves (where applicable) Refine should be a shorter step where we look for new concepts

18. Rapid Miner - Refine We add one process step to fix up some of the issues in the first extraction Filter Tokens sets a lower limit for the length of an extracted term/attribute

19. Rapid Miner results after first refinement

20. The first extraction with SPSS SPSS Uses a Wizard to specify the extraction steps

21. SPSS Extraction Results – Top 20 Terms SPSS Is counting respondents not occurrences Synonyms are used from the dictionaries

22. 10 stars, 10/10, 100 % correct, 100% accurate, 100% correct, 100% grade a, 5 star, 5 stars, 5-star, ^ best $, ^ great $, a must, a nice plus, a plus, a+, a++, aagood, above and beyond, above excellence, absolute life saver, absolute word class, acceptional, admirable, all was well, allright, alright, always a please, amazing, among the best, among the very best, appreciable, appreciative, award winning, awesome, awesopme, awsome, beenfantastic, best asset, best of all, best possible, beyond expectation, beyond expectations, big asset, big beast, big hit, big hits, big kudos to, big plus, blow all others away, blows all others away, blows the doors off, brilliant $, can not be beat, can't be beat, can't beat, cannot be beat, capable, capible, class service, compliment, compliment one another well, congrats, congratulations, copious, cutting edge, cutting-edge, dandy, delight, deluxe, deserves a raise, deserves credit, does that well, doing her best, doing his best, doing their best, done very well, dynamite, exccellent, excelent, excellant, excellence, excellet, excelllent, excepional, exceptional, exceptionl, execellent, exelant, exelent, exellant, exellecent, exellent, exlt, expectional, exquis, exquise, exquises, exquisite, exquisitely $, extraordinary, extrodinary, fabulous, fairly well, fanatstic, fantabulous, fantasic, fantastic, fantatic, finest, first class, first-class, first-rate, five stars, formidable, frantastic, given me the most, godsend, goes over well, goodd, gooood, graet, grat, grea, greaat, great pleasure, greate, greatest, greeeeeeeaaattttt, gret, greta, hats down, hats off, head and shoulders better, heavenly, high hats off, ideal, impecable, impeccable, impress, impresses me most, impressive, in an orderly fashion, incomparable, incredibe, incredible, increible, indisputable, ingenious, inpecable, invaluable, is still the best, it was a pleasure, knock socks off, knock spots off, kudos, kudos to, laudable, lifesaver, made an impression, made the difference, magnificent, marvellous, marvelous, my compliments to, nicest, number 1, number one, oustanding, out of the woods, out of the world, out of this world, outperform, outperforming, outsanding, outstanding, peachy, perfect, perfection, perfectly done, phenomenal, phenominal, pleasure of working with, prettier, pretty good, quintessential, reach a ten, real good, real nice, remarkable, right direction, rock $, rocked my world, second to none, sensational, smashing, spectacular, spendid, splendid, stand head & shoulders above, stand head and shoulders above, standing head & shoulders above, standing head and shoulders above, stands head & shoulders above, stands head and shoulders above, stood head & shoulders above, stood head and shoulders above, strong positive, superb, supurb, surpassed my expectations, surreal, sweetheart, ten stars, terric, terrific, terrifig, the best, the best one so far, the best thing, the highlight of, the only one that works, thebest, think highly, think very highly, to die for, top notch, top quality, top ranked, top-flight, top-notch, top-of-the-line, top-ranked, top-ranking, topflight, topnotch, topranked, topranking, tremedous, tremendous, tried and proven, trmendous, turn out good, two thumbs up, unbeatable, unmatched, unmnatched, unparalleled, unquestionable, unquestionnable, unsurpassed, up 2 standard, up 2 standards, up 2 usual standards, up to standard, up to standards, up to usual standards, up to your usual standards, up-beat, upbeat, utmost, v-good, well done, went above and beyond my expectations, woderful, womderful, wondeful, wonderful, wonderfull, wonedeful, wonederful, would be the smartest, wounderful Synonyms for “Excellent” 10 stars, 10/10, 100 % correct, 100% accurate, 100% correct, 100% grade a, 5 star, 5 stars, 5-star, ^ best $, ^ great $, a must, a nice plus, a plus, a+, a++, aagood, above and beyond, above excellence, absolute life saver, absolute word class, acceptional, admirable, all was well, allright, alright, always a please, amazing, among the best, among the very best, appreciable, appreciative, award winning, awesome, awesopme, awsome, beenfantastic, best asset, best of all, best possible, beyond expectation, beyond expectations, big asset, big beast, big hit, big hits, big kudos to, big plus, blow all others away, blows all others away, blows the doors off, brilliant $, can not be beat, can't be beat, can't beat, cannot be beat, capable, capible, class service, compliment, compliment one another well, congrats, congratulations, copious, cutting edge, cutting-edge, dandy, delight, deluxe, deserves a raise, deserves credit, does that well, doing her best, doing his best, doing their best, done very well, dynamite, exccellent, excelent, excellant, excellence, excellet, excelllent, excepional, exceptional, exceptionl, execellent, exelant, exelent, exellant, exellecent, exellent, exlt, expectional, exquis, exquise, exquises, exquisite, exquisitely $, extraordinary, extrodinary, fabulous, fairly well, fanatstic, fantabulous, fantasic, fantastic, fantatic, finest, first class, first-class, first-rate, five stars, formidable, frantastic, given me the most, godsend, goes over well, goodd, gooood, graet, grat, grea, greaat, great pleasure, greate, greatest, greeeeeeeaaattttt, gret, greta, hats down, hats off, head and shoulders better, heavenly, high hats off, ideal, impecable, impeccable, impress, impresses me most, impressive, in an orderly fashion, incomparable, incredibe, incredible, increible, indisputable, ingenious, inpecable, invaluable, is still the best, it was a pleasure, knock socks off, knock spots off, kudos, kudos to, laudable, lifesaver, made an impression, made the difference, magnificent, marvellous, marvelous, my compliments to, nicest, number 1, number one, oustanding, out of the woods, out of the world, out of this world, outperform, outperforming, outsanding, outstanding, peachy, perfect, perfection, perfectly done, phenomenal, phenominal, pleasure of working with, prettier, pretty good, quintessential, reach a ten, real good, real nice, remarkable, right direction, rock $, rocked my world, second to none, sensational, smashing, spectacular, spendid, splendid, stand head & shoulders above, stand head and shoulders above, standing head & shoulders above, standing head and shoulders above, stands head & shoulders above, stands head and shoulders above, stood head & shoulders above, stood head and shoulders above, strong positive, superb, supurb, surpassed my expectations, surreal, sweetheart, ten stars, terric, terrific, terrifig, the best, the best one so far, the best thing, the highlight of, the only one that works, thebest, think highly, think very highly, to die for, top notch, top quality, top ranked, top-flight, top-notch, top-of-the-line, top-ranked, top-ranking, topflight, topnotch, topranked, topranking, tremedous, tremendous, tried and proven, trmendous, turn out good, two thumbs up, unbeatable, unmatched, unmnatched, unparalleled, unquestionable, unquestionnable, unsurpassed, up 2 standard, up 2 standards, up 2 usual standards, up to standard, up to standards, up to usual standards, up to your usual standards, up-beat, upbeat, utmost, v-good, well done, went above and beyond my expectations, woderful, womderful, wondeful, wonderful, wonderfull, wonedeful, wonederful, would be the smartest, wounderful

23. Adding Wordnet to our R (/RapidMiner) analysis library("wordnet") setDict ("C:/Wordnet/WordNet-3.0/dict") synonyms("excellent", "ADJECTIVE") [1] "excellent" "fantabulous" "first-class" "splendid"

24. Analytics to aid refinement

25. Job … Fair Students are asking for more “stuff” at the job fair

26. R Extraction Results – Top 20 Terms

27. Onward to analysis *This is an anonymised example

28. Onward to analysis • 3. Analyse

29. A High Level Comparison *IBM/SPSS have a Text Analytics option for Data Mining which may be more scalable – we haven’t tested yet

30. Our current conclusions • Dictionaries help in the initial extraction • But it is almost inevitable you will want to extend them to get to the specificity of the study. If the study domain is very specific you can build your own dictionaries in all 3 tools. A lot of social media monitoring starts with libraries of regular expressions built from the ground up. • Open Source tools like R and Rapid Miner will continue to improve with “packages” added by the R community • There is no “silver bullet”. The Refine step will typically require a lot of manual input • Especially in the initial “build” phase • More is required on larger surveys • But the ROI – in time and/or cost - should be clear • And the results more robust and reliable

31. A journey into Text AnalyticsThank-you & Questions John McConnell Analytical People jmcconnell@AnalyticalPeople.com