Data Mining w/ NO DATA

1. Data Mining w/ NO DATA Timothy D’Auria BostonDecision.com April 16th, 2012 Disclaimer: Boston Decision believes the information contained herein to be accurate. However, Boston Decision, LLC makes no guarantees and no warranties, written, oral or implied, including without limitation any implied warranties of merchantability, fitness, or accuracy. Recipient assumes all responsibility for use of the information contained herein.

2. Boston Decision • MINE – PREDICT – AUTOMATE • Provide the skills, resources, expertise

5. Example

8. Example

9. Example

10. Demonstration

12. Data Mining ≠ Querying

13. How we left off… • “For the next demonstration, we need a volunteer from our last meetup.  • We're looking for a small company, perhaps a start-up, in any industry.”

15. NO DATA!!

16. … finger on the pulse marketing

17. 2 Questions • Who is the target market? • What does the target market want?

18. The Data • Bureau of Labor Statistics • Consumer Expenditure Survey (http://www.bls.gov/cex/) • Bureau of Economic Analysis • Personal Consumption Expenditures (http://www.bea.gov/) • American FactFinder (US Census) • http://factfinder2.census.gov

19. The Technology • R Language – Open Source & Free • tm • gdata • plyr • twitteR • ggplot2 • maps • wordcloud

20. Read the Data

33. Univariate Model Limitations • 1 attribute at a time (Age, Income, etc..) • Not comprehensive (can’t look at age, income together) • Ignores relationships

34. Multivariate Model = A Next Step • Control for Age while looking at income • Take many more factors into account

35. A Naïve Geographic Scoring Algorithm • Age 25 – 34, $80K+, Associates or Bachelors • Created a scoring algorithm that ranks MA counties in these 3 areas.

38. Jewelry Blogosphere Chatter in Middlesex? What does the target want?

41. Read Tweets

42. Results • 562 Tweets from the last 9 days

43. We can extract all sorts of info

44. Want the text!

45. Let’s Mine The Text • What is a Corpus? • A collection of text “documents” • Not necessarily literal documents! • Sentence • Text Box • Web Page • Paragraph • A Word Doc

46. Construct the Corpus

47. Some “Cleanup” • Remove punctuation • Set all to lowercase • Remove words that provide no information

48. Term-Document Matrix

49. What terms appear most frequently? • "amp" "black" "bracelet" "check" "chicos" "crystal" "earrings" "etsy" "glass" "gold" "hawaiian" "jewelry" "love" "necklace" "nwot" "silver" "via" "vintage " "white"