1 / 45

Innovation Culture 创新方法论

The Networked Economy (17) : Information Management, Strategy, and Innovation 网络经济 : 信息管理 , 战略 , 和创新. Innovation Culture 创新方法论. Three ingredients for innovation 创新三要素. Methodology 方法 Statistics 统计学 Data Analysis 数据探测 Machine Learning 机器学习. Data 数据 Computer Science 计算机科学

keala
Download Presentation

Innovation Culture 创新方法论

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Networked Economy (17) :Information Management, Strategy, and Innovation网络经济: 信息管理, 战略, 和创新 Innovation Culture创新方法论

  2. Three ingredients for innovation 创新三要素 Methodology方法 Statistics统计学Data Analysis数据探测 Machine Learning机器学习 Data数据 Computer Science计算机科学 Database Research数据库研究 Domain expertise知识领域 Behavioral Economics行为经济学 Marketing市场营销学 Finance 金融学 plus: Culture

  3. Q: Understanding cross-channel shopping behavior读解跨渠道购物行为 • Drive online shoppers to traditional channels through trackable means 通过跟踪手段鼓励网上购物者到传统渠道消费 • E.g., giving them codes to redeem如：抵换折扣 • Drive offline shoppers to website鼓励店铺消费者上网购物 • E.g., Banana Republic giving offers with offline purchase to first online purchase如： Banana Republic对其线下销售提供首次网上销售的机会 • Survey shoppers调查购物者 • Survey employees与员工交谈 • Merge online and off-line transaction databases 合并网上和网下交易数据库 • Collect additional data收集其他数据 • Video录像 • RFIDs…电子标签…

  4. Approaches towards understanding customers • Stated preferences消费者宣称的主观偏好 • Surveys调研 • Online网上 • Offline网下 • Revealed preferences实际研究显示的客观偏好 • Analyze clicks分析点击 • Methodologies 研究方法 • Server data网络服务器数据 • Observe the store观察消费者行踪 • Browser data (e.g., toolbar)浏览器数据(如：工具条） • Follow customer across sites观察网民的跨网浏览 Gain insights by combining surveys with click analysis通过对网上调查的点击情况进行分析能够获得启发

  5. Analyze clicks分析点击 • Definitions定义 • Visit访问 • Visit begins with first click on the site始于网站上的第一次点击 • To avoid counting redirects, require minimum of 5 seconds until next click 至少等5秒再点击，以区别自动重定向 • Visit ends after inactivity on the site of a specified time (e.g.,30 minutes)已终止网站活动一段时间（15到30分钟），访问即结束 • Session停留 • Within a site, session and visit used interchangeably在同一网站内，访问和停留是交替进行的 • Across sites (i.e., browser - or proxy data), session denotes a continuous string of visits跨网站（通过浏览器或代理数据） ，停留显示了一系列持续的访问 • Includes competitor sites and others包括访问竞争对手网站等 • Purchase购买 • A visit that contains a order confirmation page from the site网络访问中最后出现订购确认页面

  6. Browser collected data浏览器收集的数据 • Advantages优点 • Advantage of browser-side data 主要优点 • Behavior across sites跨网站的用户行为 • Of course, also behavior within site (for a smaller subset of server log)在某特定网站上的行为 • Caching of web pages no problem (compared to site data and proxy data)网页缓存 • Disadvantages缺点 • User needs to download software需要网民下载软件 • Privacy issues隐私问题 • MediaMetrix provides URL’s onlyMediaMetrix仅提供URL数据 • No purchase data而非购买数据 • No site characteristics无其它网站特征

  7. An early study of online customer behavior Johnson et al. (1999): The Data 约翰逊等就网络消费行为所作的早期研究：数据 • MediaMetrix Panel from July 1997 through Feb. 99. (June 98 for CD, Books) MediaMetrix调查小组，从1997年7月到1999年2月（1998年6月对CD、图书的调查） • Total of 19,465 panelists, capture every URL.总计19465个成员，抓取每个URL • There was still little shopping June ’97-Feb. ’99 97年6月-99年2月，购物行为仍然很少

  8. Research Questions 对一些问题的研究 • Characterize paths through website 刻画访问网站的路径特征 • “Modeling Online Browsing and Path Analysis Using Clickstream Data”通过利用点击流数据来为在线浏览和路径分析建立模型by Alan L. Montgomery, Shibo Li, KannanSrinivasan, and John C. Liechty.Marketing Science (2004). • Predict intention and modality of the visit 预测访问者的访问意图和模式 • “Seize the Occasion” 抓住机会by Horacio D. Rozanski, Gerry Bollman, and Martin Lipman. Strategy and Business (2001). • Understand and influence conversion 理解并影响转变率 • “A Model of Web Site Browsing Behavior Estimated on Clickstream Data” 在点击流数据的评估中被认为是网站浏览行为的模范by Randolph E. Bucklin and CatarinaSismeiro, J of Marketing Research 40 (2003). • “Dynamic Conversion Behavior at E-Commerce Sites” 电子商务网站中活跃的转变行为by Wendy W. Moe and Peter S. Fader. Management Science (2004). • Compute and apply customer network value 测定并利用消费者网络价值 • “Mining the Network Value of Customers” 挖掘消费者的网络价值by Pedro Domingos and Matt Richardson, KDD-2001. ACM Press.

  9. 加入购物车

  10. 加入购物车 戴维斯的专辑 Also want to make recommendations!添加推荐! 去收银台 等一下！再加16.01美元就能……

  11. 购买戴维斯专辑的消费者 也买了另两张专辑 想买戴维斯专辑的消费者 同时浏览了其它同类专辑 Also want to make recommendations!添加推荐! • Substitutes (buy instead of)替代（选择购买） • Customers who shopped for X also shopped for Y (based on clicks)Z买了X的消费者也会再 • Complements (buy in addition to)互补（额外购买） • Customers who bought X also bought Y买了X的消费者会再买Y

  12. Result: Right vs Left对比结果：左还是右 • Metrics　衡量标准 • Conversion rate: Percentage of visits placing an order转化率：下订单的浏览者所占的比例 • Order size: Number of additional (from the second page) items put into cart 订单大小：（从第二页起）新购商品数量 • Result　结果 • “Your Shopping Cart” on right is about 1% better than on left “Your Shopping Cart”置于右侧比置于左侧的效果提高1% • Some details　细节

  13. Amazon Shopping Cart Recommendations亚马逊的购物车推荐 • Situation and idea • Add an item to your shopping cart at a website在网站上把某种产品加入你的购物车 • Most sites show the cart大多数网站都有购物车 • At Amazon, Greg Linden had the idea of showing recommendations based on cart items在亚马逊，格雷格提出了根据购物车显示推荐 • Evaluation评价 • Pro: cross-sell more items优点：多买产品 • Con: distract people from checking out – VP asked to stop work on this idea缺点：使人们在结帐时受到干扰- VP 提出停止这个想法 • As with many new things, hard to decide正如许多新鲜事物，很难判断 • A/B test was runA/B 实验还在继续 • Feature is live on the site这项特点在网站上很流行 • from Greg Linden’s Blog:http://glinden.blogspot.com/2006/04/early-amazon-shoppingcart.html

  14. Discussion讨论 • Strength and weaknesses of such experiments?此类试验的优缺点? • Examples　举例 • Search Inside the Book　图书内容全文搜索 • Sponsored Links广告链接 • Pricing, promotions, …　　定价、促销…… • Short-term hit vs sustainable long-term effect短期冲击还是长期持续效果 • The science behind it • Generalize from individual experiments to create scientific insights总结试验结果，树立科学观点 • Design experiments appropriately精心设计试验

  15. Controlled Experiments核对实验 • Many names for same concept一些概念的名称 • A/B testsA/B 实验 • Treatment and control groups处理和控制团队 • Controlled experiments核对实验 • Randomized Experimental Design随机实验设计

  16. Controlled Experiments核对实验 • Main idea主要思路 • Randomly split traffic between two versions在两个版本之间随机分流 • Treatment: New idea(s)处理：新想法 • Control: Current live version控制：目前的现存版本 • Collect metrics of interest搜集兴趣度量 • Analyze分析 • Statistical tests统计实验 • Data mining数据挖掘

  17. How old is this methodology?方法论的历史 • First known controlled experiment in the 1700s核对实验初次被人们熟知是在18世纪 • British captain noticed lack of scurvy in Mediterranean ships英国的船长发现在地中海的船只上存在坏血病（因缺乏维生素C所致） • Had half the sailors eat limes (treatment), half did not (control)一半的船员吃柠檬（治疗），一半的不吃（控制） • Experiment was so successful, British sailors are still called limeys实验非常成功，至今英国的海员还被称为limeys（英国佬） • Note: Success despite no understanding of vitamin C deficiency注意：尽管对缺乏维生素C不了解，但是成功了

  18. Advantages of Controlled Experiments核对实验的优点 • Controlled experiments test for causal relationships, not simply correlations核对实验是关于因果关系的实验，而非简单的相关。 • They insulate external factors它们与外部因素无关 • They are a standard requirement for drug approval by US Federal Drug Administration (FDA )它们被美国联邦药品管理局作为药品审核的标准要求 • But like most great things, there are problems and it’s important to recognize them…但是正如大多数事情一样，它们也存在问题，并且认清这些问题很重要

  19. Issues with Controlled Experiments核对实验的相关问题 If you don‘t know where you are going, any road will take you there如果你不知道要往哪里去，任何路都可以带你去。 —Lewis Carroll李维斯卡尔 • Organization has to agree on key metric(s) to improve组织不得不在一些主要的度量准则上取得一致来提高 • While it may seem obvious that we need to know if we’re improving, it’s not easy to get clear agreement因而很显而易见，我们需要了解我们是否正在提高，得到清晰的一致意见并不容易 • If nothing else, bringing this question to the surface is a great benefit to the org!如果没有其他的，那么把这个问题提到台面上来对于组织有很大的益处

  20. Issues with Controlled Experiments核对实验的问题 • Quantitative metrics, not always explanations of “why”定量的度量原则，不能总是用来解释“为什么” • For example, we may know that lemons work against scurvy, but not why; it may take a while to understand vitamin C deficiency例如：我们可能知道柠檬可以治疗坏血病，但是并不知道为什么；可能需要一些时间才能了解释维生素C缺乏 • Data Mining may help identify segments where difference is large, leading to better understanding数据挖掘可以帮助识别出不同之处，进一步增加了解 • Usability studies also useful at explaining可用性研究在解释方面也非常有用

  21. Issues with Controlled Experiments控制实验的问题 • Primacy effect首要的影响 • Changing navigation in a website may degrade the customer experience (temporarily), even if the new navigation is better网站导航的改变将会减少顾客的使用（暂时的），即使新的导航比较好 • Evaluation may need to focus on new users, or run for a long period评价应该注重于新用户，或者是长期的运营 • Multiple experiments多项实验 • Even though the methodology shields an experiment from other changes, statistical variance increases making it harder to get significant results即使这种方法使得实验免于改变，统计的变化还是使得得到更好的结果变的艰难 • It is useful to avoid multiple changes to the same “area.” QA also becomes harder when tests interact对于同一区域避免多项变化比较有用，但是当实验时互动的时候，也会变得艰难

  22. Issues with Controlled Experiments控制实验的问题 • Consistency/contamination一致性与缺陷 • On the web, assignment is usually cookie-based, but people may use multiple computers, erase cookies, etc. Typically a small issue在网络上，任务通常是基于cookie的，但是人们可以使用多个计算机，删除cookie等等。通常这只是小问题 • Launch events / media announcements sometimes preclude controlled experiments开展事件/媒体公告有时可以排除控制实验 • The journalists need to be shown the “new” version记者需要看到新的版本

  23. Issues with Controlled Experiments核对实验的问题 • Short-term vs. Long-term短期与长期Hard to assess long term effects, such as customer abandonment很难评估长期影响，例如客户流失 • Example: if you optimize for ads for click-through revenues, you might plaster the site with ads. Long-term concerns should be part of metric (e.g., revenue per pixels of real estate on the window)例如：如果你们为了点击收入而优化广告，你可能要用广告来重新建设你的网站。长期的考虑应该是规测的一部分。（例如：窗口上每个象素的收入）

  24. 蓝色长裤 搜索： 广告链接

  25. Long-term effects 长期影响 • Displaying Google’s “sponsored links” within Amazon.com’s site? 在亚马逊网站上显示Google的“友情链接”？ • Initial effect positive…初始效果是积极的…… but how to model whether this helps or hurts the company in the long run?…但是怎样模拟这给公司带来的长期影响是好是坏？ • Offer free shipping 提供免费送货 • Competitors quickly adopt, how to model the overall game? 竞争对手很快就会跟进，怎样模拟整个博弈过程？ • Pricing, Promotions… 定价，促销…

  26. Lesson : A/B Experiments第一课： A/B试验 • (1) Formulate relevant questions 设计问题 • (2) “Ask” your customers:让你的客户回答问题 • Decide on set of metrics确定衡量标准 • Create a set of actions设计行动 • Run A/B test开始A/B试验 • Learn from the experiment总结试验结果 • Act行动

  27. Action: Invest USD 10M to improve customer satisfaction行动：投资1000万提高客户满意度 • Base decision on analysis of behavioral data将决策建立在对行为数据的分析基础上 • Quantify数量化 • Model模型 • Act行动 • Consider 考虑 • Increase selection? 增加选择？ • Increase availability? 保证供给？ • Reduce clutter on web site? 减少网站的杂乱性？ • Improve product search algorithms? 改进产品搜索算法？ 所以，如大家所见，自从我们停用投诉表以后，用户满意度得到了显著的提升

  28. Survey 调查问卷 • “…. Please let us know what you plan to do at Amazon.com today”“……请告诉我今天你计划在亚马逊网站上做什么” • Free responses, hand-coded into non-exclusive categories自愿回复，将回复进行分类编码，归入一些非互斥的类型 • Multiple assignments possible可能回复多种可能 • Average: 1.2 categories per response平均：每个回复涵盖1.2个种类 • Number of responses: 1023回复数量：1023 • Response rate: 3.1%回复率：3.1% • Date: February 11-12, 2003日期：2003年二月11-12日

  29. Why do people visit?　为何访问网站

  30. Why do people visit?　人们为什么访问网站？

  31. Stated vs revealed preferences 自认的偏好与实际的偏好 • Obtain insights by combining individual survey response with click analysis:结合个人调查问卷的回复与点击情况分析，得出一些结论 • Look at those who ended up buying something:最终购物的网民中 • Only about one-half of those making a purchase indicated that they wanted to buy something in this visit只有1/2事先称计划购物 • Look at those who said they wanted to buy something:最初想来购物的网民中 • Only about one-third of those indicating intent to buy ended up making a purchase in that visit只有1/3最终完成购物  

  32. Customer satisfaction客户满意度 • Relate satisfaction rating to:把满意度评价与下列两项联系起来： • Visit intent (from survey)访问意图（来源于调查问卷） • Actions (from clickstream)行动（从其点击行为看） • Intermediate goal中期目标 • Predict satisfaction预测满意度 • Ultimate goal最终目标 • Increase satisfaction提高满意度 • Number of responses: 6117回复的数量：6117 • Dates: March 15–24, 2003日期：2003年3月15-24日

  33. Outcome of the analysis分析结果 • Created a9.com for web search and product search诞生了a9.com，致力于网络和产品研究的附属机构

  34. How long ago did a customer first look at the detail page of an item eventually purchased? (i.e., conditioning on purchase) 对于最终购买的物品，该顾客第一次访问该物品的详细页面发生在多久以前？（视购买的东西不同而不同） 在购买和访问相关详细信息之间相隔的周数 购买日

  35. How long do customers take between adding an item to cart and checking out? 在访问与订购之间的时间 5%of items are purchased after having been in the cart for more than one week5%的商品从放入购物车到最后购买之间的时间长于一周 90.5% of ordered items were in cart for less than 24 hours before purchase所订购的商品中90.5%是在购买前24小时之内放入购物车中的 天数 (6)-37

  36. What makes great groups Great Groups?是什么使得一个优秀的团队优秀？ Source来源: Warren Bennis, USC, 1997http://www.leadertoleader.org/knowledgecenter/L2L/winter97/bennis.html

  37. A shared dream共同的梦想 • At the heart of every Great Group is a shared dream.每一个伟大的团队的核心是一个共同的梦想。 • All Great Groups believe that they are on a mission from God, that they could change the world, make a dent in the universe. They are obsessed with their work. It becomes not a job but a fervent quest. That belief is what brings the necessary cohesion and energy to their work. 每一个伟大的团队都坚信他们有着上帝赋予的改变世界的使命。他们痴迷于他们的工作，工作已经不是所谓的工作而是一种热烈的寻求。 这种信仰赋予他们工作的凝聚力和能量。 • E.g., Google wants to make the world’s information accessible例如： Google（谷歌）的使命：让全世界所有的信息都可以访问 • Source来源: Warren Bennis, USC, 1997http://www.leadertoleader.org/knowledgecenter/L2L/winter97/bennis.html

  38. Price to be part of the group高额的代价是团队的一部分 • Members pay a personal price. 团队成员付出个人代价 • Membership in a Great Group isn‘t a day job; it is a night and day job. Divorces, affairs, and other severe emotional fallout are typical, especially when a project ends. So groups strike a Faustian bargain for the intensity and energy that they generate. 一个优秀团队的成员不仅是白天工作，而是白天晚上都工作。离婚，婚外情和其它严重的感情波动，特别是当一个项目结束的时候。所以，团队在与其产生的凝聚力和能量之间讨价还价。

  39. Curators, not creators监护人，而非创造者 • Great Groups make strong leaders. 优秀的团队造就强有力的领导者。 • On one hand, they‘re all nonhierarchical, open, and very egalitarian. Yet they all have strong leaders. That’s the paradox of group leadership. You cannot have a great leader without a Great Group -- and vice versa. In an important way, these groups made the leaders great. The leaders … were connoisseurs of talent, more like curators than creators.一方面，优秀的团队是不分等级，公开、平等的。，然而他们都有非常强的领导者。这是团队领导的矛盾。没有优秀的团队就没有优秀的领导者，反过来，优秀的团队使得领导者更加优秀。领导者们是人才的专家，更像是监护人而非创造者。

  40. Abandon egos for the dream为了梦想放弃自我 • Great groups manage conflict by abandoning individual egos to the pursuit of the dream. 团队成员在追求梦想过程中通过放弃自我来解决冲突。 • At a critical point in the Manhattan Project a great chemist who later served as Dwight Eisenhower‘s chief scientific advisor threatened to quit because he couldn’t get along with a colleague. Project leader Robert Oppenheimer simply said, “George, how can you leave this project? The free world hangs in the balance.” So conflict, even with these diverse people, is resolved by reminding people of the mission. 曼哈顿项目的一个关键点上一名卓越的化学家，后来在德怀特 艾森豪威尔公司任首席科学顾问，因为不能和一名同事融洽相处而面临辞职。该项目领导人Robert Oppenheimer 仅仅说到：“乔治，你怎么能离开这个项目呢？” 因此，冲突特别是和不同的人发生的，最终通过提醒他团队的使命而解决。 • E.g., Amazon, Google et al focus on measurement.亚马逊，谷歌等等，专注于测量。

  41. Recruiting聘用 • Great Groups are the product of meticulous recruiting. 优秀的团队是精心聘用的产物。 • Cherry-picking the right talent for a group means knowing what you need and being able to spot it in others. It also means understanding the chemistry of a group. Candidates are often grilled, almost hazed, by other members of the group and its leader. You see the same thing in great coaches. They can place the right people in the right role. And get the right constellations and configurations within the group. 精心聘用适合的人才意味着明白你所需要的类型，并能在其他人中发现他们。这也意味着对团队组成的了解。应聘者通常被考核，甚至经受一些被其它团队成员和领导者制造的挫折。你会在优秀的领导者身上发现相同的东西。他们把合适的人放在合适的位置。使整个团队人才济济，结构合理。 • Hiring practice at Amazon, Microsoft et al. And firing practice (“top grading”)亚马逊，微软等的聘用原则：雇用最优秀的人

  42. Age年龄 • Great Groups are usually young. 优秀的团队通常比较年轻。 • The average age of the physicists at Los Alamos was about 25. Oppenheimer -- “the old man” -- was in his 30s. But Great Groups are also young in their spirit, ethos, and culture. Most important, because they‘re young and naive, group members don’t know what‘s supposed to be impossible, which gives them the ability to do the impossible.” Great Groups don’t lack the experience of possibilities. Los Alamos 的物理学者平均年龄是25岁，其中，奥本海默是最年长的，三十多岁。优秀的团队在精神，气质和文化方面也同样年轻。最重要的是，因为团队成员的年轻率真，他们不知道什么是不可能的事情，从而使他们有能力去做很多不可能的事情。优秀的团队并不缺乏可能的经验。 • Average age of employees at Google in 2007: 28 years.谷歌：2007年员工的平均年龄：28岁

  43. Deliver成果 • Real artists ship. 真正的艺术家 • Steve Jobs constantly reminded his band of Apple renegades that their work meant nothing unless they brought a great product to market. In the end, Great Groups have to produce a tangible outcome external to themselves. Most dissolve after the product is delivered; but without something to show for their efforts, the most talented assemblage becomes little more than a social club or a therapy group.史蒂夫 乔布经常提醒他的团队（苹果叛徒乐队）除非为市场创造优秀的产品否则他们的工作一文不值。最终，优秀的团队必须有切实的成果。大多数团队在产品推出后解散，他们的努力有时并不会显现。最有才华的团队比一个社交俱乐部或治疗团队要大的多。 • Yes!是的

More Related