Peer Interaction Effectively, yet Infrequently, Enables Programmers to Discover New Tools Emerson Murphy-Hill North Carolina State University Gail Murphy University of British Columbia
Background Emerson’s Problem • I was making a bunch of new user interfaces for refactoring tools • They made it easier for programmers to restructure their code • But what if people didn’t know of the tools existed?
Background Gail’s Problem • Was working on a recommender system called Spyglass that recommended tools in Eclipse • In a longitudinal study, only 2 out of 7 participants thought that the recommendations were useful But we really expected that a recommender system would help! = ? MS Clippy
Spyglass (and Clippy) is similar to recommendations made by peers, which may be effective • Literature suggests it’s similar; recommender system should “guide and advise an user similar to a knowledgeable colleague or assistant” [Fisher 84] • Literature suggests learning from peers works “exceedingly effectively” [Lave and Wegner 91] Why we Thought Spyglass Would Work But we really expected that a recommender system would help! = ? MS Clippy • But we don’t really know how tool recommendations are made between peers
Peer Recommendation You discover when a peer is watching you not use a tool when you could be, then she recommends it Peer Observation You discover by watching a peer using a tool that you didn’t know about We call these both Peer Interaction, one kind of discovery mode A bit more about what we were looking for
Sent invitations to 62 participants of Open Source Bridge 2009 Asked personal contacts at 7 large companies to distribute 18 participants total All programmed as part of their jobs, though not necessarily “Programmers” Participants
Semi-structured phone, Skype, or chat interviews • Gave participants a list of tools, asked participants to pick a tool and tell me how you discovered it • Gave participants a list of discovery modes, asked participants to pick 2 most effective • Asked participants to recall instances of peer interaction • Asked about perceptions of Spyglass Methodology Frequency Effectiveness Process RecSys
Frequency Effectiveness Process RecSys
RecSys Process Frequency Effectiveness
RecSys Process Frequency Effectiveness One cause: Learner associates use with context of use
RecSys Process Frequency Effectiveness One cause: distributed, asynchronous development.
After we outlined what Spyglass does, several participants immediately mentioned Clippy “…and Clippy should just go die as far as I’m concerned.” Or put more politely: “Honestly; I bet the [recommender system] would have better success rate [than a peer] at recommending things that I would like, but that doesn’t mean that I would trust the [recommender system] more.” RecSys Process Frequency Effectiveness
For recommender systems: No matter how accurate, how timely, how politely, or how valuable the recommendation is, programmers don’t trust recommender systems But what if, rather than foisting recommendations on people, why don’t we just encourage more frequent peer interaction? Implications
How can people who work asynchronously and remotely learn from each other? Making Peer Interaction More Frequent http://www.flickr.com/photos/totalaldo/2400635097 http://www.flickr.com/photos/novecentino/2339687721/
For each developer, keep a local history of: • what tools he uses • a full “movie” of his development history Compare tool histories between all developers Suggest either: • To Rob: “Your co-worker, Tim, uses ToolX. Would you like to see an example of him using it yesterday?” or • To Tim: “Your coworker, Rob, doesn’t seem to know about ToolX. Would you like to share with him an example of you using it?” Making Peer Interaction More Frequent http://www.flickr.com/photos/totalaldo/2400635097 http://www.flickr.com/photos/novecentino/2339687721/
Conclusion Programmers have thousands of useful tools available.Peer interaction can help programmers effectively discover new tools,but they need support for more frequent interactions.