Spoken Programs or Programmers say the darndest things

Spoken Programs or Programmers say the darndest things PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on
  • Presentation posted in: General

Download Presentation

Spoken Programs or Programmers say the darndest things

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


1. Spoken Programs or Programmers say the darndest things! Andrew Begel, Susan L. Graham Computer Science Division, EECS University of California, Berkeley

2. VL/HCC 2005 Motivation Programmers conventionally use keyboard Long hours at keyboard leads to higher risk of RSI Can a programmer code using speech? Can a computer understand what the developer says? If the user can say anything, this makes it really difficult to understand. Provide motivations for doing this work. Learning about more natural modes of interaction Put why programming by voice on this slide. Introduce programming by voice as a solution. We use conventionalized programmign languages bcause of training and tools. Not the best way to do things, but too hard to change, so instead we’ll modify the Input to the programming language to make it easier for people. (programming ambiguous because it makes the compielr and validation easier). Already existing tools for adapting to programming languages (declaration before def) Given the programming domain Here are the problems we have with them. If the user can say anything, this makes it really difficult to understand. Provide motivations for doing this work. Learning about more natural modes of interaction Put why programming by voice on this slide. Introduce programming by voice as a solution. We use conventionalized programmign languages bcause of training and tools. Not the best way to do things, but too hard to change, so instead we’ll modify the Input to the programming language to make it easier for people. (programming ambiguous because it makes the compielr and validation easier). Already existing tools for adapting to programming languages (declaration before def) Given the programming domain Here are the problems we have with them.

3. VL/HCC 2005 Programming by Voice Our Goal Find out how developers use code verbally. Use this to develop a naturally verbalizable input form. Build development environment that supports verbal authoring, navigation, modification. Extend conventional compiler analyses to support ambiguities generated by speech. Learn how developers can use voice-based programming, and iterate design. speak whole story, then say we’re going to talk about the first one. What we want to do: comp, nav, edit speak whole story, then say we’re going to talk about the first one. What we want to do: comp, nav, edit

4. VL/HCC 2005 Programming by Voice Challenges Speech is inherently ambiguous. Programming languages and tools were designed to be unambiguous. Speech tools are poorly suited for programming tasks. Programmers are not used to verbal software development. We use conventionalized programmign languages bcause of training and tools. Already existing tools for adapting to programming languages (declaration before def) (programming not ambiguous because it makes the compiler and validation easier). Introduce programming by voice as a solution. Not the best way to do things, but too hard to change, so instead we’ll modify the input to the programming language to make it easier for people.We use conventionalized programmign languages bcause of training and tools. Already existing tools for adapting to programming languages (declaration before def) (programming not ambiguous because it makes the compiler and validation easier). Introduce programming by voice as a solution. Not the best way to do things, but too hard to change, so instead we’ll modify the input to the programming language to make it easier for people.

5. VL/HCC 2005 Programming by Voice for (int i = 0; i < 10; i++ ) { ¦ }

6. VL/HCC 2005 Current Tools are Awkward! for (¦ ; ; ) { } Existing tools badly adapt VR to programming (No compiler technology) More explicitly list out loud what the awkwardness is. Requires users to spell identifiers VR finds small words hard to understand. prescriptive way to enter the program once code is in editor, it can no longer be edited by template. code entry is over-stylized no solution for nav and editing Existing tools badly adapt VR to programming (No compiler technology) More explicitly list out loud what the awkwardness is. Requires users to spell identifiers VR finds small words hard to understand. prescriptive way to enter the program once code is in editor, it can no longer be edited by template. code entry is over-stylized no solution for nav and editing

7. VL/HCC 2005 for (int i = 0;¦ ; ) { } More explicitly list out loud what the awkwardness is. More explicitly list out loud what the awkwardness is.

8. VL/HCC 2005 for (int i = 0; i < 10;¦ ) { } More explicitly list out loud what the awkwardness is. More explicitly list out loud what the awkwardness is.

9. VL/HCC 2005 for (int i = 0; i < 10; i++ ) { ¦ } More explicitly list out loud what the awkwardness is. not the first person to worry about this stuff, here’s what people have to live with now. More explicitly list out loud what the awkwardness is. not the first person to worry about this stuff, here’s what people have to live with now.

10. VL/HCC 2005 How do Programmers Speak Code? 10 programmers read Java code out loud (Begel ‘05) Graduate students in Computer Science Five knew Java, five did not Five were native English speakers, five were not Five were educated in U.S.A., five were not Read pre-written code into tape recorder As if speaking to a sophomore-level CS undergrad who knows Java, but does not know the program Most programmers spoke the same way

11. VL/HCC 2005 for (int i = 0; i < 10; i++ ) { ¦ } Natural Language Verbalization Didn’t have to verbalize parens, semicolons, etc. Didn’t have to verbalize parens, semicolons, etc.

12. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

13. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

14. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

15. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

16. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

17. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

18. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

19. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

20. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

21. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

22. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

23. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

24. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

25. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

26. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

27. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

28. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

29. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

30. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

31. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

32. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

33. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

34. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

35. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

36. VL/HCC 2005 How Do Programmers Speak Code? “Think about how to say this” “Think about how to say this”

37. VL/HCC 2005 Design Tradeoffs

38. VL/HCC 2005 Spoken Java Semantically identical to Java Syntactically easier to say than Java Design methodology generalizable to any computer language

39. VL/HCC 2005 A More Natural Way to Code public class Symbol implements Serializable { ¦ } Didn’t have to verbalize parens, semicolons, etc. Didn’t have to verbalize parens, semicolons, etc.

40. VL/HCC 2005 A More Natural Way to Code public class Symbol implements Serializable { static HashMap hashtbl = new HashMap(); ¦ } All punctuation has English equivalents Open Brace, End For Loop Most punctuation is optional Provide verbalization for all abbreviations Relaxed phrasing for better fit with English (int)foo ? “cast foo to integer” foo = 6 ? “set foo to 6” foo[i]++ ? “increment the ith element of array foo” All punctuation has English equivalents Open Brace, End For Loop Most punctuation is optional Provide verbalization for all abbreviations Relaxed phrasing for better fit with English (int)foo ? “cast foo to integer” foo = 6 ? “set foo to 6” foo[i]++ ? “increment the ith element of array foo”

41. VL/HCC 2005 A More Natural Way to Code public class Symbol implements Serializable { static HashMap hashtbl = new HashMap(); } ¦ All punctuation has English equivalents Open Brace, End For Loop Most punctuation is optional Provide verbalization for all abbreviations Relaxed phrasing for better fit with English (int)foo ? “cast foo to integer” foo = 6 ? “set foo to 6” foo[i]++ ? “increment the ith element of array foo” All punctuation has English equivalents Open Brace, End For Loop Most punctuation is optional Provide verbalization for all abbreviations Relaxed phrasing for better fit with English (int)foo ? “cast foo to integer” foo = 6 ? “set foo to 6” foo[i]++ ? “increment the ith element of array foo”

42. VL/HCC 2005 A More Natural Way to Code for (int i = 0; i < 10; i++ ) { ¦ } Didn’t have to verbalize parens, semicolons, etc. Didn’t have to verbalize parens, semicolons, etc.

43. VL/HCC 2005 Too Many Ambiguities for (int i = 0; i < 10; i++ ) { ¦ } A human could eyeball this.A human could eyeball this.

44. VL/HCC 2005 Sometimes It’s Non-Obvious fore *= 8; file.tooLode.times = won ¦

45. VL/HCC 2005 SPEED: Speech Editor Build an editor that supports naturally verbalized programs SPEED: SPEech EDitor Based on Dragon Naturally Speaking, Eclipse IDE, Harmonia Spoken Java Language for Composition Spoken Command language for Navigation, Editing, Template instantiation, Refactorings, Search Visual feedback for training

46. VL/HCC 2005 Future Study - SPEED Usability Goal: Understand how SPEED can be used by expert programmers Train expert Java programmers on SPEED Author new code Build a Linked List data structure with associated algorithms Modify existing code Change abstraction representation and update algorithms Say this is for expert java programmers. Rephrase system understanding (everything else is about the person, but this is about the system). Find some terminology from AI systems evaluations (false positive rate etc) Say this is for expert java programmers. Rephrase system understanding (everything else is about the person, but this is about the system). Find some terminology from AI systems evaluations (false positive rate etc)

47. VL/HCC 2005 Metrics User Metrics Task completion time Vocabulary mistakes, Forgotten vocabulary Grammatical mistakes, Grammatical substitutions SPEED Metrics Word tokenization errors Disambiguation errors Irresolvable ambiguities

48. VL/HCC 2005 Spoken Programs Learn how developers speak code out loud Analyze speech and categorize ambiguities Use study to design naturally verbalizable input for programming Create programming tools designed for spoken interaction Evaluate design and tools by studying programmers using voice for software development fix animation fix animation

49. VL/HCC 2005 End of Talk

  • Login