1 / 21

Supporting Test-Driven Development of Graphical User Interfaces Using Agile Interaction Design

Third International Conference on Software Testing, Verification, and Validation Workshops 2010 IEEE ICSTW. Theodore D. Hellmann, Ali Hosseini-Khayat , Frank Maurer. Supporting Test-Driven Development of Graphical User Interfaces Using Agile Interaction Design. 報告人 : 歐陽玉慧. 2011/5/30.

zamir
Download Presentation

Supporting Test-Driven Development of Graphical User Interfaces Using Agile Interaction Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Third International Conference on Software Testing, Verification, and Validation Workshops 2010 IEEE ICSTW. Theodore D. Hellmann, Ali Hosseini-Khayat, Frank Maurer Supporting Test-Driven Development of Graphical User Interfaces Using AgileInteraction Design 報告人: 歐陽玉慧 2011/5/30

  2. outline • Abstract • I. INTRODUCTION • II. PROBLEM • III. RELATED WORK • IV. OUR APPROACH • V. LIMITATIONS • VI. FUTURE WORK • REFERENCES

  3. Abstract • TDD of GUIs 目前的兩難: • TDD 需要發展中的app有一定的穩定度, 來避免tests 去頻繁變動, 但 GUIs 在開發過程中卻很有可能要常變. • 建立GUI tests 最簡單的方法就是用capture-replay tool , 但必須GUI先存在才能使用. • 本文介紹一種新的user interface TDD(UITDD) 方法, 利用CRT 由 low-fidelity prototypes 紀錄 test scripts. • 可使 GUItests 更好寫 且 不需要先有GUI

  4. I. INTRODUCTION • TDD的好處與欠缺 • 好處: • encourages communication between customers and developers, • increases programmer confidence, • increases software quality, • decreases bug density without decreasing productivity[1][2] • 欠缺: 大部分的TDD 研究仍未解決TDD of graphicaluser interfaces (GUIs), 兩個最大的挑戰: • GUIs 在開發過程中很有可能要常變  使得tests要經常修改(很不直覺) • 建立GUI tests 最簡單的方法 capture-replay tool (CRT), 必須GUI先存在才能定義 tests.

  5. I. INTRODUCTION(cont’) • 本文的作法與好處 • 作法:利用CRT結合 low-fidelity prototyping tool • 收集 details of userstories • 用這些stories建立系統的 low-fidelity prototype. • 進行prototype的usabilityevaluations找出且修復UIflaws • 重複 repeated 上述流程直到 prototype 夠stable. • 用最後的prototype 加上預期的GUIbehavior 資訊, CRT 就能記錄複雜的 acceptance tests. • 此tests 就能被發展中的GUI去run確保符合user期待(在prototype 上的定義)

  6. I. INTRODUCTION(cont’) • 本文的作法與好處 • 好處:利用夠明確的prototype進行agileinteraction design 能得到兩大好處 • 開發初期就能找出usability concerns, 最後的GUI就比較不用常改 • 若prototype 能加上一些自動化資訊(用來識別與確認widgets), 同時這些資訊實際實作中又可維護,  tests就能用prototype紀錄 且 在實際GUI上replayed.

  7. II. PROBLEM • TDD核心優勢與應用在GUI-based app之屏障 • 核心優勢: • tests不用code存在就可先寫, 之後就可以寫讓 tests pass的code. • 要能重整既有的code, 而不用擔心會破壞 tests. • 兩大屏障讓TDD無法有效應用在GUI-based app: • 重整或修改GUIs就會破壞其tests (even 改變跟semantic 無關) • GUI tests 很難用人工撰寫, 雖可用CRT簡化但GUI一定要先存在, 這代表UITDD一定只能靠人工撰寫.

  8. II. PROBLEM (cont’) • GUI tests之問題與可能的solutions • 雙重負擔:(GUI常變, 其test suite就要重複維護) • 維護費用會持續不斷(substantial) • 若開發者視錯誤為“the test’s fault”而非程式可能的bugs就會更不願意改程式且不會認真看待 failing tests( 這比上一個問題更嚴重, 因為TDD之核心 benefits 就是 developer confidence) • Solutions • Team 可試著減小GUI及其test suite必要的改變 • 現已有可自動修改或夠簡單的方法可修改 broken GUI tests(他人作法: Section III, 作者方法:Section IV)

  9. III. RELATED WORK • A. Capture-Replay Tools • CRTs:運用紀錄 GUI的interactions並且存成 a sequence of actions可在GUI上replayed(但根本的難題就是 widgets 無法被 test code 存取) 1. 初版CRTs解法: 只用 keyboard input 及 screen position of mouseclicks 來紀錄的 test script可簡單的replay這些動作, 但卻無法驗證正確的系統行為, 且依賴screen coordinates有很大的缺點(對GUI之non-semanticchanges非常敏感,如重排 widgets 都會造成 test failures) 2. 次版CRTs作法: 用object maps(存夠多的widget資訊測試時就可作fuzzy match [5], [3])的 testing, 可使tests應變力更強,test時可存取widgets去測試其behavior, 但此法很難寫code(因為要正確定位widget所需的資訊量太大) 3. Keyword-based testing是最近發展的GUI testing技術, 利用給每個widgets設定unique keyword(而非儲存大量的widgets資訊),好處就是測試時只要找一個keyword 且與特定widget之interact(可視為一個透過CRTs寫test的簡單且健全的方法)

  10. III. RELATED WORK • B. User Interface Test Driven Development • 近年來已有發展出可支援UITDD的工具 • 提供 framework 支援 widgets identification 與interaction更簡單且健全作法,簡化人工 GUI test(更健全的做法還有把測試存到intermediateform 及用來測試 widgets 的intermediate representation以輔助 test maintenance [6].) • 缺點: • 雖可減少UITDD的effort,但仍要用人工撰寫code(容易產生錯誤, 且有agile team證實此法要一直修改tests, 用CRTs重新紀錄tests都比修改原本的tests還要快)

  11. III. RELATED WORK • C. Support for Test Script Maintenance • 支援test maintenance 的tool也是最近的研究主題: • Compiler-inspired approach[9]: 不需要人工互動就能建立一個 legal的test steps sequence自動取代 broken test 事件 • TIGOR system[5]: 利用加入明確的 GUI test scripts type 資訊來簡化 manual maintenance • REST[3]: 產生App程式內widgets與test內的使用之 關聯資訊來建議 test script where and why 可能 fail • Actionable Knowledge Models [10]: 把 tests 存在中間 model中, 可把導致test failure 的根源紀錄在single location而不用人工放在 individual test scripts.

  12. IV. OUR APPROACH • A. Tools • ActiveStory Enhanced (supports agile interaction design) • 讓usability engineers能建立 low-fidelity prototypes. • 這些prototypes是由一組有各式各樣UIstates 及“hot zones”(涵蓋了引起states間轉換的特定範圍的 clear transparent buttons)的pictures所構成 • 可很便宜的建構及調整low-fidelity prototypes, 讓ActiveStory Enhanced 可以用很便宜且分散進行多人的可用性測試(usability tested)

  13. IV. OUR APPROACH(cont’) • A. Tools • LEET(利用 Microsoft’s User Interface Automation Framework (UIAF) 所發開發的CRT [12]) • 此Framework 可提供 keyword-basedtesting(此法基本) • 唯一可用來辨識widget的屬性就是利用其AutomationID, 所以就可能從ActiveStory Enhanced prototype 來記錄test 並且可replay在actualGUI上. • 這可利用在每個hot zone附加一個AutomationID並確定GUI上相對的 widget 也會使用同一個ID來達成.

  14. IV. OUR APPROACH(cont’) • B. Process • Process • 建 User stories: 用ActiveStory Enhanced 建GUI low-fidelityprototype • Usabilityevaluations: 進行此prototypes的可用性評估, 可減少最後的GUI所需的改變(low-fidelity prototypes 可快速建構且少成本, 對 iteratedagileusability evaluations最為理想) • 加入自動化資訊: 把2.得到的 stableprototype加上自動化資訊就可進行複雜的verifications • 建tests: 用LEET紀錄prototype上的互動為一組 acceptancetests • Run:此 tests就能在開發GUI-based app過程中執行testing

  15. IV. OUR APPROACH(cont’) • B. Process • 好處: • UITDD沒有了附加限制:可用最簡單的工具CRT來建立tests, 代表不用人工撰寫 test code--用既有工具就可建UITDD • 降低維護成本:因為實作前已經先做了usability testing, 這個部分來來回回的設計工作可因為這項好處及user購買量的增加來得到補償 • 減少broken tests:雖有工具可輔助修正broken tests, 但最佳的作法還是在broken tests第一次發生時就減少之(這可避免上述其他UITDD方法的遇到的問題)

  16. IV. OUR APPROACH(cont’) • C. Example • UITDD以Windows 7calculator為例, 設計考量含: • keys representing numbers, • keys representing operators, • a display 放在最上面顯示輸入的數字或運算結果 • 此例目前就只先考慮addition feature. • In this story, the fivebutton, plus button, nine button, and equals button areclicked in that order, and we expect that the displayshouldread “14” at the end.

  17. IV. OUR APPROACH(cont’) • C. Example AutomationID會設成“Five”(當實際的GUI也把實際的“5”按鍵設為一樣的ID, test 就會找到並且按下它(如prototype所設計)) hot zone 1.建 storyboard(Fig1): 此 test sequence 可在ActiveStory Enhanced上的prototype建構出的 storyboard 2. 建test(Fig2): 用 LEET紀錄此prototype之 sequence of interactions 作為 test script 在UIAF, 用來顯示text的widgets會設Name屬性對應該text

  18. IV. OUR APPROACH(cont’) 5. 當 interface 被完成後, 此測試仍可適當地運作Fig4. • C. Example 3. 實作GUI: 用WPF實作可將每個widgetname屬性設定的值視為自動 AutomationIDs. 4. Runtest: 先修改 Fig2.的 Start動作指向實際的GUI 結果如 Fig3. (因為還沒寫邏輯所以再跑Fig2第二行到最後test會先失敗, 讓test可找到並執行動作是因為keyword-based的測試,容許widgetsresized, moved, even改成相似types都不會破壞test script,)

  19. V. LIMITATIONS • Evaluation is upcoming(本法可用在UITDD上, 但尚未證實其 practicality or usefulness) • 短期:要求開發者進行簡單應用系統的不同功能的UITDD, 提供本法對於個人是否實用及對應用系統發展有多少幫助之相關資訊 • 長期:實際用在業界一段時間確認本法對開發團隊的實際可用性 • 未可證實之假設: • 對GUI之prototype重複進行usability evaluations, 開發過程中開發者對GUI要修改的量就會比較少 • 本法測試所用之low-fidelity prototype 有其限制: • 除了 buttons, hyperlinks 其他如text boxes, draggable items, gestures還沒有工具支援,還是必須人工撰寫測試

  20. VI. FUTURE WORK • 完成evaluation後, 就開始探索mixed-fidelityUITDD(結合low-fidelity prototypes 的手繪元件與實際的widgets) • 建立prototype widgets與app實際功能的橋梁 • 漸進式的替換 low-fidelity prototype 成 functionalwidgets, GUI就能由這些 prototype 漸進地開發 (此法可避免用 low-fidelity prototype 紀錄的tests 但卻在不同GUI 上執行的gap)

  21. REFERENCES [1] R Jeffries and G. Melnik, "Guest Editors' Introduction: TDD - The Art of Fearless Programming," IEEE Software, pp. 24-30, 2007. [2] N. Nagappan, E. M. Maximilien, T. Bhat, and L. Williams, "Realizing Quality Improvement through Test Driven Development: Results and Experiences of Four Industrial Teams," in Empirical Software Engineering, 2008, pp. 289-302. [3] M. Grechanik, Q. Xie, and F. Chen, "Maintaining and Evolving GUIDirected Test Scripts," in IEEE 31st International Conference on Software Engineering, 2009, pp. 408-418. [4] A. Holmes and M. Kellogg, "Automating Functional Tests Using Selenium," in AGILE 2006, 2006, pp. 270-275. [5] C. Fu, M. Grechanik, and Q. Xie, "Inferring Types of References to GUI Objects in Test Scripts," in International Conference on Software Testing, Verification, and Validation, 2009, pp. 1-10. [6] W. Chen, T. Tsai, and H. Chao, "Integration of Specification-Based and CR-Based Approaches for GUI Testing," in 19th International Conference on Advanced Information Networking and Applications,2005, pp. 967-972. [7] A. Ruiz and Y. W. Price, "GUI Testing Made Easy," in Testing: Academic and Industrial Conference - Practice and Research Techniques, 2008, pp. 99-103. [8] A. Ruiz and Price Y. W., "Test-Driven GUI Development with TestNGand Abbot," in IEEE Software, 2007, pp. 51-57. [9] A. M. Memon and M. L. Soffa, "Regression Testing of GUIs," in ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2003, pp. 118-127. [10] Z. Yin, C. Miao, Z. Shen, and Y. Miao, "Actionable Knowledge Model for GUI Regression Testing," in IEEE/WIC/ACM International Conference on Intelligent Agent Technology, 2005, pp. 165-168. [11] Ali Hosseini-Khayat, YaserGhanam, Shelly Park, and Frank Maurer, "ActiveStory Enhanced: Low-Fidelity Prototyping and Wizard of Oz Usability Tool," in Agile Processes in Software Engineering and Extreme Programming, 2009, pp. 257-258. [12] Theodore D. Hellmann. (2010) LEET (LEET Enhances Exploratory Testing) - CodePlex. [Online]. http://leet.codeplex.com/

More Related