madara kats automated testing with distributed knowledge and reasoning n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
MADARA KATS: Automated Testing with Distributed Knowledge and Reasoning PowerPoint Presentation
Download Presentation
MADARA KATS: Automated Testing with Distributed Knowledge and Reasoning

Loading in 2 Seconds...

play fullscreen
1 / 25

MADARA KATS: Automated Testing with Distributed Knowledge and Reasoning - PowerPoint PPT Presentation


  • 102 Views
  • Uploaded on

MADARA KATS: Automated Testing with Distributed Knowledge and Reasoning. James Edmondson Vanderbilt Institute for Software Integrated Systems Mail: james.r.edmondson@vanderbilt.edu Project site: http://madara.googlecode.com. KATS: Table of Contents. Scenario. Overview. Modeling.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'MADARA KATS: Automated Testing with Distributed Knowledge and Reasoning' - yair


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
madara kats automated testing with distributed knowledge and reasoning

MADARA KATS:Automated Testing with Distributed Knowledge and Reasoning

James Edmondson

Vanderbilt Institute for Software Integrated Systems

Mail: james.r.edmondson@vanderbilt.edu

Project site: http://madara.googlecode.com

kats table of contents
KATS: Table of Contents

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Motivating scenario
  • Solution overview
  • Modeling tests
  • Disseminating and evaluating test knowledge
  • Automating Android smartphone tests with MAML
  • Demonstration
  • Conclusions
kats motivating scenario
KATS: Motivating Scenario

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Three computers and two smart phones connected to a network
  • The two smart phones are connected to two of the computers

network

kats motivating scenario1
KATS: Motivating Scenario

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • The phones communicate with system services on the server
  • The phones must wait for the system services to warm up before connecting (5 seconds)

server

System

Services

network

kats motivating scenario2
KATS: Motivating Scenario

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • A race condition exists if App 2 is started immediately after App 1
  • How do we automatedly test for this type of scenario?

server

Phone

App 1

System

Services

network

Phone

App 2

kats motivating scenario3
KATS: Motivating Scenario

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • If the test fails, we may have to run a separate test on another host

server

Phone

App 1

System

Services

network

Process

Results

Process

Results

Phone

App 2

Phone

App 3

Process

Results

Process

Results

  • After the tests are performed, each host must process its log results
kats motivating scenario4
KATS: Motivating Scenario

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Scenario Requirements:
  • Ability to sequence tests, preferably without a need for a centralized server (single point of failure)
  • Ability to sequence different types of programs (e.g. Python results parsers, C++ services, and Java smart phone applications)
  • Ability to disseminate knowledge of test failure and success
  • Configurable knowledge reasoning service to make sense of the incoming information
  • Portable to Windows, Linux and other operating systems
madara architectural overview
MADARA: Architectural Overview

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • EXTERNAL DEPENDENCIES
  • C++ for speed and portability
  • ACE for portable threads and OS calls
  • DDS for transport of knowledge
  • Android SDK for Android smartphone dev
  • Monkeyrunner for sending events to phones

KATS

MAML

KaRL

Monkeyrunner

ACE

DDS

Android SDK

C++

Python

Java

  • DEVELOPED TECHNOLOGIES
  • MAMLfor instrumentation of phones
  • KaRLfor knowledge and reasoning
  • KATS for automated testing
kats testing entity life cycle
KATS: Testing Entity Life Cycle

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Each testing entity has eight life cycle phases which flow linearly until completion. Each phase is disabled by default.

Barrier

Application Launch

5.

1.

Precondition

Postlaunch

6.

2.

  • The barrier condition is met when all potentially distributed entities participating in a test have reached the first phase of their life cycle.

Temporal delay

Postcondition

3.

7.

Postdelay

Exit

4.

8.

kats testing entity life cycle1
KATS: Testing Entity Life Cycle

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Each testing entity has eight life cycle phases which flow linearly until completion. Each phase is disabled by default.

Barrier

Application Launch

5.

1.

Precondition

Postlaunch

6.

2.

  • The precondition is a knowledge requirement that must be true before proceeding (e.g. ServicesStarted for Phone App 1)

Temporal delay

Postcondition

3.

7.

Postdelay

Exit

4.

8.

kats testing entity life cycle2
KATS: Testing Entity Life Cycle

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Each testing entity has eight life cycle phases which flow linearly until completion. Each phase is disabled by default.

Barrier

Application Launch

5.

1.

Precondition

Postlaunch

6.

2.

  • The temporal delay is a platform-neutral sleep of at least the specified number of seconds

Temporal delay

Postcondition

3.

7.

Postdelay

Exit

4.

8.

kats testing entity life cycle3
KATS: Testing Entity Life Cycle

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Each testing entity has eight life cycle phases which flow linearly until completion. Each phase is disabled by default.

Barrier

Application Launch

5.

1.

Precondition

Postlaunch

6.

2.

  • The postdelay is a knowledge expression to evaluate after the temporal delay has finished.

Temporal delay

Postcondition

3.

7.

Postdelay

Exit

4.

8.

kats testing entity life cycle4
KATS: Testing Entity Life Cycle

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Each testing entity has eight life cycle phases which flow linearly until completion. Each phase is disabled by default.

Barrier

Application Launch

5.

1.

Precondition

Postlaunch

6.

2.

  • The application launch creates a heavy weight process to run the user-specific executable

Temporal delay

Postcondition

3.

7.

Postdelay

Exit

4.

8.

kats testing entity life cycle5
KATS: Testing Entity Life Cycle

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Each testing entity has eight life cycle phases which flow linearly until completion. Each phase is disabled by default.

Barrier

Application Launch

5.

1.

Precondition

Postlaunch

6.

2.

  • The postlaunch phase begins with the evaluation of user-provided knowledge expressions and extends until the application exits or is killed

Temporal delay

Postcondition

3.

7.

Postdelay

Exit

4.

8.

kats testing entity life cycle6
KATS: Testing Entity Life Cycle

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Each testing entity has eight life cycle phases which flow linearly until completion. Each phase is disabled by default.

Barrier

Application Launch

5.

1.

Precondition

Postlaunch

6.

2.

  • The postcondition is evaluated after the application has exited. A return value is made available to the user-specified knowledge expression

Temporal delay

Postcondition

3.

7.

Postdelay

Exit

4.

8.

kats testing entity life cycle7
KATS: Testing Entity Life Cycle

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Each testing entity has eight life cycle phases which flow linearly until completion. Each phase is disabled by default.

Barrier

Application Launch

5.

1.

Precondition

Postlaunch

6.

2.

  • During the exit phase, the testing entity cleans up and ends its life cycle

Temporal delay

Postcondition

3.

7.

Postdelay

Exit

4.

8.

kats testing entity modeling
KATS: Testing Entity Modeling

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • A GME DSML paradigm is provided for modeling all testing entities:
  • Visual interface for configuring entities and their life cycles
  • The modeled entity attributes are generated into an XML file
  • Recursive includes are allowed to chain entity groups and provide for intricate testing sequences
  • Provides specialized KATS processes like the KATS Observer, which allows for knowledge monitoring of active tests
kats disseminating knowledge
KATS: Disseminating Knowledge

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • KATS uses the MADARA KaRL reasoning engine
  • Portable reasoning engine built for distributed real-time knowledge services
  • Microsecond latency on evaluations
  • Capable of evaluating millions of knowledge rules per second
  • Built in C++ on top of ACE
  • Provides network transports for two DDS vendors
    • Open Splice Community Edition (open source)
    • NDDS (licensed and closed source)
kats disseminating knowledge1
KATS: Disseminating Knowledge

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • KaRL provides an intuitive multimodal logic
  • Supports declarative or rule-based expressions
    • .kats.result == 2 => PhoneApp1.failed = 1
    • PhoneApp1.failed && Services.running && (StartTest2 = 1)
  • Supports local variables (variable starts with a period) which are not disseminated
  • Global variable values will be disseminated to all entities in the domain when changed
  • First class support for 264 knowledge values per variable
  • No available ontologies for non-integer types but can be supported
kats smart phones and maml
KATS: Smart Phones and MAML

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • So… what does this have to do with Smart Phones?
  • MADARA contains a Python library called MAML which allows for instrumenting an Android smart phone from a connected host machine
    • Uses a combination of Android Monkeyrunner and the Android Debug Bridge
    • Can also run instrumentation apks (installed programs that manipulate a different installed program) and JUnit tests
  • The MAML library is open source and may be downloaded separately from the rest of the MADARA source code
kats smart phones and maml1
KATS: Smart Phones and MAML

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • So… what the heck is Monkeyrunner?
  • The functional parts of Monkeyrunnerallow for sending KeyEvents (like mimicking a user clicking a button, etc.) to the smart phone
  • Also allows for taking snapshots of the screen and saving it to disk
  • Wait… so what does this MADARA MAML thingy do?
  • Encapsulates the working Monkeyrunner functionality
  • Fixes the broken parts of the API
  • Provides mechanisms for directly using the Android Debug Bridge
  • Provides convenience functions for instrumentinglogcat from Python
  • Provides a much more user friendly library of functions to send KeyEventsthan Monkeyrunner does
kats smart phones and maml2
KATS: Smart Phones and MAML

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Okay… but what does MAML have to do with KATS?
  • Python scripts that use the MAML library may be fitted with the testing entity life cycle of KATS to instrument smart phones
  • KATS are MAMLs…
kats demo the thing already
KATS: Demo the thing already!

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Why are you looking at this screen?
  • Our powerpoint instrumentation feature is broken…
kats conclusions
KATS: Conclusions

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • KATS features a flexible test entity life cycle model that provides fine-grained sequencing support for automating almost any test, including most that require microsecond precision to replicate race conditions between hosts
  • The KATS paradigm for GME provides an intuitive interface for configuring all aspects of tests and their sequencing
  • The KaRL engine provides a dynamic knowledge and reasoning service that is built into appropriate KATS life cycle phases
  • The MAML Python library provides a extensible interface for instrumentingAndroid Smart Phone applications
  • One day, these technologies may contribute to a distributed, automated deployment that walks your dog and waters your plants while you are playing with robots at work.
kats conclusions1
KATS: Conclusions

Scenario

Overview

Modeling

Disseminating

Smart phones

Demo

Conclusions

  • Email
  • James Edmondson (james.r.edmondson@vanderbilt.edu)
  • Websites
  • MADARA project site (http://madara.googlecode.com)
  • Distributed Reasoner blog (http://distributedreasoner.blogspot.com)