Ontology evolution and regression analysis insights into ontology regression testing
Download
1 / 42

Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing - PowerPoint PPT Presentation


  • 141 Views
  • Uploaded on

Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing. Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler. Motivation. Current studies of Ontology Evaluation tend to: Focus on individual ontology versions

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing' - holden


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Ontology evolution and regression analysis insights into ontology regression testing

Ontology Evolution and Regression AnalysisInsights into Ontology Regression Testing

Maria Copeland

Rafael Goncalvez

Robert Stevens

Bijan Parsia

Uli Sattler


Motivation
Motivation

Current studies of Ontology Evaluation tend to:

  • Focus on individual ontology versions

  • Focus on shifts in the gross statistics

    In either case we don’t get objective and systematic evaluations of the life span of the ontology


Our goal is to extract insightful and useful information out of all the existing versions of an ontology


Ontology testing challenge
Ontology Testing Challenge of all the existing versions of an ontology

  • How do we systematically identify test areas?

  • How do we systematically analyse change impacts to the ontology?

    How can we effectively minimise testing efforts and cost

    and still achieve adequate testing coverage


Software testing
Software Testing of all the existing versions of an ontology


Software regression testing what is it
Software Regression Testing of all the existing versions of an ontologyWhat is it?

  • It is a test activity to systematicallyre-test existing components after software changes

  • It test against current and updated requirements


Software regression testing testing aspects
Software Regression Testing of all the existing versions of an ontologyTesting Aspects

  • Testing at the functional requirements

    • Unit Level

    • System Level

  • Testing at the non-functional requirements


Software regression testing testing plan
Software Regression Testing of all the existing versions of an ontologyTesting Plan

  • Defines testing criteria

  • Identifies test area or components

  • Test execution strategy

  • Test evaluation strategy

  • Updates test and other relevant documentation


Software regression testing process
Software Regression Testing of all the existing versions of an ontologyProcess

Change Detection

Impact Analysis

Define Test Plan

Build Test Suite

Run Test

Evaluate Results

Report Results


Ontology regression testing
Ontology Regression Testing? of all the existing versions of an ontology


Change detection
Change Detection of all the existing versions of an ontology

  • Explicit Changes

    • Asserted logical and annotation axioms

    • Properties

    • Classes

  • Implicit Changes?

    • Subsumption changes

    • Entailment changes


Impact analysis
Impact Analysis of all the existing versions of an ontology

  • Previous Version or all versions?

    • Intentional Difference analysis?

    • Justifications analysis?

  • Information Content?

    • Asserted content?

    • Entailed content?

  • Requirements Impact?

    • Functional and Non-functional?

    • Which ones do we test?


Define test plan
Define Test Plan of all the existing versions of an ontology

  • Test criteria?

  • Test area? Do we have test areas?

  • How can the test be systematically run?

  • Can results be interpreted?


Can we systematically
Can we systematically: of all the existing versions of an ontology

  • Build test suites?

  • Run tests?

  • Evaluate results?

  • Re-run tests if necessary?


Manual vs automated test suites
Manual vs. Automated Test Suites of all the existing versions of an ontology

  • Manually test cases

    • Check against a methodology

    • Eyeballing

  • Automated test cases

    • Satisfiability

    • Inconsistency


Manual vs automated test suites1
Manual vs. Automated Test Suites of all the existing versions of an ontology

  • Manually test cases

    • Time consuming

    • Subjective

    • Unsystematic

  • Automated test cases

    • Reasoner based

    • Limited in scope


Can we expand the range of automatic test suites
Can we expand the range of automatic test suites? of all the existing versions of an ontology


Yes by analysing ontology dynamics
YES of all the existing versions of an ontologyBy Analysing Ontology Dynamics


What are ontology dynamics
What are Ontology Dynamics? of all the existing versions of an ontology

  • Periods of growth, decline, and stability

  • Axioms presence

  • Types of axioms presence (e.g. continual, interrupted)

  • Sequence editing types and patterns


Axiom life span we expect axioms with constant unchanged presence
Axiom Life Span - We expect: of all the existing versions of an ontologyAxioms with Constant Unchanged Presence

αi

O1

Oi

Ontology Versions


Axiom life span we expect axioms that are modified
Axiom Life Span - We expect: of all the existing versions of an ontologyAxioms that are Modified

αi

Split or Merge

O1

Oi

Oi+n

Ontology Versions


Axiom life span we expect axioms that enter and leave the ontology
Axiom Life Span - We expect: of all the existing versions of an ontologyAxioms that Enter and Leave the Ontology

αi

O1

Oi

Oi+n

Ontology Versions


Ncit ontology dynamics
NCIt Ontology Dynamics of all the existing versions of an ontology


National cancer institute thesaurus ncit
National Cancer Institute Thesaurus (NCIt) of all the existing versions of an ontology

  • The National Cancer Institute (NCI) is a U.S. government funded organisation for the research of causes, treatment, and prevention of cancer

  • The NCIt is an ontology written in the Web Ontology Language (OWL) which supports the development and maintenance of a controlled vocabulary about cancer research

  • Multiple publications about process, quality control, usage, and critiques

  • Publicly available monthly releases and concept change logs

    Rich source of ontology evolution data


Ncit dynamics axioms life span analysis
NCIt Dynamics – Axioms Life Span Analysis of all the existing versions of an ontology


Ncit change dynamics detailed view of axiom life span
NCIt Change Dynamics – Detailed View of Axiom Life Span of all the existing versions of an ontology

Top Ten Frequency Distributions


Ncit axiom life span axioms with constant unchanged presence
NCIt Axiom Life Span: of all the existing versions of an ontologyAxioms with Constant Unchanged Presence

6.73%

αi

O1

O103

NCIt 2003 – 2012 103 Versions


Ncit axiom life span axioms that enter and leave the ontology
NCIt Axiom Life Span: of all the existing versions of an ontologyAxioms that Enter and Leave the Ontology

αi

O1

Oi

Oi+n

NCIt 2003 – 2012 103 Versions


Ncit axiom life span axioms with gaps between unchanged presence
NCIt Axiom Life Span: of all the existing versions of an ontologyAxioms with Gaps between Unchanged Presence

Effectually Added

Effectually Removed

Effectually Added

Effectually Removed

αi

O I≠ αi

αi

O I≠ αi

O1

Oi

Oi+n

Ok

Ok+n

NCIt 2003 – 2012 103 Versions


Ncit axiom life span axioms with gaps between unchanged presence1
NCIt Axiom Life Span: of all the existing versions of an ontologyAxioms with Gaps between Unchanged Presence

O I≠ αi

Effectually Added

Ineffectually Removed

Effectually Removed

Effectually Added

αi

αi

O I= αi

O1

Oi

Oi+n

Ok

Ok+n

NCIt 2003 – 2012 103 Versions


Ncit axiom life span axioms with gaps between unchanged presence2
NCIt Axiom Life Span: of all the existing versions of an ontologyAxioms with Gaps between Unchanged Presence

Ineffectually

Removed

Ineffectually Added

Ineffectually Added

Ineffectual Removed

αi

  • O I= αi

αi

  • O I= αi

O1

Oi

Oi+n

Ok

Ok+n

NCIt 2003 – 2012 103 Versions


Ncit dynamics editing patterns
NCIt Dynamics – Editing Patterns of all the existing versions of an ontology


Ncit regression analysis main finding
NCIt Regression Analysis – Main Finding of all the existing versions of an ontology

This means that we are able to identify ‘bugs’, the sequence pattern of these bugs, and their location!


Ncit regression analysis indicative of faults in sequence of changes
NCIt Regression Analysis: of all the existing versions of an ontologyIndicative of Faults In Sequence of Changes

Effectually Added

Effectually Added

Ineffectually Removed

Effectually Removed

Effectually Removed

Effectually Added

Effectually Removed

Effectually Added

?

?

αi

αi

αi

αi

O I= αi

O I≠ αi

O I≠ αi

O1

Oi

Oi+n

Ok

Ok+n

O1

Oi

Oi+n

Ok

Ok+n

NCIt 2003 – 2012 103 Versions

NCIt 2003 – 2012 103 Versions

?

?


Ncit regression analysis suggestive of faults in sequence of changes
NCIt Regression Analysis: of all the existing versions of an ontologySuggestive of Faults In Sequence of Changes

Ineffectually Removed

Effectually Added

O I= αi

αi

O1

Ok

Ok+n

NCIt 2003 – 2012 103 Versions

Refactoring


Ncit regression analysis suggestive of faults in sequence of changes1
NCIt Regression Analysis: of all the existing versions of an ontologySuggestive of Faults In Sequence of Changes

Ineffectually

Added

Ineffectually Removed

Ineffectually Removed

Effectually Added

O I= αi

αi

αi

?

O1

Ok

Oi

Oi+n

Ok+n

NCIt 2003 – 2012 103 Versions

Ineffectually

Added

O I= αi

Ok

O1

NCIt 2003 – 2012 103 Versions


From change dynamics to ontology regression testing
From Change Dynamics to Ontology Regression Testing of all the existing versions of an ontology


Systematically build test suites
Systematically Build Test Suites of all the existing versions of an ontology

Indicative of Faults In Sequence of Changes


Automated test suites fault detection
Automated Test Suites - Fault Detection of all the existing versions of an ontology

  • It provides systematic regression test for all version of the ontology

  • It conclusively identifies content regression and content refactoring

  • It suggests other faults based on regression sequence

    It is efficient and cheap to run


And there are still more potential benefits
And there are still more potential benefits … of all the existing versions of an ontology


Automated test suites fault detection1
Automated Test Suites - Fault Detection of all the existing versions of an ontology

  • Entailment Set Studies

  • Sub Domain Dynamics

  • Ontology Classes Dynamics


Thanks
Thanks of all the existing versions of an ontology


ad