4
This presentation is the property of its rightful owner.
Sponsored Links
1 / 1

Towards Parallel Nonmonotonic Reasoning with Billions of Facts PowerPoint PPT Presentation


  • 78 Views
  • Uploaded on
  • Presentation posted in: General

4 IBM Research, Ireland. 1 University of Crete Department of Computer Science. Motivation for this Paper.

Download Presentation

Towards Parallel Nonmonotonic Reasoning with Billions of Facts

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Towards parallel nonmonotonic reasoning with billions of facts

4 IBM Research, Ireland

1 University of Crete Department of Computer Science

Motivation for this Paper

Situation: In the last two of years, there has been significant progress in parallel reasoning, scaling reasoning up to 100 billion triples. Nevertheless, current approaches have been restricted to monotonic reasoning, namely RDFS and OWL-horst, or have not been evaluated for scalability

Problems: Poor quality data (e.g., involving inconsistency or incompleteness), could easily lead to reasoning triviality when considering rules based on monotonic formalisms

Purpose of this paper: Analyze how parallelization, using the MapReduce framework, can be used to reason with defeasible rules over huge data sets

 Evaluate our approach in terms of scalability

Towards Parallel Nonmonotonic Reasoning with Billions of Facts

INPUT

Facts in multiple files

MAP phase Input

<position in file, fact>

IliasTachmazidis1,2, Grigoris Antoniou 2,3, GiorgosFlouris2, Spyros Kotoulas4

MAP phase Output

<argument,predicate>

Reduce phase Output

(Final Output)

<Conclusions after reasoning>

Reduce phase Input

<argument, list(predicates)>

File01

-------------------

bird(eagle)

bird(owl)

<0, bird(eagle)>

<eagle, bird>

<11, bird(owl)>

{tahmazid,antoniou,[email protected] , [email protected]

¬ flies(eagle)

<eagle, <bird, brokenWing>>

<owl, bird>

<0, bird(pigeon)>

Grouping/Sorting

<owl, <bird, brokenWing>>

¬ flies(owl)

File02 

-------------------

bird(pigeon)

brokenWing(eagle)

brokenWing(owl)

<pigeon, bird>

<13, brokenWing(eagle)>

3 University of Huddersfield, UK

<pigeon, <bird>>

flies(pigeon)

<30, brokenWing(owl)>

<eagle, brokenWing>

2 Foundation for Research and Technology - Hellas, Institute of Computer Science

<owl, brokenWing>

http://www.csd.uoc.gr

http:// www.ibm.com

http://www.ics.forth.gr

http://www. www.hud.ac.uk

Defeasible Logic

Syntax

Advantages

Defeaters

e.g. brokenWing(X)↝¬ flies(X)

Priority Relation (acyclic relation on the set of rules)

e.g.r: bird(X)flies(X)

  • r’: brokenWing(X)  ¬ flies(X)

  • r’ > r

  • Facts

    • e.g. bird(eagle)

  • Strict Rules

    • e.g. bird(X)animal(X)

  • Defeasible Rules

    • e.g. bird(X)flies(X)

  • Suitable for encoding commonsense knowledge and reasoning

  • Avoid triviality of inference due to low-quality data

  • Low complexity

    • The consequences of a defeasible theory D can be computed in O(N) time, where N is the number of symbols in D

MapReduce

Framework

Hadoop

MapReduce Framework

  • Hadoop is used heavily, among others, by IBM, Yahoo!, Facebook and Twitter

  • Hadoop includes

    • a distributed file system, HDFS

    • a MapReduce framework

    • a web monitoring interface

  • MapReduce is a software framework

  • introduced by Google in 2004

  • Deals with:

    • very large amounts of data (many terabytes)

    • process data fairly quickly

    • use very large numbers of commodity machines (thousands)

  • Operates exclusively on <key, value> pairs

  • Want an infrastructure that takes care of management tasks

    • distribution of data

    • management of fault tolerance

    • collecting results

  • For a specific problem

    • developer writes a few routines

    • routines plug into the general interface

Rule set

MapReduce Algorithm

Algorithm

Description

Apply Reduce Function/

Perform Reasoning

Split input by lines

Apply Map Function

r1 : bird(X)  animal(X),

r2 : bird(X)  flies(X),

r3 : brokenWing(X) ¬ flies(X),

r3 > r2

map(Long key, String value) :

// key: position in document (irrelevant)

// value: document line (a fact)

argumentValue = extractArgumentValue(value);

predicate = extractPredicate(value);

reduce(String key, Iterator values) :

// key: argument value

// values : list of predicates (facts)

List listOfFacts;

Reasoner reasoner = Reasoner.getCopy();

for each v in values

listOfFacts.add(v);

reasoner.Reason(listOfFacts);

Emit(key , reasoner.getResults());

Experimental

Results

Scaled speedup for various dataset sizes

Runtime in minutes as a function of dataset size, for various numbers of nodes

13th International Conference on Principles of Knowledge Representation and Reasoning (KR-2012)

This work was partially supported by the PlanetData NoE (FP7:ICT-2009.3.4, #257641)


  • Login