Integration of agent and data mining
This presentation is the property of its rightful owner.
Sponsored Links
1 / 50

Integration of Agent and Data Mining PowerPoint PPT Presentation


  • 91 Views
  • Uploaded on
  • Presentation posted in: General

Integration of Agent and Data Mining. Longbing Cao University of Technology, Sydney. Content. Introduction Agents can enrich data mining Data mining can improve agents Ontology-based integration of agents and data mining Demo Conclusions and directions. INTRODUCTION.

Download Presentation

Integration of Agent and Data Mining

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Integration of agent and data mining

Integration of Agent and Data Mining

Longbing Cao

University of Technology, Sydney


Content

Content

  • Introduction

  • Agents can enrich data mining

  • Data mining can improve agents

  • Ontology-based integration of agents and data mining

  • Demo

  • Conclusions and directions


Integration of agent and data mining

INTRODUCTION


Data mining multiagent research group at uts

Data mining & multiagent research group at UTS

  • Cross disciplinary researchers interacting at the group

  • Integrated research of data mining and multi-agent system

    • http://datamining.it.uts.edu.au

  • Real-world applications of the integration

    • Capital markets

    • F-Trade


Agents as a new computing paradigm for complex problems

Agents as a new computing paradigm for complex problems

  • Strengths

    • Analyze and understand complex systems

    • Deal with nonfunctional requirements

    • Handle social complexity such as distribution, dynamics, interaction, evolution, self-organization

    • Build flexible infrastructure

  • Weaknesses

    • Lack machine learning capability

    • Lack in-depth analytics

    • Lack knowledge representation


Data mining and knowledge discovery as an effective tool for in depth analysis

Data mining and knowledge discovery as an effective tool for in-depth analysis

  • Strengths

    • Deep data analysis

    • Deep knowledge discovery

  • Weaknesses

    • Nothing related to system infrastructure

    • Deal with social complexity such as distribution, dynamics


Bilateral enhancement of agents and data mining by the integration

Bilateral enhancement of agents and data mining by the integration

  • Agents can enrich data mining

  • Data mining can improve agents

  • Mutual enhancement: integration between data mining and multi-agent system


Integration of agent and data mining

AGENTS can ENRICH DATA MINING


Building agent based data mining systems

Building agent-based data mining systems

  • Agent-based data mining system

    • F-Trade

  • Agent-based distributed data mining system

    • Agent-based distributed data mining systems, such as BODHI, PADMA, JAM, Papyrus

  • Agents for multiple data source mining

  • Agents for web mining


Data mining models as agents

Data mining models as agents

  • Intelligent data mining agents – modeling data mining algorithms as agents

  • Data mining model integrator – integrating data mining algorithms

  • Data mining model planner – smartly managing data mining algorithms

  • Data mining model recommender – recommending appropriate algorithms


Agent based mediation and management of distributed and large scale data sources

Agent-based mediation and management of distributed and large-scale data sources

  • Data gateway agents for connecting data sources

  • Distributed data preprocessor agent

  • Data integrator agents for data integration

  • Agents for data clustering

  • Agents for ensemble mining in distributed data

  • Agents for data sampling and assumption


User and interaction agents for data mining

User and interaction agents for data mining

  • Human agent interaction for data mining

  • Agents for interactive mining

  • Agents in human-guided mining

  • Domain knowledge management using agents

  • User agents for preparing mining reports

  • Agents for circulating mining results


Case study 1 f trade

Users/CMCRC/Instituations

(Anybody,anytime,anywhere, from MAS & KDD & Finance)

Applications developers

Network

(Internet & LAN)

F-Trade

(open automated enterprise services, and personalized services)

KDD Researchers

(Frequent and abnormal patterns discovery, optimization of trading strategies, correlation analysis)

AAMAS Researchers

(OCAS, AOSE, OADI, OSOAD)

(Services for system components,algorithm and multiple data sources)

Data Sources

(Diff. Providers: AC3, HK market, CSFB, etc. Diff. Formats: FAV, ODBC, JDBC, OLEDB, etc.

)

Case study 1 -- F-Trade

  • Aims/Motivations:

  • Research Service Provider for AAMAS and data mining

  • Integrated Infrastructure for Financial Trading and Mining Support


Case study 1 f trade1

Case study 1 -- F-Trade

System infrastructure


Case study 1 f trade2

Case study 1 -- F-Trade

F-TRADE: Financial Trading Rules Automated Development & Evaluation


Case study 1 f trade3

Case study 1 -- F-Trade

Algorithm as an agent


Case study 1 f trade4

Case study 1 -- F-Trade

  • AgentService

  • RegisterAlgorithm(algoname;inputlist;inputconstraint;outputlist;outputconstraint;)

  • Description:

  • This agent service involves accepting registration application submitted by role PluginPerson, checking validity of attribute items, creating name and directory of the algorithm, and generating universal agent identifier and unique algorithm id.

  • Role: PluginPerson

  • Pre-conditions:

    • A request of registering an algorithm has been activated by protocol SubmitAlgoPluginRequest

  • A knowledge base storing rules for agent and service naming and directory

  • Type: algorithm.[datamining/tradingsignal]

  • Location: algo.[algorithmname]

  • Inputs: inputlist

  • InputConstraints: inputconstraint[;]

  • Outputs: outputlist

  • OutputConstraints: outputconstraint[;]

  • Activities: Register the algorithm

  • Permissions:

    • Read supplied knowledge base storing algorithm agent ontologies

  • Read supplied algorithm base storing algorithm information

  • Post-conditions:

    • Generate unique agent identifier, naming, and locator for the algorithm agent

  • Generate unique algorithm id

  • Exceptions:

    • Cannot find target algorithm

  • There are invalid format existing in the input attributes

Agent plug-and-play


Case study 1 f trade5

Case study 1 -- F-Trade

Agent for multiple data sources management


Case study 1 f trade6

Case study 1 -- F-Trade

Agent for reporting


Case study 2 agent based weka

Case study 2 – agent-based WEKA


Case study 3 ensemble

Case study 3 – ensemble


Integration of agent and data mining

DATA MINING can IMPROVE AGENTS


Data mining driven multiagent learning

Data mining-driven multiagent learning

  • DM-driven learning in MAS

    • Coordination learning

    • Individual learning

    • Group/collective learning

    • Distributed learning

    • Online/offline learning


Data mining driven evolution and adaptation in mas

Data mining-driven evolution and adaptation in MAS

  • Evolution of MAS based on hidden rules, so mine these rules and fill into the agent knowledge base for designing evolutionary agent systems

  • Adaptive capability mining for enhancing agent’s adaptation

  • Self-organization rule mining


Data mining for agent communication planning and dispatching

Data mining for agent communication, planning and dispatching

  • Cluster and classification

  • Class/segment-based communication

  • Class-based planning and dispatching


Dm based user modeling

DM-based User modeling

  • Modeling user behavior from DM

    • Game player modeling

    • Trader’s behavior modeling

    • Trader’s role modeling

  • User-agent interaction based on user modeling

    • Trader agents’ interface design

    • Trader-agent interaction rule design


Dm based user servicing

DM-based User servicing

  • DM-based agents for serving users

    • Visualization mining for reporting

    • Customer-relationship management for customer care

  • DM-based recommender agents

    • Stock recommender

    • In-depth rule recommender

    • Trading rule-stock recommender


Case study learning

Case study - learning

  • Agent learning via machine learning

    • Reinforcement learning

    • Evolutionary multiobjective methods

    • Evolutionary algorithm

    • Markov decision process

    • Temporal difference method


Case study user modeling

Case study – user modeling

  • Trader’s behavior modeling

  • Trader’s role modeling

    • Market order

    • Limit order


Integration of agent and data mining

MarketOrderLargeMarketOrder

January

February

Large market orders analysis


Case study servicing

Case study - servicing

  • Pairs trading

    • Mining correlated stock pairs

    • Correlated stock miner agent

    • Stock pairs recommender

    • Pairs trading strategy solution


Case study servicing1

Case study - servicing

  • Optimized rules

    • Mining in-depth rules

    • In-depth rule miner agent

    • User interface agent

    • Optimized rules recommender

    • Optimized trading strategy solution


Case study servicing2

Case study - servicing

  • Rule-stock pairs

    • Mining rule-stock pairs

    • Rule-stock pair mining agent

    • User interface agent

    • Rule-stock pair recommender

    • Trading strategy solution


Integration of agent and data mining

Return on investment


Integration of agent and data mining

ONTOLOGY-BASED INTEGRATION OF AGENTS AND DATA MINING


Ontology for domain understanding and interaction

Ontology for domain understanding and interaction

  • Domain ontology for understanding the domain problems

  • Problem-solving ontology

  • Task ontology

  • Method ontology


Ontology for knowledge management

Ontology for knowledge management

  • Ontology for organizing agent systems

  • Ontology for organizing mining algorithms

  • Ontology for user interaction

  • Managing domain ontology/task ontology/problem-solving ontology/method ontology


Ontology based system architecture

Ontology-based system architecture

  • Multi-domain ontological space

    • Related problem domains

    • Agent ontology domain

    • Data mining ontology domain

  • Hybrid ontology structure for organizing ontologies crossing multiple domains


Ontological engineering for the integration

Ontological engineering for the integration

  • Ontology namespace

  • Ontology mapping structure

  • Semantic rules for ontology mapping

  • Ontology transformation

  • Ontology query

  • Ontology search and discovery


Integration of agent and data mining

-  (part_of (A, B) part_of (B, C)) part_of (A, C)

  •  (substitute_to (A, B) substitute_to (B, C)) 

  • substitute_to (A, C)


Integration of agent and data mining

Rule 4.

-  (A AND B), B ::= substitute_to(A, B)

A OR B, the resulting output is A or B

Rule 5.

-  (A AND B), B ::= disjoint_to(A, B)

A AND B, the resulting output is A and B


Integration of agent and data mining

DEMO


Integration of agent and data mining

CONCLUSIONS and DIRECTIONS


  • Login