Computational intelligence
Download
1 / 41

Computational Intelligence - PowerPoint PPT Presentation


  • 156 Views
  • Uploaded on

Computational Intelligence. John Sum Institute of Technology Management National Chung Hsing University Taichung, Taiwan ROC. OUTLINE. Historical Background Computational Intelligence Example Problems Methodology Model Structure Model Parameters Parametric Estimation Discussion

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Computational Intelligence' - kizzy


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Computational intelligence

Computational Intelligence

John Sum

Institute of Technology Management

National Chung Hsing University

Taichung, Taiwan ROC


Outline
OUTLINE

  • Historical Background

  • Computational Intelligence

  • Example Problems

  • Methodology

    • Model Structure

    • Model Parameters

    • Parametric Estimation

  • Discussion

  • Conclusion

Computational Intelligence


History
HISTORY

Computational Intelligence


History1
HISTORY

  • 1940 – First computing machine

  • 1957 – Perceptron (First NN model)

  • 1965 – Fuzzy Logic (Rules)

  • 1960s – Genetic Algorithm

  • 1970s – Evolutionary Computing

Computational Intelligence


History2
HISTORY

  • 1980s

    • Neural Computing

    • Swarm Intelligence

  • 1990s (Hybrid)

    • Fuzzy Neural Networks

    • NFG, FGN, GNF, etc

Computational Intelligence


History3

CI

IS

SC

HISTORY

  • Beyond 1990s: Research areas converge

    • Computational Intelligence

    • Softcomputing

    • Intelligent Systems

  • Covering

    • Adaptive Systems

    • Fuzzy Systems

    • Neural Networks

    • Evolutionary Computing

    • Data Mining

AS FS DM EC

NN SA PSO GA

MCMC SL

Computational Intelligence


Computational intelligence1
COMPUTATIONAL INTELLIGENCE

  • Computational Intelligence

    • Heuristic algorithms (or models) such as in fuzzy systems, neural networks and evolutionary computation.

    • Techniques that use Simulated annealing, Swarm intelligence, Fractals and Chaos Theory, Artificial immune systems, Wavelets, etc.

Computational Intelligence


Computational intelligence2
COMPUTATIONAL INTELLIGENCE

  • Goal: Problem Solving

    • Financial forecast

    • Customer segmentation (CRM)

    • Supply chain design (SCM)

    • Business process re-engineering

    • System control

    • Pattern recognition

    • Image compression

    • Homeland security

Computational Intelligence


Computational intelligence3
COMPUTATIONAL INTELLIGENCE

  • Underlying structure of the model is unknown, or the model is known but it is too complicated

  • Example: DJI versus HIS (Time Series)

  • Define system structure

    • NL model (NN, ODE, etc.)

    • Rule-based system

  • Parametric estimation

    • Deterministic search (Gradient descent or Newton’s method)

    • Stochastic search (SA or MCMC)

Computational Intelligence


Computational intelligence4
COMPUTATIONAL INTELLIGENCE

  • Underlying model structure is known

  • Example: Manufacturing process (SCM)

  • Define the objective to be maximized

    • Examples: Completion time, Cost, Profit

  • Optimization

    • Linear programming, ILP, NLP

    • Deterministic search (Gradient descent or Newton’s method)

    • Stochastic search (SA or MCMC)

Computational Intelligence


Eg1 nonlinear dynamic system
EG1: Nonlinear Dynamic System

Computational Intelligence


Eg2 nonlinear function
EG2: Nonlinear Function

Computational Intelligence


Eg3 car price
EG3: Car Price

  • Predict the price of a car based on

    • Specification of an auto in terms of various characteristics

    • Assigned insurance risk rating

    • Normalized losses in use as compared to other cars

    • Number of attributes: 25

    • Missing values: Yes!

Computational Intelligence


Eg3 car price1
EG3: Car Price

Computational Intelligence


Eg4 purchasing preference
EG4: Purchasing Preference

Structural Equation Model

Bayesian Network

Feedforward Network

Fuzzy Logic

Computational Intelligence


Eg5 financial time series
EG5: Financial Time Series

Computational Intelligence


Eg5 financial time series1
EG5: Financial Time Series

  • What would happen in the next trading day? (Time series prediction problem)

    • Closing value

    • Open value

    • UP or DOWN

  • Time series prediction + trading rules

    • What should I do tomorrow? HOLD, SELL or BUY

    • When should I BUY and SELL?

Computational Intelligence


Remarks on eg1 eg5
Remarks on EG1 ~ EG5

Computational Intelligence


Computational intelligence statement of problem
COMPUTATIONAL INTELLIGENCEStatement of Problem

  • Given a set of data collected (or measured) from a system (probably an unknown system), devise a model (by whatever structure, technique, method in CI) that mimics the behavior of that system as ‘good’ as possible.

  • Making use of the devised model to

    • (1) interpret the behavior of the system,

    • (2) predict the future behavior of the system,

    • (3) control the behavior of the system,

    • (4) make money.

Computational Intelligence


Methodology
METHODOLOGY

  • Step 1: Data Collection

    • Experiments or measurements

    • Questionnaire

    • Magazine

    • Public data sets

  • Step 2: Model Structure Assumption

    • IF it is known, SKIP this step.

    • ELSE, DEFINE a model structure

Computational Intelligence


Methodology1
METHODOLOGY

  • Step 3: Parametric Estimation

    • Gradient descent

    • Newton’s method

    • Exhaustive search

    • Genetic algorithms (*)

    • Evolutionary algorithms (*)

    • Swarm intelligence

    • Simulated annealing (*)

    • Markov Chain Monte Carlo (*)

Computational Intelligence


Methodology2
METHODOLOGY

  • Step 4: Model Validation (is it a reasonable good model)

    • Hypothesis test

    • Validation/Testing set

    • Leave one out validation

  • Step 5: Model Reduction (would there be a simpler model that is also reasonable good)

    • AIC, BIC, MDL

    • Pruning (using testing set)

Computational Intelligence


Methodology3
METHODOLOGY

  • Beyond Model Reduction

    • Any redundant input

    • Any redundant sample (or outlier)

    • Any better structure (alternative)

    • How do we determine a ‘good’ model

Computational Intelligence


Nn model structures

Perceptron

Multilayer Perceptron (MLP or BPN)

Adaptive Resonance Theory Model (ART)

Competitive Learning (CL)

Hopfield Network, Associative Network

Bidirectional Associative Model (BAM)

Recurrent Neural Network (RNN)

Boltzmann Machine

Brain-State-In-A-Box (BSB)

Radial Basis Function Network (RBF Net)

Bayesian Networks

Self Organizing Map (SOM or Kohonen Map)

Learning Vector Quantization (LVQ)

Support Vector Machine (SVM)

Support Vector Regression (SVR)

PCA, ICA, MCA

Winner-Take-All Network (WTA)

Spike neural networks

Remarks

Not all of them is able to learn, eg BSB, WTA

Might need to combine two structures to solve a single problem

Multiple definitions on the ‘neuron’

NN MODEL STRUCTURES

Computational Intelligence


Nn model structures1
NN MODEL STRUCTURES

  • Supply Chain Management (Optimization Problem)

    • Hopfield Network

  • Customer Segmentation (Clustering Problem)

    • CL, SOM, LVQ, ART

  • Dynamic Systems Modeling

    • RNN, Recurrent RBF

  • Car Price/NL Function (Function Approximation)

    • MLP, RBF Net, Bayesian Net, SVR, +SOM/LVQ

  • Financial TS (FA or Time Series Prediction)

    • RNN, SVR, MLP, RBF Net, + SOM/LVQ

Computational Intelligence


Fuzzy model structure
FUZZY MODEL STRUCTURE

Computational Intelligence


Fuzzy model structure1
FUZZY MODEL STRUCTURE

Computational Intelligence


Nn model parameters
NN MODEL PARAMETERS

  • MLP

    • Input Weights

    • Output Weights

    • Neuron model

  • RNN

    • Input Weights

    • Output Weights

    • Recurrent Weights

    • Neuron model

Computational Intelligence


Nn model parameters1
NN MODEL PARAMETERS

Computational Intelligence


Nn model parameters2
NN MODEL PARAMETERS

Computational Intelligence


Nn model parameters3
NN MODEL PARAMETERS

Computational Intelligence


Fuzzy model parameters
FUZZY MODEL PARAMETERS

Computational Intelligence


Parametric estimation
PARAMETRIC ESTIMATION

Computational Intelligence


Parametric estimation gradient descent
PARAMETRIC ESTIMATION Gradient Descent

Computational Intelligence


Paramertic estimation genetic algorithm
PARAMERTIC ESTIMATIONGenetic Algorithm

Computational Intelligence


Paramertic estimation genetic algorithm1
PARAMERTIC ESTIMATIONGenetic Algorithm

Computational Intelligence


Paramertic estimation genetic algorithm2
PARAMERTIC ESTIMATIONGenetic Algorithm

Computational Intelligence


Discussions
DISCUSSIONS

  • CI is not the only method (or structure) to solve a problem.

  • Even it can solve, its performance might not be better than other methods.

  • Should compare with other well-known or existing methods

Computational Intelligence


Discussions1
DISCUSSIONS

  • SCM Problem

    • LP, LIP, NLP

    • Lagrangian Relaxation

    • Cutting Plane

    • CPLEX

  • Function Approximation

    • Polynomial Series

    • Trigonometric Series

    • B-Spline

Computational Intelligence


Conclusions
CONCLUSIONS

  • IF

    • The problem to be solved has been well formulated

    • The structure has been selected

    • The objective function to evaluation the goodness of a parametric vector has been defined

  • THEN

    • Every problem is just an optimization problem

Computational Intelligence


John sum pfsum@nchu edu tw
JOHN SUM ([email protected])

  • Taiwan HK-Chinese, PhD (98) and MPhil (95) from CUHK, BEng (92) from PolyU HK.

  • Taught in HK Baptist University (98-00), OUHK (00) and PolyU HK (00-04), Chung Shan Medical University (05-07)

  • Adj. Associate Prof., Institute of Software, CAS Beijing (99-02)

  • Short visit: CityU HK, Griffith University in Australia, FAU, Boca Raton FL US, CAS in Beijing, Ching Mai University in Thailand.

  • Assist. Prof., IEC (07-09), Asso. Prof., ITM (09-) NCHU Taiwan

  • 2000 Marquis Who's Who in the World.

  • Senior Member of IEEE, CI Society, SMC Society (05-)

  • GB Member, Asia Pacific Neural Network Assembly (09-)

  • Associate Editor of the IJCA (05-09)

  • Research Interests include NN, FS, SEM, EC, TM

Computational Intelligence


ad