Loading in 5 sec....

Computational IntelligencePowerPoint Presentation

Computational Intelligence

- By
**kizzy** - Follow User

- 156 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Computational Intelligence' - kizzy

**An Image/Link below is provided (as is) to download presentation**
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

Presentation Transcript

### Computational Intelligence

John Sum

Institute of Technology Management

National Chung Hsing University

Taichung, Taiwan ROC

OUTLINE

- Historical Background
- Computational Intelligence
- Example Problems
- Methodology
- Model Structure
- Model Parameters
- Parametric Estimation

- Discussion
- Conclusion

Computational Intelligence

HISTORY

Computational Intelligence

HISTORY

- 1940 – First computing machine
- 1957 – Perceptron (First NN model)
- 1965 – Fuzzy Logic (Rules)
- 1960s – Genetic Algorithm
- 1970s – Evolutionary Computing

Computational Intelligence

HISTORY

- 1980s
- Neural Computing
- Swarm Intelligence

- 1990s (Hybrid)
- Fuzzy Neural Networks
- NFG, FGN, GNF, etc

Computational Intelligence

IS

SC

HISTORY- Beyond 1990s: Research areas converge
- Computational Intelligence
- Softcomputing
- Intelligent Systems

- Covering
- Adaptive Systems
- Fuzzy Systems
- Neural Networks
- Evolutionary Computing
- Data Mining

AS FS DM EC

NN SA PSO GA

MCMC SL

Computational Intelligence

COMPUTATIONAL INTELLIGENCE

- Computational Intelligence
- Heuristic algorithms (or models) such as in fuzzy systems, neural networks and evolutionary computation.
- Techniques that use Simulated annealing, Swarm intelligence, Fractals and Chaos Theory, Artificial immune systems, Wavelets, etc.

Computational Intelligence

COMPUTATIONAL INTELLIGENCE

- Goal: Problem Solving
- Financial forecast
- Customer segmentation (CRM)
- Supply chain design (SCM)
- Business process re-engineering
- System control
- Pattern recognition
- Image compression
- Homeland security

Computational Intelligence

COMPUTATIONAL INTELLIGENCE

- Underlying structure of the model is unknown, or the model is known but it is too complicated
- Example: DJI versus HIS (Time Series)
- Define system structure
- NL model (NN, ODE, etc.)
- Rule-based system

- Parametric estimation
- Deterministic search (Gradient descent or Newton’s method)
- Stochastic search (SA or MCMC)

Computational Intelligence

COMPUTATIONAL INTELLIGENCE

- Underlying model structure is known
- Example: Manufacturing process (SCM)
- Define the objective to be maximized
- Examples: Completion time, Cost, Profit

- Optimization
- Linear programming, ILP, NLP
- Deterministic search (Gradient descent or Newton’s method)
- Stochastic search (SA or MCMC)

Computational Intelligence

EG1: Nonlinear Dynamic System

Computational Intelligence

EG2: Nonlinear Function

Computational Intelligence

EG3: Car Price

- Predict the price of a car based on
- Specification of an auto in terms of various characteristics
- Assigned insurance risk rating
- Normalized losses in use as compared to other cars
- Number of attributes: 25
- Missing values: Yes!

Computational Intelligence

EG3: Car Price

Computational Intelligence

EG4: Purchasing Preference

Structural Equation Model

Bayesian Network

Feedforward Network

Fuzzy Logic

Computational Intelligence

EG5: Financial Time Series

Computational Intelligence

EG5: Financial Time Series

- What would happen in the next trading day? (Time series prediction problem)
- Closing value
- Open value
- UP or DOWN

- Time series prediction + trading rules
- What should I do tomorrow? HOLD, SELL or BUY
- When should I BUY and SELL?

Computational Intelligence

Remarks on EG1 ~ EG5

Computational Intelligence

COMPUTATIONAL INTELLIGENCEStatement of Problem

- Given a set of data collected (or measured) from a system (probably an unknown system), devise a model (by whatever structure, technique, method in CI) that mimics the behavior of that system as ‘good’ as possible.
- Making use of the devised model to
- (1) interpret the behavior of the system,
- (2) predict the future behavior of the system,
- (3) control the behavior of the system,
- (4) make money.

Computational Intelligence

METHODOLOGY

- Step 1: Data Collection
- Experiments or measurements
- Questionnaire
- Magazine
- Public data sets

- Step 2: Model Structure Assumption
- IF it is known, SKIP this step.
- ELSE, DEFINE a model structure

Computational Intelligence

METHODOLOGY

- Step 3: Parametric Estimation
- Gradient descent
- Newton’s method
- Exhaustive search
- Genetic algorithms (*)
- Evolutionary algorithms (*)
- Swarm intelligence
- Simulated annealing (*)
- Markov Chain Monte Carlo (*)

Computational Intelligence

METHODOLOGY

- Step 4: Model Validation (is it a reasonable good model)
- Hypothesis test
- Validation/Testing set
- Leave one out validation

- Step 5: Model Reduction (would there be a simpler model that is also reasonable good)
- AIC, BIC, MDL
- Pruning (using testing set)

Computational Intelligence

METHODOLOGY

- Beyond Model Reduction
- Any redundant input
- Any redundant sample (or outlier)
- Any better structure (alternative)
- How do we determine a ‘good’ model

Computational Intelligence

Multilayer Perceptron (MLP or BPN)

Adaptive Resonance Theory Model (ART)

Competitive Learning (CL)

Hopfield Network, Associative Network

Bidirectional Associative Model (BAM)

Recurrent Neural Network (RNN)

Boltzmann Machine

Brain-State-In-A-Box (BSB)

Radial Basis Function Network (RBF Net)

Bayesian Networks

Self Organizing Map (SOM or Kohonen Map)

Learning Vector Quantization (LVQ)

Support Vector Machine (SVM)

Support Vector Regression (SVR)

PCA, ICA, MCA

Winner-Take-All Network (WTA)

Spike neural networks

Remarks

Not all of them is able to learn, eg BSB, WTA

Might need to combine two structures to solve a single problem

Multiple definitions on the ‘neuron’

NN MODEL STRUCTURESComputational Intelligence

NN MODEL STRUCTURES

- Supply Chain Management (Optimization Problem)
- Hopfield Network

- Customer Segmentation (Clustering Problem)
- CL, SOM, LVQ, ART

- Dynamic Systems Modeling
- RNN, Recurrent RBF

- Car Price/NL Function (Function Approximation)
- MLP, RBF Net, Bayesian Net, SVR, +SOM/LVQ

- Financial TS (FA or Time Series Prediction)
- RNN, SVR, MLP, RBF Net, + SOM/LVQ

Computational Intelligence

FUZZY MODEL STRUCTURE

Computational Intelligence

FUZZY MODEL STRUCTURE

Computational Intelligence

NN MODEL PARAMETERS

- MLP
- Input Weights
- Output Weights
- Neuron model

- RNN
- Input Weights
- Output Weights
- Recurrent Weights
- Neuron model

Computational Intelligence

NN MODEL PARAMETERS

Computational Intelligence

NN MODEL PARAMETERS

Computational Intelligence

NN MODEL PARAMETERS

Computational Intelligence

FUZZY MODEL PARAMETERS

Computational Intelligence

PARAMETRIC ESTIMATION

Computational Intelligence

PARAMETRIC ESTIMATION Gradient Descent

Computational Intelligence

PARAMERTIC ESTIMATIONGenetic Algorithm

Computational Intelligence

PARAMERTIC ESTIMATIONGenetic Algorithm

Computational Intelligence

PARAMERTIC ESTIMATIONGenetic Algorithm

Computational Intelligence

DISCUSSIONS

- CI is not the only method (or structure) to solve a problem.
- Even it can solve, its performance might not be better than other methods.
- Should compare with other well-known or existing methods

Computational Intelligence

DISCUSSIONS

- SCM Problem
- LP, LIP, NLP
- Lagrangian Relaxation
- Cutting Plane
- CPLEX

- Function Approximation
- Polynomial Series
- Trigonometric Series
- B-Spline

Computational Intelligence

CONCLUSIONS

- IF
- The problem to be solved has been well formulated
- The structure has been selected
- The objective function to evaluation the goodness of a parametric vector has been defined

- THEN
- Every problem is just an optimization problem

Computational Intelligence

JOHN SUM ([email protected])

- Taiwan HK-Chinese, PhD (98) and MPhil (95) from CUHK, BEng (92) from PolyU HK.
- Taught in HK Baptist University (98-00), OUHK (00) and PolyU HK (00-04), Chung Shan Medical University (05-07)
- Adj. Associate Prof., Institute of Software, CAS Beijing (99-02)
- Short visit: CityU HK, Griffith University in Australia, FAU, Boca Raton FL US, CAS in Beijing, Ching Mai University in Thailand.
- Assist. Prof., IEC (07-09), Asso. Prof., ITM (09-) NCHU Taiwan
- 2000 Marquis Who's Who in the World.
- Senior Member of IEEE, CI Society, SMC Society (05-)
- GB Member, Asia Pacific Neural Network Assembly (09-)
- Associate Editor of the IJCA (05-09)
- Research Interests include NN, FS, SEM, EC, TM

Computational Intelligence

Download Presentation

Connecting to Server..