1 / 41

Computational Intelligence

Computational Intelligence. John Sum Institute of Technology Management National Chung Hsing University Taichung, Taiwan ROC. OUTLINE. Historical Background Computational Intelligence Example Problems Methodology Model Structure Model Parameters Parametric Estimation Discussion

kizzy
Download Presentation

Computational Intelligence

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computational Intelligence John Sum Institute of Technology Management National Chung Hsing University Taichung, Taiwan ROC

  2. OUTLINE • Historical Background • Computational Intelligence • Example Problems • Methodology • Model Structure • Model Parameters • Parametric Estimation • Discussion • Conclusion Computational Intelligence

  3. HISTORY Computational Intelligence

  4. HISTORY • 1940 – First computing machine • 1957 – Perceptron (First NN model) • 1965 – Fuzzy Logic (Rules) • 1960s – Genetic Algorithm • 1970s – Evolutionary Computing Computational Intelligence

  5. HISTORY • 1980s • Neural Computing • Swarm Intelligence • 1990s (Hybrid) • Fuzzy Neural Networks • NFG, FGN, GNF, etc Computational Intelligence

  6. CI IS SC HISTORY • Beyond 1990s: Research areas converge • Computational Intelligence • Softcomputing • Intelligent Systems • Covering • Adaptive Systems • Fuzzy Systems • Neural Networks • Evolutionary Computing • Data Mining AS FS DM EC NN SA PSO GA MCMC SL Computational Intelligence

  7. COMPUTATIONAL INTELLIGENCE • Computational Intelligence • Heuristic algorithms (or models) such as in fuzzy systems, neural networks and evolutionary computation. • Techniques that use Simulated annealing, Swarm intelligence, Fractals and Chaos Theory, Artificial immune systems, Wavelets, etc. Computational Intelligence

  8. COMPUTATIONAL INTELLIGENCE • Goal: Problem Solving • Financial forecast • Customer segmentation (CRM) • Supply chain design (SCM) • Business process re-engineering • System control • Pattern recognition • Image compression • Homeland security Computational Intelligence

  9. COMPUTATIONAL INTELLIGENCE • Underlying structure of the model is unknown, or the model is known but it is too complicated • Example: DJI versus HIS (Time Series) • Define system structure • NL model (NN, ODE, etc.) • Rule-based system • Parametric estimation • Deterministic search (Gradient descent or Newton’s method) • Stochastic search (SA or MCMC) Computational Intelligence

  10. COMPUTATIONAL INTELLIGENCE • Underlying model structure is known • Example: Manufacturing process (SCM) • Define the objective to be maximized • Examples: Completion time, Cost, Profit • Optimization • Linear programming, ILP, NLP • Deterministic search (Gradient descent or Newton’s method) • Stochastic search (SA or MCMC) Computational Intelligence

  11. EG1: Nonlinear Dynamic System Computational Intelligence

  12. EG2: Nonlinear Function Computational Intelligence

  13. EG3: Car Price • Predict the price of a car based on • Specification of an auto in terms of various characteristics • Assigned insurance risk rating • Normalized losses in use as compared to other cars • Number of attributes: 25 • Missing values: Yes! Computational Intelligence

  14. EG3: Car Price Computational Intelligence

  15. EG4: Purchasing Preference Structural Equation Model Bayesian Network Feedforward Network Fuzzy Logic Computational Intelligence

  16. EG5: Financial Time Series Computational Intelligence

  17. EG5: Financial Time Series • What would happen in the next trading day? (Time series prediction problem) • Closing value • Open value • UP or DOWN • Time series prediction + trading rules • What should I do tomorrow? HOLD, SELL or BUY • When should I BUY and SELL? Computational Intelligence

  18. Remarks on EG1 ~ EG5 Computational Intelligence

  19. COMPUTATIONAL INTELLIGENCEStatement of Problem • Given a set of data collected (or measured) from a system (probably an unknown system), devise a model (by whatever structure, technique, method in CI) that mimics the behavior of that system as ‘good’ as possible. • Making use of the devised model to • (1) interpret the behavior of the system, • (2) predict the future behavior of the system, • (3) control the behavior of the system, • (4) make money. Computational Intelligence

  20. METHODOLOGY • Step 1: Data Collection • Experiments or measurements • Questionnaire • Magazine • Public data sets • Step 2: Model Structure Assumption • IF it is known, SKIP this step. • ELSE, DEFINE a model structure Computational Intelligence

  21. METHODOLOGY • Step 3: Parametric Estimation • Gradient descent • Newton’s method • Exhaustive search • Genetic algorithms (*) • Evolutionary algorithms (*) • Swarm intelligence • Simulated annealing (*) • Markov Chain Monte Carlo (*) Computational Intelligence

  22. METHODOLOGY • Step 4: Model Validation (is it a reasonable good model) • Hypothesis test • Validation/Testing set • Leave one out validation • Step 5: Model Reduction (would there be a simpler model that is also reasonable good) • AIC, BIC, MDL • Pruning (using testing set) Computational Intelligence

  23. METHODOLOGY • Beyond Model Reduction • Any redundant input • Any redundant sample (or outlier) • Any better structure (alternative) • How do we determine a ‘good’ model Computational Intelligence

  24. Perceptron Multilayer Perceptron (MLP or BPN) Adaptive Resonance Theory Model (ART) Competitive Learning (CL) Hopfield Network, Associative Network Bidirectional Associative Model (BAM) Recurrent Neural Network (RNN) Boltzmann Machine Brain-State-In-A-Box (BSB) Radial Basis Function Network (RBF Net) Bayesian Networks Self Organizing Map (SOM or Kohonen Map) Learning Vector Quantization (LVQ) Support Vector Machine (SVM) Support Vector Regression (SVR) PCA, ICA, MCA Winner-Take-All Network (WTA) Spike neural networks Remarks Not all of them is able to learn, eg BSB, WTA Might need to combine two structures to solve a single problem Multiple definitions on the ‘neuron’ NN MODEL STRUCTURES Computational Intelligence

  25. NN MODEL STRUCTURES • Supply Chain Management (Optimization Problem) • Hopfield Network • Customer Segmentation (Clustering Problem) • CL, SOM, LVQ, ART • Dynamic Systems Modeling • RNN, Recurrent RBF • Car Price/NL Function (Function Approximation) • MLP, RBF Net, Bayesian Net, SVR, +SOM/LVQ • Financial TS (FA or Time Series Prediction) • RNN, SVR, MLP, RBF Net, + SOM/LVQ Computational Intelligence

  26. FUZZY MODEL STRUCTURE Computational Intelligence

  27. FUZZY MODEL STRUCTURE Computational Intelligence

  28. NN MODEL PARAMETERS • MLP • Input Weights • Output Weights • Neuron model • RNN • Input Weights • Output Weights • Recurrent Weights • Neuron model Computational Intelligence

  29. NN MODEL PARAMETERS Computational Intelligence

  30. NN MODEL PARAMETERS Computational Intelligence

  31. NN MODEL PARAMETERS Computational Intelligence

  32. FUZZY MODEL PARAMETERS Computational Intelligence

  33. PARAMETRIC ESTIMATION Computational Intelligence

  34. PARAMETRIC ESTIMATION Gradient Descent Computational Intelligence

  35. PARAMERTIC ESTIMATIONGenetic Algorithm Computational Intelligence

  36. PARAMERTIC ESTIMATIONGenetic Algorithm Computational Intelligence

  37. PARAMERTIC ESTIMATIONGenetic Algorithm Computational Intelligence

  38. DISCUSSIONS • CI is not the only method (or structure) to solve a problem. • Even it can solve, its performance might not be better than other methods. • Should compare with other well-known or existing methods Computational Intelligence

  39. DISCUSSIONS • SCM Problem • LP, LIP, NLP • Lagrangian Relaxation • Cutting Plane • CPLEX • Function Approximation • Polynomial Series • Trigonometric Series • B-Spline Computational Intelligence

  40. CONCLUSIONS • IF • The problem to be solved has been well formulated • The structure has been selected • The objective function to evaluation the goodness of a parametric vector has been defined • THEN • Every problem is just an optimization problem Computational Intelligence

  41. JOHN SUM (pfsum@nchu.edu.tw) • Taiwan HK-Chinese, PhD (98) and MPhil (95) from CUHK, BEng (92) from PolyU HK. • Taught in HK Baptist University (98-00), OUHK (00) and PolyU HK (00-04), Chung Shan Medical University (05-07) • Adj. Associate Prof., Institute of Software, CAS Beijing (99-02) • Short visit: CityU HK, Griffith University in Australia, FAU, Boca Raton FL US, CAS in Beijing, Ching Mai University in Thailand. • Assist. Prof., IEC (07-09), Asso. Prof., ITM (09-) NCHU Taiwan • 2000 Marquis Who's Who in the World. • Senior Member of IEEE, CI Society, SMC Society (05-) • GB Member, Asia Pacific Neural Network Assembly (09-) • Associate Editor of the IJCA (05-09) • Research Interests include NN, FS, SEM, EC, TM Computational Intelligence

More Related