1 / 75

Introduction to Neural Networks

適応システム理論 ガイダンス. Introduction to Neural Networks. Kenji Nakayama Kanazawa University, JAPAN. PPT ファイルの入手方法 http://leo.ec.t.kanazawa-u.ac.jp/~nakayama/ Education 授業科目  Lecture Subjects 適応システム理論 平成22年度 ガイダンス PPT ファイル. Neural Networks. Network Structures Multi-layer Network

colby
Download Presentation

Introduction to Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. 適応システム理論 ガイダンス Introduction to Neural Networks Kenji Nakayama Kanazawa University, JAPAN

  2. PPTファイルの入手方法http://leo.ec.t.kanazawa-u.ac.jp/~nakayama/Education授業科目 Lecture Subjects適応システム理論平成22年度 ガイダンスPPTファイル

  3. Neural Networks Network Structures Multi-layer Network Recurrent Network Learning Algorithms Supervised Learning Unsupervised Learning Functions Pattern Mapping and Classification Estimation and Prediction Associative Memory Optimization and Minimization

  4. Multi-Layer Neural Networks

  5. Artificial Neuron Model

  6. Activation (Nonlinear) Function of Neuron Active Inactive Space Separation

  7. Pattern Classification by Single Neuron

  8. Linearly Inseparable Problem

  9. Two Layer Neural Network

  10. Pattern Classification by Two-Layer NN- Region Separation by Hidden Units-

  11. Pattern Classification by Two-Layer NN- Class Separation by Output Unit -

  12. Learning of Connection Weights in Single-Layer NN Gradient Method is minimized

  13. Learning of Connection Weights in Multi-Layer NN- Error Back Propagation Algorithm - Gradient Method Chain Rule in Derivative

  14. Learning Process (Initial State) u=0

  15. Learning Process (Middle State) u=0

  16. Learning Process (Middle State) u=0

  17. Learning Process (Convergence) u=0

  18. Training and Testing for Pattern Classification

  19. Application 1 Prediction of Fog Occurrence

  20. Number of Fog Occurrence Fog is observed every 30 minutes

  21. Neural Network for Prediction

  22. Weather Data ・Temperature ・Atmospheric Pressure ・Humidity ・Force of Wind ・Direction of Wind ・Cloud Condition ・Past Fog Occurrence  ・・・・・ 20 kinds of weather data are used

  23. Connection Weights from Input to Hidden Unit

  24. Connection Weights from Hidden to Output Fog will occur Fog won’t occur

  25. FFT of Connection Weights Used for Predicting Fog Input→Hidden Unit #6 Input→Hidden Unit #10

  26. FFT of Connection Weights for Predicting No Fog Input→Hidden Unit #3 Input→Hidden Unit #14

  27. Prediction Accuracy of Fog and No Fog

  28. Application 2 Nonlinear Time Series Prediction

  29. Examples of Nonlinear Time Series Examples of Nonlinear Time Series Sunspot Lake Level Chaotic Series

  30. Nonlinear Predictor Combining NN and Linear Filter

  31. Prediction Accuracy by Several Methods Prediction Accuracy by Several Methods

  32. Application 3 Prediction of Machine Deformation

  33. Numerically Controlled Cutting Machine Cutting Tool Objective

  34. Deformation of Cutting by Temperature Change

  35. Machine Temperature Change in Time

  36. Deviation of Cutting by Temperature Change Tolerance

  37. Prediction of Deformation Using NN Tolerance

  38. Predicting Protein Secondary Structure Application 4

  39. Comparison of Predition Accuracy in (A)Single NN in [6], (B)Single NN with η=0.00001, (C)Single NN with Random Noise and η=0.001 (A) (B) (C) Total accuracy Q3 (A) (B) (C) α-helix (A) (B) (C) β-sheet

  40. Brain Computer Interface Application 5

  41. Feature Extraction Classification Measure Brain WF Control Machine Feature Classifier Brain WF Brain Computer Interface (BCI) • Measure brain waveforms for subject thinking something (mental tasks). • Analyze brain waveforms and estimate what kind of mental tasks does the subject imagine. • Control computer or machine based on the estimation. Mental tasks Measure Brain WF Feature Extraction Classification Control Machine Brain WF Feature Classifier

  42. Approaches • FeatureAmplitude of Fourier transform • ClassificationMulti-layer neural networks • Brain waveformsColorado State University http://www.cs.colorado.edu/eeg/

  43. Five Mental Tasks • Baseline: Nothing to do (Relax). • Multiplication: Calculate 49×78 for example. • Letter: Writing a sentence of letter. • Rotation: Imagine rotating a 3-D object. • Count: Writing numbers in order on a board.

  44. Measuring Brain Waveform • Number of electrodes: 7ch C3, C4, P3, P4, O1, O2, EOG • Measuring time: 10 sec • Sampling frequency: 250Hz2500 samples per channel

  45. Brain Waveform Segmental Processing Amplitude of FFT Averaging Nonlinear Normali- zation Input for NN Pre-Processing of Brain Waveform • Segmental processing • Amplitude of Fourier transform • Reduction of # of samples by averaging • Nonlinear normalization of data

  46. ・・・ Segmental Processing • Brain waveform of 10 sec is divided into segments of 0.5 sec. • Mental tasks are estimated at each 0.25 sec.(↓) 0.5 sec 0.5 sec 0.5 sec 0.5 sec ・・・ Fourier transform for each segment

  47. Reduction of # of Samples • # of samples are reduced from 125 to 20 by averaging the successive samples of waveform. # of samples: 125 # of samples: 20

  48. Nonlinear Normalization for Amplitude of Fourier Transform • Amplitude of FFT is nonlinearly normalized in order to use samples having small values.

  49. Nonlinear Normalization for Amplitude of Fourier Transform 1  2  3  4  5  6  7 Channel:  1  2  3  4  5  6  7 Nonlinear Normalization 7 channels are arranged at input nodes (10×7=70 samples)

  50. Simulation Setup • 2 subjects • Hidden units: 20 • Learning rate: 0.2 • Initial connection weights: Random numbers distributed during -0.2~0.2 • Threshold for rejection: 0.8

More Related