ART 2: self-organization of stable category recognition codes ofr analog input patterns

1. ART 2: self-organization of stable category recognition codes ofr analog input patterns Authors: Gail A. Carpenter , Stephen Grossberg Advisor: Dr. Hsu Graduate: Yu-Wei Su Intelligent Database System Lab, IDSL

2. Outline • Motivation • Objective • Adaptive Resonance Architectures • ART 1: binary input patterns • ART 2: analog input patterns • ART 2 design principles • ART 2 equations • Experiment • Conclusion • Opinion Intelligent Database System Lab, IDSL

3. Motivation • ART 1 network can only response to arbitrary sequences of binary input patterns. Intelligent Database System Lab, IDSL

4. Objective • ART 2 networks self-organize stable recognition categories in response to arbitrary sequences of analog input patterns, as well as binary input patterns Intelligent Database System Lab, IDSL

5. Adaptive Resonance Architectures • ART networks encode new input patterns, by changing the weights, or long-term memory(LTM) traces, of a bottom-up adaptive filter. • The filter is contained in pathways leading from a feature representation field(F1) to a category representation field(F2) whose nodes undergo cooperative and competitive interactions Intelligent Database System Lab, IDSL

6. Adaptive Resonance Architectures( cont.) • Processing cycle of bottom-up adaptive filtering, code selection, read-out of a top-down learned expectation, matching, and code reset Intelligent Database System Lab, IDSL

7. Adaptive Resonance Architectures( cont.) Intelligent Database System Lab, IDSL

8. Adaptive Resonance Architectures( cont.) • The matching criterion is determined by a vigilance parameter that controls activation of the orienting subsystem • Higher vigilance imposes a stricter matching criterion • Gain control acts to adjust overall sensitivity to patterned inputs and to coordinate the separate, asynchronous functions of the ART subsystems Intelligent Database System Lab, IDSL

9. ART 1: binary input patterns Intelligent Database System Lab, IDSL

10. ART 2: analog input patterns Intelligent Database System Lab, IDSL

11. ART 2 design principles • Stability-plasticity trade-off • STM processing must be defined in such a way that s sustained new input pattern does not wash away previously learned information • Search-direct access trade-off • The familiar input pattern directly accesses its recognition code Intelligent Database System Lab, IDSL

12. ART 2 design principles( cont.) • Match-reset trade-off • Be able to recognize and react to arbitrarily small differences between an active F1 STM pattern and the LTM pattern being read-out from an established category • STM invariance under read-out of matched LTM • The property is achieved by designing the bottom and middle levels of F1 so that the STM activity patterns are not changed Intelligent Database System Lab, IDSL

13. ART 2 design principles( cont.) • This invariance property enables the bottom and middle F1 levels to nonlinearly transform the input pattern in a manner that remains stable during learning • Coexistence of LTM read-out and STM normalization • F1 receive an additional input when LTM read-out occurs and bottom F1 level enables an input to be normalized Intelligent Database System Lab, IDSL

14. ART 2 design principles( cont.) • No LTM recoding by superset input • One or more of the top-down LTM traces equal zero or very small. When this occurs, the STM activities of these F1 nodes are suppressed • Stable choice until reset • Only orienting subsystem can cause a change in the chosen F2 code Intelligent Database System Lab, IDSL

15. ART 2 design principles( cont.) • Contrast enhancement, noise suppression, and mismatch attenuation by non signal functions • A combination of normalization and nonlinear feedback processes within F1 determines a noise criterion and enables signal from noise • Rapid self-stabilization • Unstable learning system can be made more stable by making the learning rate so slow Intelligent Database System Lab, IDSL

16. ART 2 design principles( cont.) • Normalization • Used nonspecific inhibitory interneurons • Local computations • STM and LTM computations use only information available locally and in real time Intelligent Database System Lab, IDSL

17. ART 2 equations • ART2 STM equations: F1 i=1…M, is the total excitatory input to ith node and is the total inhibitory input ratio between the STM relaxation time and the LTM relaxation time Also B≡C≡0 Intelligent Database System Lab, IDSL

18. ART 2 equations( cont.) Intelligent Database System Lab, IDSL

19. ART 2 equations( cont.) • ART2 STM equations: F2 If Tj=max{Tj: the jth F2 node has not been reset On the current trial} otherwise 0<d<1,For all j<>J,dzji/dt=0, dzij/dt=0 If F2 is inactive, If the Jth F2 node is active Intelligent Database System Lab, IDSL

20. ART 2 equations( cont.) • ART2 LTM equations Intelligent Database System Lab, IDSL

21. ART 2 equations( cont.) • ART2 reset equation:the orienting subsystem • The degree of match between an STM pattern at F1 and an active LTM pattern is determined by the vector r=(r1….rM) P<=||r||, 0<p<1 Intelligent Database System Lab, IDSL

22. ART 2 equations( cont.) • Match-reset trade-off : choice of top-down initial LTM values Intelligent Database System Lab, IDSL

23. ART 2 equations( cont.) Intelligent Database System Lab, IDSL

24. Experiment Intelligent Database System Lab, IDSL

25. Conclusion • ART 2 can process analog input pattern successfully and Intelligent Database System Lab, IDSL

26. Opinion • The class of ART can make a research to handle categorical data Intelligent Database System Lab, IDSL