Slow Intelligence Systems Session and Panel

1. Slow Intelligence SystemsSession and Panel

2. Panelists • Erland Jungert • Francesco Colace • Tiansi Dong • Shi-Kuo Chang (Moderator)

3. Outline • Motivation • Introduction to SIS • Application: Ontological Filters • Application: Topic/Trend Detection • Discussion

4. Motivation: Common Characteristics ofNew Generation Information Systems • Connected • Multiple sourced • Knowledge-based • Personalized • Hybrid

5. Smarter Planet • We are all now connected - economically, technically and socially. Our planet is becoming smarter via integration of information scattered in many different data sources: from the sensors, on the web, in our personal devices, in documents and in databases, or hidden within application programs. Often we need to get information from several of these sources to complete a task. Examples include healthcare, science, the business world and our personal lives. (Quoted from Josephine M. Cheng, IBM Fellow and Vice President of IBM Research)

6. (courtesy of IBM)

7. Hybrid Intelligence • While processor speed and storage capacity have grown remarkably, the geometric growth in user communities, online computer usage, and the availability of data is in some ways is even more remarkable. Hybrid Intelligence offers great opportunities we have to harness this data availability to build systems of immense potential. While today s large scale systems are evolutionarily based on the distributed computing technologies envisioned in the 70 s and 80 s, sheer scaling has led to many unanticipated challenges. (quoted from Alfred Z. Spector, Vice President, Research and Special Initiatives, Google, USA)

8. Hybrid Intelligence Users and computers doing more than either could individually (quoted from Alfred Z. Spector, Google).

9. Slow Intelligence Systems • Slow Intelligence Systems are general-purpose systems characterized by being able to improve performance over time. • A slow intelligence system is • a system that (i) solves • problems by trying different • solutions, (ii) is context- • aware to adapt to different • situations and to propagate • knowledge, and (iii) may • not perform well in the • short run but continuously • learns to improve its • performance over time.

10. Slow Intelligence Systems • Slow Intelligence Systems are general-purpose systems characterized by being able to improve performance over time through a process involving • Enumeration

11. Slow Intelligence Systems • Slow Intelligence Systems are general-purpose systems characterized by being able to improve performance over time through a process involving • Enumeration • Propagation

12. Slow Intelligence Systems • Slow Intelligence Systems are general-purpose systems characterized by being able to improve performance over time through a process involving • Enumeration • Propagation • Adaptation

13. Slow Intelligence Systems • Slow Intelligence Systems are general-purpose systems characterized by being able to improve performance over time through a process involving • Enumeration • Propagation • Adaptation • Elimination

14. Slow Intelligence Systems • Slow Intelligence Systems are general-purpose systems characterized by being able to improve performance over time through a process involving • Enumeration • Propagation • Adaptation • Elimination • Concentration

15. Slow Intelligence Systems • Slow Intelligence Systems are general-purpose systems characterized by being able to improve performance over time through a process involving • Enumeration • Propagation • Adaptation • Elimination • Concentration • Slow Decision Cycle • to complement Fast • Decision Cycle

16. Slow Intelligence Systems • A SIS continuously learns, searches for new solutions and propagates and shares its experience with other peers. • From the structural point of view, a SIS is a system with multiple decision cycles such that actions of slow decision cycle(s) may override actions of quick decision cycle(s), resulting in poorer performance in the short run but better performance in the long-run.

17. SIS Basic Building Block (BBB)

18. Advanced Building Block (ABB)

19. SIS is a component-based systembuilt from BBBs and ABBs

20. The SIS Testbed for Healthcare Systems

21. Ontological Filters for Slow Intelligence Systems Shi-Kuo Chang, Emilio Zegarra, Francesco Colace and Massimo De Santo

22. SIS Application to Product Configuration Productionof personalized or custom-tailored goods or services to meet consumers' diverse and changing needs

23. Figure 6 - Ontological Filter and the Slow Intelligent System Ontological Filter and Slow Intelligence System

24. A Scenario • A customer would like to buy a Personal Computer in order to play videogames and surf on the internet. • He knows that he needs an operating system, a web browser and an antivirus package. • In particular, the user prefers a Microsoft Windows operating system. He lives in the United States and prefers to have a desktop. He also prefers low cost components.

25. Ontological Transform for Product Configurator

26. Building Topic/Trend Detection System based on Slow Intelligence Chia-Chun Shih & Ting-Chun Peng Institute for Information Industry Taipei, Taiwan

27. An online trend detection system requirescareful resource allocation and automatic algorithm adaptationto process huge size of heterogeneous data. • This research adopts Slow Intelligence, which provides a framework for systems with insufficient computing resources to gradually adapt to environments, to response the challenges. • Four Slow Intelligence subsystems are proposed, and each subsystem targets a challenge in designing online topic/trend detection systems.

28. Introduction • Topic Detection and Tracking (TDT) • Initiated by DARPA at 1996 • discover the topical structure in unsegmented streams of news reporting as it appears across multiple media • Tasks: • Topic Detection • Topic Tracking • First Story Detection • Story Segmentation • Link Detection

29. Topic/Trend Detection System • Objective • Detect current hot topics and to predict future hot topics based on data collected from Social Media • Three components • Crawler & Extractor: Collect data and extract information from Social Media • Topic Extractor: Detect hot topics from a set of text documents • Trend Detector: Detect trends (future hot topics) based on currently available data Current Hot topics Crawler & Extractor Topic Extractor Trend Detector Social Media Future Hot topics

30. (cont’d) Topic/Trend Detection System • Crawler & Extractor Social Media HTML documents User’s Keywords of Interests Web Crawler Text documents Web data DB Topic Extractor Information Extractor * Extract articles and metadata (title, author, content, etc) from semi-structured web content Crawler & Extractor

31. (cont’d) Topic/Trend Detection System • Topic Extractor Web data DB Current topics Topic Word Extraction Topic Word Clustering • Apply TF-IDF scheme to generate Top-N topic words for each document • Apply clustering algorithm to cluster topic words into topic groups. The topic groups are treated as “topics” Current Hot topics Hot topic extraction • Apply aging theory to find hot topics Topic Extractor

32. Topic Trend (Future Hot Topics) Current topics Trend Estimation Algorithms (cont’d) Topic/Trend Detection System • Trend Detector Trend Detector

33. T/TD System with Slow Intelligence • Four complexitiesof designing online topic/trend detection systems • 1. It is unlikely to collect all web data based on limited amount of computing resources. The system needs to develop data collection strategies which can concentrate limited resources on collecting important web data.

34. (cont’d) T/TD System with Slow Intelligence • 2. Many computation methods are available for estimating trends. If parameter settings are also taken into account, there are too many combinations to choose. Furthermore, Internet is a changing environment, which means current best solution may not perform well in the future. The system needs to automatically (or at least quasi-automatically) find best solution from many alternatives in a changing environment.

35. (cont’d) T/TD System with Slow Intelligence • 3. The crawler needs to revisit websites to collect up-to-date data in hourly or daily intervals. Each site has different amount of to-be-update data and different policy to restrict frequent access, which are unknown beforehand. The system needs to find feasible data collection schedule based on past experience.

36. (cont’d) T/TD System with Slow Intelligence • 4. Any changes in web pages may disrupt Extractors. It needs automatic repair mechanism for Extractorsif many websites are being monitored. The repair mechanism needs to detect errors of Extractors, find alternatives, and choose the best solution from alternatives to fix the disrupted Extractors. Crawler & Extractor

37. (cont’d) T/TD System with Slow Intelligence 1. SIS to help restrict the range of data collection Knowledge of data Knowledge of algorithm

38. (cont’d) T/TD System with Slow Intelligence 2. SIS to help select and adapt trend detection algorithms

39. (cont’d) T/TD System with Slow Intelligence 3. SIS to help scheduling Crawler

40. (cont’d) T/TD System with Slow Intelligence 4. SIS to help adapt Extractors

42. Discussion • There are a large number of intelligent systems, quasi-intelligent systems and semi-intelligent systems that are "slow". Distributed intelligence systems, multiple agents systems and emergency management systems are mostly slow intelligence systems that exhibit the characteristics of multiple decision cycles.

43. Discussion (continued) • Since time is relative, "slow" intelligence systems for some can also be "fast" for others. • A slow intelligence system can evolve into a fast intelligence system. • A framework for knowledge-based software engineering.

44. Q&A

45. TheEnd