Deep Learning

1. Deep Learning https://store.theartofservice.com/the-deep-learning-toolkit.html

2. Computer vision - Related fields • neural net and deep learning based image and feature analysis and classification) have their background in biology. https://store.theartofservice.com/the-deep-learning-toolkit.html

3. Machine learning - Representation learning • Deep learning algorithms discover multiple levels of representation, or a hierarchy of features, with higher-level, more abstract features defined in terms of (or generating) lower-level features https://store.theartofservice.com/the-deep-learning-toolkit.html

4. Artificial neural network - History • In the 1990s, neural networks were overtaken in popularity in machine learning by support vector machines and other, much simpler methods such as linear classifiers. Renewed interest in neural nets was sparked in the 2000s by the advent of deep learning. https://store.theartofservice.com/the-deep-learning-toolkit.html

5. Artificial neural network - Recent improvements • Between 2009 and 2012, the recurrent neural networks and deep feedforward neural networks developed in the research group of Jürgen Schmidhuber at the IDSIA|Swiss AI Lab IDSIA have won eight international competitions in pattern recognition and machine learning.http://www.kurzweilai.net/how-bio-inspired-deep-learning-keeps-winning-competitions 2012 Kurzweil AI Interview with Jürgen Schmidhuber on the eight competitions won by his Deep Learning team 2009–2012 For example, multi-dimensional long short term memory (LSTM)Graves, Alex; and Schmidhuber, Jürgen; Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks, in Bengio, Yoshua; Schuurmans, Dale; Lafferty, John; Williams, Chris K https://store.theartofservice.com/the-deep-learning-toolkit.html

6. Artificial neural network - Recent improvements • Deep learning feedforward networks, such as convolutional neural networks, alternate convolutional layers and max-pooling layers, topped by several pure Statistical classification|classification layers https://store.theartofservice.com/the-deep-learning-toolkit.html

7. Andrew Ng • He researches primarily in Artificial Intelligence, machine learning, and deep learning. His early work includes the Stanford Autonomous Helicopter project, which developed one of the most capable autonomous helicopters in the world, and the STAIR (STanford Artificial Intelligence Robot) project, which resulted in ROS (Robot Operating System)|ROS, a widely used open source software|open-source robotics software platform. https://store.theartofservice.com/the-deep-learning-toolkit.html

8. Andrew Ng - Machine learning research • Among its notable results was a neural network trained using deep learning algorithms on 16,000 CPU cores, that learned to recognize higher-level concepts, such as cats, after watching only YouTube videos, and without ever having been told what a cat is. https://store.theartofservice.com/the-deep-learning-toolkit.html

9. Ben Goertzel - Papers • * Goertzel, Ben (2011). Integrating a Compositional Spatiotemporal Deep Learning Network with Symbolic Representation/Reasoning within an Integrative Cognitive Architecture via an Intermediary Semantic Network. Proceedings of AAAI Symposium on Cognitive Systems, Arlington VA https://store.theartofservice.com/the-deep-learning-toolkit.html

10. Ben Goertzel - Papers • * Goertzel, Ben (2011). Imprecise Probability as a Linking Mechanism Between Deep Learning, Symbolic Cognition and Local Feature Detection in Vision Processing. Proceedings of AGI-11, Lecture Notes in AI, Springer Verlag [ http://goertzel.org/VisualAttention_AGI_11.pdf] https://store.theartofservice.com/the-deep-learning-toolkit.html

11. Serbo-Croatian - Croatian linguists • : At the end of the 15th century [in Dubrovnik and Dalmatia], sermons and poems were exquisitely crafted in the Croatian language by those men whose names are widely renowned by deep learning and piety. https://store.theartofservice.com/the-deep-learning-toolkit.html

12. Pattern recognition - Regression analysis|Regression algorithms (predicting real number|real-valued labels) • *Neural networks and Deep learning|Deep learning methods https://store.theartofservice.com/the-deep-learning-toolkit.html

13. Deep learning • 'Deep learning' is a set of algorithms in machine learning that attempt to learn in multiple levels of representation, corresponding to different levels of abstraction. It typically uses artificial neural networks. The levels in these learned statistical models correspond to distinct levels of concepts, where higher-level concepts are defined from lower-level ones, and the same lower-level concepts can help to define many higher-level concepts. https://store.theartofservice.com/the-deep-learning-toolkit.html

14. Deep learning • Deep learning is part of a broader family of machine learning methods based on learning representations. An observation (e.g., an image) can be represented in many ways (e.g., a vector of pixels), but some representations make it easier to learn tasks of interest (e.g., is this the image of a human face?) from examples, and research in this area attempts to define what makes better representations and how to learn them. https://store.theartofservice.com/the-deep-learning-toolkit.html

15. Deep learning • Ronan Collobert has said that deep learning is just a buzzword for neural nets https://store.theartofservice.com/the-deep-learning-toolkit.html

16. Deep learning - Introduction • The term deep learning gained traction in the mid-2000s after a publication by Geoffrey Hinton and Ruslan Salakhutdinov[http://www.cs.toronto.edu/~hinton/absps/tics.pdf Learning multiple layers of representation] https://store.theartofservice.com/the-deep-learning-toolkit.html

17. Deep learning - Introduction • In 1992, Jürgen Schmidhuber had already implemented a very similar idea for the more general case of unsupervised deep hierarchies of recurrent neural networks, and also experimentally shown its benefits for speeding up supervised learning.Jürgen Schmidhuber|Schmidhuber, Jürgen; Learning complex, extended sequences using the principle of history compression., Neural Computation, 4(2):234-242, 1992Jürgen Schmidhuber|Schmidhuber, Jürgen; My First Deep Learning System of 1991 + Deep Learning Timeline 1962-2013, http://www.idsia.ch/~juergen/firstdeeplearner.html https://store.theartofservice.com/the-deep-learning-toolkit.html

18. Deep learning - Introduction • Advances in hardware have been an important enabling factor for the resurgence of neural networks and the advent of deep learning, in particular the availability of powerful and inexpensive graphics processing units (GPUs) also suitable for general-purpose computing on graphics processing units|general-purpose computing https://store.theartofservice.com/the-deep-learning-toolkit.html

19. Deep learning - Introduction • and has attracted the attention of such thinkers as Ray Kurzweil, who was hired by Google to do deep learning research. https://store.theartofservice.com/the-deep-learning-toolkit.html

20. Deep learning - Introduction • Gary Marcus has expressed skepticism of deep learning's capabilities, noting that https://store.theartofservice.com/the-deep-learning-toolkit.html

21. Deep learning - Fundamental concepts • The appropriate number of levels and the structure that relates these factors is something that a deep learning algorithm is also expected to discover from examples. https://store.theartofservice.com/the-deep-learning-toolkit.html

22. Deep learning - Fundamental concepts • Deep learning algorithms often involve other important ideas that correspond to broad a priori beliefs about these unknown underlying factors https://store.theartofservice.com/the-deep-learning-toolkit.html

23. Deep learning - Fundamental concepts • Many deep learning algorithms are actually framed as unsupervised learning, e.g., using many examples of natural images to discover good representations of them. Because most of these learning algorithms can be applied to unlabeled data, they can leverage large amounts of unlabeled data, even when these examples are not necessarily labeled, and even when the data cannot be associated with labels of the immediate tasks of interest. https://store.theartofservice.com/the-deep-learning-toolkit.html

24. Deep learning - Deep learning in artificial neural networks • Deep Learning Neural Networks date back at least to the 1980 Neocognitron by Kunihiko Fukushima https://store.theartofservice.com/the-deep-learning-toolkit.html

25. Deep learning - Deep learning in artificial neural networks • Another method is the long short term memory (LSTM) network of 1997 by Sepp Hochreiter|Hochreiter Jürgen Schmidhuber|Schmidhuber.Sepp Hochreiter|Hochreiter, Sepp; and Jürgen Schmidhuber|Schmidhuber, Jürgen; Long Short-Term Memory, Neural Computation, 9(8):1735–1780, 1997 In 2009, deep multidimensional LSTM networks demonstrated the power of deep learning with many nonlinear layers, by winning three ICDAR 2009 competitions in connected handwriting recognition, without any prior knowledge about the three different languages to be learned.Graves, Alex; and Schmidhuber, Jürgen; Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks, in Bengio, Yoshua; Schuurmans, Dale; Lafferty, John; Williams, Chris K https://store.theartofservice.com/the-deep-learning-toolkit.html

26. Deep learning - Deep learning in artificial neural networks • As of 2011, the state of the art in deep learning feedforward networks alternates convolutional layers and max-pooling layers,D https://store.theartofservice.com/the-deep-learning-toolkit.html

27. Deep learning - Deep learning in artificial neural networks • Such supervised deep learning methods also were the first artificial pattern recognizers to achieve human-competitive performance on certain tasks.D. C. Ciresan, U. Meier, J. Schmidhuber. Multi-column Deep Neural Networks for Image Classification. IEEE Conf. on Computer Vision and Pattern Recognition CVPR 2012. https://store.theartofservice.com/the-deep-learning-toolkit.html

28. Deep learning - Deep learning in the human brain • These models share the interesting property that various proposed learning dynamics in the brain (e.g., a wave of neurotrophic growth factor) conspire to support the self-organization of just the sort of inter-related neural networks utilized in the later, purely computational deep learning models, and which appear to be analogous to one way of understanding the neocortex of the brain as a hierarchy of filters where each layer captures some of the information in the operating environment, and then passes the remainder, as well as modified base signal, to other layers further up the hierarchy https://store.theartofservice.com/the-deep-learning-toolkit.html

29. Deep learning - Deep learning in the human brain • The theory of deep learning therefore sees the coevolution of culture and cognition as a fundamental condition of human evolution.Shrager, J., Johnson, M https://store.theartofservice.com/the-deep-learning-toolkit.html

30. Handwriting recognition - Results since 2009 • Recent GPU-based deep learning methods for feedforward networks by Dan Ciresan and colleagues at IDSIA won the ICDAR 2011 offline Chinese handwriting recognition contest; their neural networks also were the first artificial pattern recognizers to achieve human-competitive performanceD https://store.theartofservice.com/the-deep-learning-toolkit.html

31. Mind uploading in fiction - Literature • * Clyde Dsouza's Memories with Maya (2013) looks at how deep learning processes, and 'Digital Breadcrumbs' left behind by people (tweets, Facebook updates, blogs) combined with memories of living relatives can be used to re-construct a mind and augment it with narrow AI libraries. The resulting 'Dirrogate' or Digital Surrogate can be thought of as a posthumous mind upload. https://store.theartofservice.com/the-deep-learning-toolkit.html

32. Multi-layer perceptron - Applications • but have since the 1990s faced strong competition from the much simpler (and relatedR. Collobert and S. Bengio (2004). Links between Perceptrons, MLPs and SVMs. Proc. Int'l Conf. on Machine Learning (ICML).) support vector machines. More recently, there has been some renewed interest in backpropagation networks due to the successes of deep learning. https://store.theartofservice.com/the-deep-learning-toolkit.html

33. Long short term memory - Applications • *Human action recognitionM. Baccouche, F. Mamalet, C Wolf, C. Garcia, A. Baskurt. Sequential Deep Learning for Human Action Recognition. 2nd International Workshop on Human Behavior Understanding (HBU), A.A. Salah, B. Lepri ed. Amsterdam, Netherlands. pp. 29–39. Lecture Notes in Computer Science 7065. Springer. 2011 https://store.theartofservice.com/the-deep-learning-toolkit.html

34. Jürgen Schmidhuber • Between 2009 and 2012, the recurrent neural networks and deep feedforward neural networks developed in his research group have won eight international competitions in pattern recognition and machine learning.[http://www.kurzweilai.net/how-bio-inspired-deep-learning-keeps-winning-competitions 2012 Kurzweil AI Interview] with Jürgen Schmidhuber on the eight competitions won by his Deep Learning team 2009-2012 In honor of his achievements he was elected to the European Academy of Sciences and Arts in 2008. https://store.theartofservice.com/the-deep-learning-toolkit.html

35. John F. Kennedy University - Service to Community • In keeping with its namesake, John F. Kennedy University is committed to being of service to the local communities. As part of this commitment and in recognition of the fact that deep learning often happens outside the classroom, JFK University offers students the opportunity to gain practical experience through the following clinical and internship opportunities. https://store.theartofservice.com/the-deep-learning-toolkit.html

36. Waldorf education - Educational scholars • This enables deep learning that goes beyond studying for the next test.Fanny Jiménez, Wissenschaftler loben Waldorfschulen, Die Welt, 27 September 2012 Deborah Meier, principal of Mission Hill School and MacArthur grant recipient, whilst having some quibbles about the Waldorf schools, stated: The adults I know who have come out of Waldorf schools are extraordinary people https://store.theartofservice.com/the-deep-learning-toolkit.html

37. Camille Paglia - Feminism • its deep learning and massive argument are unsurpassed) as well as Germaine Greer, but Time magazine critic Martha Duffy wrote that Paglia does not hesitate to hurl brazen insults at several feminists including Greer, whom Paglia accused of becoming a drone in three years as a result of her early success; Paglia also called activist Diana Fuss' output just junk – appalling! Showalter calls Paglia unique in the hyperbole and virulence of her hostility to virtually all the prominent feminist activists, public figures, writers and scholars of her generation, mentioning Carolyn Heilbrun, Judith Butler, Carol Gilligan, Marilyn French, Zoe Baird, Kimba Wood, Susan Thomases, and Hillary Clinton as targets of her criticism. https://store.theartofservice.com/the-deep-learning-toolkit.html

38. Pope Benedict XIV - Ascension to the papacy • This appears to have assisted his cause for winning the election, which also benefited from his reputation for deep learning, gentleness, pomp, wisdom, and piety in policy https://store.theartofservice.com/the-deep-learning-toolkit.html

39. Aleph (psychedelic) - Aleph-4 • Effects: profound and deep learning experiences - Alexander Shulgin https://store.theartofservice.com/the-deep-learning-toolkit.html

40. Pierre Baldi - Career • Pierre Baldi's research include artificial intelligence, statistical machine learning, and data mining, and their applications to problems in the life sciences in genomics, proteomics, systems biology, computational neuroscience, and, recently, deep learning. https://store.theartofservice.com/the-deep-learning-toolkit.html

41. Blissymbols - Semantics • Bliss’s concern about semantics finds an early referent in John Locke,Locke, J. (1690). An Essay Concerning Human Understanding. London. whose Essay Concerning Human Understanding prevented people from those vague and insignificant forms of speech that may give the impression of being deep learning. https://store.theartofservice.com/the-deep-learning-toolkit.html

42. Torch (machine learning) • 'Torch' is an open source deep learning library for the Lua (programming language)|Lua programming language https://store.theartofservice.com/the-deep-learning-toolkit.html

43. Torch (machine learning) - Applications • the Facebook AI Research Group,[http://www.kdnuggets.com/2014/02/exclusive-yann-lecun-deep-learning-facebook-ai-lab.html KDnuggets Interview with Yann LeCun, Deep Learning Expert, Director of Facebook AI Lab] the Computational Intelligence, Learning, Vision, and Robotics Lab at NYU,[http://cilvr.nyu.edu/doku.php?id=code:start CILVR Lab Software] MADBITS,[http://code.madbits.com/wiki/doku.php Machine Learning with Torch7] IBM,[https://news.ycombinator.com/item?id=7928738 Hacker News] Yandex[https://www.facebook.com/yann.lecun/posts/10152077631217143?comment_id=10152089275552143offset=0total_comments=6 Yann Lecun's FaceBook Page] and the Idiap Research Institute.[https://www.idiap.ch/scientific-research/resources/torch IDIAP Research Institute : Torch] It is used and cited in 240 research papers.[http://scholar.google.ca/scholar?cites=9993075313749753697as_sdt=2005sciodt=0,5hl=en Google Scholar results for Torch: a modular machine learning software library citations] For comparison, Theano (software)|Theano, a similar library written in Python (programming language), C and CUDA, has 138 citations.[http://scholar.google.ca/scholar?cites=8194189194999260817as_sdt=2005sciodt=0,5hl=en Theano: a CPU and GPU math expression compiler] Torch has been extended for use on Android (operating system)|Android[https://github.com/soumith/torch-android Torch-android GitHub repository] and iOS.[https://github.com/clementfarabet/torch-ios Torch-ios GitHub repository] It has been used to build hardware implementations for data flows like those found in neural networks.[http://pub.clement.farabet.net/ecvw11.pdf NeuFlow: A Runtime Reconfigurable Dataflow Processor for Vision] https://store.theartofservice.com/the-deep-learning-toolkit.html

44. Restricted Boltzmann machine • Restricted Boltzmann machines can also be used in deep learning networks. In particular, deep belief networks can be formed by stacking RBMs and optionally fine-tuning the resulting deep network with gradient descent and backpropagation. https://store.theartofservice.com/the-deep-learning-toolkit.html

45. Tensor Network Theory - Neural Networks and Artificial Intelligence • Other applications include teaching computers how to recognize handwriting, speech, and traffic signs by using deep learning which utilizes artificial neural networks. https://store.theartofservice.com/the-deep-learning-toolkit.html

46. Socratic questioning - Pedagogy • It teaches us the value of developing questioning minds in cultivating deep learning https://store.theartofservice.com/the-deep-learning-toolkit.html

47. Google Brain • 'Google Brain' is an unofficial name for a deep learning research project at Google. https://store.theartofservice.com/the-deep-learning-toolkit.html

48. Google Brain - History • Stanford University professor Andrew Ng who, since around 2006, became interested in using deep learning techniques to crack the problem of artificial intelligence, started Google's Deep Learning project (which would later acquire the name Google Brain) in 2011 as one of the Google X projects. The project's first in-depth coverage was in the New York Times in November 2011. https://store.theartofservice.com/the-deep-learning-toolkit.html

49. Google Brain - History • In March 2013, Google hired Geoffrey Hinton, a leading researcher in the deep learning field, and acquired the company DNNResearch Inc. headed by Hinton. Hinton said that he would be dividing his future time between his university research and his work at Google. https://store.theartofservice.com/the-deep-learning-toolkit.html

50. Google Brain - History • Moreover, In December 2012, futurist and inventor Ray Kurzweil, author of The Singularity is Near, joined Google in a full-time engineering director role, but focusing on the deep learning project. It was reported that Kurzweil would have unlimited resources to pursue his vision at Google. However, he is leading his own team, which is independent of Google Brain. https://store.theartofservice.com/the-deep-learning-toolkit.html