Symbolism & Connectionism

Symbolism & Connectionism • An overview by Erik Borra • For the course Philosophy of Mind 2003

What this talk is about. • What is symbolism? The first computer metaphor. • What is connectionism? The second computer metaphor. • I’ll show you • how they relate to the cognitive sciences • to which philosophies you can attribute them • how they answer some questions we saw in the course on Philosophy of Mind.

Symbolism: the first computer metaphor • First computers in the 50’s • According to cognitivists: • Although totally different qua structure and working, • the human brain and the digital computer are two instantiations • of the same thing. • The first computer metaphor: cognition is something that • shows intelligent behavior by manipulating symbols with • formal rules.

Symbolism: the first computer metaphor At its core, the serial digital computer is a machine that manipulates symbols. It takes individual symbols (or strings of symbols) as its input, applies a set of stored algorithms (a program) to that input, and produces more symbols (or strings of symbols) as its output. These steps are performed one at a time (albeit very quickly) by a central processor. Because of this serial constraint, problems to be solved by the First Computer Metaphor must be broken down into a hierarchical structure that permits the machine to reach solutions with maximum efficiency.

Symbolism: paradigm 1 • mental relations have a combinatorial syntax and semantics , in which a) there is difference between structurally atomic and structurally molecular representations (bv A or (A & B) ) b) structurally molecular expressions have syntactic constituents that are themselves either structurally molecular or are structurally atomic (hierarchical) c) the semantic content of a (molecular) representation is a function of the semantic contents of its syntactic parts, together with its constituent structure. This is the same as saying symbolism is committed to ‘complex’ mental representations or ‘symbol structures’

Symbolism: paradigm 2 • Structure sensitivity of processes Because classical mental representations have a combinatorial structure, it is possible for classical mental operations to apply to them by reference to their form. The result is that a paradigmatic classical mental process operates upon any mental representation that satisfies a given structural description and transforms it into a mental representation that satisfies another structural description.

Apple(X) Red(X) & Round(X) … Color(X) -> Red(X) or Orange(X) Shape(X) -> Round(X) or … Symbolism: graphical explanation • E.g. recognize a red apple input = symbol(s) -> algorithms who work on input -> output = more symbol(s) Input: Program: Output: Red(X) Round(X) … if (Orange(X) & Round(X) … ) then Orange(X) if (Red(X) & Round(X) …) then Apple(X) … Graphical: Apple(X)

Symbolism: paradigms united Mental relations have a combinatorial syntax and semantics Structure sensitivity of processes • Both these claims define Classical models, and they can be taken quite literally; they constrain the physical realizations of symbol structures. • Symbolist Claim: the physical properties onto which the structure of the symbols is mapped are the very properties that cause the system to behave as it does. The physical counterparts of the symbols, and their structural properties, cause the system’s behavior.

Symbolism: working revisited • There is an inference system working on the symbols that derives possible solutions out of a set of hypotheses. There is only a focus on the syntax of the symbols and the semantics (what they refer to) are disregarded. • The initial expressions (premises) must be true before taking inference in a formal system. • Rephrased: Symbolists acknowledge not only causal relations among the semantically evaluable objects that they posit, but also a range of structural relations. Of which constituency is paradigmatic

Symbolism: philosophic context 1 • There is a symbol level of representation which constitutes a language of thought • Some (philosophical) schools related to symbolism: • Representationalism • Realism • Rationalism • Nativism • Functionalism

Symbolism: philosophic context 2 • The first computer metaphor: cognition is something that shows intelligent behavior by manipulating symbols with formal rules. • The philosophy, that goes from Descartes’ “naturas simples” till the early Wittgensteinian “Tractatus Logico-Philosophicus”, states that there are absolute primitive simple elements (context-free) and logical relations in a subject who mirror the primitive elements and their relations of the world. GOFAI made this an empirical claim. GOFAI would find these elements and relations.

Symbolism: philosophic context 3 • Another implicit assumption in Western Philosophy was a symbolic information-processing in the brain. This tradition states that to understand a domain you need a theory for it. This theory should formulate the relation between objective context-free elements in terms of abstract principles like laws, rules and programs.

Symbolism: philosophic context 4 • Heidegger and the late Wittgenstein didn’t approve that the world could be represented by a set of context-free elements – in contrast to Husserl. Heidegger stated that the context, our world, and our everyday practices to deal with it easily, are not things were we think about, but are part of our socialization, which forms us the way we are. • What common-sense understanding amounts to might well be everyday know-how. By ‘know-how’ we do not mean procedural rules but knowing what to do in a vast number of special cases. (E.g. biking) Very hard and cumbersome to formalize!!!

Symbolism: Summing up • Based on first computer metaphor • Logical rules that operate on (structures of) symbols • Hierarchical • Serial • Efficient and fast • …

Symbolism: problems • Discrete representations. The symbols that are manipulated by a serial digital computer are discrete entities. They either are or are not present in the input • Absolute rules the program also consists of discrete rules • Learning as programming discrete decisions, all that really counts is already there from the beginning • The hardware/software distinction (functionalism; Frued, Gesell, Baldwin, Piaget …) • Symbolist approaches do not address issues of intentionality or meaning (E.g. The Chinese Room Argument) • Rules tend to have exceptions (E.g. The red apple can be a little green as well) • No (satisfying) generalization • …

The Switch After a lot of years and a lot of effort it still doesn’t work well Think about the red apple There were philosophical objections against the theories on which symbolism was founded (Late Wittgenstein, Heidegger, …) Maybe we would be better of if we could find a computational model (or class of models) in which it would be easier to organize and study the mutual constraints that hold between mental and neural development.

Connectionism: the opposite of symbolism Connectionism: systems that can exhibit intelligent behavior without storing, retrieving, or otherwise operating on structured symbolic expressions

Connectionism: a neural network • Based on an abstract view of a neuron • Artificial neurons are connected to form large networks • The connections determine the function of the network • Connections can often be formed by learning and do not need to be ‘programmed’

Connectionism: an artificial neuron

Connectionism: A simple example 1

Connectionism: the second computer metaphor Connectionist networks are networks consisting of very large numbers of simple but highly interconnected “units “. Certain assumptions are generally made both about the units and the connections: Each unit is assumed to receive real-valued activity (either excitatory or inhibitory or both) along its input lines. Typically the units do little more than sum this activity and change their state as a function (usually a threshold function) of this sum. Each connection is allowed to modulate the activity it transmits as a function of an intrinsic (but modifiable) property called its “weight”. Hence the activity on an input line is typically some non-linear function of the state of activity of its sources. The behavior of the network as a whole is a function of the initial state of activation of the units and of the weights on its connections, which serve as its only form of memory.

Connectionism: properties 1 • Neural networks can learn input-output associations • Do not operate on symbols in the usual sense of the term • No clear distinction between processor and memory • No homunculus

Connectionism: properties 2 – learning • learning by example • learning is a natural property of neural nets (inductive learning) • For a connectionist network learning is not done by adding or modifying propositions but rather by changing the connection weights between the simple units. Weights partly determine the state of the network. • knowledge is a property of the connection strengths that hold between the respective input and output layers; the machine can be said to “know” a pattern when it gives the correct output for a given class of inputs (including novel members of the input class that it has never seen before, i.e. generalization)

Connectionism: properties 3 – solutions • The solution is an emergent property of the system as a whole, a global pattern produced by independent, local computations • A neural network converges to some solution but will not be static. This is similar to the brain. (Cell growth and death) • Good pattern recognition • they are able to retrieve an output on the basis of incomplete input cues • they show graceful degradation • eventually the network will become overloaded with to many patterns

Connectionism: more … The network is a dynamical system which, once supplied with initial input, spreads excitations and inhibitions among its units. In some types of network, this process does not stop until a stable state is achieved. To understand a connectionist system as performing a cognitive task, it is necessary to supply an interpretation. This is typically done by viewing the initial activations supplied to the system as specifying a problem, and the resulting stable configuration as the system’s solution to the problem.

Connectionism: Summing up 1 • Neural plausibility (to some extent) • Parallel processing • graceful degradation • capacity to learn from experience and generalize • Distributed representations (same units can participate in many different patterns -> degrees of similarity in patterns)

Connectionism: Summing up 2 • Graded rules • Learning as structural change • Software as hardware • Non-linear dynamics • Emergent form • (elaborated) associationism

Connectionism: Intelligence?? • In neural nets you cannot directly explain what the nodes and weights are about nor what happens. Still, neural nets can be viewed as detecting higher order properties. Intelligence, defined as the knowledge of a certain set of associations on a certain domain, can be defined – as well – in terms of relations between some higher order properties and skills in a domain. • It is not the case that explaining properties have to consist the essential structure to make a theory about it, as the rationalists say. If the net receives a new association of input and output then the interpretation of the nodes will change. The properties of the nodes are no invariant properties of the domain.

Connectionism: problems • A large number of parameters to specify by the programmer • Not all learning is gradual • Catastrophic interference (E.g. trauma) • Constraints by programmer on what type of generalizations is possible. • Complexity • … • there is a lot more according to symbolists, but that would lead us to far. Some of these you can find in Fodor and Phylyshyn. Though most of their objections are later proven false: systematicity, productivity, recursion, … )

Connectionism: Philosophy and a little of the debate • Both: ?No dualism but causal closure and ?mental cause • Both have a lot of a priori assumptions • A symbolic system is purely nativist. Connectionist systems can be constructivist • The issue between symbolists and connectionists is not about the explicitness of rules: Symbolism is not per se committed to the idea that the explicit rules mediate the etiology of behavior. And it is not about the reality of representational states: symbolists and connectionists are all representational realists. And it is not about nonrepresentational architecture; a connectionist neural network can perfectly well implement a symbolist architecture at the cognitive level • Debate is about which is the right representation / implementation of cognition

Bridging the gap • Smolensky showed last Wednesday that children can learn a language (prefer some strategies) on the basis of examples. They actively adjust the preference of the order of strategy to be consistent with the examples they get. • Smolensky showed partly that a logic can be implemented in a network but that this logic doesn’t need to be a static native one. Children can adjust according to the examples they get and make structural changes. This couldn’t be possible with logic alone. Cause with logic you cannot start with for instance predicate logic and end up with a structural different logic, for instance epistemic logic. • A logic is bounded to its preprogrammed symbols and structures and cannot generalize or cause a paradigm shift. Some neuronal networks do have this propery, as in Smolensky (2003).

Bridging the gap • Smolenski is an integrationalist • He sees connectionism and symbolism as two sides of the same coin. • Contrast with eliminativists (Churchland, McCleland) and implementationalists (the hardcore symbolists).

Gebruikte literatuur en pointers naar meer literatuur • Fodor and Phylyshyn, Connectionism and Cognitive Architecture: A critical analysis • Classicalism and Cognitive Architecture, T. van Gelder & Niklasson • Making a Mind vs Modelling a brain, Dreyfus and Dreyfus, 1988 (samenvatting te vinden op http://gene.wins.uva.nl/~ekborra/ • Connectionism and the Mind, Bechtel & Abrahamsen, 2002 • Connectionism and the Study of Change, E. Bates & J. Elman • Artificial Intelligence: A Modern Approach, Stuart & Russel, 1995 Deze presentatie is te downloaden van http://gene.wins.uva.nl/~ekborra/ • Connectionisme: http://www.neuromod.org/ • Cognitive Sciences: http://www.csca.uva.nl/

Symbolism & Connectionism