ABSTRACT

1. ABSTRACT

2. BACKGROUND: A believable experience is not necessarily the experience of something real or existent. A believable experience does not necessarily resemble to an experience with real or existent entities, at least not in the sense that believable entities simulate the physical aspect and behavior of real entities. Believability is a form of judgment associated to specific emotional reactions and the conditions that trigger believability are not necessarily coincident with reality or realism; one significant condition seems to be represented by the respect of the users’ or audience’s activated expectations. OBJECTIVES: Identify the conditions that allow a believable experience with virtual or fictional entities even in absence of simulation of the phenomenological aspect and behavior of real entities. METHODS: Investigate the role played by expectations in the judgment of believability, describe types of expectations especially in relationship to different types of knowledge and their differential impact on believability. TITLE:Believability and expectations

3. RESULTS: Definition and types of expectations Expectations are mental states that range from beliefs based on symbolic knowledge to representations based on enactive forms of knowledge or connections and rules of perception produced by selection and repeated experience. When falsified by non-matching events, expectations give rise to surprise and revision (including anticipatory beliefs about future states of affairs) or at least to a sense of wrongness. Role of expectations in believability Experimental activity concerning the perception and interaction with both real and virtual worlds shows that the frustration of expectations and the violation of coherence play a negative role in adaptive and social behaviors and that the perceptual system actively works to maintain coherence. The awareness of violation of coherence and the frustration of expectations provoke a reaction of surprise and alerts the subject to the presence of some form of error, hence diminishing believability. Activated and de-activated of expectations Not all the expectations that users and audience hold are activated during a virtual or fictional experience. The activation and de-activation of expectations is a major instrument for enhancing believability. Certain expectations can be derived from other more general expectations or from expectations that are hold by the subject in virtue of the context of the cognitive activity and the intentions of the subject (volatile expectations). But also the expectations that are hold by the subject in a non-derived way need to be activated in order to produce cognitive effects. Epistemic and pragmatic goals on the side of the subject and the context of the cognitive activity are two components of the activation of expectations. Even the de-activation of expectations concerns expectations that have been previously activated. Different types of expectations present a different aptness to acquisition and modification through learning and training and a different profile of activation or de-activation.

4. CONCLUSIONS: The acquisition of knowledge about the expectations that are hold by potential users of VR applications, their differential potential of activation/de-activation, the instruments for the production of new knowledge at different levels and for the activation and de-activation of the desired expectations is a necessary condition for the design of believable experiences in VR. In addition to psychological and theoretic research advancements in this direction can be produced by experimental activity directed to individuate the minimal conditions of believability by stretching the characteristics of believable experiences. Tests of believability should include both subjective reports based on self-awareness (judgment of believability) and behavioral measures concerning the emotional consequences of the violation of expectations.

5. BACKGROUND • Believability is a judgment. When we consider a certain experience as believable we do not necessarily consider the experience as being true, in the sense of being an experience with real, existing objects. Neither we consider that experience as being susceptible of becoming true of the real world, either in the future, as show by the fact that fictional characters as Alexis Karamazoff or Wil Coyote can be considered as believable. • ‘Un-believable’ has a different meaning than ‘un-true’ because the judgment of un-believability implies impossibility and not only falsity: something which is judged as un-believable is something that can’t be as it appears (to perception or reasoning). • Hence, the judgment of believability is a judgment of possibility or impossibility, at least within a certain context (such as the one represented in the virtual or fictional world). • Possibility and impossibility are referred to the presence of a violation of coherence, both in the sense of the presence of a conflict between current experience and in the sense of the violation of past expectations. In daily life when someone exclaims: “I can’t believe it!” we guess that something unexpected has happened. • In normal cases one does not notice that the real world is going on in a believable fashion: the real world is trivially believable and it is only when expectations are unfulfilled that the problem of believability arises. • It is especially in the framework of fictional virtual entities that the need for a characterization of the notion of believability arises. Readers and audience of works of fiction, users of VR worlds in fact are used to find a certain virtual or fictional experience as more or less believable and to judge the fictional or virtual world with its inhabitants in terms of believability. • The judgment of believability is associated with a certain number of behavioral and emotional reactions. • Negative believability is associated with surprise. • Positive believability is associated with the possibility of putting in place a number of actions and emotions as if the experience were true (make believe game).

6. Two considerations follow: • Since no problem of existence in the real world is at stake, when we consider a certain experience as believable we just accept it as plausible or possible under certain conditions - such as the context of a certain virtual world - and as suitable for certain purposes - such as the purpose of playing a certain game of make-believe in which one can act or react as if the experience he is undertaking with the virtual world were true -. • If the notion of reality has no role to play in the judgment that a certain experience is believable, then the adherence of the experience with the experienced reality cannot be a criterion for characterizing believability. The judgment of believability seems to be especially related to the existence of different kinds ofexpectations and to the fact of respecting or violating them.

7. OBJECTIVES • Fictional and virtual worlds constitute a vast domain: narrative, films, dramatic arts in general ad VR in its different declinations. A good characterization of the notion of believability should apply to all these situations, and to believability in the real world too. • In order to respond to the requirement of generality, the characterization of the notion of believability should be minimalist: provide the minimal necessary conditions for believability and eventually provide the rules for gaining more and more believability and for producing believability in different situations. In this way, general characteristics of instruments for testing believability and general factors for enhancing believability can be indicated that are valid for all kinds of situations. • In the literature about dramatic arts, classic animation, and more recently computer animation for films and VR, interactive drama and VR in general, the issue of individuating the minimal conditions for believability is not directly addressed, and artists and designers normally provide a list of conditions that are considered as suitable for producing believable experiences and that are more or less related to the problem of realism and to the objective of designing or creating rich characters and agents.

8. METHODS • Characterize the notion of expectation • Describe different forms of expectations in respect to • The reaction of surprise • Their behavioral and emotional reactions • Their relationship with automatic/conscious mental states and processes • Their relationship to different types of knowledge • Describe the role of expectations upon believability through the description of the • Effects of violation of expectations upon adaptive behavior • Effects of the violation of expectations upon epistemic judgments • Describe the ways in which expectations can be acquired, derived, activated and de-activated • Indicate the conditions for testing believability in terms of expectations and for enhancing believability by the suitable activation/de-activation of expectations

9. DISCUSSIONExpectations and surprise • Expectations/Surprise • Expectations explain the reaction of surprise toward certain events and surprise constitutes a crucial element in the characterization and in the individuation of expectations, even when they are not linguistically expressed. • When an event produces that has never been experienced before two reactions can be observed in people: they are surprised by the event or they just accept the event and integrate the informational contents it represents. If someone is surprised, it means he was holding some expectation concerning that event that do not match with the event as it has produced. Expectations are hence necessary in order to justify the reaction of surprise and surprise can be characterized as being the reaction to the frustration of an expectation (Davidson, 2004). • The possibility of being surprised can hence function as a test (surprise test) for the capacity of entertaining expectations, or more generally beliefs • Someone puts his hand in his pocket and finds a coin. If he his surprised in finding the coin, then he comes to believe that his previous belief about his pockets and coins was false. Hence, he is aware that there is an objective reality which is independent from (previous) beliefs (Davidson, 1982. • Since surprise is “a telling betrayal of the subject’s having expected something else.” (Dennett, 2001, p. 982), the reaction of surprise can be used in order to shed light on the expectations of the subject.

10. Expectations: Active, Passive and Volatile • Let us have a look at some examples of surprise in relationship to unfulfilled expectations: • (1) I want to wear my brown shoes; I have a belief of my brown shoes being in the white shoe box or: I expect to find them there. But they are not in the white box, hence I am surprised. • (2) They are there but they have no strings. I am surprised. • (3) I believe that my brown shoes have strings, hence I expect to find strings if I check my belief against reality or if I go to take my brown shoes. • (4) I’ve been surprised in not finding strings to my shoes; of course, I expected to find them there. • (5) I’ve been surprised in not finding strings to my brown shoes; it must be that I expected to find them there, even if I can’t remember to have ever entertained such an expectation • The examples (2), (4) and (5) show that one is not always aware of entertaining a certain expectation and that the expectations one can gain access to through the reaction of surprise can present different levels of awareness: • in case (4) the subject re-gains awareness on a belief he did entertain; in case (5) the subject apparently was not aware of holding such a belief before the non-matching event was produced. • Case (5) can receive two interpretations. • In the first interpretation, case (5) can be associated with the “removed” psychological states described in psychoanalysis: the subject entertains a certain belief or intention which is removed far from his conscious states; this belief or intention shows itself in the subject’s behaviors and the subject can be forced to admit its existence, for instance in virtue of the interpretation of certain behaviors. • In cognitive psychology terms, one can consider the existence of two types of expectations: passive and active, with corresponding forms of surprise. • Active beliefs and expectations are expectations that are generically under the focus of attention and passive expectations are potentially derivable beliefs and expectations that are not actually under the focus of attention.

11. This distinction gives rise to a corresponding distinction between two types of surprise Ortony & Partridge, 1987 : • Surprise which results from the input proposition conflicting with an active, already derived expectation • Surprise which results from the input proposition conflicting with a passive and practically deducible expectation • An analogous distinction is operated by Lorini, 2005: • Mismatch-based Surprise: generated by the mismatch between active predictions and the explained raw sensor data (function of Unexpectedness). The mismatch is based on the existence of three conditions: sensory data which must be interpreted, a theory based on general conceptual knowledge or general knowledge concerning a certain domain (in any case, general knowledge), a set of active expectations composed of beliefs and expectations of different degrees of certainty (expectations are active in the sense that they are under the focus of attention of the subject, in contrast with potentially derivable beliefs that are not actually under the focus of attention). The set of expectations and the theory are consistent. On the basis of the theory, the raw data are interpreted (inferential approach to perception) and compared with the set of expectations; a certain mismatch and relative surprise can arise. The degree of the mismatch is calculated by taking into account the degree of certainty of the set of expectations that are referred to the explanation provided for the raw data and also the degree of certainty of expectations concerning the correlation of couples of elements of the theory (degree of dependence between the perceived features, after interpretation). This second quantity is not considered into other theories for calculating surprise, in which surprise is function of the unexpectedness of perceived single features only (see Macedo & Cardoso, 2001). The degree of the mismatch has also an effect on the explanatory value of the hypothesis or explanation of the raw data, when the mismatch is great, the explanation is revised for a better one. • Passive Prediction-based Surprise: generated by a later mismatch between the explained raw sensor data and a set of expectations that is derived from the set of active expectations and the theory. It can give rise to a further mismatch.

12. In the second interpretation, the subject is right in not remembering himself entertaining the belief that his brown shoes have strings, because he did not. He did entertain the general belief that shoes have strings and the local belief that there are brown shoes in the white box, but not the local belief or expectation that the brown shoes have strings. • The concept of Volatile expectations (Casati & Pasquinelli, Forthcoming) must be introduced: • There is no need for individuals to hold all the expectations like (5) all the time, and in particular to hold all the beliefs that correspond to all of these expectations all the time. Common people minds would be a little bit overloaded. • It is just possible that people hold some general beliefs, such as some general beliefs about the ontology of our world (the entities that furnish our world and the laws that describe their behavior), with no belief, for instance, about the entities that do not furnish our world. • In special occasions expectations that are related to general beliefs are frustrated, as it happens when a gremlin appears. • The expectation that gremlins do not exist is hence a volatile expectation, generated by a certain context and related to some general belief about the entities that furnish our world, with no corresponding specific belief. • Volatile expectations can also be derived from local beliefs and expectations, not only from general beliefs and relative expectations. • The difference between the concept of volatile expectations and the concept of passive or derived expectations consists in the fact that volatile expectations are generated only when precise conditions are given; these conditions can be related to the context of the experience and/or to the goals of the subject. The conditions that operate the generation of volatile expectations are relevant for the content of the generated expectations. Hence, the content of volatile expectations depends both on the beliefs that are hold by the subject and on the conditions of their generation. On the contrary the content of passive expectations seems to depend only on the expectations and beliefs that are hold by the subject in an active way. • Volatile expectations are not expectations that are hold by that subject and that are not under the focus of attention: they are not hold by the subject up to the moment in which they are unfulfilled by some event. • Only the reaction of surprise alerts the individual about the fact that the sudden appearance of a gremlin is an unexpected event to him, and only at that point the individual can express the belief –expectation that gremlins do not exist, or at least that they do not appear out of the blue.

13. Non-volatile expectations are expectations that are hold by the subject and that do not need special conditions in order to be generated in a particular occasion. • Nevertheless, even if they do not need to be generated, non-volatile expectations might need to be activated. • In fact, not all the expectations and beliefs that are hold by a subject are relevant at any moment and in any condition. This fact might explain the analogies and differences between the emotional and behavioral reactions that are evoked by a certain event in the real world and the same event in a fictional world (the so-called ‘paradox of fiction’). • Independently of the fact of being derived or direct, non-volatile expectations could hence be considered as passive or active depending on if they are relevant or activated in a certain situation or if the are not relevant and passive in that situation. • Two hypothesis can be advanced: • Hypothesis of activation. Activation of expectations might be coincident with focusing the attention on them but not necessarily with the fact of the content of the expectations becoming aware or being symbolically expressed. As for volatile expectations, certain factors such as the context of the experience or the goals of the subject might in fact select some expectations between all the expectations that are hold by the subject, and only the selected expectations would be active and would be relevant (have effects) on the emotional and behavioral reactions of the subject. • Hypothesis of de-activation. All the expectations that are hold by the subject in a non-derived way are active, and special conditions of context and goals de-activate the expectations that are not relevant for the situation. • The necessity of activation and de-activation and the conditions that produce the activation or de-activation of expectations will be investigated later in this document. • Following the two hypotheses we hence have the following types of expectations: • Hypothesis of activation: volatile expectations that are not hold by the subject but are generated by context/goals, non-volatile passive expectations that are hold by the subject but are not relevant for the current context/goals and are hence not activated, non-volatile active expectations that are relevant for the current context/goals and that are in fact under the focus of attention in virtue of the activating effect of the context/goals condition. • Hypothesis of de-activation: volatile expectations that are not hold by the subject but are generated by context/goals, non-volatile expectations that are hold by the subject and are thus all active even if they are not relevant for the current context/goal unless the context/goal condition de-activates them.

14. Expectations: Future • (1) overtly expresses an expectation as a belief about a future state of affairs. In (2) no belief about future states of affairs is overtly expressed. Once the reaction of surprise has been produced it is possible to evince that the subject did entertain the belief that the brown shoes have strings or that shoes have strings as in (4) and (5). • A belief becomes an expectation once the belief is considered in the perspective of future states of affair, as in (3). This is in accord with (Castelfranchi, Castelfranchi & Lorini, 2003) who characterize expectations as part of a class of mental states that consist in more or less certain anticipatory representations about future states of affairs; this class includes expectations, hypotheses and forecasts or predictions. • Hypotheses consist in the belief that the future state p is possible; • Forecasts in the belief that the future state p is probable (that is, in the belief that the chance threshold for p has been exceeded); • Predictions in the belief that the future state p is quite certain (probability esteemed next to 100%) • Expectations are predictions or forecasts associated to a motivational component (a goal, drive, motive, concern). • As shown in (3) goals can be represented by epistemic intentions (verify one’s own belief) or by pragmatic intentions (act on some object which is believed to present certain characteristics). • But in (4) and (5) the existence of an expectation and the content of the belief that constitutes the expectation are evinced only après-coup, because the subject becomes aware of entertaining that certain belief only after the non-matching state of affairs has produced. • In (4) the non-matching event allows the subject to recover an expectation that he did know to entertain; in (5) the subject is informed about the fact that surprise is a telling betrayal of expectations, hence he is forced to admit that he did entertain a certain expectation, even if he was not aware of that. • Expectations are hence beliefs that can be expressed in the form of anticipations, at least après-coup - after that a certain state of affair has provoked a reaction of surprise and hence alerted the subject to the fact that he must have entertained a certain expectation.

15. Expectations: Goals • Goals activate expectations [Castelfranchi & Lorini, 2003]. • And also, expectations are different from other purely anticipatory states because they matter for the subject, the subject is involved, has motivations concerning his predictions that induce him to make predictions [Castelfranchi, ]. • Since goals can be realized or discarded, not any goal implies (activates) expectations, but only active non-realized goals do. • Expectations activate Epistemic Control (a sub-category of epistemic actions, that is, actions directed to gain knowledge about the world): active perception, monitoring, testing, matching activities. Epistemic control includes goal-directed, rule-based actions (proto-intentions, anticipatory classifiers) ad intentional actions with explicit representations of the expected state. • Goal-directed systems are different from reactive systems because the formers present purposive behaviors activated by a mismatch between the current state of the world (perception) and the state of the world represented as a goal (explicit representation).

16. On the basis of the accord between belief and goal, expectations can be described as being negative or positive (Castelfranchi, Castelfranchi & Lorini, 2003). • Negative expectations have goal opposite to the prediction; positive expectation: have the conformable goal of the expectation. • A positive expectation is always present in order to satisfy a need to confirm one’s own capacity of prediction or predictability as a major meta-goal of the cognitive system. Epistemic agents are in fact in any case frustrated and disoriented by surprise, by non-concordant events because they represent failures of their capacity to predict, hence to control (Miceli, 2002). • There are hence four possible scenarios: negative expectations (prediction that p, goal that not-p), positive expectations (prediction that p, goal that p), ambivalent expectations (belief that p, goal that not-p and goal that p), neutral expectations (belief that p, goal that p and not goal that not-p). • But negative expectations are always ambivalent in virtue of the presence of the positive expectations that confirms the subject’s capacity of prediction. • Castelfranchi describes two major types of expectation in relationship to the active or passive character of the experience. • Expectations are active if the goal implies action on the side of the subject. • Expectations are passive if the realization of the goal does not depend on him, but he just executes matching procedures for controlling if the goal has been realized (by others, by nature). • Goals in fact include also epistemic intentions that are not necessarily related to actions and other desires that are not realized by the action of the subject who entertains the expectation; hence expectations are broader than goal-directed actions, for instance the desire that one’s own predictions are confirmed. This goal is related to the importance for the subject of representing himself as a successful predictor and on the importance of mechanisms of control. • Once quantitative parameters are added (both on the side of beliefs, that can present different degrees of certainty or strength, and on the side of goals, that can present a greater or lesser value or subjective importance for the subject), other anticipatory states can be characterized (Castelfranchi, Castelfranchi & Lorini, 2003): • Hopes can be characterized as positive expectations with different levels of certainty (hopes or hope-casts) and fears as negative expectations with any degree of certainty (from fears to fear-casts) and value.

17. The question arises if fictional and virtual worlds can present expectations in the sense of anticipatory states with goals or only predictions. • The question is relevant because the effects of validation and invalidation are different for expectations and for predictions: an expectation that reveals wrong is a frustration or goal failure for the goal that p and an invalidation, falsification, prediction-failure for the belief that p. • In relationship to the positive or negative expectation and the nature of the validating or invalidating event (and consequent belief), in fact there will be the following forms of invalidation: • No surprise but achievement when a positive expectation is validated; • Frustration with negative surprise or disappointment when a positive expectation (hope or hope-cast) is invalidated; • Positive surprise or relief when a negative expectation (fear or fear-cast) is invalidated; • Frustration with no surprise when a negative expectation is validated. • The degree of disappointment and relief are in relation with both the strength of the belief and the value of the goal. • Frustration or goal-failure is the falsification of a goal and belief-failure is the falsification of a belief. • The strength of the effects of invalidating beliefs can be measured because it is function of the strength of the belief (expressed in terms of the degree of certainty) and of the strength of the goal (expressed in terms of the subjective value of the goal). • The surprise component is in relation to the failure of the prediction (hence its strength is in relation to the degree of certainty of the prediction) and the disappointment/relief component is in relation to the frustration of the goal (hence its strength is in relation to the value of the goal). (See Lorini, 2005). • For this reason, surprise is activated even in case of failure of a prediction in a negative expectation, when no frustration is evoked.

18. As a first consideration, the distinction between expectations and predictions is not always clear. • (6) A certain experience in the visual sensory modality activates the anticipation of a corresponding experience in the touch sense modality, only on the basis of connections based on statistical co-activation. • (7) a subject intends to catch a ball (goal of catching a ball) and evaluates the shape of the ball in order to catch it properly with his hands. On the basis of the visual information the subject pre-shapes his hand in relationship to the shape and dimension of the ball. • The behavior in (7) seems to be a purely anticipatory one, because no goal is directly associated with it. Nevertheless, the behavior described in case (7) is common to goal-directed and reactive systems: reactive systems react to a ball with sensorimotor matching and actions plans even if no explicit (in the sense of symbolic) goal of catching a ball can be ascertained. It all depends if we associate the general goal to the different predictions that follow. • (8) a subject intends to dance and is a skilled dancer. The subject puts in action a number of movements that are based on automatic action plans and sensorimotor matchings. No specific goal is related to the single matchings and plans. The dance behavior is hence composed of general goals and hence of goal-directed behaviors, but also of purely reactive behaviors activated by the context (state of muscles, sensory feedbacks). • If only automatic, reactive actions activated by the context are present, the behavior in (8) is not a dance (as the so-called dance of bees is not a dance but a reactive behavior in presence of certain conditions); it is the presence of goals, hence of symbolic representations, that makes (8) be a case of dance. Nevertheless, it is not clear which of the behaviors put in place in the case (8) are pure predictions and which ones are expectations. • Hence the distinction between expectations and predictions becomes less clear.

19. Additionally, according to (Castelfranchi, Giardini, Lorini, Tummolini, 2003) beliefs have a tendency to become expectations. • Simple beliefs about future actions and events tend to become full expectations (with goals) in reason of the component of prediction and self-efficacy which is implicated in anticipatory states: cognitive agents make predictions in order to avoid anxiety and stress, act in order to find their predictions validated by facts and feel distress in case of invalidation. Hence, when a cognitive agent predicts a certain event, he has a tendency to act as if he wanted the prediction to be confirmed, even if it is a negative prediction. • An invalidated hope-cast is felt not as a simple distress but as an ill-treatment. People have a tendency to add a normative component to their strong predictions. Since the belief that p is well grounded and the agents wants it to happen, p turns to be something that is bound to happen. • Miceli, 2002 considers the need for predictability as a meta-goal: it is not represented (at the conscious or unconscious level) but has a function on the regulation of mind processes. • In analogy with the concern for predictability that would be present in any anticipatory state, we could consider that a general concern for coherence constitutes another regulatory goal or meta-goal. • As we will see in fact, when a violation of expectation or violation of coherence is detected by the system, the system reacts with a tentative to eliminate the inconsistency and re-establish coherence. • Miceli, 2002 considers regulatory principles as the re-establishment of coherence and the need for prediction as meta-goals because they do not need to be present in the mind as representations on the basis of which the mind reasons and plans (for instance when calculating costs and benefits) , but as functions or procedures that are implemented, for instance when a contradiction is detected. • It is important to maintain the functional meta-goal level distinct from the goal-level: even if meta-goals can become goals when assumed as representations they do not need to be represented. • The need for coherence is testified by the existence of procedures for eliminating contradiction when it is detected, because contradiction produces unpleasant cognitive states (Festinger, 1957; Carlsmith & Aronson, 1963; Cooper & Fazio, 1984; Fazio & Cooper, 1983) and by manifestations of the principle of minimal change in belief revision ( Gardenfors, 1992; Harman, 1986). • Thus, if the content of the experience is not coherent, the subject experiences both prediction failure (invalidation of the belief or in general of the anticipation) and frustration or disappointment in reason of the invalidation of the coherence procedure. Frustration will be more or less strong in connection with the strength of coherence procedure. We can in fact imagine that the need forcoherence is stronger when action is involved because violations of coherence block action than when the organism is just a spectator. The spectator can be more disappointed when he holds prevision with a high degree of certainty about the enjoyment he will obtain from the spectacle or when the fact of enjoying the spectacle has a great value for him.

20. (9) The user of a VR world can act in the virtual environment and hence have goals that concern his actions and their result, for instance the goal of surviving during a pilot test and the belief that the executed action is efficient for the goal. The results’ of the user actions are the object of active expectations, because they involve beliefs and goals. • (10) In the case of passive media, the spectator of a movie can make certain predictions or forecasts concerning the story which is represented (what will happen to the hero, for instance); if the spectator is involved he can also have a preference for certain events to happen (the hero is saved in a difficult situation). The preference constitutes a form of goal, and we are hence in presence of an expectation, passive expectation in this case. • When the expectations in (9) and (10) are unfulfilled the users and spectators are surprised in virtue of the invalidation of expectations and also disappointed, at different degrees according to the respective degree of certainty of the belief and value of the goal. • These emotions concern the contents of the story or more in general the events that take place in the virtual and fictional worlds. • We can hence conclude that users and spectators hold expectations (with goals) when the events that happen in the frame-work of the virtual or fictional world are concerned (contents of the fictional or virtual world). • The case of passive medias (such as cinema, radio) is different from the case of interactive media, where the subject can actively operate on the fictional or distant world (VR systems but also telephone). In the case of the interaction with a virtual environment, the user can have the goal of chasing a gremlin or catching a virtual ball, which is not the case for the spectator of a movie, even of a SF movie featuring gremlins or of a movies featuring baseball. • Only experiences with interactive media can present active expectations, because only in this case action on the side of the subject is allowed.

21. Surprise: Believability • The problem arises of the relationship between surprise and believability. • The examples in (9) and (10) show that invalidated expectations are tolerated at the level of the story, at least a certain number of invalidations and invalidations of a certain kind. The user or spectator accept the situation and change their beliefs, as in the case of the surprise test. • What makes an experience of mismatch that causes surprise be un-believable? • The judgment of un-believability can be applied to single experiences or to long course experiences, such as a film or an entire experience with a virtual world.

22. We can advance several hypothesis, that are not mutually exclusive: • Believability is related to the degree of surprise or to the degree of violation of the expectation, in two senses: • distance between the expectation and the invalidating belief: • (11) I expected it would snow, and it rains; • (12) I expected it would snow and it’s 40 degrees. In (12) the distance between expectation and invalidating belief is greater than in (11). It can be hypothesized that the greater the distance the greater the surprise. It is not clear if great distance alone can give rise to an experience which appears wrong. • Degree of certainty of the invalidated expectation: the invalidation of beliefs with high degree of certainty (predictions) is more suitable to produce un-believability. The degree of certainty can, on its side, depend on different conditions: • Active expectations and passive expectations. For instance, • (13) the expectations concerning what a movie’s hero will do next cannot have a high level of certainty, because the spectator knows that it is up to the writer and director to decide about the contents of the story. • The type of knowledge to which the expectation is related: the type of belief or type of knowledge involved can be relevant because different types of belief might present different levels of certainty.

23. Believability of a certain experience with long duration is related to the quantity of surprising effects or of violations of coherence : the presence of a certain number of violations of coherence could be tolerated, also in reason of their degree of surprise, but an excessive amount of violations of coherence would make the global experience un-believable • Believability of a certain experience is related to the quality of the surprise, hence to the invalidation of the goal and not only, as in 1. and 2. to the invalidation of belief: relief would be more tolerated than disappointment. • Believability depends on the existence of a certain threshold of subjective tolerance to the invalidation of expectations, which could be related to • The emotional state of the subject • The cognitive state of the subject, in particular: will to believe vs. will to ascertain, hence to the presence of general goals concerning the global experience

24. Believability depends on the type of expectation, in three senses: • The type of knowledge to which the expectation is related: the type of belief or type of knowledge involved can be relevant because different types of belief might be more or less easy to revise. This issue will be discussed later in this document where the different types of knowledge expectations are related to will be analyzed.

25. The type ofcontent of the expectation: the invalidation of expectations such as in (9) ad (10) that concern the contents of the world seems to be more tolerated than the invalidation of expectations that concern the way of presentation of the contents of the fictional or virtual world - (13) Spectators and users of VR environments make predictions or have expectations concerning the complexity and accuracy in the representation of the motor behavior of a represented character because of the level of complexity and accuracy of the representation of the physical aspect of the same character. An example of this kind of expectation or prediction and of the disrupting effects of its invalidation are described by Garau et al. (2003) in an experiment featuring a discrepancy between levels of realism for the representation of the physical aspect and the representation of motor behavior. • Four conditions are tested: in the first one realistic physical aspect is associated with a less-realistic behaviour, in the second one realistic physical aspect is associated with realistic behaviour, in the third one non-realistic physical aspect is associated with realistic behaviour and finally non-realistic physical aspect is associated with non-realistic behaviour. The non-realistic physical aspect is represented by a sort of sticky human and the behaviour is the avatar’s expressiveness represented by gaze: in the realistic option gaze is associated with the speaking and listening turns during the communication between the avatar and the human (inferred gaze), in the less-realistic option random gaze is used. Four effects are measured through a questionnaire: face-to-face effectiveness, sense of co-presence, involvement and partner evaluation. The results indicate the following impact of behavioral realism with different levels of visual realism: for the lower realism avatar, the more realistic inferred gaze behaviour reduces face-to-face effectiveness, sense of co-presence and partner evaluation, but has no effect on involvement; for the higher realism avatar the more realistic inferred gaze behaviour increases face-to-face effectiveness, sense of co-presence and partner evaluation. It hence seems that for lower realistic avatars, the realistic behaviour has a consistently negative effect. The opposite is true for more realistic avatars. In other words, it seems that low fidelity in one domain, physical aspect or motor behaviour, demands the same level of low fidelity in the other domains and that consistency between the visual aspect and the motor behaviour is necessary in order to produce effective communication and co-presence in the case of the interaction of human beings with artificial human-like entities. • It is proposed that realism in one domain (physical aspect or motor behaviour) raises in the audience or users a certain number of expectations concerning the other domains; for instance a realistic representation of the motor behaviour of an entity raises certain expectations about the aspect of the entity. The effect of the violation of these expectations is more powerful than the effect of the motor behaviour. The relation between the audience or users and the artificial entity is disrupted.

26. The type of experience during which expectations are expressed. Certain experiences might alert to the possibility of implausibility and put in place implausibility tests, and also generate a special type of surprise, of the kind described by Lorini, 2005, as Implausibility-based surprise: • Lorini, 2005 affirms that even if surprise is always generated by expectation’s failure, three types of surprise can be distinguished: mismatch-based, passive prediction-based and implausibility-based surprise: • Implausibility-based Surprise: generated by a test about the plausibility of the explained raw sensor data in relation with my active belief system (function of Incredulity). Implausibility or incredulity is not generated from a mismatch, but is generated by some form of simulation: the agent assumes a certain explanation of the raw data to be a possible explanation and before assuming it the distance between the explanation and the expectations and knowledge is measured.

27. Implausibility tests could be put in place in special situations, such as experiences with representations, when the agent is somehow alerted to the possibility that what he perceives does not correspond to reality, or special perceptual conditions such as the presence of disturbances that create uncertainty (mist) or even special conditions of the perceiver that make him unsure about his senses (drug). • In case of representations, such as it is the case for fictional and virtual worlds, one is not disposed to revise his own previous beliefs just because they conflict with the contents of the representation. If a conflict exists, the contents of the representation are considered as un-believable and the expectations are maintained. • Nevertheless, a conflict precedes the reaction of surprise and the judgment of un-believability of the current experience. • On the contrary, when experience with the real world is at stake and a conflict is ascertained with previous beliefs, perception is normally more trusted than knowledge (at least in daily conditions, and not in the frame-work of scientific research, where even observational proposition are interpreted in the light of the theory). • We can say that perception is evidence against which beliefs are normally revised, unless certain conditions occur that alert the perceiver that the perceptual experience cannot be trusted more than expectations and knowledge.

28. The different attitude toward perception of the real world and perception of representations could explain the fact that believability is much more a problem in virtual and fictional experiences than in in reality. In the experience with the real world, one does not start with thinking “let see if this is plausible or not, let us control if what I perceive is plausible”. • It is not only a matter of certainty of the expectations, but of a different attitude to the revision of beliefs when perception of the real world or perception of fictional and virtual world are concerned. • Hence, the fact of being aware of the presence of a representation would activate a plausibility test (or considerations about the plausibility of the experience) and would explain the fact that in case of conflict with expectations or other type of conflict, surprise gives rise to un-believability rather than to belief revision. • The cited situations would give rise to a special form of surprise, especially related to un-believability, which is implausibility-based surprise. • It can be hypothesized that normal mismatch-based surprise can give rise to implausibility-based surprise even in normal perceptual conditions: for instance in the case of a great number of conflicts or of conflict of great strength, the perceiver could begin to put into doubt the perceptual evidence, start some simulation procedures and be startled with implausibility based surprise.

29. Belief revision • In any case the problem of un-believability and surprise seems to be related to the problem of belief revision following the experience of a conflict. • When an expectation conflicts with another or with an invalidating belief and surprise arises three possible consequences are: • The invalidating belief is accepted as true, and the belief which is contained in the expectation is revised. It is the case of (9) and (10) and of the surprise test. • The invalidating belief is not accepted, but the belief contained in the expectation is maintained. This case resembles to other cases of assimilation to expectations. We will see that assimilation as a way for solving the conflict between two inconsistent contents is quite a common solution for the perceptual and cognitive systems. • The assimilation can be unconscious, when the subject does not notice the conflict • or the subject is aware of the presence of a discrepancy and of the value of the discrepant elements and chooses to rely on his past beliefs more than on current evidence. Hence the invalidating belief is rejected as inconsistent with a belief which is not revised, hence as false. The fact that the subject is aware of refuting the perceptual evidence creates a special condition because, in the mean time, the subject knows that perception is normally reliable and he is hence faced with a special kind of surprise, of the type of implausibility-based surprise. This case hence concerns perception; • In the case of cognitive evidence (reasoning) rather than perceptual evidence, the subject does not experience the same kind of implausibility.

30. Finally, the subject cannot say which of the conflicting expectations is true or false, he only knows that something must be wrong. • The invalidating belief cannot be accepted but it is not possible to operate assimilation or to find other solutions, and the conflict remains unsolved. • Or there is no necessity of choosing between one of the conflicting expectations, but the simple fact of a conflict makes the experience appear un-believable because the subject is alerted to the presence of an error. This situation might be common for special violations of expectations that will be described later as synchronic and not diachronic violations of expectations, and for expectations, as in (13) that do not concern the content of the experience but its way of presentation, hence structural elements that can pass unnoticed at the level of symbolic beliefs but operate at the automatic, sub-personal level. Also in this case, the awareness that something must be wrong (the sense of wrongness and impossibility) creates the conditions for a special kind of surprise based on implausibility. • Both 2.2 and 3. are cases of un-believability in the sense of implausibility, characterized by a special type of surprise connected to impossibility or implausibility.

31. When a virtual or fictional world or agent or object presents a certain level of violation of coherence and expectations the users or spectators react with a special form of surprise (in the sense of alert to error) which does not give rise to belief revision but to a judgment of implausibility which concerns the current experience: the actual experience is rejected. • The rejected experience is not simply considered as false: it can’t be true because, if it were true, it would create a conflict, a violation of coherence, with other experiences or with expectations that cannot be dismissed (because they are too strong, too important for the subject, etc.) or with expectations for which the current experience does not give sufficient reasons for revising (because the current experience is known to be fictional or virtual, because the subject cannot trust his senses, etc.). • It can be suggested that coherence is a regulatory principle for the cognitive system and that violations of coherence are perceived as impossible conditions. This issue will be discussed later in detail when the consequences of violations of coherence for the cognitive system will be analized. • The rejected experience is hence un-believable because if it were true a conflict or violation of coherence would arise, an impossible situation which is unacceptable for the cognitive system. • The fact of considering what would happen if the experience were true is a form of simulation and suggests the existence of plausibility-implausibility tests that are practiced in particular conditions. • When the test gives a negative result (a result that indicates that conflict will ensue), a judgment of un-believability is expressed which means: if it were true, conflict would ensue, but this is impossible, hence the experience is un-believable. • When the test gives a positive result (a result that indicates that conflict will not ensue), a judgment of believability is expressed, which means: if it were true, no conflict would ensue, hence it is possible, at least within the present context. • In the case of virtual and fictional experience a mismatch can be at the origin of the necessity of practicing a simulation test of plausibility. A simulation test can be motivated by particular conditions of uncertainty, but also by the perception of a mismatch with expectations or between current experiences. In this latter case, surprise based on mismatch could precede the judgment of believability or un-believability.

32. DISCUSSIONExpectations and knowledge • The different hypotheses expressed in the first part concerning the relationship between surprise and believability evidence the importance of knowing which types of knowledge are involved in the formation of expectations and which are their different characteristics in terms of degree of certainty, susceptibility to revision, type of experience and type of conflicts to which they give rise. • The different types of knowledge that give rise to expectations that play a role in the believability of experiences with virtual and fictional worlds should hence be classified and the different characteristics of each type of expectation should be analyzed.

33. Knowledge: expectations are related to different types of knowledge, only the case of symbolic knowledge gives rise to beliefs, hence to expectations based on beliefs. • According to Dennett, “Surprise is only possible when it upsets belief. But there are examples of non-linguistic expectations.” (Dennett, 2001, p. 98) • (1) and (3) are examples of linguistically expressed expectations; in (4) and (5) the expectation is linguistically expressed only after that a state of affairs has produced which does not fulfill some of the subject’s belief. We can say that in cases (4) and (5) a belief is entertained (or at least it can be derived from entertained beliefs) even if it is not linguistically expressed (at least not before the surprising event). • But there are also other possibilities in which expectations are not linguistically expressed, and even cases in which expectations do not necessarily have a belief for content. Dennett is right in the sense that the outcome of the upsetting of this type of expectations is not a full reaction of surprise, but a reaction which share many analogies with surprise and which consists in a sense of wrongness and bizarreness.

34. Expectations in (1), (3), (4), (5) are all based on beliefs, that can be linguistically or not linguistically expressed but that have all a symbolic nature. • They are based upon a form of knowledge which contents are organized and represented in a symbolic form: symbolic knowledge, the kind of abstract knowledge which is proper for cognitive functions as language and mathematics [Bruner, 1966; Bruner, 1968]. • Two types of symbolic knowledge can be described that are relevant for distinguishing between types of expectations: • Scientific knowledge • Commonsense knowledge

35. Scientific knowledge • Scientific knowledge is a special form of symbolic knowledge which is true of the world and justified. • A recently appeared discussion about the believability of the planets and worlds depicted by the Star Wars saga, conducted by two scientists specialized in astrophysics and extraterrestrial life [Lovgren, 2005], suggests the specific role that expectations might play in the characterization of the notion of believability. The interviewed scientists, B. Betts (a planetary scientist at the Planetary Society in Pasadena, California) and S. Shostak (a senior astronomer at theSETI - Search for Extraterrestrial Intelligence Institute in Mountain View, California), affirm having trouble buying some planets of the “galaxy far, far away” because they are “beyond our current expectations or measurements”. Following their discussion we evince that their judgment on the believability of the planets and worlds depicted in Star Wars depends on the expectations based on what scientific knowledge tells about planets and worlds: a credible world is a world that could exist without violating the expectations that scientists have. It is a possible world relatively to scientific knowledge, even if it is not necessarily an existing world, or a world that will ever exist. Science fiction is not scientific discovery, it can just be scientifically plausible. • The knowledge of the two scientists that judge of the believability of the Star War’s worlds is specialized and scientific (astrophysics, astrobiology). So are the expectations they hold and against which they judge the believability of their experience. • Also VR experiences can involve scientific knowledge, as it can be the case for training and simulation for medical applications. • Nevertheless, other spectators of Star Wars saga or of other Science Fiction movies or VR users do not necessarily hold scientific knowledge about the contents of the fictional or virtual world with relative expectations. In considering the conditions that are relevant for the judgment of believability, we must then take into account different kinds of knowledge ad relative expectations. One type of knowledge which certainly seems to be involved is the so-called commonsense knowledge.

36. Commonsense knowledge • Many types of commonsense knowledge can be described: commonsense knowledge is expressed in fact by the beliefs that most people entertain about the entities and events that populate their world. These beliefs range from very general beliefs concerning the way the world is constituted (the structure of objects and events) to specific beliefs concerning the laws that organize our world and the specific entities with which it is filled in. • A first type of commonsense knowledge makes reference to the way people parse the world in objects in first instance. • [Kölher, ] for instance describes people naïve experience as consisting first of all of objects, “their properties and changes, which appear to exist and to happen quite independently of us” [11, p. 1, 2] • Commonsense knowledge of the type of naïve, qualitative or folk physics makes reference to the aspect of the world as most of the people think about it, rather than to the world as physicists think about it. This form of commonsense knowledge is expressed by beliefs (eventually by theories) and generates symbolic expectations, as it is the case for scientific knowledge; contrarily to scientific knowledge it is not necessarily correct or justified. Folk physics can be at odds with scientific physics and in this sense be false. Commonsense knowledge of this type is exemplified by two widely hold naïve physics theories: the “force” of sucking and the “impetus” of motion; both theories adequately describe everyday phenomena, even if they are considered as inappropriate by physicists.

37. Other domains must be added to physics, concerning to which common people hold more or less justified and correct beliefs, theories and symbolic expectations: folk psychology (theories about the functioning of mind and other persons’ inner states), but also geography, biology, ecology, zoology, etc. • In fact, we can suppose that a belief such as the following are shared by most of common people: • (14) gremlins do not existin our world • As we have seen before, it is not necessary that expectations such as (6) are explicitly hold. One could have never had the occasion to think to gremlins up to the moment he is surprised in seeing a gremlin popping out from a shoe box or appearing out of the blue. Nevertheless, the expectation that gremlins do not exist can be derived from other expectations or beliefs concerning the type of biological entities that live in our world. The reaction of surprise reveals hence the presence of unfulfilled, stable or volatile, direct or derived expectations, such as the expectation that gremlins do not exist (belief about the biological entities that populate our world) or the expectation that in our world objects do not appear out of the blue (belief about the physics of our world). • Other beliefs can be as commonly shared as (14) that concern folk psychology or biology.

38. Commonsense knowledge is characterized by the fact of being largely shared and by the fact of constituting a form of symbolic knowledge which is not necessarily as justified and exact as scientific knowledge is. • Variations in the first of the two characteristic produce forms of commonsense knowledge that are not so largely shared and that eventually can be bound to restricted groups of individuals in virtue of their acquaintance with specific portions of the world or even with worlds other than the real one , such as worlds of fiction or virtual worlds. Commonsense knowledge about physical facts related to the behavior of oceans is plausibly more diffused in populations that live next to the ocean, and knowledge about cartoon physics is certainly bound to cartoon spectators. • Cartoon physics groups the physical laws that apply in fictional worlds, and in particular in Warner cartoons (Looney Toones and Merry Melodies). These laws differ from commonsense physics’ laws of our real world. But people know them by experience with Warner’s fiction and are no more surprised by the fact that • (15) Any body suspended in space will remain in space until made aware of its situation. • This is not really commonsense knowledge, because it does not concern the world which is common to most people, but it is sufficiently shared.

39. It should be investigated which other types of beliefs and expectations people have developed in virtue of their acquaintance with fictional media that concern the experience of fictional contents. • The absence of surprise and other emotional reactions tells us for instance that people have acquired a commonsense knowledge about what a motion picture is: it is not the real world, nor necessarily a truthful representation of the real world; as a matter of fact, it’s long time people do not leave their sits in panic at the sight of a motion picture featuring a train or a horse [Stoffregen, 2003]. • (16) Giant leeches are fictional characters, not real ones, hence they cannot hurt me. • In spite of the fact that fictional worlds are not taken for the real one or for real worlds in general, they continue to produce strong emotional reactions and to be judged as more or less believable. • The experience of virtual worlds is still too limited and varied for producing theories and laws such as those described under the label of “cartoon physics”. But it is possible that a diffused experience with virtual worlds could produce a form of acquaintance with the limits and possibilities of this form of representation and for consequence modify the expectations that are hold during the experience of a virtual world. • Expectations derived from commonsense knowledge can hence be as general as the expectation that the world is constituted of objects or they can make reference to the specific properties of specific objects in specific conditions, depending on the level of ontological specification. • They can also be more or less commonly shared depending on the level of universality of the experiences that produce the sort of knowledge they are based on.

40. Other forms of knowledge exist that are not of symbolic nature. • Enactive knowledge • Bruner, 1966; Bruner, 1968 describes three systems or ways of organizing knowledge and three correspondent forms of representation of the interaction with the world: enactive, iconic and symbolic. Each mode of organizing knowledge is dominant through a specific developmental phase, but is nevertheless present and accessible throughout. So, all types of representations are present in the adult mind and are part of his cognitive performances. • Symbolic knowledge. • Iconic knowledge is based on visual structures and recognition. • Enactive knowledge is constructed on motor skills, such as manipulating objects, riding a bicycle, etc. Enactive representations are acquired by doing. • The notion of motor skills and enactive knowledge suggests the existence of corresponding expectations that are not expressed in symbolic terms (and not only not linguistically expressed). • These expectations are related to the acquired skills or motor habits. • (17) Merleau-Ponty, 1946 for instance describes a woman who uses to wear a hat with a long feather: when passing through a door the woman will precisely lean her head in order to preserve her hat. Such a movement indicates the presence of expectations about the relationship between the dimension of the door and the dimension of the hat of which the woman is not necessarily aware. • Another example is represented by the phenomenon called “Aristotle’s illusion” which exemplifies the effects of expectations based on motor habits and the consequence of these motor habits through learning in the form of long training. • Aristotle illusion seems to be related to some form of mastery and apprenticeship of motor actions and relative perceptual consequences. The illusion is in fact related to the motor possibilities of the fingers of the subject and disappears after the subject has followed a long training and he has acquired new motor skills with his fingers. In seems thus plausible, in the light of some experiments conducted by Benedetti, to suggest that the motor skills of the fingers are relevant for Aristotle’s illusion to occur and to disappear.

41. (18) The phenomenon described as Aristotle’s illusion presents the following characteristics: if one crosses two adjacent fingers one over the other and then touches with the two crossed fingertips a small ball, one will have the feeling of touching two balls. Benedetti, 1985 points out that we are so accustomed to feeling one single object between the fingers, that feeling two objects with crossed fingers provokes surprise. • Aristotle’s illusion is one of the oldest observations about perception; in fact, the phenomenon was first described in Aristotle’s Metaphysics and On Dreaming. Successively, it was analyzed at the end of the XIX century and at the beginning of the XXth (see Ponzo, 1910; Tastevin, 1937) and finally by Benedetti, 1991, 1985, 1985, 1988, 1988, 1990. Aristotle’s illusion is also taken into account by Merleau-Ponty, 1945. • A variant of Aristotle’s illusion consists in the two crossed fingers touching one’s nose, giving rise to the impression of perceiving two noses. The phenomenon is not only restricted to the fingertips, but has also been described at the level of lips, tongue, face, scrotum and ears (see Ponzo, 1910; Tastevin, 1937): when the skin is displaced from its resting position, and a small ball is touched with the displaced skin, the perception of a double ball arises. • Another variant is obtained without crossing fingers, by displacing the cutaneous surface of the fingertips Benedetti, 1985. The fingertips are pressed against each other by the aid of two devices placed laterally to each finger (in this case, the third and fourth fingers). A plastic sphere is pressed against the fingers and subjects are asked whether they perceived one or two touches. This condition provokes the occurrence of Aristotle’s illusion. This finding is in agreement with the fact that it is possible to evoke tactile diplopia even at other body sites, through skin displacement. Nevertheless, the occurrence of Aristotle’s illusion seems also to be connected to the range of action of the fingers. • A different form of the phenomenon is described in 1855 by Czermak as inversion of the sensation when the fingers are crossed (see Ponzo, 1910; Tastevin, 1937): if one touches with crossed fingers an object which presents a sharp point on one side and a convex surface on the other, then one perceives the sharp point in the location where the convex surface is and viceversa. • More recently, the phenomenon has been investigated by Benedetti, 1985, 1986 who has described Aristotle’s illusion as a form of somesthetic or tactile diplopia. The doubling of the object perceived with crossed fingers reminds in fact the doubling of a visual image. Even if an analogous of the Aristotle’s phenomenon exists for the visual system, the haptic modality presents the specificity (as previously stated even in the case of the SWI) that two types of receptors are involved: superficial, tactile receptors and deep kinesthetic receptors (which is characteristic of the haptic touch).

42. It has been proposed that the lack of unity of the object perceived with crossed fingers is due to the fact that the perception with crossed fingers is not normal Husserl, 1990. Original work published 1952 and that the position with crossed fingers is not part of the normal bodily motor activities Merleau-Ponty, 1945. • According to Merleau-Ponty, the body schema is composed of the familiar motor skills or motor habits of the body. Since skills or habits have both motor and perceptual properties, the body schema is composed of both the motor and perceptual possibilities of the body. The acquisition of a new motor skill or habit is equivalent to the reorganization of the body schema. • Crossing the fingers, for instance, is an artificial movement which goes beyond the motor possibilities of the fingers. For this reason, the body schema is not able to comprise the crossed fingers as one organ directed to one and the same motor project or intention. Thus, the crossed fingers act separately and give rise to separate sensations that cannot be unified in one percept. The existence of motor habits of the fingers can be considered as responsible for the occurrence of Aristotle’s illusion, that is, for the experience of perceiving two objects instead of one when the object is sensed with two crossed fingers. • Tastevin, 1937 has provided an explanation for the occurrence of the Aristotle’s illusion which is based on the activity of the neuromuscular apparatus: the two crossed fingers are perceived to be at the position they would achieve with voluntary muscular effort; beyond that limit, the neuromuscular apparatus does not provide any further information. When the fingers are passively crossed in an artificial position (beyond the limit of the voluntary movement) and stimulated, the sensation of the stimulus is referred back to the limiting position. Thus, the spatial location of the stimulus is perceived in the natural limit position.

43. The experiments recently conducted by Benedetti indicate that it is not simply the distance from a normal position that provokes the illusion, but the existence of skills with uncrossed fingers (the normal position) that are no more valid with crossed fingers (the anomalous position). The perception with crossed fingers is thus referred back to the position with uncrossed fingers, which is the normal position and the position for which the subject has developed perceptual and motor skills. In fact, Aristotle’s illusion disappears following suitable training with crossed fingers in association with the acquisition of new motor and perceptual skills. • In a first experiment, Benedetti, 1985 has tested the hypothesis that tactile information with crossed fingers is processed as if the fingers were not crossed. Subjects are asked to identify the position of a small ball. The position is expressed as the angle between the ball and a sharp point which is equally in contact. In the uncrossed condition the third finger is in contact with the sharp point; the sharp point is placed at the center of a circle and the ball is placed at 0° at the right of the sharp point. In the crossed condition the fourth finger is in contact with the sharp point and the third finger the ball, which is still in the same position, even if the subjects are informed that the ball may assume different positions. In the uncrossed position the ball is judged to be at an average angle of 3° with the point; with the third finger crossed over the fourth, the perceived angle increases to 96°; with the third finger crossed under, the perceived angle decreases to -115°. Both 96° and -115° values are located on the left of the fourth finger touching the point, even if in the crossed position the third finger is on the right of the fourth one. • Thus, when the fingers are crossed, tactile spatial information seems to be processed as if fingers were uncrossed (third finger on the left of the fourth one). • In addition, a difference is noticed between the situation with the third finger crossed over the fourth finger and the situation with the third finger crossed under. When the third finger is crossed over, the ball is perceived to be above the sharp point in contact with the fourth finger; when the third finger is crossed under, the ball is perceived below the sharp point and the fingers are perceived as uncrossed. In fact, when the third finger is under the fourth, the third finger is referred to a position which is also lower than the fourth finger.

44. A second experiment is directed to test the second part of the hypothesis emitted by Tastevin, that is, beyond certain limits the perceived location of tactile stimuli does not vary. • [Benedetti, 1985] assumes that the limit is not the limit of the voluntary movement; in fact, the illusion occurs even when the fingers are crossed voluntarily. Since the sensation with crossed fingers is referred back to the position with uncrossed fingers, the individuated limit is the limit of crossing: the point at which the transition between the position with uncrossed fingers and the position with crossed fingers occurs (with the hand in the position in which the two fingers are aligned with one finger under the other). • Tactile sensations with crossed fingers are referred to two points (96° and -115°); these points are assumed to represent the limits of the functional range of action of the fingers: the spatial excursion of the fingers beyond which the perceived location of tactile events does not vary. 96° is nearer to the objective limit of crossing of the fingers (which is 90°). The difference can be explained by the fact that the movement of the third finger under the fourth is more limited, thus, the perceived location of tactile stimuli will become invariant farther from the objective limit of the crossing. • The second experiment makes use of a different apparatus than the first one (the 0° is on the left, while in the first experiment it was on the right; the range of normal position is between 90° and -90°; the range with the third finger crossed over the fourth is between 90° and 180°; for the third finger crossed under is between -180° and -90°), so that the limits are 84° (180° - 96°) and 65° (180° - 115°) and saturation of tactile information (no variations in the perceived position) is expected at these points. The fourth finger of the subjects is immobilized and put in contact with a sharp point and the third finger is again passively moved over and under the fourth one and in contact with a small ball. The results seem to confirm the expected saturation effect: tactile sensations with crossed fingers are perceived at 80° and -70°. Within this functional range of action the tactile spatial sensation follows and reproduces almost exactly the effective spatial position of the fingers; beyond the indicated values, the experience does not change.

45. The experiments by Benedetti show that the perception of tactile stimuli with crossed fingers is referred to the perception of tactile stimuli with uncrossed fingers, that is, to the normal situation and the normal position of the fingers. A given pair of fingers has a functional range of action within which spatial perception is correct and beyond which the location of tactile stimuli is perceived incorrectly. The objects touched with crossed fingers are perceived as having the spatial properties of the extreme limits of the range of action of the fingers. What mediates the perception of the object with crossed fingers is thus something related to the range of action of the fingers, but not the representation of the position of the fingers, which is not altered by the fact of crossing (the subject of the illusion describes his fingers as crossed). Aristotle’s illusion is thus related to a form of knowledge which is based on the acquisition of skills and not on the existence of explicit representations of the position of the body parts (fingers). • Aristotle’s illusion testifies the role of sensorimotor learning. • Benedetti also excludes the possibility that Aristotle’s illusion depends on the perception of the position of the fingers. The perceived location of the tactile stimulus in fact does not co-vary with the perceived position of the fingers [Benedetti, 1988]. When the two perceptions are compared, it appears that whatever the position of the crossed fingers (specifically 0°, 45° and 90° are tested for the third finger being crossed over the fourth), the perceived position of the stimulus (a ball, whose position, as in the previously described experiments, is plotted against the position of a sharp point stimulus applied to the fourth finger) remains unvaried (when subjects are asked to place the third finger at 0° they place it at -5° and perceive the ball to be located at -4°; for the request of placing the finger at 45°, the finger is placed at 40° and the ball perceived at 3°; for the finger to be placed at 90°, it is really positioned at 87° and the ball is perceived at -10°). In this experiment the fingers are crossed voluntarily and the third finger is charged with a little weight in order to make it necessary for it to exert a continuous muscular effort to maintain the position. Thus, Aristotle’s illusion occurs both in the passive and active condition of crossing fingers and position sense has no effect on the perceived position of the stimulus, at least with crossed fingers.

46. Another experiment by Benedetti, 1991 investigates the effects of motor-perceptual learning on the disappearance of Aristotle’s illusion. In fact, Benedetti, 1991 has tested whether or not the individuated range of action of the fingers can be modified by a long-lasting crossing. The subjects crossed the third finger over the second and were asked to go back to their daily lives with crossed fingers for variable periods, from 60 to 183 days (with short periods of rest with uncrossed fingers); some of the subjects also underwent special training. Spatial perception with crossed and uncrossed fingers and the perception of the position of the fingers were tested at intervals in the modality adopted for the experiments described in Benedetti, 1985 and Benedetti, 1988. Again, since the actual position of the ball is at 0°, an error greater than 90° indicates that the ball is perceived as if the fingers were uncrossed, while an error smaller than 90° indicates that the ball is perceived on the correct side. A decrease of the error from 90° is observed for all subjects. Hence, all the subjects learned to perceive the ball on the correct side with the second and third finger. A test performed over the non-trained third and fourth finger always elicited perception as if the fingers were uncrossed. • The results indicate that Aristotle’s illusion disappears after a period of training with crossed fingers. • Even when perception with crossed fingers became correct, perception with uncrossed fingers still remained correct too. In addition, no saturation effect is observed for the trained fingers, but there is linear co-variation between the effective position and the perceived position of the stimulus. The last observations indicate that no adaptation has occurred, but there has been an extension of the range of action of the fingers, which now includes the crossed position. • The observed perceptual modifications (extension of the range within which perception varies following the variations of the stimuli) are accompanied by corresponding motor modifications. The percentage of correct movements (the number of times a stimulus is rejoined correctly) greatly improves in correspondence with the dropping of perceptual errors. Thus motor and perceptual performances show a good correspondence. • The extension of the range of action suggests the existence of plastic changes: the touch system seems to develop according to the pattern of hand exploration and is not to be rigidly pre-determined. If the fingers are located in new and unusual positions, the touch system develops in a new and unusual way. In this sense, the acquisition of a new perceptual competence implies the acquisition of a new motor capability.

47. Following the repartition of forms of knowledge provided by Bruner, we can consider the expectations like the one in (17) and (18) that are based on connections between perception and action and that are acquired in virtue of the acquisition of motor skills and habits as enactive expectations. • Enactive expectations can be evinced from the existence of correct action plans and some forms of illusions. • When an action plan fails the subject can be surprised or can modify its perceptual outcome in order to fit with his own expectations; the second option gives rise to illusions or in general special perceptual effects.

48. A reaction of surprise is associated with Aristotle’s illusions, as with the others illusions. • Illusions have the same function than surprise. As a matter of fact, it is possible to demonstrate that the reaction of surprise is directly or indirectly associated with illusions: illusions we are immediately aware of give directly rise to a reaction of surprise, while illusions we are not immediately aware of provoke a reaction of surprise only indirectly, when additional information makes the subject aware of being victim of an illusion or of the presence of discrepant information. • Hence it is still in virtue of the reaction of surprise that the subject can become aware of the presence of frustrated expectations. • The reaction of surprise which arises when two objects are perceived instead of one might be provoked by the conflict between the visual and the tactile information or between the tactile sensation which is presently experienced and the knowledge that only one object is really sensed with the two fingers. In both cases surprise arises in response to a violation of the coherence of the perceptual experience.

49. In addition to learnt motor skills and to the acquisition of motor habits by use, other cases can be described in which perceptual anticipations, laws of perception, sensorimotor matching activities and intersensory connections produce forms of surprise or illusions and indicate the presence of expectations. • In all these cases expectations operate at an automatic, sensorimotor level. • They can be considered as forms of representations (sensorimotor representations) or they can be considered the result of the existence of forms of connections between perceptual systems or motor and perceptual systems with no need to make recourse to the notion of representation (sensorimotor connections, laws of perception and other forms of connections between perceptual systems such as intersensory connections) • In any case, these expectations do not make reference to some form of belief or theory but are based on the existence of connections between perceptual experiences or between motor actions and perceptual experiences. Selection and learning from experience are at the origin of this kind of knowledge and relative expectations Stein & Meredith, 1993. • They can be the result of long term or short term apprenticeship (as in the case of the acquisition of perceptual and/or motor habits, perceptual and/or motor habits motor skills), statistic learning, associative links, probability of activation, or even be the result of connections, associations and laws produced by selection. • They can be general and largely shared, as it is the case for symbolic commonsense knowledge, or be specific and local, being the fruit of individual learning and of the existence of motor and perceptual conditions that are specific to some groups of individuals. • Time for the acquisition of new knowledge varies with the type of activity but it is plausible that the acquisition of new perceptual and motor connections and the eventual modification of early acquired and shared connections will require more time for taking place, if even it can possibly take place. • It seems possible, in any case, that suitable training plays an effect in producing new connections as those described in the second form of commonsense knowledge, thank to the existence of neural plasticity Bach-y-Rita, 1982; Benedetti, 1991. • Finally, the interaction with the virtual world can be the occasion for the user to acquire new symbolic, iconic and enactive knowledge, and, through suitable training, to develop new perceptual and motor connections, hence to give rise to a large set of new expectations. It is important to understand which are the most suitable instruments for producing new learning acquisition at different levels when this is desirable and how to exploit the different acquisitions, for instance in order to supply to VR systems limitations.

50. Sensorimotor connections: • Perceptual anticipation • Illusions are according to Berthoz the result of the projective nature of perception: not only they show that the perceptual system is much more than a system for the passive reception of stimuli, but they can reveal which type of hypothesis the brain has been emitting. • (19) Kohnstamm phenomenon (involuntary arm abduction that follows sustained isometric contraction of the deltoid and supraspinatus muscles, usually performed by pushing the upper extremity forcibly and against an immovable vertical surface - say a wall - while standing closely beside it). It is explained as a reaction of the postural muscles due to the fact that during the pushing phase the brain has adapted to one condition where immobility is associated with constant muscular effort; when the arm is released from the contact with the wall the brain continues to apply the same force since, in virtue of the apprenticeship made, the brain has the illusion that the obstacle is still where it used to be. • Berthoz hence asserts that perception in general is a projective activity and considers perceptual projections as forms of expectation or anticipation of the future: hypotheses are continuously emitted by the brain regarding the state of the world and confronted with the actual stimulation. In particular, the brain possesses mechanisms for anticipating the perceptual consequences of action and the perceptual results of the changes that the subject has provoked in the environmental condition.