A. Bozzini, S.Siccardi

1. Towards a conceptual replication of the Robertson-Roy experiments with mediums. A preliminary study with a psychometric subject. A. Bozzini, S.Siccardi

2. The Robertson-Roy Protocol (RRP): main goals • Terms used in the present discussion: • medium or psychic (interchangeable): whoever claims he can tell something without "normally" knowing it • recipient: the person to whom the medium addresses a number of statements (that constitute a set, or a questionnaire) • non-recipient or control: somebody that judges a set as if it had been addressed to him/her • The Protocol addresses some points: • what is the part played by non-paranormal factors? E.g.: • body language and appearance of the participant • verbal responses of the recipient to the medium statements • participants believing themselves to be / not to be the recipient • participants being / not being the recipient • participants not knowing whether they are the recipient • statements made by mediums are so general that they can be accepted by everyone?

3. The Robertson-Roy Protocol (RRP): main features • The psychic does not meet the participants until the end of the experiment • The statements are recorded, one piece of information on each line, one set of statements comprising of the statements provided to one recipient • The sets of statements are copied and given both to the recipient and to a set of people (non-recipients or controls) to whom they were not addressed • The whole RRP considers 9 types of subjects (in green those applicable in our replication):

4. Our replication: state-of-the-art • We started discussing the RRP and its applicability in our situation • We tried to plan a set of experiments that would last a long time • We conduced a few informal and formal experiments • We carried out the computations and compared numerical results to first hand impressions • We are interested in understanding factors that can favour or inhibit the psi, not just to demonstrate that PSI exists • We found no evidence of PSI, but have some methodological points to discuss for future research

5. The RRP basic computations: 1) without weighting • whenever we give a set to somebody to evaluate it, we define f = (Nr. of accepted statements in a set) / (Total Nr. of statements of the set) • f is computed for recipients and non-recipients for all the sets of statements of a group of experiments • The median of the non-recipients' fs is computed • The recipients' fs are compared with the median; supposing that we have n recipients and that r are above the median, the probability is computed with the binomial formula: P = (nr) (1/2)r(1/2)n-r • The original study of RR with 10 mediums, 44 recipients and 407 non recipients found a P as small as 5.37 x 10-11

6. The RRP basic computations: 2) weighting • Weighting is used to evaluate the relevance of statements and sets of statements • No judgements of investigators, recipients, non-recipients etc. • Giving m statements to n non-recipients, the weight of statement i is: wi = 1 – (Nr. of non-recipients that accepted i)/n • Accordingly, one can compute and compare: • the weight of the r statements accepted by the recipient: W=Si=1r wi and the mean weight W/r • the total weight of the m statements of the set: Ws=Si=1m wi and the mean weight Ws/m • the total weight of the rj statements accepted by the j-th non-recipient: Wj=Si=1 rj wi and the mean weight Wj/ rj

7. The RRP main steps – and comments – in our realization Room 1: people, experimenters Room 2: psychic, experimenters some "tokens" are collected and sent to the psychic room in sealed envelopes a subset of the envelopes is opened; the psychic "reads" the tokens the psychic's readings are tape recorded; then the experimenters listen to the tape and write down a list of short/simple sentences. The list(s) is (are) printed and sent back to room 1 all those who sent a "token" judge the whole set of sentences, without knowing which, if any, were written for them Data are stored for statistical analysis – the formal experiment ends "Token" = anything the psychic can use to put himself in contact with the recipient People can meet the psychic for an informal discussion The same sentences are later submitted to "external" controls

8. Our experiments: the psychic, the RRP realization, the physical environment, etc. • Our psychic: man, age 60, non professional psychic, psychometrist • 5 sessions, 3 of which were formal ones (+ preliminary sessions to understand how he is used to working)

9. The rules to obtain simple sentences (1) • The statements are recorded, one piece of information on each line... • Rules to divide a complex sentence, so that a yes/no or true/false answer is appropriate: • complex sentences should be divided into simpler ones • e.g. if the psychic says that the object he is holding "belongs to a woman with dyed blonde hair", we get the sentences: • the obj belongs to a woman • the obj belongs to a woman with blonde hair • the obj belongs to a woman with dyed blonde hair • the same when the psychic says a generic sentence followed by a more precise one, eg.: • from "the obj's owner suffers from breathing problems, maybe due to allergy" we get: • "the obj's owner suffers from breathing problems" + the original complete sentence

10. The rules to obtain simple sentences (2) • Sometimes the psychic says vague statements about his perceptions of an object, a part of the recipient's body, etc., like "I feel something regarding your liver". Then we use the standard formula: • in your life you experienced facts, situations, moods, in consequence of which your liver has become relevant for you • When the psychic reports about something of unreal, fictious, symbolic, etc., like "I see a heart upside down", we use the other standard formula: • a heart upside down is or has been important for you as a metaphor or in some other sense • Of course we write all sentences so that no sexual inference can be made (using "he/she" etc. every time) • We feel that the process of writing down the sentence is a crucial one, and that it could introduce a positive or (more often in our experience) negative bias: "adding or moving a comma we experimenters could change a sentence meaning"

11. Sorting sentences: what we are actually measuring • As in many other tasks, there is a basic choice to make: • the basic piece of information that we want to measure is a single sentence or • the basic piece of information is a set • As we chose the single sentence, we wanted to destroy the set coherence, so we sorted the sentences (e.g. in alphabetical order) • If the information that the psychic can obtain is of a synthetic type, we could destroy or diminish its significance • So we think that this is a pivotal point to understand something about the PSI mechanism: • nobody can "recognize" him/herself in a section of a questionaire • each single sentence must be evaluated apart

12. The sheep-goat scale measurement • One of the most often reported points in parapsychology is that the attitude toward PSI would influence the results • We used the 26 items Tobacyk Revised Paranormal Belief Scale, without adding nor removing any items, and with a 7 points likert scale • It doesn't address the psi belief in a strict sense, but has a number of subscales, including: witchcraft, psi, traditional religious beliefs, spiritualism, precognition and superstition • It seemed appropriate as the expectancy one has when consulting a psychic should depend on some of the quoted subscales

13. Other measurements we considered • The anagraphical data, i.e. age, sex and education: we had a lot of internal discussion about the opportunity of having or avoiding • homogeneous groups of recipients • a mix of recipients so that they would reflect the general population • a group of controls that reflects the recipient group composition • a group of controls very different from the recipient group, etc. • The attendants' mood. • When in public conferences with many more attendants than recipients and controls • We would like to measure the general public attitude, nervousness, scepticism, etc. in order to see if it correlates with the experimental results • We were able to do it just once (it is quite boring for participants) • The recipients motivation: if a recipient is really looking for "an answer" about some personal problems, is it easier for the psychic to establish a contact? • The experiment time, in order to check sidereal time influence in the long run

14. The applicability likert: motivation and examples • We considered that some piece of information can be general in itself, but specific when it excedes a medium degree • Example (for an experiment in Italy): • "the recipient can speak Dutch" is per se a specific sentence • "the recipient can speak English" is not (more than 50% of the average population can – more or less) • but maybe the recipient has a specific relationship with the English language, e.g. he is a professional translator, or a native English speaker • We decided to use a second copy of the sentences, asking people to mark on a 1 to 5 scale "how much" each sentence is applicable • Only the maximum score is considered as an acceptance, and the statistics are repeated • By the way, we obtain a measure of the attention of the subjects, considering the coherence between this questionnaire and the previous one

15. The emotional valence likert: motivation and examples • Another possibility is that the psychic picks a topic that is generic, but that is emotionally relevant for the recipient. • It can even happen that a sentence is false, but that it hits a relevant point for the recipient. Example: • "the recipient has travelled to China" • he never has, but he is passionate about Chinese culture and has dreamed of going there since he was a child • Of course, we are not trying to save anyway what the psychic is saying, we are interested in what can catch his attention • We decided to use a third copy of the sentences, asking people to mark on a 1 to 5 scale "how emotional" each sentence is • Only the maximum score is considered as an acceptance, and the statistics are repeated

16. Some more data analysis: significance • Since when we started to plan our research, we wondered "how many" sessions, recipients and controls should we gather in order to attain significance • We tried two different ways to proceed from binomial probabilities to significance: • referring to the median of non-recipients, we built a 2x2 table and then used a chi square test with 1 degree of freedom • randomization

17. Formal and informal sessions: basic results • Informal session: • each of the 5 recipients judged all the 5 sets, although they had listened to the psychic, so we had 20 "controls" • all the recipients were above the median both weighted and unweighted • binomial P=0.03125; chi square = 0.041 • we used oly the main questionnaire (yes/no) • Formal sessions:

18. Formal sessions: cumulative basic results • Putting together the 3 formal sessions, we have 9 recipients and 36 controls (the recipients themselves judging the statements that were not written for them) • Of course, to make sense, we have to add a possibly large number of other controls; we call them "external" because they were not present at the experiments and they obviously know that they are not the recipients

19. Formal experiments: the impact of external controls • We consider 3 group of external controls • a set of 4, that were recruited ad hoc for the first office session; they completed all the formats (yes/no and likert) and generally matched the recipients • a set of 4, that completed all the formats for the second office session; they again generally matched the recipients • a set of 4, that completed one format for each of the formal sessions; this group did not match some of the recipients' characteristics

20. Results obtained adding external controls Individual sessions: Cumulative results:

21. Differences in the audience, motivation, belief in psi, etc. Correlation with data • Factors of the sheep-goat questionnaire and the s-g score as predictors of the number of statements that a participant accepts • On one occasion, we tried to measure the audience mood and expectations • We are planning measures of participants' motivation for: 1) echology of the situation 2) response quality evaluation

22. Significance through data randomization • A standard way is to build the NxN table of recipients/sets • The sum of the principal diagonal elements represents the actual outcome for the recipients • We permutate the columns (or the rows) in all the possible ways, and compare the principal diagonal elements sums • The probability that the outcome is due to chance is obtained by the percentage of lower results • But, as we have a few recipients in each session, usually we get a 3x3 table: the method is not so useful f fractions or weights computed as requested by the RRP

23. Significance through data randomization - continued Still another way of significance computing is to consider all the sets of N that is possible to extract from the whole set of controls and compare their distribution with the recipients Does not take into account how many controls have been used to compute the median Binomial p Does not take into account how much the recipients are near the median or extreme chi square Uses all the available information, but can become heavy to compute with increasing controls' number randomization

24. Our experiments significance using randomization Individual sessions with external controls: We did not use the method for cumulative results: more powerful algorithms are needed.

25. Some more data analysis: exceptional hits or "star" hits • Question: do psychics "often" get a "little more" information than expected by chance or "seldom" a "lot"? • In other terms: however we measure it, does it follow • a normal distribution, with a higher mean value than expected by chance or • a different (e.g. a heavy tail) distribution? • Note that some human characteristics and behaviour do follow heavy tail distributions • wealth (Pareto distribution) • citation index of scientific publications • degree of scale-free networks • visits to websites

26. Star hits in our experiments A definition of star hit for our experiments is not trivial. We used the data of the applicability likert and tried the formula: n+1 S (vik – 1) (vlk – 1) ( 1 - ___________ ) . ______ 4 n 4 Plk = i = 1, i<>l vlk (vik) is the score (from 1 to 5) assigned by the l-th (i-th) non recipient to the k-th statement; n is the number of non-recipients, in the sum is included the recipient; if N is the number of statements, we obtain n x N values 0 <= Plk <= 1, the chance distribution

27. Star hits in our experiments - continued For recipients we use the analogous formula: n S (vik – 1) (vk – 1) ( 1 - ___________ ) . ______ 4 n 4 i = 1 Pk = vk is the score (from 1 to 5) assigned by the recipient to the k-th statement; the other variables are the same as in the previous formula if N is the number of statements we obtain N values 0 <= Pk <= 1, the psychic distribution we put together all the statements of all the sessions and obtain two cumulated distributions to compare

28. Star hits: the distributions

29. Confusion data in our experiments • We use Confusion Matrices to investigate similarity between the fractions of accepted statements and their weight, and the 3 types of questionnaires • In general, numbers and weights seem to show an analogous structure within a questionnaire, while questionnaire types seem to differ from each other • More data are needed to draw even the most basic hypothesis

30. Some more details about confusion data • Confusion matrices (CM) record how easily two targets are confused, a larger off-diagonal entry reflecting greater confusion between two data • Some characteristics of CMs are of standard use, e.g. asymmetry, indicating whether two targets are interchangeable or one prevails • Probably the most common use of CMs in psychology is comparison of confusion among the senses Structure similarity

31. Some more data analysis: evaluation of answers coherence • The yes/no questionnaire and the likert for applicance should be in agreement • We compared the results to evaluate the participants' coherence and attention during compilation • scores 3,4,5 in the likert compared with "yes", 1 and 2 with "no" • How to help participants during compilation?

32. When the recipients meet the psychic: face to face behaviour vs. quantitative data • At the end of the sessions, the psychic meets the participants: • he "reads" objects tht had been collected but not used for the formal session, if any • we read from our notes the statements to the recipients • From recipients' reactions we often expect some good hits, but when we look at their written answers we don't find many • Examples: • "you are interested in a research about water power" • the recipient works at a pool for rehabilitation – uses/studies how water can help to heal • answers: "yes", "almost true", "associated to a weak emotion" • "you are interested in a particular specialization of medicine" • the recipient is preparing statistics for medicine • answers: "no", "almost true", "associated to a strong, specific emotion" • Possible explanations: • our style in writing statements is poor • written "too cool", spoken "too warm" • participants are looking for knowledege/advice on some hot topic and don't care about other matters, expecially written, however true • Conversation Analysis might shed some light

33. Is the RRP applicable to other kinds of psychics? • Mediums vs. psychics • A lady claiming an "etheroscopy" gift • Astrologers and others • Non psychics • Random generated sentences

34. Acknowledgements • We are grateful to • AISM (Associazione Italiana Scientifica di Metapsichica) for their support; • to Mr. E. L., our psychic, for his patience; • to dr. Massimo Biondi for many suggestions and hints • Any comments and suggestions will be greatly appreciated – please email them to info@progettopsi.it