Irla Lee Zimmerman Violette G. Steiner Roberta Evatt Pond

1. Irla Lee ZimmermanViolette G. Steiner Roberta Evatt Pond

2. Agenda • Characteristics of PLS4 • Reliability Evidence • Validity Evidence

3. Characteristics • Purpose: Identify children who have a language disorder or delay • Age: From Birth to 6 years, 11 months • Administration: Individually administered Who can administer, score and interpret the test?

4. Test Content - Core Subscales • Auditory Comprehension (61 items): Evaluates how much language a child understands.

5. Test Content - Core Subscales 2. Expressive Communication (68 items): Determine how well a child communicates with others.

6. Test Content - Supplemental Assessments

7. Material • Examiner’s Manual • Picture Manual • Record Form • Manipulatives Administration Time • Age < 1 year 20-40 mins • 1 year < Age <4 years 30-40 mins • 4 years < Age < 7 years 25-45 mins

8. Flowchart 0. Before Beginning the Test 1. Administer Test 2. Score Test 3. Interpret Test Scores

9. 0. Before Beginning the Test • Before you Begin: Collect relevant information regarding the child’s motor, vision, and hearing status. Some accommodations that you make will not interfere with the ability to administer the test in a standardized fashion. • Test Environment: Provide a supportive, friendly test environment. It is important that the child and the caregiver are comfortable and relaxed so that you can obtained best performance from the child. Begin with a task that that you have determined that the child will do successfully. • Caregiver’s Participation: Caregiver’s Participation is critical. - give credit for passing specific test items if the caregiver can provide examples of the child’s behavior at home - caregiver should not reword directions - caregiver should not intervene or prompt the child unless you ask for assistance. • Handling Materials: Place in front of the child only those materials that you will need for the task in hand. After using the manipulatives to administer a task, immediately return the objects to the container.

10. Test Administration Steps: 1. Determine Chronological Age 2. Determine Starting Point • Start points for children with mildly- or moderately-impaired communication • Start points for children with severely- or profoundly-impaired communication 3. Administer Tasks • Practice Tasks are included for a number of items • Sequence: Either subscales can be administered first • Repeating Directions: It is acceptable to repeat the directions or the stimulus for the child only once. • Additional Cues and Support: Do not provide any additional cues or support, other than what is mentioned in the manual.

11. Role Play with David Jr. Auditory Comprehension (AC) 21. Identifies photographs of familiar objects. (18-23 months) 32. Follows two step, related commands without cues a. Open the box and give me the bear. b. Get the cup and give the bear a drink. c. Get the cup and give the cup to me. (30-35 months) 61. These pictures show different times of the year, like fall and spring. Point to the picture that shows: a. winter b. fall c. spring (78-83 months)

12. Role Play with David Jr. Expressive Communication (EC) 3. Responds to speaker by smiling. (birth to 2 months) 22. Uses five to ten words. (18-23 months) 31. Uses plurals (30-35 months) • Completes analogies (42-47 months) 61. Defines words. (72-77 months)

13. Test Scoring Steps: 1. Establish the Basal level 2. Establish the Ceiling Level 3. Complete the Scoring Table

14. Setting the Basal Level

15. Setting the Ceiling Level

16. Scoring Table

17. Test Score Interpretation Interpreting Test Scores: 1.Task analysis checklist 2. Profiles

18. Task Analysis Checklist

19. Profiles

20. Evidence of Reliability

21. Importanceof Reliability • A measure must be (mostly) reliable before it can be considered valid

22. Test-Retest Reliability • Sample of 218 children from the standardization sample • Ages from 2 to 5 yrs. 11 months • 117 females and 101 males • Between test interval ranged from 2 to 14 days-believed to be long enough to minimize practice effects, but not so long that the construct being measured is changed

23. Subscales (Auditory & Expressive) Total language Score Ranged between .82 and .95 Ranged between .90 and .97 Reliability Coefficients

24. Internal Consistency

25. Inter-Rater Reliability • Most of this test is scored objectively, but some questions on the Expressive Communication subscale are subject to scorer interpretation • 15 Individuals, Elementary School Special Education teachers were used in to examine inter-rater reliability • 3-Weeks to practice and learn the test

26. Inter-Rater Reliability • Correlation between raters on Expressive Communications scores was r=.99 • Very high inter-rater reliability

27. Modified Administration forSpecial Populations • It is important to note that modified administration test scores cannot be compared to standardized administration scores • Modified administration scores can be useful for describing language behaviors that a child can or cannot do (criterion referenced).

28. Accommodations • Severe Physical Impairments • Can help a child stabilize an arm for pointing (Norm Reference) • Point to each Picture and allow child to nod yes or no (Criterion Reference) • Children with Visual Impairments • Position pictures and manipulatives with the child’s visual field (Norm Referenced) • If a severe visual impairment, allow the child to pick up and examine manipulatives for as long as necessary (Criterion Referenced)

29. Evidence of Validity

30. Types of Evidence Provided • Test content • Response Processes • Internal Structure • Relationship with other Variables • Consequence of Testing

31. Summary of What we KnowAccording to the makers of the PLS-4 • The PLS-4 offers a through and balanced sample of language behaviors (evidence of validity based on test content) • PLS-4 measures the processes it is intended to measure, i.e., language skills (evidence of validity based on response processes) • PLS-4 demonstrates the inter-relatedness of constructs related to assessing language (evidence of validity based on internal structure) • PLS-4 scores are highly correlated with scores obtained from other valid measures of language ability and the test consistently differentiates children who are language disordered form children who are not (evidence of validity based on relationships with other variables) • PLS-4 has no known negative consequences of testing (evidence of validity based on the consequences of testing)

32. Evidence Based on Test Content Validity evidence related to test content is supported when the content area(s) being measured are generally accepted as the proposed construct (content relevance) and when the content areas are accepted to be an adequate sampling of these areas (content coverage).

33. Evidence Based on Test Content Evidence was gathered by: • An exhaustive literature review and user survey provided information about language skills that the test should address; • The test scope and sequence maps test tasks to the areas tested; • Content, bias, and task reviews were conducted to verify breadth and appropriateness of task/subitem coverage and task/subitem formats; • All tasks and subitems were reviewed by experts in the field for evidence of construct-irrelevent components and construct under-representation.

34. Evidence Based on Test Content • The evidence was presented by discussing each construct, task and subitem. The test developers explained each construct, provided a rational for its inclusion, and discussed how it is measured. • They conclude that the evidence supports the contention that the PLS-4 has both requisite content coverage and content relevance.

35. Evidence Based on response Processes • Evidence must show that the task formats elicit responses in the desired manner, hence tapping into the intended skills. • Evidence comes from an analysis of the process intended to be assess, analysis of the child’s response process, and evaluation of observers’ and/or scorers’ interpretation of behavior and/or scores.

36. Evidence Based on response Processes Methods used to collect evidence • Review of each task • Pilot-tests (analyze child responses • Tryout practice cases (for test users/scorers) • PLS-4 suggests that the evidence supports the hypothesis that the desired processes are being elicited

37. Evidence Based on Internal Structure • Evidence is provided by examining the tasks and subscales in order to determine how they relate to one another • This was done by: • Analyzing the internal consistency of the subscales for evidence of high homogeneity. • Tests indicated that the PLS-4 has highly homogenous scales • Analyzing the internal structure for correlation between the two subscales • Fairly highly correlated, indicating that the scales are unique while also having a good deal in common (the desired result)

38. Evidence Based on relations to other Variables • An examination of the relationship of test scores with variables external to the test; such as other variables, measures, and tests that are meant to measure/test the same constructs.

39. Evidence Based on relations to other Variables Convergent Evidence • PLS-4 was found to be highly correlated with both the Denver II and the PLS-3 Test-Criterion Relationships • Have not been able to test the predicative power of the test because of problems of early intervention and successful treatment

40. Evidence Based on relations to other Variables Clinical Validity Studies • Completed studies with four clinical groups, children identified as having a: language disorder, developmental delay, autism, and a hearing impairment. • In each case the PLS-4 was able to differentiate between children that were developing typically and the clinical groups.

41. Evidence Based on Consequence of Testing • To date, there is no evidence to suggest that the PLS-4 has any negative consequences for children when it is used as intended.

42. Overall Evaluation • The PLS-4 developers appear to have successfully made the case for the validity of their test in each category of evidence.