1 / 18

The Art (and Science) of Defining a Construct

The Art (and Science) of Defining a Construct. Tamara Shore Dr. Robin Anderson Center for Assessment and Research Studies James Madison University. What is a Construct?. An Attribute of people Motivation Depression Sociocultural Competency Qualitative Attributes or Structures

zaria
Download Presentation

The Art (and Science) of Defining a Construct

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Art (and Science) of Defining a Construct Tamara Shore Dr. Robin Anderson Center for Assessment and Research Studies James Madison University

  2. What is a Construct? • An Attribute of people • Motivation • Depression • Sociocultural Competency • Qualitative Attributes or Structures • People have it or they don’t (Amnesia) • Quantitative Attribute • People have it to some degree (Anxiety) • Inferences are made about the attribute (Cronbach & Meehl, 1955)

  3. Validity “The assessment instrument has content validity to the degree that it taps the targeted construct and facilitates valid...judgments.” (Haynes, Richard, & Kubany, 1995)

  4. Validation of the Construct • Review existing research in literature • Formulate a theory of the construct • Position the construct with related constructs (Nomological network) • Generate hypotheses about construct relationships • Test, interpret and refine (Cronbach & Meehl, 1955)

  5. Three Aspects of Construct Validation Substantive Stage: Definition of the Theoretical and Empirical Domains of the Construct Structural Stage: Internal Relations among Observed Variables External Stage: Relations Among Constructs (convergent & divergent) (Benson, 1998)

  6. Begin to Define the Construct (Substantive Stage) • Theoretical • Cultural Intelligence- the ability to construct innovative ways of conceptualizing, data gathering, and operating, in a new culture • General cultural competency - the meta-cultural perspective • Styles of Communication (verbal and nonverbal) • Conflict styles • Prejudice and power • Intercultural Communication • Intercultural Competence (INCA Motivation, Skills/Knowledge, & Behavior) • Empirical • How the construct is operationalized

  7. Challenges to the Process of Defining a Construct Do you have the same definition? Are enough related constructs included to adequately represent the full domain? Do you operationalize the construct in the same way as others do? Have you teased apart related constructs that should be included from those that should NOT be included?

  8. Model of Construct • Specify content knowledge • How detailed? • Specify procedural knowledge • Cognitive processes, communication skills • Specify pathways of development from novice to expert on all facets of the construct.

  9. Our current working definition • Socio-cultural Competence: An individual's ability to discern equitable and inequitable circumstances for persons involved in any situation, correctly identify environmentally and culturally influenced behavior and responses by all, and communicate effectively using all available data.

  10. Construct Explication • Compile a list of specific behaviors, beliefs, and attitudes that demonstrate the presence of the construct • Also compile a list of behaviors, beliefs, and attitudes that do NOT indicate the presence of the construct • Use these lists to determine the nature and content of the test items.

  11. I. Cognitive Skills Critical Thinking Clarification and analysis of information supporting own position Identification and exploration of information supporting multiple viewpoints.   Reflective Thinking Involves the assumption that knowledge is uncertain and can be created in the internal process of constructing meaning Dialectical Thinking Systems thinking Comparative Thinking Self – awareness i.     Knowledge of own cultural programming                ii.     Knowledge of beliefs and values Awareness of self in relation to others Our List of Characteristics

  12. Our List of Characteristics II. Intercultural Competency Attitudes Valuing of all cultures Curiosity and discovery Interaction posture (non-judgmental) Role behavior Ethnocentrism – ethnorelativism Willingness to self-correct

  13. Our List of Characteristics III.Procedural Skills Communication Verbalization Mindful Listening Face management Non-verbalized communication – (un)conscious Identification & General Knowledge of: Conflict Styles Value PatternsValue judgments result from ignorance of how our culture shapes our values - errors result from not using a systematic and integrated method of coming to value conclusions or unexamined value principles Cultural Adaptation Prejudice and Power(Mindsets and Stereotypes)

  14. Our List of Characteristics IV. Motivation Mindfulness(as a bridging concept between cognition and affect - a self-conscious concern for and evaluation of one’s thinking process) Open - mindedness Tolerance of Ambiguity Suspension of judgment Respectfulness Empathy

  15. Wolcott’s Critical Thinking Rubric (Novice to Expert)

  16. Our Conceptual Map of the Domain

  17. Where to go from here… Allow a lot more time to define your construct than you were planning Examine existing instruments to determine their ability to fully assess the domain If no instrument fully assesses the domain and you must develop one, plan enough time for defining the construct. Review the Standards for Educational and Psychological Testing Have fun with the process!

  18. References APA Board of Educational Affairs Task Force on Psychology Major Competencies, APA Guidelines for the Undergraduate Psychology Major. www.apa.org/ed/resources.html, retrieved 4/20/2008. Bennett, J. & Salonen, R. (2007). Intercultural communication and the new american campus. Change March/April, 46-50. Benson, J. (1998). Developing a strong program of construct validation: A test anxiety example. Educational Measurement: Issues and Practice, 17, 10-17. Earley, P.C, & Ang, S. (2003). Cultural intelligence: Individual interactions across cultures. Educational Measurement: Issues and Practice, 17, 10-17. Ferrara, S., & DeMauro, G.E., (2006). Standardized assessment of individual achievement in K-12. In R.L. Brennan (Ed.), Educational Measurement (4th ed., pp. 579-621). Westport, CT: American Council on Education/Praeger. Ferrara, S. (2006). Toward a psychology of large-scale educational achievement testing: Some features and capabilities. Educational Measurement: Issues and Practice, 4, 2- 5. Gorin, J. (2006). Test design with cognition in mind. Educational Measurement: Issues and Practice, 4, 21-35. Haynes, S.N., Richard, D.C.S., & Kubany, E.S. 1995. Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238-247. Yershova, Y., Dejaeghere, J., & Mestenhauser, J. (2000). Thinking not as usual: Adding the intercultural perspective. Journal of Studies in International Education, 4(1), 39-78.

More Related