The future of assessment
Download
1 / 31

Discovering Talent and Potential - PowerPoint PPT Presentation


  • 106 Views
  • Uploaded on

The future of assessment. Dr Maretha Prinsloo. Discovering Talent and Potential. Discovering Talent and Potential. The future of assessment. Demands on assessment : dynamic complexity integrated and contextualised nature of human functioning

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Discovering Talent and Potential' - jerry


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

The future of assessment

Dr MarethaPrinsloo

Discovering Talent and Potential

Discovering Talent and Potential


The future of assessment
The future of assessment

  • Demands on assessment:

  • dynamic complexity

  • integrated and contextualised nature of human functioning

  • collective consciousness & the evolution of consciousness

  • ethical considerations

  • societal needs

  • technology

  • theoretical trends

Discovering Talent and Potential


The future of assessment1
The future of assessment

Thus much room for improvement of current assessment methodologies


The future of assessment2
The future of assessment

  • Psychometrics criticised for:

  • little innovation

  • commercial viability

  • “We have a relatively stable, mature, market consisting of a few standard assessment formats … which are liberally branded by a variety of companies. These tests barely differ in content, accuracy, or utility; they differ solely in terms of their marketing.” (Barrett, 2011)

  • Barrett regards conventional psychometrics as a

  • “pathology of science”

Discovering Talent and Potential


The future of assessment3
The future of assessment

  • Psychometrics:

  • use of limited experimental designs

  • weak quantification

  • weaker assessment methodologies

  • causal inference based on correlational models

Discovering Talent and Potential


The future of assessment4
The future of assessment

  • Research focus on:

  • the number of constructs

  • the level at which these are to be conceptualized

  • the structure of the model (single, multiple, hierarchical, facet, phases, interactive, cube, overlapping , linear-causal, quadrants…)

  • the statistical methods to be used (correlations or correlations)

  • the constructs e.g. IQ, personality and EQ

  • researched via individual differences

Discovering Talent and Potential


The future of assessment5
The future of assessment

  • The Scientific status of Psychology

  • explanatory sciences

  • descriptive sciences

  • speculative sciences

  • These fields are characterized by differences in:

  • ontology (entities / object of study)

  • epistemology (how we get to know)

  • methodology (investigative techniques)

Discovering Talent and Potential


The future of assessment6
The future of assessment

Psychology: a descriptive science with a weak rational basis and only some empirical grounding

David Freedman (1991), a mathematician, recommends a “low-tech” approach which “relies on intimate knowledge of the subject matter and a meticulous research design …”

“Simplicity after complexity” given robust design and rigour

Discovering Talent and Potential


The future of assessment7
The future of assessment

  • Why do we want to test

  • commercial viability

  • statistical challenges posed

  • pragmatic considerations of HR

  • clinical applications

  • educational development

Discovering Talent and Potential


The future of assessment8
The future of assessment

The future of assessment:

- Constructs - Methodology

- Innovation - Dynamic modeling

- Integration - Contextualisation

- Internet - Statistics

- Effectiveness of assessment - Legal & ethical

- Practical utility - International collaboration

- Psychophysiological


The future of assessment9
The future of assessment

  • Future constructs:

  • Intelligence → cognitive processing →wisdom → intuition

  • Personality assessment → motivation → consciousness

  • Integrity → compassion

  • Leadership → intention

Discovering Talent and Potential


The future of assessment10
The future of assessment

Future methodology

Statistics not well suited to Psychology

Johnson (1936):

"Those data should be measured which can be measured; those which cannot be measured should be treated otherwise.”

“Statistics has failed to demonstrate the true structure of the attributes it is studying. … It has failed to understand the difference between quantity and quality as distinguishable modes of being”

(Grice, 2012)

Discovering Talent and Potential


The future of assessment11
The future of assessment

  • Statistics in Psychometrics:

  • averages

  • group membership and category

  • bell curve (not for random data / qualitative differences)

  • standard deviation

  • correlations

  • null hypotheses (no proof)

Discovering Talent and Potential


The future of assessment12
The future of assessment

Dilemma: meaning lost via testing and statistical analysis

Resulted in: devising more complex analytical techniques without reconsidering the fundamental premises

But, psychology operates in a subtle, complex, dynamic and interactive world, one that appears random because we don’t understand the dynamics involved

Although statistical inference has not served psychology that well, formal analysis of data is important in any science

Discovering Talent and Potential


The future of assessment13
The future of assessment

  • Statistics

  • “Substantive methodological synergies”

  • (Marsh et al 2009)

  •  integrating multilevel and structural equation approaches

  • to control measurement and sampling error

Discovering Talent and Potential


The future of assessment14
The future of assessment

  • A call for Innovation:

  • An approach which caters for the dynamic and complex nature of human functioning is required

    • Brad Woods (2011):

      • very little progress except for Observation Oriented Modelling (OOM) – behaviour seen as goal directed and purposeful, not mechanical and responsive

    • Barrett (2011, 2012): Innovations include:

      • Cognitive Process Profile (CPP)

      • Read et al’s (2010) neural net research in personality

      • Gigerenzerand colleagues from the ABC group: "fast and frugal heuristics”

Discovering Talent and Potential


The future of assessment15
The future of assessment

  • Dynamic modeling

  • Move from broad generalizations into more precisely modelingthe individual

  • Not for psychometrics and statisticians, but for “scientist-philosopher-heuristics specialists”(Barrett)

  • Consider technology platforms and business intelligence / rules for dynamic assessment

Discovering Talent and Potential


The future of assessment16
The future of assessment

  • Dynamic modeling

  • to capture the output of a "complex", self-organizing, adaptive system

  • may require a new kind of systems-modeling

  • where trajectory mapping for an individual is the goal

  • rather than "measurement" of psychological attributes

  • The issue is thus not "psychometrics - measurability", but the "outcome-consequences" of a dynamic complex system (Barrett, 2011)

Discovering Talent and Potential


The future of assessment17
The future of assessment

  • Dynamic modelingtechniques:

  • Neural net applications and algorithmically based, “if-then” expert systems

    • CPP

    • OOM

Discovering Talent and Potential


The future of assessment18
The future of assessment

  • Integration of study fields and constructs

  • Human functioning is integrated

    • (physiological, emotional, motivational, psychological, spiritual and collective factors)

  • Holistic assessment practices

  • Human functioning is contextualised

  • Integrationof disciplines (e.g. psychology & neurology / sociology …)

  • Integration of nomological networks of concepts

  • Discovering Talent and Potential


    The future of assessment19
    The future of assessment

    • Building of an integrated science

    • Across epistemologies: both emic (meaningful to actor) and etic (objective description) perspectives

    • Across disciplines

    • Across schools of psychology

    • Across constructs

    Discovering Talent and Potential


    The future of assessment20
    The future of assessment

    • Contextualisation of assessment results

    • Human functioning involves individual, collective and contextual components

    • Currently dealt with via:

    • competency approach

    • person-job-matching

    • situational judgement methodology

    • Challenge: cross-cultural bias

    Discovering Talent and Potential


    The future of assessment21
    The future of assessment

    • Internet options and delivery

    • Spread to a wider audience

    • The individual taking charge of own growth

    • Social networks

    • New functionalities e.g. alternative item generation

    Discovering Talent and Potential


    The future of assessment22
    The future of assessment

    • Ensuring the effectiveness of assessment

    • Determining the actual value that tests add to the user

    • Evidence of the utility of assessment is required to guide investment in psychometric research and application

    • But how to measure it

    • Maybe also systems modelingplus rigorous evidence-base designs

    Discovering Talent and Potential


    The future of assessment23
    The future of assessment

    • Ethical and legal implications

    • Current pragmatic value of psychometrics

    • Benefits of the test subject: self-assessment and self-development offerings, job applications

    • Feuerstein: assessment should have a developmental impact

    • Cross-cultural bias and fairness

    • Addressing values of business leadership

    • Intellectual property challenges

    Discovering Talent and Potential


    The future of assessment24
    The future of assessment

    • Practical utility of assessment

    • Clinical purposes

    • Legal field

    • Moral-ethical matters

    • Optimising the man-machine interface

    • Advertising

    • Learning disabilities

    • Relationship counselling

    • Self-development

    Discovering Talent and Potential


    The future of assessment25
    The future of assessment

    International collaboration and group research

    Barrett suggests large scale international academic collaboration

    Discovering Talent and Potential


    The future of assessment26
    The future of assessment

    Psychophysiological

    Medical imaging to integrate functional and structural aspects of mental functioning

    Discovering Talent and Potential


    The future of assessment27
    The future of assessment

    • Summary

    • Theoretical integration

    • New constructs, models and nomological networks

    • New research methodologies (dynamic modeling)

    • New delivery mechanisms

    • New analytical techniques

    • New purposes

    • Contextualisation

    • Practical everyday uses

    • Ethical approach

    • Measurement of impact

    Discovering Talent and Potential


    The future of assessment28
    The future of assessment

    • Conclusion

    • The psychometrics paradigm thus needs to make way for something more:

    • comprehensive

    • relevant

    • dynamic

    • purposeful

    Discovering Talent and Potential


    Thank you for your valuable time

    011 – 884 0878 www.cognadev.com

    Discovering Talent and Potential


    ad