using concept map approaches to communicate and present knowledge l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Using concept map approaches to communicate and present knowledge PowerPoint Presentation
Download Presentation
Using concept map approaches to communicate and present knowledge

Loading in 2 Seconds...

play fullscreen
1 / 102

Using concept map approaches to communicate and present knowledge - PowerPoint PPT Presentation


  • 237 Views
  • Uploaded on

Using concept map approaches to communicate and present knowledge . University of Oulu, Finland EDTECH A41857 (1 credit) – Challenges, Problems, & Future of EdTech Wednesday March 30, 2005 Dr. Roy Clariana Penn State University email: RClariana@psu.edu home: www.personal.psu.edu/rbc4.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Using concept map approaches to communicate and present knowledge' - elina


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
using concept map approaches to communicate and present knowledge
Using concept map approaches to communicate and present knowledge

University of Oulu, Finland

EDTECH A41857 (1 credit) – Challenges,

Problems, & Future of EdTech

Wednesday March 30, 2005

Dr. Roy Clariana

Penn State University

email: RClariana@psu.edu

home: www.personal.psu.edu/rbc4

"First we build the tools, then they build us!" -- Marshall McLuhan

goals
goals
  • Your take aways:
    • Some experiences with collaborative concept mapping, mindmapping
    • Some understanding of how/why it works
    • Some examples that you could implement on Monday morning in your classroom or in you research
  • Your Digital Portfolio for future reference and for sharing
1 credit option
1 credit option

Digital portfolio – Formulate as a group a  digital portfolio of mindmapping, which you may utilize in the future in your studies or work.

  • For teachers, provide specific examples for using mind mapping in your classroom
  • For researchers, provide specific examples for using mind mapping in your research
2 credit option
2 credit option?
  • Digital Portfolioplus a
  • White paper – a 5-10 page (double-spaced) persuasive review of some aspect of mind mapping, i.e., scripting MM in CSCL, MM as an artifact, etc. [Based on your intuition, describe how a MM can work, this is your first iteration of a “solution”. The White papers is a “soft sell” for your “solution” that describes the problem (90% of the document) and then states clearly how your solution solves the problem (10%). Avoid straw man arguments.]
if you are interested
If you are interested…
  • Manuscript for presentation – I hope that we can publish this experience, i.e., based on several projects we will do, together we formulate questions, collect and analyze data, write… (this will likely go beyond the workshop time frame and is also more open-ended)

For example:

How does interaction develop/evolve in online collaborative mind mapping?

What scripts can support online collaborative mind mapping?

agenda for today
Agenda for today
  • Welcome and introductions all around
  • Q&A
  • Brief overview of concept maps
  • Intro to Cmap tools software
  • Brainstorm activity (group roles)
  • Set up Project 1 (see handout)
  • Set up Project 2 (see handout)
  • Does anyone have any student essays that we can use in Project 3 on Monday?

Click here for projects handout

some terminology

Some foundation stuff

some terminology
  • Concept map – diagrams indicating interrelationships among concepts and representing conceptual frameworks within a specific domain of knowledge (vanBoxtel)
  • Concept map – a visual set of nodes and arcs (a network representation) that embodies the relationships among the set of concepts. Also called knowledge maps, mindmaps, semantic maps (Turns, et al.).
  • Nodes – terms/complexes/concepts (usually nouns, things, examples, ideas, categories, people, locations…)
  • Links (arcs) – lines connecting nodes, usually labeled with a relationship term (usually verbs)
  • Propositions – node-link-node combinations, also called “soup” (ketti) by IHMC

contrast

Vygotsky

Turns, Atman, & Adams, 2000

mindmaps vs concept maps
Mindmaps vs. concept maps

Bahr (2004) using concept maps to teach English to German students

mindmap of group knowledge anni anna paula esa ja herkko source is the second floor hallway

vaihto

kyllin

usein

demot,

konkreettiset

esimerkit

sama

työtapa

liian pitkään

pelkkä

kuunteleminen

tekeminen

vs.

pelkkä

kalvoshow

työtavat

vältä !

oppilaiden erot

oppettajan

vaikutus-

mahdollisuudet

muista

huumori !

huomaa

erot

hitaat

nopeat

tukiopetus.

apu

lisätehtäviä

konkretisoi !

luokkakohtaiset

erot

yllätä !

kikkoja

opettajan

oma tarina

haasta,

kysele !

ikäluokka

vaikuttaa

vilkas luokka

elävöittää

kytke

oppilaan

arkeen !

perustele !

erityisen

paljon

kikkoja

hiljainen

luokka

huiputa !

liikuta

oppilas

ylös

penkistä

istumajärjestys !

ei

palautetta

opettajalle

näennäinen

keskittyminen ?

Mindmap of “group” knowledge (Anni, Anna, Paula, Esa, ja Herkko), source is the second floor hallway
mindmaps vs concept maps10
Mindmaps vs. concept maps
  • My question is, do concept maps or do mindmaps fit better with the Finnish language?
tools to support mapping
Tools to support mapping
  • Yellow stickies!! Pencil and paper may be best for your classroom
  • Software – PowerPoint is pretty good
  • Inspiration is good but expensive
  • CMAP tool is free, but your tech person will have to agree to support it
  • At least 22 other tools are available, some free some not
other concept map automatic scoring approaches
Other concept map automatic scoring approaches
  • CMap tools (IHMC) that we will use today
  • C-TOOLS – Luckie (PI), University of Michigan NSF grant available: http://ctools.msu.edu/ctools/index.html
  • TPL-KATS – University of Central Florida (e.g., Hoeft, Jentsch, Harper, Evans, Bowers, & Salas, 1990). TPL-KATS: concept map: a computerized knowledge assessment tool. Computers in Human Behavior, 19 (6), 653-657.
  • SEMNET – http://www.semanticresearch.com/about/
  • CMAT – Arneson & Lagowski, University of Texas, http://chemed.cm.utexas.edu
  • Plus 22 other non-scoring map tools, Inspiration, Kidspiration
some previous uses of mapping
Some previous uses of mapping
  • Usually involve individuals working alone, and involve text in some way
  • Some collaborative strategies have been used
  • Lets look at a few…
using a student mindmap to capture a text note taking
Using a student mindmap to “capture” a text (note taking)

Mindmap notes

Textbook

Text text text text text text text text text text text

memo

text

text

text

text

Examples?

student

using a student mindmap to capture research on a topic
Using a student mindmap to “capture” research on a topic

text

Text text text text tex Text text text text textt

Mindmap notes

text

Text text text text tex Text text text text textt

memo

text

text

www

text

text

Examples?

video

student

video

then using the mindmap to write an essay
Then using the mindmap to write an essay

Mindmap notes

essay

Text text text text text text text text text text text

memo

text

text

text

text

Examples?

student

using a researcher drawn mindmap to capture an interview transcript
Using a researcher drawn mindmap to “capture” an interview transcript

attribute

theory note

issue

Interview 1

Interview 1

Text text text text text text text text text text text

memo

text

text

text

text

The capability and experience of the person coding the text is critical…

coder

using a group drawn mindmap to capture an interview
Using a group drawn mindmap to “capture” an interview

Interview 1

text

text

text

text

Qs

The capability and experience of the person coding the text is critical…

interviewer

example of dyad collaboration not online
Example of dyad collaboration (not online)

Note the attentional effects of the artifact

Mindmap artefact

Verbal discussion (taped)

Observations:

On task

Abstract talk

3-propositions/min

Question

Answer

Criticize

Conflict

Elaboration

Co-construction

Analyze the discussion

Blah blah blah blah Blah blah

Blah blah blah blah Blah blah

The incredible value of talk!

Hannah

Inferred:

Active use of prior knowledge

Acknowledged problems

Look for meaningful relations

Negotiation

Yergin

Problem: Sometimes unscientific notions are ingrained

Shared objects play an important role in negotiation and co-construction

van Boxtel, van der Linden, Roelofs, & Erkens (2002)

chiu et al example of an online collaboration
Chiu et al. example of an online collaboration

Mindmap artefact

Mindmap session lasted 80 minutes. 3 x 12 online groups, communicate by chat, 745 messages were exchanged (avg. of 62 per group).

creates

Online chat

H: WE should …

J: Did you see…

Y: Yeah, but …

Etc.

Etc.

Only the lead could alter the mindmap

Jari

Hannah

(lead)

The ‘other 2 members used chat to “advise”

Researchers

Analyzed the chat text

And the mindmap

Yergin

p.22, Chiu, Huang, & Chang (2000)

project 1 and 2
Project 1 and 2
  • We will experiment with two online collaboration approaches
  • Project 1 is a synchronous concept map collaboration using Cmap tools software
  • Project 2 is an asynchronous concept map collaboration using PowerPoint software and email
  • But next, we will try brainstorming with Cmap tools to become familiar with the tools and process before setting up Project 1

Click here for projects handout

first mind map cscl roles
First Mind map CSCL roles…

Mindmap activity…

  • Starter: You work as a discussion moderator. Your assignment is to engage your group members to the discussion by asking questions and commenting. And if the wrapper makes small summaries during discussion you can utilize his or her work to raise new questions. Active participation in the discussions is essential.
  • Wrapper: Your assignment is to sum up the discussion. If you think it is easier you can summarize frequently and weave ideas together. For example, if five participants of your group are having a discussion about collaborative and co-operative learning you can summarize their main points during the discussion. An alternative way is to sum up the discussions in the end of article-videoclip task (and the last course assignment). Please overview your group's discussions and make a brief summary of the main topics. Active participation in the discussions is essential.
  • Group member: Your assignment is to participate actively into discussions by asking questions making comments and stating arguments. You are expected to be a critical inquirer.
  • Evaluator (an optional role): You are required to evaluate your group's work during the course. Please focus on the group interaction and group dynamics, for example how the starters, wrappers and group members performed during the discussions and last course assignment. The tutors inform you when to perform evaluations. Notice that you are also a deputy starter and a deputy wrapper if the originally named persons are not available. If you are called to work as a starter or wrapper please see the instructions given above. The role of evaluators are used only if you have not had a role of starter or wrapper during this course.
cluster analysis
Cluster analysis

enter

Brainstorming

(corpus list)

Sorting

(move like terms closer)

Build consensus!

Naming Clusters

(name the categories/themes)

Merging & Pruning

(combine like terms,

delete or move unlike terms,

synthesize terms)

and if necessary

Sorting Clusters

(move like clusters closer)

Naming broad themes

(name the cluster of clusters)

E-document (to save/print)

brainstorm then make the map
Brainstorm, then make the map
  • Open IHMC Cmap tools
  • Fill in personal information on first use (I’ll tell you what to type in here)
  • Click Other Places
  • Open brainstorm file
  • Click collaborate icon

if necessary

  • Type in your first name
  • Collaborate
now go back and add small group roles
Now go back andadd Small Group Roles

Mindmap activity…

Group Task Roles

Initiator-contributor. Proposes new ideas or approaches to group problem solving; may suggest a different approach to procedure or organizing the problem-solving task

Information seeker. Asks for clarification of suggestions; also asks for facts or other information that may help the group deal with the issues at hand

Opinion seeker. Asks for clarification of the values and opinions expressed by other group members

Information giver. Provides facts, examples, statistics, and other evidence that pertains to the problem the group is attempting to solve

Opinion giver. Offers beliefs or opinions about the ideas under discussion

Elaborator. Provides examples based on his or her experience or the experience of others that help to show how an idea or suggestion would work if the group accepted a particular course of action

Coordinator. Tries to clarify and note relationships among the ideas and suggestions that have been provided by others

Etc..

project 1 cmap tools synchronous collaboration
Project 1 – Cmap tools synchronous collaboration

Set day and time to join online …….

(see the Project handout)

project 1
Project 1

IHMC Public Cmaps conv v2 on Jan 22 2004

project 128
Project 1

Oulu EDTECH Public

project 2 overview of pass the soup
Project 2 – Overview of “Pass the soup”

PowerPoint file

Email to

Email to

Email to

Email to

(see the Project handout)

project 2 pass the soup powerpoint file
Project 2 – “Pass the soup” PowerPoint file

Slide 1 – mindmap is developed bit-by-bit here by the group by adding only 3 to 5 elements and then emailing it to the next person on the list

Slide 2 – numbered list of names of group members with email address, other instructions

Slide 3, 4, etc. – comments about changes that you want to make, suggestions, etc.

how to use ala reader
How to use ALA-Reader

Monday, April 4, 2005

agenda for today32
Agenda for today
  • Debrief “pass the soup” activity, and come up with a better Finnish name for it
  • Q&A
  • Brief overview of my concept map assessment research
  • ALA-Reader demo (English language essays)
  • Set up Project 3 for Finnish (see handout)
  • How can we find Finnish essays for use in Project 3?
final map for project 2 team 1
Final map for Project 2: Team 1

Click her to

See progression

Of this map

final map for project 2 team 2
Final map for Project 2: Team 2

Click her to

See progression

of this map

debriefing
Debriefing
  • What happened?
  • What worked?
  • What did not work?
  • What would you do differently next time?
  • If you like, write this up as a team for your final paper.
my research interests
My research interests

prototypes

  • Mind map assessment – automatic scoring software tool called ALA-Mapperhttp://www.personal.psu.edu/rbc4/ala.htm
  • Essay assessment – automatic scoring software tool called ALA-Readerhttp://www.personal.psu.edu/rbc4/score.htm
  • for Latent Semantic Analysis (LSA) see: http://www.personal.psu.edu/rbc4/frame.htm
novak
Novak
  • Novak says “Concept maps were first developed in our research program in 1972 as a way to represent changes in children’s understanding of science concepts over the 12-year span of schooling.  We were using modified Piagetian clinical interviews to assess changes in their knowledge over time, but we found the interview transcripts were too difficult to analyze for changes in specific aspects of the children’s knowledge.  Instead we prepared concept maps from the interviews.” 

From: http://wwwcsi.unian.it/educa/mappeconc/jdn_an2.html

first uses to represent knowledge in a visual format
First uses… to represent knowledge in a visual format

The primary parts of the system are the heart, blood cells, and vessels. The human heart, a pump, is made of cardiac muscle

Cardiac muscles have a unique feature of forming connections between two adjacent cardiac cells. This allows the muscle cells to contract powerfully and quickly involuntarily

The brain is unable to increase or decrease the heart's beating

The heart is comprised of four chambers; two upper chambers called atriums, and two lower chambers called ventricles

The blood flows through the right side to the lungs where it picks up oxygen. The blood then returns to the right. Next, it flows into the left where it I xxxx

tissue within the body by approximately 9 pints of blood through 100,000 miles of vessels

The primary parts of the system are the heart, blood cells, and vessels. The human heart, a pump, is made of cardiac muscle

Cardiac muscles have a unique feature of forming connections between two adjacent cardiac cells. This allows the muscle cells to contract powerfully and quickly involuntarily

The brain is unable to increase or decrease the heart's beating

The heart is comprised of four chambers; two upper chambers called atriums, and two lower chambers called ventricles

The blood flows through the right side to the lungs where it picks up oxygen. The blood then returns to the right. Next, it flows into the left where it I xxxx

The human circulatory system is a transportation system. Nutrients and oxygen are carried to living tissue within the body by approximately 9 pints of blood through 100,000 miles of vessels

The primary parts of the system are the heart, blood cells, and vessels. The human heart, a pump, is made of cardiac muscle

Cardiac muscles have a unique feature of forming connections between two adjacent cardiac cells. This allows the muscle cells to contract powerfully and quickly involuntarily

The brain is unable to increase or decrease the heart's beating

The heart is comprised of four chambers; two upper chambers called atriums, and two lower chambers called ventricles

The blood flows through the right side to the lungs where it picks up oxygen. The blood then returns to the right. Next, it flows into the left where it I xxxx

Novak interview data

Was science content knowledge

Mind Map

finnish research with concept maps
Finnish research withconcept maps…
  • Mainly for knowledge representation for instructional use but also for representing the structure of a curriculum and for group communication
  • Pasi Eronen, Jussi Nuutinenn and Erkki Sutinen, (http://www.cs.joensuu.fi/pages/avt/concept.htm), Joensuu (computer science)
  • Mauri Ählberg, Helsinki (education) and Erkki Rautama (computer science)
  • University of Art and Design, Helsinki (http://www2.uiah.fi/~araike/papers/articles/CinemaSense_Collaborative_Cinemastudies_DeafWay2002.htm) (see also: Future Learning Environment 3)
  • Text graphs (Helsinki): http://www.cs.hut.fi/Research/TextGraph/
  • Kari Lehtonen, Helsinki Polytechnic, concept maps as a portfolio component (http://cs.stadia.fi/~lehtonen/DPF/dpf-berlin-02-muotoiltu.doc)
  • Also School astronomy and Vocational Training and Education 
  • 4th IEEE International Conference on Advanced Learning Technologies Joensuu, Finland, August 30 - September 1, 2004
concept map for assessment score validity
Concept map for assessment: score validity???
  • Concept maps contains propositions
  • These propositions scores are generally considered to be valid and reliable measures of science content knowledge organization (Ruiz-Primo, Schultz, Li, Shavelson, CREST in California. . .).

essays

interviews

tests

observations

slide41
e.g.,…
  • Rye and Rubba (2002) reported that traditional concept map scores were related to California Achievement total test scores (r = 0.73). (Note that Crocker and Algina say that validation coefficients rarely exceed r=0.50.)
  • Concept maps (cognitive maps, concept maps) may be an appropriate approach for assessing structural knowledge (Jonassen, Beissner, & Yacci, 1993).
  • For example, concept maps have been used to visualize the change from novice to expert.
scoring concept maps
Scoring Concept Maps
  • Traditionally, concept maps are scored by teachers or trained raters using scoring rubrics (e.g., Lomask’s rubric)
  • Although this marking approach is time consuming and fairly subjective, map scores usually correlate well with more traditional measures of science content knowledge (multiple choice, fill-in-the blank, and essays)
  • Complex scoring rubrics decrease the concept map score reliability (so keep scoring simple)
scoring concept maps43
Scoring Concept Maps

C3 describes our automatic system for scoring

concept maps:

collect –>convert –> compare

  • Collect raw map data
  • Convert raw data into a mathematical network representation
  • Compare the mathematical network representation of two maps (e.g., student to teacher, student to expert, student to student)
1 collect raw data
1. Collect raw data

What raw data can a computer “extract” from a concept map?

  • Term counts – in open-ended maps, count required terms included
  • Propositions – a link connecting two terms and a link label
  • Associations – geometric distance between pairs of terms. Small values indicate stronger relationship.
link and distance data

(n2-n)/2 pair-wise comparisons

Link and distance data

Most approaches use only link label

information, usually called “propositions”.

link and distance
Link and distance
  • Link data (propositions) – are the common way to compare/assess concept maps
  • Distance data – not common, based on James Deese’s (1965) ideas on the structure of association in language and thought, card-sorting task approaches (Vygotsky in Luria, 1979, Miller, 1969), Kintsch and Landauer’s ideas on representing text structure, and neural network methods (Elman, e.g., 1995)
using our finnish mind map example

See next slide

Using our Finnish Mind Map example
  • Borrowed from Anni, Anna, Paula, Esa, ja Herkko
  • Found in the hallway on the second floor
slide48

vaihto

kyllin

usein

demot,

konkreettiset

esimerkit

sama

työtapa

liian pitkään

pelkkä

kuunteleminen

tekeminen

vs.

pelkkä

kalvoshow

työtavat

vältä !

oppilaiden erot

oppettajan

vaikutus-

mahdollisuudet

muista

huumori !

huomaa

erot

hitaat

nopeat

tukiopetus.

apu

lisätehtäviä

konkretisoi !

luokkakohtaiset

erot

yllätä !

kikkoja

opettajan

oma tarina

haasta,

kysele !

ikäluokka

vaikuttaa

vilkas luokka

elävöittää

kytke

oppilaan

arkeen !

perustele !

erityisen

paljon

kikkoja

hiljainen

luokka

huiputa !

liikuta

oppilas

ylös

penkistä

istumajärjestys !

ei

palautetta

opettajalle

näennäinen

keskittyminen ?

collect mind map raw data
Collect Mind Map raw data

9 main terms selected here (ALA-Mapper max=30)

selecting terms
Selecting terms
  • Selecting important terms (and their synonyms) is a critical step (for example, singular value decomposition in LSA derives terms). We use an expert(s) to determine terms.
  • Goldsmith, Johnson, and Acton (1991)
predictive validity of pfnets directly relates to the number of terms used
predictive validity of PFNets directly relates to the number of terms used

0,80

0,70

0,60

0,50

0,40

predictive validity

So, perhaps the predictive validity of Concept Maps (and essays) directly relates to the number of terms used

0,30

0,20

0,10

0,00

0

10

20

30

Number of terms

Goldsmith, Johnson, and Acton (1991)

2 convert raw data into scores
2. Convert raw data into scores
  • Currently, we use a data reduction and comparison approach called Pathfinder network representation (PFNet, Schanveldt, 1990). Our future research will consider additional approaches, such as MDS and data-mining. http://interlinkinc.net/Pathfinder.html
  • PFNets describe the least weighted path to connect the terms
  • Scores are established by comparing the participant’s PFNet to a referent (expert) PFNet, and calculating the number of common links (the intersection)

Visual example 

compare student to expert referent
Compare student to expert referent

O

6 of 8 common links

O

Expert Referent PFNet

Student PFNet

poindexter and clariana

#1st

Poindexter and Clariana
  • Participants – 23 undergraduate students in intro EdPsyc course (Penn State Erie)
  • Food rewards for participation
  • Setup – complete a demographic survey and how to make a concept map lesson
  • Text based lesson interventions – instructional text on the “heart” with either proposition specific or relational lesson approach

Poindexter, M. T., & Clariana, R. B. (in press). The influence of relational and proposition-specific processing on structural knowledge and traditional learning outcomes. International Journal of Instructional Media, 33 (2), in press. link to doc file

treatments
Treatments
  • Relational condition, participants were required to “unscramble” sentences (following Einstein, McDaniel, Bowers, & Stevens, 1984) in one paragraph in each of the five sections or about 20% of the total text content
  • Proposition-specific condition (following Hamilton, 1985), participants answered three or four adjunct constructed response questions (taken nearly verbatim from the text) provided at the end of each of the five sections, for a total of 17 questions covering about 20% of the total text content (no feedback was provided).
posttests
Posttests
  • Concept map (use 26 terms provided)
    • Link-based common scores
    • Distance-based common scores
  • Multiple-choice tests (Dwyer, 1976)
    • Identification (20)
    • Terminology (20)
    • Comprehension (20)
analysis
Analysis
  • MANOVA (relational, proposition-specific, and control) and five dependent variables including ID, TERM, COMP, Map-prop, and Map-assoc.
  • COMP was significance, F = 5.25, MSe = 17.836, p = 0.015, none of the other dependent variables were significance.
  • Follow-up Scheffé tests revealed that the proposition-specific group’s COMP mean was significantly greater than the control group’s COMP mean (see previous Table).
correlations
Correlations

Map-link

Map-link

Map-distance

All sig. at p<.05

Compare to Taricani & Clariana

next 

taricani and clariana replication of poindexter and clariana
Taricani and Clariana – Replication of Poindexter and Clariana

TermComp

Link data 0.78 0.54

Distance data 0.48 0.61

Taricani, E. M. & Clariana, R. B. (in press). A technique for automatically scoring open-ended concept maps. Educational Technology Research and Development, 53 (4), in press.

compare these two
Compare these two . . .

Taricani & Clariana TermComp

Link data 0.78 0.54

Distance data 0.48 0.61

Poindexter & Clariana TermComp

Link data 0.77 0.53

Distance data0.69 0.71

clariana koul salehi

# 2nd

Clariana, Koul, & Salehi
  • Participants – A group of 24 practicing teachers enrolled in CI 400
  • Lesson intervention – while researching online, completed concept maps in pairs (newsprint & yellow stickies) to describe the structure and function of the heart and then individually wrote essays on this topic from their maps.

Clariana, R. B., Koul, R., & Salehi, R. (in press). The criterion related validity of a computer-based approach for scoring concept maps. International Journal of Instructional Media, 33 (3), in press.

posttests64
Posttests

Essays

  • Multiple-raters using holistic rubric
  • Computer-derived LSA Essay scores (http://www.personal.psu.edu/rbc4/frame.htm)

Concept Maps

  • Multiple-raters using Lomask’s rubric
  • ALA-Mapper PFNet link and distance agreement with an expert
correlation matrix
Correlation matrix

Human

Computer

Map

Essay

LSA

Link

Map

1

Essay

0.49

1

LSA

0.31

0.73

1

Link data

0.36

0.76

0.83

1

Distance data

0.60

0.77

0.71

0.82

1

p

< .05 shown in boldface type.

Many investigators have noted the close relationship between maps and essays.

overview tools to score essays
Overview: Tools to score Essays
  • ETS – PEG (Project Essay Grade), e-rater, Criterion and other products… http://www.ets.org/research/erater.html
  • Walter Kintsch (and Landau) at CU-Boulder – Latent semantic analysis, many uses, i.e., score online training for the Army - http://lsa.colorado.edu/
  • Vantage Learning essay scoring products - http://www.vantagelearning.com/

ALA-Reader: http://www.personal.psu.edu/rbc4/score.htm

ala reader
ALA-Reader

Text

PFNet

… an electrical signal starts the heartbeat, by causing the atrium to contract. The blood then flows through the pulmonary valve into the pulmonary artery and then into the lungs. Once inside the lungs, the blood gives up the carbon dioxide (cleansed) and receives oxygen. This oxygenated blood …

atrium

contract

P valve

P artery

lungs

cleansed

oxygenated

Link array

clariana koul

# 3rd

Clariana & Koul
  • Participants – Again, a group of 24 practicing teachers enrolled in CI 400
  • Lesson – while researching the topic “the structure and function of the heart” online, students completed concept maps using Inspiration software and later wrote an essay on this topic from their maps.

Clariana, R.B., & Koul, R. (2004). A computer-based approach for translating text into concept map-like representations. In A.J.Canas, J.D.Novak, and F.M.Gonzales, Eds., Concept maps: theory, methodology, technology, vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sep 14-17, pp.131-134. http://cmc.ihmc.us/papers/cmc2004-045.pdf

posttests69
Posttests

Essays

  • Multiple-raters using holistic rubric
  • Computer-derived LSA Essay scores (http://www.personal.psu.edu/rbc4/frame.htm)

Concept Maps

  • Multiple-raters using Lomask’s rubric
  • ALA-Mapper PFNet link and distance agreement with an expert
  • ALA-Reader PFNet link scores (from 1 to 5)

(so far, only looked at essay scores) 

ala rater pfnet scores
ALA-Rater PFNet scores
  • The scores for each text and rater-pair are shown ordered from best to worst.
  • ALA-Reader scores were moderately related to the combined text score, Pearson r = 0.69, and ranked 5th overall.
demo ala reader
Demo ALA-Reader
  • Download ALA-Reader.exe
  • Create terms file (can include 2 synonyms)
  • Create 2 expert baseline reference texts called expert1.txt and expert2.txt (i.e., Instructor, best student)
  • Use it (type in the students essay file name)
  • Files created
    • Summary file called report.txt
    • Multiple *.prx files (PRX folder)

Available at: www.personal.psu.edu/rbc4

agenda
agenda

Today is a hands-on demonstration day

  • Brief overview of the ideas
  • SPSS for representing
  • Pathfinder
  • KU-Mapper

My intent, you will know enough to begin to use these approaches

eliciting structural knowledge
Eliciting structural knowledge
  • Every method for eliciting knowledge should be viewed as “sampling”
  • Caution, never forget the likely effects of contiguity (time, space, etc.) dominating over semantics (meaning)

essays

interviews

tests

observations

dave s ideas
Dave’s ideas

Knowledge

elicitation

Knowledge

comparison

Knowledge

representation

Jonassen, Beissner, & Yacci (1993), page 22

dave s ideas77
Dave’s ideas

word

associations

semantic

proximity

similarity

ratings

card

sort

relatedness

coefficients

Knowledge

elicitation

ordered

recall

quantitative

graph

comparisons

scaling

solutions

graph

building

free

recall

C of PFNets

qualitative

graph

comparisons

Knowledge

comparison

additive

trees

Knowledge

representation

expert/

novice

hierarchical

clustering

Trees

Dimensional

Networks

MDS – multidimensional scaling

ordered

trees

link

weighted

principal

components

cluster

analysis

minimum

spanning

trees

Pathfinder

nets

Jonassen, Beissner, & Yacci (1993), page 22

eliciting structural knowledge78
Eliciting structural knowledge
  • Vygotsky (in Luria, 1979); Miller (1969) card-sorting approaches
  • Deese’s (1965) ideas on the structure of association in language and thought
  • Kintsch and Landauer’s ideas on representing text structure, and latent semantic analysis
  • Recent neural network representations (e.g., Elman, 1995)
analyzing deese free association data with mds
Analyzing Deese free association data with MDS
  • Hands-on with MDS in SPSS
    • A good description of MDS: http://www.statsoft.com/textbook/stmulsca.html
    • (Aside: a good description of Factor analysis: http://www.statsoft.com/textbook/stfacan.html )
  • Hands-on with Pathfinder KNOT
deese free recall data p 56
Deese, free recall data (p.56)

100 participants are shown a list of related words, one at a time, and asked to free recall a related term

Full array (n * n): 19 x 19 = 361

Half array ((n2 – n)/2): ((19 x 19) –19 )/2 = 171

Deese, J. (1965). The structure of associations in language and thought. Baltimore, MD: John Hopkins Press, page 56

deese free recall data p 5681
Deese, free recall data (p.56)

Full array (n * n): 19 x 19 = 361

Half array ((n2 – n)/2): ((19 x 19) –19 )/2 = 171

Deese, J. (1965). The structure of associations in language and thought. Baltimore, MD: John Hopkins Press, page 56

using mds in spss
Using MDS in SPSS
  • Start SPSS and open the deese.sav file
  • Under Analyze, select Scale, then select Multidimensional Scaling (ALSCAL)…
  • Move Variable from left to right
  • Create distances from data
  • Model
  • Options

Next page

side issue the mds obtains alternate e g enantiomorphic visual representations
Side issue, the MDS obtains alternate (e.g., enantiomorphic) visual representations

Is this map correct?

Helsinki

Oulu

Both are “correct”.

Tampere

Pori

Jyväsklyä

Jyväsklyä

Tampere

Pori

Helsinki

Oulu

geographic data, for example, may be oriented in different ways

how good is the representation
How good is the representation?
  • many dimensions (as many as 19) reduced to 2 dimensions
  • Check the “stress” value to estimate how strained the results are

An algorithmic, power, approach rather than based on a model so no assumptions about data structure are required…

side trip
Side trip
  • Wordnet: http://wordnet.princeton.edu/

http://wordnet.princeton.edu/cgi-bin/webwn

  • What is the Visual Thesaurus? – The Visual Thesaurus offers stunning visual displays of the English language. Looking up a word creates an interactive visual map with your word in the center of the display, connected to related words and meanings.
  • Type “bird” in at: http://www.visualthesaurus.com/trialover.jsp
pathfinder network pfnet analysis
Pathfinder Network (PFNet) analysis
  • Pathfinder is a mathematical approach for representing and comparing networks, see: http://interlinkinc.net/index.html
  • Pathfinder data reduction is based on the least weighted path between nodes (terms), so for example, Deese’s 171 data points become 18 data points. Only the salient or important data is retained.
  • Pathfinder PFNet uses, for example:
    • Library reference analysis
    • Measuring Team Knowledge (Nancy J. Cooke) next slide
    • Use google to see many more
pathfinder for cognitive task analysis
Pathfinder for cognitive task analysis

Shope, DeJoode, Cooke, and Pedersen (2004)

pfnet of same data
PFNet of same data

Now let’s try Pathfinder analysis of the

same Deese data set…

  • Find the pfnet folder
  • Double-click to run PCKNOT.bat (notice the bat extension, see next slide below)
  • We will do it together
pfnet of deese data

sky

summer

blue

spring

sunshine

garden

yellow

color

flower

nature

butterfly

cocoon

moth

wing

bees

bird

fly

insect

bug

PFNet of Deese data
mds and pfnet of deese data
MDS and PFNet of Deese data

SPSS MDS

Pathfinder KNOT PFNet

mds and pfnet data reduction
MDS and PFNet data reduction
  • MDS uses all of the data points to reduce the dimensions in the representation, and so may be improperly driven by noise in the data or by unimportant data points
  • Pathfinder uses only the most important data
transition to your real life example
Transition to your real life example
  • Finally, you will collect *real* data (using my KU-Mapper software) and
  • analyze it with Pathfinder KNOT
ku mapper
KU-Mapper
  • Your data, determine 15 important terms in your research area (Finnish and English), create a “terms.txt” file with the 15 terms
  • Run KU-Mapper (do all 3 tasks: pair-wise, list-wise, and card sort)
  • Use KNOT to analyze and compare all three prx files

Download KU-Mapper from: http://www.personal.psu.edu/rbc4/KUmapper.htm

debrief your data activity
Debrief your data activity
  • What happened?
  • What worked?
  • What did not work?
  • What would you do differently next time?
  • If you like as your final paper, describe how you might use this approach.
final thoughts
Final thoughts…
  • I enjoyed working with you
  • If you want a credit,
    • Email to let me know this
    • Then be sure to send me you paper via email as soon as possible
possible research question on optimal scripts under vs over scripting cscl
Possible research question on optimal scripts: Under- vs. over-scripting CSCL

Amount of collaboration

with crash?

linear

S-curve

J-curve

Amount of scripting 

Amount of scripting 

Amount of scripting 

Some possibilities

generative learning strategies

+

+ +

generative learning strategies

Generative learning(Jonassen, 1988)

  • recall - repetition, rehearsal, review, mnemonics
  • integration - learner paraphrases, generates questions, generates examples
  • organization - learner analyzes key ideas by creating headings, underlining keywords, outlining, categorizing (i.e., invent table categories, populate a table with existing ideas)
  • elaboration - generate mental images, create physical diagrams, sentence elaboration (i.e., invent stuff to fill cells in a table)
i just think systemically and n dimensionally on paper with imagery
I just "think" systemically and "n-dimensionally" on paper, with imagery…
  • My essential skill is simply--If you can explain it to me, I can draw a picture of it. It doesn't matter if it's something totally new to me, if you can make a coherent explanation, and let me understand it. I can "visualize it" and make a picture that shows you what you said.
  • This is why I work in aerospace. I'm able to sit down with SME's (Subject Matter Experts-in any discipline), let them do a "data-dump" and put a sketch in their hand at the end of the conversation that "say's it all". This skill is vital to helping disparate technical types talk to each other (communication across cultural barrier of the "dialect" of the various technical disciplines). It also provides a way for ideas to get from that rough-semi coherent stage and into a practical and "do-able" condition.

For example,

  • One day I found myself working a Kelly Temp job for a bunch of Boeing System Analysts doing a JAD (joint application development) project to design a computing architecture for a new tooling system for the 777. The first drawing came by accident, started a huge argument, and eventually (2 weeks later) resolved in a group wide "a-hah"... that put everyone on the same wavelength-allowing the new system to be built a lot more "right" than usual, quicker than usual.

From: http://visual.wiki.taoriver.net/moin.cgi/MichaelErickson