Fusing Wireless Sensor Data to Measure Small-Group Collaborative Processes in Real-Time

1. Fusing Wireless Sensor Data to Measure Small-Group Collaborative Processes in Real-Time Gregory K. W. K. Chung, Girlie C. Delacruz, Linda F. de Vries, Cecile H. Phan, UCLA/CRESST Mani B. SrivastavaDepartment of Electrical Engineering, UCLA Raul Alarcon, Seeds University Elementary School Annual Conference of the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) September 10-11, 2002 Los Angeles, CA

2. Project Background • Project focus is on engineering issues • Wireless networks, embedded systems, sensors, location tracking, speech recognition, middleware, location services, data mining • Deploy in demonstration classroom for system validation • Engineering: Does the hardware and software work? • Education: Can we use the technology to assess small group collaboration?

3. iBadge 1.75W x 2.75H x 0.5D inches 2.3 oz

4. Research Challenge • Devise data fusion strategy to measure collaborative processes from simple behavioral data • Location (x,y) • Head orientation (degrees from true north) • Speech on/off

5. Research Issues • How can sensor-based data be transformed into measures useful for instructional and assessment purposes? • Feasibility • Can we develop a behavioral calculus for individual and group behavior in a classroom context? • Can the calculus be operationalized with sensor-based measures? • Validity issues • How accurate are sensor-based measures in detecting individual and group interaction patterns? • How well do sensor-based measures detect changes in individual and group interaction? Relate to external measures of performance and achievement?

6. Inferential (e.g., members are collaborating) construct behavioral primitive Descriptive (e.g., facing speaker, speaking) Location, orientation, speech on/off Voltage Behavioral Calculus • • • atomic-level measure atomic-level measure • • • sensor data sensor data

7. Bayesian Networks • Model causal relations of phenomena • Graphical network representation • Nodes represent variables (observable, unobservable) • Links represent dependencies between variables • Conditional probabilities associated with each node • API available • Microsoft MSBNx API; Hugin API • Moment-to-moment updates to network

8. Collaboration Engagement Interaction Existence of Group Member speaking? Member facing speaker? Orientation toward group centroid Neighbors Simplified Bayesian Network Unobservable (12 nodes) Observable (48 nodes)

9. Pilot Test • Test approach to measure group processes • No real data yet -- badges in production • Simulate data from badges to exercise model under expected range of conditions • Create scenarios of group interaction -- define the processes that should be occurring • Carry out scenarios with mannequins and measure mannequin location and orientation (Claymation) • Compare probabilities yielded from Bayesian network to specifications of what should be occurring

10. Pilot Test

11. Pilot Test • 6 Scenarios • 1 scenario - boundary conditions • 5 scenarios - small group vignettes • Intended to represent the range of small group behaviors (drawn from 10 hours of video across 2 weeks) • 169 snapshots • 3 raters judged match/no match between probabilities and our a priori specifications of what should be occurring

12. Results • Ratings • Four variables per snapshot, 169 snapshots • Overall collaboration, interaction, engagement, group • Collaboration, interaction, engagement - 82% match • Existence of group - 88% match • Overall collaboration, interaction - 0% match for one scenario • Model sensitive to misidentification of groups

13. Tool Interface

15. Simulated Groups I Preliminary Bayesian Model Classroom Observation Next Steps Simulated Groups II Classroom Trial System Validation • Tool Interface • iBadge w/researchers going through same scenarios • False positives • False negatives • Relationship of sensor measures to performance measures and external measures