The Networked Education Database - PowerPoint PPT Presentation

The networked education database
1 / 32

  • Uploaded on
  • Presentation posted in: General

The Networked Education Database. Matthew Pittinsky Ph.D. Candidate, Teachers College, Columbia University. NED: A Vision. Schools have long invested in student administrative systems.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

The Networked Education Database

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

The networked education database

The Networked Education Database

Matthew PittinskyPh.D. Candidate, Teachers College, Columbia University

Ned a vision

NED: A Vision

  • Schools have long invested in student administrative systems.

  • Schools are now adopting eLearning systems equipped with gradebooks, class rosters and Web-based survey (assessment) tools.

  • Both are Internet-enabled.

  • Could “generic” and custom data be collected through school systems automatically, and anonymously, massively reducing the cost and complexity of educational research?

The networked education database


The Problem…

Original data collection

Original Data Collection

  • Requires precious classroom time.

  • Informed consents difficult to secure.

  • Customizing instruments for context across sites and time is costly and discourages certain types of data collection (e.g. sociometric).

  • Data entry and coding inhibits sharing and re-use.

  • Incomplete responses undermine results and inhibit certain types of data collection (e.g. full classrooms).

Major secondary datasets

Major Secondary Datasets

  • International studies (e.g. TIMSS).

  • Federal studies (e.g. NELS, HSB).

  • State data warehouses (e.g. Florida K-20 Education Data Warehouse (EDW)).

  • Sponsored private studies (e.g. AddHealth).

Secondary datasets issues

Secondary Datasets: Issues

  • Requires tough trade-off’s when operationalizing specific research questions.

    • e.g. classmate effect studies.

  • Often based on stratified samples, not whole classrooms and schools.

    • e.g. same-teacher class periods.

  • Rarely longitudinal within academic years.

  • Rarely contextualized (e.g. relationship questions that require roster).

The networked education database


The Solution…

Ned a dataset

NED: A Dataset

  • Classroom-level data.

  • Same-teacher data.

  • Sociometric & social-psychological data.

  • Longitudinal data.

  • Multi-site data.

  • At scale…

Ned a data collection model

NED: A Data Collection Model

  • Asynchronous (outside class time)

  • Automatic (pre-scheduled, add/drops)

  • Contextual (draws on system data to generate questions)

  • Non-duplicative (uses already stored or entered data where possible)

  • Complete (form checks)

  • Anonymous (unique ID)

  • Efficient (paperless and coded)

  • Sustainable (self-perpetuating)

How ned works

How NED Works

  • School installs NED extension and marks participating classes.

  • School’s eLearning system automatically posts survey based on schedule (w/ announcement).

  • Participant provides consent.

  • Survey is delivered through eLearning system GUI.

  • Survey draws on class context (roster, subject matter, student information, etc.) when phrasing customized questions.

  • Survey enforces certain completion rules.

  • Survey responses stored in special encrypted tables that self-delete after posting.

  • Survey responses and pre-existing data (demographic, gradebook) are packaged and securely posted to NED.

  • New students are “caught up” when added to course.

  • Teacher knows how many completes, but not who.

The networked education database

Scaling NED

Participant anonymity

Participant Anonymity

  • Data arrives to NED as secondary data (anonymous and coded).

  • Survey responses are tagged with unique participant IDs.

  • Generic data tagged with same ID and automatically merged with survey responses (gradebook, demographic, course overlap).

  • Structure of unique participant ID allows for sorting by class and school, however...

  • Data arrives at NED without any knowledge of participant’s school or classroom identity.

Participant confidentiality

Participant Confidentiality

  • Survey responses are encrypted and automatically self-delete on local server.

  • No school official has access to student responses or completion status.

  • Data transmission is via secure protocol.

Ned data feed example

NED Data Feed: Example

The networked education database


The Pilot…

Ned pilot

NED Pilot

  • Custom extension to the Blackboard Learning System.

  • Solicited 15 sites, 5 agreed, 3 ultimately participated.

    • All secondary schools (2 private / 1 public), 3 different states and regions.

  • 18 teachers, 37 classes, 732 participants

  • Three pre-scheduled NED survey administrations (October, January, May).

    • Each “live” for two weeks

  • NED staff know site names, but not names of participating schools (if district), teachers or class information.

  • Students provided incentive to participate.

  • Surveys included questions from other datasets to compare responses.

  • Approximately 250 development hours.

Ned status

NED Status

  • First administration launched 10-16 and ends today (10-30).

  • Several showstopper technical issues identified and resolved.

  • One site dropped out.

  • % participation and incompletes will be assessed 11-1.

Ned pilot walkthrough

NED Pilot: Walkthrough

Ned pilot walkthrough1

NED Pilot: Walkthrough

Ned pilot walkthrough2

NED Pilot: Walkthrough

Ned pilot walkthrough3

NED Pilot: Walkthrough

Ned pilot walkthrough4

NED Pilot: Walkthrough

Ned pilot walkthrough5

NED Pilot: Walkthrough

Ned pilot walkthrough6

NED Pilot: Walkthrough

Ned pilot walkthrough7

NED Pilot: Walkthrough

Implementation issues

Implementation Issues

  • Not a standard “building block;” required custom coding.

    • Bb installations vary, affecting custom code.

  • Not Bb’s standard survey tool.

    • Survey formatting limited.

    • Save and start, adaptive, and timing features limited.

    • Low ease of use (e.g. self-reference not grayed in sociometric questions; matrix questions scroll off screen without freezing roster).

  • Gradebook entries are user-defined, without a standard taxonomy.

  • Many schools create one mega-site for all class periods.

    • Pilot leans away from core subjects.

    • Pilot leans away from same-teacher course sections.

  • Many schools do not use Bb as their gradebook or student profile of record.

  • Relying on teacher responses for student-level data not always viable (e.g. mixed age-grade classes).

Implementation issues1

Implementation Issues

  • Required “enterprise license” of Blackboard.

  • Data transmission via local SQL scripts, not Web service.

  • IP address of sending site could allow for matching of school name with unique ID schema.

  • Different participant ID’s across classes (if student changed class periods).

  • System reports fragmented and unusable without additional programming.

  • Total eligible population not included in system report.

  • Ideal survey length difficult to asses.

  • Anonymity and class time impact concerns during site solicitation.

  • Will students participate?

Future directions

Future Directions

  • Implement through standardized APIs and via eLearning system’s survey tool.

  • Pilot with larger number of sites.

  • Pilot with smaller, more frequent surveys.

  • Pilot with full site participation across all classes and grades.

  • Pilot with full age-grade population over time.

  • Include non eLearning systems (e.g. TPR) and non Bb eLearning systems.

    • Formalize vendor NED interface program.

  • Expand to higher education.

Ned imagine

NED: Imagine

  • A national dataset.

  • Fed from tens of thousands of sites.

  • Collecting unique classroom-level data.

  • Throughout the academic year and a student’s educational career.

  • With minimal site-specific maintenance.

  • Efficiently and cost effectively.

Ned team

NED Team

  • Technical

    • Tim Streightiff, lead developer

    • Linda Merryman, project director

    • Basheer Azizi, database engineer

  • Functional

    • Matthew Pittinsky, principal investigator

    • Gary Natriello, principal investigator

The networked education database1

The Networked Education Database

A Joint Research Project


Matthew Pittinsky:

Gary Natriello:

  • Login