NDIA  3
This presentation is the property of its rightful owner.
Sponsored Links
1 / 41

Center for Intelligent Systems Research GW Transportation Research Institute PowerPoint PPT Presentation


  • 108 Views
  • Uploaded on
  • Presentation posted in: General

NDIA 3 rd Annual Intelligent Vehicle Systems Symposium Driving Simulator Experiment: Detecting Driver Fatigue by Monitoring Eye and Steering Activity. Center for Intelligent Systems Research GW Transportation Research Institute

Download Presentation

Center for Intelligent Systems Research GW Transportation Research Institute

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

NDIA 3rd Annual Intelligent Vehicle Systems Symposium Driving Simulator Experiment:Detecting Driver Fatigue by Monitoring Eye and Steering Activity

Center for Intelligent Systems Research

GW Transportation Research Institute

The George Washington University, Virginia Campus, 20101 Academic Way, Ashburn, VA 20147

Dr. Azim Eskandarian, Riaz Sayed (GWU)


Research objective

Research Objective

Conduct Simulator Experiment and Analyze the Data, to search for a system for automatic detection of drowsiness based on driver’s performance


Significance of the problem

Significance of the Problem

  • Drowsiness/Fatigue Related Accident Data:

  • NHTSA Estimates 100,000 drowsiness/fatigue related Crashes Annually

  • FARS indicates an annual average of 1,544 fatalities

  • Fatigue has been estimated to be involved in 10-40% of crashes on highways (rural Interstate)

  • 15% of single vehicle fatal truck crashes

  • Fatigue is the most frequent contributor to crashes in which a truck driver was fatally injured


Significance of the problem1

Significance of the Problem

  • A drowsy/sleepy driver is unable to determine when he/she will have an uncontrolled sleep onset

  • Fall asleep crashes are very serious in terms of injury severity

  • An accident involving driver drowsiness has a high fatality rate because the perception, recognition, and vehicle control abilities reduces sharply while falling asleep

  • Driver drowsiness detection technologies can reduce the risk of a catastrophic accident by warning the driver of his/her drowsiness


Driver drowsiness detection techniques

Driver Drowsiness Detection Techniques

1. Sensing of driver physical and physiological phenomenon

  • Analyzing changes in brain wave or EEG

  • Analyzing changes in eye activity and Facial expressions

  • Good detection accuracy is achieved by these techniques

  • Disadvantages:

    • Electrodes have to be attached to the body of the driver for sensing the signals

    • Non-contact type sensing is also highly dependant on environmental conditions


  • Driver drowsiness detection techniques1

    Driver Drowsiness Detection Techniques

    2. Analyzing changes in performance output of the vehicle hardware

    • Steering, speed, acceleration, lateral position, and braking etc.

  • Advantages:

    • No wires, cameras, monitors or other devices are to be attached or aimed at the driver

    • Due to the non-obtrusive nature of these methods they are more practically applicable


  • Approach for drowsiness detection and driver warning

    Approach for Drowsiness Detection and Driver Warning


    Experiment

    Experiment

    • Conducted in the Vehicle Simulator Lab of the CISR. GWU VA Campus, Ashburn VA.

    • Twelve subjects between the ages of 23 and 43

    • Test Scenario consisted of a continuous rural Interstate highway, with traffic in both directions Speed limit of 55 mph.

    • Morning session 8 – 10 am

    • Night session 1 – 3 am


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    CISR Driving Simulator


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Eye Tracking Equipment


    Sample data from simulator

    Sample Data From Simulator

    RUN#ZONETIMESPEEDLIMCRASHBCRASHVLANEXBRAKEFORBRAKETAP

    1 035 0 0 0 0 0

    1 2.135 0 0 0 0 0

    1 4.235 0 0 0 0 0

    1 6.235 0 0 0 0 0

    1 8.335 0 0 0 0 0

    STEERPOSSTEERVARLATPLACE LATPLVARSPEEDSPEEDVARSPEEDDEV

    -0.1 0-0.09 0 53.71 0 -4.65

    0.2 0-0.22 0 53.71 0 -4.65

    0.4 0-0.31 0 53.71 0 -4.65

    0 0-0.35 0 53.71 0 -4.65


    Lateral position of vehicle

    Lateral Position of Vehicle


    Power spectrum density for vehicle lateral position

    Power Spectrum Density for Vehicle Lateral Position


    Steering angle filter correction for curves

    Steering Anglefilter correction for curves


    Hypothesis

    Hypothesis

    • The hypothesized relationship between driver state of alertness and steering wheel position is that under an alert state, drivers make small amplitude movements of the steering wheel, corresponding to small adjustments in vehicle trajectory, but under a drowsy state, these movements become less precise and larger in amplitude resulting in sharp changes in trajectory (Planque et al. 1991).


    A hybrid artificial neural network architecture

    Unsupervised Layer : Clustering

    Competitive Algorithm

    Supervised Layer: Classification

    Feedforward Algorithm

    A Hybrid Artificial Neural Network Architecture

    Wj1

    2

    8 X 8


    Hybrid artificial neural network architecture

    Hybrid Artificial Neural Network Architecture


    Ann training for unsupervised competitive layer

    ANN Training for Unsupervised Competitive Layer

    1. Initialize the weight vector randomly for each neuron.

    2. Present the input vector X(n) .

    3. Compute the winning neuron using the Euclidean distance as a metric.

    Where Wi= [w1, w2, …. w8]T is the weight vector of neuron i.

    bi is the bias to stop the formation of dead neurons.


    Ann training competitive layer continued

    ANN Training Competitive Layer Continued

    • N number of time a neuron wins in competitive layer

    •  and are learning constants and o(n) is the outcome of the present competition (=1 if neuron wins & else = 0).

    • Ci initially set to small random value

    • 4. Update the weight vector of the winning neuron Wi* only.

    • 5. Continue with step (2) two until change in the weight vectors reaches a minimum value.


    Ann training competitive layer continued1

    ANN Training Competitive Layer Continued

    • The competitive algorithm moves the weight vectors of all the neurons closer to the center of the clusters.

    • Each neuron (or set of neurons) of the competitive layer represents a cluster.

    • The Output of the neuron is 1 if it wins the competition and 0 if it losses.

    • The Output of the Competitive layer is an

      n-dimensional binary vector T(n) = [t1, t2, …….., tn]T .


    Ann training for supervised feed forward layer

    ANN Training for supervised feed forward layer

    • Step 1: Initialize the synaptic weights and the thresholds to small random numbers.

    • Step 2: Present the network with an epoch of training exemplars

    • Step 3: Apply Input vector X(n) to the input layer and the desired response d(n) to the output layer of neurons. The output of each neuron is calculated as


    Ann training continued

    ANN Training Continued


    Ann training continued1

    ANN Training Continued

    • N = No. of training sets in one epoch

    •  = Learning rate parameter

    •  = Momentum constant

    • Step 5: Iterate the computation by presenting new epochs of training examples until the mean square error (MSE) computed over entire epoch achieve a minimum value. MSE is given by:


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    ANN Training Parameters

    • Hybrid architecture using an unsupervised clustering algorithm and a classifier (Back propagation learning algorithm in batch mode)

    • Tanhyperbolic activation function, with output range from –1 to 1

    • Variable learning rate and momentum were used

    • Cross validation during training


    Input discretization of steering angle

    Input Discretization of Steering Angle

    Algorithm to select r (ranges) for each driver to compensate performance variability between drivers

    Discretized steering angle for one driver :


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Accounting for Individual Driver Behaviors

    • Some drivers are more “sensitive” to vehicle lateral position and make very accurate corrections to the steering for lane keeping while other are less “sensitive” and make less accurate corrections.

    • The result is a low amplitude signal (steering angle) for more “sensitive” drivers and relatively high amplitude signal for less “sensitive” drivers.

    • Larger values for Pk will make the descritization ranges wider to accommodate large amplitude while small values will make them shorter for small amplitudes.

    • Therefore, same ANN (8-dimensional descritization) can be used


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Input Discretization of Eye closures

    • Eye closure data is recorded at 60 Hz

    • Ci = No. of zero’s in 1 second of data

    • Ci is further discretized according to the following scheme


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Input Discretization of Eye closures

    Algorithm to select r (ranges) for each driver to compensate eye closure variability between drivers

    P values are representative of variability of eye closures (blinking) for each driver

    Sample of a few seconds of Discretized Eye closures for one driver :


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Input Vector

    • The two vectors are combined to form a 12 dim vector J(T)

    • Vector J(T) is summed over 15 sec time interval to get the input vector X(n)


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Input and Desired Output Vector

    Each row represents the sum of discretized input over a selected time interval, e.g., 15 sec.


    Ann performance during training

    ANN Performance During Training


    Ann test data

    ANN Test Data

    • Driving data from 12 subjects available

    • 1 subject night session not recorded due to equipment error.

    • 1 subject morning data not available, software error.

    • Remaining 10 were used for training ANN and testing results,

    • NOTE: training data and testing of the ANN were not the same, Testing data selected randomly from the sets not used in the training


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Performance

    SLEEP

    Wake

    MSE

    0.0550

    0.0554

    NMSE

    0.2205

    0.2218

    MAE

    0.1259

    0.1245

    Min Abs Error

    0.0000

    0.0000

    Max Abs Error

    0.9857

    0.9806

    r

    0.8851

    0.8840

    Percent Correct

    92.3000

    93.0000

    Results

    Actual Totals

    Actual Totals

    Network Output

    Network Output

    Wake

    Wake

    Sleep

    Sleep

    Wake

    Wake

    193

    193

    179

    179

    14

    14

    False Alarm

    False Alarm

    Sleep 207

    Sleep 207

    16

    16

    191

    191

    Mis

    Mis

    -

    -

    classified

    classified

    Crash Prediction:

    All crashes that occurred due to

    driver falling asleep during the experiment were

    predicted before the crash occurred.


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Morning and Night session results


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Morning and Night session results


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Morning and Night session results


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Morning and Night session results


    Ndia 3rd annual intelligent vehicle systems symposium driving simulator experiment detecting driver fatigue by moni

    Morning and Night session results


    Time before crash when the ann generated a first warning

    Time Before Crash When the ANN Generated a first Warning


    Conclusions

    Conclusions

    • A non-intrusive method of drowsiness detection using steering data is possible

    • A method using ANN is developed and successfully predicts drowsiness (91% Success Rate)

    • Method is solely based on driver’s (Vehicle) steering performance

    • Same method may be applied to detection of fatigue or other related driver performance

    • Further refining and validation of the algorithm is recommended

    • Capturing individual driver’s steering while drowsy requires additional research


    Recommended additional research

    Recommended Additional Research

    • Additional Simulator Experiments

      • Validate the Developed Algorithm

      • Additional Road Conditions

      • More Diversified Group of Drivers

    • Road (Experimental) Tests in an Instrumented Vehicle

    • Further Refining the Algorithm Based on the Road Test Data

    • Testing of Other Fatigue Related Scenarios

    • Research on Warning Systems Integrated With This Detection System


  • Login