Telecom Network Fault Prediction

1. Telecom Network Fault Prediction H. K. Yuen Department of Management Sciences City University of Hong Kong

2. Outline • Problem Formulation • Variable Selection • Model Development • Model Implementation

3. Problem Formulation • Overview • Messages about network performances are generated from transmission stations • Messages are examined manually • Messages are classified as urgent fault or non-urgent fault • To build a model to predict whether a received signals an urgent fault or not

4. Problem Formulation • The Data • 5,924 past messages were collected • Each message contains 1,082 variables • Each message was examine manually • The decision "Urgent" or "Non-Urgent" was set as the target variable • Urgent case = "True" Non-Urgent case = "Null"

5. Problem Formulation • Distribution of the Target Variable Null True

6. Problem Formulation • Selection of Cases • Use the Sampling node of Enterprise Miner (EM) to select a sample

7. Variable Selection • Using all of the variables in the model is not practical • Impractical to examine the associations between the target variable and the other input variables manually • The Tree node and the Variable Selection node of Enterprise Miner were employed

8. Variable Selection • Process flow

9. Variable Selection • Some results from Tree1 • A total of 23 variables are selected as input

10. Model Development • Data are partitioned into three parts • Training (50%) • Validation (25%) • Testing (25%) • Two possible model selection criteria: • The one that most accurately predicts the response (either "True" or "Null") • The one that generates the highest expected profit

11. Model Development • Modified Profit Vector • Neural Network models with different setting were developed • Model Output: Prob(Target variable="True")

12. Model Development • Process flow • Model Manager

13. Model Development • How to choose a model with the most predictive power? • Sensitivity: # of predicted "True" / # actual "True" • Specificity: # of predicted "Null" / # actual "Null" • Cutoff point: Observations with predicted probability of the target event greater than a cutoff point are classified as "True"

14. Winner Lower Cutoff Higher Model Development • Receiver Operating Characteristic Chart (ROC) All "True" Sensitivity All "Null" 1-Specificity

15. Model Development • Correct Classification Chart • Displays the prediction accuracy for each actual target level across a range of cutoff values Cutoff

16. p  0.5 p  0.15 else Class 1 Class 2 Class 3 Send technician Examine the signal manually Ignore the signal Implementing the Model An incoming signal with predicted Prob(target variable = "True) = p

17. Actual Implementing the Model • Results of classification • Benefits: • Saving in manpower • Faster response time to problems