The Incorporation of Prior Knowledge and Constraints through Bayesian Analysis into a Neuro-Fuzzy In...
Download
1 / 28

Tom Musicka Centre for Process Analytics and Control Technology University Of Newcastle - PowerPoint PPT Presentation


  • 132 Views
  • Uploaded on

The Incorporation of Prior Knowledge and Constraints through Bayesian Analysis into a Neuro-Fuzzy Inference System Framework. Tom Musicka Centre for Process Analytics and Control Technology University Of Newcastle Contact : [email protected] Introduction. Problem Definitions

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Tom Musicka Centre for Process Analytics and Control Technology University Of Newcastle' - uyen


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

The Incorporation of Prior Knowledge and Constraints through Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

Tom Musicka

Centre for Process Analytics and Control Technology

University Of Newcastle

Contact : [email protected]


Introduction
Introduction Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Problem Definitions

  • Data Analysis

    • PreScreening

    • Data Validation

  • Neuro-Fuzzy Modelling

  • Model Comparison

    • Results

  • Bayesian Framework

    • Results

  • Conclusions

  • Future Work


Data mining
Data Mining Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Data Capture and Treatment

  • ‘Analysis of observational data sets to find unsuspecting relationships and to summarise the data in a number of understandable and useful ways’

    • Prediction

    • Classification

    • Detection of Relationships

    • Explicit Modelling

    • Clustering

    • Deviation Detection


Corus process and scheduling description
Corus - Process and Scheduling Description Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Hot Strip Steel Rolling - Corus

    • Multigrade Operation

    • Multistage Process

      • Reheating

      • Multi-Pass Reversing Rougher

      • Multi-Pass Finishing Rolling

      • Run Out Table (ROT) Cooling

      • Coiling

      • Testing


Corus problem definition
Corus - Problem Definition Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Virtual Test House

  • Mechanical Property Prediction

    • Ultimate Tensile Strength

    • Yield Stress

    • Elongation

  • Project Description:

    • ‘To produce a modelling system capable of accurately and consistently predicting the the mechanical properties of Steel rolled at Hot Strip Mills within Corus UK.’


Shell process and scheduling description
Shell - Process and Scheduling Description Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Refinery High Viscosity Index (HVI) Operation - Shell

    • Multicrude Operation

    • Continuous and Batch Operations

    • Pressure to Minimise Working Capital

      • Statistical Visualisation

        • Masters Thesis

    • Four Unit Process

      • High Vacuum Unit (HVU)

      • Furfural Extraction Unit (FEU)

      • Propane De-Asphalting Unit (PDU)

      • MEK De-Waxing Unit (MDU)


Shell problem definition
Shell - Problem Definition Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Slack Wax Oil Content (SWOC) Prediction

  • MDU Produces Lube Oil and Slack Wax

    • Slack Wax Contains Oil

    • Useful to know the % Oil Content

  • Project Description:

    • ‘To produce a model capable of accurately and consistently predicting the SWOC. This Model will be considered within the feasibility report for the purchase of an online physical measurement system’


Data characteristics
Data Characteristics Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Multiple Input, Single Output Models

  • Discrete and Continuous Variables

  • Missing Data

  • Process Noise

  • Outliers

  • Multiple Operating Regions

    • Strong Clustering

    • Overlapping Regions

    • Local Modelling Techniques

  • Non-Linear Relationships


Data screening
Data Screening Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Univariate Checks

  • Time Series Visualisation

  • Multivariate Statistics

    • Principal Component Analysis

    • Outlier Detection

    • Process Visualisation

  • Multi-Modal Data

    • Local PCA Models

      • Process Grades

      • Fuzzy Clustering

    • MixPCA


Data analysis principal component analysis
Data Analysis - Principal Component Analysis Bayesian Analysis into a Neuro-Fuzzy Inference System Framework


Local principal component analysis
Local Principal Component Analysis Bayesian Analysis into a Neuro-Fuzzy Inference System Framework


Data splitting
Data Splitting Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Data is Divided into 3 Data Sets

    • Training

      • Data to Build Model

    • Validation

      • Data used to Define Training Stopping Epoch

      • Improve Generalisation

    • Testing

      • Data Used to Quote Model Results

      • Unseen by Model in Training Phase

    • Ratio 70:15:15


Data splitting1
Data Splitting Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Outliers Removed

    • Fully Populated Data Set

  • Require Representation of Full Scope of Process Within Training Data

    • Random Selection

      • Sample from Data Distribution

      • With Respect to Operating Region

    • Kennard-Stone Selection

      • Sample from Flat Distribution

      • Mahalanobis Distance Metric for Selection

      • Analysis for Each Operating Region

      • Ensures Training Set Covers Full Scope


Advanced modelling studies
Advanced Modelling Studies Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Projection to Latent Structures

    • Linear

    • Predictive Extension of PCA

  • Neural Networks

    • Non-linear

    • ‘Black-Box’

  • Neuro-Fuzzy Inference Systems

    • Adaptive Neuro-Fuzzy Inference System - ANFIS

    • Fuzzy Rulebase - Model Switching / Merging

    • Local Linear Models

  • Compromise between Accuracy, Interpretation and Simplicity


Anfis structure

Input Data Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

S

Input Fuzzification

Normalised Rulebase

Local Models

Network Output

ANFIS Output

ANFIS Structure


Corus model comparison results
Corus - Model Comparison Results Bayesian Analysis into a Neuro-Fuzzy Inference System Framework


Shell model comparison results
Shell - Model Comparison Results Bayesian Analysis into a Neuro-Fuzzy Inference System Framework


Model comparison results corus and shell
Model Comparison Results - Corus and Shell Bayesian Analysis into a Neuro-Fuzzy Inference System Framework


Bayesian methods
Bayesian Methods Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Mathematical Foundation to Model Optimisation

  • Incorporates Use of Prior Knowledge

  • Advantage over Maximum Likelihood Models

    • Bayes Rule

      • Posterior = (Likelihood * Prior)/Evidence

    • Evidence is Constant for Given Model Structure


Bayesian methods1
Bayesian Methods Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Posterior - p(w|D,H)

    • Probability of Parameters, w, Representing Data, D, for Given Model, H

  • Likelihood - p(D|w,H)

    • Probability of the Model Fitting the Data

    • Accounts for Least Squares Modeling Power

  • Prior - p(w|H)

    • Incorporates Prior Knowledge into Training

    • Model Constraints

    • Independent of Data


Bayesian methods2
Bayesian Methods Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Bayesian Model Averaging

    • Accounts for Model Uncertainty

    • Avoids Use of Single Model

  • Prior Distribution

    • Incorporation of Process Knowledge

    • Favour Certain Characteristics

      • Model Interpretability

      • Model Accuracy


Bayesian anfis banfis
Bayesian ANFIS - BANFIS Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Single ANFIS Structure

    • Process Knowledge

    • Cluster Analysis

  • Bayesian Model Averaging

    • Sample Rulebase Parameters

      • Markov Chain Monte Carlo

        • Rejection Sampling

      • Single Local Model Learning Stage

      • Cover Important Parameter Space


Banfis results comparison
BANFIS - Results Comparison Bayesian Analysis into a Neuro-Fuzzy Inference System Framework


Banfis results comparison corus
BANFIS - Results Comparison - Corus Bayesian Analysis into a Neuro-Fuzzy Inference System Framework


Banfis results comparison shell
BANFIS - Results Comparison - Shell Bayesian Analysis into a Neuro-Fuzzy Inference System Framework


Banfis advantages and disadvantages
BANFIS - Advantages and Disadvantages Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Advantages

    • Improved results

    • Incorporates Parameter Uncertainty

    • Avoids Optimisation Training Problems

    • Avoids Single Network Solution

    • Incorporation of Prior Knowledge

  • Disadvantages

    • Complex Hierarchical Analysis

    • Network Interpretation Issues


Conclusions
Conclusions Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • Data Validation Methods

    • Screening

    • Local Models

    • Essential for Non-linear Modelling Methods

  • Compromise between:

    • Accuracy

    • Interpretation

    • Model Complexity

  • Results

    • Improvements in Accuracy and Data Analysis

    • Implementation of Bayesian Framework


Acknowledgements
Acknowledgements Bayesian Analysis into a Neuro-Fuzzy Inference System Framework

  • CPACT

  • TCD

  • Paul Kitson - Corus

  • Phil Jonathan, Paul Blackhurst - Shell


ad