Loading in 2 Seconds...
Loading in 2 Seconds...
Linear and NonLinear Regression and Estimation Techniques. Course presentation EE-671 Presented by- Manesh Meena. Content. Introduction General Regression model Parameter estimation in LR Nonlinear and weighted Regression NARMAX modeling
Presented by- ManeshMeena
General Regression model
Parameter estimation in LR
Nonlinear and weighted Regression
Route training in mobile robotics through system identification
Application of Regression analysis
-x is the predictor (independent) variable
-y is the response (dependent) variable
We speciﬁcally want to indicate how y varies as a function of x.
Additional variables may be considered for the purpose of reducing the prediction errors on the predicted value of y. For example, Revenue, Gender, Annual milage, Number of children etc...could be included in the regression function. So, quite generally, doing regression is looking for the "best" function :
y = f(x1, x2, ..., xp)
There are two main ways of calculating the parameters of a regression model :
y(x) = m(x) + ɛ(x)
-Natural variations due to exogenous factors
Therefore, y(x) is also a random variable
The error is additive
These assumptions are only on the error term.
ɛ(x) = y(x) − m(x)
How to Estimate parameter m(x)??
Example: Relating Shoe Size to Height using footprint impressions
Simple linear regression assumes that m(x) is of the parametric form
m(x) = β0 + β1x
which is the equation for a line.
Which line is the best estimate??
yi = β0 + β1*xi + ɛi (i = 1, 2, . . . , n)
Where yi ≡ y(xi) is the response value for observation i,
β0 and β1 are the unknown parameters (regression coefﬁcients),
xi is the predictor value for observation i
ɛi ≡ ɛ(xi) is the random error for observation i
Deﬁne a Loss Function L(y(x), g(x)) ,
which describes how far g(x) is from y(x)
The Risk or expected loss is R(x)=E[L(y(x),g(x))]
The best predictor minimizes the Risk (or expected Loss)
g∗(x) = arg min E[L(y(x), g(x))]
Nonlinear regression takes the general form
y(x) = m(x; β) + ɛ(x)
for some speciﬁed function m(x; β) with unknown parameters β.
Making same assumptions as in linear regression (A1-A3), the least squares solution is still valid.
Non-linear regression is an iterative procedure in which the number of iterations depend on how quickly the parameters converge.
Consider the risk functions we have considered so far
R(β) = ∑(yi − m(xi; β))^2
Each observation is equally contributes to the risk
Weighted regression uses the risk function
so observations with larger weights are more important
To represent nonlinear models NARMAX(nonlinear autoregressive moving average with exazenous input) representation is used.
For multiple input, single output noiseless systems, this model takes the form
where y(n) and u(n) are the sampled output and input signals at time n respectively, Ng and Na are the regression orders of the output and input respectively. f() is a non-linear function.
The notation ARMA(p,q) refers to the model with p autorgressive terms and q moving average terms
This model contains AR(p) and MA(q) models.
The NARMAX methodology breaks the modeling problem into the following steps:
Determine model structure and parameters based on estimation dataset.
Validate the model using validation dataset.
The initial structure of NARMAX polynomial is determined by the inputs u and output y and the input and the output time-lags Nu and Ng.
The general rule in choosing the suitable inputs for the model is that at least some of them should be causing the output.
But not all of them are significant contributors to the computation of the output.
The final structure of the estimated NARMAX model will indicate only significant inputs.
Before any removal of the model terms an equivalent auxiliary model is computed from the original NARMAX model. The model terms of the auxiliary model are orthogonal.
The calculation of the auxiliary model parameters and refinement of the model’s structure is an iterative process.
Each iteration involves three steps.
1. Estimation of model parameters using the estimation dataset.
2.Model validation using the validation dataset.
3.Removel of noncontributing terms.
After the model validation step, if there is no significant error between the model predicted output and the actual output, non-contributing terms are removed in order to reduce the size of the polynomial.
To determine the contribution of a model term to the output the Error Reduction Ratio (ERR) is computed for each term, which is the percentage reduction in the total mean-squared error as a result of including the term under consideration.
Model terms with the ERR under certain threshold are removed from the model polynomial during the refinement process.
In the following iteration if the error is higher as a result of last removal of the model term then these are reinserted back into the model and the model equation is considered as final.
Finally NARMAX model parameters are computed from the auxiliary model.
1. initially the robot was driven manually several times through the specific route to be learned.
2. During this time robots sensor values and rotational velocities were logged.
3. The data collected was then used for estimation and validation of NARMAX model.
4. then the model was put on the robot and executed in order to record a further set of data that was used to test the model’s performance.
After manual control for 1 hour all the sonar and laser measurements were taken.
The values delivered by the laser scanner were averaged in 12 sectors of 15 degrees each(laser bins) to obtain a 12 dimensional vector of laser spaces.
These laser bins as well as 16 sonar values were inverted so that large values indicate close-by-objects.
Finally, the sonar and laser readings at each instant were normalized by minimum sonar and laser readings respectively at that instant.
All these values are input into the model.
This time only laser sensor was used and pre-processed same as in route-1
The characteristics of the NARMAX model obtained are Nu=0, Ny=0, Ne=0 and degree=3.
The initial model had 573 terms but just 94 remained after removal process of non-contributing terms.
Statistical space occupancy tests along x and y axis confirms that there is no significant difference between the two trajectories(~.05)
This time robot had to go through two narrow passes and many symmetries.
To obtain the model normalized and inverted bins were used.
The best NARMAX model uses Nu=0, Ny=0 and degree of polynomial=2.
The initial model had 97 terms but after removal of non-contributing ones just 73 remained.
Once again the model was properly able to learn the trajectory with no significant difference in the space occupancy.
In this route robot had to start from position labelled A and had to reach to point labelled B.
TO model the route’s behaviour ARMAX modeling was used which is the linear polynomial equivalent of NARMAX i.e. degree of polynomial is one.
In this experiment regression order of output (Ny) was 0 and that of input(Nu) was 8.
This model has successfully learned the route from A to B which was again confirmed using statistical analysis.
Trend line analysis
Risk analysis for investment
“Route Training in mobile robotics: System Identification”- Ulrich Nehmzow and S. Billings
is linear or nonlinear?