Lectures 7&8: Statistical & Bayesian Parameter Estimation. Dr Martin Brown Room: E1k Email: [email protected] Telephone: 0161 306 4672 http://www.eee.manchester.ac.uk/intranet/pg/coursematerial/. W7&8: Outline.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Dr Martin Brown
Room: E1k
Email: [email protected]
Telephone: 0161 306 4672
http://www.eee.manchester.ac.uk/intranet/pg/coursematerial/
for some constant c
^
=>
=>
v2
v1
^
q
Eigenvalue/Eigenvector Pictureq
Quadratic Form and Singular Equationsy(t)
p(e)
s
0
e
Gaussian, Additive Measurement Noiser(t)
“residual”
x(t)
Model
q

^
+
y(t)
x(t)
output measurement
Plant
q
+
+
e(t)
e ~ N(0,s2)
y ~ N(xTq,s2)
Single variable
y
x
Data
Least squares parameter estimate
Well determined l = 12.8
l = 0.52
poorly determined
Note that the largest error bar (most uncertainty) occurs when t=2.
When t=1, the error bar is zero, because the input is identically 0.
Bayes rule can be used as a framework for parameter estimation:
p(qD) is the posterior conditional data distribution
p(q) is the prior estimate of the parameter values
p(Dq) is the data likelihood/conditional density function
p(D) is the unconditional density function
Remember, Bayes rule is derived from expressing the joint distribution in terms of the conditional and marginal distributions:
But the two terms on the left hand side are the same, so:
Note:
We wish to calculate the parameter values that maximise this quantity – its mean. Note that there is also a variance (see earlier)
Identically distributed means a common variance
Independent means that the joint distribution is formed from the product of the individual distributions
“every value is
equally likely”
p(q)
q
Uninformative Prior & MLE^
^