MLE, MAP, Bayes
Estimation: ML, Bayes, MAP
Principles and calculus at http://www-users.cselabs.umn.edu/classes/Fall-2008/csci5525/notes/estimate.pdf
Estimation
Simple explanation here:
09.11.10Given the evidence X, MLE considers the parameter vector Θ to be a constant and seeks out that value for the constant that provides maximum support for the evidence. ML does NOT allow us to inject our prior beliefs about the likely values for Θ in the estimation calculations.
MAP allows for the fact that the parameter vector Θ can take values from a distribution that expresses our prior beliefs regarding the parameters. MAP returns that value for Θ where the probability prob(Θ|X) is a maximum.
Both ML and MAP return only single and specific values for the parameter Θ. Bayesian estimation, by contrast, calculates fully the posterior distribution prob(Θ|X). Of all the Θ values made possible by this distribution, it is our job to select a value that we consider best in some sense. For example, we may choose the expected value of Θ assuming its variance is small enough.
The variance that we can calculate for the parameter Θ from its posterior distribution allows us to express our confidence in any specific
value we may use as an estimate. If the variance is too large, we may declare that there does not exist a good estimate for Θ.
Viterbi algorithm Demo
En excellent tutorial on Viterbi Algorithm was fount at http://www.telecom.tuc.gr/~ntsourak/demo_viterbi.htm
Covariance Matrix
Good description and simple explanation is given here
Covariance matrix
The variance of a variable is a measure of the dispersion of the values taken by the variable around its mean value.
The covariance matrix generalizes the concept of variance to random vectors, or sets of random variables.
PCA and covariance matrix both are well explained. You can see some illustrations with different covariance matrix types.
All materials are from this site http://www.aiaccess.net/English/Glossaries/GlosMod/e_gm_covariance_matrix.htm#Animation_covariance%20matrix
| Posted in Statistics | No Comments »