Marginal probabilities. Compute marginals of variables (given model parameters \(\mathbf{\theta}\)): \(p(x_i\mid \mathbf{\theta})=\sum_{\mathbf{x}': x_i'=x_i}p(\mathbf{x}'\mid \mathbf{\theta}).\) (or posterior distribution, aka, query probabilities)
Technique: Variable elimination to avoid the computational complexity that is exponential in dimension
- Why it works
- Use the fact that some factors only involve a small number of variables
- By computing intermediate factors and caching the results, we avoid duplicated calculations
Q: What if we calculated a particular query probability \(p(x_1\mid x_6)\), and now we want to calculate \(p(x_4\mid x_3)\)? How to share the work across them.
A: This motivates the study of more sophisticated graph representation methods, including factor graphs and tree representation of UG.