September 29, 2016

Lecture 7 Main Points Once Again

  • Marginal probabilities. Compute marginals of variables (given model parameters \(\mathbf{\theta}\)): \(p(x_i\mid \mathbf{\theta})=\sum_{\mathbf{x}': x_i'=x_i}p(\mathbf{x}'\mid \mathbf{\theta}).\) (or posterior distribution, aka, query probabilities)

  • Technique: Variable elimination to avoid the computational complexity that is exponential in dimension

  • Why it works
    • Use the fact that some factors only involve a small number of variables
    • By computing intermediate factors and caching the results, we avoid duplicated calculations
  • Q: What if we calculated a particular query probability \(p(x_1\mid x_6)\), and now we want to calculate \(p(x_4\mid x_3)\)? How to share the work across them.

  • A: This motivates the study of more sophisticated graph representation methods, including factor graphs and tree representation of UG.

Induced Graphs: Variable Elimination: \(p(x_1, x_6)\)