Computational Modeling Fall 2005 For today: HomeworkSeventeenPointFive (Bayesian readings) Today's outline: 0) Bayesian inference, examples and coding For Monday: HomeworkEighteen Examples from Chapter 3 of MacKay, "Information Theory, Inference, and Learning Algorithms" (The web page explains that "The same copyright rules will apply to the online copy of the book as apply to normal books. [e.g., copying the whole book onto paper is not permitted.]" This implies that the principle of fair use applies to the online book; that is, I can distribute copies of a single chapter for class. I believe that the copyright notice on the handout is intended to refer to the whole book.) To solve the Euro problem, we have to take a few steps. Step 1: choosing among simple hypotheses Exercise 3.1, Exercise 3.4, Exercise 3.12 Trick 1: don't bother computing denominators do all the normalization at the end. Trick 2: likelihood ratio is the product of ratios Trick 3: don't bother computing priors Often we only care about the likelihoods, or the likelihood ratio. Trick 4: log likelihood is the sum of log likelihoods Step 2: inferring (continuous) parameters Exercise 3.3 Exercise 3.5 (bent coin, page 52) Trick 5: discretize the distribution of parameters Step 3: comparing a simple hypothesis with a compound Exercise 3.15 (Euro problem) Trick 6: the likelihood of a compound is the sum of the likelihoods of the components, which is the normalization constant of the update. Fun problems: Exercises 3.8 and 3.9 (Monty Hall problem and variations) Exercise 3.11 (Bayes in the courtroom) Trick 7: there is no Trick 7