Computational Modeling
Fall 2005

For today: HomeworkNineteen (Euro problem)

Today's outline:

1) FFT redux

2) confused Monty

3) estimating p_a

4) the Euro problem

For Monday (after break):

1) if you are missing any homeworks, get caught up

2) otherwise, read the optional reading at the end of
   HomeworkNineteen


FFT comments
------------

1) label axes

2) check the order of growth

several people wrote implementations of fft that look like this:

def fft(h):
    N = len(h)
    H = [fft_one(h, k) for k in range(N)]

def fft_one(h, k):
    N = len(h)
    if N==1: return h[0]
    even = h[0::2]    
    odd = h[1::2]
    W = exp(2*pi*1j*k/N)
    return fft_one(even, k) + W * fft_one(odd, k)

What's wrong with this solution?


Confused Monty
--------------

Unfortunately, this turned out to be the "confused Allen" problem.

A correct analysis requires us to consider six cases, not the
three I had on the board.

The car is behind    1               2               3

Monty
opens 2            stick             X            switch

Monty
opens 3            stick           switch            X

Of the six cases, sticking wins 2, switching wins 2, and
the contest gets cancelled twice.



Hypothesis testing
------------------

Classical analysis of the Euro problem:

1) Form a hypothesis: the Belgian Euro is biased

2) Form an alternative, "null hypothesis"

   The Belgian Euro has p_a = 0.5

3) Forget all about Step 1, because from now on, it is irrelevant.

4) Define a measurement of strangeness.

   strangeness = | actual number of heads - expected number of heads | 

5) Compute a p-value, which is the probability that a data set
   would be as strange or stranger than the actual data,
   conditioned on the null hypothesis.

6) If the p-value is lower than 0.5%, you can "reject the
   null hypothesis at a 95% confidence level"

   6a) Alternatively, some people think it is ok to compute
       a p-value and then "reject the null hypothesis at the 
       1-p confidence level."

7) Notice that Step 6 doesn't actually tell us anything about
   the orginal hypothesis.

8) Notice that this analysis isn't exactly wrong; it just asks
   the wrong question.

9) Notice that this analysis involves several choices that are
   at least as arbitrary as choosing a prior.

10) Stop using classical hypothesis testing (at least for problems
    like this).


Bayesian analysis requires the user to state the alternative
hypotheses carefully, which is a good thing.

H0: the coin is unbiased, so p_a = 0.5 +- epsilon

H1: the coin is biased, but I don't know what the bias is
    (uniform prior, maybe exclude the middle)

Comparing these two hypotheses, the data lend some evidence
in favor of H0.


Alternatively, someone might have posed

H1': the coin is biased, and I think p_a is between 0.55 and 0.75

Comparing H0 and H1', the data lend weak evidence in favor of H1'

"Weak evidence" means that people with different priors might
(still) not agree after seeing this data.