Baye's Rule

\[ P(\theta | x) = \frac{P(x | \theta) P(\theta)} {P(x)} \]

There is a disease that you can get tested for. Before seeing the results of the test, my prior belief that I have the disease is \(P(D)\). This belief is the sum of two things:

1. The belief that I have the disease and my test is positive = \(P(+|D)P(D)\) Here, you can think of \(P(+|D)\) as the fraction of my belief devoted to this case.
2. The belief that I have the disease and my test is negative = \(P(+|D)P(D)\)

If I test positive, then I now should re-weight my beliefs. I now consider my previous beliefs in the two cases where I test positive:

1. I test positive and I do not have the disease = \(P(+|\bar{D})P(\bar{D})\)
2. I test positive and I have the disease = \(P(+|D)P(D)\)

I need to re-weight these beliefs (disease and no disease) so that they add to 1. The belief that I have the disease is then \[\frac{P(+, D)}{P( +)} = \frac{P( + | D)P(D)}{P( + )} \]

This is a very rational way to update my beliefs, because I am simply taking my old beliefs, discarding the ones I now know to be untrue, and re-weighting my remaining beliefs.