Bayesian Bags

There are 3 bags; one containing a white counter and a black one, another two white and a black, and the third 3 white and a black. It is not known in what order the bags are placed. A white counter is drawn from one of them, and a black from another. What is the chance of drawing a white counter from the remaining bag?

(Source: The Mathematical Recreations of Lewis Carroll: Pillow Problems and a Tangled Tale)

The key to solving this one—my way, not Carroll's way, at least—is knowing that, although the order in which the bags is placed is not known outright, the draws give evidence about what order might have been used and thus about what bag remains. More concretely, if $P(R = n \mid D)$, where $R$ stands for the remaining bag, $n$ is which bag it is, and $D$ stands for the particular draws observed in this case, can be determined for each value of $n$ from one to three inclusive, that is, the conditional probability that bag number $n$ remains, given that a white counter was drawn from one and a black counter from the other, then the problem is an issue of multiplying each respective probability by the more obvious probability of drawing a white counter from that possibly remaining bag, and then adding all three terms.

From Bayes' theorem, which, along with the concept of conditional probability, needs to be understood before going further, it can be known in this case that:

\[ P(R = n \mid D) = \frac{P(D \mid R = n) \times P(R = n)}{P(D)} \]

The first step towards going further is working out the numerators for each of the three probabilities. The denominator will be the sum of all those numerators. It will be assumed that any order of bags is equally possible. This means that the prior probability $P(R = n)$ is $\frac{1}{3}$ in each case. For the first bag to remain, one of two mutually exclusive events had to have happened, given the evidence: either a white tile was drawn from the second bag and a black tile from the third, or a white tile was drawn from the third and a black tile from the second. So in this case:

\begin{align*}
P(R = 1 \mid D) &= \frac{P(D \mid R = 1) \times P(R = 1)}{P(D)} \\
&= \frac{\left(\frac{2}{3} \times \frac{1}{4} + \frac{3}{4} \times \frac{1}{3}\right) \times \frac{1}{3}}{P(D)}
\end{align*}

As it has been said, $P(D)$ is not immediately relevant here. Another thing worth pointing out is that, because each prior probability is $\frac{1}{3}$, that value can be factored out of the summation that will take place in the denominator and then allowed to cancel with the same value in the numerator in each instance. So this problem is in fact the same as finding all the likelihood terms $P(D \mid R = n)$, then dividing each by the sum of all the likelihoods to find each posterior probability. Having said that, $P(D \mid R = 1)$ is equal to $\frac{5}{12}$. Repeating this process for bags #2 and #3 results in $P(D \mid R = 2)$ and $P(D \mid R = 3)$ both being $\frac{1}{2}$. This means that the sum of all the likelihoods is $\frac{17}{12}$. Then:

\begin{align*}
P(R = 1 \mid D) &= \frac{\frac{5}{12}}{\frac{17}{12}} \\
&= \frac{5}{17} \\
P(R = 2 \mid D) &= \\
P(R = 3 \mid D) &= \\
&= \frac{\frac{1}{2}}{\frac{17}{12}} \\
&= \frac{6}{17}
\end{align*}

Lastly, given the chain rule of probability, it can be known that:

\[ P(R = n, N = \text{white} \mid D) = P(N = \text{white} \mid R = n, D) \times P(R = n \mid D) \]

If, for example, $n$ is one, then the value of this equation is: $\frac{1}{2} \times \frac{5}{17}$, because the first bag contains one white and one black counter. Summing over all possible values of $n$ results in an expected value for the probability of drawing a white counter from the remaining bag (whether that one might be):

\[ \frac{1}{2} \times \frac{5}{17} + \frac{2}{3} \times \frac{6}{17} + \frac{3}{4} \times \frac{6}{17} \]

Or, rearranging:

\[ \frac{1}{17} \times \left(\frac{5}{2} + 4 + \frac{9}{2}\right) \]

So now it can be known that the final answer is $\frac{11}{19}$.