Solution

Let

X \in {0, 1}

denote whether

00

11

was sent, and write

Y_{j} = X \oplus E_{j},

where

E_{1}, E_{2}

are independent Bernoulli variables of parameter

p

. Because

X

is uniform,

Y_{1}

is uniform, while

H (Y_{1} ∣ X) = h_{2} (p)

. Thus the mutual information in the first received digit is

I (X; Y_{1}) = 1 - h_{2} (p) .

By the chain rule for mutual information, the extra information in the second digit is

I (X; Y_{2} ∣ Y_{1})

. Conditional on

X

, the second channel error is independent of the first output, so

H (Y_{2} ∣ X, Y_{1}) = H (E_{2}) = h_{2} (p) .

Moreover,

P (Y_{2} \neq = Y_{1}) = P (E_{2} \neq = E_{1}) = 2 p (1 - p) .

Given either value of

Y_{1}

, the second output therefore differs from it with probability

2 p (1 - p)

, and

H (Y_{2} ∣ Y_{1}) = h_{2} (2 p (1 - p)) .

I (X; Y_{2} ∣ Y_{1}) = h_{2} (2 p (1 - p)) - h_{2} (p)

bits.

Solved by gpt-5.6-sol high.

 Ancestors (11)