FinalExamofInformationTheoryin2023Fall
XuanGuang
Collating:Mathzwj
1.Letp(x,y)begivenby
?101?
1??
8?011?
?112?
??
(1)CalculateH(X),H(Y),H(X|Y)andI(X;Y).
(2)CalculateD(pX||pY)andD(pY||pX).
(3)DrawaVenndiagramforquantitiesin(1).
2.Denoteaprobabilitydistributionp=(p,p,?,p)andletp=maxp.Show
12ni
1?i?n
that:
(1)Hp??plogp?1?plog1?p.
()()()
(2)Hp??logp.
()
(3)Hp?21?p.
()()
3.ConsiderarandomvariableXthattakessixvalues?A,B,C,D,E,F?with
probabilities,,,,,.
0.30.250.20.10.10.05
(1)ConstructabinaryHuffmancodefortherandomvariableandfinditsaverage
length.
(2)ConstructaquaternaryHuffmancodefortherandomvariable[i.e.,acodeover
analphabetoffoursymbols(callthema,b,candd)]andfinditsaveragelength.
(3)Constructabinarycodefortherandomvariablebystartingwiththequaternary
Huffmancodein(2)andconvertingthesymbolsintobinaryusingthemapping
a→00,b→01,c→10andd→11.Findtheaveragelengthofthebinarycode
constructedbythisprocess.
(4)ForanyrandomvariableY,letLHbetheaveragelengthofthebinaryHuffman
co