9/3/03
6
Maximum and Minimum
Entropy
¥
Maximum entropy is achieved when all
signals are equally likely
Ð
No ability to guess; maximum surprise
Ð
H
max
= lg
N
¥
Minimum entropy occurs when one symbol
is certain and the others are impossible
Ð
No uncertainty; no surprise
Ð
H
min
= 0