Information Theory

Step 1: Define Entropy. The entropy of a source is a measure of its average information content per symbol. For a source with M possible symbols, the entropy H is given by: where is the probability of the i-th symbol.
Step 2: Understand the condition for maximum entropy. The entropy is maximized when all symbols are equally likely. In this case, for all i. The maximum entropy is then:
Step 3: Apply the formula to the given problem. The question asks for the entropy associated with \textit{one image pixel}. A single pixel is the "symbol". The number of possible symbols (distinguishable intensity levels) is M = 8. The maximum entropy occurs when each of the 8 levels is equally probable. Since , The unit of entropy is bits per symbol (in this case, bits per pixel). The total number of pixels ( ) is irrelevant for calculating the entropy of a single pixel.