CS6800 15F: Homework 02

Assigned: Friday, September 25, 2015
Due: Friday, October 02, 2015

Last modified:


General Instructions

  1. Feel free to work with others on this assignment. However, you must acknowledge with whom you worked, and you must write up your own solutions.

  2. All of the problems below are from the text, Cover and Thomas, 2nd edition, or refer to problems from the text.


Problems

Points: 20 pts per problem.
  1. Problem 2.1
  2. Repeat Problem 2.1, part (a), for a biased coin where the probability of heads is p.
  3. Problem 2.27. Note: From the grouping property and two other (simpler) properties, one can derive entropy, axiomatically. See Problem 2.46.
  4. Problem 2.10

    Note: 2H(X) is referred to as the perplexity of X. It is a measure of the "effective" size of the range of X, with respect to the underlying distribution over this range.

  5. Problem 2.28

    Hint: You will likely make use of a convexity argument. See CT Section 2.6 for the definition of convexity; we will cover this in class as well.