Category Archives: Information Theory

Walking Downhill

Posted on July 12, 2010 by

The problem asks you to show that . If , then the solution is quite easy. Set the gradient equal to zero and solve the system of equations for . Since the function is convex, this is the minimum point, and … Continue reading →

Posted in Information Theory, Papers | Leave a comment

Shannon Meets Shannon

Posted on July 6, 2010 by

He’s met almost everyone else: Wiener, Bode, Bellman, Carnot, Tesla, Marconi, and of course, Shortz. Bad jokes aside, in an attempt to understand the inverse water filling solution from rate-distortion theory better, I put together some rough notes attempting to connect it and the sampling … Continue reading →

Posted in Information Theory, Signal Processing | Leave a comment