
Recent Posts
Archives
Categories
Meta
Blogroll
Category Archives: Information Theory
The Bug
As I read through “Information theoretic inequalities”, it felt like the icing on the cake: a simpler proof of a result from my Master’s thesis. The result itself wasn’t the focus of my Master’s but instead facilitated the proof of a number of other results, … Continue reading
Posted in Information Theory, The Bug
Leave a comment
Ages of Three Daughters
My uncle’s in town and just reminded me of a puzzle that he’d asked me over ten years ago. A census worker goes to a house and the person who answers the door tells the census worker that she has three daughters. “I need to know … Continue reading
Posted in Information Theory, Puzzle
1 Comment
Le Poisson
It was the spring of 2003; Toby Berger and I were both in the common room of Phillips Hall, and there was half an hour before his information theory class was set to start. Toby was advising my honors thesis, a project that was exploring … Continue reading
Posted in Biology, Information Theory, Probability
Leave a comment
Passing Notes in Class
I just got back from a fourday camping trip for the Fourth. While roasting marshmallows and hiking along trails, I managed to fall into a few puzzles. Justin posed a puzzle that we’d formulated a while ago: a multiperson variation … Continue reading
Posted in Information Theory, Papers, Puzzle
Leave a comment
Walking Downhill
The problem asks you to show that . If , then the solution is quite easy. Set the gradient equal to zero and solve the system of equations for . Since the function is convex, this is the minimum point, and … Continue reading
Posted in Information Theory, Papers
1 Comment
Shannon Meets Shannon
He’s met almost everyone else: Wiener, Bode, Bellman, Carnot, Tesla, Marconi, and of course, Shortz. Bad jokes aside, in an attempt to understand the inverse water filling solution from ratedistortion theory better, I put together some rough notes attempting to connect it and the sampling … Continue reading
Posted in Information Theory, Signal Processing
1 Comment