After having toyed with a few vaguely-functional languages over the past few years (Scala being a personal favourite), I decided to get my hands dirty with my old favourite from my undergrad days: Haskell. I fondly remember evenings of page-long lambda-calculus beta reductions in preparation for the final FP module exam. I seemed to be the only one in the year who enjoyed lambda calculus. The idea of describing what something *is*Â rather than the steps to achieve it (as in imperative programming) is very persuasive and elegant, and the `brutal purity’ of Haskell (a term I borrowed from John Carmack) forces you to think in this way.

But I digress. Part of my work over the past six months or so has related to information-theoretic measures. Although I’ve implemented efficient entropy computations in a couple of different architectures (raising a few interesting asides which I’ll probably write up in future posts), I had fun trying out a possibly naive shannon entropy definition in Haskell which I’m hoping will serve the purpose of reminding me of just how bad I was at the start of this learning experience!

1 2 3 4 5 6 7 8 9 10 11 |
-- shannon entropy for input shannon :: [Int] -> Float shannon xs = -sum [ p * logBase 2 p | p <- pmf xs ] -- probability mass function for input pmf :: [Int] -> [Float] pmf xs = [(fromIntegral l) / (fromIntegral tot) | (x,l) <- bins xs] where tot = (sum [l | (x,l) <- bins xs]) -- convert to unique set with counts as (el,cnt) bins :: [Int] -> [(Int,Int)] bins = map (xs -> (head xs, length xs)) . group . sort |

I’m hoping that in time as I work my way through this excellent book that I can reduce these definitions somewhat. I’m guessing that there is a cheaper way to produce a binned distribution for example using whatever set types are available. I’m at least happy thatÂ binsÂ is a function composition for now.