lore-oriented programming: the good, the "meh", and the ugly of haskell tuning

Following my earlier pithy musings on haskell, I thought I would expand the topic matter to accommodate my shallow thoughts on something I only hinted at in that post - namely the dark art of tuning haskell.

disclaimer: before you proceed, remember that this is my personal blog, I do not claim to be a haskell master, and you are not obligated to read anything I write or have an opinion on it. this is not reddit link-bait...if you waste a day discussing this on reddit, that is your own business.

2012-12-03: discussion on reddit

As the title implies, haskell tuning is an art located deep in the recesses of lore-oriented programming. What is "lore-oriented programming"? My own term for any tool whose effective use relies on lore - secrets, legends, things-left-out-of-the-docs, etc. My tool of choice in the 90s - perl, is a classic case of a lore-oriented tool. Implicit variables, default args, default behavior, default coercions, cryptic regexes, tips, tricks and techniques that befuddle the junior programmer and provide job security for those in-the-know. Don't get me wrong - I still love perl, and I'm not at all put-off by its status as a lore-oriented language, but lore-oriented it is.

Haskell tuning is also lore-oriented. You'll find some scattered wiki pages, some long-forgotten threads in haskell-cafe, some stackoverflow posts...all of which constitute the lore of tuning haskell, and almost entirely non-obvious to the average coder. I won't enumerate all of the techniques here - I can't, I'm not a haskell master and it is lore after all. I do offer my thoughts on what I have been exposed to in the past.

Before you proceed, I am fully aware of the fact that haskell was not originally conceived as a performance/efficiency focused tool. I do try to take haskell at face-value, so don't look at any of my criticisms as damning the overall value of haskell as a platform for experimentation. on to the lists...

the good

• types - the strong type system provides a lot of instances that tend to map well to real resources, which offers the dual promise of efficiency and safety.
• unboxing - this is a reliable technique that tends to deliver consistent results and doesn't violate the spirit of idiomatic haskell.
• concurrency/parallelism - haskell has incredible support for exploiting cutting-edge techniques to improve program performance using concurrency and parallelism. I find this support to be consistent with the spirit of idiomatic haskell.
• lazy/strict libraries - in many cases, library authors have offered both lazy and strict variants of their packages. If you know what you are doing, you can effectively exploit these choices for effective tuning.
• monad/pure distinction - helps in isolating behaviors, which can provide for reliable tuning results.
• tools - threadscope, criterion, etc.

the "meh"

• "preferred" libraries - very much steeped in lore, and not uniformly distributed in hackage. You might be smart and use Data.Text in place of String in your code, but what about all of the libraries you are using? Same with Vectors vs plain lists, etc etc. An obvious case here to make a major clean break in haskell and make the preferred approach the default.
• strictness annotations - just as likely to explode in the face of someone misusing them.
• never-ending space leaks - a "meh" because it should be obvious even to intermediate coders that a lazily-evaluated language will use a lot of memory. That said, these leaks are almost impossible to get rid. space leaks severely limit where haskell can be deployed. I can't deploy a haskell web-server for example, that uses five times the memory per connection as a java web server...this places an artificial limit on how many users I can serve. With enough work I can eliminate a lot of these leaks...or just use java and move on to other tasks. (2012-12-03: reddit user vagif disagrees)
• types - (again!) from time to time we all feel like we are serving the type system instead of it serving us. When misused, the type system can create inflexibilities that make it impossible to tune code.

the ugly

• Foreign.* - Foreign is of course part of how haskell provides a great FFI, but I find it is abused as an escape hatch to make haskell programs faster. For honesty's sake, all shootout contributions that feature Foreign libraries should be rewritten. They are as much of a cheat as the php contributions that lean on c libs. I really wish the shootout was tightened up in this regard.
• unsafe* - I consider the unsafe* family of functionality to also be a major cheat.
• UNPACK - self-explanatory.
• hidden traps - why just today on haskell-cafe, it was revealed that div should be replaced by quot in certain situations. Who is going to think of this? I'm certain many of the haskell masters themselves were surprised to read this. Every language has hidden traps, haskell isn't alone here.

last update 2012-12-02