OTAS Blog

  • Market Commentary
  • New Product Features
  • Function Of The Week
  • Reports
  • Quantitative Developers
  • About us

Experiences with Haskell Development at OTAS Technologies

Posted by Alex Bates on March 2, 2015
Posted in: Quantitative Developers. Tagged: Haskell.

This post discusses my experiences as a software engineer at OTAS Technologies, developing almost exclusively in Haskell for the past six months. Prior to this I’d learnt about and used functional languages solely in academia, in the context of type theory, with little development experience. Here are my views on some pretty fundamental issues I deal with day-to-day.

Tooling

The Haskell toolchain is not simple. Certainly, it took me several weeks before I adopted the system I currently use, although happily this system works successfully. The Haskell platform ships with GHC and Cabal. Cabal is the Haskell build system and dependency manager, and it is at once powerful and fragile. Usage of Cabal is well documented and its operation is not complicated, and yet many resources and tutorials do not stress the importance of sandboxes in Haskell development.

Sandboxes provide a mechanism to isolate the build process of individual Haskell projects. By default Cabal will use a shared (global) package database for storing compiled packages, dependencies of your library/program. However, sharing compiled packages leads to breakages when their dependencies are updated, an affliction known as `Cabal hell’. Sandboxes solve this issue by isolating (sandboxing) compiled dependencies per project, at the expense of vastly longer setup times, as all dependencies need to re-downloaded and built. However, their use is crucial in Haskell development and I’m unsure why this isn’t enforced by Cabal.

Isolated build environments with sandboxes are half the story to reliable Haskell development. It is necessary to future-proof your packages, ensuring upgrades to dependencies break neither your package or another dependency. `Stackage’ is a Hackage alternative that provides restrictions on packages, ensuring they are mutually-compatible. By restricting your projects to package dependencies specified by the Stackage snapshot you guarantee your package is buildable for all time. Certainly I make use of these snapshots in all my Haskell projects. I have not yet needed to upgrade a snapshot to use a newer version of a dependency, although I anticipate the process will be simple. Certainly Stackage enables me to upgrade dependencies in a controlled fashion.

I currently advise Haskellers to install the Haskell toolchain local to their user, and not through an OS-level package manager. The reason for this is twofold. Firstly, package managers frequently have an outdated version of GHC. Secondly, package managers also provide packages of popular Haskell libraries, which are then installed to the global package database. This interferes with the sandbox approach described above, and the two are not compatible.

Note that I have not explored packaging solutions such as Nix, as I find my current approach satisfactory, although I will be interested in seeing how the project develops.

Libraries

The Haskell ecosystem includes some fantastic software, available for instant use as libraries served by Hackage. Spend enough time using Hackage and you become familiar with many of the authors! A key skill when developing Haskell projects is identifying libraries that match your needs, the architecture of your program, and your other dependencies. Many people would expect the functionality provided by some of these libraries to be shipped with the compiler (`random’ and `time’ come to mind..). However, I see it as an advantage that core functionality is spread across many independent developers, as features are developed and pushed faster. The ecosystem certainly moves fast!

Certain libraries provide types and combinators that seemingly alter the very language. The prototypical example is Edward Kmett’s `lens’ library, which provides an extremely general, elegant and powerful framework for navigating data structures. Usage of the lens library is straightforward, but the underlying mathematical concepts are smart, and very interesting to learn about.

It takes experience to understand the gravity of your choice of library. There are libraries which attempt to perform the same functionality (pipes and conduit, warp and happstack etc.) but are architected in vastly different ways, such that converting between the two is far from trivial. For example, the lens library forms the sole interface to my OTAS Base wrapper, and is so heavily used that upgrading the dependency version requires careful consideration.

Using Haskell day-to-day

Haskell development is, frankly, awesome. When I describe my job, I frequently say I’m `living the programming dream’ (having the autonomy to choose my software tools and having full control over my projects are two reasons for my happiness, too). Programming in a purely functional language allows you to produce smarter, more correct code, period. The type system forces you to request and record the behavior of your function in its type. If your function needs access to a read-only environment, wrap its type in the Reader transformer. Need to log? Use the Writer monad. If you need to characterise failure, incorporate an Error type into your stack.

Because this correctness and clarity stems from restricting the behaviour of your functions, you do need to think carefully about what exactly they need to do. In particular, adding a feature to the software can often require a fundamental change in the monad transformer stack at the heart of your program. Therefore it is imperative you carefully consider the direction the project will take even before you begin development, and always consider the implications of the types you choose.

One source of difficulty for me is knowing to what extent I should generalise my types. This is often the case with monadic stacks based on IO. Annoyingly, core IO actions provided by the Haskell Prelude are of a concrete IO type (e.g. putStrLn “Hello, world!” :: IO ()), and one must explicitly import and use `liftIO’ from the `MonadIO’ typeclass when performing IO actions deeper in the stack. Handling exceptions requires similar machinery. Ideally, all IO actions would utilise generalised typeclasses such as MonadIO, `MonadBase’ and `MonadBaseControl’ as fully general frameworks for exception handling and IO actions. However, prematurely generalising your types complicates development, and furthermore many libraries do not currently support the most general interface possible. In practice fully general types leads to much cleaner code, although reasoning about the types and debugging becomes more difficult.

Finally, I’m beginning to notice limitations in the language which would greatly improve some aspects of development, although it seems unlikely they will see a solution in the near future. Foremost in this is Haskell’s record system. I would like to see a lens-based anonymous record system implemented in GHC, which would solve the overloaded records problem and enable `first class’ record manipulations. Alas, this would be a new dialect of Haskell! Additionally, the Prelude supports many outdated and inefficient concepts deemed beginner friendly (the String type, for example) and which are only now being improved upon (the applicative-monad proposal and the foldable-traversable proposal are two key imminent changes).

Certainly though, my experience with Haskell has been extremely positive. I’d be skeptical of anyone disregarding Haskell for any problem domain due to performance (except perhaps embedded or OS-level software) or through lack of libraries. Certainly at OTAS, barring exceptional circumstances I’ll be encouraging all future projects to be implemented in Haskell.

Posts navigation

← Record NASDAQ Insider Buying Last Month
NASDAQ Looks Cheap At 5000 →
  • Recent Posts

    • Navigating Event Risk
    • Data Rules, OK !
    • Carrefour : The not-so super, market
    • Tracking and Auditing Traders’ Decision-Making
    • Intraday Screener – Identifies which orders need a bit more TLC
  • Archives

    • October 2017
    • September 2017
    • July 2017
    • June 2017
    • May 2017
    • April 2017
    • March 2017
    • January 2017
    • December 2016
    • November 2016
    • October 2016
    • September 2016
    • August 2016
    • July 2016
    • June 2016
    • May 2016
    • April 2016
    • March 2016
    • February 2016
    • January 2016
    • December 2015
    • November 2015
    • October 2015
    • September 2015
    • August 2015
    • July 2015
    • June 2015
    • May 2015
    • April 2015
    • March 2015
    • February 2015
    • January 2015
    • December 2014
    • November 2014
    • October 2014
    • September 2014
  • Authors

    Simon Maughan
    Simon
    Maughan
    Ben Lock
    Ben
    Lock
    OTAS Tech
    OTAS
    Tech
    Hugo Martay
    Hugo
    Martay

    Alex
    Bates

    James
    Whitehead
Proudly powered by WordPress Theme: Parament by Automattic.