All posts tagged Hackage

Here at OTAS, we run a private, internal instance of Hackage to easily share our Haskell packages and documentation amongst our developers. It’s really a core part of our Haskell development workflow and, once fully set-up, integrates seamlessly with Stack. Unfortunately, I’ve found deploying the server to be tricky and poorly-documented, so I’ve written this guide to help make the process easier.

We use Stack for building all our Haskell projects, so we do not explicitly install either GHC or cabal-install on our machines. I’ll assume you have a working installation of Stack on your machine before you begin. Cabal-install users should be able to follow easily enough.

Building the Hackage Server

I’ve found it very frustrating to build the hackage-server packages listed on Hackage. At the time of writing, two version, 0.4 and 0.5.0, are listed, both of which have an extended list of dependencies which Cabal’s solver isn’t able to resolve for me. As such, I went directly to the Github source and decided to build the most recent commit at the time of writing, e0bd7fc2a8701a807a5551afc1ab1e9df4179e81.

git clone git@github.com:haskell/hackage-server.git 
cd hackage-server
git checkout e0bd7fc2a8701a807a5551afc1ab1e9df4179e81

Stack successfully generated this stack.yaml file (using stack init --solver) which you can copy into your working directory to reproduce the build. Simply run stack build to generate the executables.

Running Hackage

Before we’re ready to initialise the server we need to generate private keys using hackage-repo-tool. First build and copy this program to your path with Stack then generate and install the keys.

hackage-repo-tool create-keys --keys keys
cp keys/timestamp/.private datafiles/TUF/timestamp.private
cp keys/snapshot/.private datafiles/TUF/snapshot.private
hackage-repo-tool create-root --keys keys -o datafiles/TUF/root.json
hackage-repo-tool create-mirrors --keys keys -o datafiles/TUF/mirrors.json

Now we’re ready to initialise the server! In the following command, substitute the administrator username and password with your chosen credentials.

./hackage-server init --admin=user:password --static-dir=datafiles/

Finally, run the server on your chosen port.

./hackage-server run --port=4050 --static-dir=datafiles/

Now that your server’s up and running (and assuming your firewall’s correctly configured) you’ll be able to access the local Hackage instance through your browser. There you’ll find documentation for administrative functions such as user management. This wiki page may be helpful.

Automatically Building Documentation

We use hackage-build to automatically generate documentation for uploaded packages. This program requires cabal-install, its dependencies and GHC to be accessible on your PATH. Fortunately, we can use Stack to install cabal-install for us!

stack install alex happy cabal-install

Here’s where things get a little tricky. You’ll need to use a different instance of hackage-build for each version of GHC that your packages require to build. In my case, I have packages depending on base versions and, so I require GHC versions 7.8.4 and 7.10.2. I’ve installed both of these locally, off-path, at $HOME/.local/bin/ghc-7.8.4 and $HOME/.local/bin/ghc-7.10.2 respectively.

Of course, Hackage will document your packages by building them with cabal-install, which may well fail to find a successful build plan. Before switching over to Stack, I used Stackage snapshots to constrain the range of packages cabal-install considered. Therefore, all my older packages build with lts-1.0, and all my newer packages with lts-3.12. In the future, I’ll release packages with using a more recent snapshot, but I’ll always control when this happens.

For me, then, the best course is to run a hackage-build instance for each snapshot I’ve used. Documentation will therefore certainly be built by this instance for a package known to build with the corresponding snapshot. For each snapshot, I initialise hackage-build in a separate directory using the username and password of the account with which I wish to upload the documentation with. Note that account needs to be enabled as a trustee.

./hackage-build init http://localhost:4050 --init-username=user --init-password=password --cache-dir=build-cache-lts-1.0 # For lts-1.0

In the newly created build-cache-lts-1.0 directory I edit the cabal-config file to include both Hackage and the local Hackage as remote-repos, and I add the constraints specified by the corresponding snapshot. The first few lines of the file look like this.

remote-repo: localhost:http://localhost:4050
remote-repo: hackage.haskell.org:http://hackage.haskell.org/packages/archive
remote-repo-cache: build-cache-lts-1.0/cached-tarballs
constraint: AC-Vector ==2.3.2
constraint: BlastHTTP ==1.0.1
constraint: BlogLiterately ==

Remember, I perform the above sequence for each of my two snapshots. Finally, I can run each build client with its corresponding GHC version. For the lts-1.0 process, I use the following command.

PATH=$HOME/.local/bin/ghc-7.8.4/bin:$PATH ./hackage-build build --cache-dir=build-cache-lts-1.0

From here, it’s simple to script the builder to run continuously. Also check out the output of ./hackage-build --help for more options.

Enabling access through Stack

To alert Stack to your new internal Hackage, include its details in the package-indices section of your global config file (~/.stack/config.yaml). Instructions are provided here.

See Also

hackage-server wiki

Edsko’s commit on settings up TUF for the server