Developing Medical Software in Scala and Haskell

Just recently, I gave a talk at the CUFP (Commercial Users of Functional Programming) workshop about our experience with developing software in the functional programming languages Haskell and Scala. The slides of the talk are available here, and there is even a video recording. Enjoy! Any feedback is welcome, just post a comment below.

By the way, CUFP hosted a whole bunch of very interesting talks. They are all available on youtube, just search for "CUFP 2012".

UIs for Hierachical iPad Apps

If you're in a hurry: tl;dr version

At factis research we developed FRLayeredNavigationController, a view controller compositor which enables you to develop -- as we think -- great iPad apps. Influenced by the UI of the Twitter and Soundcloud iPad apps we designed a new navigation concept for our product Checkpad MED. Since we love open source software and know what we owe the community we'll again try to give something back: FRLayeredNavigationController gives iPad developers a simple drop-in replacement for UINavigationController for great UIs. This screencast shows a very simplistic app made with FRLayeredNavigationController.

Full-blown post

Motivation

The Apple iPad is a great mobile device which is due to its big screen capable of displaying a lot of content at the same time. This capability makes the iPad a great device for apps with complex, structured data. A question Apple leaves to its developers is how to guide the users through deeply hierachical content.

On the iPhone the usual way is to use UINavigationControllers. Because of the small screen, UINavigationControllers are great on the iPhone, only the currently most relevant information is displayed. The sole trace of the navigation path from app start to the current content is the title of the second-to-last view controller in the back button at the upper left side of the screen.

On the iPad UINavigationControllers alone are fine, too, when all your view controllers really have an iPad-screenful of content. But, most apps have some kind of low-content view controllers which are primararilly made to bring the users to the actual content, stepping through several layers of the hierarchy. Often, these controllers are simple UITableViews which look bad when they are displayed fullscreen on the iPad. To solve this problem, Apple gave us UISplitViewController: the navigation-only part on the smaller left side (or even in a UIPopoverController) and the content right next to the navigation in a somewhat bigger frame. This concept works great and looks good as demonstrated at least by the iPad Settings and Mail apps. Compare them to their iPhone counterparts and you will see how nicely they work on these very different devices with minor changes only.

UISplitViewControllers work great when your content hierarchy's depth is two: One navigation layer and one content layer. Three layers is still okay, since you can use a UINavigationController inside the left or right side of the UISplitViewController. But if the depth of your content hierarchy exceeds three, it does not fit nicely in the UISplitviewController somehow, you have to go other ways.

At factis research we particularily liked the Twitter and Soundcloud iPad apps and decided to go a similar path with our product Checkpad MED. We called our solution FRLayeredNavigationController because it enables developers to build schrieblesque apps with multiple hierarchy layers.

How does it work?

The basic idea is not to completely hide a view controller when a new view controller is being shown.  FRLayeredNavigationController still shows some pixels of the view controllers the user passed through until reaching the currently active one. This will give the user a good idea of where (s)he currently is in the navigation hierarchy. If the user should need more information of these partly covered view controllers, (s)he can always reveal the covered pixels by intuitive pan gestures. The user will think of a stack of paper and has similar interaction options. Probably you get a better idea when looking at these screenshots:

regular view

after revealing more information in the level-2 view controller (the "Teams" view controller) by swiping  to the right

Also watch these two iPad screencasts to get the idea: http://youtu.be/v_tXD_mL05E and http://youtu.be/q66HX2td_uc .

Technically, it uses view controller composition as introduced in iOS 5. It manages multiple "Layers" which are UIViewControllers you provide. The moving of the layers is triggered by a UIPanGestureRecognizer and some logic when to move which layer. It looks and feels very nice due to the nice iOS animation support.

Can I use FRLayeredViewController, too?

Sure, it's open sourced and available at GitHub! If you have any problems, feel free to contact us.

What is the API like?

Have a look at the API documentation, it's quite similar to UINavigationController's API from Apple.

Say, you have a UIViewController and you're using UINavigationController from Apple. To show a new layer to the user, you would do:

[self.navigationController pushViewController:test animated:YES];

with FRLayeredNavigationController it's quite similar:

[self.layeredNavigationController pushViewController:test
                                           inFrontOf:self
                                        maximumWidth:NO
                                            animated:YES];

The maximumWidth parameter is how you tell FRLayeredNavigationController whether the new view controller is content (give it the maximal width still available) or not.

Just as UINavigationController, it also supports UIBarItems and a configurable title view. For example, a UIViewController which has been pushed onto a FRLayeredNavigationController could have the following viewWillAppear method to add a back button on the upper left corner:

- (void)viewWillAppear:(BOOL)animated
{
    self.layeredNavigationItem.leftBarButtonItem =
        [[UIBarButtonItem alloc]
         initWithImage:[UIImage imageNamed:@"back-button.png"]
                 style:UIBarButtonItemStylePlain
                target:self
                action:@selector(backButtonClicked)];

    self.layeredNavigationItem.leftBarButtonItem.style =
        UIBarButtonItemStyleBordered;
}

Have a look at FRLayeredNavigationController and FRLayeredNavigationItem to see what's possible with FRLayeredNavigationController.

How can I embed FRLayeredViewController in my project?

It's explained step-by-step in the README file and in this screencast.

stm-stats: Retry statistics for STM transaction

Two days ago, Stefan Wehr talked at the Haskell in Leipzig workshop about the experiences that factis research had with Haskell when creating real-world applications. When he mentioned that we have code that keeps count of retries of STM transaction, someone from the audience asked us to publish the code. So, after some refactoring to make the interface generally usable, here it is. The stm-stats package provides a few functions (in Control.Concurrent.STM.Stats) that can replace atomically: trackSTM and variants that allow you to name the transaction or have more control about the warnings that will be emitted when the number of retries exceeds a certain value. The following code demonstrates how the module works:

import Control.Concurrent
import Control.Concurrent.STM
import Control.Monad

import Control.Concurrent.STM.Stats

main = do
var <- trackSTM $ newTVar 0
forkIO $ forM_ [1..23] $ \i -> do
    threadDelay (100*1000)
    trackNamedSTM "writer" $ writeTVar var i
putStrLn "Starting reader..."
trackNamedSTM "reader" $ do
    i <- readTVar var
    when (i < 23) retry
putStrLn "Reader finished."
dumpSTMStats

When run, you will see this output:

Starting reader...
STM transaction reader finished after 23 retries
Reader finished.
STM transaction statistics (2011-10-09 16:26:27.226675 UTC):
Transaction     Commits    Retries      Ratio
_anonymous_           1          0       0.00
reader                1         23      23.00
writer               23          0       0.00

PS: As I usually post on my own blog, I should explain why I from now on will also post on the factis research company blog. Factis research relies heavily on Free Software and wants to contribute back to the community where possible. But in the course of every-day work, this sometimes falls by the wayside. Therefore, I was hired to help out as the community interface: My task is identifying, packaging and uploading components of internal code that are of wider interest, following up on user requests and bug reports, and talk about it. This module is the first brought to you by this new strategy, but expect more to come.

Talk about developing commercial software in Haskell

Yesterday, I gave a talk at the Haskell in Leizpig workshop about our experience in developing commercial software in Haskell. You can find the slides (in german) here. I was really surprised by the large number of attendees (about 50), given that the workshop's language was german (except the very interesting talk by Kevin Hammond).

New Version of HTF with Diffs, Colors, and Pretty-printing

I've just uploaded version 0.8.1.0 of HTF to hackage. HTF (Haskell Test Framework) allows you to define unit tests, QuickCheck properties, and black box tests in an easy and convenient way. We use HTF at work all the time, where it has proven to be quite valuable for organizing our test suite. An earlier blog post describes HTF in more detail.

The new version comes with some cool new features:

• Support for diffs and pretty-printing. If an equality assertions fails, you now get a proper diff of the two values involved. If possible, the values are pretty-printed (thanks to Edward Yang and his groom package for inspiration). Here's an example:

  [TEST] Main:diff (TestHTF.hs:68)
  assertEqual failed at TestHTF.hs:68
  * expected:
  PlusExpr
    (PlusExpr
       (MultExpr
          (PlusExpr (Variable "foo")
             (MultExpr (Literal 42) (Variable "bar")))
          (PlusExpr (Literal 1) (Literal 2)))
       (Literal 581))
    (Variable "egg")
  * but got:
  PlusExpr
    (PlusExpr
       (MultExpr
          (PlusExpr (Variable "foo")
             (MultExpr (Literal 42) (Variable "bar")))
          (PlusExpr (Literal 2) (Literal 2)))
       (Literal 581))
    (Variable "egg")
  * diff:
  6c6
  <         (PlusExpr (Literal 1) (Literal 2)))
  ---
  >         (PlusExpr (Literal 2) (Literal 2)))
  *** Failed!
      

• As you can see from the example above, HTF now supports colored output.
• There's a new commandline option --quiet which causes HTF to produce output only if absolutely necessary (e.g. for failed test cases).

Just get HTF from hackage, it now also works with GHC 7.0.* and 7.2.1!

xmlgen: a feature-rich and high-performance XML generation library

I’ve released xmlgen to hackage just a few days ago. Xmlgen is a pure Haskell library with a convenient API for generating XML documents. It provides support for all functionality defined by the XML information set and offers good performance and low memory usage. In our company, we developed xmlgen because we wanted the readability of XML literals (as for example provided by the hsp library) without the drawbacks of a custom preprocessor (wrong line numbers in error messages, non-compositionality).

In this blog article, I’ll show you how to use the combinators provided by xmlgen to generate the following XML document:

<?xml version="1.0"?>
<people>
  <person age="32">Stefan</person>
  <person age="4">Judith</person>
</people>

First, we import some modules:

> import Data.Monoid
> import qualified Data.ByteString.Lazy as BSL
> import Text.XML.Generator -- the main xmlgen module

Then we continue by generating the person element.

> genPersonElem :: (String, String) -> Xml Elem
> genPersonElem (name, age) =
>     xelem "person" $ xattr "age" age <#> xtext name

The xelem combinator constructs an XML element from an element name and from the children of the element. Xmlgen provides overloaded variants of xelem to support a uniform syntax for the construction of elements with qualified and unqualified names and with different kinds of children. The <#> combinator separates the element’s attributes from the other children (sub-elements and text nodes). The combinators xattr and xtext construct XML attributes and text nodes, respectively.

The result of an application of xelem has type Xml Elem, whereas xattr has result type Xml Attr. This distinction is important so that attributes and elements can not be confused. The result type of the xtext combinator is Xml Elem; we decided against an extra type for text nodes because for xmlgen’s purpose text nodes and elements are almost interchangeble.

The types Xml Elem and Xml Attr are both instances of the Monoid type class. Constructing a list of elements from a list of persons and their ages is thus quite easy:

> genPersonElems :: [(String, String)] -> Xml Elem
> genPersonElems = foldr mappend mempty . map genPersonElem

The pattern shown above is quite common, so xmlgen allows the following shorthand notation using the xelems combinator.

> genPersonElems' :: [(String, String)] -> Xml Elem
> genPersonElems' = xelems . map genPersonElem

We are now ready to construct the final XML document:

> genXml :: Xml Doc
> genXml = let people = [("Stefan", "32"), ("Judith", "4")]
>          in doc defaultDocInfo $ xelem "people" (genPersonElems people)

For convenience, here is a standalone variant of the genXml function:

> genXml' :: Xml Doc
> genXml' =
>   let people = [("Stefan", "32"), ("Judith", "4")]
>   in doc defaultDocInfo $
>        xelem "people" $
>          xelems $ map (\(name, age) -> xelem "person" (xattr "age" age <#> xtext name)) people

Xmlgen supports various output formats through the overloaded xrender function. Here we render to resulting XML document as a lazy byte string:

> outputXml :: IO ()
> outputXml = BSL.putStrLn (xrender genXml')

Loading the current file into ghci and evaluating outputXml produces the following result:

*Main> outputXml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<people
><person age="32"
>Stefan</person
><person age="4"
>Judith</person
></people
>

This blog post introduced most but not all features of the xmlgen API. Check out the documentation.

Happy hacking and have fun!

Author: Stefan Wehr

Empty GHC profile files and signal handling

We recently installed signal handlers in our Haskell application to clean up some stuff on program termination. Later we realised that we couldn’t do profiling anymore because the generated .prof files were empty. We were catching SIGTERM and SIGINT (using System.Posix.Signals) to run our cleanup signal handler. As signal handlers are run in their own thread, we had to use “exitImmediately” (from System.Posix.Process) instead of “exitWith” to terminate the application. Unfortunately that call really exits immediately and doesn’t give the RTS the opportunity to write the profile.

The GHC Commentay  http://hackage.haskell.org/trac/ghc/wiki/Commentary/Rts/Signals explains that the default interrupt handler calls ”interruptStgRts” to exit the program. We now import this C-function via FFI and call it in our signal handler.

Update! We lose the ability to specify an exit code when using “interruptStgRts” so we now call “shutdownHaskellAndExit” instead.

Here’s an example program that demonstrates the use of System.Posix.Process and the FFI call to “shutdownHaskellAndExit”:

{-# LANGUAGE ForeignFunctionInterface #-}
import Control.Concurrent
import Foreign.C( CInt )
import System.IO
import System.Posix.Process
import System.Exit
import System.Posix.Signals

foreign import ccall "shutdownHaskellAndExit" shutdownHaskellAndExit :: CInt -> IO ()

firstSigINT :: IO ()
firstSigINT =
    do hPutStrLn stderr "caught interrupt (shutdown takes 2s)"
       hFlush stderr
       installHandler keyboardSignal (Catch secondSigINT) (Just emptySignalSet)
       threadDelay (2*1000*1000)
       shutdownHaskellAndExit 13

secondSigINT :: IO ()
secondSigINT =
    do hPutStrLn stderr "forcing immediate exit"
       hFlush stderr
       exitImmediately (ExitFailure 13)

main =
    do installHandler keyboardSignal (Catch firstSigINT) (Just emptySignalSet)
       let busyloop i = busyloop (i+1)
       busyloop 0

Author: David Leuschner

New version of HTF: now backwards-compatible with HUnit

I’ve just uploaded version 0.5.0.0 of the Haskell Test Framework (HTF) to hackage. The new version allows for backwards-compatibility with the HUnit library. So, for example, say you have the following existing HUnit test:

test_fac = 
    do assertEqual "fac 0" 1 (fac 0)
       assertEqual "fac 3" 6 (fac 3)

To let the HTF collect your unit tests automatically, you just need to add the following line at the top of your source file:

{-# OPTIONS_GHC -F -pgmF htfpp -optF --hunit #-}

The pragma above specifies that the source file should be run through HTF’s preprocessor htfpp in HUnit-backwards-compatibility mode. The preprocessor attaches precise location information to all assertions and collects all unit tests and all QuickCheck properties in a fresh variable called allHTFTests.

If you start with your unit tests from scratch, you should leave out the -optF --hunit flag because it releaves you from providing location information such as "fac 0" and "fac 1" for your testcases by hand. The pragma should then look as follows:

{-# OPTIONS_GHC -F -pgmF htfpp #-}

See the HTF tutorial for more information.

Thanks to Magnus Therning, who convinced me to add the HUnit-backwards-compatibility layer to the HTF.

Author: Stefan Wehr

Speeding up your cabal builds - Part II

Last time, I blogged about how linking your binaries against an internal library might speed up your cabal builds. This time, I show how you can avoid building certain binaries at all.

In our company, we work at a rather large Haskell project. The cabal file specifies more then ten binaries, so it takes rather long to build all of them. But often, you only need one or two of these binaries, so building them all is a waste of time.

Unfortunately, cabal does not allow you to build only a subset of your binaries. One workaround is the set the buildable flag in your .cabal file to false for the binaries you don’t want to build. However, this approach is rather unflexible because you need to edit the .cabal file and do a cabal configure after every change.

The solution I present in this article allows you to specify the binaries to build as arguments to the cabal build command. For example, if you want to build only binary B, you invoke cabal as cabal build B and cabal only builds binary B.

To get this working, all you need to do is writing a custom Setup.hs file:

import Data.List
import System.Exit
import Control.Exception

import Distribution.Simple
import Distribution.Simple.Setup
import Distribution.PackageDescription hiding (Flag)
import Distribution.PackageDescription.Parse
import Distribution.Verbosity (normal)

_CABAL_FILE_ = "DociGateway.cabal"

-- enable only certain binaries (specified on the commandline)                                                                                   
myPreBuildHook ::  Args -> BuildFlags -> IO HookedBuildInfo
myPreBuildHook [] flags = return emptyHookedBuildInfo
myPreBuildHook args flags =
    do let verbosity = case buildVerbosity flags of
                         Flag v -> v
                         NoFlag -> normal
       descr <- readPackageDescription verbosity _CABAL_FILE_
       let execs = map fst (condExecutables descr)
           unbuildableExecs = execs \\ args
       mapM_ (checkExistingExec execs) args
       putStrLn ("Building only " ++ intercalate ", " args)
       return (Nothing, map (\e -> (e, unbuildable)) unbuildableExecs)
    where
      unbuildable = emptyBuildInfo { buildable = False }
      checkExistingExec all x =
          if not (x `elem` all)
             then do putStrLn ("Unknown executable: " ++ x)
                     throw (ExitFailure 1)
             else return ()

main = defaultMainWithHooks $ simpleUserHooks { preBuild = myPreBuildHook }

That’s all! Don’t forget the set the Build-Type in your .cabal file to Custom. I’ve tested this approach with cabal-install version 0.8.2, using version 1.8.6 of the Cabal library.

Happy hacking and have fun!

Author: Stefan Wehr

Speeding up your cabal builds

Every waited too long for your cabal builds to finish? If that’s because you have multiple executable sections in your .cabal file, then there might be a solution.

By default, cabal rebuilds all relevant object files for each executable in separation. In other words, object files are not shared between executables. So if you have n executables and m source files, then cabal needs n * m compilation steps plus n link steps to rebuild the executables, no matter whethe any source file contributes to multiple executables.

Starting with cabal 1.8, there is a better solution, provided your executables have some source files in common. In this case, you might build a library from these common source files and then link the executables against the library. In the example above, if all n executables use the same set of m source files, then you end up with m compilation steps plus n + 1 link steps. Sounds good, doesn’t it?!

Here is a simple .cabal file that demonstrates how linking against an internal library works:

Name:                test
Version:             0.1
Synopsis:            test package for linking against internal libraries
Author:              Stefan Wehr
Build-type:          Simple
Cabal-version:       >=1.8 -- IMPORTANT

Library
  Hs-source-dirs: lib -- IMPORTANT
  Exposed-modules: A
  Build-Depends: base >= 4

Executable test-exe
  Build-depends: base >= 4, test, -- link against the internal library
  Main-is: Main.hs -- imports A
  Hs-source-dirs: prog  -- IMPORTANT

There are some things to consider:

• The Cabal-Version must be greater or equal 1.8.
• The library and the executable must not use common source directories, otherwise the compiler does not pick the library but recompiles the source files.
• The library must be mentioned in the Build-depends of the executable

Running cabal build now gives the following output:

Building test-0.1...
[1 of 1] Compiling A                ( lib/A.hs, dist/build/A.o )
Registering test-0.1...
[1 of 1] Compiling Main             ( prog/Main.hs, dist/build/test-exe/test-exe-tmp/Main.o )
Linking dist/build/test-exe/test-exe ...

No rebuilding of A when compiling Main!!!

This feature of cabal isn’t mentioned in the manual, at least I didn’t find it. Further, there seems to be no changelog for cabal. I found out about this feature by browsing the bug tracker for cabal. Is there a better way to get informed of new features of cabal?

Note: I successfully tested this with cabal-install version 0.8.2 (cabal library 1.8.0.4). I couldn’t get it to work with cabal-install version 0.8.0.

Author: Stefan Wehr

Cross-Compiling DLLs with Linux

When working with a Linux-driven work environment, it is nice to be able to also compile your Windows projects under Linux. One question that arises is how to compile DLLs. Thankfully this is very straighforward process using MinGW.

Creating the DLL

Simply create a file example_dll.c with the fitting header example_dll.h

#include "example_dll.h"

int example_function(int n) {
   return n*42;
}

#ifndef EXAMPLE_DLL_H__
#define EXAMPLE_DLL_H__

int example_function(int n);

#endif

Then just compile it with:
$> i586-mingw32msvc-gcc -shared example_dll.c -o example.dll

and viola, you have your DLL ready to use. This is just a simple example DLL, but with this method it is possible to create full-blown DLLs with thousands of lines of code. When you keep your code clean and platform-independet you can compile the same code into a shared library for Linux and a DLL for Windows and even link against other dynamic libraries like OpenSSL or libcurl, though it is advisable to use GNU Automake and GNU Libtool when creating larger projects to ease the hassle of the growing command lines, especially because of different options for Windows and Linux. GNU Automake will take care of all that automatically, also when cross-compiling.

Using the DLL

Using the DLL is just as you would expect it. In this example just create a file use_dll.c with following content:

#include <stdio.h>
#include "example_dll.h"

int main() {
   int res = example_function(13);
   printf("%d should be %d!\n", res, 13*42);

   return 0;
}

Then your program compiles as simple as this, ready to use on any Windows system:
$> i586-mingw32msvc-gcc use_dll.c example.dll -o example.exe

Using GNU Automake

Creating DLLs with GNU Automake and GNU Libtool isn't difficult either. With your working Automake setup, simply add the macro
AC_LIBTOOL_WIN32_DLL
to your configure.ac and GNU Libtool will create clean DLLs for your project when configured for cross-compiling.

Using the DLL with MSVC

To link the DLL against a project in MSVC you will have to generate a .lib file, and for that you will have to generate a .def file. So when compiling on your Linux machine just add the following parameter to your gcc comandline:
-Wl,--output-def,example.def
which will tell the linker to output the .def file as example.def. Then on your Windows machine with a installation of some kind of MSVC compiler execute following command:
lib /machine:i386 /def:example.def
to compile the .def into a .lib which you can then link against in your project. Don't forget to do this step every time your API changes...

Author: Jonathan Dimond

MVars in Objective-C

MVars (mutable variables) are a well-known synchronization primitive in the functional programming language Haskell (see here for the API documentation). An MVar is like a box, which can be either empty or full. A thread trying to read from an MVar blocks until the MVar becomes full, a thread writing to an MVar blocks until the MVar becomes empty.

Recently, I had the need for MVars in Objective-C. I’m sure, I could have solved the problem with other synchronization mechanisms from Apple’s API, but as we all know, programmers are too lazy to read API documentation and programming MVars in Objective-C is fun anyway. I started with this simple interface for MVars:

@interface MVar : NSObject {
  @private
    NSCondition *emptyCond;
    NSCondition *fullCond;
    id value;
    BOOL state;
}
// Reads the current value from the MVar, blocks until a value is available.
- (id)take;
// Stores a new value into the MVar, blocks until the MVar is empty.
- (void)put:(id)val;
// Creates an MVar that is initial filled with the given value.
- (id)initWithValue:(id)val;
@end

Here is a trivial nonsense program that uses the MVar interface to solve the producer-consumer problem:

#import "MVar.h"
#define N 1000
@implementation MVarTest
- (void)producer:(MVar *)mvar {
    for (NSInteger i = 0; i < N; i++) {
        [mvar put:[NSNumber numberWithInteger:i]];
    }
}

- (void)consumer:(MVar *)mvar {
    for (NSInteger i = 0; i < N; i++) {
        NSNumber *n = [mvar take];
        // do something with n
    }
}

- (void)main {
    MVar *mvar = [[[MVar alloc] init] autorelease];
    [NSThread detachNewThreadSelector:@selector(producer:)
                       toTarget:self withObject:mvar];
    [self consumer:mvar];
}
@end

Let's come back to the implementation of MVars. The condition variables emptyCond and fullCond signal that the MVar is empty/full. The variable state stores the stateof the MVar (empty/full). With this in hand, the actual implementation of the MVar class is straightforward:

#import "MVar.h"
#import "Common.h";

#define STATE BOOL
#define EMPTY NO
#define FULL YES

@interface MVar ()
@property (nonatomic,retain) id value;
@end

@implementation MVar

// Notifies waiting threads that the state of the MVar has changed.
- (void)signal:(STATE)aState {
    NSCondition *cond = (aState == FULL) ? fullCond : emptyCond;
    [cond lock];
    self->state = aState;
    [cond signal];
    [cond unlock];
}

- (id)take {
    [fullCond lock];
    while (state != FULL) {
        [fullCond wait];
    }
    id res = self.value;
    self.value = nil;
    [fullCond unlock];
    [self signal:EMPTY];
    return res;
}

- (void)put:(id)aValue {
    [emptyCond lock];
    while (state != EMPTY) {
        [emptyCond wait];
    }
    self.value = aValue;
    [emptyCond unlock];
    [self signal:FULL];
}

// Creates an MVar that is initially empty.
- (id)init {
    if ((self = [super init])) {
        self->emptyCond = [[NSCondition alloc] init];
        self->fullCond = [[NSCondition alloc] init];
        [self signal:EMPTY];
    }
    return self;
}

- (id)initWithValue:(id)aValue {
    self = [self init];
    [self put:aValue];
    return self;
}

- (void)dealloc {
    [emptyCond release];
    [fullCond release];
    [value release];
    [super dealloc];
}

@synthesize value;
@end

Please let me know if you find any bugs in the code shown in this article.

Happy hacking and have fun!

Author: Stefan

DPM: Darcs Patch Manager

I’ve just released the initial version of DPM on Hackage! The Darcs Patch Manager (DPM for short) is a tool that simplifies working with the revision control system darcs. It is most effective when used in an environment where developers do not push their patches directly to the main repository but where patches undergo a reviewing process before they are actually applied. Here is a short story that illustrates how would use the DPM in such sitations.

Suppose that Dave Developer implements a very cool feature. After polishing his patch, Dave uses darcs send to send the patch:

  $ darcs send host:MAIN_REPO
  Tue Mar 16 16:55:09 CET 2010  Dave Developer <dave@example.com>

    * very cool feature
  Shall I send this patch? (1/1)  [ynWsfvplxdaqjk], or ? for help: y
  Successfully sent patch bundle to: patches@example.com

After the patch has been sent to the address patches@example.com, DPM comes into play. For this example, we assume that mail devivery for patches@example.com is handled by some mailfilter program such as maildrop (http://www.courier-mta.org/maildrop/) or procmail (http://www.procmail.org/). The task of the mailfilter program is the add all patches sent to patches@example.com to the DPM database. This is achieved with the DPM command add:

  $ dpm add –help
  add: Put the given patch bundles under DPM’s control (use ‘-’ to read from stdin).
  Usage: add FILE…

  Command options:

  Global options:
    -r DIR  –repo-dir=DIR                  directory of the darcs repository
    -s DIR  –storage-dir=DIR               directory for storing DPM data
    -v      –verbose                       be verbose
            –debug                         output debug messages
            –batch                         run in batch mode
            –no-colors                     do not use colors when printing text
            –user=USER                     current user
            –from=EMAIL_ADDRESS            from address for emails
            –review-address=EMAIL_ADDRESS  email address for sending reviews
    -h, -?  –help                          display this help message

Now suppose that Dave’s patch is in the DPM database. A reviewer, call him Richard Reviewer, uses the DPM command list to see what patches are available in this database:

  $ dpm list –help
  list: List the patches matching the given query.

  Query ::= Query ‘ + ‘ Query  — logical OR
          | Query ‘ ‘   Query  — logical AND
          | ‘^’ Query          — logical NOT
          | ‘{‘ Query ‘}’      — grouping
          | ‘:’ Special
          | String

  Special is one of "undecided", "rejected", "obsolete", "applied",
  "reviewed", "open", or "closed", and String is an arbitrary sequence
  of non-whitespace characters not starting with ‘^’, ‘{‘, ‘}’, ‘+’, or ‘:’.

  If no query is given, DPM lists all open patch groups.

  Usage: list QUERY …

  Command options:

  Global options:
    -r DIR  –repo-dir=DIR                  directory of the darcs repository
    -s DIR  –storage-dir=DIR               directory for storing DPM data
    -v      –verbose                       be verbose
            –debug                         output debug messages
            –batch                         run in batch mode
            –no-colors                     do not use colors when printing text
            –user=USER                     current user
            –from=EMAIL_ADDRESS            from address for emails
            –review-address=EMAIL_ADDRESS  email address for sending reviews
    -h, -?  –help                          display this help message

In our example, the output of the list command might look as follows:

  $ dpm -r MAIN_REPO -s DPM_DB list
    very cool feature [State: OPEN]
      7861 Tue Mar 16 17:20:45  2010 Dave Devloper <dave@example.com>
           State: UNDECIDED, Reviewed: no
           added
    some other patch [State: OPEN]
      7631 Tue Mar 16 13:15:20  2010 Eric E. <eric@example.com>
           State: REJECTED, Reviewed: yes
           added
    …

(The -r option specifies a directory containing the DPM database. Initially, you simply create an empty directory. The -s option specifies the path to the darcs repository in question.)

DPM groups all patches with the same name inside a patch group. Patch groups allow keeping track of multiple revisions of the same patch. In the example, the patch group of name very cool feature has only a single member, which is the patch Dave just created. The patch is identified by a unique suffix of its hash (7861 in the example). The output of the list command further tells us that no reviewer decided yet what to do with the patch (its in state UNDECIDED).

At this point, Richard Reviewer reviews Dave’s patch. During the review, he detects a minor bug so he rejects the patch:

  $ dpm -r MAIN_REPO -s DPM_DB review 7861
    Reviewing patch 7861
    Starting editor on DPM_DB/reviews/2010-03-16_7861_swehr_24166.dpatch
      <inspect patch in editor>
    Mark patch 7861 as reviewed? [Y/n] y
    Patch 7861 is in state UNDECIDED, reject this patch? [y/N] y
    Enter a comment: one minor bug
    Marked patch 7861 as reviewed
    Moved patch 7861 to REJECTED state
    Send review to Dave Developer <dave@example.com>? [Y/n] y
    Mail sent successfully.

Now Dave Developer receives an email stating that has patch has been rejected. The email also contains the full review so that Dave sees why the patch has been rejected. Thus, Dave starts fixing the bug, does an amend-record of the patch, and finally sends the patch again. (Alternatively, he could also create a new patch with exactly the same name as the original patch.)

  $ darcs send MAIN_REPO
  Tue Mar 16 16:55:09 CET 2010  Dave Developer <dave@example.com>
    * very cool feature
  Shall I send this patch? (1/1)  [ynWsfvplxdaqjk], or ? for help: y
  Successfully sent patch bundle to: patches@example.com

Once the email is received, the improved patch is added to the DPM database. The output of the list command now looks like this:

  $ dpm -r MAIN_REPO -s DPM_DB list
    very cool feature [State: OPEN]
      2481 Tue Mar 16 17:50:23  2010 Dave Devloper <dave@example.com>
           State: UNDECIDED, Reviewed: no
           added
      7861 Tue Mar 16 17:20:45  2010 Dave Devloper <dave@example.com>

           State: REJECTED, Reviewed: yes
           marked as rejected: one minor bug
    some other patch [State: OPEN]
      7631 Tue Mar 16 13:15:20  2010 Eric E. <eric@example.com>
           State: REJECTED, Reviewed: yes
           added
    …

The patch 2481 is the improved revision of the original patch 7861. It is in the same group as the original patch because both patches have the same name. Richard Reviewer reviews the improved patch and has no complains anymore:

  $ dpm -r MAIN_REPO -s DPM_DB review 2481
    Reviewing patch 2481
    Starting editor on DPM_DB/reviews/2010-03-16_2481_swehr_876102.dpatch
      <inspect patch in editor>
    Mark patch 2481 as reviewed? [Y/n] y
    Patch 2481 is in state UNDECIDED, reject this patch? [y/N] n
    Enter a comment: ok
    Marked patch 2481 as reviewed
    Send review to Dave Developer <dave@example.com>? [y/N] n

At this point, Richard Reviewer applies the patch with the very cool feature:

  $ dpm apply 2481
    About to apply patch 2481
    Entering DPM’s dumb (aka interactive) apply command.
    Future will hopefully bring more intelligence.

    Instructions:
    =============
    – Press ‘n’ until you reach
      Tue Mar 16 17:50:23  2010 Dave Devloper <dave@example.com>

        * very cool feature
      (Hash: 20100316162041-c71f4-871aedab8f4dd3bd042b9188f1496011c7dd2481)
    – Press ‘y’ once
    – Press ‘d’

    Tue Mar 16 17:50:23  2010 Dave Devloper <dave@example.com>
      * very cool feature
    Shall I apply this patch? (1/1)  [ynWsfvplxdaqjk], or ? for help: y
    Finished applying…
    Patch 2481 applied successfully
    Send notification to author Dave Developer <dave@example.com> of patch 2481? [Y/n] y
    Mail sent successfully.

Applying a patch closes the corresponding patch group. Per default, the list command doesn’t display closed patch groups, but we can force it to do so with the :closed query:

  $ dpm list :closed
    very cool feature [State: CLOSED]
      2481 Tue Mar 16 17:50:23  2010 Dave Devloper <dave@example.com>

           State: APPLIED, Reviewed: yes
           marked as applied: -
      7861 Tue Mar 16 17:20:45  2010 Dave Devloper <dave@example.com>
           State: REJECTED, Reviewed: yes
           marked as rejected: one minor bug
      …

Author: Stefan Wehr

HTF: a test framework for Haskell

After nearly 5 years of inactivity, I've finally managed to upload a new version of the Haskell Test Framework (HTF) to Hackage. The HTF is a test framework for the functional programming language Haskell. The framework lets you define unit tests (http://hunit.sourceforge.net), QuickCheck properties (http://www.cs.chalmers.se/~rjmh/QuickCheck/), and black box tests in an easy, uniform and convenient way. The HTF uses a custom preprocessor that collects test definitions automatically. Furthermore, the preprocessor allows the HTF to report failing test cases with exact file name and line number information.

Here's a short tutorial on how to use the HTF. It assumes that you are using GHC for compiling your Haskell code. (It is possible to use the HTF with other Haskell environments, only the steps taken to invoke the custom preprocessor of the HTF may differ in this case.) Note that a hyperlinked version of this tutorial will shortly be available on http://hackage.haskell.org/package/HTF.

Suppose you are writing a function for reversing lists:

myReverse :: [a] -> [a]
myReverse [] = []
myReverse [x] = [x]
myReverse (x:xs) = myReverse xs

To test this function using the HTF, you first create a new source file with a OPTIONS_GHC pragma in the first line.

{-# OPTIONS_GHC -F -pgmF htfpp #-}

This pragma instructs GHC to run the source file through htfpp, the custom preprocessor of the HTF. The following import statements are also needed:

import System.Environment ( getArgs )
import Test.Framework

The actual unit tests and QuickCheck properties are defined like this:

test_nonEmpty = do assertEqual [1] (myReverse [1])
                   assertEqual [3,2,1] (myReverse [1,2,3])

test_empty = assertEqual ([] :: [Int]) (myReverse [])

prop_reverse :: [Int] -> Bool
prop_reverse xs = xs == (myReverse (myReverse xs))

When htfpp consumes the source file, it replaces the assertEqual tokens (and other assert-like tokens, see Test.Framework.HUnitWrapper) with calls to assertEqual_, passing the current location in the file as the first argument. Moreover, the preprocessor collects all top-level definitions starting with test_ or prop_ in a test suite with name allHTFTests of type TestSuite.

Definitions starting with test_ denote unit tests and must be of type Assertion, which just happens to be a synonym for IO (). Definitions starting with prop_ denote QuickCheck properties and must be of type T such that T is an instance of the type class Testable.

To run the tests, use the runTestWithArgs function, which takes a list of strings and the test.

main = do args <- getArgs
          runTestWithArgs args reverseTests

Here is the skeleton of a .cabal file which you may want to use to compile the tests.

Name:          HTF-tutorial
Version:       0.1
Cabal-Version: >= 1.6
Build-type:    Simple

Executable tutorial
Main-is: Tutorial.hs
Build-depends: base, HTF

Compiling the program just shown (you must include the code for myReverse as well), and then running the resulting program with no further commandline arguments yields the following output:

Main:nonEmpty (Tutorial.hs:17)
*** Failed! assertEqual failed at Tutorial.hs:18
expected: [3,2,1]
but got:  [3]

Main:empty (Tutorial.hs:19)
+++ OK

Main:reverse (Tutorial.hs:22)
*** Failed! Falsifiable (after 3 tests and 1 shrink):
[0,0]
Replay argument: "Just (847701486 2147483396,2)"

* Tests:    3
* Passed:   1
* Failures: 2
* Errors:   0

(To check only specific tests, you can pass commandline arguments to the program: the HTF then runs only those tests whose name contain at least one of the commandline arguments as a substring.)

You see that the message for the first failure contains exact location information, which is quite convenient. Moreover, for the QuickCheck property Main.reverse, the HTF also outputs a string represenation of the random generator used to check the property. This string representation can be used to replay the property. (The replay feature may not be useful for this simple example but it helps in more complex scenarios).

To replay a property you simply use the string representation of the generator to define a new QuickCheck property with custom arguments:

prop_reverseReplay =
  withQCArgs (\a -> a { replay = read "Just (10603948072147483396,2)"}) 
                     prop_reverse

To finish this tutorial, we now give a correct definition for myReverse:

myReverse :: [a] -> [a]
myReverse [] = []
myReverse (x:xs) = myReverse xs ++ [x]

Running our tests again on the fixed definition then yields the desired result:

Main:nonEmpty (Tutorial.hs:17)
+++ OK

Main:empty (Tutorial.hs:19)
+++ OK

Main:reverse (Tutorial.hs:22)
+++ OK, passed 100 tests.

Main:reverseReplay (Tutorial.hs:24)
+++ OK, passed 100 tests.

* Tests:    4
* Passed:   4
* Failures: 0
* Errors:   0

The HTF also allows the definition of black box tests. See the documentation of the Test.Framework.BlackBoxTest module for further information.

Author: Stefan Wehr

Set operations on the unix shell

Some commands to apply set operations to files...

Union:

sort set1 set2 | uniq > lines_in_set1_OR_set2

Intersection:

sort set1 set2 | uniq -d > lines_in_set1_AND_set2

Difference:

comm -23 set1 set2 > lines_in_set1_BUT_NOT_in_set2

Autor: David Leuschner