Wednesday, March 17, 2010

DPM: Darcs Patch Manager

I’ve just released the initial version of DPM on Hackage! The Darcs Patch Manager (DPM for short) is a tool that simplifies working with the revision control system darcs. It is most effective when used in an environment where developers do not push their patches directly to the main repository but where patches undergo a reviewing process before they are actually applied. Here is a short story that illustrates how would use the DPM in such sitations.

Suppose that Dave Developer implements a very cool feature. After polishing his patch, Dave uses darcs send to send the patch:

  $ darcs send host:MAIN_REPO
  Tue Mar 16 16:55:09 CET 2010  Dave Developer <dave@example.com>

    * very cool feature
  Shall I send this patch? (1/1)  [ynWsfvplxdaqjk], or ? for help: y
  Successfully sent patch bundle to: patches@example.com

After the patch has been sent to the address patches@example.com, DPM comes into play. For this example, we assume that mail devivery for patches@example.com is handled by some mailfilter program such as maildrop (http://www.courier-mta.org/maildrop/) or procmail (http://www.procmail.org/). The task of the mailfilter program is the add all patches sent to patches@example.com to the DPM database. This is achieved with the DPM command add:

  $ dpm add –help
  add: Put the given patch bundles under DPM’s control (use ‘-’ to read from stdin).
  Usage: add FILE…

  Command options:

  Global options:
    -r DIR  –repo-dir=DIR                  directory of the darcs repository
    -s DIR  –storage-dir=DIR               directory for storing DPM data
    -v      –verbose                       be verbose
            –debug                         output debug messages
            –batch                         run in batch mode
            –no-colors                     do not use colors when printing text
            –user=USER                     current user
            –from=EMAIL_ADDRESS            from address for emails
            –review-address=EMAIL_ADDRESS  email address for sending reviews
    -h, -?  –help                          display this help message

Now suppose that Dave’s patch is in the DPM database. A reviewer, call him Richard Reviewer, uses the DPM command list to see what patches are available in this database:

  $ dpm list –help
  list: List the patches matching the given query.

  Query ::= Query ‘ + ‘ Query  — logical OR
          | Query ‘ ‘   Query  — logical AND
          | ‘^’ Query          — logical NOT
          | ‘{‘ Query ‘}’      — grouping
          | ‘:’ Special
          | String

  Special is one of "undecided", "rejected", "obsolete", "applied",
  "reviewed", "open", or "closed", and String is an arbitrary sequence
  of non-whitespace characters not starting with ‘^’, ‘{‘, ‘}’, ‘+’, or ‘:’.

  If no query is given, DPM lists all open patch groups.

  Usage: list QUERY …

  Command options:

  Global options:
    -r DIR  –repo-dir=DIR                  directory of the darcs repository
    -s DIR  –storage-dir=DIR               directory for storing DPM data
    -v      –verbose                       be verbose
            –debug                         output debug messages
            –batch                         run in batch mode
            –no-colors                     do not use colors when printing text
            –user=USER                     current user
            –from=EMAIL_ADDRESS            from address for emails
            –review-address=EMAIL_ADDRESS  email address for sending reviews
    -h, -?  –help                          display this help message

In our example, the output of the list command might look as follows:

  $ dpm -r MAIN_REPO -s DPM_DB list
    very cool feature [State: OPEN]
      7861 Tue Mar 16 17:20:45  2010 Dave Devloper <dave@example.com>
           State: UNDECIDED, Reviewed: no
           added
    some other patch [State: OPEN]
      7631 Tue Mar 16 13:15:20  2010 Eric E. <eric@example.com>
           State: REJECTED, Reviewed: yes
           added
    …

(The -r option specifies a directory containing the DPM database. Initially, you simply create an empty directory. The -s option specifies the path to the darcs repository in question.)

DPM groups all patches with the same name inside a patch group. Patch groups allow keeping track of multiple revisions of the same patch. In the example, the patch group of name very cool feature has only a single member, which is the patch Dave just created. The patch is identified by a unique suffix of its hash (7861 in the example). The output of the list command further tells us that no reviewer decided yet what to do with the patch (its in state UNDECIDED).

At this point, Richard Reviewer reviews Dave’s patch. During the review, he detects a minor bug so he rejects the patch:

  $ dpm -r MAIN_REPO -s DPM_DB review 7861
    Reviewing patch 7861
    Starting editor on DPM_DB/reviews/2010-03-16_7861_swehr_24166.dpatch
      <inspect patch in editor>
    Mark patch 7861 as reviewed? [Y/n] y
    Patch 7861 is in state UNDECIDED, reject this patch? [y/N] y
    Enter a comment: one minor bug
    Marked patch 7861 as reviewed
    Moved patch 7861 to REJECTED state
    Send review to Dave Developer <dave@example.com>? [Y/n] y
    Mail sent successfully.

Now Dave Developer receives an email stating that has patch has been rejected. The email also contains the full review so that Dave sees why the patch has been rejected. Thus, Dave starts fixing the bug, does an amend-record of the patch, and finally sends the patch again. (Alternatively, he could also create a new patch with exactly the same name as the original patch.)

  $ darcs send MAIN_REPO
  Tue Mar 16 16:55:09 CET 2010  Dave Developer <dave@example.com>
    * very cool feature
  Shall I send this patch? (1/1)  [ynWsfvplxdaqjk], or ? for help: y
  Successfully sent patch bundle to: patches@example.com

Once the email is received, the improved patch is added to the DPM database. The output of the list command now looks like this:

  $ dpm -r MAIN_REPO -s DPM_DB list
    very cool feature [State: OPEN]
      2481 Tue Mar 16 17:50:23  2010 Dave Devloper <dave@example.com>
           State: UNDECIDED, Reviewed: no
           added
      7861 Tue Mar 16 17:20:45  2010 Dave Devloper <dave@example.com>

           State: REJECTED, Reviewed: yes
           marked as rejected: one minor bug
    some other patch [State: OPEN]
      7631 Tue Mar 16 13:15:20  2010 Eric E. <eric@example.com>
           State: REJECTED, Reviewed: yes
           added
    …

The patch 2481 is the improved revision of the original patch 7861. It is in the same group as the original patch because both patches have the same name. Richard Reviewer reviews the improved patch and has no complains anymore:

  $ dpm -r MAIN_REPO -s DPM_DB review 2481
    Reviewing patch 2481
    Starting editor on DPM_DB/reviews/2010-03-16_2481_swehr_876102.dpatch
      <inspect patch in editor>
    Mark patch 2481 as reviewed? [Y/n] y
    Patch 2481 is in state UNDECIDED, reject this patch? [y/N] n
    Enter a comment: ok
    Marked patch 2481 as reviewed
    Send review to Dave Developer <dave@example.com>? [y/N] n

At this point, Richard Reviewer applies the patch with the very cool feature:

  $ dpm apply 2481
    About to apply patch 2481
    Entering DPM’s dumb (aka interactive) apply command.
    Future will hopefully bring more intelligence.

    Instructions:
    =============
    – Press ‘n’ until you reach
      Tue Mar 16 17:50:23  2010 Dave Devloper <dave@example.com>

        * very cool feature
      (Hash: 20100316162041-c71f4-871aedab8f4dd3bd042b9188f1496011c7dd2481)
    – Press ‘y’ once
    – Press ‘d’

    Tue Mar 16 17:50:23  2010 Dave Devloper <dave@example.com>
      * very cool feature
    Shall I apply this patch? (1/1)  [ynWsfvplxdaqjk], or ? for help: y
    Finished applying…
    Patch 2481 applied successfully
    Send notification to author Dave Developer <dave@example.com> of patch 2481? [Y/n] y
    Mail sent successfully.

Applying a patch closes the corresponding patch group. Per default, the list command doesn’t display closed patch groups, but we can force it to do so with the :closed query:

  $ dpm list :closed
    very cool feature [State: CLOSED]
      2481 Tue Mar 16 17:50:23  2010 Dave Devloper <dave@example.com>

           State: APPLIED, Reviewed: yes
           marked as applied: -
      7861 Tue Mar 16 17:20:45  2010 Dave Devloper <dave@example.com>
           State: REJECTED, Reviewed: yes
           marked as rejected: one minor bug
      …

Author: Stefan Wehr

Tuesday, March 16, 2010

HTF: a test framework for Haskell

After nearly 5 years of inactivity, I've finally managed to upload a new version of the Haskell Test Framework (HTF) to Hackage. The HTF is a test framework for the functional programming language Haskell. The framework lets you define unit tests (http://hunit.sourceforge.net), QuickCheck properties (http://www.cs.chalmers.se/~rjmh/QuickCheck/), and black box tests in an easy, uniform and convenient way. The HTF uses a custom preprocessor that collects test definitions automatically. Furthermore, the preprocessor allows the HTF to report failing test cases with exact file name and line number information.

Here's a short tutorial on how to use the HTF. It assumes that you are using GHC for compiling your Haskell code. (It is possible to use the HTF with other Haskell environments, only the steps taken to invoke the custom preprocessor of the HTF may differ in this case.) Note that a hyperlinked version of this tutorial will shortly be available on http://hackage.haskell.org/package/HTF.

Suppose you are writing a function for reversing lists:

myReverse :: [a] -> [a]
myReverse [] = []
myReverse [x] = [x]
myReverse (x:xs) = myReverse xs

To test this function using the HTF, you first create a new source file with a OPTIONS_GHC pragma in the first line.

{-# OPTIONS_GHC -F -pgmF htfpp #-}

This pragma instructs GHC to run the source file through htfpp, the custom preprocessor of the HTF. The following import statements are also needed:

import System.Environment ( getArgs )
import Test.Framework

The actual unit tests and QuickCheck properties are defined like this:

test_nonEmpty = do assertEqual [1] (myReverse [1])
assertEqual [3,2,1] (myReverse [1,2,3])

test_empty = assertEqual ([] :: [Int]) (myReverse [])

prop_reverse :: [Int] -> Bool
prop_reverse xs = xs == (myReverse (myReverse xs))

When htfpp consumes the source file, it replaces the assertEqual tokens (and other assert-like tokens, see Test.Framework.HUnitWrapper) with calls to assertEqual_, passing the current location in the file as the first argument. Moreover, the preprocessor collects all top-level definitions starting with test_ or prop_ in a test suite with name allHTFTests of type TestSuite.

Definitions starting with test_ denote unit tests and must be of type Assertion, which just happens to be a synonym for IO (). Definitions starting with prop_ denote QuickCheck properties and must be of type T such that T is an instance of the type class Testable.

To run the tests, use the runTestWithArgs function, which takes a list of strings and the test.

main = do args <- getArgs
runTestWithArgs args reverseTests

Here is the skeleton of a .cabal file which you may want to use to compile the tests.

Name:          HTF-tutorial
Version: 0.1
Cabal-Version: >= 1.6
Build-type: Simple

Executable tutorial
Main-is: Tutorial.hs
Build-depends: base, HTF

Compiling the program just shown (you must include the code for myReverse as well), and then running the resulting program with no further commandline arguments yields the following output:

Main:nonEmpty (Tutorial.hs:17)
*** Failed! assertEqual failed at Tutorial.hs:18
expected: [3,2,1]
but got: [3]

Main:empty (Tutorial.hs:19)
+++ OK

Main:reverse (Tutorial.hs:22)
*** Failed! Falsifiable (after 3 tests and 1 shrink):
[0,0]
Replay argument: "Just (847701486 2147483396,2)"

* Tests: 3
* Passed: 1
* Failures: 2
* Errors: 0

(To check only specific tests, you can pass commandline arguments to the program: the HTF then runs only those tests whose name contain at least one of the commandline arguments as a substring.)

You see that the message for the first failure contains exact location information, which is quite convenient. Moreover, for the QuickCheck property Main.reverse, the HTF also outputs a string represenation of the random generator used to check the property. This string representation can be used to replay the property. (The replay feature may not be useful for this simple example but it helps in more complex scenarios).

To replay a property you simply use the string representation of the generator to define a new QuickCheck property with custom arguments:

prop_reverseReplay =
withQCArgs (\a -> a { replay = read "Just (10603948072147483396,2)"})
prop_reverse

To finish this tutorial, we now give a correct definition for myReverse:

myReverse :: [a] -> [a]
myReverse [] = []
myReverse (x:xs) = myReverse xs ++ [x]

Running our tests again on the fixed definition then yields the desired result:

Main:nonEmpty (Tutorial.hs:17)
+++ OK

Main:empty (Tutorial.hs:19)
+++ OK

Main:reverse (Tutorial.hs:22)
+++ OK, passed 100 tests.

Main:reverseReplay (Tutorial.hs:24)
+++ OK, passed 100 tests.

* Tests: 4
* Passed: 4
* Failures: 0
* Errors: 0

The HTF also allows the definition of black box tests. See the documentation of the Test.Framework.BlackBoxTest module for further information.

Author: Stefan Wehr