Rash thoughts about .NET, C#, F# and Dynamics NAV.

Test-Driven development (TDD) is a well known software development technique and follows the mantra “Red-Green-Refactor”. Behavior-Driven Development (BDD) is a response to TDD and introduces the idea of using natural language to express the Unit Test scenarios.

There are a lot of popular testing frameworks around which can be used for BDD including xUnit.net ,NUnit, StoryQ, MSpec, NSpec and NBehave. Most of them can be used with fluent interfaces and therefore provides a good readability of the sources. Some of them even provide the possibility to generate a spec in natural language out of passed Unit tests.

What is a spec?

“A specification is an explicit set of requirements to be satisfied by a material, product, or service.”

American Society for Testing and Materials (ASTM) definition

A spec is an important document for the communication process – it enables domain experts to communicate with developers. But how can you verify the compliance with the spec? The answer is: you have to write unit tests. Even with the mentioned frameworks there is a lot of work to do in order to translate a spec scenario into a Unit Test.

Question 7 in the famous Joel Test is “Do you have a spec?”.

The idea of NaturalSpec is to give domain experts the possibility to express their scenarios directly in compilable Unit Test scenarios by using a Domain-specific language (DSL) for Unit Tests. NaturalSpec is completely written in F# – but you don’t have to learn F# to use it. You don’t even have to learn programming at all.

Example 1 – Specifying a list

Let’s consider a small example. If we want to test a new List implementation a spec could look like this:

[<Scenario>]
let When_removing_an_3_from_a_small_list_it_should_not_contain_3() =
  Given [1;2;3;4;5]              // “Arrange” test context
    |> When removing 3           // “Act”
    |> It shouldn't contain 3    // “Assert”
    |> It should contain 4       // another assertion
    |> Verify                    // Verify scenario

I used BDD style here and expressed my scenario in a quite natural language. As the comments are indicating the scenario is following the Arrange Act Assert (“AAA”) pattern.

With the Keyword “Given” I can create a test context (the objects I want to test). In this sample I created a list with 5 elements. With the keyword “When” I call a function which does something with my test context. In this case I want to remove the value 3. In the Assert section (keywords “It should” or “It shouldn’t”) I can give some observations, which should hold for my manipulated test context.

When I run this scenario via a NUnit runner (i am using TestDriven.Net) I get the following output:

Scenario: When removing an 3 from a small list it should not contain 3

– Given [1; 2; 3; 4; 5]
– When removing 3
=> It should not contain 3
=> It should contain 4
==> OK

Example 2 – Specifying a factorial function

If you implement factorial function the spec could look like this:

[<Scenario>]
let When_calculating_fac_5_it_should_equal_120() =
  Given 5
    |> When calculating factorial
    |> It should equal 120
    |> Verify    

[<Scenario>]
let When_calculating_fac_1_it_should_equal_1() =
  Given 1
    |> When calculating factorial
    |> It should equal 1
    |> Verify          

[<Scenario>]
let When_calculating_fac_0_it_should_equal_0() =
  Given 0
    |> When calculating factorial
    |> It should equal 1
    |> Verify

And the output of NaturalSpec would look like this:

Scenario: When calculating fac 0 it should equal 0

– Given 0
– When calculating factorial
=> It should equal 1
==> OK

Scenario: When calculating fac 1 it should equal 1

– Given 1
– When calculating factorial
=> It should equal 1
==> OK

Scenario: When calculating fac 5 it should equal 120

– Given 5
– When calculating factorial
=> It should equal 120
==> OK

Getting started

Of course you can use NaturalSpec to specify C# objects. I see my post "Using NaturalSpec to create a spec for C# projects" for a small sample.

You can download NaturalSpec at GoogleCode and follow the “Getting started” tutorial in order to write your first automatically testable spec.

I am very interested in your feedback. Do you like the syntax? What should I change? Do you consider using a spec tool like NaturalSpec?

Sometimes we need a function which finds the m smallest elements in a list or array.

We can use the idea of bubble sort to maintain an sorted array of the m smallest elements so far. This idea gives us a O(m*n) algorithm which solves our problem:

let bubble array element =
  let rec bubbleStep i =
    if i < (array |> Array.length) then
      match array.[i] with
        | None -> 
           if i = array.Length - 1 || array.[i+1] <> None then
             array.[i] <- Some element
                          
           bubbleStep (i+1)
        | Some x -> 
           if element < x then
             if i = 0 then 
               array.[i] <- Some element 
             else
               array.[i-1] <- Some x
               array.[i] <- Some element
             bubbleStep (i+1)
  
  bubbleStep 0
  array    
  
let mMin seq m =
  Seq.fold bubble (Array.create m None) seq

We can easily generalize this function to compute maximum and minimum:

let bubble f array element =
  let rec bubbleStep i =
    if i < (array |> Array.length) then
      match array.[i] with
        | None -> 
           if i = array.Length - 1 || array.[i+1] <> None then
             array.[i] <- Some element
                          
           bubbleStep (i+1)
        | Some x -> 
           if f element x then
             if i = 0 then 
               array.[i] <- Some element 
             else
               array.[i-1] <- Some x
               array.[i] <- Some element
             bubbleStep (i+1)
  
  bubbleStep 0
  array    
    
let mBubble f seq m =
  Seq.fold (bubble f) (Array.create m None) seq
  
let mMin seq m = mBubble (<) seq m
let mMax seq m = mBubble (>) seq m

If we don’t want to store Option-Values we can use this simplification:

let bubble f m array element =
  let rec bubbleStep i =
    if i < (array |> Array.length) then
      let x = array.[i]
      if f element x then
         if i = 0 then 
           array.[i] <- element 
         else
           array.[i-1] <- x
           array.[i] <- element
         bubbleStep (i+1)
  
  if Array.length array < m then
    let newArray = Array.append [|element|] array 
    newArray 
      |> Array.sort 
          (fun a b -> 
            if a = b then 0 
            elif f a b then 1 else -1)
    newArray
  else
    bubbleStep 0
    array    
    
let mBubble f seq m =    
  Seq.fold (bubble f m) Array.empty seq
  
let mMin seq m = mBubble (<) seq m
let mMax seq m = mBubble (>) seq m

Am Dienstag dem 16.6.2009 werde ich bei der .NET Developer Group Braunschweig einen Vortrag zum Thema “Funktionale Programmierung in F#” halten. Veranstaltungsort ist das Restaurant Kilian’s Raffinerie.

“Funktionale Programmiersprachen nehmen seit geraumer Zeit einen hohen Stellenwert in der Wissenschaft ein. Demnächst könnte es eine dieser Sprachen sogar aus dem Forschungsbereich direkt in den Mainstream schaffen. Visual Studio 2010 wird neben C# und VB.NET die funktionale Programmiersprache F# als dritte Hauptsprache anbieten. Der Vortrag soll einen Einblick in funktionale Konzepte und deren Umsetzung in F# geben. Insbesondere soll auf “Funktionen höherer Ordnung”, Typinferenz, Currying, Pattern Matching, “Unveränderlichkeit” und parallele Programmierung eingegangen werden.”

[Abstract]

Warning: Creating default object from empty value in /www/htdocs/w007928a/blog/wp-content/plugins/wp-referencesmap/GoogleMapsWrapper.php on line 55

Irgendwie bringt Google Maps doch immer wieder neue Kuriositäten. Wenn man sich den Fußweg von Barcelona nach Marseille anzeigen lässt, dann schlägt das Programm eine Fährtour über Genua und Korsika vor und weist auch ausdrücklich darauf hin, dass es dort möglicherweise keine Bürgersteige gibt. 😉

In the last post I show how we can calculate clusters of n-dimensional Objects with the K-means-Algorithm in F#. This time I will simplify the code by using the built-in type vector.

First of all we redefine the interface for “clusterable” objects:

type IClusterable =
  abstract DimValues: vector with get

This time we do not need any dimension information – the vector type will care about this. Now we can reduce our distance function to:

let calcDist (p1:vector) (p2:vector) = (p1 - p2).Norm

The function Norm calculates the 2-Norm of a vector.

So calcDist will still compute the n-dimensional Euclidean distance.

The next step is the definition of a n-dimensional Centroid type. This time we will only use a type alias of vector:

type Centroid = vector

The calculation of the centroid and the centroid error can be written as:

let calcCentroid (items:'a list when 'a :> IClusterable)
     (oldCentroid:Centroid) =
  let count = items.Length
  if count = 0 then oldCentroid else  
  let sum =
    items
      |> List.fold_left 
        (fun vec item -> vec + item.DimValues) // vector addition
        (nullVector oldCentroid)
   
  sum * (1./(count |> float))  // multiply with scalar
     

let calcError centroid (items:'a list when 'a :> IClusterable) =
  let sq x = x * x

  items
    |> List.sum_by (fun i -> calcDist centroid i.DimValues |> sq)

The calcError function is straight forward but not perfect in terms of performance. As we use the calcDist function and hence the 2-Norm the function will first calculate a square root and then squares the result.

The rest of the code is pretty much same as in the last post:

/// creates n-dimensional 0 - vector
let nullVector (v:vector) = v * 0.
    
type Cluster<'a> when 'a :> IClusterable =
  {Items: 'a list;
   Count: int;
   Centroid: Centroid;
   Error: float}

  member x.Print =
     printfn "(Items: %d, Centroid: %A, Error: %.2f)"
       x.Count x.Centroid x.Error

  static member CreateCluster (item:'a) =
    let items = [item]
    let empty = nullVector item.DimValues

    let centroid = calcCentroid items empty
    {new Cluster<'a> with
      Items = items and
      Count = 1 and
      Centroid = centroid and
      Error = calcError centroid items}

  member x.EvolveCluster (items:'a list) =
    let l = items.Length
    let centroid = calcCentroid items x.Centroid
    {x with
      Items = items;
      Count = l;
      Centroid = centroid;
      Error = calcError centroid items} 

let rand = new Random()  

let InitClusters (k:int)
      (items:'a array when 'a :> IClusterable)  =
  let length = items.Length
  let getInitItem() = items.[rand.Next length]
  Array.init k (fun x ->
                  getInitItem()
                    |> Cluster<'a>.CreateCluster)
                    
                    
let AssignItemsToClusters k (clusters:Cluster<'a> array)
      (items:'a seq when 'a :> IClusterable) =
  if k <= 0 then failwith "KMeans needs k > 0"
  let findNearestCluster (item:'a) =
    let minDist,nearest,lastPos =
      clusters
        |> Array.fold_left
            (fun (minDist,nearest,pos) cluster ->
               let distance = calcDist item.DimValues (cluster.Centroid)
               if distance < minDist then
                 (distance,pos,pos+1)
               else
                 (minDist,nearest,pos+1))
            (Double.PositiveInfinity,0,0)
    nearest

  let assigned =
    items
      |> Seq.map (fun item -> item,findNearestCluster item)

  let newClusters = Array.create k []
  for item,nearest in assigned do
    newClusters.[nearest] <- item::(newClusters.[nearest])

  clusters
    |> Array.mapi (fun i c -> c.EvolveCluster newClusters.[i])  

    
let calcClusterError (clusters:Cluster<'a> array) =
  clusters
    |> Array.fold_left (fun acc cluster -> acc + cluster.Error) 0.  
                      
let kMeansClustering K epsilon
       (items:'a array when 'a :> IClusterable) =
  let k = if K <= 0 then 1 else K
  let rec clustering lastError (clusters:Cluster<'a> array) =
    let newClusters =
      AssignItemsToClusters k clusters items
    let newError = calcClusterError newClusters

    if abs(lastError - newError) > epsilon then
      clustering newError newClusters
    else
      newClusters,newError    

  InitClusters k items
    |> clustering Double.PositiveInfinity

This vector approach shortens the code for the kMeans algorithm, but has a little longer runtime. I am interested in any suggestions.

The K-means-Algorithm is one of the simplest unsupervised learning algorithms that solve the well known clustering problem. In this article I will show how we can implement this in F#.

First of all we define an interface for “clusterable” objects:

The next step is to define a distance function. We will use n-dimensional Euclidean distance here.

Now we define a n-dimensional Centroid type. Our kMeans-Algorithm will minimize the squared distances to k centroids (or “means”).

The centroid of a finite set of n-dimensional points x₁, x₂, …, x_n is calculated as

We made a small modification – if we don’t have any items assigned to a cluster the centroid won’t change. This is important for the robustness of the algorithm.

Now we can calculate the squared errors to a centroid:

For storing cluster information we create a cluster type:

Now we need a function to initialize the clusters and a function to assign a items to the nearest cluster:

The last step is a function which calculates the squared error over all clusters:

Now it is an easy task to write the kMeans algorithm:

We can test this algorithm with random 2-dimensional points:

Next time I will show how we can simplify the code by using F#’s built-in type vector instead of array.

Am Freitag dem 29.5.2009 werde ich in Leipzig ein kostenloses “.NET Bootcamp” zum Thema “Funktionale Programmierung in F#” leiten. Die Anmeldung wird ab März auf der Veranstaltungsseite der .NET User Group Leipzig möglich sein.

“Funktionale Programmiersprachen nehmen seit geraumer Zeit einen hohen Stellenwert in der Wissenschaft ein. Demnächst könnte es eine dieser Sprachen sogar aus dem Forschungsbereich direkt in den Mainstream schaffen. Visual Studio 2010 wird neben C# und VB.NET die funktionale Programmiersprache F# als dritte Hauptsprache anbieten. Das .NET Bootcamp zu F# soll einen Einblick in funktionale Konzepte und deren Umsetzung in F# geben. Insbesondere soll auf “Funktionen höherer Ordnung”, Typinferenz, Currying, Pattern Matching, “Unveränderlichkeit” und parallele Programmierung eingegangen werden.”

Veranstaltungsabstract

Update: Die Anmeldung ist nun möglich.

Update: Die Fragen zum BootCamp können nun hier herunter geladen werden.

Am 20.01.2009 findet zwischen 19:30 und 21:30 Uhr der 1. Stammtisch der .NET User Group Leipzig im Jahr 2009 statt. Treffpunkt ist das TELEGRAPH Café in Leipzig und als besonderes Highlight hat sich Ralf Westphal angekündigt. Eine Anmeldung ist nicht nötig.

Warning: Creating default object from empty value in /www/htdocs/w007928a/blog/wp-content/plugins/wp-referencesmap/GoogleMapsWrapper.php on line 55

Since I am working with hybrid solutions (with F# and C# projects in it) I had to deactivate ReSharper. ReSharper had a problem with analyzing my F# sources (see JIRA bug entry #79203). The result was that every single F# defined type and function was marked as an error. I nearly got crazy. On one hand I got used to all the nice ReSharper refactorings (and the NUnit runner) and on the other I got all these false positive errors.

But from now on this hard times are over. Today I tested build 1153 (see nightly builds for version 4.5) – and everything works fine.

Thank you guys at JetBrains. 🙂

Seit dem 22.12.2008 ist die Add On Datenbank für NAV2009 mit den Modulen Zahlungsverkehr und Kostenrechnung zum Download auf Partnersource verfügbar.

“Dieses Release beinhaltet folgende Dateien:

• Klassische fdb Datenbank
• MDF Datenbank für SQL Server
• Onlinehilfe für den Zahlungsverkehr (DEU und ENU)
• Changes.doc für Zahlungsverkehr und Kostenrechnung

Hinweis:
Die Objekte für den Zahlungsverkehr beinhalten auch die SEPA Funktionalität.
Die Onlinehilfe für die Kostenrechnung wird umgehend nachgereicht.”

[Quelle Partnersource]

Weitere Links:

• Deutsche Vollversion von Dynamics NAV 2009
• Update Toolkit
• Launch Portal für Microsoft Dynamics NAV 2009