Rash thoughts about .NET, C#, F# and Dynamics NAV.

The new version 0.27 of “FAKE – F# Make” comes with new syntactic sugar for build targets and build dependencies. Don’t be afraid the old version is still supported – all scripts should still work with the new version.

The problem

Consider the following target definition:

As you can see we are having a lot of “magic strings” for the target names and the dependency definitions. This was always a small shortcoming in FAKE, since this doesn’t allow refactoring and may result in runtime errors.

One of my goals for “FAKE – F# Make” is to remove these strings in future versions. Unfortunately this is not that easy, because it causes a lot of internal issues. In particular logging to the build server is much harder if you don’t have a target name.

The first step

As posted in a bitbucket comment by cipher we could use the “dynamic lookup operator” to remove some of the magic strings without breaking any internal code.

As a result we can rewrite the above sample as:

All magic strings suddenly disappeared. I think this syntax looks really nice, but unfortunately the strings are not really gone, since the token Default is only checked at runtime.

The idea for future versions

Since the new syntax is really just syntactic sugar I’m always interested in a better solution. Currently I’m working on a syntax using monads. The result could look like this:

This way the magic string are really gone, but my current problem is retrieving the target name from the let-binding name. Please leave a comment if you have an idea to solve this issue.

Yesterday I released “FAKE – F# Make” version 0.14 with xUnit.net support. The usage is very easy and similar to the usage of NUnit:

This sample works perfectly with TeamCity and creates a html-page per test project in addition:

If you want to publish the xUnit.net test results in CruiseControl.NET just modify the build script a little:

Now follow the steps in the CrusieControl.NET documentation. You will need to download the xUnitSummary.xsl file and save it to your webdashboard directory. If everything works correctly you should see something like this:

Since version 0.12 the FAKE build system provides an easy way to setup build configurations for CruiseControl.NET.

“CruiseControl.NET is an Automated Continuous Integration server, implemented using the Microsoft .NET Framework.”

[thoughtworks.org]

In this article I will show you how you can set up a FAKE build script in CruiseControl.NET. We will use the CalculatorSample which you can download from the FAKE Download page.

If you want to know how this build script works and how you could create one for your own projects please read the “Getting started with FAKE”-tutorial.

If you want to set up “FAKE – F# Make” build configurations in TeamCity please read “Integrating a "FAKE – F# Make" build script into TeamCity”.

Installing CruiseControl.NET

You can download CruiseControl.NET from thoughtworks.org. After the installation process (further instructions) you should see an empty dashboard:

Installing Subversion

The CalculatorSample is using Subversion (SVN) for SourceControl, so we need to download SVN from subversion.tigris.org. I’m using the “Windows MSI installer with the basic win32 binaries” and installed them under “c:\Program Files (x86)\Subversion\”.

Installing FxCop

The CalculatorSample is using FxCop, so we need to download and install it next.

“FxCop is an application that analyzes managed code assemblies (code that targets the .NET Framework common language runtime) and reports information about the assemblies, such as possible design, localization, performance, and security improvements. Many of the issues concern violations of the programming and design rules set forth in the Design Guidelines for Class Library Developers, which are the Microsoft guidelines for writing robust and easily maintainable code by using the .NET Framework.”

[MSDN]

Creating a FAKE Project

Now create a new folder for the CalculatorSample sources. I’m using “d:\Calculator\” for the rest of the article.

The next step is to modify the CruiseControl.NET config file (“c:\Program Files (x86)\CruiseControl.NET\server\ccnet.config” on my machine):

In this configuration I set up a trigger which checks every 30 sec. for changes in my CalculatorSample project.

If SVN finds changes FAKE.exe is called with my build script (completeBuild.fsx).

After the build I want to merge the FxCop and NUnit output files with my build results to create a build report.

Configuring the dashboard

In order to provide a nicer output on the dashboard we need to modify the BuildPlugins section in the dashboard.config file (“c:\Program Files (x86)\CruiseControl.NET\webdashboard\dashboard.config” on my machine):

As you can see we use the nant.xsl to transform the FAKE output to HTML.

Starting the build

Now if everything is configured correctly, you can tell CruiseControl.NET to run your build (press “Start” for your build project on the dashboard):

Now CruiseControl.NET should use SVN to checkout the CalculatorSample sources and run the build. The output on the project page for build 1 should look like this:

You can also inspect the NUnit and FxCop results:

Please feel free to give feedback if you have any problems with this article.

I just released a new version of my Open Source Build Automation Framework “FAKE – F# Make”. You can read more about FAKE on the project website or in the Getting started with "FAKE – F# Make"-article.

Although the new release contains many bugfixes, I only want to show the two major improvements here.

1. FAKE 0.10 uses FSI instead of FSC

From now on FAKE uses the “F# Interactive” (fsi.exe) instead of the F# Compiler (fsc.exe) to run the build scripts, which brings two major improvements.

No TempPath for compiled binaries needed

Due to the fact that FAKE scripts are no longer compiled at the beginning of the build process, we don’t need a temporary folder for the created binaries.

Loading modules at runtime

The #load command in F# scripts allows us to load modules at runtime. Now we are able to put reusable Targets or TargetTemplates (see below) into external build script files.

2. TargetTemplates

TargetTemplates provide an easy way to reuse common Targets. Let’s consider a (very) small sample:

This Target “TraceHello” traces a “Hello World” string into our build log. Now we want it to be slightly more generic and to trace a custom string. We can do this by using a TargetTemplate:

Now we have a template (or a function which generates targets) that gets a string for the target name and a string for the trace text and generates  a usable target:

Of course the TargetTemplate function is generic and can be used with any tuple as parameter:

One of the nice properties of functional programming languages is the easy extensibility of custom functions. Let’s consider a simple F# function (from “FAKE – F# Make”) for a recursive directory copy:

open System
open System.IO

/// Copies a directory recursive
/// Thanks to Robert Pickering http://strangelights.com/blog/
///  param target: target directory : string
///  param source: source directory : string
let CopyDir target source =
  Directory.GetFiles(source, "*.*", SearchOption.AllDirectories)
    |> Seq.iter (fun file ->
      let newFile = target + file.Remove(0, source.Length)
      printf "%s => %s" file newFile
      Directory.CreateDirectory(Path.GetDirectoryName(newFile)) |> ignore
      File.Copy(file, newFile, true))

If we want to allow users to set custom file filters, we can add a third parameter:

/// Copies a directory recursive
/// and allows to filter the files
/// Thanks to Robert Pickering http://strangelights.com/blog/
///  param target: target directory : string
///  param source: source directory : string
///  param filterFile: FilterFunction: string -> bool
let CopyDirFiltered target source filterFile =
  Directory.GetFiles(source, "*.*", SearchOption.AllDirectories)
    |> Seq.filter filterFile
    |> Seq.iter (fun file ->
      let newFile = target + file.Remove(0, source.Length)
      printfn "%s => %s" file newFile
      Directory.CreateDirectory(Path.GetDirectoryName(newFile)) |> ignore
      File.Copy(file, newFile, true))

Now we can define some filter functions:

/// Exclude SVN files (path with .svn)
/// excludeSVNFiles: string -> bool
let excludeSVNFiles (path:string) = not <| path.Contains ".svn"

/// Includes all files
/// allFiles: string -> bool 
let allFiles (path:string) = true

Now it is possible to use CopyDirFiltered in the following ways:

/// Copies all files <=> same as CopyDir
CopyDirFiltered "C:\\target" "C:\\source" allFiles

/// Copies all files except SVN files
CopyDirFiltered "C:\\target" "C:\\source" excludeSVNFiles

/// Copies all files only if random number <> 2
let r = new Random()
CopyDirFiltered "C:\\target" "C:\\source" (fun path -> r.Next(5) <> 2)

Extensibility of functions in C#

Of course we can do the same thing in C# 3.0:

/// <summary>
/// Copies a directory recursive
/// and allows to filter the files
/// </summary>
/// <param name="target">The target.</param>
/// <param name="source">The source.</param>
/// <param name="fileFilter">The file filter.</param>
public static void CopyDirFiltered(string target, string source,
                                   Func<string, bool> fileFilter)
{
    string[] allFiles = Directory.GetFiles(
        source, "*.*", SearchOption.AllDirectories);
    foreach (string file in from f in allFiles
                            where fileFilter(f)
                            select f)
    {
        string newFile = target + file.Remove(0, source.Length);
        Console.WriteLine("{0} => {1}", file, newFile);
        Directory.CreateDirectory(Path.GetDirectoryName(newFile));
        File.Copy(file, newFile, true);
    }
}

Now it is easy to use the C# function with lambdas:

“A lambda expression is an anonymous function that can contain expressions and statements, and can be used to create delegates or expression tree types.”

[MSDN]

Func<string, bool> filterSVN = x => !x.Contains(".svn");
Func<string, bool> allFiles = x => true;

/// Copies all files <=> same as CopyDir
CopyDirFiltered("C:\\target", "C:\\source", allFiles);

/// Copies all files except SVN files
CopyDirFiltered("C:\\target", "C:\\source", filterSVN);

/// Copies all files only if random number <> 2
var r = new Random();
CopyDirFiltered("C:\\target", "C:\\source", path => r.Next(5) != 2);

Keeping this simple technique in mind allows to create very flexible functions.

Yesterday I was talking about F# at the .NET Developer Group Braunschweig. It was my first talk completely without PowerPoint (just Live-Coding and FlipChart) and I have to admit this is not that easy. But the event was really a big fun and we covered a lot of topics like FP fundamentals, concurrency and domain specific languages (of course I showed “FAKE – F# Make”).

Now I have a bit time before I go to the next BootCamp in Leipzig. Today Christian Weyer will show us exciting new stuff about WCF and Azure.

In the meanwhile I will write here about another important question (see first article) from the F# BootCamp in Leipzig:

Question 4 – Try to explain “Currying” and “Partial Application”. Hint: Please show a sample and use the pipe operator |>.

Obviously this was a tricky question for FP beginners. There are a lot of websites, which give a formal mathematical definition but don’t show the practical application.

“Currying … is the technique of transforming a function that takes multiple arguments (or more accurately an n-tuple as argument) in such a way that it can be called as a chain of functions each with a single argument”

[Wikipedia]

I want to show how my pragmatic view of the terms here, so let’s consider this small C# function:

public int Add(int x, int y)
{
   return x + y;
}

Of course the corresponding F# version looks nearly the same:

let add(x,y) = x + y

But let’s look at the signature: val add : int * int –> int. The F# compiler is telling us add wants a tuple of ints and returns an int. We could rewrite the function with one blank to understand this better:

let add (x,y) = x + y

As you can see the add function actually needs only one argument – a tuple:

let t = (3,4)         // val t : int * int
printfn "%d" (add t)  // prints 7 – like add(3,4)

Now we want to curry this function. If you’d ask a mathematician this a complex operation, but from a pragmatic view it couldn’t be easier. Just remove the brackets and the comma – that’s all:

let add x y = x + y

Now the signature looks different: val add : int -> int –> int

But what’s the meaning of this new arrow? Basically we can say if we give one int parameter to our add function we will get a function back that will take only one int parameter and returns an int.

let increment = add 1      // val increment : (int -> int)
printfn "%d" (increment 2) // prints 3

Here “increment” is a new function that uses partial application of the curryied add function. This means we are fixing one of the parameters of add to get a new function with one parameter less.

But why are doing something like this? Wouldn’t it be enough to use the following increment function?

let add(x,y) = x + y       // val add : int * int -> int
let increment x = add(x,1) // val increment : int -> int
printfn "%d" (increment 2) // prints 3

Of course we are getting (nearly) the same signature for increment. But the difference is that we can not use the forward pipe operator |> here. The pipe operator will help us to express things in the way we are thinking about it.

Let’s say we want to filter all even elements in a list, then calculate the sum and finally square this sum and print it to the console. The C# code would look like this:

var list = new List<int> {4,2,6,5,9,3,8,1,3,0};
Console.WriteLine(Square(CalculateSum(FilterEven(list))));

If we don’t want to store intermediate results we have to write our algorithm in reverse order and with heavily use of brackets. The function we want to apply last has to be written first. This is not the way we think about it.

With the help of curried functions, partial application and the pipe operator we can write the same thing in F#:

let list = [4; 2; 6; 5; 9; 3; 8; 1; 3; 0]

let square x = x * x

list
 |> List.filter (fun x -> x % 2 = 0) // partial application
 |> List.sum
 |> square
 |> printfn "%A"                     // partial application

We describe the data flow in exactly the same order we talked about it. Basically the pipe operator take the result of a function and puts it as the last parameter into the next function.

What should we learn from this sample?

1. Currying has nothing to do with spicy chicken.
2. The |> operator makes life easier and code better to understand.
3. If we want to use |> we need curryied functions.
4. Defining curryied functions is easy – just remove brackets and comma.
5. We don’t need the complete mathematical theory to use currying.
6. Be careful with the order of the parameter in a curryied function. Don’t forget the pipe operator puts the parameter from the right hand side into your function – all other parameters have to be fixed with partial application.

Easy TeamCity integration is one of the major goals for the FAKE build system.

“TeamCity is a Java-based build management and continuous integration server from JetBrains, creators of IntelliJ IDEA and ReSharper.”

[Wikipedia]

In this article I will show you how you can set up a FAKE build script in TeamCity. We will use the CalculatorSample which you can download from the FAKE Download page. If you want to know how this build script works read the “Getting started with FAKE”-tutorial.

Installing TeamCity

You can download the free professional edition of TeamCity from http://www.jetbrains.com/teamcity/. After the installation process (further instructions) you should be ready to configure your first build:

Creating a FAKE Project

Now create a new project and add a build configuration:

Attach a VCS root

The next step is to attach a VCS root. For this sample we will use the Calculator SVN root at http://fake.googlecode.com/svn/trunk/Samples/Calculator/.

[The only issue with this setting is, that you can’t change the path to the F# compiler. It is fixed to c:\Program Files (x86)\FSharp-1.9.6.2\bin\. If your path is different just reinstall F# to this location or use your own Version Control System.]

Choosing a build runner

We can use the Command Line build runner to start the completeBuild.fsx build script via Fake.exe. As the build provides NUnit test results we instruct TeamCity to import them:

If you want you could also add a build trigger to your build script:

Running the build

Now if everything is configured correctly, you can run your build and the output should look like:

You can also inspect the NUnit and FxCop results:

“FAKE – F# Make” is intended to be an extensible build framework. That’s why I tried to make it as easy as possible to create custom tasks. This posts shows how we can create a (very simple) custom task in C# which gives a random number.

First of all open Visual Studio and create a new C# class library called my MyCustomTask and create a class called RandomNumberTask:

using System;

namespace MyCustomTask
{
    public class RandomNumberTask
    {
        public static int RandomNumber(int min, int max)
        {
            var random = new Random();
            return random.Next(min, max);
        }
    }
}

Now compile this project and put the resulting assembly and debug symbols into the tools/FAKE path of your project. Now you can use your CustomTask in the build script:

#light
// include Fake libs
#I "tools\FAKE"
#r "FakeLib.dll"

// include CustomTask
#r "MyCustomTask.dll"
open Fake 

// open CustomNamespace
open MyCustomTask

// use custom functionality
let x = RandomNumberTask.RandomNumber(2,13)
sprintf "RandomNumber: %d" x |> trace

You can use every .Net class with FAKE. Just put the assembly in the right folder (tools/FAKE) and include it with the #r command.

If you want to use FAKE standard functionality (like globbing) within your CustomTask project, just reference FakeLib.dll and explore the FAKE namespace.

In the last article I showed how we can use “FAKE – F# Make” to set up a build script which

1. Cleans up old build outputs
2. Compiles our main projects
3. Compiles test projects
4. Uses NUnit to test our assemblies
5. Zips the assemblies to a deploy folder.

This time we will improve the same Calculator sample (download here) with a task for FxCop, so please make sure you succeeded with the last article.

“FxCop is a free static code analysis tool from Microsoft that checks .NET managed code assemblies for conformance to Microsoft’s .NET Framework Design Guidelines.”

[Wikipedia]

Setting up FxCop

First of all you have to download and install FxCop. Now open build.fsx from you Calculator sample folder and add a new target “FxCop” to the targets section:

Target "FxCop" (fun () ->
  let assemblies =
    !+ (buildDir + @"\**\*.dll")
      ++ (buildDir + @"\**\*.exe")
      |> Scan
  FxCop
    (fun p ->
      {p with
        // override default parameters
        ReportFileName = testDir + "FXCopResults.xml";
        ToolPath = @"c:\Program Files (x86)\Microsoft FxCop 1.36\"})
    assemblies
)

In the dependencies section modify the dependencies of the “Test” targets:

"Test" <== ["BuildApp"; "BuildTest"; "FxCop"]

That’s it. If you run your build script you will get new *.xml file in the .\test\-folder:

There are a lot of parameters for the FxCop task. Some are described on the project page.

Letting the build fail

If you were using MSBuild before you might know how hard it is to let MSBuild fail your build if FxCop reports any errors or warnings.

With FAKE the only thing you have to do is setting the “FailOnError” parameter:

Target "FxCop" (fun () ->
  let assemblies =
    !+ (buildDir + @"\**\*.dll")
      ++ (buildDir + @"\**\*.exe")
      |> Scan  

  FxCop
    (fun p ->
      {p with
        // override default parameters
        ReportFileName = testDir + "FXCopResults.xml";
        FailOnError = FxCopyErrorLevel.CriticalWarning;
        ToolPath = fxCopPath})
    assemblies
)

If you activate this option FxCop errors will cause your build to fail. Possible values are:

• FxCopyErrorLevel.Warning
• FxCopyErrorLevel.CriticalWarning
• FxCopyErrorLevel.Error
• FxCopyErrorLevel.CriticalError
• FxCopyErrorLevel.ToolError
• FxCopyErrorLevel.DontFailBuild

The values are cummulative. If you choose FxCopyErrorLevel.CriticalWarning the build will fail for critical warnings, errors, critical errors and FxCop tool errors but not for simple warnings. The default is FxCopyErrorLevel.DontFailBuild.

In this tutorial I will describe how you can set up a complete build infrastructure with “FAKE – F# Make”. You will learn:

• How to automatically compile your C# or F# projects
• How to automatically run NUnit UnitTests on your projects
• How to zip the output to a deployment folder

Install F#

“FAKE – F# Make” is completely written in F# and all build scripts will also be written in F#, but this doesn’t imply you have to learn programming in F#. In fact the “FAKE – F# Make” syntax is very easy to learn. But if you need to you can use the complete power of F# and the .NET Framework.

But in order to get things working we need to install the F# environment. You can download the F# Oktober 2009 CTP from the Microsoft F# Developer Center.

Download Calculator Sample

Now download the latest CalculatorSample-*.zip from the FAKE Download site. This sample includes 3 tiny projects and basically the following structure:

• src\app
  • Calculator (Command line)
  • CalculatorLib (Class library)
• src\test
  • Test.CalculatorLib (NUnit test library)
• tools
  • FAKE Assemblies
  • NUnit
• build.bat
• build.fsx
• completeBuild.bat
• completeBuild.fsx
• Calculator.sln

Getting “FAKE – F# Make” started

Open the file tools\FAKE\FakeLib.dll.config and check if the default configuration matches your system environment:

If you run the build.bat from the command line then your first FAKE script (build.fsx) will be executed. If everything works fine you will get the following output:

Now open the build.fsx with Visual Studio. It should look like this:

As you can see the code is really simple. The very first lines include the FAKE libraries and are vital in all FAKE build scripts.

Then the Default target is defined. A target definition contains of two important parts. The first is the name of the target (here “Default”) and the second is an action (here a simple trace of a “Hello world” string). The action is defined as a lambda expression.

The last line runs the “Default” target – which means it executes the defined action.

Cleaning the last build output

The first thing in nearly every build scenario is to clean the output of the last build. Please modify the build.fsx file to the following:

We introduced some new concepts in this snippet. At first we defined a global property called “buildDir” with the relative directory path of our future build folder.

In the Clean target we want the CleanDir task to clean up our build directory. This simply deletes all files in the old folder or creates it if necessary.

In the dependencies section we say that the Default target is dependent of the Clean target. In other words Clean is a prerequisite of Default and will be run before the execution of Default:

Building the application

In the next step we want to compile our application, which means we want to compile all projects under /src/app with MSBuild.

We defined a new build target named “BuildApp” which compiles all project files given in the property appReferences via the MSBuild task. The output will be copied to buildDir.

In order to find the right project files “FAKE – F# Make” scans the folder src/app/ and all subfolders with the given pattern. Therefore a similar FileSet definition like in NAnt or MSBuild (see project page for details) is used.

In addition the target dependencies are modified. Now Default is dependent of BuildApp and BuildApp needs Clean as a prerequisite.

This means the execution order is: Clean ==> BuildApp ==> Default.

Building Test projects

Now our main application will be built automatically and it’s time to build the test application. We use pretty the same concepts as before:

This time we define a new target “BuildTest”, which compiles all C# projects below src/test/ in Debug mode and we mark the target as a dependency of Default.

As you can see the Clean target is now a dependency of BuildApp and BuildTest, since we also have to clean the test dir. Although the Clean target is referenced twice the “FAKE – F# Make” runtime ensures that it will only be executed once.

The execution order is now Clean ==> BuildApp ==> BuildTest ==> Default.

Testing the test assemblies with NUnit

Now all our projects will be compiled and we can use the NUnit task in order to test our test assemblies:

Our new target “Test” scans the test directory for test assemblies and runs them with NUnit. The mysterious part (fun p –> …) simply overrides the default parameters of the NUnit task.

The execution order is now Clean ==> BuildApp ==> BuildTest ==> Test ==> Default.

Deploying a zip file

Now we want to deploy a *.zip file containing our application:

The new target “Deploy” scans the build directory for all files but zip-files. The result will be zipped to \deploy\Calculator.zip via the Zip task.

The execution order is now Clean ==> BuildApp ==> BuildTest ==> Test ==> Deploy ==> Default whereas Default now is only used as a entry point.

What’s next?

Now your build file should look like completeBuild.fsx and you are ready to write your own FAKE scripts. If you have any questions or suggestions feel free to comment on this post.

In the next article I will show how we can add FxCop to our build in order to check the code quality.