Rash thoughts about .NET, C#, F# and Dynamics NAV.

"Every solution will only lead to new problems."

Monday, 21. December 2015

FsAdvent: Automatic re-build and background tasks for suave.io websites

Filed under: .NET,F#,FAKE - F# Make,Visual Studio — Steffen Forkmann at 14:12 Uhr

[This post my second entry of the F# advent calendar 2015 series. You can also read the first post about “Using Async.Choice in Paket“]

Recently I was asked to help with a website that was based on suave.io. The task was to transform the existing F# script based suave.io website to something that can be run in the enterprise infrastructure and also performs some background tasks like creating Excel reports recurringly.

Suave as a (windows) service

A lot of the suave.io samples work with simple .fsx script files. This approach is indeed simple and allows fast prototyping with the use of the REPL. Even more complex websites like fssnip.net are using this approach with great success (code here). One of the benefits of script based suave solutions is that they are often equipped with a FAKE build script that rebuilds the website automatically if someone changes a file. One prominent example is FsReveal:

But sometimes you want to have a “real Visual Studio project” with debugging support and your enterprise might want to install your website as a Windows service. So the first step was to convert the F# script based approach into a project and adding TopShelf layer to make it runnable as Windows service. Thanks to a short blog post by Tomas Jansson this was super easy. It’s basically just installing TopShelf.FSharp via Paket and adding a single file with the config. Look into Tomas’s blog post to see the details.

After moving to the TopShelf model we still wanted to have automatic website rebuild whenever someone touched the source code. This little FAKE script makes this possible:

Running background jobs

The website already allowed to generate Excel reports by clicking somewhere in the UI. In addition to that we wanted to create the Excel reports recurringly in background jobs. Scott Hanselman has a blog post about “How to run Background Tasks in ASP.NET” and shows a couple of solutions that while optimized for ASP.NET would probably work with suave.io as well. But in this case we wanted to have something that syncronizes with the IO of the website so we decided to use F#’s MailboxProcessor feature a.k.a. “agents”. (As always: Scott Wlaschin has a nice introduction to agents.)

As a small intro into background jobs we start with F# counter agent sample by Tomas Petricek:

This little snippet shows an agent that calculates averages of the messages that it receives. Now let’s add a background job that recurringly sends messages to the counter agent:

In order to use this we need a bit of ugly infrastructure code that we will hide in a library:

So let’s try this out in the F# interactive:

Since this works as expected we can use it in our website:

Since all the user triggered reporting will also go through this taskAgent we ensure that IO is synchronized.

Sample project

In https://github.com/forki/backgroundjobs you can find a sample project which does exactly the two points from above.

As you can see the website performs background jobs and is rebuilding/restarting automatically when a source code file is saved.

Tags: , ,

Monday, 30. November 2015

F# advent calendar: Using Async.Choice in Paket

Filed under: .NET,C#,Diverses,F#,PLINQ — Steffen Forkmann at 0:50 Uhr

[This post is part of the F# advent calendar 2015 series.]

Prologue: What is Paket?

Paket is a dependency manager for .NET with support for NuGet packages and GitHub repositories. It enables precise and predictable control over what packages the projects within your application reference. If you want to learn how to use Paket then read the “Getting started” tutorial and take a look at the FAQs.

Paket file structure

Async computations in F#

Asynchronous Workflows are a very old F# feature (they already shipped in 2007) and they are used in many places in Paket. In this article I want to highlight one of the nice applications that F#’s async model allows you.

Many readers are probably familiar with C#’s async and await keywords (IIRC released with C# in 2012), but F#’s async feature works a little different. You can read more about some of “Asynchronous gotchas in C#” in Tomas Petricek’s excellent blog post. Most of these gotchas come from the fact that C# is starting all async tasks automatically while F# wraps the logic in data and allows you to run it explicitly. For a really good introduction to asynchronous programming with F# I can recommend Scott Wlaschin’s blog post.

The issue: retrieving version numbers from NuGet feeds

Paket’s package resolution algorithm (if you are interested in algorithms then read more) needs to know which versions are avalaible for a given package. Usually users specify more than one NuGet feed and different NuGet feeds support different protocols. The following code shows how older Paket versions retrieved all version numbers for a given package across all configured NuGet feeds:

As you can see getAllVersions tries 4 different NuGet protocols per source and returns the first result that is not None. Every getVersionsViaProtocol call performs async web request. In GetAllVersions we run this function for every NuGet source in parallel and combine the results. This code is very similar to the “async web downloader” sample in Scott Wlaschin’s async blog post and basically the “hello world” of asynchronous programming. 

Code cleanup

The getAllVersions is deeply nested and it’s hard to understand what’s going on. With the help of List.tryPick we can rewrite the code as:

List.tryPick returns the first result that is not None. So instead of nesting multiple match and let! expressions we use a higher-order-function to encapsulate the same pattern.

Introducing Async.Choice

After using this code for a while in Paket we noticed that different NuGet server implementations each implement a different subset of the 4 protocols and differ very much in the response times. So there was no order of the protocols that would work good for all server implementations. But what if we could query all 4 protocols in parallel and just take the fastest response?

The change is relatively easy and we can rewrite the GetAllVersions as:Instead of running the web requests synchronously and in order we run the four web request per NuGet feed in parallel and take the first response that is not None.

Implementation of Async.Choice

Unfortunately Async.Choice is not part of the standard FSharp.Core library and it turns out it is not easy to implement. There are many different implementations floating around on the internet. The one we use in Paket is by Eirik Tsarpalis and taken from fssnip. One of the advantages of this version is that the automatic cancellation of tasks works nicely. Since we are running lots and lots of web requests in parallel and always take the first response we want to cancel all the other pending web requests. Otherwise we would basically DDOS the feeds with useless requests (and yes that happened to us ;-)). Since Async.Choice is very useful we sent a pull request to the Visual F# project and hope that one day it will be in the box. 


Treating computations as data has even more advantages. The m-brace project created a new version of computation workflows called “cloud”:

cloud is a computation workflow builder and allows you to run your computations in the cloud. In contrast to async we don’t only control when something gets executed but also where. They even have a Cloud.Choice, which allows you to run tasks asynchronous in the cloud and take the first succeeding result.

Btw: async and cloud aren’t language keywords like C#’s async and await, but implemented as normal F# code in libraries. If you want to define your own computation expression builders, then the MSDN docs are a good starting point.

Tags: , , ,

Friday, 4. January 2013

F# and Microsoft Dynamics NAV 2013 OData Services

Filed under: C#,Diverses,Dynamics NAV 2009,Dynamics NAV 2013,F#,Visual Studio — Steffen Forkmann at 13:52 Uhr

In my last post I described how we can access Dynamics NAV 2009 SOAP web services from F# and the benefits we get by using a type provder. Since version 2013 it’s also possible to expose NAV pages via OData. In this article I will show you how the OData type provider which is part of F# 3 can help you to easily access this data.

Exposing the data

First of all follow this walkthrough and expose the Customer Page from Microsoft Dynamics NAV 2013 as an OData feed.

Show the available companies

Let’s try to connect to the OData feed and list all available companies. Therefore we create a new F# console project (.NET 4.0) in Visual Studio 2012 and add references to FSharp.Data.TypeProviders and System.Data.Services.Client. With the following snippet we can access and print the company names:

As you can see we don’t need to use the “Add Service Reference” dialog. All service type are generated on the fly.

Access data within a company

Unfortunately Dynamics NAV 2013 seems to have a bug in the generated metadata. In order to access data within a company we need to apply a small trick. In the following sample we create a modified data context which points directly to a company:

Now we can start to access the data:

As you can see this approach is very easy and avoids the problem with the manual code generation. If you expose more pages then they are instantly available in your code.

As with the Wsdl type provider you can expose the generated types from this F# project for use in C# projects.

Further information:

Tags: , , ,

Thursday, 3. January 2013

F# and Microsoft Dynamics NAV 2009 Web Services

Filed under: C#,Dynamics NAV 2009,F#,Navision,Visual Studio,WCF — Steffen Forkmann at 9:25 Uhr

If you are a Dynamics NAV developer you have probably heared of the web services feature which comes with the 2009 version. In this walkthrough you can learn how to create and consume a Codeunit Web Service. This method works very well if you only need to create the C# proxy classes once. If you have more than one developer, an automated build system, changing web services or many web services then you will come to a point where this code generation system is very difficult to handle.

Microsoft Visual F# 3.0 comes with feature called “type providers” that helps to simplify your life in such a situation. For the case of WCF the Wsdl type provider allows us to automate the proxy generation. Let’s see how this works.

Web service generation

In the first step we create the Dynamics NAV 2009 Codeunit web service in exactly the same way as in the MSDN walkthrough.

Connecting to the web service

Now we create a new F# console project (.NET 4.0 or 4.5) in Visual Studio 2012 and add references to FSharp.Data.TypeProvidersSystem.Runtime.Serialization and System.ServiceModel. After this we are ready to use the Wsdl type provider:

At this point the type provider creates the proxy classes in the background – no “add web reference” dialog is needed. The only thing we need to do is configuring the access security and consume the webservice:

Access from C#

This is so much easier than the C# version from the walkthrough. But if you want you can still access the provided types from C#. Just add a new C# project to the solution and reference the F# project. In the F# project rename Program.fs to Services.fs and expose the service via a new function:

In the C# project you can now access the service like this:

Changing the service

Now let’s see what happens if we change the service. Go to the Letters Codeunit in Dynamics NAV 2009 and add a second parameter (allLetters:Boolean) to the Capitalize method. After saving the Codeunit go back to the C# project and try to compile it again. As you can see the changes are directly reflected as a compile error.

In the next blog post I will show you how you can easily access a Dynamics NAV 2013 OData feed from F#.

Tags: , , ,