{"id":1742,"date":"2016-04-25T13:44:22","date_gmt":"2016-04-25T13:44:22","guid":{"rendered":"http:\/\/www.navision-blog.de\/blog\/?p=1742"},"modified":"2016-04-26T11:11:50","modified_gmt":"2016-04-26T11:11:50","slug":"make-failure-great-again-a-small-journey-into-the-f-compiler","status":"publish","type":"post","link":"http:\/\/www.navision-blog.de\/blog\/2016\/04\/25\/make-failure-great-again-a-small-journey-into-the-f-compiler\/","title":{"rendered":"&#8220;Make failure great again&#8221; &#8211; a small journey into the F# compiler"},"content":{"rendered":"<p>Giving good error messages is very important for modern compilers. The language <a href=\"http:\/\/elm-lang.org\/\">Elm<\/a> is famous for giving <a href=\"http:\/\/elm-lang.org\/blog\/compiler-errors-for-humans\">excellent user-friendly error messages<\/a>. Parts of the F# community decided to improve the F# compiler messages\u00a0as well. We want to make that a community effort and hope many people join us and help to make the compiler emit better error messages.\u00a0\u00a0<a href=\"http:\/\/compositional-it.com\/\">Isaac Abraham<\/a>\u00a0is mainatining a <a href=\"https:\/\/github.com\/Microsoft\/visualfsharp\/issues\/1103\">list with possible targets<\/a>, so if you are interested in helping then take a look at this list.<\/p>\n<p>Unfortunately working on compilers it&#8217;s not exactly the easiest task. In this post I want to show you how I tried to improve one compiler warning and hope this helps others to get started.<\/p>\n<p>The task for this post is to improve the compiler warning\u00a0for the following code:<\/p>\n<p><script src=\"https:\/\/gist.github.com\/forki\/032899ef2220d94743c67c2e8d1ceb1f.js\"><\/script><\/p>\n<p>Currently the compiler shows the following:<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/IgnoreWarning.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/IgnoreWarningS.png\" alt=\"Original compiler warning\" \/><\/a><\/p>\n<p>According to <a href=\"https:\/\/github.com\/Microsoft\/visualfsharp\/issues\/1109\">issue #1109<\/a>\u00a0we want to turn it into:<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/AssignmentWarning.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/AssignmentWarningS.png\" alt=\"New descriptive compiler warning\" \/><\/a><\/p>\n<p>As you can see the new message is giving the user more details about the specific situation and also refers to a common error of newcomers.<\/p>\n<p>Please notice that this is probably not the final wording for this warning, but once we have finished this\u00a0it will be easy to iterate on the concrete warning.<\/p>\n<h5>Getting started<\/h5>\n<p>The first thing to is to get the compiler code on your machine. I&#8217;m using Windows and Visual Studio 2015 here. So what you want to do is the following:<\/p>\n<ul>\n<li>Fork <a href=\"https:\/\/github.com\/Microsoft\/visualfsharp\">https:\/\/github.com\/Microsoft\/visualfsharp<\/a> on github<\/li>\n<li>Start a &#8220;Developer command prompt for Visual Studio 2015&#8221; in admin mode<\/li>\n<li>Run `git clone [your clone url]` and change into the new directory<\/li>\n<li>Run `git remote add upstream\u00a0https:\/\/github.com\/Microsoft\/visualfsharp.git`<\/li>\n<li>Optionally: If you want to follow the exact steps from this post, then it makes sense to set your clone to the same <a href=\"https:\/\/github.com\/Microsoft\/visualfsharp\/commit\/8a2e393999f440ea93769a288c37172d98db455a\">commit where I started<\/a>. You can do that by executing\u00a0&#8220;git reset &#8211;hard\u00a08a2e393999f440ea93769a288c37172d98db455a&#8221;<\/li>\n<li>Run\u00a0`build.cmd`<\/li>\n<\/ul>\n<p>This last step will download the internet for you, so don&#8217;t be too surprised when you find the lastest &#8220;Game of Thrones&#8221; episode somewhere inside your packages folder.<\/p>\n<p>Anyways, if everything works then the output should look like the following:<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/BuildWorks.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/BuildWorksS.png\" alt=\"Build works from command line\" \/><\/a><\/p>\n<p>If the build doesn&#8217;t work for you then read the <a href=\"https:\/\/github.com\/Microsoft\/visualfsharp\/blob\/master\/DEVGUIDE.md\">DEVGUIDE<\/a> very very carefully. If that doesn&#8217;t help then please open an issue in the <a href=\"https:\/\/github.com\/Microsoft\/visualfsharp\/issues\">Visual F# issue tracker<\/a>.<\/p>\n<h5>Reproducing the compiler warning<\/h5>\n<p>If the build works from command line we should try to reproduce the error with our own freshly built compiler. Let&#8217;s open the solution in Visual Studio 2015, set the startup project to &#8220;FsiAnyCPU&#8221; and hit F5 to start a debug session in Visual Studio. If everything works we will see a F# Interactive Window and Visual Studio is in Debug mode:<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/DebugInVS.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/DebugInVSsmall.png\" alt=\"Debug works in Visual Studio\" \/><\/a><\/p>\n<p>Now we paste our code into the command line window:<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/IgnoreWarning.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/IgnoreWarningS.png\" alt=\"Original compiler warning\" \/><\/a><\/p>\n<p>Using the F# Interactive in debug mode is an easy way to do compiler debugging. There are other possibilities (like using fsc.exe), but the F# Interactive allows you to debug the compilation of single lines in isolation from the code above. It&#8217;s really neat.<\/p>\n<h5>Where to start with\u00a0hacking<\/h5>\n<p>Since we now know how to reproduce the compiler warning, we want to make actual code changes. But where!?<\/p>\n<p><img src=\"http:\/\/www.navision-blog.de\/images\/whatnext.gif\" alt=\"Where??\" \/><\/p>\n<p>An easy way to get started is by using\u00a0the old error message as a search string. When we use Visual Studio&#8217;s solution-wide search and look for &#8220;This expression should have type&#8221; we find exactly two locations inside a file called FSStrings.resx file.<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/ErrorText.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/ErrorTextS.png\" alt=\"Searching for original compiler warning\" \/><\/a><\/p>\n<p>So this gives us a new search string &#8220;UnitTypeExpected1&#8221;.<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/CompileOps.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/CompileOpsS.png\" alt=\"Searching for UnitTypeExpected1\" \/><\/a><\/p>\n<p>This looks actually like we found something interesting in CompileOps.fs. Let set a breakpoint and see if it stops when we reproduce our warning:<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/Breakpoint.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/BreakpointS.png\" alt=\"Debugging\" \/><\/a><\/p>\n<p>Cool, so this seems to be the place where the warning gets emitted. Now we need to find the place where the discriminated union case &#8220;UnitTypeExpected&#8221; is created. After another little search we are in TypeChecker.fs:<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/TypeChecker.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/TypeCheckerS.png\" alt=\"Type checker\" \/><\/a><\/p>\n<p>We can try to set another breakpoint and see what happens in that code. The interesting part is the inner else part. <\/p>\n<p><img src=\"http:\/\/www.navision-blog.de\/images\/Baby.gif\" alt=\"Baby\" \/><\/p>\n<p>The code in there looks scary and cryptic at first, but it actually\u00a0shows the power of using a ML dialect for building compilers. Inside this code the compiler already knows that we want to emit a warning. By pattern matching on the abstract syntax tree (AST) we decide if the left side of the expression is a property. In that case the compiler seem to emit a different warning (see also that code above from CompileOps.fs).<\/p>\n<p>So after hitting the breakpoint and looking in the value of\u00a0<em>exprOpt<\/em>\u00a0I&#8217;m pretty sure we are in the right place:<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/UsingDebugger.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/UsingDebuggerS.png\" alt=\"Inspect AST\" \/><\/a><\/p>\n<p>So somewhere inside that AST we can actually find our name <em>x<\/em>. Now we have all we need.<\/p>\n<h5>Making the code changes<\/h5>\n<p>It&#8217;s good practice to never make code changes on the master branch. So let&#8217;s create a feature branch with &#8220;git checkout -b assignment-warning&#8221;.<\/p>\n<p>Now let&#8217;s start by adding our new compiler warning to FSStrings.resx:<\/p>\n<p><script src=\"https:\/\/gist.github.com\/forki\/ee5d256df7370b6155810ac3868be823.js\"><\/script><\/p>\n<p>Now we go back to TypeChecker.fs and try to come up with some pattern matching on the AST that finds our special case. After playing around a bit I found:<\/p>\n<p><script src=\"https:\/\/gist.github.com\/forki\/23375eec72dc8e1be8193eba1569a315.js\"><\/script><\/p>\n<p>This doesn&#8217;t look super nice, but it follows one of the important rules from the<a href=\"https:\/\/github.com\/Microsoft\/visualfsharp\/blob\/master\/CONTRIBUTING.md#coding-guidelines\"> contributor guide<\/a> &#8211; &#8220;use similar style to nearby existing code&#8221;. We probably want to tidy this up later when initial code review is done. But in the beginning we should stay as close to the original as possible.<\/p>\n<p>Since we decided\u00a0to store additional data, this\u00a0already tells us we need to extent the\u00a0UnitTypeExpected type in TypeChecker.fsi and TypeChecker.fs. After that we can go to\u00a0CompileOps.fs and use that data:<\/p>\n<p><script src=\"https:\/\/gist.github.com\/forki\/28e41949d1fb7c3e1fb063a140e60087.js\"><\/script><\/p>\n<p>That&#8217;s basically all we need to do. Let&#8217;s fix the the remaining errors (don&#8217;t worry the compiler will show you all the places) and then reproduce the warning again:<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/AssignmentWarning.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/AssignmentWarningS.png\" alt=\"New descriptive compiler warning\" \/><\/a><\/p>\n<p>Wow. We did it. We actually changed the compiler. And it didn&#8217;t even hurt that much.<\/p>\n<p><img src=\"http:\/\/www.navision-blog.de\/images\/YayCompilers.gif\" alt=\"Compiler changed\" \/><\/p>\n<h5>Unit tests<\/h5>\n<p>Compiler tests (especially if you want to tests negative results like warnings or exceptions) can&#8217;t be easily done in standard unit tests. The F# team came up with a different approach where the test runner compiles a .fs file and you can use regular expressions to match against the compiler output.<\/p>\n<p>In order to run the existing test suite call &#8220;build.cmd all&#8221;.<\/p>\n<p><img src=\"http:\/\/www.navision-blog.de\/images\/RunTests.gif\" alt=\"Are you afraid?\" \/><\/p>\n<p>This will take a while, but after all tests are done we should see:<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/TestResults.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/TestResultsS.png\" alt=\"Test results\" \/><\/a><\/p>\n<p>So good news is we didn&#8217;t break anything, but the bad news is that we don&#8217;t have tests covering our case. So let&#8217;s add one.<\/p>\n<p>We create tests\/fsharpqa\/Source\/Warnings\/WarnIfPossibleAssignment.fs:<\/p>\n<p><script src=\"https:\/\/gist.github.com\/forki\/a54c8147edb5aa36d4c2a58e17a04d95.js\"><\/script><\/p>\n<p>Now we need to include the test in a test suite. Since there isn&#8217;t a good fitting existing suite we need to create tests\/fsharpqa\/Source\/Warnings\/env.lst:<\/p>\n<p><script src=\"https:\/\/gist.github.com\/forki\/f5d135c3b9a9b81c9d2701efb7678ffa.js\"><\/script><\/p>\n<p>And we need to register this new suite in \/tests\/fsharpqa\/Source\/test.lst:<\/p>\n<p><script src=\"https:\/\/gist.github.com\/forki\/c4205c34f8d0dd5ec04820a7b052a8af.js\"><\/script><\/p>\n<p>Now we can run the test with &#8220;src\\RunTests.cmd release fsharpqa Misc01&#8221; (more details can be found in the <a href=\"https:\/\/github.com\/Microsoft\/visualfsharp\/blob\/master\/TESTGUIDE.md#runtestscmd\">TESTGUIDE<\/a>):<\/p>\n<p><a href=\"http:\/\/www.navision-blog.de\/images\/TestFailed.png\"><img src=\"http:\/\/www.navision-blog.de\/images\/TestFailedS.png\" alt=\"Test failed\" \/><\/a><\/p>\n<p>So at least we now know that our test gets executed \ud83d\ude09<br \/>\n The concrete test results can be found in a file called 258,259FSharpQA_Failures.log alongside our WarnIfPossibleAssignment.fs:<\/p>\n<p><script src=\"https:\/\/gist.github.com\/forki\/e68a3dcf9aed5535fe55c9fbc46eb988.js\"><\/script><\/p>\n<p>So it seems we cannot match the same line twice. For simplicitly of this blog post we will go with a reduced test:<\/p>\n<p><script src=\"https:\/\/gist.github.com\/forki\/620d4b885c300642ec1e7343517cde1a.js\"><\/script><\/p>\n<h5>Wrapping it up<\/h5>\n<p>After we ran &#8220;build.cmd all&#8221; again it&#8217;s now time for the final phase. Commit the changes locally and use &#8220;git push origin assignment-warning&#8221; to publish your code.<br \/>\nNow go to the <a href=\"https:\/\/github.com\/Microsoft\/visualfsharp\">Visual F# GitHub repo<\/a> and create a pull request with an explanation of what you did and what you intended to fix. Often it&#8217;s also useful to include one or two screenshots.<\/p>\n<p>Your PR will then go through code review process. Usually you will get a lot of questions and probably you need to change some things in your proposal. This process can take a while so\u00a0bring a bit of patience.<\/p>\n<p>Anyways, here is <a href=\"https:\/\/github.com\/Microsoft\/visualfsharp\/pull\/1115\">my PR for this change<\/a>\u00a0&#8211; wish me<\/p>\n<p><img src=\"http:\/\/www.navision-blog.de\/images\/goodluck.gif\" alt=\"Good luck!\" \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Giving good error messages is very important for modern compilers. The language Elm is famous for giving excellent user-friendly error messages. Parts of the F# community decided to improve the F# compiler messages\u00a0as well. We want to make that a community effort and hope many people join us and help to make the compiler emit [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1,448,8],"tags":[],"_links":{"self":[{"href":"http:\/\/www.navision-blog.de\/blog\/wp-json\/wp\/v2\/posts\/1742"}],"collection":[{"href":"http:\/\/www.navision-blog.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.navision-blog.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.navision-blog.de\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.navision-blog.de\/blog\/wp-json\/wp\/v2\/comments?post=1742"}],"version-history":[{"count":54,"href":"http:\/\/www.navision-blog.de\/blog\/wp-json\/wp\/v2\/posts\/1742\/revisions"}],"predecessor-version":[{"id":1796,"href":"http:\/\/www.navision-blog.de\/blog\/wp-json\/wp\/v2\/posts\/1742\/revisions\/1796"}],"wp:attachment":[{"href":"http:\/\/www.navision-blog.de\/blog\/wp-json\/wp\/v2\/media?parent=1742"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.navision-blog.de\/blog\/wp-json\/wp\/v2\/categories?post=1742"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.navision-blog.de\/blog\/wp-json\/wp\/v2\/tags?post=1742"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}