Writing efficient and reliable code with F# Type Providers

When we refactor code how can we be confident that we don't break anything?

3 of the most important things that allow us to refactor fearlessly are:

Side effect free - or pure - expressions
Statically typed expressions
Tests

In this article we will solely focus on the aspect of side effects and strictly speaking on how to identify them. Being able to identify side effects in our programs clearly is the precondition for eliminating them.

Why avoid side effects?

Continue reading →

Have you ever wanted to try out PureScript but were lacking a good way to get started?

If you

Have some prior functional programming knowledge - maybe you know Haskell,Elm,F#,or Scala,etc.
Want to solve a small task with PureScript
And want to get started quickly

This post is for you!

In this post we will walk through setting up and implementing a small exemplary PureScript application from scratch.

Continue reading →

In this article I would like to present a minimal implementation of a music data type and everything that is needed to turn that into audible sound from an Elm application.

We will see how to transcribe an existing composition - an excerpt from Chick Corea's Children's Songs No. 6 - and listen to the result right here,embedded in this article.

From a music data type to performance

My colleague Jonas recently pointed out the presentation Making Algorithmic Music by Donya Quick to me. Donya Quick shows how she uses the Haskell library Euterpea to produce algorithmic music.

It got me really excited about the idea of porting this to Elm and to be able to use this in web applications.

In the following we will see the core data types and algorithms from Euterpea ported to Elm. To focus on the core concepts the implementation is stripped down to the minimum that is required to transcribe and perform an existing polyphonic piece of music (for a single instrument).

Continue reading →

This post is about how to implement well structured,and purely functional command line applications in Scala using PureApp.

PureApp originated in an experiment while refactoring out some glue code of an interactive command line application. At the same time it was inspired by the Elm Architecture Pattern,and scalaz's SafeApp,as well as scalm.

To show the really cool things we can do with PureApp,we will implement a self-contained example application from scratch.

This application translates texts from and into different languages. And it provides basic user interactions via the command line.

The complete source code is compiled with tut. Every output (displayed as code comments) is generated by tut.
Continue reading →

In this post we will see how applicatives can be used for validation in Scala. It is an elegant approach. Especially when compared to an object-oriented way.

Usually when we have operations that can fail,we have them return types like Option or Try. We sequence operations and once there is an error the computation is short circuited and the result is a None or a Failure.

Applicatives allow us to compose independent operations and evaluate each one. Even if an intermediate evaluation fails. This allows us to collect error messages instead of returning only the first error that occurred.

A classic example where this is useful is the validation of user input. We would like to return a list of all invalid inputs rather than aborting the evaluation after the first error.

Scala Cats provides a type that does exactly that. So let's dive into some code and see how it works.

Continue reading →

After some initial struggles,the chapter Functional Parsers from the great book Programming in Haskell by Graham Hutton,where a basic parser library is built from scratch,significantly helped me to finally understand the core ideas of parser combinators and how to apply them to other programming languages other than Haskell as well.

While I recently revisited the material and started to port the examples to Scala I wasn't able to define a proper monad instance for the type Parser[A].

The type Parser[A] alias was defined like this:

type Parser[A] = String =>Option[(A,String)] // defined type alias Parser

To test the monad laws with discipline I had to provide an instance of Eq[Parser[A]]. Because Parser[A] is a function,equality could only be approximated by showing degrees of function equivalence,which is not a trivial task.

Also the implementation of tailRecM was challenging. (I couldn't figure it out.)

Using existing abstractions

Continue reading →

Most config libraries use a stringly typed approach.

Some handle runtime failures due to invalid configuration schemas by leveraging data types like Option or Result to represent missing values or errors. This allows us to handle these failures by either providing default values or by providing decent error messages.

This is a good strategy that we should definitely stick to.

However,the problem with default values is that we might not even notice if the configuration is broken. This could potentially fail in production. In any case an error e.g. due to a misspelled config property will be observable at runtime at the earliest.

Wouldn't it be a great user experience (for us developers) if the compiler told us if the configuration schema is invalid? Even better,imagine we could access the configuration data in a strongly typed way like any other data structure,and with autocompletion.

Moreover,what if we didn't have to write any glue code,not even when the configuration schema changes?

This can be done with the costs of an initial setup that won't take more than probably around 5 minutes.

Continue reading →

In this post I will look at a practical example where the combined application (through monad transformers) of the state monad and the either monad can be very useful.

I won't go into much theory,but instead demonstrate the problem and then slowly build it up to resolve it.

You don't have to be completely familiar with all the concepts as the examples will be easy to follow. Here is a very brief overview:

Continue reading →

If your API overflows with Boolean parameters,this is usually a bad smell.

Consider the following function call for example:

toContactInfoList(csv,true,true)

When looking at this snippet of code it is not very clear what kind of effect the two Boolean parameters will have exactly. In fact,we would probably be without a clue.

We have to inspect the documentation or at least the parameter names of the function declaration to get a better idea. But still,this doesn't solve all of our problems.

The more Boolean parameters there are,the easier it will be for the caller to mix them up. We have to be very careful.

Moreover,functions with Boolean parameters must have conditional logic like if or case statements inside. With a growing number of conditional statements,the number of possible execution paths will grow exponentially. It will become more difficult to reason about the implementation code.

Can we do better?

Sure we can. Lambdas and combinators come to the rescue and I'm going to show this with a simple example,a refactoring of the function from above.

This post is based on a great article by John A De Goes,Destroy All Ifs — A Perspective from Functional Programming.

I'm going to take John's ideas that he backed up with PureScript examples and present how the same thing can be elegantly achieved in Scala.

Continue reading →

I'm currently working on an sbt plugin that generates Scala case classes at compile time to model JSON API responses for easy deserialization especially with the Scala play-json library.

The plugin makes it possible to access JSON documents in a statically typed way including auto-completion. It takes a sample JSON document as input (either from a file or a URL) and generates Scala types that can be used to read data with the same structure.

Let's look at a basic example,an app that prints the current Bitcoin price to the console.

Continue reading →

F# type providers are just awesome because they help to write very efficient and reliable code.

In this post I will show this by implementing a simple, but real-world-like scenario with some F# type providers.

Type providers provide the types, properties and methods to get access to external data sources of various kinds without having to write a lot of boiler-plate code. This makes coding very efficient.

Additionally they offer static types that represent external data and that the compiler will check at compile time. This makes coding very reliable.

So let's look at the scenario that we are going to implement...

Scenario

We are going to implement a command line tool with the following features:

Import email addresses and names from a CSV file into a MS SQL database
Delete all emails from the database

The user should provide the CSV file path as a command line argument.

The user should specify the connection string in the application's configuration file.

Implementation

Main types

We will begin with creating a new F# console application project with Visual Studio and then by adding the following types:

type Email = Email of string

type Name = Name of string

type MailingListEntry = {
    Email:Email
    Name:Name }

Accessing the Database

For database access we will use the SqlDataConnection type provider, which is a native F# feature.

We have to add references to FSharp.Data.TypeProviders, as well as System.Data, and System.Data.Linq.

Now we can set-up the type provider with the connection string of the database, which we will substitute later with the connection string from the application's configuration file.

module DataAccess =
    open Microsoft.FSharp.Data.TypeProviders

    [<Literal>]
    let ConnectionString = @"Data Source=(LocalDb)\V11.0;Initial Catalog=MailingListDb;Integrated Security=True"
    type DbSchema = SqlDataConnection<ConnectionString>

When writing the mapper from the domain type to the type that represents the table, we get strong types and autocompletion:

alt text

Here is the code for inserting a list of MailingListEntry and for deleting all rows from the table:

let insert cs list =
    let db = DbSchema.GetDataContext(cs)
    list
    |> List.map (fun e -> toDto e.Email e.Name)
    |> db.MailingList.InsertAllOnSubmit
    |> db.DataContext.SubmitChanges

let delete cs =
    let db = DbSchema.GetDataContext(cs)
    db.MailingList
    |> db.MailingList.DeleteAllOnSubmit
    |> db.DataContext.SubmitChanges

This code is also strongly typed and we get IntelliSense support:

alt text

What if the database schema changes...

If the database schema changes and our database related code is not updated, our program will not compile.

This is really great because we don't have to think or care about database and code getting out of sync anymore. The compiler will just point it out to us if there has been a schema change.

This way we can always have very high confidence in our data access code. The code becomes very reliable.

E.g. even if we only change the name of a column:

alt text

... we get a compilation error:

alt text

Reading from a CSV file

To read data from the CSV file we will use the CSV Type Provider.

For installation via NuGet we run the following command in the Package Manager Console:

Install-Package FSharp.Data -Version 2.2.5

This is the format of the CSV file:

email, name
john.doe@example.com, John Doe
jane.doe@example.com, Jane Doe

Here is the complete code for reading the email addresses and names from the file and again we get nice IDE support:

alt text

Reliability because of type inference

Again we can have very high confidence that our CSV data access code works because the TP obtains the type names from the header (first row) and infers the types from the values (subsequent rows).

If the structure looked like this e.g.:

email, name
3.14, John Doe
2.72, Jane Doe

... the type of Email would be inferred to be decimal and we would get the following error:

alt text

Parsing command line arguments

Now we have to parse the command line arguments to specify the correct command (Import or Delete) as well as the path to the CSV file.

Argu is not a real type provider, but it has some similarities. Mainly it makes parsing CLI arguments really easy and it transforms the provided arguments into a strongly typed list that we can pattern match against.

Installation:

Install-Package Argu

This is how we can define the arguments for our program:

module Arguments =
    open Argu

    type CliArguments =
        | Import of fileName:string
        | Delete
    with 
        interface IArgParserTemplate with
            member s.Usage = match s with Import _ -> "import csv data" | Delete -> "delete all entries"

With this setup the following command line inputs will be parsed correctly:

.\MaillingList.exe --import ..\..\mailinglist.csv

.\MaillingList.exe --delete

Now the function getCmds converts the input and returns a list of type CliArgument:

let getCmds args =
    let parser = ArgumentParser.Create<CliArguments>()
    let results = parser.Parse args
    results.GetAllResults()

Reading from the application's configuration file

The last type provider that we will use is FSharp.Configuration.

Installation:

Install-Package FSharp.Configuration

After we set it up with the name of the application's configuration file name again we get autocompletion when retrieving the connection string:

alt text

Again if the connection string is missing in the configuration file, the application won't compile.

Putting it all together

Now we can put everything together.

First we inject the connection string by partially applying the database access functions to the connection string:

type Settings = AppSettings<"app.config">
let private import list = DataAccess.insert Settings.ConnectionStrings.MyMailingListDb list
let private delete() = DataAccess.delete Settings.ConnectionStrings.MyMailingListDb

Then we write a function that handles a single command of type CliArgument:

let private handle cmd =
    match cmd with
    | Import fileName -> readFromCsvFile fileName |> import
    | Delete          -> delete()

Finally we need a function run that retrieves the list of commands and handles each of them:

let private run args =
    getCmds args
    |> List.iter handle

[<EntryPoint>]
let main argv = 
    run argv
    0

Then we can call run from the main method.

Here is the complete program in less than 80 lines of code:

type Email = Email of string

type Name = Name of string

type MailingListEntry = {
    Email:Email
    Name:Name }

module DataAccess =
    open Microsoft.FSharp.Data.TypeProviders

    [<Literal>]
    let ConnectionString = @"Data Source=(LocalDb)\V11.0;Initial Catalog=MailingListDb;Integrated Security=True"
    type DbSchema = SqlDataConnection<ConnectionString>

    let private toDto (Email email) (Name name) =
        DbSchema.ServiceTypes.MailingList(Email = email, Name = name)

    let insert cs list =
        let db = DbSchema.GetDataContext(cs)
        list
        |> List.map (fun e -> toDto e.Email e.Name)
        |> db.MailingList.InsertAllOnSubmit
        |> db.DataContext.SubmitChanges

    let delete cs =
        let db = DbSchema.GetDataContext(cs)
        db.MailingList
        |> db.MailingList.DeleteAllOnSubmit
        |> db.DataContext.SubmitChanges

module CsvAccess =
    open FSharp.Data

    type MailingListData = CsvProvider<"mailinglist.csv">

    let readFromCsvFile (fileName:string) = 
        let data = MailingListData.Load(fileName)
        [for row in data.Rows do
            yield { Email = Email row.Email; Name = Name row.Name}]

module Arguments =
    open Argu

    type CliArguments =
        | Import of fileName:string
        | Delete
    with 
        interface IArgParserTemplate with
            member s.Usage = match s with Import _ -> "import csv data" | Delete -> "delete all entries"

    let getCmds args =
        let parser = ArgumentParser.Create<CliArguments>()
        let results = parser.Parse args
        results.GetAllResults()

module Program =
    open FSharp.Configuration
    open Arguments
    open CsvAccess

    // inject connection srtring
    type Settings = AppSettings<"app.config">
    let private import list = DataAccess.insert Settings.ConnectionStrings.MyMailingListDb list
    let private delete() = DataAccess.delete Settings.ConnectionStrings.MyMailingListDb

    let private handle cmd =
        match cmd with
        | Import fileName -> readFromCsvFile fileName |> import
        | Delete          -> delete()

    let private run args =
        getCmds args
        |> List.iter handle

    [<EntryPoint>]
    let main argv = 
        run argv
        0

Conclusion

We have implemented a simple console application for managing the import of email addresses from a CSV file into a database.

We have used the following type providers

and the library Argu.

The implementation with the help of those libraries is really easy and efficient because we don't need to write a lot of boiler-plate code.

Also our code is very reliable because the external data is type checked at compile time.

Please note that the application will still crash at run-time if the external data sources do not match the expected structure. Adding run time error handling would be a topic for another article. But concerning reliability at compile time this is probably the best we can get.

The code is available on GitHub.

Leif Battermann

Functional Programming Fu

Identify Side Effects And Refactor Fearlessly

Why avoid side effects?

PureScript Case Study And Guide For Newcomers

Elm And The Algorithm Of Music

From a music data type to performance

Interactive Command Line Applications In Scala –Well Structured And Purely Functional

How To Use Applicatives For Validation In Scala And Save Much Work

Parsers in Scala built upon existing abstractions

Using existing abstractions

Strongly Typed Configuration Access With Code Generation

Error and state handling with monad transformers in Scala

Use lambdas and combinators to improve your API

Modelling API Responses With sbt-json –Print Current Bitcoin Price

Leif Battermann

Functional Programming Fu

Writing efficient and reliable code with F# Type Providers

Scenario

Implementation

Main types

Accessing the Database

What if the database schema changes...

Reading from a CSV file

Reliability because of type inference

Parsing command line arguments

Reading from the application's configuration file

Putting it all together

Conclusion