How To Use Applicatives For Validation In Scala And Save Work

When we refactor code how can we be confident that we don't break anything?

3 of the most important things that allow us to refactor fearlessly are:

Side effect free - or pure - expressions
Statically typed expressions
Tests

In this article we will solely focus on the aspect of side effects and strictly speaking on how to identify them. Being able to identify side effects in our programs clearly is the precondition for eliminating them.

Why avoid side effects?

Continue reading →

Have you ever wanted to try out PureScript but were lacking a good way to get started?

If you

Have some prior functional programming knowledge - maybe you know Haskell,Elm,F#,or Scala,etc.
Want to solve a small task with PureScript
And want to get started quickly

This post is for you!

In this post we will walk through setting up and implementing a small exemplary PureScript application from scratch.

Continue reading →

In this article I would like to present a minimal implementation of a music data type and everything that is needed to turn that into audible sound from an Elm application.

We will see how to transcribe an existing composition - an excerpt from Chick Corea's Children's Songs No. 6 - and listen to the result right here,embedded in this article.

From a music data type to performance

My colleague Jonas recently pointed out the presentation Making Algorithmic Music by Donya Quick to me. Donya Quick shows how she uses the Haskell library Euterpea to produce algorithmic music.

It got me really excited about the idea of porting this to Elm and to be able to use this in web applications.

In the following we will see the core data types and algorithms from Euterpea ported to Elm. To focus on the core concepts the implementation is stripped down to the minimum that is required to transcribe and perform an existing polyphonic piece of music (for a single instrument).

Continue reading →

This post is about how to implement well structured,and purely functional command line applications in Scala using PureApp.

PureApp originated in an experiment while refactoring out some glue code of an interactive command line application. At the same time it was inspired by the Elm Architecture Pattern,and scalaz's SafeApp,as well as scalm.

To show the really cool things we can do with PureApp,we will implement a self-contained example application from scratch.

This application translates texts from and into different languages. And it provides basic user interactions via the command line.

The complete source code is compiled with tut. Every output (displayed as code comments) is generated by tut.
Continue reading →

In this post we will see how applicatives can be used for validation in Scala. It is an elegant approach. Especially when compared to an object-oriented way.

Usually when we have operations that can fail,we have them return types like Option or Try. We sequence operations and once there is an error the computation is short circuited and the result is a None or a Failure.

Applicatives allow us to compose independent operations and evaluate each one. Even if an intermediate evaluation fails. This allows us to collect error messages instead of returning only the first error that occurred.

A classic example where this is useful is the validation of user input. We would like to return a list of all invalid inputs rather than aborting the evaluation after the first error.

Scala Cats provides a type that does exactly that. So let's dive into some code and see how it works.

Continue reading →

After some initial struggles,the chapter Functional Parsers from the great book Programming in Haskell by Graham Hutton,where a basic parser library is built from scratch,significantly helped me to finally understand the core ideas of parser combinators and how to apply them to other programming languages other than Haskell as well.

While I recently revisited the material and started to port the examples to Scala I wasn't able to define a proper monad instance for the type Parser[A].

The type Parser[A] alias was defined like this:

type Parser[A] = String =>Option[(A,String)] // defined type alias Parser

To test the monad laws with discipline I had to provide an instance of Eq[Parser[A]]. Because Parser[A] is a function,equality could only be approximated by showing degrees of function equivalence,which is not a trivial task.

Also the implementation of tailRecM was challenging. (I couldn't figure it out.)

Using existing abstractions

Continue reading →

Most config libraries use a stringly typed approach.

Some handle runtime failures due to invalid configuration schemas by leveraging data types like Option or Result to represent missing values or errors. This allows us to handle these failures by either providing default values or by providing decent error messages.

This is a good strategy that we should definitely stick to.

However,the problem with default values is that we might not even notice if the configuration is broken. This could potentially fail in production. In any case an error e.g. due to a misspelled config property will be observable at runtime at the earliest.

Wouldn't it be a great user experience (for us developers) if the compiler told us if the configuration schema is invalid? Even better,imagine we could access the configuration data in a strongly typed way like any other data structure,and with autocompletion.

Moreover,what if we didn't have to write any glue code,not even when the configuration schema changes?

This can be done with the costs of an initial setup that won't take more than probably around 5 minutes.

Continue reading →

In this post I will look at a practical example where the combined application (through monad transformers) of the state monad and the either monad can be very useful.

I won't go into much theory,but instead demonstrate the problem and then slowly build it up to resolve it.

You don't have to be completely familiar with all the concepts as the examples will be easy to follow. Here is a very brief overview:

Continue reading →

If your API overflows with Boolean parameters,this is usually a bad smell.

Consider the following function call for example:

toContactInfoList(csv,true,true)

When looking at this snippet of code it is not very clear what kind of effect the two Boolean parameters will have exactly. In fact,we would probably be without a clue.

We have to inspect the documentation or at least the parameter names of the function declaration to get a better idea. But still,this doesn't solve all of our problems.

The more Boolean parameters there are,the easier it will be for the caller to mix them up. We have to be very careful.

Moreover,functions with Boolean parameters must have conditional logic like if or case statements inside. With a growing number of conditional statements,the number of possible execution paths will grow exponentially. It will become more difficult to reason about the implementation code.

Can we do better?

Sure we can. Lambdas and combinators come to the rescue and I'm going to show this with a simple example,a refactoring of the function from above.

This post is based on a great article by John A De Goes,Destroy All Ifs — A Perspective from Functional Programming.

I'm going to take John's ideas that he backed up with PureScript examples and present how the same thing can be elegantly achieved in Scala.

Continue reading →

I'm currently working on an sbt plugin that generates Scala case classes at compile time to model JSON API responses for easy deserialization especially with the Scala play-json library.

The plugin makes it possible to access JSON documents in a statically typed way including auto-completion. It takes a sample JSON document as input (either from a file or a URL) and generates Scala types that can be used to read data with the same structure.

Let's look at a basic example,an app that prints the current Bitcoin price to the console.

Continue reading →

In this post we will see how applicatives can be used for validation in Scala. It is an elegant approach. Especially when compared to an object-oriented way.

Usually when we have operations that can fail, we have them return types like Option or Try. We sequence operations and once there is an error the computation is short circuited and the result is a None or a Failure.

A classic example where this is useful is the validation of user input. We would like to return a list of all invalid inputs rather than aborting the evaluation after the first error.

Scala Cats provides a type that does exactly that. So let's dive into some code and see how it works.

The user type

The complete code of this article is compiled with tut to make sure it is complete and works.

We use Cats version 1.0.1 and the sbt plugin kind projector version 0.9.4.

We are going to need some imports:

import cats._
// import cats._

import cats.data._
// import cats.data._

import cats.implicits._
// import cats.implicits._

Now we define the type User:

final case class User(
  name: String,
  age: Int,
  email: String
)
// defined class User

An abstract user validator

Next we will define an abstract user validator.

We will only say that the validate function returns a user, wrapped into the abstract type F.

This has some benefits. Since it is often an important design choice which wrapper type to choose for F we defer this decision as long as possible.

Additionally it allows us to use different interpreters in different contexts, say one for testing and one for production.

trait UserValidator[F[_]] {
  def createValidUser(name: String, age: Int, email: String): F[User]
}

object UserValidator {
  def apply[F[_]](implicit ev: UserValidator[F]): UserValidator[F] = ev

  def validate[F[_]: UserValidator, E](name: String,
                                       age: Int,
                                       email: String): F[User] =
    UserValidator[F].createValidUser(name, age, email)
}

An `Id` interpreter

At this point we can already implement a simple Id interpreter that does no validation at all. It just constructs a user instance and returns it.

val userValidatorIdInterpreter = new UserValidator[Id] {
  def createValidUser(name: String, age: Int, email: String): Id[User] =
    User(name, age, email)
}
// userValidatorIdInterpreter: UserValidator[cats.Id] = $anon$1@49cfb62b

If we want to call the validate function, we have to have an implicit instance of UserValidator[F] in the scope.

implicit val userValidatorInterpreter = userValidatorIdInterpreter
// userValidatorInterpreter: UserValidator[cats.Id] = $anon$1@49cfb62b

println(UserValidator.validate("John", 25, "john@example.com"))
// User(John,25,john@example.com)

println(UserValidator.validate("John", 25, "johnn@example"))
// User(John,25,johnn@example)

println(UserValidator.validate("John", -1, "john@gexample"))
// User(John,-1,john@gexample)

println(UserValidator.validate(".John", -1, "john@gexample"))
// User(.John,-1,john@gexample)

Ok, let's now look at how applicatives can help us with real validation and error handling.

How applicatives work

Applicative is a type class that can be mapped over. And is has two operations called pure and ap.

pure is just a constructor that lifts values into an Applicative.

ap or the infix version <*> applies a value to a function, where both the value and the function are wrapped inside the same Applicative type.

So if F is the type of our Applicative we can apply F[A] to an F[A => B] to get an F[B]. It works just like normal function application, only that everything happens inside the F.

Note that the type B in F[A => B] can also be a function. Therefore we can continue to apply values to our F until we are left with a final result.

Here is an example of how to apply values of type A, B, and C to the function A => B => C => D to get a D at the end:

F[A => (B => (C => D))] <*> F[A] = F[B => (C => D)]

F[B => (C => D)]        <*> F[B] = F[C => D]

F[C => D]               <*> F[C] = F[D]

The parentheses are for clarification and can be omitted. Note that the function inside the F has to be curried.

We can also write this in one line:

F[A => B => C => D] <*> F[A] <*> F[B] <*> F[C] = F[D]

With these semantics it is possible to implement an instance for F that encapsulates conditional logic that we can use for error handling and validation.

Fortunately we do not have to implement such instances ourselves. They are already provided by Scala Cats.

In fact every monad instance such as Option or List e.g. has also an Applicative instance.

Here is an example with Option:

def add(a: Int, b: Int) = a + b
// add: (a: Int, b: Int)Int

((add _).curried).pure[Option] <*> Option(2) <*> Option(5)
// res5: Option[Int] = Some(7)

The validation logic

Because we still do not want to commit to an applicative instance too early we are going to implement the validation logic in an abstract way.

Regardless of what type we are going to use for validation later we only want to implement the validation logic once.

To do that, however, we have to have at least one constraint for our abstract type F: It has to have an instance of ApplicativeError. Which is a special kind of applicative. It has an additional error type and additional operations. Of which we are only going to need raiseError.

The error type

First we create a custom error type for the user validation:

sealed trait UserValidationError
// defined trait UserValidationError

case object NameNotValid extends UserValidationError
// defined object NameNotValid

case object AgeOutOfRange extends UserValidationError
// defined object AgeOutOfRange

case object EmailNotValid extends UserValidationError
// defined object EmailNotValid

The error constructor

The applicative instances from Cats have different error types. Some have Throwable or Unit. Others have generic error types. And yet others have generic error types with constraints.

Therefore we have to tell our generic validator how to construct an abstract error type from the concrete type that we just defined. We do this by passing a function UserValidationError => E to the UserValidator constructor.

The validation functions

Let's implement functions to validate name, age, and email address. Each function returns either a valid value or an error wrapped into the abstract type F.

Here is how we could validate the email address e.g.:

def validateEmail(email: String): F[String] =
  if (email.matches("^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"))
    email.pure[F]
  else A.raiseError(mkError(EmailNotValid))

Combining validations

Finally we will combine the validated name, age, and email address with applicative composition.

After lifting the User.apply function into the Applicative we can just apply the validated values:

def createValidUser(name: String, age: Int, email: String): F[User] = {
  (User.apply _).curried.pure[F] <*> validateName(name) <*>
    validateAge(age) <*> validateEmail(email)
}

Here is the complete generic user validator:

def userValidator[F[_], E](mkError: UserValidationError => E)(
    implicit A: ApplicativeError[F, E]): UserValidator[F] =
  new UserValidator[F] {

    def validateName(name: String): F[String] =
      if (name.matches("(?i:^[a-z][a-z ,.'-]*$)")) name.pure[F]
      else A.raiseError(mkError(NameNotValid))

    def validateAge(age: Int): F[Int] =
      if (age >= 18 && age < 120) age.pure[F]
      else A.raiseError(mkError(AgeOutOfRange))

    def validateEmail(email: String): F[String] =
      if (email.matches("^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"))
        email.pure[F]
      else A.raiseError(mkError(EmailNotValid))

    def createValidUser(name: String, age: Int, email: String): F[User] = {
      (User.apply _).curried.pure[F] <*> validateName(name) <*>
        validateAge(age) <*> validateEmail(email)
    }
  }
// userValidator: [F[_], E](mkError: UserValidationError => E)(implicit A: cats.ApplicativeError[F,E])UserValidator[F]

Usage

Now it's quite easy to create concrete UserValidator instances. We have to provide the target type, the error type and the error constructor function.

The concrete `UserValidator` instances

Here are some examples for Option, Try and Either.

(To make the code more concise we use the kind projector plugin.)

import scala.util.Try
// import scala.util.Try

val userValidatorOptionInterpreter =
  userValidator[Option, Unit](_ => ())
// userValidatorOptionInterpreter: UserValidator[Option] = $anon$1@7e284fb6

val userValidatorTryInterpreter =
  userValidator[Try, Throwable](err => new Throwable(err.toString))
// userValidatorTryInterpreter: UserValidator[scala.util.Try] = $anon$1@42f4b47b

val userValidatorEitherInterpreter =
  userValidator[Either[UserValidationError, ?], UserValidationError](identity)
// userValidatorEitherInterpreter: UserValidator[[β$0$]scala.util.Either[UserValidationError,β$0$]] = $anon$1@3ca91abe

Doing some validation

Let's bring an implicit validator into scope and try different inputs:

implicit val userValidatorInterpreter = userValidatorTryInterpreter
// userValidatorInterpreter: UserValidator[scala.util.Try] = $anon$1@42f4b47b

println(UserValidator.validate("John", 25, "john@example.com"))
// Success(User(John,25,john@example.com))

println(UserValidator.validate("John", 25, "johnn@example"))
// Failure(java.lang.Throwable: EmailNotValid)

println(UserValidator.validate("John", -1, "john@gexample"))
// Failure(java.lang.Throwable: AgeOutOfRange)

println(UserValidator.validate(".John", -1, "john@gexample"))
// Failure(java.lang.Throwable: NameNotValid)

As we can see from the results, it works. Only error messages are not collected.

Collecting error messages

If we want to collect error messages we can use the type Validated from Cats.

val userValidatorValidatedInterpreter =
  userValidator[Validated[NonEmptyList[UserValidationError], ?],
                NonEmptyList[UserValidationError]](NonEmptyList(_, Nil))
// userValidatorValidatedInterpreter: UserValidator[[β$0$]cats.data.Validated[cats.data.NonEmptyList[UserValidationError],β$0$]] = $anon$1@36f4fb60

implicit val userValidatorInterpreter = userValidatorValidatedInterpreter
// userValidatorInterpreter: UserValidator[[β$0$]cats.data.Validated[cats.data.NonEmptyList[UserValidationError],β$0$]] = $anon$1@36f4fb60

println(UserValidator.validate("John", 25, "john@example.com"))
// Valid(User(John,25,john@example.com))

println(UserValidator.validate("John", 25, "johnn@example"))
// Invalid(NonEmptyList(EmailNotValid))

println(UserValidator.validate("John", -1, "john@gexample"))
// Invalid(NonEmptyList(AgeOutOfRange, EmailNotValid))

println(UserValidator.validate(".John", -1, "john@gexample"))
// Invalid(NonEmptyList(NameNotValid, AgeOutOfRange, EmailNotValid))

The only constraint here is that the error type has to have a Semigroup instance. This holds e.g. for List or NonEmptyList. So as long as this holds (and the compiler will tell us) we don't have to worry about this detail.

Conclusion

Let's recap what we did:

We created a type class UserValidator that defined the algebra for creating valid User instances
This allowed us to implement a "fake" Id interpreter
Then we implemented the validation logic in a generic way. With the only constraint that the target type had to be an ApplicativeError
Then we were able to define UserValidator interpreters for Option, Try, Either or Validated. With almost no overhead. And Without duplicating any validation logic.
We did not commit to any target type
When changing the target type later, we can leave the application code untouched. We merely have to replace the implicit interpreter.

I think this is really cool. Also note that the code is purely functional. And we didn't have to implement any low level logic for collecting messages. This is handled by the Validated type and just works.

Leif Battermann

Functional Programming Fu

Identify Side Effects And Refactor Fearlessly

Why avoid side effects?

PureScript Case Study And Guide For Newcomers

Elm And The Algorithm Of Music

From a music data type to performance

Interactive Command Line Applications In Scala –Well Structured And Purely Functional

How To Use Applicatives For Validation In Scala And Save Much Work

Parsers in Scala built upon existing abstractions

Using existing abstractions

Strongly Typed Configuration Access With Code Generation

Error and state handling with monad transformers in Scala

Use lambdas and combinators to improve your API

Modelling API Responses With sbt-json –Print Current Bitcoin Price

Leif Battermann

Functional Programming Fu

How To Use Applicatives For Validation In Scala And Save Much Work

The user type

An abstract user validator

An `Id` interpreter

How applicatives work

The validation logic

The error type

The error constructor

The validation functions

Combining validations

Usage

The concrete `UserValidator` instances

Doing some validation

Collecting error messages

Conclusion

Why avoid side effects?

From a music data type to performance

Using existing abstractions

The user type

An abstract user validator

An Id interpreter

How applicatives work

The validation logic

The error type

The error constructor

The validation functions

Combining validations

Usage

The concrete UserValidator instances

Doing some validation

Collecting error messages

Conclusion

An `Id` interpreter

The concrete `UserValidator` instances