Identify Side Effects And Refactor Fearlessly

When we refactor code how can we be confident that we don't break anything?

3 of the most important things that allow us to refactor fearlessly are:

Side effect free - or pure - expressions
Statically typed expressions
Tests

In this article we will solely focus on the aspect of side effects and strictly speaking on how to identify them. Being able to identify side effects in our programs clearly is the precondition for eliminating them.

Why avoid side effects?

Continue reading →

Have you ever wanted to try out PureScript but were lacking a good way to get started?

If you

Have some prior functional programming knowledge - maybe you know Haskell,Elm,F#,or Scala,etc.
Want to solve a small task with PureScript
And want to get started quickly

This post is for you!

In this post we will walk through setting up and implementing a small exemplary PureScript application from scratch.

Continue reading →

In this article I would like to present a minimal implementation of a music data type and everything that is needed to turn that into audible sound from an Elm application.

We will see how to transcribe an existing composition - an excerpt from Chick Corea's Children's Songs No. 6 - and listen to the result right here,embedded in this article.

From a music data type to performance

My colleague Jonas recently pointed out the presentation Making Algorithmic Music by Donya Quick to me. Donya Quick shows how she uses the Haskell library Euterpea to produce algorithmic music.

It got me really excited about the idea of porting this to Elm and to be able to use this in web applications.

In the following we will see the core data types and algorithms from Euterpea ported to Elm. To focus on the core concepts the implementation is stripped down to the minimum that is required to transcribe and perform an existing polyphonic piece of music (for a single instrument).

Continue reading →

This post is about how to implement well structured,and purely functional command line applications in Scala using PureApp.

PureApp originated in an experiment while refactoring out some glue code of an interactive command line application. At the same time it was inspired by the Elm Architecture Pattern,and scalaz's SafeApp,as well as scalm.

To show the really cool things we can do with PureApp,we will implement a self-contained example application from scratch.

This application translates texts from and into different languages. And it provides basic user interactions via the command line.

The complete source code is compiled with tut. Every output (displayed as code comments) is generated by tut.
Continue reading →

In this post we will see how applicatives can be used for validation in Scala. It is an elegant approach. Especially when compared to an object-oriented way.

Usually when we have operations that can fail,we have them return types like Option or Try. We sequence operations and once there is an error the computation is short circuited and the result is a None or a Failure.

Applicatives allow us to compose independent operations and evaluate each one. Even if an intermediate evaluation fails. This allows us to collect error messages instead of returning only the first error that occurred.

A classic example where this is useful is the validation of user input. We would like to return a list of all invalid inputs rather than aborting the evaluation after the first error.

Scala Cats provides a type that does exactly that. So let's dive into some code and see how it works.

Continue reading →

After some initial struggles,the chapter Functional Parsers from the great book Programming in Haskell by Graham Hutton,where a basic parser library is built from scratch,significantly helped me to finally understand the core ideas of parser combinators and how to apply them to other programming languages other than Haskell as well.

While I recently revisited the material and started to port the examples to Scala I wasn't able to define a proper monad instance for the type Parser[A].

The type Parser[A] alias was defined like this:

type Parser[A] = String =>Option[(A,String)] // defined type alias Parser

To test the monad laws with discipline I had to provide an instance of Eq[Parser[A]]. Because Parser[A] is a function,equality could only be approximated by showing degrees of function equivalence,which is not a trivial task.

Also the implementation of tailRecM was challenging. (I couldn't figure it out.)

Using existing abstractions

Continue reading →

Most config libraries use a stringly typed approach.

Some handle runtime failures due to invalid configuration schemas by leveraging data types like Option or Result to represent missing values or errors. This allows us to handle these failures by either providing default values or by providing decent error messages.

This is a good strategy that we should definitely stick to.

However,the problem with default values is that we might not even notice if the configuration is broken. This could potentially fail in production. In any case an error e.g. due to a misspelled config property will be observable at runtime at the earliest.

Wouldn't it be a great user experience (for us developers) if the compiler told us if the configuration schema is invalid? Even better,imagine we could access the configuration data in a strongly typed way like any other data structure,and with autocompletion.

Moreover,what if we didn't have to write any glue code,not even when the configuration schema changes?

This can be done with the costs of an initial setup that won't take more than probably around 5 minutes.

Continue reading →

In this post I will look at a practical example where the combined application (through monad transformers) of the state monad and the either monad can be very useful.

I won't go into much theory,but instead demonstrate the problem and then slowly build it up to resolve it.

You don't have to be completely familiar with all the concepts as the examples will be easy to follow. Here is a very brief overview:

Continue reading →

If your API overflows with Boolean parameters,this is usually a bad smell.

Consider the following function call for example:

toContactInfoList(csv,true,true)

When looking at this snippet of code it is not very clear what kind of effect the two Boolean parameters will have exactly. In fact,we would probably be without a clue.

We have to inspect the documentation or at least the parameter names of the function declaration to get a better idea. But still,this doesn't solve all of our problems.

The more Boolean parameters there are,the easier it will be for the caller to mix them up. We have to be very careful.

Moreover,functions with Boolean parameters must have conditional logic like if or case statements inside. With a growing number of conditional statements,the number of possible execution paths will grow exponentially. It will become more difficult to reason about the implementation code.

Can we do better?

Sure we can. Lambdas and combinators come to the rescue and I'm going to show this with a simple example,a refactoring of the function from above.

This post is based on a great article by John A De Goes,Destroy All Ifs — A Perspective from Functional Programming.

I'm going to take John's ideas that he backed up with PureScript examples and present how the same thing can be elegantly achieved in Scala.

Continue reading →

I'm currently working on an sbt plugin that generates Scala case classes at compile time to model JSON API responses for easy deserialization especially with the Scala play-json library.

The plugin makes it possible to access JSON documents in a statically typed way including auto-completion. It takes a sample JSON document as input (either from a file or a URL) and generates Scala types that can be used to read data with the same structure.

Let's look at a basic example,an app that prints the current Bitcoin price to the console.

Continue reading →

When we refactor code how can we be confident that we don't break anything?

3 of the most important things that allow us to refactor fearlessly are:

Side effect free - or pure - expressions
Statically typed expressions
Tests

Why avoid side effects?

In the book "Functional Programming in Scala" Runar Bjarnason and Paul Chiusano state that a pure function "has no observable effect on the execution of the program other than to compute a result given its inputs".

This has significant benefits:

Reason about the behavior

We can safely replace any expression in our program with the value that it evaluates to - known as substitution model. We can interpret the code by substituting equals by equals as if the program was a mathematical expression. This is also called equational reasoning.

All relevant in- and outputs are declared in a function's type. This gives us an incredible advantage when we try to understand and reason about the code.

Testability

A side effect free program is deterministic. For the same input it will always return the same result.

This also helps reasoning as well as testability because:

It is much easier to verify that the output is correct because it is deterministic
And it is easier to write tests because all relevant inputs can be passed as arguments

Refactor without fear

We can always assign a sub-expression to a variable and replace all occurrences of the sub-expression with that variable. When we do this refactoring we can be absolutely sure that the behavior of the program remains unchanged.

How to identify side effects

The following sections will contain a compiled list of common examples of side effects that we encounter in every day programming.

The intention is to provide a compendium of typical examples of side effects that helps the reader to improve their ability to spot similar side effects in their own code.

There are multiple techniques that can be applied to eliminate side effects. A good first advice is to strictly separate side effecting code from pure code. However, this is beyond the scope of this post.

Instead will focus on identifying side effects.

Referential transparency

A function or an expression is either pure (has no side effects) or impure (has side effects). There is no in between.

If we can substitute any sub-expression of a program with the value it evaluates to without changing the behavior of the program the code is free of side effects.

This property is also known as referential transparency.

What is referential transparency, by Impure Pics used with kind permission

A test that reveals side effects

To proof that an expression is not referentially transparent - and therefore has side effects - we only need to show a single example where the substitution of an expression by its value changes the behavior of the program.

E.g. if the programs p1 and p2 in the following listing do not behave the same they are not referentially transparent.

val p1 = (<expr>, <expr>)

val p2 = {
  val a = <expr>
  (a, a)
}

Here is an example of a pure program, p1 and p2 produce the same result and there is no other observable effect:

val p1 = (1 + 2, 1 + 2)
// p1: (Int, Int) = (3, 3)

val p2 = {
  val a = 1 + 2
  (a, a)
}
// p2: (Int, Int) = (3, 3)

Compendium of common side effects

Let's now look at a list of typical examples of side effects. It is not exhaustive but it should contain the most common cases. If you find any other good examples I will be more than happy to include them here.

Every type of side effect is presented with a small proof according to the test from above.

Program output

`println`

val print1 = (println("hi"), println("hi"))
// hi
// hi
// print1: (Unit, Unit) = ((), ())

val print2 = {
  val a = println("hi")
  (a, a)
}
// hi
// print2: (Unit, Unit) = ((), ())

Program behavior:

print1 prints "hi" twice whereas print2 prints "hi" only once

Intuition:

Change something in the outside world
Often returns void or Unit

Similar:

Write to, change, or delete a file
Insert, update, delete in a database
Call external services (e.g. unsafe HTTP methods like POST, PUT or DELETE)
Send a message to a message broker or pub/sub system
etc. …

Elimination strategies:

Separate from pure code and only execute at the latest possible moment (at the end of the world)
Use a type for encoding side effects as pure values, e.g. IO
Use writer monad

Program input

`readLine`

val read1 = (scala.io.StdIn.readLine, scala.io.StdIn.readLine)
// p2a: (String, String) = ("foo", "bar")

val read2 = {
  val a = scala.io.StdIn.readLine
  (a, a)
}
// p2b: (String, String) = ("foo", "foo")

Program behavior:

read1 prompts for input twice whereas read2 prompts only once
Therefore the result of read2 will always be a tuple of the same values whereas the result of read1 may contain different values depending of the user input

Intuition:

Read a value from the outside world
Sometimes does not take parameters
Not deterministic

Similar:

Read from a file
Read from a database
Get a result from an external service (e.g. safe HTTP methods like GET,HEAD or OPTIONS)
Consume messages from a broker or pub/sub system
Read environment variables or system properties
etc. …

Elimination strategies:

Strictly separate from pure code
Use a type for encoding side effects as pure values, e.g. IO

Mutable State

Mutable variables

val mutable1 = {
  var counter = 0
  def inc() = {counter += 1; counter}
  (inc(), inc())
}
// mutable1: (Int, Int) = (1, 2)

val mutable2 = {
  var counter = 0
  def inc() = {counter += 1; counter}
  val a = inc()
  (a, a)
}
// mutable2: (Int, Int) = (1, 1)

Program behavior:

In this example the side effect is the mutation of the variable counter. Like in the previous examples the value is incremented twice in the first program and only once in the second one.

Intuition:

Watch out for the var key word

Similar:

Anything that is mutable

Elimination strategies:

Simply do not use mutable variables
And if you have to: Don't
And if you still have to: Contain the side effect locally within the smallest possible scope and make sure it does not leak
Use MVar or Ref
Use the state monad

Mutable objects

Similar to mutable variables.

class Thing() {
  private var position_ : Int = 0
  def move(distance: Int) = position_ += distance
  override def toString: String = s"Thing(position = ${this.position_})"
}

val obj1 = {
  val x = new Thing()
  ({x.move(1); x}, {x.move(1); x})
}
// obj1: (Thing, Thing) = (Thing(position = 2), Thing(position = 2))

val obj2 = {
  val x = new Thing()
  val a = {x.move(1); x}
  (a, a)
}
// obj2: (Thing, Thing) = (Thing(position = 1), Thing(position = 1))

Program behavior:

Again, similar to the previous examples the refactored version only performs the mutation once instead of twice

Similar:

Anything that is mutable

Elimination strategies:

Use object or case class instead of class
Instead of mutating the object return a new object

`Iterator`

val iter1 = {
  val xs = List(1,2,3).iterator

  (xs.next, xs.next)
}
// iter1: (Int, Int) = (1, 2)

val iter2 = {
  val xs = List(1,2,3).iterator

  val a = xs.next
  (a, a)
}
// iter2: (Int, Int) = (1, 1)

Program behavior:

Similar to the previous examples the refactored version only performs the mutation once instead of twice

Elimination strategies:

Don't use it
If you have to: Contain the side effect locally and make sure it doesn't leak

`ListBuffer`

import scala.collection.mutable.ListBuffer

val buffer1 = {
  val xs = ListBuffer(1, 2)
  (xs.append(3), xs.append(3))
}
// buffer1: (ListBuffer[Int], ListBuffer[Int]) = (
//   ListBuffer(1, 2, 3, 3),
//   ListBuffer(1, 2, 3, 3)
// )

val buffer2 = {
  val xs = ListBuffer(1, 2)
  val a = xs.append(3)
  (a, a)
}
// buffer2: (ListBuffer[Int], ListBuffer[Int]) = (
//   ListBuffer(1, 2, 3),
//   ListBuffer(1, 2, 3)
// )

Program behavior:

Similar to the previous examples the refactored version only performs the mutation once instead of twice

Similar:

StringBuilder

Elimination strategies:

Don't use it
If you have to: Contain the side effect locally and make sure it doesn't leak

Random

`scala.util.Random`

import scala.util._

val random1 = (Random.nextInt, Random.nextInt)
// random1: (Int, Int) = (-749222584, 1952393011)

val random2 = {
  val a = Random.nextInt
  (a, a)
}
// random2: (Int, Int) = (1459812504, 1459812504)

Program behavior:

The first program likely produces two different random values whereas the refactored version produces the same value twice

Intuition:

Does not take parameters
Not deterministic

Elimination strategies:

Use a purely functional number generator (optionally combined with the state monad)
Use a type for encoding side effects as pure values, e.g. IO

`java.util.UUID.randomUUID`

import java.util.UUID

val randomUuid1 = (UUID.randomUUID, UUID.randomUUID)
// randomUuid1: (UUID, UUID) = (
//   fa1f4167-ed59-45ad-92b8-f4dfca2fbee9,
//   3dfa47c2-78f4-406e-9c61-364ca60d646a
// )

val randomUuid2 = {
  val a = UUID.randomUUID
  (a, a)
}
// randomUuid2: (UUID, UUID) = (
//   006f53a4-b73f-4237-9e18-6e2e1ceb7291,
//   006f53a4-b73f-4237-9e18-6e2e1ceb7291
// )

Program behavior:

The first program produces two different UUIDs whereas the refactored version produces only one

Intuition:

Does not take parameters
Not deterministic

Elimination strategies:

Create a UUID from a seed (be aware that obtaining a useful seed can also be a side effect)
Use a type for encoding side effects as pure values, e.g. IO

Get the current time

`System.currentTimeMillis`

def f(): String = { Thread.sleep(1); "done" }

var time1 = {
  val start = System.currentTimeMillis
  f()
  val end = System.currentTimeMillis
  (start, end)
}  
// time1: (Long, Long) = (1580764607046L, 1580764607047L)  

var time2 = {
  val a = System.currentTimeMillis
  val start = a
  f()
  val end = a
  (start, end)
}
// time2: (Long, Long) = (1580764607049L, 1580764607049L)

Program behavior:

The first program produces different start and end times whereas the refactored version produces the same start and end time

Intuition:

Does not take parameters
Not deterministic

Similar:

Instant.now
ZonedDateTime.now
LocalDateTime.now
…

Elimination strategies:

Sometimes you can get away with passing the current time as a parameter after obtaining it at the entry point of the program (which follows the strategy of separating pure and impure code)
If that does not work: Use a type for encoding side effects as pure values, e.g. IO

Exceptions

val except1 = Try {
  try {
    throw new Exception
  } catch {
    case _: Throwable => 42
  }
}
// except1: Try[Int] = Success(42)

val except2 = Try {
  val a: Int = throw new Exception
  try {
    a
  } catch {
    case _: Throwable => 42
  }
}
// except2: Try[Int] = Failure(java.lang.Exception)

Program behavior:

The two programs produce different results

Elimination strategies:

Do not catch exceptions!
If you have to: Don't
If you still have to:
- Never use exceptions for control flow!
- And catch them either right where they might occur or at the outer most level of your program
Do not throw exceptions
If you throw exceptions: Do not catch them. Ever. Period.
Encode errors as pure values using Either

The end?

Well, I hope this is not the end. I would like to update and improve this list of common examples of side effects in the future.

So if you encounter any interesting situation related to side effects that is not at least briefly covered here, please send me some code.

Also, if you have any further questions about the examples from above or if you have a piece of code where you are not sure if it is pure or not, do not hesitate to contact me or leave a comment.

Thanks for reading!

The code was compiled with mdoc.