PureScript Case Study And Guide For Newcomers

From a music data type to performance

My colleague Jonas recently pointed out the presentation Making Algorithmic Music by Donya Quick to me. Donya Quick shows how she uses the Haskell library Euterpea to produce algorithmic music.

It got me really excited about the idea of porting this to Elm and to be able to use this in web applications.

In the following we will see the core data types and algorithms from Euterpea ported to Elm. To focus on the core concepts the implementation is stripped down to the minimum that is required to transcribe and perform an existing polyphonic piece of music (for a single instrument).

If your API overflows with Boolean parameters,this is usually a bad smell.

Consider the following function call for example:

toContactInfoList(csv,true,true)

When looking at this snippet of code it is not very clear what kind of effect the two Boolean parameters will have exactly. In fact,we would probably be without a clue.

We have to inspect the documentation or at least the parameter names of the function declaration to get a better idea. But still,this doesn't solve all of our problems.

The more Boolean parameters there are,the easier it will be for the caller to mix them up. We have to be very careful.

Moreover,functions with Boolean parameters must have conditional logic like if or case statements inside. With a growing number of conditional statements,the number of possible execution paths will grow exponentially. It will become more difficult to reason about the implementation code.

Can we do better?

Sure we can. Lambdas and combinators come to the rescue and I'm going to show this with a simple example,a refactoring of the function from above.

This post is based on a great article by John A De Goes,Destroy All Ifs — A Perspective from Functional Programming.

I'm going to take John's ideas that he backed up with PureScript examples and present how the same thing can be elegantly achieved in Scala.

Continue reading →

Have you ever wanted to try out PureScript but were lacking a good way to get started?

If you

Have some prior functional programming knowledge - maybe you know Haskell, Elm, F#, or Scala, etc.
Want to solve a small task with PureScript
And want to get started quickly

This post is for you!

In this post we will walk through setting up and implementing a small exemplary PureScript application from scratch.

Prerequisites

We assume that we want to run our application from a Node.js environment, maybe an AWS Lambda or some other existing Node.js application.

Node.js should be installed

Project setup

Let's start a new project which we call index-ttl with:

mkdir index-ttl && cd index-ttl && npm init

When prompted for test command enter:

npx spago test

(you can use the defaults for everything else)

Now we will install PureScript, spago a PureScript build tool and purty for code formatting:

npm install --save-dev purescript spago purty

(Note: If the above command fails depending on OS you might have to install the Debian package: libncurses5.)

To initialize a new spago project run

npx spago init

Let's see if our freshly initialized project works by running:

npm test

The output should look similar to this:

root@f883950b5408:/index-ttl# npm test

> index-ttl@1.0.0 test /index-ttl
> npx spago test

[info] Installation complete.
[info] Build succeeded.

You should add some tests.
[info] Tests succeeded.

JavaScript interop

The spago init command has generated some files. The file src/Main.purs contains the main function, the application's entry point.

To call this function from JavaScript we need to bundle the project first.

Given a file index.js in the root folder of the project with the following content:

const Main = require('./Main')

Main.main()

We can bundle and run PureScript project with the command:

npx spago bundle-module --to Main.js && node index.js

Optionally we can add the above command to the scripts property in the package.json file:

"scripts": {
  "test": "npx spago test",
  "start": "npx spago bundle-module --to Main.js && node index.js"
},

Then run the application with npm start.

Code Editor

If you do not have any other preferences I suggest using VS Code with the following extensions:

PureScript IDE
- AFAIK you have to install PureScript globally with npm install -g purescript for this extension to work properly
vscode-purty
- Specify npx purty as the path to the purty executable in the settings. Alternatively you can install purty globally with npm install -g purty

Ready to go

No we have everything in place to start hacking. Maybe you are good on your own now and do not need to read further?

If this is the case my last advice is to refer you to Pursuit, a documentation platform and search engine for PureScript packages (equivalent to Hoogle for Haskell).

In the following sections we will walk through an exemplary implementation of a PureScript application and see e.g. how to

Write tests
Do dependency injection
Handle I/O
Make HTTP requests
And parse JSON

Implementation

We are going to build an application that removes expired indices from Elasticsearch.

This is an ideal task for an exemplary PureScript application because

It involves I/O - in this case calls to the Elasticsearch REST API
It involves some processing logic which is not too complex
And it is a self-contained, real-world example

Use case

Depending on the kind and load of data that is stored in Elasticsearch it often makes sense to partition the data by time intervals based on index names tagged with a timestamp, e.g. the date. Consider the following indices:

index_2020_01_18
index_2020_01_19
index_2020_01_20
index_2020_01_21
index_2020_01_22
index_2020_01_23

Assume that we establish a 3 day time to live (TTL) and the current date is Jan 23, 2020. This means that all indices dated before Jan 20, 2020 should be removed, namely:

index_2020_01_18
index_2020_01_19

Given an Elasticsearch hostname and port, the current date, and a TTL in days our app should delete expired indices as described above. The signature of a function could look like this:

deleteExpiredIndices :: Hostname -> Port -> Today -> Ttl -> Aff Unit

But before we get to the implementation of this function we first need to install some additional packages, define some types and write some tests.

Packages

So far the setup has been pretty generic. But now we will install a few npm and PureScript packages specifically for this tutorial:

npm install xhr2
npx spago install aff datetime spec partial refs affjax argonaut-codecs aff-promise formatters

Note that xhr2 is needed for affjax (an AJAX library) if the app is not running in the browser.

spec is needed for testing.

Imports

In the file src/Main.purs we need the following imports:

import Prelude
import Affjax (printError)
import Affjax as AX
import Affjax.ResponseFormat as ResponseFormat
import Control.Promise (fromAff, Promise)
import Data.Argonaut.Core (Json)
import Data.Argonaut.Decode (decodeJson)
import Data.Array (filter, mapMaybe, takeEnd)
import Data.Bifunctor (lmap)
import Data.Date (diff)
import Data.DateTime (Date, canonicalDate, date)
import Data.Either (Either(..), either, note)
import Data.Enum (class BoundedEnum, toEnum)
import Data.Formatter.DateTime (unformatDateTime)
import Data.HTTP.Method (Method(..))
import Data.Int (toNumber, fromString)
import Data.Maybe (Maybe(..))
import Data.String (split, Pattern(..))
import Data.Time.Duration (Days(..))
import Data.Traversable (traverse, sequence)
import Effect (Effect)
import Effect.Aff (Aff)
import Effect.Class.Console (log)

Types, types, types

These types we have already used/seen above:

newtype Hostname
  = Hostname String

newtype Port
  = Port Int

newtype Ttl
  = Ttl Days

newtype Today
  = Today Date

Additionally we need a type that represents an index with a Show instance as well as a representation for an index with a date:

newtype Index
  = Index String

instance showIndex :: Show Index where
  show (Index index) = index

type DatedIndex
  = { name :: Index, date :: Date }

If we wanted to enforce even more type safety we could have put the types into their own modules. By only exposing smart constructors that parse the given arguments we would ensure only valid instances. But this is a bit too much ceremony in this particular situation for my taste. However, the larger the application gets, the more I would tend towards such an approach.

To be able to unit test our application we will pass in the dependencies - in this case the Elasticsearch API calls - as a record of functions:

type ElasticsearchClient
  = { indices :: Aff (Array Index)
    , deleteIndex :: Index -> Aff Unit
    }

Aff represents an asynchronous effect similar to Haskell's IO or Scala's cats.effect.IO or monix.eval.Task.

Now we can change the function signature slightly and also provide a dummy implementation to make it compile:

deleteExpiredIndices :: ElasticsearchClient -> Today -> Ttl -> Aff Unit
deleteExpiredIndices client today ttl = pure unit

Testing

For testing we will use purescript-spec.

We will need the following imports in the file test/Main.purs:

import Prelude
import Data.Array ((:))
import Data.Date.Component (Month(..))
import Data.DateTime (canonicalDate, Date)
import Data.Enum (toEnum)
import Data.Maybe (fromJust)
import Effect (Effect)
import Effect.Aff (Aff, launchAff_)
import Effect.Class (liftEffect)
import Effect.Ref as Ref
import Main (Index(..), Today(..), Ttl(..), deleteExpiredIndices)
import Partial.Unsafe (unsafePartial)
import Test.Spec (describe, it)
import Test.Spec.Assertions (shouldEqual)
import Test.Spec.Reporter.Console (consoleReporter)
import Test.Spec.Runner (runSpec)
import Data.Time.Duration (Days(..))
import Data.Int (toNumber)

Here is a single-example based test for our use case:

main :: Effect Unit
main =
  launchAff_
    $ runSpec [ consoleReporter ] do
        describe "Main" do
          describe "deleteExpiredIndices" do
            it "should delete expired indices" do
              result <- liftEffect $ Ref.new []
              let
                expected =
                  (map show)
                    [ Index "foo_2019_11_15"
                    , Index "foo_2020_01_06"
                    ]
              deleteExpiredIndices
                (client result)
                (Today $ date 2020 January 10)
                (Ttl $ Days $ toNumber 3)
              actual <- liftEffect $ (map show) <$> Ref.read result
              actual shouldEqual expected
  where
  client result =
    { indices:
      pure
        [ Index "foo_2020_01_11"
        , Index "foo_2020_01_10"
        , Index "foo_2020_01_09"
        , Index "foo_2020_01_08"
        , Index "foo_2020_01_07"
        , Index "foo_2020_01_06"
        , Index "foo_2019_11_15"
        , Index "index_without_date"
        ]
    , deleteIndex: \index -> liftEffect $ Ref.modify_ ((:) index) result :: Aff Unit
    }

  date :: Int -> Month -> Int -> Date
  date year month day = unsafePartial fromJust $ (\y d -> canonicalDate y month d) <$> toEnum year <*> toEnum day

Note that we created a test stub for the ElasticsearchClient that returns a hard coded list of indices. And it stores a list of deleted indices in a mutable reference of type Ref (Array Index) which we can use later to make the test assertion.

If we run the test with npm test or npx spago test it fails as expected with the following output:

Main » deleteExpiredIndices
  ✗ should delete expired indices:

  [] ≠ ["foo_2019_11_15","foo_2020_01_06"]

Summary
0/1 test passed

[error] Tests failed: exit code: 1

Let's make the test succeed!

This implementation should suffice:

findExpired :: Today -> Ttl -> Array DatedIndex -> Array DatedIndex
findExpired (Today today) (Ttl ttl) = filter (\indexDate -> (diff today indexDate.date) > ttl)

determineDate :: Index -> Maybe DatedIndex
determineDate (Index name) = { name: Index name, date: _ } <$> dateOrError
  where
  parsed = name # split (Pattern "_") # takeEnd 3

  dateOrError = case parsed of
    [ year, month, day ] ->
      canonicalDate
        <$> strToEnum year
        <*> strToEnum month
        <*> strToEnum day
    _ -> Nothing

  strToEnum ∷ forall e. BoundedEnum e => String -> Maybe e
  strToEnum = fromString >=> toEnum

deleteExpiredIndices :: ElasticsearchClient -> Today -> Ttl -> Aff Unit
deleteExpiredIndices client today ttl = do
  indices <- client.indices
  let
    datedIndices = indices # mapMaybe determineDate

    expired = findExpired today ttl datedIndices
  void $ client.deleteIndex traverse (expired # map _.name)

The function deleteExpiredIndices

Calls the indices function from the Elasticsearch client
Determines the dates by parsing the index names by calling determineDate
Determines which indices are expired by calling findExpired
And for each expired index calls deleteIndex from the Elasticsearch client

What is left?

Yes, we need to provide a real Elasticsearch client and integrate everything with our index.js.

If we query indices in JSON format we get result similar to this:

curl 'localhost:9200/_cat/indices?format=json&pretty'

[
  {
    "health" : "yellow",
    "status" : "open",
    "index" : "index_2020_01_22",
    "uuid" : "-gvBWXdPTjudCyVN-oxRKw",
    "pri" : "1",
    "rep" : "1",
    "docs.count" : "0",
    "docs.deleted" : "0",
    "store.size" : "230b",
    "pri.store.size" : "230b"
  },
  {
    "health" : "yellow",
    "status" : "open",
    "index" : "index_2020_01_23",
    "uuid" : "qLWKjqAQR5eubp5RWIn8HA",
    "pri" : "1",
    "rep" : "1",
    "docs.count" : "0",
    "docs.deleted" : "0",
    "store.size" : "230b",
    "pri.store.size" : "230b"
  }
]

The only relevant field of the JSON output is index. So let's create a type for that:

type ElasticsearchIndex
  = { index :: String }

A decoder for this can simply be derived with the help of the purescript-argonaut-codecs library by calling decodeJson.

The HTTP calls to the Elasticsearch API is done with affjax:

elasticsearchClient :: Hostname -> Port -> ElasticsearchClient
elasticsearchClient (Hostname hostname) (Port port) =
  { indices:
    indices
      <#> (map _.body)
      <#> (lmap printError)
      <#> (_ >>= indexFromJson)
      >>= either (\err -> log err $> []) pure
      <#> (map (_.index >>> Index))
  , deleteIndex:
    \(Index index) ->
      delete index >>= either (printError >>> log) (const (log $ "Deleted index: " <> index))
  }
  where
  url = "http://" <> hostname <> ":" <> show port

  indexFromJson :: Json -> Either String (Array ElasticsearchIndex)
  indexFromJson = decodeJson

  indices =
    AX.request
      ( AX.defaultRequest
          { url = url <> "/_cat/indices?format=json"
          , method = Left GET
          , responseFormat = ResponseFormat.json
          }
      )

  delete index =
    AX.request
      ( AX.defaultRequest
          { url = url <> "/" <> index
          , method = Left DELETE
          , responseFormat = ResponseFormat.ignore
          }
      )

In the main function we parse all inputs and call deleteExpiredIndices. The result is transformed to a Promise to provide JavaScript interop.

main :: String -> String -> String -> String -> Effect (Promise Unit)
main datetime ttl hostname port = do
  fromAff $ deleteExpiredIndices
    <$> maybeClient
    <*> dateOrError
    <*> ttlOrError
    # sequence
    >>= either log pure
  where
  dateOrError = unformatDateTime "YYYY-MM-DDTHH:mm:ssZ" datetime # map (date >>> Today)

  ttlOrError = fromString ttl # map (days >>> Ttl) # note "Invalid TTL"

  portOrError = fromString port # map Port # note "Invalid port"

  maybeClient = elasticsearchClient (Hostname hostname) <$> portOrError

This can be called from the index.js like this:

Main.main("2020-01-21T00:00:00Z")("3")("localhost")("9200")()

Finally bundle and run with npm start or npx spago bundle-module --to Main.js && node index.js.

The complete source code from this post can be found on GitHub.

Conclusion

How to get started with a new technology can often best be shown by an example.

This post contains a complete and self-contained example of how to write a PureScript application and how to integrate it into an existing project.

We have covered:

Generic project setup with spago
JavaScript interop
Tests
Dependency injection
I/O and side effects
HTTP requests
JSON

Hope this gives you the edge on getting started with PureScript.

I'd love to hear how it works out for you! And I'm happy too about any other feedback.

🙏

Leif Battermann

Functional Programming Fu

Identify Side Effects And Refactor Fearlessly

Why avoid side effects?

PureScript Case Study And Guide For Newcomers

Elm And The Algorithm Of Music

From a music data type to performance

Interactive Command Line Applications In Scala –Well Structured And Purely Functional

How To Use Applicatives For Validation In Scala And Save Much Work

Parsers in Scala built upon existing abstractions

Using existing abstractions

Strongly Typed Configuration Access With Code Generation

Error and state handling with monad transformers in Scala

Use lambdas and combinators to improve your API

Modelling API Responses With sbt-json –Print Current Bitcoin Price

Leif Battermann

Functional Programming Fu

PureScript Case Study And Guide For Newcomers

Prerequisites

Project setup

JavaScript interop

Code Editor

Ready to go

Implementation

Use case

Packages

Imports

Types, types, types

Testing

Let's make the test succeed!

What is left?

Conclusion