by Leif

PureScript Case Study And Guide For Newcomers

Have you ever wanted to try out PureScript but were lacking a good way to get started?

If you

  • Have some prior functional programming knowledge - maybe you know Haskell, Elm, F#, or Scala, etc.
  • Want to solve a small task with PureScript
  • And want to get started quickly

This post is for you!

In this post we will walk through setting up and implementing a small exemplary PureScript application from scratch.

Prerequisites

We assume that we want to run our application from a Node.js environment, maybe an AWS Lambda or some other existing Node.js application.

Project setup

Let's start a new project which we call index-ttl with:

mkdir index-ttl && cd index-ttl && npm init

When prompted for test command enter:

npx spago test

(you can use the defaults for everything else)

Now we will install PureScript, spago a PureScript build tool and purty for code formatting:

npm install --save-dev purescript spago purty

(Note: If the above command fails depending on OS you might have to install the Debian package: libncurses5.)

To initialize a new spago project run

npx spago init

Let's see if our freshly initialized project works by running:

npm test

The output should look similar to this:

root@f883950b5408:/index-ttl# npm test

> index-ttl@1.0.0 test /index-ttl
> npx spago test

[info] Installation complete.
[info] Build succeeded.

You should add some tests.
[info] Tests succeeded.

JavaScript interop

The spago init command has generated some files. The file src/Main.purs contains the main function, the application's entry point.

To call this function from JavaScript we need to bundle the project first.

Given a file index.js in the root folder of the project with the following content:

const Main = require('./Main')

Main.main()

We can bundle and run PureScript project with the command:

npx spago bundle-module --to Main.js && node index.js

Optionally we can add the above command to the scripts property in the package.json file:

"scripts": {
  "test": "npx spago test",
  "start": "npx spago bundle-module --to Main.js && node index.js"
},

Then run the application with npm start.

Code Editor

If you do not have any other preferences I suggest using VS Code with the following extensions:

  • PureScript IDE
    • AFAIK you have to install PureScript globally with npm install -g purescript for this extension to work properly
  • vscode-purty
    • Specify npx purty as the path to the purty executable in the settings. Alternatively you can install purty globally with npm install -g purty

Ready to go

No we have everything in place to start hacking. Maybe you are good on your own now and do not need to read further?

If this is the case my last advice is to refer you to Pursuit, a documentation platform and search engine for PureScript packages (equivalent to Hoogle for Haskell).

In the following sections we will walk through an exemplary implementation of a PureScript application and see e.g. how to

  • Write tests
  • Do dependency injection
  • Handle I/O
  • Make HTTP requests
  • And parse JSON

Implementation

We are going to build an application that removes expired indices from Elasticsearch.

This is an ideal task for an exemplary PureScript application because

  • It involves I/O - in this case calls to the Elasticsearch REST API
  • It involves some processing logic which is not too complex
  • And it is a self-contained, real-world example

Use case

Depending on the kind and load of data that is stored in Elasticsearch it often makes sense to partition the data by time intervals based on index names tagged with a timestamp, e.g. the date. Consider the following indices:

  • index_2020_01_18
  • index_2020_01_19
  • index_2020_01_20
  • index_2020_01_21
  • index_2020_01_22
  • index_2020_01_23

Assume that we establish a 3 day time to live (TTL) and the current date is Jan 23, 2020. This means that all indices dated before Jan 20, 2020 should be removed, namely:

  • index_2020_01_18
  • index_2020_01_19

Given an Elasticsearch hostname and port, the current date, and a TTL in days our app should delete expired indices as described above. The signature of a function could look like this:

deleteExpiredIndices :: Hostname -> Port -> Today -> Ttl -> Aff Unit

But before we get to the implementation of this function we first need to install some additional packages, define some types and write some tests.

Packages

So far the setup has been pretty generic. But now we will install a few npm and PureScript packages specifically for this tutorial:

npm install xhr2
npx spago install aff datetime spec partial refs affjax argonaut-codecs aff-promise formatters

Note that xhr2 is needed for affjax (an AJAX library) if the app is not running in the browser.

spec is needed for testing.

Imports

In the file src/Main.purs we need the following imports:

import Prelude
import Affjax (printError)
import Affjax as AX
import Affjax.ResponseFormat as ResponseFormat
import Control.Promise (fromAff, Promise)
import Data.Argonaut.Core (Json)
import Data.Argonaut.Decode (decodeJson)
import Data.Array (filter, mapMaybe, takeEnd)
import Data.Bifunctor (lmap)
import Data.Date (diff)
import Data.DateTime (Date, canonicalDate, date)
import Data.Either (Either(..), either, note)
import Data.Enum (class BoundedEnum, toEnum)
import Data.Formatter.DateTime (unformatDateTime)
import Data.HTTP.Method (Method(..))
import Data.Int (toNumber, fromString)
import Data.Maybe (Maybe(..))
import Data.String (split, Pattern(..))
import Data.Time.Duration (Days(..))
import Data.Traversable (traverse, sequence)
import Effect (Effect)
import Effect.Aff (Aff)
import Effect.Class.Console (log)

Types, types, types

These types we have already used/seen above:

newtype Hostname
  = Hostname String

newtype Port
  = Port Int

newtype Ttl
  = Ttl Days

newtype Today
  = Today Date

Additionally we need a type that represents an index with a Show instance as well as a representation for an index with a date:

newtype Index
  = Index String

instance showIndex :: Show Index where
  show (Index index) = index

type DatedIndex
  = { name :: Index, date :: Date }

If we wanted to enforce even more type safety we could have put the types into their own modules. By only exposing smart constructors that parse the given arguments we would ensure only valid instances. But this is a bit too much ceremony in this particular situation for my taste. However, the larger the application gets, the more I would tend towards such an approach.

To be able to unit test our application we will pass in the dependencies - in this case the Elasticsearch API calls - as a record of functions:

type ElasticsearchClient
  = { indices :: Aff (Array Index)
    , deleteIndex :: Index -> Aff Unit
    }

Aff represents an asynchronous effect similar to Haskell's IO or Scala's cats.effect.IO or monix.eval.Task.

Now we can change the function signature slightly and also provide a dummy implementation to make it compile:

deleteExpiredIndices :: ElasticsearchClient -> Today -> Ttl -> Aff Unit
deleteExpiredIndices client today ttl = pure unit

Testing

For testing we will use purescript-spec.

We will need the following imports in the file test/Main.purs:

import Prelude
import Data.Array ((:))
import Data.Date.Component (Month(..))
import Data.DateTime (canonicalDate, Date)
import Data.Enum (toEnum)
import Data.Maybe (fromJust)
import Effect (Effect)
import Effect.Aff (Aff, launchAff_)
import Effect.Class (liftEffect)
import Effect.Ref as Ref
import Main (Index(..), Today(..), Ttl(..), deleteExpiredIndices)
import Partial.Unsafe (unsafePartial)
import Test.Spec (describe, it)
import Test.Spec.Assertions (shouldEqual)
import Test.Spec.Reporter.Console (consoleReporter)
import Test.Spec.Runner (runSpec)
import Data.Time.Duration (Days(..))
import Data.Int (toNumber)

Here is a single-example based test for our use case:

main :: Effect Unit
main =
  launchAff_
    $ runSpec [ consoleReporter ] do
        describe "Main" do
          describe "deleteExpiredIndices" do
            it "should delete expired indices" do
              result <- liftEffect $ Ref.new []
              let
                expected =
                  (map show)
                    [ Index "foo_2019_11_15"
                    , Index "foo_2020_01_06"
                    ]
              deleteExpiredIndices
                (client result)
                (Today $ date 2020 January 10)
                (Ttl $ Days $ toNumber 3)
              actual <- liftEffect $ (map show) <$> Ref.read result
              actual shouldEqual expected
  where
  client result =
    { indices:
      pure
        [ Index "foo_2020_01_11"
        , Index "foo_2020_01_10"
        , Index "foo_2020_01_09"
        , Index "foo_2020_01_08"
        , Index "foo_2020_01_07"
        , Index "foo_2020_01_06"
        , Index "foo_2019_11_15"
        , Index "index_without_date"
        ]
    , deleteIndex: \index -> liftEffect $ Ref.modify_ ((:) index) result :: Aff Unit
    }

  date :: Int -> Month -> Int -> Date
  date year month day = unsafePartial fromJust $ (\y d -> canonicalDate y month d) <$> toEnum year <*> toEnum day

Note that we created a test stub for the ElasticsearchClient that returns a hard coded list of indices. And it stores a list of deleted indices in a mutable reference of type Ref (Array Index) which we can use later to make the test assertion.

If we run the test with npm test or npx spago test it fails as expected with the following output:

Main » deleteExpiredIndices
  ✗ should delete expired indices:

  [] ≠ ["foo_2019_11_15","foo_2020_01_06"]

Summary
0/1 test passed

[error] Tests failed: exit code: 1

Let's make the test succeed!

This implementation should suffice:

findExpired :: Today -> Ttl -> Array DatedIndex -> Array DatedIndex
findExpired (Today today) (Ttl ttl) = filter (\indexDate -> (diff today indexDate.date) > ttl)

determineDate :: Index -> Maybe DatedIndex
determineDate (Index name) = { name: Index name, date: _ } <$> dateOrError
  where
  parsed = name # split (Pattern "_") # takeEnd 3

  dateOrError = case parsed of
    [ year, month, day ] ->
      canonicalDate
        <$> strToEnum year
        <*> strToEnum month
        <*> strToEnum day
    _ -> Nothing

  strToEnum ∷ forall e. BoundedEnum e => String -> Maybe e
  strToEnum = fromString >=> toEnum

deleteExpiredIndices :: ElasticsearchClient -> Today -> Ttl -> Aff Unit
deleteExpiredIndices client today ttl = do
  indices <- client.indices
  let
    datedIndices = indices # mapMaybe determineDate

    expired = findExpired today ttl datedIndices
  void $ client.deleteIndex traverse (expired # map _.name)

The function deleteExpiredIndices

  • Calls the indices function from the Elasticsearch client
  • Determines the dates by parsing the index names by calling determineDate
  • Determines which indices are expired by calling findExpired
  • And for each expired index calls deleteIndex from the Elasticsearch client

What is left?

Yes, we need to provide a real Elasticsearch client and integrate everything with our index.js.

If we query indices in JSON format we get result similar to this:

curl 'localhost:9200/_cat/indices?format=json&pretty'

[
  {
    "health" : "yellow",
    "status" : "open",
    "index" : "index_2020_01_22",
    "uuid" : "-gvBWXdPTjudCyVN-oxRKw",
    "pri" : "1",
    "rep" : "1",
    "docs.count" : "0",
    "docs.deleted" : "0",
    "store.size" : "230b",
    "pri.store.size" : "230b"
  },
  {
    "health" : "yellow",
    "status" : "open",
    "index" : "index_2020_01_23",
    "uuid" : "qLWKjqAQR5eubp5RWIn8HA",
    "pri" : "1",
    "rep" : "1",
    "docs.count" : "0",
    "docs.deleted" : "0",
    "store.size" : "230b",
    "pri.store.size" : "230b"
  }
]

The only relevant field of the JSON output is index. So let's create a type for that:

type ElasticsearchIndex
  = { index :: String }

A decoder for this can simply be derived with the help of the purescript-argonaut-codecs library by calling decodeJson.

The HTTP calls to the Elasticsearch API is done with affjax:

elasticsearchClient :: Hostname -> Port -> ElasticsearchClient
elasticsearchClient (Hostname hostname) (Port port) =
  { indices:
    indices
      <#> (map _.body)
      <#> (lmap printError)
      <#> (_ >>= indexFromJson)
      >>= either (\err -> log err $> []) pure
      <#> (map (_.index >>> Index))
  , deleteIndex:
    \(Index index) ->
      delete index >>= either (printError >>> log) (const (log $ "Deleted index: " <> index))
  }
  where
  url = "http://" <> hostname <> ":" <> show port

  indexFromJson :: Json -> Either String (Array ElasticsearchIndex)
  indexFromJson = decodeJson

  indices =
    AX.request
      ( AX.defaultRequest
          { url = url <> "/_cat/indices?format=json"
          , method = Left GET
          , responseFormat = ResponseFormat.json
          }
      )

  delete index =
    AX.request
      ( AX.defaultRequest
          { url = url <> "/" <> index
          , method = Left DELETE
          , responseFormat = ResponseFormat.ignore
          }
      )

In the main function we parse all inputs and call deleteExpiredIndices. The result is transformed to a Promise to provide JavaScript interop.

main :: String -> String -> String -> String -> Effect (Promise Unit)
main datetime ttl hostname port = do
  fromAff $ deleteExpiredIndices
    <$> maybeClient
    <*> dateOrError
    <*> ttlOrError
    # sequence
    >>= either log pure
  where
  dateOrError = unformatDateTime "YYYY-MM-DDTHH:mm:ssZ" datetime # map (date >>> Today)

  ttlOrError = fromString ttl # map (days >>> Ttl) # note "Invalid TTL"

  portOrError = fromString port # map Port # note "Invalid port"

  maybeClient = elasticsearchClient (Hostname hostname) <$> portOrError

This can be called from the index.js like this:

Main.main("2020-01-21T00:00:00Z")("3")("localhost")("9200")()

Finally bundle and run with npm start or npx spago bundle-module --to Main.js && node index.js.

The complete source code from this post can be found on GitHub.

Conclusion

How to get started with a new technology can often best be shown by an example.

This post contains a complete and self-contained example of how to write a PureScript application and how to integrate it into an existing project.

We have covered:

  • Generic project setup with spago
  • JavaScript interop
  • Tests
  • Dependency injection
  • I/O and side effects
  • HTTP requests
  • JSON

Hope this gives you the edge on getting started with PureScript.

I'd love to hear how it works out for you! And I'm happy too about any other feedback.

🙏