Elm Applicatives & Json Decoders ∷ [ toast.al ]

Originally for Elm v0.17. Updated on for Elm v0.18.

One of the first things you will notice is Json.Decode.map and Json.Decode.map8, and the numbers between. Soon you will ask: how do decode structures larger than map8—where is map9, map10, and, more importantly, mapN? The documentation will lead you to elm-decode-pipeline which is not inherently bad, but hides the underlying concept from users with a DSL.

In Elm, Json.Decode.Decoder is an applicative functor—a fancy functional programming term that we’ll attempt to demystify. By leveraging the power of map and apply, we should be able to wrangle even the toughest JSON thrown our way. I’ll be the first to admit that initially I found JSON decoding incredibly confusing—literally to the point where I abandoned some projects early on because I didn’t know how to decode some scary JSON.

What we’re building towards:

Pokémon Viewer working demo

If you want to follow along and run the code for yourself

This install Elm, its dependencies, and start up Elm Reactor for you to point your browser at http://localhost:8000.

git clone https://github.com/toastal/elm-applicatives-and-json-decoders.git
cd elm-applicatives-and-the-json-decoders
npm install
npm start

Quick Fly-By at Applicatives via `Maybe`

So even the most novice Elm developer knows how Maybe works—it’s a Just a or Nothing.

To go from a Just 1 to a Just 3 we’d use map because Maybe is a functor.

Maybe.map ((+) 2) (Just 1) == Just 3
--=> True

The a in Just a can also be a function.

So what happens if we had a Just (+) with the addition infix operator… how do we use this to add in an applicative manner to add Just 1 and Just 2?

Applicatives have 2 terms:

pure, singleton
apply, ap, <*>

Pure

pure is a function from something a that creates a singleton list for the default case of an applicative of the same type a (which is why it sometimes goes by the name singleton). But to make it more clear, let’s look at the type signature of pure in Haskell.

pure :: Applicative f => a -> f a

Knowing that, let’s look at some examples of Elm singletons:

Maybe

Just : a -> Maybe a

Result

Ok : a -> Result x a

Set

singleton : comparable -> Set comparable

List

flip (::) [] : a -> List a

Apply

Next we peek at the ability to lift values in an applicative with apply (Haskell):

liftA :: Applicative f => (a -> b) -> f a -> f b
(<*>) :: f (a -> b) -> f a -> f b

So what is exactly is Just (+)?

foo : Maybe (number -> number -> number)
foo =
    Just (+)

Looking at the type signature, it’s a Maybe holding the addition function, (+).

What is Maybe.Extra.andMap?

andMap : Maybe (a -> b) -> Maybe a -> Maybe b

That looks an awful lot like apply/lift… So let’s use it:

import Maybe.Extra as Maybe

-- foo = Just (+)

bar : Maybe (number -> number)
bar =
    foo |> Maybe.andMap Just 1

And let’s apply values to completion

baz : Maybe number
baz =
    bar |> Maybe.andMap Just 2


isJust3 : Bool
isJust3 =
    baz == Just 3
--=> True

And the same thing, creating an infix for andMap/apply:

import Maybe.Extra as Maybe


singleton : a -> Maybe a
singleton =
    Just


infixl 2 <*>
(<*>) : Maybe (a -> b) -> Maybe a -> Maybe b
(<*>) =
    flip Maybe.andMap


isJust3 : Bool
isJust3 =
    (singleton (+) <*> Just 1 <*> Just 2) == Just 3
--=> True


isNothing : Bool
isNothing =
    (singleton (+) <*> Just 1 <*> Nothing) == Nothing
--=> True


isAlsoNothing : Bool
isAlsoNothing =
    (singleton (+) <*> Nothing <*> Just 2) == Nothing

Look at the the demo MaybeApplicative.elm.

So where have we seen something like this?

-- given the function foo…
foo : number -> number -> number
foo x y =
    x * y


-- partially apply in a 1
foo' : number -> number
foo' =
    foo 1


-- Easter Egg: we’ve created a monoid
foo' 37 == 37
--=> True

The (space) operator is function application ;)

So how does this relate to Json Decoders?

Looking in the docs for Json.Decode we have succeed:

succeed : a -> Decoder a

Looks pretty pure and singleton-y to me…

And in Json.Decode.Extra we have andMap and its flipped infix |:

andMap : Decoder a -> Decoder (a -> b) -> Decoder b
(|:) : Decoder (a -> b) -> Decoder a -> Decoder b

Well that’s obviously the apply function…

So let’s apply (heh heh) our knowledge

We’ll start by creating some hand-rolled, artisnal JSON:

coolJson : String
coolJson =
    """
    [
        {
            "foo": 0,
            "bar": true
        },
        {
            "foo": 1,
            "bar": true
        },
        {
            "foo": 2,
            "bar": false
        }
    ]
    """

Create some type alias to represent these cool data items

type alias CoolItem =
    { foo : Int
    , bar : Bool
    }

We are also going to need to know about field (whose old infix was := prior to Elm v0.18) with the signature String -> Decoder a -> Decoder a

So field is used to apply the given decoder given a string for a key in a JSON object (e.g. “foo” will be decoded as an integer).

import Json.Decode as Decode


fooDecoder : Decoder Int
fooDecoder =
    Decode.field "foo" Decode.int

Now that we’re set up, let’s create a `CoolItem` decoder using applicative-style

A reminder about Elm types, CoolItem can be used as a constructor for our CoolItem, which is this case is CoolItem : (Int -> Bool -> CoolItem).

So what happens when when apply in our foo decoder?

Let’s look at some type signatures and find out:

import Json.Decode as Decode exposing (Decoder)
import Json.Docode.Extra as Decode exposing ((|:))


-- Creating out Decoder our CoolItem constructor
baz : Decoder (Int -> Bool -> CoolItem)
baz =
    Decode.succeed CoolItem


-- When we apply a decoder for our first property, foo => Int
qux : Decoder (Bool -> CoolItem)
qux =
    Decode.succeed CoolItem
        |: Decode.field "foo" Decode.int


-- And the full decoder, which completes the CoolItem constructor
coolItemDecoder : Decoder CoolItem
coolItemDecoder =
    Decode.succeed CoolItem
        |: Decode.field "foo" Decode.int
        |: Decode.field "bar" Decode.bool

So now that we have a Decoder CoolItem, all we have to do now is set up our app to decode the actual JSON string into a List CoolItem.

import Html exposing (Html, text)
import Json.Decode as Decode


view : a -> Html String
view =
    text << toString


main : Html String
main =
    coolJson
        |> Decode.decodeString (Decode.list coolListDecoder)
        |> view

Decode.decodeString : Decoder a -> String -> Result String a

Decode.list : Decoder a -> Decoder (List a)

But, go look at the demo JsonDecodeApplicative.elm.

Thinking About Elm Applicatives

So how do we find Applicatives in a language like Elm without type classes? Look for type signatures, common names, or think about what the singleton and andMap would be.

In Elm you’ll see the term singleton or succeed (like Decoder and Task) for pure. …And most of the time you’ll see andMap, for <*>.

So let’s see some in action which uses Decode.map and nested (|:) with some real JSONHTTP requests because some folks want to see a more real-world example about how to put this together with Task and Cmd well: Pokémon Viewer demo PokemonViewer.elm.

Takeaway

If you’re just looking into the base Json.Decode that comes in the core, you’ll see that Json.Decode.mapN stops at map8. Many people dead-end here and don’t know where to look next when their objects are bigger. Other blog articles you might find online will often show you how this can be done with elm-decode-pipeline or something else, but they are showing you how instead of why. The why for why this works is applicatives—which is based in math and laws. With the applicative style, we have a tool that can cover all cases as well as an understand about what’s going on under Elm’s hood. An important caveat though, is that these will fail silently when you write bad decoders. This can be solved using Results and the Run A Decoder of the documentation—or using the mapN functions when possible.

Special thanks to fresheyeball for explaining this shit to me.

toast.al