SimulatedHttp, functors and sed

· Allanderek's blog

#elm #programming

In this post I'm going to describe an awkwardness encountered when using elm-program-test to simulate HTTP events in the program being tested. I will then describe ML functors, a feature of the SML/Ocaml module system and show how these would solve the awkwardness. I'll then show how it's pretty simple to hack together a "poor-person's-functor" and use that to solve the aforementioned awkwardness.

# An awkwardness when simulating HTTP for testing

If you haven't used elm-program-test to test an entire Elm program I recommend trying it out. I've found that it not only does the obvious part of helping to build robust tests for Elm programs, but also helps me structure the Elm program in a way that is better for testing, but also just generally better.

In order to use elm-program-test you have to write an auxiliary update function for your program that returns a ProgramTest.SimulatedEffect rather than a Cmd. In order to do this without completely re-writing the program (and thus more or less negating the point of the testing), elm-program-test advises you to re-write your update function so that it returns a custom Effect type. Then in the real program you translate that Effect into a Cmd Msg, and for the tests you translate it into a ProgramTest.SimulatedEffect. This is best described with some Elm types:

1type Effect
2    = GetPosts
3    | GetPostComments
4update : Msg -> Model -> (Model, Effect)
6perform : Model -> Effect -> Cmd Msg
8simulate : Model -> Effect -> SimulatedEffect Msg

Now, helpfully, the modules in elm-program-test for creating simulated effects, tend to have the same API as their equivalent modules for real commands (in some cases they are incomplete as yet). For example SimulatedEffect.Http has the same API as Http from elm/http. This means it's relatively trivial to write the perform and simulate modules. In fact, they can be, essentially identical:

 1perform : Model -> Effect -> Cmd Msg
 2perform model effect =
 3    case effect of
 4        GetPosts ->
 5            Http.get
 6                { url = "/api/posts"
 7                , expect = Http.expectJson PostsReceived (Decode.list Post.decoder)
 8                }
 9        GetPostComments ->
10            Http.get
11                { url = "/api/posts"
12                , expect = Http.expectJson PostCommentsReceived (Decode.list Post.commentDecoder)
13                }
15simulate : Model -> Effect -> SimulatedEffect Msg
16simulate model effect =
17    case effect of
18        GetPosts ->
19            SimulatedHttp.get
20                { url = "/api/posts"
21                , expect = SimulatedHttp.expectJson PostsReceived (Decode.list Post.decoder)
22                }
23        GetPostComments ->
24            SimulatedHttp.get
25                { url = "/api/posts"
26                , expect = SimulatedHttp.expectJson PostCommentsReceived (Decode.list Post.commentDecoder)
27                }

In this very simple example the two functions are essentially identical aside from using the Http and SimulatedEffect.Http modules. This is generally true, even although the logic might be much more complicated. For example, you may have to check if the user is logged in, and if so send an authenticated request. Additionally, some effects, might not be simulated at all, for example Dom effects such as focusing are not yet simulated (though you could fake with a simulated port call). Anyway the point is, there ends up being quite a bit of duplicated code.

# ML functors

ML has a pretty powerful module system. In truth, although it is powerful, even when using O'caml to develop a compiler, I still very rarely found that I needed to reach for the full power of the module system. Functors just didn't come up very often. What are functors? They are essentially the equivalent to a module, that a function is to a value. So you can think of them as functions over modules. So you can write a module A, which takes another module B as an argument. The argument is specified as a module signature. This means that A is now a functor. You can apply the functor A to more than one other module as long as you have multiple modules that satisfy the signature B. The normal use cases for functors are pretty similar to the use cases for Haskell's type classes.

A common example is a dictionary/set, here is such an example written in a fantasy version of Elm with functors (and multiple modules within a single file):

 1signature Compare
 2    type Item
 3    compare : Item -> Item -> Order
 5functor Set (Item : Compare)
 6    type Set
 7        = Empty
 8        | Node Set Item Set
 9    empty : Set
10    empty =
11        Empty
12    add : Item.Item -> Set -> Set
13    add item currentSet =
14        case currentSet of
15            Empty ->
16                Node Empty item Empty
17            Node left nodeItem right ->
18                case item nodeItem of
19                    LT ->
20                        Node (add item left) nodeItem right
21                    GT ->
22                        Node left nodeItem (add item right)
23                    EQ ->
24                        currentSet
25    ...
27module IntCompare
28    type alias Item = Int
29    compare : Item -> Item -> Order
30    compare =
31module IntSet = Set(IntCompare)

Obviously a real implementation would have the other common Set functions, and you could use this to make Sets of things that aren't in Elm's comparable type class, by actually writing your own compare function. You can read the documentation for O'caml's module system here

# Effects, SimulatedEffects, and Functors

Hopefully it is pretty obvious how this solves our awkwardness with requests and simulated requests. Because the SimulatedHttp module from elm-program-test has the same API (or signature) as the standard Http module, you can easily write a functor that, given either of those two modules, produces a Perform module that has the correct type. Again using our fantasy version of Elm with functors:

 1type Effect
 2    = GetPosts
 3    | GetPostComments
 4update : Msg -> Model -> (Model, Effect)
 6module InterpretEffects(Http : <suitable-signature>, Result : <signature with return type>)
 7    perform : Model -> Effect -> Return.Cmd Msg
 8    perform model effect =
 9        case effect of
10            GetPosts ->
11                Http.get
12                    { url = "/api/posts"
13                    , expect = Http.expectJson PostsReceived (Decode.list Post.decoder)
14                    }
15            GetPostComments ->
16                Http.get
17                    { url = "/api/posts"
18                    , expect = Http.expectJson PostCommentsReceived (Decode.list Post.commentDecoder)
19                    }
20module Perform = InterpretEffects(Http, ( Cmd ) )
21module Simulate = InterpretEffects(SimulatedEffect.Http, ( SimulatedEffect ))

I've had to fudge this a bit because the return types of both modules (Cmd and SimulatedEffect) are not defined in the respective Http module, but you get the idea.

# Elm doesn't have functors

However, it's pretty simple to fake them with the use of the unix program sed. Just write the Perform module as you would, and then copy into a Simulate module, whilst modifying only the parts that change. The use of an import as can make this especially doable. First the Perform module, remember, this is translating our Effect custom type into the Elm's standard library Cmd type:

 1module Perform exposing (perform)
 3import Model exposing (Model)
 4import Model exposing (Msg)
 5import Http
 7perform : Model -> Effect -> Cmd Msg
 8perform model effect =
 9    case effect of
10        GetPosts ->
11            Http.get
12                { url = "/api/posts"
13                , expect = Http.expectJson PostsReceived (Decode.list Post.decoder)
14                }
15        GetPostComments ->
16            Http.get
17                { url = "/api/posts"
18                , expect = Http.expectJson PostCommentsReceived (Decode.list Post.commentDecoder)
19                }

Now the Simulate module that we will produce with sed:

 1module Simulate exposing (perform)
 3import Model exposing (Model)
 4import Model exposing (Msg)
 5import SimulatedEffect.Http as Http
 6import ProgramTest exposing (SimulatedEffect)
 8perform : Model -> Effect -> SimulatedEffect Msg
 9perform model effect =
10    case effect of
11        GetPosts ->
12            Http.get
13                { url = "/api/posts"
14                , expect = Http.expectJson PostsReceived (Decode.list Post.decoder)
15                }
16        GetPostComments ->
17            Http.get
18                { url = "/api/posts"
19                , expect = Http.expectJson PostCommentsReceived (Decode.list Post.commentDecoder)
20                }

So the only changes are:

  1. The module line at the top
  2. We have to add an import so that we can use the SimulatedEffect type.
  3. We have to change the Http import to SimulatedEffect.Http, because we alias that we don't need to change any of accesses to that module.
  4. Finally any uses of Cmd Msg we have to change to SimulatedEffect Msg or we could have added a type alias.

And that's it. We could also change the type of the function from perform to simulate if we really wanted ot. As promised, this is easily achieveable with a sed script:

1IMPORT_HTTP="s/import Http/import SimulatedEffect.Http as Http/g"
2IMPORT_SIMEFFECT="0,/^$/ s/^$/\nimport ProgramTest exposing (SimulatedEffect)/"
4sed "s/module Perform exposing (perform)/module Simulate exposing (perform)/g;
7${REPLACE_CMD}" src/Perform.elm > src/Simulate.elm

I put this in a file and then also call actually run the tests, so that this module is generated before every run of the tests. It's a simple sed script and adds negligible time to the test run time. I think all the parts are fairly self explanatory the 0,/^$/ s/^$ foo at the start of the IMPORT_SIMEEFFECT is basically saying "Replace the first occurrence of a blank line with the follow", because what I replace it with starts with \n we retain the blank line.

Of course in a real application, including where I actually use this, the Perform module is perhaps broken up into smaller modules. That's okay, you can have a Requests module that is translated into a SimulatedRequests module, and then do the same import translation in your main Peform -> Simulate translation. I even translate a ports module into one that isn't a ports module but uses SimulatedEffect.Ports to created simulated versions of the ports.