Joy of Elixir

18. Automated testing

This chapter is all about testing your code. Haven't we been doing that already? We've certainly be using it a few times. Isn't that testing?

I should be more specific. In this chapter, we'll be covering automated testing. We're going to write even more Elixir code that will put our current Elixir code (the stuff in lib/person.ex and lib/router.ex) through its paces.

As we develop larger and larger Elixir applications through using Elixir, testing each function of these applications in a manual way will get tedious. But still, we want to ensure that our application is functioning the way we meant it to! The way that we do this, and maintain our sanity, is through automated testing.

We'll explore here how to write these automated tests using another piece of Elixir called ExUnit. We will run these tests with a Mix task simply called test, and when that command runs we'll be able to verify if our application is working or not.

In the last section of this chapter, we'll cover how to write documentation for our code. This documentation will be helpful to anyone who wants to read it, but it can also be written in a way that runs further automated tests for our application, using something called doctests.

Introduction to ExUnit

Let's start by looking at this ExUnit thing. Elixir is made up several distinct parts, and you've now seen Elixir itself, IEx and Mix in action. The latest part to join the fray of our learning is ExUnit.

ExUnit is a tool that we can use to write automated tests for our application. Believe it or not, but we already have a test in our application. We've just been choosing not to see it at the moment. Let's look at this file now, test/people_test.exs:

defmodule PeopleTest do
  use ExUnit.Case
  doctest People

  test "greets the world" do
    assert People.hello() == :world
  end
end

When we ran mix new people one of the files that was generated was this file. Another one of the files that was generated was one called lib/people.ex. Not to be confused with lib/person.ex! Let's look at lib/people.ex now. I'll remove the documentation here to make it easier to focus on what we're doing.

defmodule People do
  def hello do
    :world
  end
end

The test in test/people_test.exs ensures that when the People.hello/0 function is called, that it returns :world. We can run this test with the test Mix task:

$ mix test

When we run this cmomand, we'll see this output:

..

Finished in 0.04 seconds
1 doctest, 1 test, 0 failures

We'll ignore the "1 doctest" part here, and focus on the "1 test" part. ExUnit is showing us here that it has ran a single test, and that there were no failures when it ran that test. This means that our People.hello/0 function is behaving. Great!

But what if it wasn't behaving? Well, let's take a look at what happens by changing this function to return something else other than :world:

defmodule People do
  def hello do
    :earth
  end
end

Let's run our tests again:

$ mix test

This time, there will be one failure:

1) test greets the world (PeopleTest)
   test/people_test.exs:5
   Assertion with == failed
   code:  assert People.hello() == :world
   left:  :earth
   right: :world
   stacktrace:
     test/people_test.exs:6: (test)

Without even having to run iex -S mix and then run People.hello to see if this function is working, we've been able to tell by running our automated test. Well, we probably knew it wouldn't work before then but... the point still stands! When our Elixir projects get larger, testing really does come in handy. It's good to practice it now on a small scale, so that we can employ it on a larger scale.

When we change the People code back:

defmodule People do
  def hello do
    :world
  end
end

And then re-run the command to run the tests:

$ mix test

We'll see everything is still working:

..

Finished in 0.04 seconds
1 doctest, 1 test, 0 failures

Now that we've seen how to use tests that already exist, we will now try (and succeed!) to write our own.

Writing our own tests

Writing our own tests will not very scary at all. In fact, we can copy a lot of what Mix has already done for us. The tests that we will write will now will be for our People.Person module, and we'll start with the full_name function. Let's create a new file at test/person_test.exs and put this content in it:

defmodule People.PersonTest do
  use ExUnit.Case
  alias People.Person

  test "full_name/1" do
    person = %Person{
      first_name: "Ryan",
      last_name: "Bigg"
    }

    assert person |> Person.full_name() === "Ryan Bigg"
  end
end

There are a two main things to remember with tests:

  1. The files do not get compiled into the "final version" of our application, so they are Elixir scripts, indicated by their ".exs" extension
  2. The common convention is to name the module for the tests after the module that's under test. We're testing the People.Person module here, and so our test follows that same pattern, just with Test on the end.

As tests are written inside of modules, we can use alias here, just like we have done in our other code. Inside this module, we use ExUnit.Case, which then gives us access to the test function that then lets us define a test.

Inside that test, we write code just like we might within a iex -S mix session. We build up a brand new person, and then pass that data through to Person.full_name/1. We then use the assert function from ExUnit to verify that the function matches the expected value.

Let's try running this test now with our new favourite command:

$ mix test

Here's what we'll see:

...

Finished in 0.05 seconds
1 doctest, 2 tests, 0 failures

Excellent! Our test is now verifying that our Person.full_name/1 function is behaving correctly.

The great thing about having these tests is that if we were to change the behaviour of Person.full_name/1 function to something else that was wrong, then this test would fail. Rather than doing that, what we'll do is change the test first, to assert that the full name is different. Just for... something different to do.

We'll be changing our test to assert that if there is no last name specified that it outputs just the first name. After all, what if people like Teller or Madonna were to use our Elixir application, we should support them too -- they have mononyms. We could go further and attempt to address the falsehoods programmers believe about names, but perhaps that's too much. Let's just do these mononym people first.

Let's add another test for these mononymic people:

defmodule PersonTest do
  use ExUnit.Case
  alias People.Person

  ...

  test "full_name/1 with mononyms" do
    teller = %Person{
      first_name: "Teller"
    }

    assert teller |> Person.full_name() === "Teller"

    madonna = %Person{
      first_name: "Madonna"
    }

    assert madonna |> Person.full_name() === "Madonna"
  end
end

This test has not one but two assertions in it! The first asserts that when we ask for Teller's full name, we get... well, we just get "Teller". The same goes for Madonna.

Let's try running this test now.

$ mix test

Here's what we'll see:

1) test full_name/1 with mononyms (PersonTest)
  test/person_test.exs:14
  Assertion with === failed
  code:  assert teller |> Person.full_name() === "Teller"
  left:  "Teller "
  right: "Teller"
  stacktrace:
    test/person_test.exs:19: (test)

Uh oh. Our test is failing! We are not as perfect as may have thought. We have two options here. First, we could change the test to expect "Teller " (with that space). The second is that we could fix the code. I personally like the latter option and so that's what we'll do.

Let's go over to lib/person.ex and take a look at our full_name function:

def full_name(%__MODULE__{
  first_name: first_name,
  last_name: last_name
}) do
  "#{first_name} #{last_name}"
end

This function takes the first_name and the last_name of the the passed in person and joins them together with a space. But now that we're working with people with only single name, we're going to need to do something different here. That something different is to use pattern matching!

The default value for a last_name is nil. So if we just have a first name but no last name specified, we could define a different full_name function to act accordingly. And then that shgould make our test happy too! Let's try that out.

def full_name(%__MODULE__{
  first_name: first_name,
  last_name: nil
}) do
  "#{first_name}"
end

def full_name(%__MODULE__{
  first_name: first_name,
  last_name: last_name
}) do
  "#{first_name} #{last_name}"
end

We now have two function clauses for full_name, one that matches when last_name is nil, and another when its any other value. In the first clause, we only output the first name.

This goes to show that in order to make a test work, sometimes we need to change the underlying code.

Let's run that test again and see what happens:

$ mix test
....

Finished in 0.05 seconds
1 doctest, 3 tests, 0 failures

Wonderful. Our Person.full_name/1 function now supports people with only a first name, as well as those with first and last names.

Documentation and tests

We've now seen one way to write our own tests. But there's two major ways to write our own tests in Elixir! The second of these is something called doctests.

We've seen doctests referred to in the test output before, we haven't talked about them yet. When we're writing Elixir code, we can document that code by leaving documentation above it. An example of this is found in the lib/people.ex file:

defmodule People do
  @moduledoc """
  Documentation for People.
  """

  @doc """
  Hello world.

  ## Examples

      iex> People.hello()
      :world

  """
  def hello do
    :world
  end
end

The @moduledoc and @doc syntax here are called module attributes, and they allow us to write documentation for either the module as a whole, or for functions. We'll just be talking about function documentation here. The """ ("triple quote") syntax here is another way of writing strings in Elixir, and is typically used to indicate strings that go over multiple lines, such as this documentation.

The documentation for the hello function here simply reads "Hello world." and then contains an example to show you how to use this function. This documentation is what will appear when we use the h helper in an IEx session:

iex> h People.hello
def hello
Hello world. ## Examples iex> People.hello() :world

Documentation, in the form of @doc and @moduledoc is the way that Elixir developers communicate how to use their code. The "Examples" that are listed here show what an expected output of hello is.

Back in Chapter 13, we saw this documentation for Enum.find_index/2:

def find_index(enumerable, fun)
Similar to find/3, but returns the index (zero-based) of the element instead of the element itself. ## Examples iex> Enum.find_index([2, 4, 6], fn x -> rem(x, 2) == 1 end) nil iex> Enum.find_index([2, 3, 4], fn x -> rem(x, 2) == 1 end) 1

This documentation is formatted in a similar way: a short description of the function, followed by some examples.

What's cool about both of these documentation examples is that they contain tests, in the form of examples. Both People.hello's documentation, and Enum.find_index/2's documentation ensure that those functions are working, just as their examples demonstrate.

Let's see a live example of this now, by writing some documentation tests for our Person.full_name function. We should document the behaviour of this function because it behaves differently depending on if there is only a first_name or a last_name, and documentation is a good way of demonstrating this. If we didn't have documentation, anyone wanting to learn how our code worked would have to read and understand the code, and that can take longer to understand than some clear-cut examples.

To document our full_name function, we'll go into lib/person.ex and add these lines before the first full_name function clause. It's important to note here that we do not need to document both clauses, as this documentation applies equally to both functions.

  @doc """
  Joins together a person's first name and last name.
  If that person only has a first name, then will only show that name.

  ## Examples

  iex> ryan = %Person{first_name: "Ryan", last_name: "Bigg"}
  iex> ryan |> Person.full_name
  "Ryan Bigg"

  iex> madonna = %Person{first_name: "Madonna"}
  iex> madonna |> Person.full_name
  "Madonna"
"""
def full_name(%__MODULE__{
      first_name: first_name,
      last_name: nil
    }) do
  "#{first_name}"
end

def full_name(%__MODULE__{
      first_name: first_name,
      last_name: last_name
    }) do
  "#{first_name} #{last_name}"
end

This new documentation now documents our full_name function, and it can be used to test the full_name function too... except there's one last thing we need to do before we can run this documentation as tests. We need to tell our tests to run the documentation. Let's go into test/person_test.exs and add a new line at the top:

defmodule PersonTest do
use ExUnit.Case
alias People.Person
doctest People.Person

When we use doctest in this test, we're telling ExUnit that we have documentation tests in the People.Person module that we would like to run, along side the other, regular tests that are defined in this file.

With that line now added, we will be able to run our tests again and see that there are more running:

$ mix test
......

Finished in 0.06 seconds
3 doctests, 3 tests, 0 failures

Okay, so the number has gone up, but how can we be really sure our documentation tests are the ones that are running here? To see the names of the running tests, we can pass an option to mix test called --trace:

mix test --trace
PersonTest
  * doctest People.Person.full_name/1 (2) (0.00ms)
  * test full_name/1 (0.00ms)
  * test full_name/1 with mononyms (0.00ms)
  * doctest People.Person.full_name/1 (1) (0.00ms)

PeopleTest
  * doctest People.hello/0 (1) (0.00ms)
  * test greets the world (0.00ms)


Finished in 0.06 seconds
3 doctests, 3 tests, 0 failures

Yes! Our doctests are indeed running. We've now seen how to write documentation in our Elixir modules that include examples that will then be incorporated into our tests. By using documentation in this way, we can ensure that our documentation is always up to date and that our code is always working.

Testing the router

Lastly, we need to talk about how to test another part of our code -- the Plug router that we added in the last chapter. To test a plug router is not as easy as calling a function and asserting on whatever comes back. But it's almost as easy!

We'll start out by creating a new test file for this, at test/router_test.exs:

defmodule RouterTest do
      use ExUnit.Case
      use Plug.Test

      @opts People.Router.init([])

      test "returns hello 'name'" do
        conn = conn(:get, "/hello/Izzy")

        conn = People.Router.call(conn, @opts)

        assert conn.state == :sent
        assert conn.status == 200
        assert conn.resp_body == "Hello Izzy!"
      end
    end

This new module starts out by using ExUnit.Case, and this gives us the ability to call that test function a little bit later on. But this module also uses Plug.Test. This module includes a few helper functions that we can use for making requests to our routers. One of these functions is conn.

Inside the test, we use conn to build up a test connection to make a request. We then send this request to the router by calling the People.Router.call function, passing in that conn. We also pass in @opts, which comes from calling the People.Router.init function at the top of this module.

Once we have called the router, we will get back a new conn, which will contain the response from the router. We check that this connection has been "sent" -- Plug's terminology for if the router has chosen to dispatch a request or not.

We also check here the status and the resp_body of the conn, making sure that it has suceeded and returned the correct message.

When we run this test, we should see that it passes:

$ mix test
.......

Finished in 0.07 seconds
3 doctests, 4 tests, 0 failures

See? I told you that it was almost as easy as calling a function and asserting on the outcome. This is made possible by the fact that routers (and plugs) are simply modules with functions. They take some arguments, and return some data. We can then assert easily on that data.

It is possible also to write tests for the People.Hello and People.Goodbye plugs, but I think that's best left as an exercise to the reader, and so I will include it at the end of this chapter in the exercises list.

And that's all there is to testing our modules and functions within Elixir. We use ExUnit to write these tests, and we can put these tests under the test directory of our application, or write them as "inline" documentation on our functions in the form of documentation tests.

Exercises

  1. Write a test that asserts Person.age/1 works correctly. This could be a regular test, or a documentation test. Why not try both?
  2. Write tests for the People.Hello and People.Goodbye plugs. Refer back to your tests for People.Router if you need to.