18. Automated testing
This chapter is all about testing your code. Haven't we been doing that already? We've certainly be using it a few times. Isn't that testing?
I should be more specific. In this chapter, we'll be covering automated testing. We're going to write even
more Elixir code that will put our current Elixir code (the stuff in lib/person.ex
and
lib/router.ex
) through its paces.
As we develop larger and larger Elixir applications through using Elixir, testing each function of these applications in a manual way will get tedious. But still, we want to ensure that our application is functioning the way we meant it to! The way that we do this, and maintain our sanity, is through automated testing.
We'll explore here how to write these automated tests using another piece of Elixir called ExUnit. We will run these
tests with a Mix task simply called test
, and when that command runs we'll be able to verify if our
application is working or not.
In the last section of this chapter, we'll cover how to write documentation for our code. This documentation will be helpful to anyone who wants to read it, but it can also be written in a way that runs further automated tests for our application, using something called doctests.
Introduction to ExUnit
Let's start by looking at this ExUnit thing. Elixir is made up several distinct parts, and you've now seen Elixir itself, IEx and Mix in action. The latest part to join the fray of our learning is ExUnit.
ExUnit is a tool that we can use to write automated tests for our application. Believe it or not, but we already
have a test in our application. We've just been choosing not to see it at the moment. Let's look at this file now,
test/people_test.exs
:
defmodule PeopleTest do
use ExUnit.Case
doctest People
test "greets the world" do
assert People.hello() == :world
end
end
When we ran mix new people
one of the files that was generated was this file. Another one of the
files that was generated was one called lib/people.ex
. Not to be confused with
lib/person.ex
! Let's look at lib/people.ex
now. I'll remove the documentation here to
make it easier to focus on what we're doing.
defmodule People do
def hello do
:world
end
end
The test in test/people_test.exs
ensures that when the People.hello/0
function is
called, that it returns :world
. We can run this test with the test
Mix task:
$ mix test
When we run this command, we'll see this output:
..
Finished in 0.04 seconds
1 doctest, 1 test, 0 failures
We'll ignore the "1 doctest" part here, and focus on the "1 test" part. ExUnit is showing us here that it has ran
a single test, and that there were no failures when it ran that test. This means that our
People.hello/0
function is behaving. Great!
But what if it wasn't behaving? Well, let's take a look at what happens by changing this function to return
something else other than :world
:
defmodule People do
def hello do
:earth
end
end
Let's run our tests again:
$ mix test
This time, there will be one failure:
1) test greets the world (PeopleTest)
test/people_test.exs:5
Assertion with == failed
code: assert People.hello() == :world
left: :earth
right: :world
stacktrace:
test/people_test.exs:6: (test)
Without even having to run iex -S mix
and then run People.hello
to see if this function
is working, we've been able to tell by running our automated test. Well, we probably knew it wouldn't work before
then but... the point still stands! When our Elixir projects get larger, testing really does come in handy. It's
good to practice it now on a small scale, so that we can employ it on a larger scale.
When we change the People
code back:
defmodule People do
def hello do
:world
end
end
And then re-run the command to run the tests:
$ mix test
We'll see everything is still working:
..
Finished in 0.04 seconds
1 doctest, 1 test, 0 failures
Now that we've seen how to use tests that already exist, we will now try (and succeed!) to write our own.
Writing our own tests
Writing our own tests will not very scary at all. In fact, we can copy a lot of what Mix has already done for us.
The tests that we will write will now will be for our People.Person
module, and we'll start with the
full_name
function. Let's create a new file
at test/person_test.exs
and put this content in it:
defmodule People.PersonTest do
use ExUnit.Case
alias People.Person
test "full_name/1" do
person = %Person{
first_name: "Ryan",
last_name: "Bigg"
}
assert person |> Person.full_name() === "Ryan Bigg"
end
end
There are a two main things to remember with tests:
- The files do not get compiled into the "final version" of our application, so they are Elixir scripts, indicated by their ".exs" extension
- The common convention is to name the module for the tests after the module that's under test. We're testing
the
People.Person
module here, and so our test follows that same pattern, just withTest
on the end.
As tests are written inside of modules, we can use alias
here, just like we have done in our other
code. Inside this module, we use ExUnit.Case
, which then gives us access to the test
function that then lets us define a test.
Inside that test, we write code just like we might within a iex -S mix
session. We build up a brand
new person, and then pass that data through to Person.full_name/1
. We then use the
assert
function from ExUnit to verify that the function matches the expected value.
Let's try running this test now with our new favourite command:
$ mix test
Here's what we'll see:
...
Finished in 0.05 seconds
1 doctest, 2 tests, 0 failures
Excellent! Our test is now verifying that our Person.full_name/1
function is behaving correctly.
The great thing about having these tests is that if we were to change the behaviour of
Person.full_name/1
function to something else that was wrong, then this test would fail. Rather than
doing that, what we'll do is change the test first, to assert that the full name is different. Just
for... something different to do.
We'll be changing our test to assert that if there is no last name specified that it outputs just the first name. After all, what if people like Teller or Madonna were to use our Elixir application, we should support them too -- they have mononyms. We could go further and attempt to address the falsehoods programmers believe about names, but perhaps that's too much. Let's just do these mononym people first.
Let's add another test for these mononymic people:
defmodule PersonTest do
use ExUnit.Case
alias People.Person
...
test "full_name/1 with mononyms" do
teller = %Person{
first_name: "Teller"
}
assert teller |> Person.full_name() === "Teller"
madonna = %Person{
first_name: "Madonna"
}
assert madonna |> Person.full_name() === "Madonna"
end
end
This test has not one but two assertions in it! The first asserts that when we ask for Teller's full name, we get... well, we just get "Teller". The same goes for Madonna.
Let's try running this test now.
$ mix test
Here's what we'll see:
1) test full_name/1 with mononyms (PersonTest)
test/person_test.exs:14
Assertion with === failed
code: assert teller |> Person.full_name() === "Teller"
left: "Teller "
right: "Teller"
stacktrace:
test/person_test.exs:19: (test)
Uh oh. Our test is failing! We are not as perfect as may have thought. We have two options here. First, we could change the test to expect "Teller " (with that space). The second is that we could fix the code. I personally like the latter option and so that's what we'll do.
Let's go over to lib/person.ex
and take a look at our full_name
function:
def full_name(%__MODULE__{
first_name: first_name,
last_name: last_name
}) do
"#{first_name} #{last_name}"
end
This function takes the first_name
and the last_name
of the passed in person and
joins them together with a space. But now that we're working with people with only single name, we're going to
need to do something different here. That something different is to use pattern matching!
The default value for a last_name
is nil
. So if we just have a first name but no last
name specified, we could define a different full_name
function to act accordingly. And then that
should make our test happy too! Let's try that out.
def full_name(%__MODULE__{
first_name: first_name,
last_name: nil
}) do
"#{first_name}"
end
def full_name(%__MODULE__{
first_name: first_name,
last_name: last_name
}) do
"#{first_name} #{last_name}"
end
We now have two function clauses for full_name
, one that matches when last_name
is
nil
, and another when its any other value. In the first clause, we only output the first name.
This goes to show that in order to make a test work, sometimes we need to change the underlying code.
Let's run that test again and see what happens:
$ mix test
....
Finished in 0.05 seconds
1 doctest, 3 tests, 0 failures
Wonderful. Our Person.full_name/1
function now supports people with only a first name, as well as
those with first and last names.
Documentation and tests
We've now seen one way to write our own tests. But there's two major ways to write our own tests in Elixir! The second of these is something called doctests.
We've seen doctests referred to in the test output before, we haven't talked about them yet. When we're writing
Elixir code, we can document that code by leaving documentation above it. An example of this is found in
the
lib/people.ex
file:
defmodule People do
@moduledoc """
Documentation for People.
"""
@doc """
Hello world.
## Examples
iex> People.hello()
:world
"""
def hello do
:world
end
end
The @moduledoc
and @doc
syntax here are called module attributes, and they
allow us to write documentation for either the module as a whole, or for functions. We'll just be talking about
function documentation here. The """
("triple quote") syntax here is another way of writing strings
in Elixir, and is typically used to indicate strings that go over multiple lines, such as this documentation.
The documentation for the hello
function here simply reads "Hello world." and then contains an
example to show you how to use this function. This documentation is what will appear when we use the
h
helper in an IEx
session:
iex> h People.hello
def hello
Hello world.
## Examples
iex> People.hello()
:world
Documentation, in the form of @doc
and @moduledoc
is the way that Elixir developers
communicate how to use their code. The "Examples" that are listed here show what an expected output of
hello
is.
Back in Chapter 13, we saw this documentation for Enum.find_index/2
:
def find_index(enumerable, fun)
Similar to find/3, but returns the index (zero-based) of the element instead of
the element itself.
## Examples
iex> Enum.find_index([2, 4, 6], fn x -> rem(x, 2) == 1 end)
nil
iex> Enum.find_index([2, 3, 4], fn x -> rem(x, 2) == 1 end)
1
This documentation is formatted in a similar way: a short description of the function, followed by some examples.
What's cool about both of these documentation examples is that they contain tests, in the form of examples. Both
People.hello
's documentation, and Enum.find_index/2
's documentation ensure that those
functions are working, just as their examples demonstrate.
Let's see a live example of this now, by writing some documentation tests for our Person.full_name
function. We should document the behaviour of this function because it behaves differently depending on if there
is only a first_name
or a last_name
, and documentation is a good way of demonstrating
this. If we didn't have documentation, anyone wanting to learn how our code worked would have to read and
understand the code, and that can take longer to understand than some clear-cut examples.
To document our full_name
function, we'll go into lib/person.ex
and add these lines
before the first full_name
function clause. It's important to note here that we do not need to
document both clauses, as this documentation applies equally to both functions.
@doc """
Joins together a person's first name and last name.
If that person only has a first name, then will only show that name.
## Examples
iex> ryan = %Person{first_name: "Ryan", last_name: "Bigg"}
iex> ryan |> Person.full_name
"Ryan Bigg"
iex> madonna = %Person{first_name: "Madonna"}
iex> madonna |> Person.full_name
"Madonna"
"""
def full_name(%__MODULE__{
first_name: first_name,
last_name: nil
}) do
"#{first_name}"
end
def full_name(%__MODULE__{
first_name: first_name,
last_name: last_name
}) do
"#{first_name} #{last_name}"
end
This new documentation now documents our full_name
function, and it can be used to test the
full_name
function too... except there's one last thing we need to do before we can run this
documentation as tests. We need to tell our tests to run the documentation. Let's go into
test/person_test.exs
and add a new line at the top:
defmodule PersonTest do
use ExUnit.Case
alias People.Person
doctest People.Person
When we use doctest
in this test, we're telling ExUnit that we have documentation tests in the
People.Person
module that we would like to run, along side the other, regular tests that are defined
in this file.
With that line now added, we will be able to run our tests again and see that there are more running:
$ mix test
......
Finished in 0.06 seconds
3 doctests, 3 tests, 0 failures
Okay, so the number has gone up, but how can we be really sure our documentation tests are the ones that are
running here? To see the names of the running tests, we can pass an option to mix test
called
--trace
:
mix test --trace
PersonTest
* doctest People.Person.full_name/1 (2) (0.00ms)
* test full_name/1 (0.00ms)
* test full_name/1 with mononyms (0.00ms)
* doctest People.Person.full_name/1 (1) (0.00ms)
PeopleTest
* doctest People.hello/0 (1) (0.00ms)
* test greets the world (0.00ms)
Finished in 0.06 seconds
3 doctests, 3 tests, 0 failures
Yes! Our doctests are indeed running. We've now seen how to write documentation in our Elixir modules that include examples that will then be incorporated into our tests. By using documentation in this way, we can ensure that our documentation is always up to date and that our code is always working.
Testing the router
Lastly, we need to talk about how to test another part of our code -- the Plug router that we added in the last chapter. To test a plug router is not as easy as calling a function and asserting on whatever comes back. But it's almost as easy!
We'll start out by creating a new test file for this, at test/router_test.exs
:
defmodule RouterTest do
use ExUnit.Case
use Plug.Test
@opts People.Router.init([])
test "returns hello 'name'" do
conn = conn(:get, "/hello/Izzy")
conn = People.Router.call(conn, @opts)
assert conn.state == :sent
assert conn.status == 200
assert conn.resp_body == "Hello Izzy!"
end
end
This new module starts out by using ExUnit.Case
, and this gives us the ability to call that
test
function a little bit later on. But this module also uses Plug.Test
. This module
includes a few helper functions that we can use for making requests to our routers. One of these functions is
conn
.
Inside the test, we use conn
to build up a test connection to make a request. We then send this
request to the router by calling the People.Router.call
function, passing in that conn
.
We also pass in @opts
, which comes from calling the People.Router.init
function at the
top of this module.
Once we have called the router, we will get back a new conn
, which will contain the response from the
router. We check that this connection has been "sent" -- Plug's terminology for if the router has chosen to
dispatch a request or not.
We also check here the status
and the resp_body
of the conn
, making sure
that it has succeeded and returned the correct message.
When we run this test, we should see that it passes:
$ mix test
.......
Finished in 0.07 seconds
3 doctests, 4 tests, 0 failures
See? I told you that it was almost as easy as calling a function and asserting on the outcome. This is made possible by the fact that routers (and plugs) are simply modules with functions. They take some arguments, and return some data. We can then assert easily on that data.
It is possible also to write tests for the People.Hello
and People.Goodbye
plugs, but I
think that's best left as an exercise to the reader, and so I will include it at the end of this chapter in the
exercises list.
And that's all there is to testing our modules and functions within Elixir. We use ExUnit to write these tests,
and we can put these tests under the test
directory of our application, or write them as "inline"
documentation on our functions in the form of documentation tests.
Exercises
-
Write a test that asserts
Person.age/1
works correctly. This could be a regular test, or a documentation test. Why not try both? -
Write tests for the
People.Hello
andPeople.Goodbye
plugs. Refer back to your tests forPeople.Router
if you need to.