Joy of Elixir

9. Working with lists

In this chapter, we'll cover some functions that Elixir has helpfully got built-in to the language to help us work with lists.

Just like there is a String module for working with strings, there is also a List module for working with lists. This module includes functions like first that will tell you what the first item in a list is:

List.first([1, 2, 3, 4])
1

And it also lets you figure out what the last item is too:

List.last([1, 2, 3, 4])
4

We looked earlier at how we could reverse a string, but how about if we wanted to reverse something else, like this list?

iex> animals_or_derivatives_of_animals = ["cat", "dog", "cow", "turducken"]

We would think that just like first and last that there should be a reverse too! We might think this because Elixir provides us a way to reverse strings -- with String.reverse("string") -- so why not lists too? Let's try it out:

iex> List.reverse(animals_or_derivatives_of_animals)
List.reverse(animals_or_derivatives_of_animals)
** (UndefinedFunctionError) function List.reverse/1 is undefined or private
    (elixir) List.reverse(["cat", "dog", "cow", "turducken"])

Uh oh, there's that red text again. It seems like that this function doesn't exist, even though we thought it did. We are lacking the superpower of being able to will functions into being at this current point in time, so we'll have to do some sleuthing to figure out why it's missing.

The computer is telling us that Elixir doesn't know about a function called List.reverse, or the function is "private". The computer (slyly) won't tell us which one of non-existence or privateness it is, but we'll assume the first case here: that the function is undefined. (We'll come back to private functions a little later on in the book.)

Ok, so we've talked about what /1 means (Izzy is now deep in thought), so let's talk about why List.reverse/1 doesn't exist.

Thank you Reader, but our princess function is in another castle module

The short version of why the List.reverse/1 function doesn't exist is because it lives in a separate module; it exists in the Enum module. The name Enum is short for Enumerable, and it's done that way because nobody has the time to write out Enumerable correctly every single time. Even as this book's author I have a hard time spelling it correctly each time! Thank goodness for autocomplete.

"What on earth is an enumerable?", Izzy cries out. Hold your horses, Izzy. We're getting to that.

Lists are a type of data in Elixir called an enumerable. Maps are also enumerables. This means that they can be enumerated through; which means that you can do something with each item in the enumerable object (a list or a map). For instance, if we were to write out our list on a piece of paper, it might look something like this:

• Cat
• Dog
• Cow
• Turducken

It's possible to write each item from the list separately from the other items in the list. Because we're able to do this we can safely say that this list is enumerable. We could try to do the same thing for a number (like 1,354), but it wouldn't make sense:

• 1
• 3
• 5
• 4

Numbers are not enumerable because it doesn't make sense for them to be written like this, unlike our list.

Similar to this, we could enumerate through a map. If we were to take one of our maps from earlier...

%{name: "Izzy", age: "30ish", gender: "Female"}

...and write each key and value pair down, they may look something like this:

Name

Izzy

Age

30ish

Gender

Female

Again, it makes sense for a map to be an enumerable because you can enumerate through each of the key and value pairs in the map.

And now for the big reveal

We need to keep in mind that when we're working with enumerable things in Elixir the function that we need might live elsewhere. While there is most certainly a List and a Map module (which we'll see in the next chapter) -- sometimes, we need to look in the Enum module too for the function we want.

We tried looking in the List module to find the reverse/1 function so that we could turn our list around but it wasn't there. We found out a short while ago that the function is actually within the Enum module, so let's try using that function. Before that, let's get our list in Elixir form again. It's been a while since we've seen it that way:

iex> animals_or_derivatives_of_animals = ["cat", "dog", "cow", "turducken"]

Since we now know that lists are enumerables, and that the List.reverse/1 function doesn't exist but we also (now) know that there's an Enum module to work with this sort of thing, we can probably guess that there's going to be an Enum.reverse/1 function. Let's try it and see:

iex> Enum.reverse(animals_or_derivatives_of_animals)
  ["turducken", "cow", "dog", "cat"]

Hooray! We were able to reverse our list.

Izzy's features relax from intense concentration to a more neutral setting and she asks, "Hey, you mentioned before you could enumerate through a list or a map, but you didn't show an example of that. What gives?" You're absolutely right, Izzy. I was too distracted with explaining why List.reverse/1 didn't exist to explain how to enumerate through an enumerable. I'm glad that someone is paying attention.

Let's all now take a look at how to do that before we move onto other functions. We've talked about enumerables briefly and it would be a shame to stop so early when you can do so much more with them than reverse them.

Enumerating the enumerables

To appease Izzy (and the masses that she leads) yet again, we're going to need to look at how to enumerate through enumerables. What this means is that we're going to get an enumerable (a list or a map) and we're going to go through each of the enumerable's items and do something with them.

Let's start with a list. How about a list of the world's top five most liveable cities, just for something different?

iex> cities = ["vienna", "melbourne", "osaka", "calgary", "sydney"]

The simplest way of enumerating through a list in Elixir is to use the Enum.each/2 function. Remember that the /2 here means that the function takes two arguments. Those two arguments are:

1. The thing we want to enumerate over; and
2. a function to run for each of the items.

Let's take a look at an example:

iex> Enum.each(cities, &IO.puts/1)

The first argument here is our list of cities. The second argument is the function that we want to run for each item in this list: specifically the built-in IO.puts/1 function. The & before this function name is referred to as the "capture operator". We saw this back in Chapter 5, but we were using it to build our own functions then:

iex> greeting = &("Hello #{&1}")

The & has another use, and that is to capture functions from modules so that we can use them later. In this example, we're capturing IO.puts/1 so that Enum.each/2 can call that function on each of the items in the cities list.

Let's take a look at what this combination of Enum.each, cities and &IO.puts/1 does:

iex> Enum.each(cities, &IO.puts/1)
vienna
melbourne
osaka
calgary
sydney
:ok

This particular combination of those three things has made Elixir output each city on a single line in our iex prompt. Then there's one more thing: that little blue :ok at the end of our output. That's a little atom value and it indicates to us that the iterating with Enum.each/2 worked successfully. We saw atoms like this used for keys in maps in previous chapters, but they can also be used in Elixir code like this too.

Let's take a closer look at what this Enum.each/2 code is doing. Specifically: what &IO.puts/1 is doing. You know in Elixir that functions can take arguments and you might have in your head that those arguments could be strings, lists or maps because this is what we've seen done previously. But function arguments can also be other functions. This is what's happening here when we're using &IO.puts/1. We're passing that function as an argument to Enum.each/2.

To get an idea of how Enum.each/2 is able to take the IO.puts/1 function and do something with it, we can use this code in the iex prompt:

iex> puts = &IO.puts/1
puts.("melbourne")
"melbourne"
:ok

With this code, we've assigned the IO.puts/1 function to a variable called puts. This is similar to what's going on inside the Enum.each/2 function: whatever function we pass as the second argument will be stored with a particular name. Then Enum.each/2 goes through each of the items and calls the provided function on it, just like we've done above.

"That last line looks exactly like how we ran functions back in Chapter 5", Izzy exclaims. Yes, Izzy! That is exactly right. Back in that chapter we defined a greeting function like this:

iex> greeting = fn (place) -> "Hello, #{place}!" end
#Function<6.52032458/1 in :erl_eval.expr/5>

We then made Elixir run this function using this syntax:

iex> greeting.("World")
"Hello, World!"

What we're doing different here with our puts = &IO.puts/1 is that we're using an inbuilt function rather than constructing one of our own. We can do this with any inbuilt function we please. Here's another example, using String.capitalize/1:

iex> cap = &String.capitalize/1
cap.("melbourne")
"Melbourne"

Here, the String.capitalize/1 function takes a string and turns the first letter into a capital letter. In this code example, we're making it so that we can call the String.capitalize/1 as if it were a function called cap. However, if we wanted to capitalise the names of our city using the String.capitalize/1 function with Enum.each/2 it wouldn't do very much:

iex> Enum.each(cities, &String.capitalize/1)
:ok

Nothing is output here besides :ok because we're not telling Elixir to output anything. It will dutifully run the function String.capitalize/1 for each item in the list, and then just as dutifully "forget" to tell us anything about the result.

If we want to see what the result would be from evaluating String.capitalize/1 for each item in the list, we're going to have to use a different function.

Map, but not the kind that you know already

The names of these cities should be capitalized because they are proper nouns, but whoever created this list neglected to capitalize them. Oops! What we should have is a list with proper capitalization:

["Vienna", "Melbourne", "Osaka", "Calgary", "Sydney"]

We tried using Enum.each/2 to give us this list of capitalized cities, but that just told us :ok and it didn't give us back a list of capitalized city names.

We're going to need a function that goes through each item in the list and applies String.capitalize/1, and then shows us the result of each application of that function. Something that would turn our existing cities list into a list like this one:

iex> cities = [
    String.capitalize("vienna"),
    String.capitalize("melbourne"),
    String.capitalize("osaka"),
    String.capitalize("calgary"),
    String.capitalize("sydney"),
  ]
["Vienna", "Melbourne", "Osaka", "Calgary","Sydney"]

But since we've already got a list we're going to have to do something to that list to turn it into what we want. That something is to use another function from the Enum module called Enum.map/2. This map function is different from the map kind of data (%{name: "Izzy"}), in that the function-called-map will take the specified enumerable and another function as arguments, then enumerate through each item in the list and run that other function for each item, and then return the result of each function run.

Enough jibber-jabber. Let's see this in actual practice:

iex> Enum.map(cities, &String.capitalize/1)
["Vienna", "Melbourne", "Osaka", "Calgary","Sydney"]

Hooray! Each of our cities now has a correctly capitalized name. What's happened here is that we've told the map function that we want it to run another function — that'd be the String.capitalize/1 function — on each item of the cities list. This is how we've been able to go from our existing list with non-capitalized city names to a new list with capitalized names.

BYO functions

One more quick example. When using Enum.each/2 or Enum.map/2 we don't necessarily need to pass a built-in function as the second argument. We could choose to pass our own function instead. Let's say that we had a list of numbers like this:

iex> numbers = [4, 8, 15, 16, 23, 42]

And then we wanted to multiply each number by a different number, let's say 2. We're lazy and we want the computer to do the heavy lifting, and so we can pass Enum.map/2 a function of our own design to acheive this goal:

iex> Enum.map(numbers, fn (number) -> number * 2 end)
[8, 16, 30, 32, 46, 84]

With our own function here, we're taking each element of the list — represented inside the function as the variable number — and then multiplying it by 2. When Enum.map/2 has finished going through the list, it outputs a new list showing us the result of running that function on all items in the list.

Reducing an enumerable

We've now looked at how to use two functions from Enum, each/2 and map/2. These are two of the most commonly used functions from this module, and that's why we looked at them. But there's a third function that I'd like to tell you about too: Enum.reduce/2. It's a very helpful function!

Enum.reduce/2 allows us to iterate through a list and gradually apply a function to each element in that list to get a final value. Think of it like when you're cooking: you're reducing all the ingredients into a meal. You don't think of a spaghetti bolognese as it being a combination of onion, garlic, beef mince, tomatoes, herbs and spaghetti. You think of it as a complete meal. When we want to combine values in Elixir to produce one final value, we would use Enum.reduce/2. Let's take a look at it now.

We might want Elixir to tell us what the average of a list of numbers is. For instance, we might want to calculate the average of some scores, say for a test. Hey look, here are some handy score numbers now (out of 100):

iex> scores = [74, 79, 89, 32, 79, 70, 32, 69, 76, 73, 88, 73, 82, 31]

I wanted to keep writing numbers above but 14 numbers just felt right for some reason. It looks like most people in this test did particularly well. Good on them! A few didn't do so great and only scored 32 and one scored 31. Anyway, let's not look too closely at the data. What we're trying to do is to calculate the average for these scores and to do that we might try to sum these numbers up ourselves on a piece of paper

74 + 79 + 89 + 32 + 79 + 70 + 32 + 69 + 76 + 73 + 88 + 73 + 82 + 31

The calculator would then tell us that the sum of these numbers is 947, as our on-paper working out should've shown us too. Go ahead and check this in your iex console too if you'd like; after all, it's just a super-powered calculator.

Once we have the sum, then we need to divide that sum by the amount of numbers that we summed in order to get the average. Here, we have 14 numbers. So the average is 947 / 14, which a calculator or the iex prompt should both tell you: 67.64285714285714. The calculator may not have as many decimal places. If we wanted to write this whole operation in Elixir code, it would be very simple:

iex> (74 + 79 + 89 + 32 + 79 + 70 + 32 + 69 + 76 + 73 + 88 + 73 + 82 + 31) / 14

We sum up all the numbers, then divide it by how many scores we had. Easy! It's easy here because there are only 14 numbers. But what if we had scores from hundreds of people? We wouldn't want to enter all this into our calculator or the iex prompt, now would we? "No way!", contributes Izzy. Exactly! So let's look at how we could use Enum.reduce/2 to do the summing up for us, to save us having to work it out all ourselves.

iex> scores = [74, 79, 89, 32, 79, 70, 32, 69, 76, 73, 88, 73, 82, 31]
[74, 79, 89, 32, 79, 70, 32, 69, 76, 73, 88, 73, 82, 31]
iex> Enum.reduce(scores, fn (score, sum) -> sum + score end)

The reduce/2 function here does exactly the same calculations as our earlier prompt: it returns 947. It does this by taking the first number, then it adds the second, third, forth and so on to get the sum. How does it do this? Well, let's walk through what this function is doing.

First, we pass reduce/2 the list of scores that we want to sum up. Then we pass it another function which takes two arguments: score and sum. Inside this function, we add these two arguments together. But what values do score and sum have inside this function? One easy way to figure that out is to get Elixir to output their values every time this function inside reduce/2 is used:

iex> Enum.reduce(scores, fn (score, sum) ->
  IO.puts(sum)
  IO.puts(score)
  IO.puts("---")
  sum + score
end)

Even though the order of the arguments inside the function are score and sum, we're outputting them here as sum and score for reasons that will become clear soon. When we run this new variant of our reduce/2 function, this will be the output:

74
79
---
153
89
---
242
32
---
274
79
---
353
70
---
423
32
---
455
69
---
524
76
---
600
73
---
673
88
---
761
73
---
834
82
---
916
31
---
947

The first number that we're outputting is the sum and the second number is the score. If we look at the first four lines of our output we'll see three numbers: 74, 79 and 153.

74
79
---
153

If we look back at our scores list definition, we can see that its first two numbers are 74 and 79.

iex> scores = [74, 79, ...]

So then where does 153 come from? The answer is easy: 74 + 79 = 153. So we can tell from the output here that our reduce/2 function is taking the first item from our list and making that the initial sum argument for our inner function. The second item then becomes the score argument. In the function, we're adding these two values together, and that results in 153. The reduce/2 function keeps going through the remaining numbers in the list, adding them one-at-a-time until we get the final result.

So that's the wonderful reduce/2 function, or at least one example of it. As you can see, it's very handy for when we want to combine (or reduce) several items into one item.