Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Mastering Elixir
Mastering Elixir

Mastering Elixir: Build and scale concurrent, distributed, and fault-tolerant applications

Arrow left icon
Profile Icon Caixinha Profile Icon Albuquerque
Arrow right icon
NZ$39.99 NZ$57.99
Full star icon Full star icon Full star icon Full star icon Empty star icon 4 (2 Ratings)
eBook Jul 2018 574 pages 1st Edition
eBook
NZ$39.99 NZ$57.99
Paperback
NZ$71.99
Subscription
Free Trial
Arrow left icon
Profile Icon Caixinha Profile Icon Albuquerque
Arrow right icon
NZ$39.99 NZ$57.99
Full star icon Full star icon Full star icon Full star icon Empty star icon 4 (2 Ratings)
eBook Jul 2018 574 pages 1st Edition
eBook
NZ$39.99 NZ$57.99
Paperback
NZ$71.99
Subscription
Free Trial
eBook
NZ$39.99 NZ$57.99
Paperback
NZ$71.99
Subscription
Free Trial

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Table of content icon View table of contents Preview book icon Preview Book

Mastering Elixir

Preparing for the Journey Ahead

Welcome to this incredible journey! This is the beginning of an odyssey that will take you through the many features of Elixir, and how to use them to build, test, deploy, and maintain applications. This journey may require some shifting of your mindset and how you think about programming (and problem-solving in general) if you're not already familiar with functional programming.

Before diving into this book, we want to point out that this introductory chapter is deliberately short. Although we'll be introducing the language, its tooling, and its ecosystem, this won't be a complete reference guide. Elixir treats documentation as a first-class citizen, and this is shown in the incredible documentation of its modules and functions. Hence, we decided to teach you how to search for what you need, and target on the core concept of bookbuilding an Elixir application. We think that this pragmatic approach is the one that delivers the most value to you, as you'll be able to follow an application right from its inception and into its production, including its deployment and monitoring, which are sometimes overlooked.

In this chapter, we will cover the following topics:

  • Data types in Elixir
  • Working with pattern matching and how to apply it to various types
  • Working with functions and bundling them in modules
  • Working with collections
  • Using classic control-flow constructs (such as case)
  • Using typespecs on your functions
  • Creating behaviours, and adopting them in other modules
  • Using protocols to make our functions polymorphic
  • Some of the most useful tools that ship with Elixir
  • Calling Erlang libraries directly
  • Interacting with operating system processes through ports

Why functional programming?

Coming from an object-oriented background ourselves, we can't emphasize enough how valuable this shift is. Adopting the functional programming paradigm brings along some benefits, such as the following:

  • The data-transformation flow becomes more evident. This is in contrast with what usually happens in object-oriented programming, where objects strive to encapsulate the data; in functional programming, we mutate data using functions. This makes the transformations that are applied to the data explicit, which in turn makes the applications written this way easier to understand.
  • Functions are (mostly) side-effect-free. With immutable data, you can be sure that the value of a certain variable you hold a reference to will remain the same, since if another function or process wants to change it, it has to create a copy first—and operate on that copy. This makes code much easier to analyze and comprehend, since you can rest assured that your variables will remain as you expect. However, note that since Elixir isn't a pure functional language, your code can still have side effects. For instance, if a function writes to a file, it creates a side effect.
  • Programs can be parallelized easily. Coming out as an added bonus of immutability, it's usually very simple to parallelize this type of program, since there is no shared state.

One possible disadvantage of having immutable data is that you can incur a performance penalty, as each time you need to change something, you must make a copy of it. This greatly depends on the implementation, and while this concern is generally valid, Elixir employs clever techniques when compiling your code to minimize this effect. For instance, in certain conditions, Elixir can just point to existing variables when creating new ones, as it knows all variables are immutable and will never change.

If you're overwhelmed with all this functional programming jargon, lie down and relax, as we will explore these concepts in greater detail throughout this book.

Elixir and Erlang

Elixir, created by José Valim, runs on the Erlang VM (also known as BEAM). Erlang, developed at Ericsson more than 30 years ago, was created to improve the reliability and scalability of the company's telecom systems. Nowadays, it is used in a number of different settings, from database systems to chat applications. Erlang has fault-tolerance and distribution baked into its design, and is famous for running systems with nine nines of reliability.

Erlang's runtime is natively distributed, given that it was designed to be deployed on multiple telecom switches simultaneously. Programs running on the Erlang VM can take advantage of this by easily distributing an application across multiple nodes, but also across multiple CPUssince multiple cores is just a specific case of a distributed system. This is an incredible selling point of the Erlang VM (and thus of Elixir), since in today's setting CPUs are not getting much faster, and we're instead seeing CPUs with an increasing number of cores coming out.

In this opening chapter, we will be introducing Elixir. Beginning with its data types, we will also look at pattern matching, anonymous and named functions, modules, and some control-flow constructs. Then, we will see how to work with collections, and then we will briefly touch on behaviours and protocols. The chapter will end with an overview of the incredible tooling Elixir provides, along with some ways to exploit the existing interoperability between Elixir and Erlang.

Elixir's data types

We will now describe Elixir's data types, which extend upon Erlang's data types. Elixir is a dynamic programming language. Consequently, you don't declare the type of each variable—it depends on the value it holds at each moment.

To improve the learning experience, we'll be providing some examples along the way. For now, we'll just use Elixir's REPL, IEx (short for Interactive Elixir). To start an IEx session, you must have Elixir installed on your machine. Elixir has an official page with instructions on how to do this if you don't have it installed,  whether using package managers, the precompiled version, or compiling from the source yourself:

http://elixir-lang.github.io/install.html 

We will not dive into the memory usage of each type in Elixir. If you're curious about this, the official Erlang documentation contains detailed information: http://erlang.org/doc/efficiency_guide/advanced.html.

Provided that you have Elixir already installed on your machine, type iex on your terminal to start a new IEx session. With this, you can run the examples present in this chapter in your machine. Note that your default iex  prompt contains a number in between parenthesis, which represents the number of expressions you've entered in the current session, such as iex>(1). To declutter the output, in our examples, we've removed this number.

We'll be exploring IEx in greater detail toward the end of this chapter, in the Tooling and ecosystems section. Throughout the following subsections, we'll be mentioning some built-in modules in Elixir. We'll explore what modules are in the Functions and modules section—for now, it's enough to know that a module is a collection of functions.

Whenever you're done with your IEx session, you can either press Ctrl C twice or Ctrl \. This will kill the operating system process that's running the Erlang runtime (along with all background jobs). Alternatively, you can stop the system in a more polite way by entering System.halt in the shell.

Integers

This type contains, as you would expect, numbers that can be written without a fractional component. The size of integers adjusts dynamically according to its magnitude—you don't have to worry about this: an integer will simply occupy more words in memory as it grows. Here's some basic arithmetic with integers:

iex> 25 + 8
33

To improve the readability of the code, you can also use underscores in between the digits of an integer, as shown here:

iex> 1_000_000 - 500_000
500000

Besides decimal, Elixir also supports integers written in binary, octal, and hexadecimal (using 0b, 0o, and 0x, respectively):

iex> 0b10001
17
iex> 0o21
17
iex> 0x11
17

Floats

In Elixir, floats are written with a decimal point, with digits before and after it, meaning that .1 is not a valid float in Elixir (as it is, for instance, in JavaScript). In Elixir, you have to be explicit and write the leading 0—so in this case, you'd write 0.1. Here's an example of the multiplication of two floats:

iex> 0.1 * 0.5
0.05

You can also write floats using the exponent notation, as shown:

iex> 0.1e3
100.0

Floats are represented in IEEE 754 double precision, which yields between 15 to 17 significant decimal digits. As usual, you should take care when comparing floats for equality.

Beware that the division operator (/) always returns a float, even if the result of the division could be an integer:

iex> 4/2
2.0

If you want to circumvent this behavior, use the auto-imported div function from the Kernel module. Also, if you want to get the remainder of a division, use the rem function.

Atoms

Atoms are a constant, whose value is its own name. They are always prefixed with a leading colon (:), followed by alphanumeric characters (and possibly _ or @). They may terminate with an exclamation or a question mark. Atoms are similar to enumerations in C and symbols in Ruby. Here are some examples of atoms:

iex> :ok
:ok
iex> :error
:error
iex> :some_descriptive_name!
:some_descriptive_name!
iex> :value@start
:value@start

You can create atoms with arbitrary characters with the following syntax:

iex> :"Atom name with arbitrary characters#$%^"
:"Atom name with arbitrary characters#$%^"

As with all data structures in Elixir, atoms can't be modified after their creation. Furthermore, they are not garbage-collected. Atoms are kept in the atom table, and upon compilation, their value is replaced by a reference to their entry on this table. This makes comparing atoms very efficient. As you'll learn throughout this book, this is one of the major use cases for atoms in Elixir, as we are constantly matching the return of a function against a certain expected atom.

Since atoms are not garbage collected, don't create atoms dynamically from sources you can't control, as you can very easily use up all of the space allocated for the atom table. For instance, if you're parsing a JSON response and creating a map out of it, don't use atoms for its keys—use strings instead (both of these types, maps and strings, will be described later in this chapter).

Booleans

Elixir has three values related to Boolean operations: true, false, and nil (where nil represents the absence of value—similar to null in most other languages). However, those are just some syntatic sugar, as internally they are represented as atoms of the same name, as you can see in the following example:

iex> true == :true
true
iex> false == :false
true
iex> nil == :nil
true

You have the common Boolean operators, orand, and not:

iex> true or false
true
iex> true and false
false
iex> not false
true

However, these operators are type-strict in their first argument: they only accept true or false. If you pass anything else as an argument, you'll get BadBooleanError.

This is where the concept of truthiness and falseness enters. Similar to what happens in Ruby or C, false and nil are treated as falsey values, and everything else is considered to be truthy. The operators that work with falsey and truthy values are && (and), || (or), and ! (not):

iex> "a value" || false
"a value"
iex> "a value" && false
false
iex> nil && "a value"
nil
iex> !"a value"
false

Notice how these operators short circuit depending on the arguments. With ||, it returns the first value that's truthy, whereas with &&, it returns the first falsey value (in both cases, in the event those conditions never happen, they return the last value).

You also have the other normal comparison operators, such as greater than (>) and inequality (!=)—you can find the full list at https://hexdocs.pm/elixir/operators.html. The one that's worth pointing out is the strict equality operator, which, besides comparing values, compares types:

iex> 3 == 3.0
true
iex> 3 === 3.0
false

Tuples

Tuples are used to group a fixed number of elements together. They can hold any value—even other tuples. They are stored contiguously in memory, which provides constant access time to elements inside a tuple. You create a tuple surrounding the elements with curly brackes ({ and }), and separate the elements with commas:

iex> {:ok, 3.14}
{:ok, 3.14}

A common usage of tuples in Elixir is to pattern-match on the result of a function to ensure its success (usually with an :ok atom) or deal with an error. We will be looking to pattern matching and functions later in this chapter.

To access an element inside a tuple, we use the elem function (from the Kernel module), providing the tuple and a zero-based index:

iex> result = {:ok, 3.14}
{:ok, 3.14}
iex> elem(result, 1)
3.14
Functions from the Kernel module are auto-imported. Thus, we don't need to prefix them with the module name.

To change the elements on a tuple, you can use the put_elem function. The arguments are similar to the elem function, but you also provide the new value for that position of the tuple:

iex> put_elem(result, 1, 1.61)
{:ok, 1.61}
iex> result
{:ok, 3.14}

Notice how the result variable hasn't changed. As we discussed in the beginning of this chapter, data in Elixir is immutable. As such, although we've updated the tuple with a new value, the original tuple hasn't changed—Elixir updated the value on a copy of the original tuple. This way our code is side-effect free, and any other function holding a reference to the result variable won't have any surprises.

The general recommendation in Elixir is that tuples should hold up to four elements—anything more than that and you probably should be using another type.

Lists

Lists are created by wrapping the elements we want inside it with square brackets ([ and ]), separating the values with commas. Internally, lists are implemented as singly linked lists, meaning that accessing the elements of a list is a O(n) operation. Lists aren't stored contiguously in memory as arrays in other languages. As with tuples, list elements can be of any type:

iex> [1, :an_atom, 0.5]
[1, :an_atom, 0.5]

We have the ++ and -- operators that are exclusive to lists, and serve to concatenate and subtract lists, respectively:

iex> [0, 1, 1] ++ [2, 3, 5]
[0, 1, 1, 2, 3, 5]
iex> [0, 1, 1] -- [1, 2, 3]
[0, 1]

To check whether a certain element is present in a list, you can use the in operator:

iex> 1 in [0, 1, 1, 2, 3, 5]
true
iex> 99 in [0, 1, 1, 2, 3, 5]
false

To get the head of a list, we use the hd function, whereas to get the tail of a list, we use the tl function:

iex> hd([0, 1, 1, 2, 3, 5])
0
iex> tl([0, 1, 1, 2, 3, 5])
[1, 1, 2, 3, 5]

Notice that the semantic of tail here is the list without its head (which is also a list), and not the last element of a list. We'll be exploring this concept in more depth, along with some more examples on how to work with lists, in the Working with collections section. For reference, you can find a detailed list of operations you can make on lists at https://hexdocs.pm/elixir/List.html.

Appending to a list is a O(n) operation, as we need to traverse the whole list. Prepending to a list is O(1). To prepend an element to a list, you can use the following syntax: [new_element | list].

Maps

Maps are key-value data structures, where both the key and the value can be of any type. They're similar to hashes in Ruby and dictionaries in Python. To create a map, you enclose your key-value pairs in %{}, and put a => between the key and the value, as we can see in the following snippet:

iex> %{:name => "Gabriel", :age => 1}
%{age: 1, name: "Gabriel"}

In this case, the keys are both of the same type, but this isn't required. If your keys are atoms, you can use the following syntax to make the map declaration simpler:

iex> %{name: "Gabriel", age: 1}
%{age: 1, name: "Gabriel"}

To access the value associated with a certain key, put the key inside square brackets in front of the map:

iex> map = %{name: "Gabriel", age: 1}
%{age: 1, name: "Gabriel"}
iex> map[:name]
"Gabriel"

As with the map declaration, when the key is an atom, we have some syntatic sugar on top of it:

iex> map.name
"Gabriel"
When you try to fetch a key that doesn't exist in the map, a KeyError error will be raised when using the map.key syntaxunlike the map[key] syntax, which will return nil.

To update a key in a map, you can use %{map | key => new_value}. If the key is an atom, we can use the same notation described previously:

iex> %{map | age: 2}
%{age: 2, name: "Gabriel"}
If you're coming from an object-oriented programming background, you may instinctively use the following syntax to change the value of a key: map[key] = new_value. Remember that in Elixir all types are immutable and you never operate on the data structure itself but always on a copy of it.

This will only work for keys that already exist in the map—this constraint allows Elixir to optimize and reuse the fields list when updating a map. If you want to insert a new key, use the put function from the Map module:

iex> Map.put(map, :gender, "Male")
%{age: 1, gender: "Male", name: "Gabriel"}

As with all other types, in the official documentation, at https://hexdocs.pm/elixir/Map.html, you can find a pretty detailed reference on what you can do with maps.

Binaries

A binary is group of consecutive bytes. You create them by surrounding the byte sequence with << and >>. Here we are creating a two-byte binary:

iex> <<5, 10>>
<<5, 10>>

In the decimal base, a byte can only contain values up to 255 (otherwise it overflows). If we want to store values greater that 255, we need to tell the runtime to use more space to store this binary:

iex> <<5, 256>>
<<5, 0>>
iex> <<5, 256::16>>
<<5, 1, 0>>

As you can see, when we specify the size (16 bits in this case) we can see that the output as an extra byte and the overflow didn't occur. The size doesn't have to be a multiple of 8. In that case, a binary is usually called a bitstring.

Most programmers will not handle data at such a low level, so your use of binaries may not be that frequent. However, they're extremely useful in certain scenarios, such as processing the header of a file to find a magic number and identify the file type, or even when dealing with network packets by hand.

Strings and charlists

Strings are binaries with UTF-8 codepoints in them. You create a string with the usual double-quote syntax:

iex> "hey, a string"
"hey, a string"

Charlists are, as the name implies, lists of character codes. You create them using the single-quote syntax:

iex> 'hey, a charlist'
'hey, a charlist'

Since this is just a list, you can use the hd function to get the code for the first character:

iex> hd('hey, a charlist')
104
You can find out the code of a certain character with the ? operator. For instance, to find out the character code of a lowercase d, you'd use ?d.

Both representations support string interpolation:

iex> "two plus two is: #{2+2}"
"two plus two is: 4"
iex> 'four minus one is: #{4-1}'
'four minus one is: 3'

Both representations also support the heredoc notation, which is most commonly used to write documentation. To create it, use three single or double quotes:

iex> """
...> a string with heredoc notation
...> """
"a string with heredoc notation\n"
iex> '''
...> a charlist with heredoc notation
...> '''
'a charlist with heredoc notation\n'
The closing delimiter of a heredoc string/charlist must be on its own line.

Elixir provides sigils as another syntax to declare strings or charlists, which can be handy if you want to include quotes inside your string. You can use ~s to create a string and ~c to create a charlist (their uppercase versions, ~S and ~C, are similar but don't interpolate or escape characters):

iex> ~s(a string created by a sigil)
"a string created by a sigil"
iex> ~c(a charlist created by a sigil)
'a charlist created by a sigil'

There's another sigil that's worth mentioning: ~r, which is used for regular expressions. In the next snippet, we're using the run function from the Regex module to exemplify the usage of the ~r sigil:

iex> Regex.run(~r{str}, "a string")
["str"]
iex> Regex.run(~r{123}, "a string")
nil

You can find the list of supported sigils (and also how to create your own!) at http://elixir-lang.github.io/getting-started/sigils.html.

The convention in the Elixir community is to only use the term string when referring to the double-quote format. This distinction is important, since their implementation is very different. Functions from the String module will only work on the double-quote format. You should always use the double-quote format, unless you're required to use a charlist—which is the case, for instance, when you're using Erlang libraries. You can use the following functions to convert between the two formats:

iex> String.to_charlist("converting to charlist")
'converting to charlist'
iex> List.to_string('converting to string')
"converting to string"

Other types

We'll now succinctly describe some other types. We'll begin with the types that build upon some types we've already described: keyword lists, ranges, mapsets, and IO lists.

Keyword lists

A keyword list is a list in which its elements have a specific format: they are tuples where the first element is an atom (the second element can be of any type), as demonstrated in the following example:

iex> [name: "Gabriel", age: 1] = [{:name, "Gabriel"}, {:age, 1}]
[name: "Gabriel", age: 1]

We can create keyword lists using the following syntax:

iex> keyword_list = [name: "Gabriel", age: 1]
[name: "Gabriel", age: 1]
iex> keyword_list[:name]
"Gabriel"

As you can see from the previous snippet, a keyword list is indeed a list of tuples, with an atom; you can access values in a keyword list using the same syntax as you would in maps. As an alternative, you can use the get function from the Keyword module. Note that this way of declaring a keyword list is just syntatic sugar, as internally this still is a list of tupleswhich means that searching for an item in a keyword list is O(n), and not O(1) as in maps.

In a keyword list, contrary to what happens in maps, you can have more than one value for a given key. Also, you can control the order of its elements. Usually, keyword lists are used to allow functions to receive an arbitrary number of optional arguments. We'll be showing an example of this when we look at named functions later in this chapter. You can find all the operations you can do on a keyword list at https://hexdocs.pm/elixir/Keyword.html.

Ranges

Ranges, again, similar to what happens in Ruby, represent an interval between two integers. To create a range, we use this:

iex> 17..21
17..21
iex> 19 in 17..21
true

Similar to what we do with a list, we can use the in operator to check whether a number is between the start and the end of a range.

MapSets

If you're looking for an implementation of a set in Elixir, you're looking for MapSet. You create and manipulate them with the functions from the MapSet module. Here are some examples:

iex> set = MapSet.new
#MapSet<[]>
iex> set = MapSet.put(set, 1)
#MapSet<[1]>
iex> set = MapSet.put(set, 2)
#MapSet<[1, 2]>
iex> set = MapSet.put(set, 1)
#MapSet<[1, 2]>

Sets, by definition, can't contain duplicates. So, inserting a value that's already there has no effect. You can find the documentation for the MapSet module at https://hexdocs.pm/elixir/MapSet.html.

There are three types, related to the underlying Erlang VM, that we have to mention before closing this section. They are as following:

  • Reference: A reference is a type created by the Kernel.make_ref function. This functions creates an almost-unique reference, which gets repeated around every 282 calls. We will not use references in this book.
  • Port: A port is a reference to a resource. The Erlang VM uses it to interact with external resources, such as an operating system process. We will talk a bit more about ports later in this chapter, when we discuss the interoperability between Elixir and Erlang.
  • PID: A PID is the type used to identify processes in the Erlang VM. You'll see PIDs in action later in this book, when we start working with Erlang VM processes.

To complete this section, there's a function that we want to highlight: the i function, which is auto-imported from the Kernel module. You can use it to find out more information about a data type. It will print information about the data type of the term you pass as an argument. Here is an example with a string:

iex> i("a string")
Term
"a string"
Data type
BitString
Byte size
8
Description
This is a string: a UTF-8 encoded binary. It's printed surrounded by
"double quotes" because all UTF-8 encoded codepoints in it are printable.
Raw representation
<<97, 32, 115, 116, 114, 105, 110, 103>>
Reference modules
String, :binary
Implemented protocols
IEx.Info, Collectable, List.Chars, String.Chars, Inspect

And, with this, we've finished our tour of the data types in Elixir! We didn't go into much detail, but with the links we left throughout this section, you'll see how incredible the documentation in Elixir is, and how easy it is to figure out what a certain function does or the purpose of a certain argument.

We will now jump into one of the most prominent features of Elixir (and also one that will definitely change how you write programs): pattern matching.

Pattern matching

When we were describing Elixir's data types, we used the = operator to bind a value of a certain type to a variable. We didn't stop there to explain what was actually going on, as the syntax is very similar to most dynamic programming languages. In fact, it is so similar that, at first glance, we assume it works the same way.

If you don't have a functional programming background, your first instinct would be to call the = operator, the assignment operator. However, in Elixir, it is called the match operator. Let's see the reason for this difference in nomenclature with some examples:

iex> x = 3
3
iex> x * 3
9
iex> 3 = x
3
iex> 2 = x
** (MatchError) no match of right hand side value: 3
We've seen our first exception being raised on our IEx session. We'll discuss them later in this chapter, when we talk about control flow.

The first two statements are analogous to what you'd see with an assignment operator—we just set the x variable to 3, and then multiply that variable by 3, giving us the expected result of 9. Now, notice what happens on the following lines. We have an integer literal on the left-hand side of the = operator, and that is a valid expression, returning a value of 3. You're seeing the match operator in action.

Similar to what you do with equations in algebra, this operator tries to match the pattern on the left-hand side to the term on the right-hand side. On the first line of the preceding snippet, this means matching the x variable on the left to the 3 term on the right.

Elixir can make this match succeed by binding the x variable to the 3 term. On the third line, we're again matching the left and right sides, and it succeeds because x has the 3 value, so both sides are equal. On the next line, we're trying to match the 2 literal to the x variable. Elixir can't find a way to make this match work, which results in an error being raised. It's important to point out that binding to variables only happens when they are on the left-hand side of the = operator—when they're at the right-hand side, the variable is simply replaced by its value.

From this snippet, we can also notice that we always have a return value—even when doing a pattern match. This is the expected behavior, since everything in Elixir is an expressionthere are no statements. Every operation you can do will always return a value, whether you're printing something to the console or making an HTTP request. While you can achieve the same things with expressions and statements, always having a return value is very useful because you can chain functions together and define the program flow according to the values being returned. In the case of pattern matching, when it is successful, we always get back the term that was matched on the right-hand side.

The match operator is not confined to bind variables to simple valuesit's actually a very powerful operator that is able to destructure complex data types (and make your code ridiculously simple to read). We will now show how you can use this operator on several of the types we've presented in the previous section, while also demonstrating other aspects of the pattern matching process.

Pattern matching on tuples

The following snippet shows how pattern matching can be done on tuples:

iex> {number, representation} = {3.1415, "π"}
{3.1415, "π"}
iex> number
3.1415
iex> representation
"π"

The process here is the same as we have described in the preceding snippet. By setting the {number, description} pattern on the left-hand side, we're stating that we expect a tuple with two values—again, if that's not the case, a MatchError will be raised. In this case, the match succeeds, and we can see that the variables number and representation are bound to the expected values.

Unlike Erlang, Elixir allows you to rebind a variable, which is why the following works:

iex> a = 1
1
iex> a = 7
7

However, a variable can only bind once per match:

iex> {a, a} = {3, 3}
{3, 3}
iex> {a, a} = {2, 3}
** (MatchError) no match of right hand side value: {2, 3}

On the first line, the match succeeds because each a is binding to the same value, 3. On the second line, we get a MatchError because we're binding a to two different values on the same match. Later in this chapter, we'll see how we can make Elixir behave like Erlang in this regard, by using the pin operator.

We can set our expectations even further, using literals on the left-hand side:

iex> {3.1415, representation} = {3.1415, "π"}
{3.1415, "π"}
iex> representation
"π"

Now our expectation is a tuple with two elements, where the first one is the 3.1415 float literal. We can use this on other Elixir types as well, such as lists or maps. This technique becomes even more fruitful when we apply it to functions, as we will see in the next section.

Pattern matching on lists

Matching on lists is akin to matching on tuples. Here's a simple example:

iex> [first, second, third] = ["α", "β", "γ"]
["α", "β", "γ"]
iex> first
"α"
iex> second
"β"
iex> third
"γ"

There's nothing new here. What if, for instance, we don't care about the second element of the list? That's where the  _ (underscore) anonymous variable is convenient:

iex> [first, _, third] = ["δ", "ε", "ζ"]
["δ", "ε", "ζ"]
iex> first
"δ"
iex> third
"ζ"

We're again matching a list with three elements, but now bind the second element to the _ variable, which means that we accept anything in that position and we won't use its value.

 

 


The _ variable can never be read fromif you do so, you will get a CompileError:

iex> _
** (CompileError) iex:83: unbound variable _

This way, Elixir is protecting you from inadvertently reading from this variable, which could easily cause unexpected behaviors in your application.

As we've mentioned on the data types section, you can use the hd and tl functions from the Kernel module to get the head and the tail of a list. You can do the same with pattern matching:

iex> [first | rest_of_list] = ["α", "β", "γ"]
["α", "β", "γ"]
iex> first
"α"
iex> rest_of_list
["β", "γ"]

While in this contrived example, this approach yields no benefit, this technique is a fundamental piece to operate on a list using recursion. We'll look at this in greater detail in the Working with collections section.

Pattern matching on maps

To use pattern matching on a map, we set our pattern with the key-value pairs we want to match on, as you can see in the following example:

iex> %{"name" => name, "age" => age} = %{"name" => "Gabriel", "age" => 1}
%{"age" => 1, "name" => "Gabriel"}
iex> name
"Gabriel"
iex> age
1

Note that in this case we're matching on all keys of the map, but this isn't necessarywe could just match on age, for instance. However, your pattern may only contain keys that exist on the map that's being matched on, otherwise MatchError will be raised.

Sometimes, you may want to match on the value of a variable, instead of rebinding it to a new value. To this end, you can use the pin operator, represented by the ^ character:

iex> name = "Gabriel"
"Gabriel"
iex> %{name: ^name, age: age} = %{name: "Gabriel", age: 1}
%{age: 1, name: "Gabriel"}
iex> %{name: ^name, age: age} = %{name: "Jose", age: 1}
** (MatchError) no match of right hand side value: %{age: 1, name: "Jose"}

As we can see in the preceding snippet, we have the name variable bound to "Gabriel". We then match a map as we did previously in this section, this time using the contents of the name variable. This is equivalent to using the "Gabriel" literal on the left-hand side. When we're trying to match against a map that has a value different than that of the pinned variable, we get a MatchError, as expected.

When working with the pin operator, the variable you're using must already be bound, as it will not bind the variable in case it doesn't exist. If you use the pin operator on a non-existent variable, you'll get a CompileError stating that the variable you're trying to use is unbound.

Pattern matching on binaries and strings

The following example shows how you can use pattern matching on binaries:

iex> <<first_byte, second_byte>> = <<100, 200>>
<<100, 200>>
iex> first_byte
100
iex> second_byte
200

As previously stated when describing binaries, this can be incredibly helpful when you're dealing with bytes directly, such as parsing a packet from a given network protocol. By applying pattern matching to binaries, you can extract bits or bytes as necessary, while your code remains extremely expressive.

Since strings are just binaries underneath, we can use the same strategy as we did in the preceding snippet:

iex> <<first_byte, second_byte>> = "YZ"
"YZ"
iex> first_byte
89
iex> second_byte
90

However, this isn't very helpful when dealing with strings, as you're getting the decimal code of the characters in UTF-8 (also, as UTF-8 is a variable width encoding, a code point may take more than one byte). To match on strings, the best approach is to use the functions from the String module, such as starts_with?, ends_with?, or contains?.

As we've explained in the beginning of this section, everything in Elixir is an expression, and in pattern matching, when the match succeeds, the right-hand side of the expression is returned. Due to this behavior and taking into account that Elixir rebinds the variables on the left-hand side, we can write expressions such as the following one, binding multiple variables to the same value:

iex> x = y = 100
100
iex> x
100
iex> y
100

Functions and Modules

Despite not being mentioned in the data types section, functions in Elixir are a type as wellin fact, they are a first-class citizen, as they can be assigned to a variable and passed as arguments to other functions.

As with most functional programming languages, functions are an important type, hence they justify having their own section, away from other built-in types.

We will start by exploring anonymous functions, followed by an explanation of modules and named functions, and then we'll end this section with a quick tour of module attributes and directives.

Anonymous functions

Anonymous functions, usually called lambdas, are created with the fn keyword, as we can see in the following example:

iex> plus_one = fn (x) -> x + 1 end
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> plus_one.(10)
11

Here, we are defining a function that takes one argument, which we've named x, and simply adds one to the provided argument. We then bind this anonymous function to a variable named plus_one, and execute it with 10 as the argument, using the syntax we can see in the preceding snippet. As expected, we get 11 back.

There is no return keyword in Elixirthe return value of a function is the value returned by its last expression.

An anonymous function can also have multiple implementations, depending on the value and/or type of the arguments provided. Let's see this in action with an example:

iex> division = fn
...> (_dividend, 0) -> :infinity
...> (dividend, divisor) -> dividend / divisor
...> end
#Function<12.99386804/2 in :erl_eval.expr/5>
iex> division.(10, 2)
5.0
iex> division.(10, 0)
:infinity

Imagine that we want a special division function, that, instead of raising ArithmeticError when dividing by 0, would just return the :infinity atom. This is what the anonymous function we see here achieves. Using pattern matching, we say that when the second argument (the divisor) is 0, we simply return :infinity. Otherwise, we just use the / arithmetic operator to perform a normal division.

Aside from the lambda with multiple bodies, notice that we prefix the unused variable with an underscore (_), as in _dividend. Besides increasing the readability of your code, following this practice will make the Elixir compiler warn you when you use a supposedly unused variable. Conversely, if you don't use a certain variable but don't prefix it with an underscore, the compiler will also warn you.

Parentheses around arguments of an anonymous function are optional. You could write the plus_one function we introduced earlier as fn x -> x + 1 end.

Beyond accepting arguments, anonymous functions can also access variables from the outer scope:

iex> x = 3
3
iex> some_fun = fn -> "variable x is #{x}" end
#Function<20.99386804/0 in :erl_eval.expr/5>
iex> some_fun.()
"variable x is 3"
iex> x = 5
5
iex> some_fun.()
"variable x is 3"

As you can see, our anonymous function can access variables from the outer scope. Furthermore, the variable can be bound to another value, and our function will still hold a reference to the value that the variable had when the anonymous function was defined. This is usually called a closure: the function captures the memory locations of all variables used within it. Since every type in Elixir is immutable, that value residing on each memory reference will not change. However, this also means that these memory locations can't be immediately garbage-collected, as the lambda is still holding references to them.

We'll end this section on anonymous functions by introducing a new operatorthe capture operator (represented by &).

This operator allows you to define lambdas in a more compact way:

iex> plus_one = &(&1 + 1)
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> plus_one.(10)
11

This syntax is equivalent to the one presented before for the plus_one function. &1 represents the first argument of this lambda function—and, more generally, &n will represent the nth argument of the function. Similar to what happens in the fn notation, the parentheses are optional. However, it's better to use them, as in a real-world application, these lambda functions become hard to read without them.

Besides providing a shorter way to define lambda functions, the capture operator can also be used with named functions. We'll explore this further in the next section.

Modules and Named Functions

In Elixir, modules group functions together, much like a namespace. Usually, functions that reside in the same module are related to one another. You create a module using the defmodule construct:

iex> defmodule StringHelper do
...> def palindrome?(term) do
...> String.reverse(term) == term
...> end
...> end
{:module, StringHelper,
<<70, 79, 82, 49, 0, 0, 4, 0, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 119, 0,
0, 0, 11, 19, 69, 108, 105, 120, 105, 114, 46, 83, 116, 114, 105, 110, 103,
72, 101, 108, 112, 101, 114, 8, 95, 95, ...>>, {:palindrome?, 1}}
iex> StringHelper.palindrome?("abcd")
false
iex> StringHelper.palindrome?("abba")
true

In the preceding example we're also creating a function inside the StringHelper module, using the def construct, that checks whether a given string is a palindrome. This is a named function, and contrary to the anonymous functions, must be created inside a module.

Function names, like variable names, start with a lowercase letter, and if they contain more than one word, they are separated by underscore(s). They may end in ! and ?. The convention in the Elixir community is that function names ending in ! denote that the function may raise an error, whereas function names ending in ? indicate that that function either returns true or falsewhich is the case of our palindrome? function.

Note that unlike anonymous functions, we don't need to put a dot between the function name and the parenthesis when calling named functions. This is deliberate and serves to explicitly differentiate calls to anonymous and named functions.

As the implementation of our palindrome? function is very small, we can inline it with the following syntax:

def palindrome?(term), do: String.reverse(term) == term

This works with other constructs that use the do ... end syntax, such as defmodule or if. We will explore if (and other classical control flow mechanisms) in the Control-flow section.

Before we go any further, as our examples are getting bigger, we must discuss how you can write Elixir code in files. As you can see in the previous example, you can define modules on an IEx sessionhowever, any typo while writing them results in having to start from the beginning.

Put the contents of the last example in a filelet's call it "string_helper.ex" (we usually name the file with the name of the module we're defining in it). Elixir source code files may have two extensions: .ex or .exs. The difference between them is that the former is compiled to disk (creating .beam files), while the latter is compiled only in memory. We mostly use the .ex extension when working on a real application, except for the test files that use the .exs extension (as there's no point in compiling them to disk).

Having your file created, you can use the elixirc command in your terminal to compile it, passing the name of the file whose contents you want compiled. More interestingly, you can pass the filename to the iex command (iex string_helper.ex in our case). This will make Elixir compile your file, which will make our StringHelper module (and its functions) available in the IEx session. If you're already inside the IEx session and want to compile a new file, you can use the c command, passing the filename as a string:

iex> c("examples/string_helper.ex")
[StringHelper]

You can also nest modules:

$ cat examples/nesting_modules.ex
defmodule Helpers do
defmodule StringHelper do
# StringHelper code goes here
end
end
In the preceding example, the line starting with # is commented. That's the syntax to comment lines in Elixir. There's no syntax for multi-line commentsif you want to comment a block of code, prepend each line of that block with #.

However, during compilation, Elixir will prepend the outer module name to the inner module name, and separate them with a dot. This is just an amenity, as there is no relationship between these two modules. This syntax is equivalent to the following one, which is used much more in Elixir applications:

$ cat examples/nesting_modules_inline.ex
defmodule Helpers.StringHelper do
# StringHelper code goes here
end

We'll now explain the concept of arity, with our palindrome? function as an example. Named functions in Elixir are identified by their module name, the function's own name, and their arity. The arity of a function is the number of arguments it receives. Taking this into account, our palindrome? function is identified as Helpers.StringHelper.palindrome?/1, where /1 represents the arity of the function. You'll be seeing this notation a lot when browsing through Elixir documentation.

This concept is important because functions with the same name but different arities are, in effect, two different functions. However, for a human, it wouldn't make much sense that two functions with the same name (but different arities) are unrelated. As such, only use the same name for different functions when they are related to one another.

The common pattern in Elixir is to have lower-arity functions being implemented as calls to functions of the same name but with a higher arity. Let's extend our module with an emphasize function:

$ cat examples/string_helper_emphasize.ex
defmodule StringHelper do
def palindrome?(term) do
String.reverse(term) == term
end

def emphasize(phrase) do
emphasize(phrase, 3)
end

def emphasize(phrase, number_of_marks) do
upcased_phrase = String.upcase(phrase)
exclamation_marks = String.duplicate("!", number_of_marks)
"#{upcased_phrase}#{exclamation_marks}"
end
end

Here, we can observe it in action:

iex> StringHelper.emphasize("wow")
"WOW!!!"
iex> StringHelper.emphasize("wow", 1)
"WOW!"
We've used the def construct to create functions. By using it, our functions are exported and can be called in other modules. If you want to change this behavior, and make a function only available within the module where it's defined, use the defp construct.

The function with an arity of 1 is implemented by simply calling emphasize/2. This is useful when you want to offer a broad interface on your module, which allows you to have some clients that simply want to call emphasize/1 and not have to specify the number of exclamation marks, but also have some other clients that want to call emphasize/2 and specify the number of exclamation marks.

When the code is as simple as in this example, this multitude of functions is not necessary, as you can achieve the same end result using default arguments. We do that by using the \\ operator in front of the argument name, and then the default value it should have:

$ cat examples/string_helper_emphasize_with_default_args.ex
def emphasize(phrase, number_of_marks \\ 3) do
upcased_phrase = String.upcase(phrase)
exclamation_marks = String.duplicate("!", number_of_marks)
"#{upcased_phrase}#{exclamation_marks}"
end

This will generate two functions with the same name and different arities, as in the last snippet. If your function has multiple bodies, as in the next example, you must define a function header with the default arguments defined there:

$ cat examples/string_helper_emphasize_with_function_header.ex
def emphasize(phrase, number_of_marks \\ 3)
def emphasize(_phrase, 0) do
"This isn't the module you're looking for"
end
def emphasize(phrase, number_of_marks) do
upcased_phrase = String.upcase(phrase)
exclamation_marks = String.duplicate("!", number_of_marks)
"#{upcased_phrase}#{exclamation_marks}"
end

In this example, we're also seeing an example of how we can use pattern matching in named functions. Note that the order in which we define our functions matters. Elixir will search from top to bottom for a clause that matches. If we had put the clause where we're matching against 0 on the second argument at the end, that definition of the emphasize function would become unreachable, as the other definition is more general and always matches. Elixir will help you avoid these situations, as it will emit a warning during compilation, alerting you of this situation.

Apart from using pattern matching (as we've seen in this example and on anonymous functions), on named functions we can use guard clauses, which extend on the pattern matching mechanism and allow us to set broader expectations on our functions. To use a guard clause on a function, we use the when clause after the list of arguments.

To see an example of this, we will use a guard clause on our palindrome? function. Up to this point, we were accepting an argument of any type. If we passed an integer to this function, an error would be raised, as we would be trying to call String.reverse on an integer. Let's change that:

$ cat examples/string_helper_palindrome_with_guard_clause.ex
def palindrome?(term) when is_bitstring(term) do
String.reverse(term) == term
end
def palindrome?(_term), do: {:error, :unsupported_type}

We now state that we're expecting bitstring as an argument. We've also created a new definition of our function, which runs when the match doesn't occur on the first definition. Here it is in action:

iex> StringHelper.palindrome?("abba")
true
iex> StringHelper.palindrome?(123)
{:error, :unsupported_type}

Using guard clauses in our functions can lead to a lot of duplication, since we may be repeating the same clause over and over again. To combat this, Elixir 1.6 introduced the defguard construct, which allows us to define clauses that can be reused.

Moreover, using this construct may improve the readability of your code, since we can extract complex guard clauses and give them descriptive names. Let's see the previous example implemented using defguard:

$ cat examples/string_helper_palindrome_with_defguard.ex
defguard is_string(term) when is_bitstring(term)

def palindrome?(term) when is_string(term) do
String.reverse(term) == term
end
def palindrome?(_term), do: {:error, :unsupported_type}

In this simple example, there's no clear advantage to using this construct. However, as your modules, along with your guard clauses, grow more complex, this technique becomes incredibly useful. Note that you can use defguardp to define a guard clause that is not exported, and can only be used within the module where it's defined.

You can use other type-checking functions in guard clauses, as well as comparison operators, and also some other functions. You can find the full list at https://hexdocs.pm/elixir/guards.html.

To end this section, we will now showcase one of the most eminent features of the language: the pipe (|>) operator. This operator allows you to chain function calls, making the flow of your functions easy to read and comprehend. This operator takes the term that's at its left, and injects it as the first argument on the function at its right. This seemingly insipid feature increases the readability of your code, which is amazing since code is read many more times than it is written. To see this operator in action, let's add some more logic to our palindrome? function: We will now remove leading or trailing whitespaces from the term we're checking, and we'll also make our comparisons case-insensitive. This is the result:

$ cat examples/string_helper_palindrome_with_pipe_operator.ex
def palindrome?(term) do
formatted_term = term
|> String.trim()
|> String.downcase()

formatted_term |> String.reverse() == formatted_term
end

While the impact may seem negligible in this simple example, you'll see the expressiveness this operator brings as we build our application throughout the book.

Module attributes, directives, and uses

Modules in Elixir may contain attributes. They're normally used where you'd use constants in other languages. You define a module attribute with the following syntax:

$ cat examples/string_helper_with_module_attribute.ex
defmodule StringHelper do
@default_mark "!"

# rest of the StringHelper code
end

Then, we can use the @default_mark module attribute inside the functions of this module. This attribute only exists at compile time, as it's replaced by its value during this process.

There are some other use cases for module attributes: you can register them, which makes them accessible at runtime. For instance, Elixir registers the @moduledoc and @doc attributes, which can be used to provide documentation for modules and functions, respectively. This documentation can then be accessed at runtime by other Elixir tools, as we'll explore in the Tooling and ecosystems section.

We're now mentioning macros for the first time. Macros are Elixir's mechanism to do meta-programminggenerating code that writes code. We will not touch macros in this introductory chapter, as they will be properly examined in Chapter 6Metaprogramming – Code that Writes Itself.

Elixir provides three lexically scoped directives to manage modules, plus a macro called use. We'll describe them now:

  • alias is used to create aliases for other modules. You use it as alias Helpers.StringHelper, as: StrHlp, and you can then refer to that module as StrHlp. The as: portion is optional, and if you don't provide it, the alias will be set to the last part of the module name.
  • We use require when we want to invoke what's defined as macros in a given module. As stated in the official documentation, is_odd/1 from the Integer module is defined as a macro. To use it in another module, you have to require it: require Integer.
  • When we want to access functions from other modules without having to use the fully-qualified name, we use import. When we import a given module, we're also automatically requiring it. If we're constantly using String.reverse/1 for instance, we can import it: import String, only: [reverse: 1]. Now we can just use reverse directly in our module. Apart from only:, you can also use except: to import all but a given number of functions from a module. Besides function names, only: and except: also accept :modules and :functions (which are self explanatory). You can also just use import without any option, but this isn't recommended, as it pollutes the scope of your modulealways try to pass the only: option when using import.
  • Last, but not least, we have use, which is a macro. This is commonly used to bring extra functionality to our modules. Beneath the surface, use calls the require directive and then calls the __using__/1 callback, which allows the module being used to inject code into our context.

For now, you don't need to know how all of this works. It is enough to know that you have these constructs to deal with modules. When we dive into macros later in this book, all of this will become much clearer.

Working with collections

Contrary to the most common programming languages, Elixir doesn't have while or do ... while constructs, which makes sense, given all data types are immutable. The way to iterate in Elixir is by using recursion, through functions that call themselves. Most of your needs when working with collections are covered by the high-level abstractions Elixir provides, meaning that you may barely use recursion when writing your Elixir applications.

Nevertheless, we'll begin this section by briefly describing recursion, and show an example of a recursive function in Elixir. Then, we'll see how we can process a collection using the Enum module, and finish the section by talking about the benefits of processing a collection lazily, and how to do it using the Stream module.

Looping through recursion

We'll show you how to create a recursive functions through two simple examples: doubling each element on a list, and multiplying consecutive elements of a list. As mentioned earlier, although you probably won't be using this in your day-to-day coding, it's still very important to understand how this work. This way, if the abstractions Elixir provides aren't enough for your use case, you can just create your own recursive functions to accomplish what you need.

Before jumping into the examples, let's explain generally how recursive functions work. In Elixir, they're usually implemented as a multi-clause function, using pattern matching to control its flow of execution. The first clause sets the condition that will stop the recursion, and is followed by other broader clauses that apply the recursion.

In the first example, we want to take a list of integers as input, and return a new list where each element is multiplied by two. Let's see the code for such a function:

$ cat examples/double.ex
defmodule Recursion do
def double([]), do: []

def double([head | tail]) do
[head * 2 | double(tail)]
end
end

Here are its results:

iex> Recursion.double([2, 4, 6])
[4, 8, 12]

Besides using multi-clause functions, we're also using pattern matching in two ways: to know when we've reached the end (and the empty list) and treat it accordingly; and to extract the head and the tail of a list, similar to what we've shown in the pattern matching section. The recursion happens when we call double(tail). As we're only passing the tail to the recursive call, we're essentially iterating through the list. When we reach an empty list, the first clause matches, we return an empty list, and all of the intermediate calls will unfold and create our new list.

What if, instead of returning a new list, we want to return a single value? We'll exemplify this by multiplying consecutive elements of a list. Here's the code to do it:

$ cat examples/multiply.ex
defmodule Recursion do
def multiply([]), do: 1

def multiply([head | tail]) do
head * multiply(tail)
end
end

Here's its use on an IEx session:

iex> Recursion.multiply([1, 2, 3])
6

The strategy is similar to the one shown in the previous example, except, instead of adding an element to a list at each step, we're now using our head as an accumulator. Also, it's important to note that, since we're doing a multiplication, our stopping condition must return 1 (the neutral element of this operation). The definition of the stopping condition varies between different problems, and is, arguably, one of the most important steps of defining a function recursively.

A common concern when dealing with recursive functions is its memory usage, as we have multiple function calls that will get into the stack. The Erlang runtime employs tail-call optimization whenever it can, which means that a recursive call won't generate a new stack push. For the runtime to do this optimization, you have to ensure that the last thing our function does is call another function (including itself)or, in other words, make a tail call. Here's our multiply function updated to make tail calls:

$ cat examples/multiply_with_tail_recursion.ex
def multiply(list, accum \\ 1)
def multiply([], accum), do: accum
def multiply([head | tail], accum) do
multiply(tail, head * accum)
end

The usual strategy is to pass an accumulator around, which enables us to use the tail-call optimization. Note that there's a trade-off here: On one hand, this optimization is important when dealing with large collections (since function calls don't consume additional memory); on the other hand, code that doesn't use this optimization is usually easier to read and comprehend, as it's usually more concise. When doing recursion, consider the advantages and disadvantages of each solution.

Eager processing with the Enum module

Having seen how recursion works in Elixir, we'll now show some examples of the abstractions that are built on top of it. We'll explore the Enum module, which contains a set of functions to work on collections. We've already seen some examples of collections in the Elixir's data types section, such as lists or maps. More generally, we can use the Enum module on collections that implement the Enumerable protocol.

We haven't yet covered protocols. We will do so in the Protocols section.

Taking the two examples from our Recursion section, let's see how they become incredibly simple to implement using the Enum module:

iex> Enum.map([2, 4, 6], &(&1 * 2))
[4, 8, 12]
iex> Enum.reduce([1, 2, 3], 1, &(&1 * &2))
6

The map function receives a collection and a lambda, and returns a new list where the lambda is applied to each element of the collection.

The reduce function receives a collection, an accumulator, and a lambda. The lambda receives the current element of the collection and the accumulator, and the result of this lambda is the accumulator for the following iteration. At the end of the iteration, reduce returns the final accumulator value.

We're using the capture operator to define a lambda. As we've previously hinted, you can also use it to capture named functions. In the following example, we're using the Integer.is_even/1 function to check which numbers are even in a collection:

iex> require Integer
Integer
iex> Enum.map([1, 2, 3], &Integer.is_even/1)
[false, true, false]

You'll see the Enum module being used in the application that we'll build throughout the book. For further usage of the Enum module, check its documentation at https://hexdocs.pm/elixir/Enum.html.

Comprehensions

Elixir provides another construct to iterate collections: comprehensions. As with the functions from the Enum module, comprehensions work on anything that implements the Enumerable protocol. Let's see a simple example:

iex> for x <- [2, 4, 6], do: x * 2
[4, 8, 12]

While, in this simple example, it is similar to Enum.map/2, comprehensions bring some other interesting features. You can, for instance, iterate over multiple collections and also apply filters. Let's see these two being applied in the following example:

iex> for x <- [1, 2, 3], y <- [4, 5, 6], Integer.is_odd(x), do: x * y
[4, 5, 6, 12, 15, 18]

Here we're doing a nested iterationfor each element of the first enumerable (which is represented by x), we will iterate through all elements of the second enumerable (represented by y). Also, we're applying a filter, and the body of our comprehension only gets executed when x is odd.

We won't be using comprehensions in the application we'll build throughout this book. However, it's important to mention them, as there are cases where using a comprehension instead of functions from the Enum module renders more elegant and expressive code

In our example, all comprehensions are returning a list, which is the default behavior. We can change that by passing the into: option, as you can see in this example:

iex> for x <- [1, 2, 3], into: %{}, do: {x, x + 1}
%{1 => 2, 2 => 3, 3 => 4}

As you can see, now we're getting a map back. The into: option takes a collection that will receive the results of the comprehension. This collection must implement the Collectable protocol. This protocol can be seen as the opposite of the Enumerable protocol, and is used to create a new structure from the values of an existing collection. This also has usage outside of comprehensionsthe Enum.into/2 function uses this protocol to create a new collection based on an enumerable.

Lazy processing with the stream module

We will now talk about a different way of processing collections, which, as functional programming, may require a shift in your mindset. Before talking about lazy processing, let's enumerate some of the shortcomings of working with the Enum module. The Enum module is referred to as being eager. This means that when processing a collection, this module will load the entire collection into memory. Furthermore, if you have a chain of functions you want to apply to a collection, the Enum module will iterate through your collection as many times as the functions are applying to it. Let's examine this further with an example:

iex> [1, 2, 3, 4, 5] \
...> |> Enum.map(&(&1 + 10)) \
...> |> Enum.zip(["a", "b", "c", "d", "e"])
[{11, "a"}, {12, "b"}, {13, "c"}, {14, "d"}, {15, "e"}]
The \ on the end of the first two lines is to stop our Elixir console from evaluating this line right away, and wait for a new line instead. This way, we can write these operations with the pipe operator on multiple lines, which makes them more readable.

We take our initial collection and iterate it to add 10 to each element inside it. This generates a new list, which is passed to our next function. This function will zip the two lists together, which will produce a new list, which is returned to us. In this simple example, we need to traverse our list twice to build the desired result.

This is where the Stream module, and lazy processing, becomes advantageous. When working with lazy enumerables, the entire collection never gets loaded into memory, and contrary to what we're accustomed to, the computations aren't made right away. The results are produced as they are needed. Let's see this same example with the Stream module:

iex> [1, 2, 3, 4, 5] \
...> |> Stream.map(&(&1 + 1)) \
...> |> Stream.zip(["a", "b", "c", "d", "e"])
#Function<66.40091930/2 in Stream.zip/1>

As you can see, we're not getting our final list back. When we feed our list to Stream.map, the list is not iterated. Instead, the functions that will be applied on it are saved into a structure (along with the collection we're working on). We can then pass this structure into the next function, which will further save a new function to be applied to our list. This is really cool! But how do we make it return the result we're expecting? Just treat it as a regular (eager) enumerable, by applying a function from the Enum module, and it will start to produce results.

To exemplify this, we'll use the Enum.take/2 function, which allows us to take a given number of items from an enumerable:

iex> [1, 2, 3, 4, 5] \
...> |> Stream.map(&(&1 + 10)) \
...> |> Stream.zip(["a", "b", "c", "d", "e"]) \
...> |> Enum.take(1)
[{11, "a"}]

As you can see, we're now getting the expected result back. Note that this is not a result of applying our computation to all the list and then just taking the first element. We've essentially only computed results for the first element, as that's all that was necessary. If you wanted to have the full list in the end, you could use the Enum.to_list/1 function.

Streams are a really nimble way to process large, or even infinite, collections. Imagine that you're parsing values from a huge CSV file, and then running some functions on them. If you're running your application on the cloud, as most of us are these days, you probably have a short amount of memory. Using lazy processing, you can avoid having to load the whole file, processing it line by line. If you're processing an infinite collection, such as an RSS feed, lazy processing is also a great solution, as you can process each element of the collection incrementally, as they arrive.

Note that while the Stream module is amazing, it will not replace your usage of the Enum module. It's certainly great for very large collections, or even if you have a big chain of functions being applied to a collection and only want to traverse it once. However, for small or even medium collections, the Stream module will perform worse, as you're adding a lot of overhead, for instance, by having to save the functions you'll apply instead of applying them right away. Always analyze your situation carefully and take this into account when choosing to use the Enum or the Stream module for a given task.

We'll be using functions from the Stream module in the application we'll build in this book. You'll learn more about the Stream module in Chapter 4, Powered by Erlang/OTP.

Elixir provides some functions that wrap most of the complex parts of building streams. If you want to build your own lazy stream, check out these functions from the Stream module: cycle, repeatedly, iterate, unfold, and resource. The full documentation for the Stream can be found at https://hexdocs.pm/elixir/Stream.html.

Control flow

We're now introducing control-flow constructs. In Elixir, they aren't used as often as in traditional imperative languages, because we can fulfill our control-flow needs, using a mix of pattern matching, multi-clause functions, and guard clauses. Whenever you're about to use one of the constructs we're presenting in this section, stop and check whether it's possible to employ a more functional approach. Code without these traditional control-flow constructs is usually easier to understand and test. If you get to a point where you have nested conditionals, it's almost guaranteed you can simplify it by using one of the approaches I mentioned earlier. Either way, you'll occasionally use these constructs, so it's important to know they exist.

if and unless

These two constructs can be used with the following syntax:

if <expression> do
# expression was truthy
else
# expression was falsy
end

unless <expression> do
# expression was falsy
else
# expression was truthy
end

As with the def construct, they can be inlined. For if, you'd do this:

if <expression>, do: # expression was truthy, else: # expression was falsy 

For both constructs, the else clause is optional. They will return nil if the main clause doesn't match and no else clause was provided.

cond

cond can be seen as a multi-way if statement, where the first truthy condition will run its associated code. This may substitute chains of if ... else if blocks. Let's see this with an example on IEx:

iex> x = 5
5
iex> cond do
...> x * x == 9 -> "x was 3"
...> x * x == 16 -> "x was 4"
...> x * x == 25 -> "x was 5"
...> true -> "none of the above matched"
...> end
"x was 5"

true in a condition will serve as a default condition, which will run when no other clause matches.

case

case accepts an expression, and one or more patterns, which will match against the return value of the expression. These patterns may include guard clauses. These patterns are matched (from top to bottom), and will run the code associated with the first expression that matches. Here is a simple example:

iex> case Enum.random(1..10) do
...> 2 -> "The lucky ball was 2"
...> 7 -> "The lucky ball was 7"
...> _ -> "The lucky ball was not 2 nor 7"
...> end
"The lucky ball was not 2 nor 7"

Note that your output may differ when running this example, as we're matching against Enum.random/1. In here, the default condition is represented by using _ in the pattern, which will match anything. Although a bit more condensed, the case construct is similar to a multi-clause function.

with

This control-flow construct, introduced in Elixir 1.2, accepts one or more expressions, a do block, and optionally an else block. It allows you to use pattern matching on the return value of each expression, running the do block if every pattern matches. If one of the patterns doesn't match, two things may happen: If provided, the else block will be executed; otherwise, it will return the value that didn't match the expression. In practice, with allows you to replace a chain of nested instances of case or a group of multi-clause functions.

To demonstrate the usefulness of with, let's see an example:

iex> options = [x: [y: [z: "the value we're after!"]]]               
[x: [y: [z: "the value we're after!"]]]
iex> case Keyword.fetch(options, :x) do
...> {:ok, value} -> case Keyword.fetch(value, :y) do
...> {:ok, inner_value} -> case Keyword.fetch(inner_value, :z) do
...> {:ok, inner_inner_value} -> inner_inner_value
...> _ -> "non-existing key"
...> end
...> _ -> "non-existing key"
...> end
...> _ -> "non-existing key"
...> end
"the value we're after!"

We're using the Keyword.fetch/2 function to get the value of a key from a keyword list. This function returns {:ok, value} when the key exists, and :error otherwise. We want to retrieve the value that's nested on three keyword lists. However, let's say that if we try to fetch a key that doesn't exist on the keyword list, we have to return "non-existing key". Let's achieve the same behavior using with, operating on the same options list as the preceding example:

iex> with {:ok, value} <- Keyword.fetch(options, :x),                
...> {:ok, inner_value} <- Keyword.fetch(value, :y),
...> {:ok, inner_inner_value} <- Keyword.fetch(inner_value, :z),
...> do: inner_inner_value
"the value we're after!"

Note that, since our expression is really small, we're using the shorthand do: syntax (but we can also use a regular do ... end block). As you can see, we're getting the same result back. Let's try to fetch a key that doesn't exist:

iex> with {:ok, value} <- Keyword.fetch(options, :missing_key),      
...> {:ok, inner_value} <- Keyword.fetch(value, :y),
...> {:ok, inner_inner_value} <- Keyword.fetch(inner_value, :z),
...> do: inner_inner_value
:error

Since we didn't provide an else block, we're getting back the value that didn't match, which is the return value of Keyword.fetch/2 when a key doesn't exist in the keyword list provided. Let's do the same, but by providing an else block:

iex> with {:ok, value} <- Keyword.fetch(options, :missing_key), 
...> {:ok, inner_value} <- Keyword.fetch(value, :y),
...> {:ok, inner_inner_value} <- Keyword.fetch(inner_value, :z) do
...> inner_inner_value
...> else
...> :error -> "non-existing key"
...> _ -> "some other error"
...> end
"non-existing key"

Since we're now providing an else block, we can now handle error cases accordingly. As you can see, else takes a list of patterns to match on. As you do with case, you can use _ as a default clause, which would run when the patterns above (if any) didn't match.

As you can see, with is a very helpful construct, which allows us to create very expressive code that is concise and easy to read. Moreover, you can control how to handle each error separately, using pattern matching inside the else block.

The first expression provided to with has to be on the same line of with itself, you'll get a SyntaxError otherwise. If you do want to have with on its own line, wrap the expressions provided to it in parentheses.

Exceptions

Much like if and else, exceptions in Elixir aren't used as much as in other popular imperative languages. Exceptions aren't used for control flow, and are left for when truly exceptional things occur. When they do, your process is usually running under a supervision tree, and upon crashing, the supervisor of your process will be notified and (possibly, depending on the strategy) restart it. Then, upon being restarted, you're back to a known and stable state, and the effects of the exceptional event are no longer present. In the Elixir and Erlang communities, this is usually referred to as the "Let it crash!" philosophy. We'll be examining this in greater detail in Chapter 3, ProcessesThe Bedrock for Concurrency and Fault Tolerance, when we talk about processes, supervisors, and supervision trees.

For now, I'll list the traditional error-handling constructs. You can raise an error with the raise construct, which takes one or two arguments. If you provide only one argument, it will raise a RuntimeError, with the argument as the message. If you provide two arguments, the first argument is the type of error, while the second is a keyword list of attributes for that error (all errors must at least accept the message: attribute). Let's see this in action:

iex> raise "Something very strange occurred"
** (RuntimeError) Something very strange occurred
iex> raise ArithmeticError, message: "Some weird math going on here"
** (ArithmeticError) Some weird math going on here

You can rescue an error by using the rescue construct (you can rescue from a try block or from a whole function, pairing it with def). You define patterns on the rescue clause. You can use _ to match on anything. If none of the patterns match, the error will not be rescued and the program will behave as if no rescue clause was present:

iex> try do
...> 5 / 0
...> rescue
...> e in ArithmeticError -> "Tried to divide by 0."
...> _ -> "None of the above matched"
...> end
"Tried to divide by 0."

Since we're not doing anything with the error, and just returning a string, we could just use ArithmeticError in the pattern. Only use this syntax if you want to capture the error itself. When none of the patterns match, we get the error back in our console:

iex> try do
...> 5 / 0
...> rescue
...> ArgumentError -> "ArgumentError was raised."
...> end
** (ArithmeticError) bad argument in arithmetic expression

Furthermore, you can also pass an else and/or an after block to the try/rescue block. The else block will match on the results of the try body when it finishes without raising any error. As for the after construct, it will always get executed, regardless of the errors that were raised. This is commonly used to clean up some resources (closing a file descriptor, for instance).

We've mentioned in this section that we don't use exceptions to control the flow of our programs, but in fact there's a special construct in Elixir for this. The syntax is similar to the one shown earlier, but you use throw instead of raise, and catch instead of rescue. As mentioned in the official Getting Started guide (http://elixir-lang.github.io/getting-started/try-catch-and-rescue.html) this should be used in situations where it is not possible to retrieve a value unless by using throw and catch. It's also mentioned that those situations are quite uncommon in practice.

Typespecs and behaviours

As already mentioned in the beginning of this chapter, Elixir is a dynamic programming language. As such, we don't declare the type of each variable, as it depends on the value each variable is bound to at each moment.

Usually dynamic programming languages yield higher productivity, as programmers don't need to declare types and can focus on developing the logic of an application. However, this comes at a cost: Certain errors, which in statically-typed languages would be caught at compile-time, may only be caught at runtime in dynamic languages. The time saved by using a dynamic language is then used (often in excess) on debugging in production.

We're not advocating for statically-typed languagesthis book is about Elixir, after all. But what if you could have the best of both worlds?

It turns out you can! Type specifications, or typespecs, are a mechanism to annotate function signatures with the respective types (arguments and return values). Typespecs are also used to create custom data types. Let's explore them in more detail, before jumping into behaviours.

Typespecs

Typespecs are written using the @spec module directive, followed by function_name(argument_type) :: return_type. This module directive is placed right before the definition of the function we're annotating.

To demonstrate how to apply typespecs to a function, let's bring back our palindrome? function:

$ cat examples/string_helper.ex
defmodule StringHelper do
def palindrome?(term) do
String.reverse(term) == term
end
end

Given that the module name is StringHelper, and that it's using functions from the String module, we can see that this function receives a string. As it uses the == operator, and also hinted at by the trailing ? on the function name, we know this function returns a Boolean. With this information, writing the typespec for this function is straightforward:

$ cat examples/string_helper_palindrome_with_typespec.ex
defmodule StringHelper do
@spec palindrome?(String.t) :: boolean
def palindrome?(term) do
String.reverse(term) == term
end
end

You can also define your own types to use in typespecs, by using the @type directive within a module (usually the module where that type is used the most). Let's say that you frequently use a tuple in your module that contains the name of a country on the first element, and its population on the second, as in {"Portugal, 10_309_573}. You can create a type for this data structure with this:

@type country_with_population :: {String.t, integer}

Then, you could use country_with_population in typespecs as you'd use any other type.

Defining a type in a module will export it, making it available to all modules. If you want your type to be private to the module where it's defined, use the @typep directive. Similar to the @moduledoc and @doc directives (to document modules and functions, respectively), you can use the @typedoc directive to provide documentation for a certain type.

In typespecs, the string type is usually represented by String.t. You can also use string but the Elixir community discourages this, with the goal of avoiding confusion with the charlist type, which represents strings in Erlang. If you use string in typespecs, the compiler will emit a warning.

Typespecs also provide great documentation, as we can quickly grasp what types this function accepts and returns. This example only shows a subset of the types you can useplease refer to the official documentation to get a full list, at

https://hexdocs.pm/elixir/typespecs.html.

Dialyzer

Dialyzer (http://erlang.org/doc/man/dialyzer.html) is a tool that ships with Erlang and performs static analysis of code. It analyses compiled .beam files, making it available for all programming languages that run on the Erlang VM (such as Elixir!). While Dialyzer can be helpful on projects that don't have typespecs (as it can, for instance, find redundant code), its power is maximized on projects that have their functions annotated with typespecs. This way, Dialyzer is able to report on typing errors, which brings you closer to the security you can get on a statically-typed language.

Although we won't be exploring Dialyzer in this book, we highly recommend its usage, as it can be very helpful. Particularly, we feel that the Dialyxir library (https://github.com/jeremyjh/dialyxir), is a great way to integrate Dialyzer into Elixir projects, as it abstracts away part of the complexity of dealing with Dialyzer directly.

Behaviours

Behaviours provide a way to describe a set of functions that have to be implemented by a module, while also ensuring that the module implements the functions in that set. If you come from an object-oriented programming background, you can think of behaviours as abstract base classes that define interfaces. After declaring the behaviour, we can then create other modules that adopt this behaviour. A behaviour creates an explicit contract, which states what the modules that adopt the behaviour need to implement. This way, we can have our business logic tied to abstractions (the behaviour) instead of concrete implementations. We can swap two implementations of the same behaviour with very little and localized change in our applicationthe place where we define the implementation of the behaviour we're going to use.

Let's demonstrate this concept with an example. We'll define a behaviour, called Presenter, that has only one function: present. We'll then define a module that adopts this behaviour, called CLIPresenter. We could also have other modules that would adopt this behaviour, such as a GraphicalPresenter. A behaviour is created using the @callback directive inside a module, providing a typespec for each function this behaviour contains. Thus, to define our Presenter behaviour, we use the following:

$ cat examples/presenter.ex
defmodule Presenter do
@callback present(String.t) :: atom
end

And now we define the module that will adopt this behaviour:

$ cat examples/cli_presenter.ex
defmodule CLIPresenter do
@behaviour Presenter

@impl true
def present(text) do
IO.puts(text)
end
end

We can see this module working in the next snippet:

iex> CLIPresenter.present("printing text to the command line")
printing text to the command line
:ok

From Elixir 1.5 onwards, we may use the @impl directive to mark the functions that are implemented as callbacks for a behaviour. In our case, we'd put the @impl directive on top of the present function:

$ cat examples/cli_presenter_with_impl_annotation.ex
defmodule CLIPresenter do
@behaviour Presenter

@impl true
def present(text) do
IO.puts(text)
end
end

We can be even more specific and use @impl Presenter, to state that this function is a callback from the Presenter behaviour. This brings two major advantages:

  • Increased readability, as it's now explicit which functions make up part of our API and which functions are callback implementations.
  • Greater consistency, as the Elixir compiler will check whether the functions you're marking with @impl are part of a behaviour your module is adopting.

Note that when you set @impl on a module, you must set it on all callback functions on that module, otherwise a warning will be issued. Furthermore, you can only use @impl on functions that are callbacks, otherwise a compilation warning will be issued as well.

You can use @optional_callbacks to mark one or more functions as optional when adopting a certain behaviour. You need to provide a keyword list, with function names as keys, and arity as the value. If we wanted our present/1 function to be optional in the Presenter behaviour, we would use:

$ cat examples/presenter_with_optional_callbacks.ex
defmodule Presenter do
  @callback present(String.t) :: atom
  @optional_callbacks present: 1
end

If we didn't implement all of the functions declared in the Presenter behaviour (by commenting the CLIPresenter.present/1 function, for instance), the Elixir compiler would emit the following warning:

warning: undefined behaviour function present/1 (for behaviour Presenter)

Behaviours help us follow the open/closed principle, since we can extend our system without modifying what we already have. If, in the future, we would need a new type of presenter, we'd just have to create a new module that adopts the Presenter behaviour, without having to change the behaviour or its current implementations. We'll be using behaviours in the application we'll build in this book, and you'll see them in action again in the next chapter.

Protocols

Throughout this introductory chapter, we've mentioned a couple of times that Elixir has protocols, with Enumerable being one of the examples. In this section, we'll dive into protocols and even define our own!

Protocols, like the behaviours we've seen in the last section, define a set of functions that have to be implemented. In that sense, both constructs serve as a way to achieve polymorphism in Elixirbeing able to display multiple forms of behavior, but all linked to a single interface. While behaviours define a set of functions that a module needs to implement, and are thus tied to a module, protocols define a set of functions that a data type must implement. This means that, with protocols, we have data type polymorphism, and we're able to write functions that behave differently depending on the type of their arguments.

Let's now see how we can create a new protocol. We'll pick up, and extend, the example present in the official Getting Started guide (at http://elixir-lang.github.io/getting-started/protocols.html). We will define a Size protocol, which will be implemented by each data type. To define a new protocol, we use the defprotocol construct:

$ cat examples/size.ex
defprotocol Size do
@doc "Calculates the size of a data structure"
def size(data)
end

We're stating that our Size protocol expects the data types that will implement it must define a size/1 function, where the argument is the data structure we want to know the size of.

You can use the @doc directive to add documentation to this function, as you normally do with named functions inside modules. We can now define the implementation of this protocol for the data types we're interested in, using the defimpl construct:

$ cat examples/size_implementations_basic_types.ex
defimpl Size, for: BitString do
def size(string), do: byte_size(string)
end

defimpl Size, for: Map do
def size(map), do: map_size(map)
end

defimpl Size, for: Tuple do
def size(tuple), do: tuple_size(tuple)
end
We didn't define an implementation for the lists, as in Elixir, size is usually used for data structures that have their size precomputed. For types where we have to compute this on demand, such as lists, the length term is used instead of size. This is further observable by looking at the name of the function used to get the dimension of a list: Kernel.length/1.

With this defined, we can see our protocol in action:

iex> Size.size("a string")
8
iex> Size.size(%{a: "b", c: "d"})
2
iex> Size.size({1, 2, 3})
3

If we try to use our protocol on a type that doesn't have an implementation defined, an error is raised:

iex> Size.size([1, 2, 3, 4])
** (Protocol.UndefinedError) protocol Size not implemented for [1, 2, 3, 4]
You can define an implementation for a protocol on all Elixir data types: Atom, BitString, Float, Function, Integer, Tuple, List, Map, PID, Port, and Reference. Note that BitString is used for the binary type as well.

Having to implement a protocol for all types may quickly become monotonous and exhausting. You can define a fallback behavior for types that don't implement your protocol by implementing the protocol for Any. Let's do this for our Size protocol:

$ cat examples/size_implementation_any.ex
defimpl Size, for: Any do
def size(_), do: 0
end

You have to define the desired behavior when a type doesn't implement your protocol. In this case, we're saying that it has a size of 0 (which might not make sense, since the data type may have a size different than 0, but let's ignore that detail).

We now have two options for this implementation to be used: Either mark the modules where we want this fallback behavior with @derive [Size] (the List module, for instance), or use @fallback_to_any true in the definition of our Size protocol. The former is more laborious as you have to annotate each module that you want to assume the behavior for Any, while the latter is simpler since you make it work on all data types just by changing the definition of your protocol. In the Elixir community, explicitness is usually preferred, and, as such, you're more likely to see the @derive approach in Elixir projects.

While implementing protocols for Elixir's data types already opens a world of possibilities, we can only fully utilize Elixir's extensibility when we mix them with structs. We haven't yet talked about structs, so we'll introduce them in the next section.

Structs

Structs are an abstraction built on top of maps. We define a struct inside a module, with the defstruct construct. The struct's name is the name of the module it's being defined in (which means you can only define one struct per module). To defstruct, we pass a keyword list, which contains the key-value pairs that define the fields that struct has, along with their default values. Let's define a Folder struct:

$ cat examples/folder.ex
defmodule Folder do
defstruct name: "new folder", files_info: [], path: nil
end

We can now use it in our IEx session:

iex> %Folder{}
%Folder{files_info: [], name: "new folder", path: nil}
iex> %Folder{}.name
"new folder"
iex> %Folder{}.files_info
[]

Elixir already has a File module, which provides several functions to deal with files. One of them is the File.stat/2, which returns a %File.Stat{} struct with information about the provided path. The files_info field in our %Folder{} struct is a list, which will contain %File.Stat{} structs as elements. Let's initialize a folder with one file:

iex> folder = %Folder{files_info: [File.stat!("string_helper.ex")]}
%Folder{files_info: [%File.Stat{access: :read_write,
atime: {{2017, 12, 31}, {16, 58, 56}}, ctime: {{2017, 12, 30}, {3, 40, 29}},
gid: 100, inode: 3290229, links: 1, major_device: 65024, minor_device: 0,
mode: 33188, mtime: {{2017, 12, 30}, {3, 40, 29}}, size: 509, type: :regular,
uid: 1000}], name: "new folder", path: nil}

Note that this example assumes you have a "string_helper.ex" file in the directory where you started iex. Also note that we're using File.stat!, which works similarly to File.stat, but, instead of returning a {:ok, result} tuple, it returns the result itself.

We now have our %Folder{} struct with one file. We can now show you the syntax to update a struct, which is similar to the one used in maps (or you can use the functions from the Map module). Assuming you also have a "recursion.ex" file on your current working directory, you can use this syntax to update the struct:

iex> folder = %Folder{ folder | files_info: [File.stat!("recursion.ex") | folder.files_info]}
%Folder{files_info: [%File.Stat{access: :read_write,
atime: {{2017, 12, 30}, {20, 8, 29}}, ctime: {{2017, 12, 30}, {20, 8, 25}},
gid: 100, inode: 3278529, links: 1, major_device: 65024, minor_device: 0,
mode: 33188, mtime: {{2017, 12, 30}, {20, 8, 25}}, size: 270, type: :regular,
uid: 1000},
%File.Stat{access: :read_write, atime: {{2017, 12, 31}, {16, 58, 56}},
ctime: {{2017, 12, 30}, {3, 40, 29}}, gid: 100, inode: 3290229, links: 1,
major_device: 65024, minor_device: 0, mode: 33188,
mtime: {{2017, 12, 30}, {3, 40, 29}}, size: 509, type: :regular, uid: 1000}],
name: "new folder", path: nil}
iex> folder.files_info
[%File.Stat{access: :read_write, atime: {{2017, 12, 30}, {20, 8, 29}},
ctime: {{2017, 12, 30}, {20, 8, 25}}, gid: 100, inode: 3278529, links: 1,
major_device: 65024, minor_device: 0, mode: 33188,
mtime: {{2017, 12, 30}, {20, 8, 25}}, size: 270, type: :regular, uid: 1000},
%File.Stat{access: :read_write, atime: {{2017, 12, 31}, {16, 58, 56}},
ctime: {{2017, 12, 30}, {3, 40, 29}}, gid: 100, inode: 3290229, links: 1,
major_device: 65024, minor_device: 0, mode: 33188,
mtime: {{2017, 12, 30}, {3, 40, 29}}, size: 509, type: :regular, uid: 1000}]

As you can see, we now have two files in our %Folder{} struct.

Although structs are implemented on top of maps, they do not share protocol implementations with the Map module. This means that you can't, out of the box, iterate on a struct, as it doesn't implement the Enumerable protocol.

We'll end our little tour of structs with two more bits of information. First, if you don't provide a default value when defining the fields of a struct, nil will be assumed as its default value. Second, you can enforce that certain fields are required when creating your struct. You do that with the @enforce_keys module attribute. If we wanted to make sure path was provided when creating our %Folder{} struct, we would define it as following:

$ cat examples/folder_with_enforce_keys.ex
defmodule Folder do
@enforce_keys :path

defstruct name: "new folder", files_info: [], path: nil
end

If you don't provide path when creating this struct, ArgumentError will be raised:

iex> %Folder{}
** (ArgumentError) the following keys must also be given when building struct Folder: [:path]
expanding struct: Folder.__struct__/1
iex:46: (file)
iex> %Folder{path: "/a/b/c/"}
%Folder{files_info: [], name: "new folder", path: "/a/b/c/"}

Bringing structs and protocols together

Now that we have the %Folder{} struct defined, we can define its implementation for the Size protocol.

We'll first define the implementation for the %File.Stat{} struct, as we can then use this to implement the protocol for %Folder{}. Here's the implementation for %File.Stat{}:

$ cat examples/size_implementations_file_stat_and_folder.ex
defimpl Size, for: File.Stat do
def size(file_stat), do: file_stat.size
end

# ...

With this in place, our implementation for our %Folder{} struct is as follows:

$ cat examples/size_implementations_file_stat_and_folder.ex
# ...

defimpl Size, for: Folder do
def size(folder) do
folder.files_info
|> Enum.map(&Size.size(&1))
|> Enum.sum()
end
end

To find out the size of a folder, we sum the size of each file it contains. As such, this implementation iterates through our files_info list, using the Size implementation for %File.Stat{} to get the size of each file, summing all the sizes in the end. In the following snippet, we can see this implementation being used on the folder variable we just defined:

iex> Size.size(folder)
779

With this, we can see the full power of mixing structs and protocols, which lets us have polymorphic functions based on the data type of their arguments. We now have a common interface, Size.size(data), that allows us to find out the size of pretty much anything we want, provided that we implement the Size protocol for the data type we're interested in.

Tooling and ecosystems

The last section of this chapter is dedicated to one of the most renowned characteristics of Elixir: its incredible tooling. Also, since we can directly use Erlang libraries, we can take advantage of a very mature ecosystem that has been around for decades. First, let's begin with Elixir's interactive shell, IEx.

IEx

In the beginning of this chapter, we provided some instructions on how to start and stop an Elixir shell. You've seen it in use throughout this chapter. We'll now show you some other interesting things you can do inside IEx.

In this chapter, we've been giving you links to the official documentation of several Elixir modules. You can access this documentation right from IEx, using the h command. Let's say you want to know more about the Enum module:

iex> h Enum
Enum
#...,

The preceding code provides a set of algorithms that enumerate over enumerables according to the Enumerable protocol.

Note that, for brevity, the output of this command was cropped.

You can even pass a fully qualified function name, and get information about it. Let's say, for example, you don't remember the order of the arguments for the Enum.map/2 function:

iex> h Enum.map/2
def map(enumerable, fun)
@spec map(t(), (element() -> any())) :: list()

Returns a list where each item is the result of invoking fun on each
corresponding item of enumerable.
For maps, the function expects a key-value tuple.
## Examples

iex> Enum.map([1, 2, 3], fn(x) -> x * 2 end)
[2, 4, 6]

iex> Enum.map([a: 1, b: 2], fn({k, v}) -> {k, -v} end)
[a: -1, b: -2]

Another very interesting feature, which ships from Elixir 1.5 onwards, is the ability to set breakpoints from IEx. To showcase this, let's create a breakpoint on the StringHelper.palindrome?/1 function we've defined in the Functions and modules section, present in the "string_helper.ex" file:

iex> break! StringHelper.palindrome?/1
1
iex> StringHelper.palindrome?("abba")
Break reached: StringHelper.palindrome?/1 (string_helper.ex:2)

1: defmodule StringHelper do
2: def palindrome?(term) when is_bitstring(term) do
3: String.reverse(term) == term
4: end

pry(1)> term
"abba"
pry(2)> whereami
Location: string_helper.ex:2

1: defmodule StringHelper do
2: def palindrome?(term) when is_bitstring(term) do
3: String.reverse(term) == term
4: end

As you can see, you could access the argument passed to the function, term, and also use whereami to show the location where the breakpoint stopped the execution. To resume the execution, you use the continue command to go to the next breakpoint, or respawn to exit and start a new shell.

From Elixir 1.6 onward, you can also use pattern matching and guard clauses when setting a breakpoint. The breakpoint we defined earlier could also be set as break! StringHelper.palindrome?("abba"), which would make our breakpoint work when that function is called with "abba" as the argument.

You can also use IEx.pry to set a breakpoint on the source-code file, instead of doing it via IEx. Use the method that's most appealing to you.

To finish this section, we'll show you a handy feature of IEx: the ability to access values from the history. Sometimes, you call a certain function and realize afterward that you wanted to bind the result of that function call to a variable. If this happens, you can use the v/1 function. You pass it a number, which represents the position of the expression from which you want to retrieve the value (starting at 1). You can also pass a negative number, which makes the position relative to the current expression in the shell. You can call this function without providing an argument, which is the same as calling v(-1)which means we're getting the value from the last expression. Let's see an example:

iex> 7 * 3
21
iex> a = v
21
iex> a * 2
42
iex> v(-3)
21

There are more things you can do with IEx, such as configuring it or connecting to remote shells. Please refer to the official documentation (at https://hexdocs.pm/iex/IEx.html) for further usage examples of IEx.

Mix

Mix is the Swiss-army knife of the Elixir tools. It's used to compile and run your application, manage your dependencies, run tests, and even for profiling your code. You can see which Mix tasks are available by typing mix help in your terminal. We won't explain its usage here, since the next chapter's purpose is to show you how you can use Mix to create and maintain an Elixir project.

ExUnit

Elixir comes with a fully fledged unit test framework: ExUnit. We'll use it to write tests for the application built in the course of this book. Chapter 9, Finding Zen Through Testing, is dedicated to testing, where we'll demonstrate how you can write tests for your application that let you sleep at night.

Erlang interoperability

As we stated at the beginning of this chapter, Elixir targets the Erlang runtime. Elixir is compiled to byte-code that can run on an Erlang VM (or BEAM), and Erlang libraries can be used in your Elixir projects (and vice versa). The philosophy in the Elixir community is to not reinvent the wheel and directly use Erlang libraries when appropriate. The creation of Elixir libraries that simply wrap an underlying Erlang library is discouraged, as you can directly call an Erlang library from your Elixir code.

We can take advantage not only of Erlang libraries, but also of their tooling. For instance, Erlang ships with a tool called Observer, which allows you to monitor your server and provides you with tons of useful information about the Erlang VM, such as the running processes or dynamic charts of load. We'll explore this tool in greater detail in Chapter 11Keeping an Eye on Your Processes, when we talk about monitoring.

To utilize an Erlang library, you write its name as an atom and then call functions on it. Elixir doesn't have a Math module, so when more advanced mathematical operators are needed, it's common to call the math Erlang library. Let's use this library to calculate the natural logarithm of 10:

iex> :math.log(10)
2.302585092994046
You can check out Erlang's standard libraries at http://erlang.org/doc/apps/stdlib/index.html.

When we introduced the data types in Elixir, we mentioned the Port type, which is a reference to a resource used by the Erlang VM to interact with external resources. Let's now see how we can use them to interact with an operating system process. To interact with ports, we use functions from the Port module. In the following example, we'll use the whoami UNIX command, which prints the username associated with the current user. To do that, we open a port and provide the name of the executable we want to run:

iex> port = Port.open({:spawn, "whoami"}, [:binary])
#Port<0.3730>

We passed the [:binary] option so that we get our result as a binary instead of a list of bytes. We now use the IEx flush() helper to print the messages received by the port:

iex> flush()
{#Port<0.3731>, {:data, "dcaixinha\n"}}
:ok

As our operating system process died after returning its result, the port is also closed. If this was not the case, we could use the Port.close/1 function to explicitly close the port. Also note that this was a very simple example. You can have more complex interactions by using the Kernel.send/2 function to dynamically send messages (that contain commands) to your port. In the official documentation for ports (which is available at https://hexdocs.pm/elixir/Port.html), you can see how this can be achieved.

If all we're interested in is running an operating-system command and getting its result back, we can use the System.cmd/3 function, which is an abstraction on top of ports that allows us to achieve this effortlessly.

Summary

We've now reached the end of the first chapter, which contains a condensed introduction to Elixir. We didn't visit every aspect of the language, but provided several links that are worth exploring. Our goal is to provide the necessary knowledge for you to follow the next chapters, where we'll build a complex application. A lot of ground was covered in this chapter, so it's normal if you don't remember every concept we covered. Let's recap the most important points:

  • Elixir is a dynamic language, and the type of a variable is determined by the value it holds.
  • Every data type is immutable, which means that you never actually change the contents of a variable, you operate on copies of it. You can, however, rebind a variable, which will make it point to a new memory location, leaving its old contents untouched.
  • Elixir code is organized in modules, which contain a set of functions.
  • Functions are first-class citizens, as you can assign them to variables and pass them as arguments to other functions.
  • Iteration in Elixir is always made through recursion. There are no while or do ... while constructs. Elixir provides a set of modules, such as Enum and Stream, that abstract the recursion and let you work with collections efficiently.
  • The usual control-flow constructs, such as if and else statements, are less common in Elixir. You still use them occasionally, but you normally use a combination of pattern matching and multi-clause functions to control the flow of your programs.
  • Exceptions aren't used for control-flow. Instead, they are used for truly exceptional events. We rely on supervision trees (which we'll discuss in Chapter 3, ProcessesThe Bedrock for Concurrency and Fault Tolerance) to recover from exceptions.
  • You can annotate your functions with typespecs, or type specifications, bringing some of the safety of static-type languages into a dynamic language. These annotations also serve as a type of documentation for your functions.
  • Elixir provides great mechanisms to have extensibility in your code, such as Behaviours and Protocols.
  • Elixir comes bundled with amazing tooling, and you can also take advantage of all the libraries in the Erlang ecosystem.

In the next chapter, we'll learn how to use Mix to create a new project, while also describing the application we'll build throughout this book. This was the only chapter that contained ad hoc examples, and in the following chapters, we'll always use our application to exemplify the concepts that we want to explain.

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • •Enhance your Elixir programming skills using its powerful tools and abstractions
  • •Discover how to develop a full-fledged file server
  • •Understand how to use Phoenix to create a web interface for your application.

Description

Running concurrent, fault-tolerant applications that scale is a very demanding responsibility. After learning the abstractions that Elixir gives us, developers are able to build such applications with inconceivable low effort. There is a big gap between playing around with Elixir and running it in production, serving live requests. This book will help you fll this gap by going into detail on several aspects of how Elixir works and showing concrete examples of how to apply the concepts learned to a fully ?edged application. In this book, you will learn how to build a rock-solid application, beginning by using Mix to create a new project. Then you will learn how the use of Erlang's OTP, along with the Elixir abstractions that run on top of it (such as GenServer and GenStage), that allow you to build applications that are easy to parallelize and distribute. You will also master supervisors (and supervision trees), and comprehend how they are the basis for building fault-tolerant applications. Then you will use Phoenix to create a web interface for your application. Upon fnishing implementation, you will learn how to take your application to the cloud, using Kubernetes to automatically deploy, scale, and manage it. Last, but not least, you will keep your peace of mind by learning how to thoroughly test and then monitor your application.

Who is this book for?

Mastering Elixir is for you if you have experience in Elixir programming and want to take it to the next level. This Elixir book shows you how to build, deploy, and maintain robust applications, allowing you to go from tinkering with Elixir on side projects to using it in a live environment. However, no prior knowledge of Elixir is required to enjoy the complex topics covered in the book.

What you will learn

  • •Use Elixir tools, including IEx and Mix
  • •Find out how an Elixir project is structured and how to create umbrella applications
  • •Discover the power of supervision trees, the basis for fault-tolerance
  • •Create a Domain-Specifc Language (DSL) that abstracts complexity
  • •Create a blazing-fast web interface for your application with Phoenix
  • •Set up an automatic deployment process for the cloud
  • •Monitor your application and be warned if anything unexpected happens

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Jul 30, 2018
Length: 574 pages
Edition : 1st
Language : English
ISBN-13 : 9781788472241
Vendor :
Apache
Category :
Languages :
Tools :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Product Details

Publication date : Jul 30, 2018
Length: 574 pages
Edition : 1st
Language : English
ISBN-13 : 9781788472241
Vendor :
Apache
Category :
Languages :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just NZ$7 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just NZ$7 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total NZ$ 193.97
Learning Elixir
NZ$56.99
Mastering Elixir
NZ$71.99
Phoenix Web Development
NZ$64.99
Total NZ$ 193.97 Stars icon
Banner background image

Table of Contents

12 Chapters
Preparing for the Journey Ahead Chevron down icon Chevron up icon
Innards of an Elixir Project Chevron down icon Chevron up icon
Processes – The Bedrock of Concurrency and Fault Tolerance Chevron down icon Chevron up icon
Powered by Erlang/OTP Chevron down icon Chevron up icon
Demand-Driven Processing Chevron down icon Chevron up icon
Metaprogramming – Code That Writes Itself Chevron down icon Chevron up icon
Persisting Data Using Ecto Chevron down icon Chevron up icon
Phoenix – A Flying Web Framework Chevron down icon Chevron up icon
Finding Zen through Testing Chevron down icon Chevron up icon
Deploying to the Cloud Chevron down icon Chevron up icon
Keeping an Eye on Your Processes Chevron down icon Chevron up icon
Other Books You May Enjoy Chevron down icon Chevron up icon

Customer reviews

Rating distribution
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
(2 Ratings)
5 star 50%
4 star 0%
3 star 50%
2 star 0%
1 star 0%
Tyler Munyon Jul 10, 2019
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Love the book so far, especially since we get to build an application that uses concepts that we are learning. Am wondering how to get the online version of the book on packt??
Amazon Verified review Amazon
Sean Bergstedt Apr 21, 2020
Full star icon Full star icon Full star icon Empty star icon Empty star icon 3
[REVISION]After re-referencing it a few more times, I've had time to reconsider my previous 1 star vote. There are some good sections on leveraging elixir's GenStage abstraction with Flow, which is unique and a pretty new addition to the Elixir scene. There are also some good sections on how to setup umbrella projects, how to integrate with SQL backend and best practices preserving coding style.The book is a bit rough around the edges, though, in terms of organization and just getting how the code works on your first read. Like I mentioned in my original review, you kind of have to hunt for what you want to learn and if you want to follow the code, you need to already have some familiarity with mix and the language. So, if you're just starting with Elixir, I'd recommend looking for one of the introductory books instead.[ORIGINAL]Lots of cross referencing and looking at the on-line source code to make sure your book code is tracking along nicely. Sometimes you have to look at something in later chapters in order to run code in earlier chapters. I ended up spending more time organizing the information the editors should have organized for me.I continue to be disappointed with Pakt Publishing. They just crank out books on volume instead of quality. At the end of the day, I don't blame the authors as I'm sure they spent a great deal of time on this, but the organization of code and flow of content is wildly out of sync with the quality one should expect for a book this price.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.