Using Julia modules and packages

Code in Julia is not limited to functions and can be organized at higher levels through modules and packages. A module is one level higher than functions under which code in Julia can be organized. A package is another level higher, can contain one or more modules, and provides functionality that can be reused by other Julia projects. Often, a web app is a package, containing a number of modules. When the package contains a project file called Project.toml, the file is also a project.

Modules

Modules are used to group together the definitions of types, functions, constants, and so on that are related. By convention, a file named M.jl will always define a module named M.

Such a module will be declared as follows (shown here for the Genie framework):

module Genie
# Loads dependencies and bootstraps a Genie app.
# Exposes core Genie functionality.
end

And it will be stored in Genie.jl.

To illustrate, let’s create a module ToDoApp inside a ToDoApp.jl file, with the ToDo struct definition and a display function (see Chapter1\modules\ToDoApp.jl in the code repository):

module ToDoApp
using Dates   # to make the Date type available
export print_todo, ToDo
mutable struct ToDo
  id::Int32
  description::String
  completed::Bool
  created::Date
  priority::Int8
end
function print_todo(todo)
  if !todo.completed
    println("I still have to do: $(todo.description)")
    print("A todo created at: ")
    helper(todo)
  end
end
function helper(todo)
   println(todo.created)
end
end

In the preceding code, we see that using is needed to bring in the definitions of the Dates module.

In the REPL, we evaluate the preceding Julia script with include(" ToDoApp.jl"). Then, we employ using .ToDoApp.

The period (.) is used here because we want to look for definitions inside the scope of the current module. Without this, we get an error:

julia> using ToDoApp
ERROR: ArgumentError: Package ToDoApp not found in current path
Import

Also, we must do using Dates so that the Date type is recognized, which is needed when making a ToDo instance:

Now, let us define our struct instance as follows:

julia> todo1 = ToDo(1, "Getting groceries", false, Date("2022-04-01", "yyyy-mm-dd"), 5)
Main.ToDoApp.ToDo(1, "Getting groceries", false, Date("2022-04-01"), 5)

We can now call the exported print_todo function:

julia> print_todo(todo1)
I still have to do: Getting groceries
A todo created at: 2022-04-01

However, the helper function is not available because it was not exported from the module:

julia> helper(todo1)
ERROR: UndefVarError: helper not defined

But we can call the helper function as follows:

ToDoApp.helper(todo1) # => 2022-04-01

When evaluating using, Julia looks in the filesystem for modules or packages in paths that are stored in the LOAD_PATH variable. By default, LOAD_PATH contains the following:

julia> LOAD_PATH
3-element Vector{String}:
"@"
"@v#.#"
"@stdlib"

The preceding code implies that first the current project is searched, then the default Julia environment, and then the standard library.

The variable @__DIR__ contains the current folder. So, another way to enable Julia to search for modules or packages in the current folder is to say push!(LOAD_PATH, @__DIR__).

Let us summarize when and how to use using:

• using MyPackage looks in the LOAD_PATH for a file called MyPackage.jl and loads the module contained in that file; all exported definitions are loaded into the current scope
• using .MyPackage: This part of the code instructs looking for definitions inside the scope of the current module, which is needed because we have previously done include ("MyPackage.jl")
• using ..MyPackage: This part of the code instructs looking for definitions inside the parent scope of the current module

Besides include and using, we can also bring in a module with import. Then, you have to prefix the name of a function or another object with its module name when it is used. For example, after import Inflector has imported the Inflector module, you have to use its to_plural function, as Inflector.to_plural(name).

The import keyword also has to be used when you want to extend functions with new methods. For example, if you want to pretty-print your own types with a new version of the show function, you first have to do import Base.show.

To bring in specific definitions, use : after using or import as follows:

import SearchLight: AbstractModel

As an example, here are the starting lines of the Genie module:

module Genie
import Inflector
include("Configuration.jl")
using .Configuration
const config = Configuration.Settings()
include("constants.jl")
import Sockets
import Logging
using Reexport
Using Revise
# rest of the code
end

Packages and projects

Packages are managed using the Git version control system and the package manager, Pkg (which is itself a package!). They are stored on GitHub, and each Julia package is named with a .jl suffix. A single GitHub repository may host one or more packages, but a good convention is one repository containing just one package.

A single package with the name P will always contain a P.jl file. By convention, this is placed in a subfolder, src. You can’t have other top-level modules in a single package.

As an example, the Genie framework, called Genie.jl, can be found at https://github.com/GenieFramework/Genie.jl.

A project is a package that contains two .toml files, which declare the packages your project depends on. You can create a project from the REPL as follows:

(@v1.8) pkg> generate MyPackage
Generating project MyPackage:
MyPackage\Project.toml
MyPackage\src/MyPackage.jl

The output will show the file structure created by the generate command.

The default Project.toml file contains the following:

name = "MyPackage"
uuid = "607adcac-db05-4b5b-9d7e-b11c396083d4"
authors = ["YourName<email-address>"]
version = "0.1.0"

A project can also contain a [deps] section, containing the names and universally unique ids (UUID) of the packages your project depends on (we will see an example of this in the next section). When adding a package to your project with the add command, the entry in the [deps] section is automatically filled in. The [compat] section constraints compatibility for the dependencies listed under [deps].

Besides Project.toml, a project can also have a manifest in the form of a Manifest.toml file, as indeed all Genie projects have. This file is generated and maintained by Pkg and, in general, should never be modified manually. The Manifest.toml file records the state of the packages in the current project environment, including exact information about (direct and indirect) dependencies of the project.

Given these two files, you can exactly reproduce the package dependency environment of a project, so this guarantees reproducibility.

Parsing a CSV file

As a simple example of how to work with packages, let’s read the data from a CSV file and display it. Suppose we have a todos.csv file that contains a header line with column names, and then line by line, the field data of our to-dos, as follows:

id, description, completed, created, priority
1, "Getting groceries", true, "2022-04-01", 5
2, "Visiting my therapist", false, "2022-04-02", 4
3, "Getting a haircut", true, "2022-03-28", 6
4, "Paying the energy bill", false, "2022-04-04", 8
5, "Blog on workspace management", true, "2022-03-29", 4
6, "Book a flight to Israel", false, "2022-04-04", 3
7, "Conquer the world", true, "2022-03-29", 1

Start up a REPL to work with the data. We already installed the CSV package previously in the Using the package mode to jump-start a project section. We’ll also need the DataFrames package to show our data as a table with columns, so go into pkg mode by typing ] and give the command: add DataFrames.

Going back to the normal REPL, type the following using command:

using CSV, DataFrames

Now, we can read in the CSV file into a DataFrame object, df, with the following command:

df = CSV.read("todos.csv", DataFrame)

You will get the following output in the REPL:

Figure 1.3 – Viewing a CSV file in a DataFrame

If the file has no header line, specify the header=false keyword argument. Also, if the data delimiter is something different, such as ;, you can specify this with delim=';'.

The CSV package has a lot more capabilities for reading and writing, which you can learn about here: https://csv.juliadata.org/stable/index.html.

Now that you’ve seen how to use modules, packages, and projects, let’s examine Julia’s internal workings a bit more.