Managing packages

Julia has a built-in package manager that allows you to fully control the combination of packages that your project can use.

In this recipe, we explain the fundamentals of how to manage packages in a default (global) project. In the Managing project dependencies recipe in Chapter 8, Julia Workflow, we discuss how you can customize which packages you use in your local project repositories.

Getting ready

In this recipe, we will use the Julia command line. Make sure that your computer is connected to the internet.

In the GitHub repository for this recipe, you will find the commands.txt file that contains the presented sequence of shell and Julia commands.

Now open your favorite terminal to execute the commands.

How to do it...

Here is a list of steps to be followed:

1. Go to the Julia command line:

julia> 

2. Press ] to switch to the package manager mode:

(v1.0) pkg>

3. We can check the initial status of packages using the status command:

(v1.0) pkg> status
    Status `~/.julia/environments/v1.0/Project.toml`

We can see that we currently have a clean environment with no additional packages installed.

4. To add the BenchmarkTools package, use the add command:

(v1.0) pkg> add BenchmarkTools
   Cloning default registries into /home/ubuntu/.julia/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed BenchmarkTools ─ v0.4.1
 Installed JSON ─────────── v0.19.0
  Updating `~/.julia/environments/v1.0/Project.toml`
  [6e4b80f9] + BenchmarkTools v0.4.1
  Updating `~/.julia/environments/v1.0/Manifest.toml`
  [6e4b80f9] + BenchmarkTools v0.4.1
  [682c06a0] + JSON v0.19.0
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8ba89e20] + Distributed
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode

5. Now, use the status command to see the versions of your installed packages:

(v1.0) pkg> status
    Status `~/.julia/environments/v1.0/Project.toml`
  [6e4b80f9] BenchmarkTools v0.4.1

Notice that only the BenchmarkTools package is visible, although more packages have been installed by Julia. They reside in the package repository but are not visible to the user unless explicitly installed. Those packages are dependencies of the BenchmarkTools package (directly or via recursive dependency).

6. After installing a package, we precompile the installed packages:

(v1.0) pkg> precompile
Precompiling project...
Precompiling BenchmarkTools
[ Info: Precompiling BenchmarkTools [6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf]

7. Exit the package manager mode by pressing the Backspace key:

(v1.0) pkg>

julia>

8. Now, we can check that the package can be loaded and used:

julia> using BenchmarkTools

julia> @btime rand()
  4.487 ns (0 allocations: 0 bytes)
0.07253910317708079

9. Finally, install the BSON package version v0.2.0 (this is not the latest version of this package, as as of writing of the book the currently released version is v0.2.1). Switch to the PackageManager mode by pressing ] and then type:

(v1.0) pkg> add [email protected]
[output is omitted]

Now you have version 0.2.0 of the BSON package installed

10. Often you want to keep the version of some package fixed to avoid its update by the Julia Package Manager, when its new version is released. You can achieve it with the pin command as follows

(v1.0) pkg> pin BSON
[output is omitted]

11. If you decide that you want to allow the Julia Package Manager to update some package that was pinned you can do it using the free command:

(v1.0) pkg> free BSON
[output is omitted]

The process of installing a specified version of some package (step 9 of the recipe) and pinning it (step 10) might be useful for you when you will need to install the exact versions of the packages that we use in this book, as in the future new releases of the packages might introduce breaking changes. The full list of packages used in this book along with their required versions is given in the To get the most out of this book section in the Preface of this book.  

How it works...

For each environment, Julia keeps information about the required packages and their versions in the Project.toml and Manifest.toml files. For the global default environment, they are placed in the ~/.julia/environments/v1.0/ folder. The first file contains a list of installed packages along with their UUIDs (see https://en.wikipedia.org/wiki/Universally_unique_identifier or https://www.itu.int/ITU-T/studygroups/com17/oid.html). The second file describes detailed dependencies of the project with exact versions of the packages used.

In this recipe, we have only made use of the basic commands of the package manager. In most cases, this is all a regular user needs to know. There are many more commands available, which you can find using the help command in package manager mode. Some of the potentially useful commands are as follows:

• add: installs the indicated package
• rm: removes the indicated package
• up: updates the indicated package to a different version
• develop: Clones the full package repository locally for development (useful in circumstances such as when wanting to use the latest master version of the package)
• up: Updates packages in the manifest (be warned that updating packages in your project might lead to incompatibilities with old code, so use this with caution)
• build: Runs the build script for packages (useful as sometimes installing packages fails to correctly build them, for example, due to external dependencies that have to be configured)
• pin: Pins the version of packages (ensures that a given package version is not changed by using other commands)
• free: Undoes a pin, develop, or stops tracking a repository

There's more...

All the commands we have described are also available programmatically using functions from the Pkg.jl package. For instance, running Pkg.add("BenchmarkTools") would install the package the same way as writing add BenchmarkTools in package manager mode in the Julia console. In order to use those functions, you have to load the Pkg package using using Pkg first.

In addition, it is important to know that many packages are preinstalled with Julia and thus do not require installation and can be directly loaded with the using command. Here is a selection of some of the more important ones, along with a brief description of their functionality:

• Dates: Works with date and time features
• DelimitedFiles: Basic loading and writing of delimited files
• Distributed: Multiprocessing
• LinearAlgebra: Various operations on matrices
• Logging: Support for logging
• Pkg: Package manager
• Random: Random number generation
• Serialization: Support for serialization/deserialization of Julia objects
• SparseArrays: Support for non-dense arrays
• Statistics: Basic statistical functions
• Test: Support for writing unit tests

Extensive coverage of these packages is given in the standard library section of the Julia manual, which can be accessed at https://docs.julialang.org/en/v1.0/.

See also

The true power of the Julia package manager is realized when you need to have different versions of packages for your different projects on the same machine. In the Managing project dependencies recipe in Chapter 8, Julia Workflow, we discuss how you can achieve this.