Package management

Julia provides a built-in package manager. Using Pkg, we can install libraries written in Julia. For unregistered packages, we can also compile them from their source or use the standard package manager of the operating system. A list of registered packages is maintained at http://pkg.julialang.org. The Pkg module is provided in the base installation. The Pkg module contains all the package manager commands.

Pkg.status() – package status

The Pkg.status() is a function that prints out a list of currently installed packages with a summary. This is handy when you need to know whether the package you want to use is installed or not. When the Pkg command is run for the first time, the package directory is automatically created. The command requires that the Pkg.status() function returns a valid list of the packages installed. The list of packages given by the Pkg.status() function are of registered versions, which are managed by Pkg. We currently have lots of packages installed, whose versions we can check, given as follows:

The Pkg.installed() function can also be used to return a dictionary of all the installed packages with their versions:

julia> Pkg.installed()
Dict{String,VersionNumber} with 54 entries:
 "Juno"              => v"0.2.5"
 "Lazy"              => v"0.11.4"
 "ZMQ"               => v"0.4.0"
 "DataStructures"    => v"0.4.6"
 "Compat"            => v"0.9.4"
 "RData"             => v"0.0.4"
 "GZip"              => v"0.2.20"
 ...

You can see that the number of packages is the same, but the way the information is presented is different. Packages can be incomplete or in more complicated states. These states are generally indicated by annotations to the right of the installed package version. Pkg.status() will print branch information to the right of the version.  

Pkg.add() – adding packages

Julia's package manager is declarative and intelligent. You only have to tell it what you want and it will figure out which version to install and resolve dependencies if there are any.

We can use Pkg.add(package_name) to add packages and Pkg.rm(package_name) to remove packages. This is the recommended way.

Otherwise, we can add the list of requirements that we want and it resolves which packages and their versions to install. The ~/.julia/v0.5/REQUIRE file contains the package requirements. We can open it using a text editor such as vi or atom, or use the Pkg.edit() function in Julia's shell to edit this file:

$ cat ~/.julia/v0.5/REQUIRE
IJulia
ScikitLearn
RDatasets
Plots
UnicodePlots
PyPlot
PlotlyJS

After editing the file, run Pkg.resolve() to install or remove the packages. This installation and removal will be based on the packages mentioned in the REQUIRE file.

Remember earlier we used Pkg.add("IJulia") to install the IJulia package? When we don't want to have a package installed on our system anymore, Pkg.rm() is used to remove the requirement from the REQUIRE file. Similar to Pkg.add(), Pkg.rm() first removes the requirement of the package from the REQUIRE file and then updates the list of installed packages by running Pkg.resolve() to match. Pkg.add() and Pkg.rm() are convenient and are generally used to add and remove requirements for a single package. But when there is a need to add or remove multiple packages, we can call Pkg.edit() to manually change the contents of REQUIRE and then update our packages accordingly. The Pkg.edit() function does not roll back the contents of REQUIRE if Pkg.resolve() fails. In that scenario, we have to run Pkg.edit() again to fix the file's contents ourselves.

Working with unregistered packages

Often, we would like to be able to use packages created by our team members or someone who has published them on GitHub but that are not in the registered packages of Pkg. Julia allows us to do that by using GitHub clones. Julia packages are hosted on GitHub repositories and can be cloned using mechanisms supported by Git. The index of registered packages is maintained at METADATA.jl. For unofficial packages, we can use the following:

julia> Pkg.clone("git://github.com/path/unofficialPackage/Package.jl.git")

Julia repository names end with .jl on GitHub (the additional .git indicates a bare GitHub repository). This prevents Julia and other languages from colliding with repositories with the same name. This also makes the search easy. Sometimes, unregistered packages have dependencies that require fulfilling before use. If that is the scenario, a REQUIRE file is needed at the top of the source tree of the unregistered package. The unregistered packages' dependencies on the registered packages are determined by this REQUIRE file. When we run Pkg.clone(url), these dependencies are automatically installed.

Pkg.update() – package update

It's good to have updated packages. Julia, which is under active development, has its packages frequently updated and new functionalities are added. To update all of the packages, type the following:

Pkg.update()

Under the hood, new changes are pulled into the METADATA file in the directory located at ~/.julia/v0.5/, and checks are run for any new registered package versions that may have been published since the last update. If there are new registered package versions, Pkg.update() attempts to update the packages that are not dirty and are checked out on a branch. This update process satisfies top-level requirements by computing the optimal set of package versions to be installed. The packages with specific versions that must be installed are defined in the REQUIRE file in Julia's directory (~/.julia/v0.5/).  

METADATA repository

Registered packages are downloaded and installed using the official METADATA.jl repository. A different METADATA repository location can also be provided if required:

julia> Pkg.init("https://julia.customrepo.com/METADATA.jl.git", "branch")

Developing packages

Julia allows us to view the source code, and, as it is tracked by GitHub, the full development history of all the installed packages is available. We can also make changes and commit to our own repository, or do bug fixes and contribute enhancements upstream. You may also want to create your own packages and publish them at some point in time. Julia's package manager allows you to do that too. It is a requirement that GitHub is installed on the system and the developer needs an account with their hosting provider of choice (GitHub, Bitbucket, and so on). Having the ability to communicate over SSH is preferred—to enable that, upload your public ssh-key to your hosting provider.

Creating a new package

It is preferable to have the REQUIRE file in the package repository. This should have the bare minimum of a description of the Julia version. For example, if we would like to create a new Julia package called HelloWorld, we would have the following:

Pkg.generate("HelloWorld", "MIT")

Here, HelloWorld is the package that we want to create and MIT is the license that our package will have. The license should be known to the package generator. Pkg.generate currently knows about MIT, BSD, and ASL but a user can use any license they want by changing LICENCE.md.

The above process creates a directory at ~/.julia/v0.5/HelloWorld. The directory that is created is initialized as a GitHub repository. Also, all the files required by the package are kept in this directory. This directory is then committed to the repository and can then be pushed to the remote repository for the world to use.