The Julia package ecosystem is under continuous development and new package versions are released on a daily basis. Most of the times this is great news, as new releases bring new features and bug fixes. However, since many of the packages are still in beta (version 0.x), any new release can introduce breaking changes. As a result, the code presented in the book can stop working. In order to ensure that your code will produce the same results as described in the book, it is recommended to use the same package versions. Here are the external packages used in this chapter and their specific versions:

 [email protected]
[email protected]
[email protected]

In order to install a specific version of a package you need to run:

pkg> add [email protected] 

For example:

pkg> add [email protected]

Alternatively you can install all the used packages by downloading the Project.toml file provided with the chapter and using pkg> instantiate as follows:

julia> download("https://raw.githubusercontent.com/PacktPublishing/Julia-Programming-Projects/master/Chapter01/Project.toml", "Project.toml")
pkg> activate . 
pkg> instantiate

In a nutshell, Julia truly is a new breed of programming language that successfully manages to combine the high performance of compiled languages with the agility of the dynamic ones, through a friendly syntax that feels natural and intuitive right from the start. Julia is fast (programs are compiled at runtime to efficient native code for multiple platforms), general (the standard library supports, out of the box, powerful programming tasks including asynchronous I/O, process control, parallel, and distributed computing, logging, profiling, package management, and more), dynamic and optionally typed (it is dynamically-typed with optional type declarations and comes with a powerful read-eval-print loop (REPL) for interactive and exploratory coding). It is also technical (excelling at numerical computing) and composable (thanks to its rich ecosystem of packages that are designed to work together seamlessly and with high performance).

Although initially it focused on addressing the needs of high-performance numerical analysis and computational science, recent releases have positioned the language in the area of general computing, with many classes of specialized functions being moved out of the core into dedicated modules. As such, it is also a great fit for client and server-side programming, due to its powerful capabilities for concurrent, parallel, and distributed computing. 

Julia implements a type system based on parametric polymorphism and multiple dispatch, it is garbage-collected, uses eager evaluation, packs a powerful regular expression engine, and can call C and Fortran functions without glue code. 

Let's take a look at the most important features of the language, the parts that make Julia stand out. If you're considering Julia for your next project, you can use this as a quick checklist against your requirements.

The key to Julia's performance is the combination between the LLVM-based just-in-time (JIT) compiler and a series of strategic design decisions that allow the compiler to generate code that approaches, and most of the times matches, the performance of C.

To give you an idea of where Julia stands in this regard, the official website provides a series of micro-benchmarks against other mainstream languages (including C, Python, R, Java, JavaScript, Fortran, Go, Rust, MATLAB, and Octave) implementing some algorithms for computing the Fibonacci sequence, the Mandelbrot set, a quicksort and a few others. They are designed to evaluate compiler performance against common code patterns such as function calls, string parsing, sorting, iterations, recursion, and more. There is a plot of the benchmarks, available at https://julialang.org/benchmarks/, which illustrates Julia's consistent top performance across all of the tests. The following plot depicts this:

The creators of Julia have carefully picked the most successful elements of syntax from other languages, with the goal of producing expressive, concise, and readable code. Julia provides powerful and expressive language constructs for high-level numerical computing, in the same way as languages such as R, MATLAB, and Python do. It builds upon the experience brought by existing mathematical programming languages but also borrows much from popular dynamic ones, such as Lisp, Perl, Python, Lua, and Ruby.

To give you a quick taste of idiomatic Julia, here's how to open a file, read it, output it, and then have the file automatically closed by Julia:

open(".viminfo") do io
    read(io, String) |> println
end  

In the preceding snippet, we open the .viminfo file for reading passing io, an IOStream instance, into the underlying code block. The stream is then read into a String that is finally displayed onto the console by piping it into the println function. The code is very readable and easy to understand if you have some coding experience, even if this is your first time looking at Julia code.

This so-called do syntax (named after the do part following the open function) is inspired by Ruby's blocks—and it is, in fact, syntactic sugar for passing anonymous functions as method arguments. It is efficiently used in the preceding example to succinctly express a powerful design pattern for safely handling files, guaranteeing that the resources are not accidentally left open.

This goes to show the amount of attention that was put by the designers of the language to make Julia safe, beginner-friendly, expressive, concise, readable, and intuitive.

Julia's type system is a key feature of the language and one that has a major impact on both its performance and productivity. The type system is dynamic and optional, meaning that the developer can, but is not required to, provide type information to the compiler. If not provided, Julia will perform type inference, which is the process of deducing the types of later values from the types of input values. This is a very powerful technique, as it frees the programmers from having to worry about types, allowing them to focus on the application logic and making for a gentler learning curve. This is especially useful for prototyping and exploratory programming, when the complete set of constraints and requirements is not known beforehand.

However, understanding and correctly using the type system offers important performance benefits. Julia allows optionally adding type information, making it possible to indicate that a certain value must be of a specific kind. This is one of the cornerstones of the language, allowing performant method dispatching and facilitating the automatic generation of efficient, specialized code for different argument types. The type system allows the definition of rich type hierarchies, with user-defined types as fast and compact as the built-in ones.

If the languages of the 70s and 80s were designed under the strict requirements imposed by the limited CPU and RAM resources, the ones in the 90s and the 2000s had the optimistic outlook that these resources are forever expanding. However, the last decade had seen something of a stagnation in this regard, with a shift toward multi-CPU, multi-core, and distributed computing. In this regard, Julia's inception only 6 years ago gave it an edge compared to older languages, putting parallel and distributed computing at its center as one of its most important features.

One of the most serious barriers in the adoption of a new language is that it takes time for the ecosystem to catch up—and in the beginning, it cannot offer libraries of the quality and richness of the already established languages. This is less of an issue now, when Julia benefits from a large, enthusiastic and continuously growing developer community. But being able to seamlessly communicate with other languages is a very efficient way to enrich existing functionality and to effortlessly supplement any missing features.

Julia has the ability to directly call C and Fortran functions (that is, without glue code)—especially important for scientific computing, where these languages have a strong presence and a long history.

Optional packages extend this capability by adding support for calling functions written in other languages, most notably Python, via PyCall. And there are others, supporting interaction with Java, C++, MATLAB, Rust, and more.

The REPL represents a language shell, an interactive computer programming environment at the command line. Julia has an excellent REPL, supporting sophisticated code inputting and evaluation. It includes powerful editing features such as searchable history, tab-completion, and syntax highlighting, to name just a few.

It also comes with three special modes—shell, which allows executing commands as if at the OS Terminal; help, which provides access to documentation without leaving the REPL; and pkg, used for installing and managing application dependencies.

Julia comes with its own powerful package manager that resolves dependencies and handles the adding and removal of extra packages. Like most modern languages, Julia has full support for Unicode. And finally, it is distributed under the liberal MIT license—it is free and open source.

If the previous section convinced you to use Julia for your next project, or at least made you curious to learn more, then it's time to set up your Julia development environment.

Julia has superb cross-platform support, running on all major operating systems. The install process is straightforward—the language can be set up on your local machine, in a virtual machine (VM), in a Docker container, or on a server somewhere in the cloud.

Let's start by looking at local installation options for the big three operating systems (Windows, Linux, and macOS). Feel free to skip directly to the right one for you.

Windows has come a long way as a development platform and there are a few good alternatives for getting Julia up and running.

The simplest option is to download the Windows installer corresponding to your platform (32 or 64-bit), from https://julialang.org/downloads/. Get the .exe and run it. Follow the standard installation procedure and in the end, you will have Julia installed as a program. Double-clicking julia.exe opens a command prompt with the Julia REPL, just like the one illustrated here:

Chocolatey is a package manager for Windows, similar to apt or yum on Linux, or brew on Mac. If you don't have it, get it by following the instructions at https://chocolatey.org.

Chocolatey has the latest version of Julia, as can be confirmed with a search, as follows:

$ choco search julia 
Chocolatey v0.10.11 
Julia 1.0.0 [Approved] 
1 packages found. 

Installing is as simple as this:

$ choco install julia 
Chocolatey v0.10.11 
Installing the following packages: 
julia 
By installing you accept licenses for the packages. 
Progress: Downloading Julia 1.0.0... 100% 
Julia v1.0.0 [Approved] 
Chocolatey installed 1/1 packages. 

One of the recent additions to Windows 10 is the Subsystem for Linux. This allows setting up a Linux development environment, including most command-line tools, utilities, and applications—directly on Windows, unmodified, and without the overhead of running a VM.

In order to be able to use the Linux Subsystem your PC must be running the 64-bit version of Windows 10 Anniversary Update or later (build 1607+). It also needs to be enabled first—so open a PowerShell as an administrator and run the following:

$ Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux 

Once the subsystem is enabled (computer restart might be required) you can choose one of the Linux versions available, directly from the Windows Store. At the time of writing, five versions were available—Ubuntu, openSUSE , SLES, Debian, and Kali.

Ubuntu is the default option for Windows 10 and has the best user ratings in the Windows Store, so let's go with that. It can be installed from https://www.microsoft.com/en-us/store/p/ubuntu/9nblggh4msv6. Alternatively, you can just open a command prompt and type $ bash. This will trigger the installation of the Ubuntu Linux Subsystem.

Once you find yourself at the shell prompt of your Linux subsystem, you can proceed and issue the commands for installing Julia. For Ubuntu you need to run the following:

$ sudo apt-get install julia

Make sure to confirm the required selections—then after a couple of minutes you should have Julia up and running.

Installing Julia on macOS is straightforward. There are two main options, depending on whether you prefer a visual installer or are more at home in front of the Terminal prompt.

Visit https://julialang.org/downloads/ and look for the macOS package (.dmg). Once it's downloaded, double-click the .dmg file and drag and drop the Julia app into the /Applications folder. Now you can simply open the Julia app—which in turn will launch a new Terminal session, loading the Julia environment, as follows:

Homebrew is a well-known package manager for macOS in the line of apt and yum on Linux. It's not really necessary for installing Julia, but it's worth setting it up as it can be very useful during development since it can seamlessly install database servers, libraries, and other components for your projects.

As per the instructions at https://brew.sh, it can be installed by running the following command in a Terminal window:

$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" 

It might take a while but once Homebrew is installed, a new command-line utility, brew, will become available.

Finally, $ brew cask install julia will download and install the latest version of Julia. In the process, it will also link the julia binary to /usr/local/bin/julia so you can interact with the language from the command line by simply typing $ julia.

As soon as you get the confirmation that the installation was successful you can run $ julia to start a new REPL session:

Julia is already available in the software repositories of the major Linux distributions, but unfortunately, these are not up to date. For example, at the time of writing, Ubuntu was providing v0.4.5 and Debian v0.4.7. The best approach is to use the generic Linux binaries provided on Julia's download page, at https://julialang.org/downloads/. 

Please follow the instructions corresponding to your Linux distribution, as indicated at https://julialang.org/downloads/platform.html#generic-binaries. 

Docker is a software technology that provides an additional layer of abstraction of operating-system-level virtualization. In plain English, Docker sets up containers that behave like VMs, but without the added overhead of starting and maintaining VMs. You can run Docker on all the major operating systems. 

Docker is widely used as a development and deployment strategy, so many technologies are readily available in the form of Docker images, and Julia is no exception.

Start by installing Docker for your platform. The official Julia container can be found in the Docker store at https://store.docker.com/images/julia. Go get it.

At the command prompt, type $ docker pull julia. Once the Julia image is configured by Docker, run it with $ docker exec -it --rm julia. This will start the container and load a new Julia REPL:

Julia Computing, the company behind the Julia programming language, offers a batteries included distribution. It's called JuliaPro and it's arguably the easiest way to get started with Julia straight away. It includes the compiler, a profiler, the Juno IDE, and over 160 top quality curated packages for plotting, data visualization, machine learning, databases, and more.

JuliaPro can be downloaded for free at https://shop.juliacomputing.com/Products/ (registration required). Once you get it, follow the install process specific to your platform. When done you'll have everything needed to begin using Julia productively.

Finally, there's also JuliaBox (https://www.juliabox.com), another free offering from Julia Computing. JuliaBox allows running a Julia Docker container on the fly, in their cloud. It provides access to IJulia Jupyter notebooks (https://github.com/JuliaLang/IJulia.jl), file sync with Google Drive, importing GitHub repositories, and many other features.

An IDE is very important when working with a programming language. A powerful source code editor, code completion, and a good linter and debugger can significantly influence the learning curve and the productivity of using a language. You will be happy to learn that there are some very good IDE and editor options for Julia—and chances are you'll find your favorite one among these.

The IDE choices reflect the pragmatism of the language as a whole. From choosing LLVM as the compiler to providing efficient ways for calling functions from other languages, to using git and GitHub to power the package manager, the Julia core team takes a don't reinvent the wheel approach. Following the same line of thinking, the Julia community has built powerful IDEs upon existing industry established editors, such as Atom and Visual Studio Code.

Juno (http://junolab.org) is the most advanced Julia IDE and the de facto editor of choice for Julia professionals. It is based on the Atom editor and it can be considered the official development tool, being also distributed with the previously mentioned JuliaPro distribution.

To get it, either download and install JuliaPro from https://juliacomputing.com/products/juliapro.html or do a manual install of Atom and the required plugins.

If you choose the manual install, first you need to download Atom from https://atom.io. Once it's up and running, go to the Settings pane (you can use the shortcut Ctrl/cmd and ,) and then go to the Install panel. Type uber-juno into the search box and press Enter. Next, click the install button on the package with the same name. Atom will pick it up from here, installing all the required Atom and Julia packages.

Once configured, the IDE options will be available in Atom's menu, under Packages > Julia. Various panes can also be enabled from here, to list variables, visualize plots, or search the documentation.

For further information, check out http://junolab.org and https://github.com/JunoLab/uber-juno/blob/master/setup.md.

Visual Studio Code is a cross-platform extendable editor from Microsoft. It is available for all the big three platforms at https://code.visualstudio.com. Once installed, run it and from the menu click View > Extensions or use the shortcut Shift and Ctrl/cmd and X. Search for julia and install the Julia extension from julialang.

The Julia support in Visual Studio Code is not (yet) as powerful as Juno, but if you prefer it, it makes for a great coding experience, providing syntax highlighting, code completion, hover help, evaluation of Julia code, linting, code navigation, and more. Visual Studio Code is also snappier and uses fewer resources than Atom, which makes it an appealing option when running on a less powerful workstation (although Atom has greatly improved in this regard with recent versions).

The extension might need a bit of help figuring out where it can find the Julia binary. If that is the case, you'll get an informative error message, asking you to set the julia.executablePath configuration option. This should point to the julia binary, and depends on your operating system and the way you installed Julia (see the previous section for details on the installation).

To set the configuration, go to Preferences > Settings (Ctrl/cmd and ,) and in the right pane, the one used to overwrite the defaults, add the following:

"julia.executablePath": "/path/to/your/julia/folder/bin/julia" 

We already mentioned JuliaBox (https://www.juliabox.com) in the previous section—it allows creating, editing, and running IJulia Jupyter notebooks in the cloud. IJulia can also be installed on the local development machine.

IJulia is a Julia language backend for the Jupyter interactive environment (also used by IPython). It allows us to interact with the Julia language using Jupyter/IPython's powerful graphical notebook, which combines code, formatted text, math, and multimedia in a single document.

Although IJulia/Jupyter is not really an IDE, nor a classical editor, it is a powerful environment for editing and executing Julia scripts, and it's especially popular for data science and scientific computing. Let's take a few moments to set it up.

Start a new Julia REPL and execute the following:

julia> using Pkg
julia> Pkg.add("IJulia")

This will install the IJulia package, while also adding a required minimal Python and Jupyter distribution called Miniconda. This Python distribution is private to Julia (not in your PATH). Once finished, continue by executing the following:

julia> using IJulia
julia> notebook()

This will open the home page of your local Jupyter install in your default browser, at http://localhost:8888/tree. From the toolbar choose New > Julia 1.0.0 (or whatever version you are currently running) to create a new notebook. You can now create rich documents using embedded executable Julia code.

If you're new to Jupyter, it's worth learning more about it. Go check it out at http://jupyter.org.

The preceding choices are the most common IDE and editor options for Julia. But there are a few more out there.

For the vim enthusiasts, there's also julia-vim (https://github.com/JuliaEditorSupport/julia-vim).

If you prefer Emacs, you'll be pleased to know that Julia supports it as well https://github.com/JuliaEditorSupport/julia-emacs.

If you'd rater go with one of the IDEs provided by JetBrains (like IntelliJ IDEA), you'll be happy to hear that a plugin is available, at https://plugins.jetbrains.com/plugin/10413-julia

Finally, there is also support for Sublime Text, available at https://github.com/JuliaEditorSupport/Julia-sublime. The plugin provides a good Julia editing experience, supporting syntax highlighting, code completion, and jumping to definition, among other things.

If you followed through the first part of the chapter, by now you should have a fully functional local Julia installation, the knowledge to start a Julia REPL session, and have your preferred IDE ready for coding. If that is not the case, please refer to the previous sections. From this point on we're getting down to business—it's time to write some Julia code!

The first thing we need to understand is how to use the powerful REPL. As a Julia developer, you'll spend a significant amount of time doing exploratory programming, interacting with the shell and the filesystem, and managing packages. The REPL will be your trusted sidekick. It's worth getting to know it well, it will save you a lot of time down the line.

The acronym REPL stands for read-eval-print loop. Simply put, it's a language-specific shell, an interactive coding environment that allows inputting expressions, evaluates them, and outputs the result.

REPLs are very useful as they provide a simple way to interact with the language, to try out ideas and prototype, facilitating exploratory programming and debugging. It is especially powerful in the context of data analysis, where one can quickly connect to a data source, load a data sample and then slice and dice, rapidly testing different hypothesis.

Julia provides an excellent REPL experience, with rich functionality that covers quick evaluation of Julia statements, searchable history, tab-completion, syntax highlighting, dedicated help and shell modes, to name just a few.

If you do not have a working Julia installation, please see the Installing Julia section.

Depending on your OS and your preferences, the REPL can be started by simply invoking $ julia with no arguments, or by double-clicking the julia executable.

You will be greeted with a screen like this one (the Julia version might be different than mine):

Now Julia is waiting for us to input our code, evaluating it line by line. You can confirm that by checking the Terminal prompt, which says julia>. This is called the julian mode. Let's take it for a spin.

Input the following lines, pressing Enter after each one:

julia> 2+2 
julia> 2^3 

So we can use Julia like a simple calculator. Not very useful, but this is only the beginning and illustrates how powerful this rapid input and feedback cycle can be when we deal with complex computations.

println is a very useful function that prints whatever value it receives, appending a new line afterward. Type the following code:

julia> println("Welcome to Julia") 

Under each line, you should see the output generated by each expression. Your window should now look like this.

julia> 2+2 
4 
julia> 2^3 
8 
julia> println("Welcome to Julia") 
Welcome to Julia 

Let's try some more. The REPL interprets one line at a time, but everything is evaluated in a common scope. This means that we can define variables and refer to them later on, as follows:

julia> greeting = "Hello" 
"Hello" 

This looks great! Let's use the greeting variable with println:

julia> println(greting) 
ERROR: UndefVarError: greting not defined 

Oops! A little typo there, and the REPL promptly returned an error. It's not greting, it's greeting. This also tells us that Julia does not allow using variables without properly initializing them. It just looked for the greting variable, unsuccessfully—and it threw an undefined variable error. Let's try that again, this time more carefully:

julia> println(greeting) 
Hello 

OK, that's much better! We can see the output: the Hello value we stored in the greeting variable.

The REPL provides a few helping features, specific to this interactive environment (they won't be available when executing a Julia script). One of these is the ans variable, automatically set up and updated by Julia.

If you type julia> 2^3—unsurprisingly, you'll get 8. Now input julia> ans—you'll get 8 again! What's up with that? ans is a special variable that exists only in the REPL and that automatically stores the last returned value. It can prove very useful when working with the REPL, but more importantly, you need to be aware of its existence so that you don't accidentally declare a variable with the same name. Otherwise, you'll run into some very hard to understand bugs with your variable's value constantly overwritten.

The REPL comes with a very powerful feature called prompt pasting. This allows us to copy-paste-execute Julia code and snippets that include both the julia> prompt and the output of the expression. It activates when pasting text that starts with julia>. In that case, only expressions starting with julia> are parsed, and all the others are ignored. This makes it possible to paste a chunk of code that has been copied from another REPL session or from the documentation, without having to scrub away prompts and outputs.

To see this in action, copy and paste the following snippet, as is:

julia> using Dates 
 
julia> Dates.now() 
2018-09-02T21:13:03.122 
julia> ans 
2018-09-02T21:13:03.122 

If all goes well, both expressions should output your current time, and not the one from the snippet, effectively replacing the results in the snippet with the results in your Julia session.

In both the Julian, pkg, and help modes you can press the Tab key after entering the first few characters of a function to get a list of all the matches:

julia> pri[TAB] 
primitive type   print             print_shortest    print_with_color  println           printstyled  

It can also be used to substitute LaTeX math symbols with their Unicode equivalents. To do this, type a backslash as the first character, then the first few characters of the symbol, then Tab. This will complete the name of the symbol or will display a list of options if there's more than one matching name. Pressing Tab again on the complete name of the symbol will perform the replacement:

julia> \pi[TAB] 
julia> π 
π = 3.1415926535897... 

julia> \om[TAB] \omega \ominus 
julia> \ome[TAB] 
julia> \omega[TAB] 
julia> ω 

Julia does not have the concept of null so you can't really deallocate a variable from memory. If, however, you need to free an expensive resource referenced by a variable, you can replace its value with something like 0 and the previous value will be automatically garbage collected. You can even invoke the garbage collector yourself straight away by calling gc().

The Julia REPL comes with four operational modes—and additional ones can be defined as needed. The currently active mode is indicated by its prompt. In the previous examples we've used the julian mode julia>, which evaluates the inputted expression. The other three available modes are help, help?>, shell, shell>, and package management, pkg>.

The active mode can be switched by typing a specific character right at the beginning of the line. The prompt will change in response, to indicate the current mode. The mode will stay active until the current line is evaluated, automatically switching back to julian (with the exception of the pkg> mode which is sticky—that is, it stays active until explicitly exited by typing backspace at the beginning of the line). The alternative modes can be exited without evaluating the expression by deleting everything on the line until the prompt changes back to julia>, or by pressing Ctrl + C.

The help mode provides access to documentation without having to get out of the REPL. To access it, simply type ? at the beginning of the line. You should see the  help?> prompt. Now you can input text, and Julia will search the documentation for matching entries, as follows:

julia> ?  
help?> println 
search: println printstyled print_with_color print print_shortest sprint isprint 
 
  println([io::IO], xs...) 
 
  Print (using print) xs followed by a newline. If io is not supplied, prints to stdout. 
 
  Examples 
  ≡≡≡≡≡≡≡≡≡≡ 
 
  julia> println("Hello, world") 
  Hello, world 
 
  julia> io = IOBuffer(); 
 
  julia> println(io, "Hello, world") 
 
  julia> String(take!(io)) 
  "Hello, world\n"  

The output supports rich formatting, via Markdown:

julia> using Profile 
help?> Profile.print 

Resulting a rich output as in the following screenshot:

More complex expressions can be queried, including macros, types, and variables.

For example, help?> @time:

Or help?> IO:

The shell mode is used to switch to a command-line interface similar to the system shell, for directly executing OS commands. To enter it, input a semicolon ; at the very beginning of the julian prompt:

julia> ;  

Upon typing ; the prompt changes (in place) to shell>:

Now we can execute system-wide commands directly, without the need to wrap them in Julia code. This will list the last ten lines of your repl_history.jl file. This file is used by Julia to keep a history of the commands executed in the REPL, so your output will be different from mine:

julia> using REPL
shell> tail -n 10 ~/.julia/logs/repl_history.jl
IO
# time: 2018-09-02 21:56:47 CEST
# mode: julia
REPL.find_hist_file()
# time: 2018-09-02 21:58:47 CEST
# mode: shell
tail -n 10 ~/.julia/logs/repl_history.jl 

While in REPL mode we can access Julia's API, making this a very powerful combo. For example, in order to programmatically get the path to the REPL history file, we can use the REPL.find_hist_file() function, as follows:

julia> REPL.find_hist_file() 
"/Users/adrian/.julia/logs/repl_history.jl" 

The path to the file will be different for you.

We can use this in the shell mode by wrapping the command in $():

shell> tail -n 10 $(REPL.find_hist_file()) 
    REPL.find_hist_file() 
# time: 2018-09-02 21:58:47 CEST 
# mode: shell 
    tail -n 10 ~/.julia/logs/repl_history.jl 
# time: 2018-09-02 22:00:03 CEST 
# mode: shell 
    tail -n 10 $(REPL.find_hist_file()) 

Similarly to the help mode, the shell mode can be exited without executing any command by pressing backspace at the beginning of the line or typing Ctrl + C.

In IJulia, the command can be executed by prefixing the input with ;, like this:

;tail -n 10 ~/.julia/logs/repl_history.jl 

Besides the help and the shell modes, there are two search modes. These are not necessarily Julia specific, being common to many *nix style editing apps.

Press the Ctrl key and the R key at the same time in order to initiate a reverse incremental search. The prompt will change to (reverse-i-search). Start typing your query and the most recent result will show. To find older results, type Ctrl + R again.

The counterpart of Ctrl + R is Ctrl + S, initiating an incremental search. The two may be used in conjunction to move through the previous or next matching results, respectively.

If you want to automatically execute some code every time you run Julia, you can add it to a special file called startup.jl. This file is not automatically created, so you'll have to add it yourself to your Julia configuration directory. Any code you add to it will be run by Julia each time it starts up. Let's have some fun and do this using Julia—and practice a bit of what we've learned so far.

First, go into shell mode and run these three commands:

shell> mkdir $(dirname(REPL.find_hist_file()))/../config 
 
shell> cd $(dirname(REPL.find_hist_file()))/../config 
/Users/adrian/.julia/config 
 
shell> touch startup.jl 

Then, in julian mode, execute the following:

julia> write("startup.jl", "println(\"Welcome to Julia!\")") 
28 

What did we just do? In shell mode, we created a new directory, called config, just one folder up from where our history file was. Then we cd into the newly created folder, where we created a new file called startup.jl. Finally, we asked Julia to add the line "println(\"Welcome to Julia!\")" to the startup.jl file. Next time we start the Julia REPL we'll be greeted by this welcome message. Check this out:

It is also possible to define a function that will be automatically called before starting a REPL session. To achieve this, you need to use the atreplinit(f) function, which registers a one-argument function f to be called before the REPL interface is initialized in interactive sessions. This function should be called from within the startup.jl file.

Let's say that we edit our startup.jl file so that it now looks like this:

println("Welcome to Julia!") 
 
atreplinit() do (f) 
  println("And welcome to you too!") 
end 

Our REPL will now greet us twice:

The atreplinit function can be used in tandem with isinteractive, which returns a Boolean true or false value that tells us whether or not Julia is running an interactive session.

In order to exit the REPL, you can type ^ D (Ctrl + D). However, that will only work if you're at the beginning of the line (when the text buffer is empty). Otherwise just type ^C (Ctrl + C) to first interrupt (or cancel) and clear the line. You can also run exit(), which will stop the execution of the current Julia process.

Your Julia installation comes with a powerful package manager called Pkg. This handles all the expected operations, such as adding and removing packages, resolving dependencies and keeping installed packages up to date, running tests, and even assisting with publishing our own packages.

Packages play a pivotal role by providing a wide range of functionality, seamlessly extending the core language. Let's take a look at the most important package management functions.

In order to be known to Pkg, the packages must be added to a registry that is available to Julia. Pkg supports working with multiple registries simultaneously—including private ones hosted behind corporate firewalls. By default, Pkg is configured to use Julia's General registry, a repository of free and open sources packages maintained by the Julia community.

Pkg is quite a powerful beast and we'll use it extensively throughout the book. Package management is a common task when developing with Julia so we'll have multiple opportunities to progressively dive deeper. We'll take our first steps now as we'll learn how to add packages—and we'll do this by stacking a few powerful new features to our Julia setup.

One of my favourite packages is called OhMyREPL. It implements a few super productive features for the Julia REPL, most notably syntax highlighting and brackets pairing. It's a great addition that makes the interactive coding experience even more pleasant and efficient.

Julia's Pkg is centered around GitHub. The creators distribute the packages as git repos, hosted on GitHub—and even the General registry is a GitHub repository itself. OhMyREPL is no exception. If you want to learn more about it before installing it—always a good idea when using code from third parties — you can check it out at https://github.com/KristofferC/OhMyREPL.jl 

Keep in mind that even if it's part of the General registry, the packages come with no guarantees and they're not necessarily checked, validated or endorsed by the Julia community. However, there are a few common sense indicators which provide insight into the quality of the package, most notably the number of stars, the status of the tests as well as the support for the most recent Julia versions.

The first thing we need to do in order to add a package is to enter the Pkg REPL-mode. We do this by typing ] at the beginning of the line:

julia>] 

The cursor will change to reflect that we're now ready to manage packages:

(v1.0) pkg> 

Pkg uses the concept of environments, allowing us to define distinct and independent sets of packages on a per-project basis. This is a very powerful and useful feature, as it eliminates dependency conflicts caused by projects that rely on different versions of the same package (the so-called dependency hell).

Given that we haven't created any project yet, Pkg will just use the default project, v1.0, indicated by the value between the parenthesis. This represents the Julia version that you're running on—and it's possible that you'll get a different default project depending on your very own version of Julia.

Now we can just go ahead and add OhMyREPL:

(v1.0) pkg> add OhMyREPL 
  Updating registry at `~/.julia/registries/General` 
  Updating git-repo `https://github.com/JuliaRegistries/General.git` 
 Resolving package versions... 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] + OhMyREPL v0.3.0 
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [a8cc5b0e] + Crayons v1.0.0 
  [5fb14364] + OhMyREPL v0.3.0 
  [0796e94c] + Tokenize v0.5.2 
  [2a0f44e3] + Base64 
  [ade2ca70] + Dates 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [76f85450] + LibGit2 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [44cfe95a] + Pkg 
  [de0858da] + Printf 
  [3fa0cd96] + REPL 
  [9a3f8284] + Random 
  [ea8e919c] + SHA 
  [9e88b42a] + Serialization 
  [6462fe0b] + Sockets 
  [8dfed614] + Test 
  [cf7118a7] + UUIDs 
  [4ec0a83e] + Unicode  

When running pkg> add on a fresh Julia installation, Pkg will clone Julia's General registry and use it to look up the names of the package we requested. Although we only explicitly asked for OhMyREPL, most Julia packages have external dependencies that also need to be installed. As we can see, our package has quite a few—but they were promptly installed by Pkg.

Sometimes we might want to use packages that are not added to the general registry. This is usually the case with packages that are under (early) development—or private packages. For such situations, we can pass pkg> add the URL of the repository, instead of the package's name:

(v1.0) pkg> add https://github.com/JuliaLang/Example.jl.git 
   Cloning git-repo `https://github.com/JuliaLang/Example.jl.git` 
  Updating git-repo `https://github.com/JuliaLang/Example.jl.git` 
 Resolving package versions... 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [7876af07] + Example v0.5.1+ #master (https://github.com/JuliaLang/Example.jl.git) 
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [7876af07] + Example v0.5.1+ #master (https://github.com/JuliaLang/Example.jl.git) 

Another common scenario is when we want to install a certain branch of a package's repository. This can be easily achieved by appending #name_of_the_branch at the end of the package's name or URL:

(v1.0) pkg> add OhMyREPL#master 
   Cloning git-repo `https://github.com/KristofferC/OhMyREPL.jl.git` 
  Updating git-repo `https://github.com/KristofferC/OhMyREPL.jl.git` 
 Resolving package versions... 
 Installed Crayons ─ v0.5.1 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0 #master (https://github.com/KristofferC/OhMyREPL.jl.git) 
  Updating `~/.julia/environments/v1.0/Manifest.toml`
  [a8cc5b0e] ↓ Crayons v1.0.0 ⇒ v0.5.1 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0 #master (https://github.com/KristofferC/OhMyREPL.jl.git)

Or, for unregistered packages, use the following:

(v1.0) pkg> add https://github.com/JuliaLang/Example.jl.git#master 

If we want to get back to using the published branch, we need to free the package:

(v1.0) pkg> free OhMyREPL 
 Resolving package versions... 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 #master (https://github.com/KristofferC/OhMyREPL.jl.git) ⇒ v0.3.0 
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [a8cc5b0e] ↑ Crayons v0.5.1 ⇒ v1.0.0 
  [5fb14364] ~ OhMyREPL v0.3.0 #master (https://github.com/KristofferC/OhMyREPL.jl.git) ⇒ v0.3.0 

That was easy, but practice makes perfect. Let's add one more! This time we'll install Revise, another must-have package that enables a streamlined development workflow by monitoring and detecting changes in your Julia files and automatically reloading the code when needed. Before Revise it was notoriously difficult to load changes in the current Julia process, developers usually being forced to restart the REPL—a time-consuming and inefficient process. Revise can eliminate the overhead of restarting, loading packages, and waiting for code to compile.

Unsurprisingly, we only have to invoke add one more time, this time passing in Revise for the package name:

(v1.0) pkg> add Revise 
Resolving package versions... 
 Installed Revise ─ v0.7.5 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [295af30f] + Revise v0.7.5 
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [bac558e1] + OrderedCollections v0.1.0
  [295af30f] + Revise v0.7.5 
  [7b1f6079] + FileWatching 

We can confirm that the operations were successful by checking our project's status, using the aptly named status command:

 (v1.0) pkg> status 
    Status `~/.julia/environments/v1.0/Project.toml` 
  [7876af07] Example v0.5.1+ #master (https://github.com/JuliaLang/Example.jl.git) 
  [5fb14364] OhMyREPL v0.3.0 
  [295af30f] Revise v0.7.5 

The status command displays all the installed packages, including, from left to right, the short version of the package's id (called the UUID), the name of the package and the version number. Where appropriate, it will also indicate the branch that we're tracking, as in the case of Example, where we're on the master branch.

Once a package has been added, in order to access its functionality we have to bring into scope. That's how we tell Julia that we intend to use it, asking the compiler to make it available for us. For that, first, we need to exit pkg mode. Once we're at the julian prompt, in order to use OhMyREPL, all we need to do is execute:

julia> using OhMyREPL
[ Info: Precompiling OhMyREPL [5fb14364-9ced-5910-84b2-373655c76a03]

That's all it takes—OhMyREPL is now automatically enhancing the current REPL session. To see it in action, here is what the regular REPL looks like:

And here is the same code, enhanced by OhMyREPL:

Syntax highlighting and bracket matching make the code more readable, reducing syntax errors. Looks pretty awesome, doesn't it?

OhMyREPL and Revise are excellent development tools and it's very useful to have them loaded automatically in all the Julia sessions. This is exactly why the startup.jl file exists—and now we have the opportunity to put it to good use (not that our heartfelt and welcoming greetings were not impressive enough!).

Here's a neat trick, to get us started—Julia provides an edit function that will open a file in the configured editor. Let's use it to open the startup.jl file:

julia> edit("~/.julia/config/startup.jl") 

This will open the file in the default editor. If you haven't yet deleted our previously added welcome messages, feel free to do it now (unless you really like them and in that case, by all means, you can keep them). Now, Revise needs to be used before any other module that we want to track—so we'll want to have it at the top of the file. As for OhMyREPL, it can go next. Your startup.jl file should look like this:

using Revise 
using OhMyREPL 

Save it and close the editor. Next time you start Julia, both Revise and OhMyREPL will be already loaded.

Julia boosts a thriving ecosystem and packages get updated at a rapid pace. It's a good practice to regularly check for updates with pkg> update:

(v1.0) pkg> update 

When this command is issued, Julia will first retrieve the latest version of the general repository, where it will check if any of the packages need to be updated.

Beware that issuing the update command will update all the available packages. As we discussed earlier, when mentioning dependency hell, this might not be the best thing. In the upcoming chapters, we will see how to work with individual projects and manage dependencies per individual application. Until then though, it's important to know that you can cherry pick the packages that you want to update by passing in their names:

(v1.0) pkg> update OhMyREPL Revise 

Pkg also exposes a preview mode, which will show what will happen when running a certain command without actually making any of the changes:

(v1.0) pkg> preview update OhMyREPL 
(v1.0) pkg> preview add HTTP 

Sometimes though we might want to ensure that certain packages will not be updated. That's when we pin them:

(v1.0) pkg> pin OhMyREPL 
 Resolving package versions... 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0

Pinned packages are marked with the ⚲ symbol—also present now when checking the status:

(v1.0) pkg> st 
    Status `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] OhMyREPL v0.3.0
  [295af30f] Revise v0.7.5 

If we want to unpin a package, we can use pkg> free:

(v1.0) pkg> free OhMyREPL 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0 
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0 
 
(v1.0) pkg> st 
    Status `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] OhMyREPL v0.3.0 
  [295af30f] Revise v0.7.5 

If you no longer plan on using some packages you can delete (or remove them), with the (you guessed it) pkg> remove command. For instance, let's say that we have the following configuration:

(v1.0) pkg> st 
    Status `~/.julia/environments/v1.0/Project.toml` 
  [7876af07] Example v0.5.1+ #master (https://github.com/JuliaLang/Example.jl.git) 
  [5fb14364] OhMyREPL v0.3.0 
  [295af30f] Revise v0.7.5 

We can remove the Example package with the following code:

(v1.0) pkg> remove Example 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [7876af07] - Example v0.5.1+ #master (https://github.com/JuliaLang/Example.jl.git) 
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [7876af07] - Example v0.5.1+ #master (https://github.com/JuliaLang/Example.jl.git) 

Sure enough, it's now gone:

(v1.0) pkg> st 
    Status `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] OhMyREPL v0.3.0 
  [295af30f] Revise v0.7.5 

Besides the explicit removal of undesired packages, Pkg also has a built-in auto-cleanup function. As package versions evolve and package dependencies change, some of the installed packages will become obsolete and will no longer be used in any existing project. Pkg keeps a log of all the projects used so it can go through the log and see exactly which projects are still needing which packages—and thus identify the ones that are no longer necessary. These can be deleted in one swoop with the pkg> gc command:

(v1.0) pkg> gc Active manifests at: `/Users/adrian/.julia/environments/v1.0/Manifest.toml` `/Users/adrian/.julia/environments/v0.7/Manifest.toml` Deleted /Users/adrian/.julia/packages/Acorn/exWWb: 40.852 KiB Deleted /Users/adrian/.julia/packages/BufferedStreams/hCA7W: 102.235 KiB Deleted /Users/adrian/.julia/packages/Crayons/e1SsX: 49.133 KiB Deleted /Users/adrian/.julia/packages/Example/ljaU2: 4.625 KiB Deleted /Users/adrian/.julia/packages/Genie/XOia2: 2.031 MiB Deleted /Users/adrian/.julia/packages/HTTPClient/ZQR55: 37.669 KiB Deleted /Users/adrian/.julia/packages/Homebrew/l8kUw: 277.296 MiB Deleted /Users/adrian/.julia/packages/LibCURL/Qs5og: 11.599 MiB Deleted /Users/adrian/.julia/packages/LibExpat/6jLDP: 127.247 KiB Deleted /Users/adrian/.julia/packages/LibPQ/N7lDU: 134.734 KiB Deleted /Users/adrian/.julia/packages/Libz/zMAun: 80.744 KiB Deleted /Users/adrian/.julia/packages/Nettle/LMDZh: 50.371 KiB
   Deleted /Users/adrian/.julia/packages/OhMyREPL/limOC: 448.493 KiB 
   Deleted /Users/adrian/.julia/packages/WinRPM/rDDZz: 24.925 KiB 
   Deleted 14 package installations : 292.001 MiB 

Package discovery is not yet as simple as it could be, but there are a few good options. I recommend starting with this list of curated Julia packages: https://github.com/svaksha/Julia.jl. It groups a large collection of packages by domain, covering topics such as AI, Biology, Chemistry, Databases, Graphics, Data Science, Physics, Statistics, Super-Computing and more.

If that is not enough, you can always go to https://discourse.julialang.org where the Julia community discusses a multitude of topics related to the language. You can search and browse the existing threads, especially the package announcements section, hosted at https://discourse.julialang.org/c/community/packages.

Of course you can always ask the community for help—Julians are very friendly and welcoming, and a lot of effort is put towards moderation, in order to keep the discussion civilized and constructive. A free Discourse account is required in order to create new topics and post replies.

Finally, https://juliaobserver.com/packages is a third party website that provides a more polished way to look for packages—and it also performs a GitHub search, thus including unregistered packages too.

Although I already touched upon the topic in the previous paragraphs, I want to close the discussion about Pkg with a word of caution. The fact that a package is registered does not necessarily mean that it has been vetted in terms of functionality or security. It simply means that the package has been submitted by the creator and that it met certain technical requirements to be added to the general registry. The package sources are available on GitHub, and like with any open source software, make sure that you understand what it does, how it should be used, and that you accept the licensing terms.

This concludes our initial discussion about package management. But as this is one of the most common tasks, we'll come back to it again and again in future chapters, where we'll also see a few scenarios for more advanced usage.

Julia is a new programming language that takes advantage of recent innovations in compiler technology to offer the functionality, ease-of-use, and intuitive syntax of dynamic programming languages at the speed of C. One if its goals is to eliminate the so-called two language problem—when the users code in a high-level language, such as R and Python, but performance-critical parts have to be rewritten in C or C++. Julia feels like a dynamic language and offers all the productivity features associated with these. But at the same time, it eliminates the performance trade-offs, proving to be productive enough for prototyping and exploratory coding, and efficient enough for performance-critical applications.

Its built-in package manager provides access to over 2,000 third-party libraries that seamlessly extend the language with powerful new features—and we've learned how to take advantage of these. And if that is not enough, Julia has the ability to call functions written in other languages, such as C, Fortran, Python, or Java, to name just a few.

Julia is free and open source (MIT licensed) and can be deployed on all the major operating systems, including Windows, the main Linux distributions, and macOS. It also comes with some very good IDE and editor options.

Now that we have successfully set up our development environment, it's time to dive deeper into Julia's syntax. In the next chapter, we'll take a look at some of the basic building blocks of the language—defining variables and constants, manipulating and using Strings and numeric types, and working with Arrays. As a testament to Julia's productivity, that's all we'll need (together with some extra packages that we'll add) in order to perform powerful exploratory data analysis on the Iris flowers dataset. Meet you in the next chapter!