Using namespaces

Clojure organizes code via namespaces, or small units of code. Each namespace contains a set of define functions. One of the recurring questions is to figure out how to do the layout of your namespaces, and how to organize your different namespaces so they relate to each other in a clean way. This is what this recipe will go through.

Getting ready

This recipe does not require any special installation steps apart from a running REPL. Also, it is recommended to keep the Clojure namespaces page open so we can refer to it quickly.

How to do it...

We will go through the steps of creating a Clojure namespace, referencing code from other namespaces, as well as loading and reloading namespaces on demand. Lastly, we will go briefly through a few concepts of how to organize those namespaces properly.

Creating a new namespace

We start here by looking at how namespaces are created. While working in the REPL, making changes to or creating a namespace mostly resorts to using the in-ns function from clojure.core:

(in-ns 'hello) 

And next time we define a var using def, we see it bound to that namespace:

hello=> (in-ns 'hello) 
#object[clojure.lang.Namespace 0x3b5fad2d "hello"] 
hello=> (def a 1) 
#'hello/a 

Great. Now, since the function resolves to var as well, we can define the function in our namespace. We need to give the full path to fn now to define the definition, so clojure.core/fn:

hello=> (def b (clojure.core/fn[c] (clojure.core/inc c))) 
#'hello/b 
hello=> (b 2) 
3 

Inspecting namespaces

Namespaces are regular objects, so on the JVM we can see their internals very easily. find-ns tells us what the object is, and on the JVM we then call .getMappings from the namespace object:

(find-ns 'hello) 
; #object[clojure.lang.Namespace 0x3b5fad2d "hello"] 
(.getMappings *1) 
.... 

Namespaces are listed and retained statically in a static field of this class on the JVM so we can refer to things later.

While in-ns helps us define namespaces, we will actually use a higher-level function named ns:

helloagain=> (ns helloagain) 
nil 
helloagain=>  

If you want to go and have a look, ns actually does quite a bit for us. The two main keywords that can be used are imports and refers.

Adding functions with :import and :refer

:import will import classes from the hosting virtual machine (JVM, JavascriptVM, and .NET VM for now). Say in Java we want to use the raw Random Java object directly, we will import it in the namespace with the following:

(ns helloagain (:import [java.util Random])) 
(Random.) 
; #object[java.util.Random 0x6337c201 "java.util.Random@6337c201"] 

We can also check the imports defined in each namespace with ns-imports:

(ns-imports 'helloagain) 
;...  

By default, Clojure does some work for us, to make sure the basic Java objects are already available in each new namespace.

In the same way as we can import Java objects, we can go along and require other Clojure namespaces in the current namespace. This is done through the :require keyword in the namespace definition:

helloagain=>  (ns helloagain (:require [clojure.string :refer :all])) 
WARNING: reverse already refers to: #'clojure.core/reverse in namespace: helloagain, being replaced by: #'clojure.string/reverse 
WARNING: replace already refers to: #'clojure.core/replace in namespace: helloagain, being replaced by: #'clojure.string/replace 
nil 
helloagain=> (reverse "helloagain") 
"niagaolleh" 

The newly referred functions can be seen using ns-refers:

(ns-refers 'helloagain) 
; ... (somewhere clojure.string ...)     

Loading namespaces from files

Now, effectively, the ns calls will mostly be located at the top of each file. Let's say we want to keep the code we have now on this helloagain namespace; we will create a helloagain.clj file and copy the following content:

(ns helloagain  
      (:import [java.util Random]) 
     (:require [clojure.string :refer :all])) 
 
(println  
     (clojure.string/reverse (str (.getName *ns*)))) 
 
 (println (Random.)) 

The namespace and thus the file can be loaded through require. For this to work, we need to validate our classpath, so let's review the way we started the REPL earlier on:

java -cp .:clojure-1.8.0-beta1.jar clojure.main  

The . makes the files in the current folder available on the classpath, so then require can look through the defined classpath and find the file helloagain.clj we have just defined:

user=>  (require 'helloagain) 
niagaolleh 
nil 
user=>  

You will notice the code is executed when calling require on the command line. The ' is required because helloagain should not be evaluated, but instead kept as a symbol.

Reloading namespaces

Now, here's something slightly more complicated. We want to add a function to the helloagain namespace:

(defn only-one [] 
   (println "only one")) 

We want to use this function in a different user namespace, so we can call the ns macro again to do this, or we can also use require directly. Supposing we have added the preceding function to our filename; simply calling require should do it for us:

user=> (require 'helloagain)  
nil 
user=>  (helloagain/only-one) 
CompilerException java.lang.RuntimeException: No such var: helloagain/only-one, compiling:(NO_SOURCE_PATH:4:2)  

Or not. What happened there? The file was not reloaded from disk, and we just got a reference to the already created namespace. The :reload keyword gives us a chance to specify that we want to reload from disk:

(require 'helloagain :reload) 
user=> (helloagain/only-one) 
; only one 

IDEs such as IntelliJ will actually, most of the time, reload the full current file from the buffer, and so the definition of the namespace can be updated.

When loading and reloading namespaces, state management becomes problematic, so we will focus on that a little bit later.

How to organize namespaces

There is great science and research being done on how to organize namespaces; even Albert Einstein had a biweekly meeting to make sure namespaces would be organized in a compatible way.

Here is a list that will help you focus on creating namespaces around coherent goals:

• Group architectural layers with different causes for concern
• Functional modules that have to define contracts to communicate with each other
• Define a public API on top of internal low-level functions

A public API namespace will have well defined contracts and extensive documentation, while low-level functions maybe be more tested but could be less well documented.

Some people have suggested grouping functions depending on the kind of data they are handling. This is also good when you do not have to deal with the relationships of those namespaces; for example, you defined a User namespace and a ShoppingList namespace, but then Users with many ShoppingList items make the relationship management between the two namespaces cumbersome. Only use this way of organization if the data handled is straightforward and simple.

Possibly the most interesting motivation to separate namespaces is the public API method.

There's more...

In the previous section, we saw that having a namespace for the public API of your library is important, so the following will describe, with the potemkin library as an example, how to extract just what you need from your code to present the API you want to present:

tools.namespace,  potemkin

Namespace for public API

potemkin has a very interesting method, named import-vars, that allows something like a copy/paste of a function from a different namespace to a current one. If we remember the previous recipe, we can simply try the potemkin dependency as follows:

lein try potemkin "0.4.1" 

Then we can select on-demand functions and clean up our public-facing namespace:

user=> (require 'potemkin) 
(potemkin/import-vars [clojure.string reverse]) 
 (user/reverse "hello")  

This makes selecting functions for our namespace clean, without any code, mostly documentation.

This also makes a case for versioning our namespace, in case some breaking changes are introduced but there is no need to make a completely new version of your code.

tools.namespace

Now, we saw just a few moments ago that we can force :reload on a namespace, but things get complicated when namespaces depend on each other, and you are not sure which one was reloaded and which one was not. This is where tools.namespace makes it easier for you to track all this for you. Let us start a REPL and try this out:

lein try org.clojure/tools.namespace "0.2.11" 

As per the doc, the refresh function will scan all the directories on the classpath for Clojure source files, read their ns declarations, build a graph of their dependencies, and load them in dependency order. (You can change the directories it scans with set-refresh-dirs.)

So, at the REPL, the function we mostly need from tools.namespace is refresh:

(require '[clojure.tools.namespace.repl :refer [refresh]]) 

Now, supposing our file helloagain.clj is in the src folder so, src/helloagain.clj, and lein can find and load the file with require:

(require '[helloagain :refer :all]) 
user=> (only-one) 
only one 

We will quickly add a new function to our file:

 (defn only-two [] 
    (println "only two"))   

And call refresh to make sure we have the latest code:

user=> (refresh) 
; :reloading (helloagain) 
user> (helloagain/only-two) 
; only two 

And that is all. We have pulled the latest from our namespace code, and the required namespaces will be reloaded as needed.