Implementation in Julia

There are many good and tested libraries for deep learning in popular programming languages:

• Theano (Python)  can utilize both CPU and GPU (from the MILA Lab at the University of Montreal)
• Torch (Lua) is a Matlab-like environment (from Ronan Collobert, Clement Farabet, and Koray Kavukcuoglu)
• Tensorflow (Python) makes use of data flow graphs
• MXNet (Python, R, Julia, C++)
• Caffe is the most popular and widely used
• Keras (Python) based on Theano
• Mocha (Julia) by Chiyuan Zhang

We will mainly go through Mocha for Julia, which is an amazing package written by Chiyuan Zhang, a PhD student at MIT.

To start, add the package as follows:

Pkg.update() 
Pkg.add("Mocha") 

Network architecture

Network architecture in Mocha refers to a set of layers:

data_layer = HDF5DataLayer(name="data", source="data-list.txt", batch_size=64, tops=[:data]) 
ip_layer   = InnerProductLayer(name="ip", output_dim=500, tops=[:ip], bottoms=[:data]) 

• The input of the...