Understanding HTTP, Go, and Echo

Echo is a performance-focused, open source Go web application framework. Go is an increasingly popular language choice for creating web applications due to the flexibility and performance built into the `net/http` standard library package that comes with the language. The `net/http` package provides many useful and powerful primitives, such as its web server implementation, request and response types and methods, uniform resource location router, as well as a clean handler interface and function type declaration.

We will begin by taking an inventory of the basic building blocks of a web application from the protocol. Then, we will explore what the `net/http` package provides, and very briefly touch on how a web application framework could build with these primitives to provide developers with structure for building successful applications. It will be shown how the Echo web application framework fills the voids left by the `net/http` package, allowing for ease of development. Finally, we will end with best practices for setting up your web application using Echo.

Within this chapter, we will cover:

A refresher of the primitives of the Hypertext Transport Protocol (HTTP)
An overview of what the net/http standard library provides
A brief explanation of why the standard library is lacking and in need of a framework
How to set up your Go environment and install the Echo Framework

HTTP basics

Two of the most critical structures within web application development are requests and responses. As with most other programs, a web application needs to take inputs, and produce outputs. As HTTP is a stateless protocol, each request made will result in a response message back to the caller. For a web application, the input comes in the form of an HTTP request, and the output as an HTTP response. The net/http package contains data structures that completely model the HTTP protocol message types, which are used by most of the many web frameworks built with Go. In the following diagram, you can see the HTTP request and response protocol data structures:

HTTP request

A request, as defined by RFC-7230 Section 3, is a text-based message, which includes the following required information at a minimum:

Request method (RFC-7231 4.3; RFC-5789 2): The method is the action the client would like to perform on the resource. Enumerated HTTP methods are a finite list of verbs, and are outlined in the preceding two RFCs. The Go net/http package contains a list of constants for referencing the various methods, as can be seen in the documentation: https://golang.org/pkg/net/http/#pkg-constants.
Request target (RFC-7230 5.3): The request target, derived from the Uniform Resource Identifier (URI), identifies the resource to which it applies the request on the server. This target, which is based on the specification, can consist of multiple parts, including a host portion, path portion, query portion, and fragment portion. The Go net/url package contains logic for parsing a Unified Resource Locator (URL), as can be seen in the documentation: https://golang.org/pkg/net/url/.
Request HTTP version (RFC-7230 2.6): The HTTP version is the textual representation of the highest minor version within the major version of the protocol that the client is able to support. Within the Go net/http web server, this version information is populated in the request type.

The following is an example of the plain text HTTP request message which will GET the resource /file.txt from the web server, from which the client would like to navigate to the HTTP/1.1 protocol:

   GET /file.txt HTTP/1.1

The first line in an HTTP message is known as the start-line of the HTTP message. If the message is a request type message, this line is more clearly known as the request-line of the message. As you can see in the Go documentation for the request type, the first five attributes of the http.Request type encompass the data from the start-line of the request: https://golang.org/pkg/net/http/#Request.

Additionally, and optionally, the request can contain the following data and metadata if applicable within the structure of the HTTP message:

Request header fields (RFC-7230 3.2): Request header fields, known as just headers, are key/value pairs of metadata associated with either of the HTTP message types, request or response. This metadata is used to further describe attributes of the request, such as the length of the content body, or authentication credentials. The key/value pairs are separated by a : character, and are enumerated directly after the start-line of the message, one line per pair. There is a helpful list of registered headers that are commonly in use, which can provide guidance on request and response metadata. These registered headers are defined here: http://www.iana.org/assignments/message-headers. Within the http.Request type, there is a Header attribute which is a key to value mapping.
Message body (RFC-7230 3.3): The message body is the payload of the request or response in the HTTP message protocol. The client and server are signaled the fact that the message body is present by the existence of the Content-Length or Transfer-Encoding headers. The Go web server implementation takes care of these headers automatically when a response is written to the response writer. The message body on a request is accessible from the Body attribute on the Request type in the net/http package. Of particular note here is that the Body is an implementation of the io.ReadCloser interface, which contains two methods: Read and Close.

The following is a more complete HTTP request message:

   POST /file.txt HTTP/1.1
        Host: localhost
        Accept: */*
        Content-Length: 8
        Content-Type: application/x-www-form-urlencoded

        hi=there

In the preceding example, the request dictates that we want to use the POST method on the `/file.txt` target resource. Our request headers specify that we are attempting to contact a particular host which is localhost, and are willing to accept any type of response content. Since we have a request body that is URL encoded, we need a Content-Type header specifying the content type as well as the length of the request body in bytes. Finally, our request body, hi=there, is submitted as the message body of the HTTP request message.

HTTP response

Very similar to the request structure, responses also contain a start-line that is designated as the response status-line. The response may include a message body and header fields, just as the request message is capable of doing. The following data is included within the response line:

HTTP-Version (RFC-7230 2.6): Similar to the request HTTP-Version field in the request-line, the HTTP-Version is the protocol major and minor version in which the server is responding.
Status-Code (RFC-7231 6): The response status-code is a three-digit integer code that explains the response of the server. There are five different classes of status codes, namely informational (1xx), successful (2xx), redirection (3xx), client Error (4xx) and server error (5xx).
Reason-Phrase: The reason-phrase of the response status-line should be ignored by the client, as it is a textual explanation concerning the status of the request.

Within Go, there is an http.Response type which contains all of the previously enumerated elements within the status-line as well as a response header structure and a message body. Typically, if you are developing a web application in Go, you will generally only interface with the response type through the http.ResponseWriter type defined (https://golang.org/pkg/net/http/#ResponseWriter), which is an interface. This interface allows the developer to populate the message header fields with the Header method, and write the status-line in the message with the WriteHeader method. If the response requires a message body in the response message, the http.ResponseWriter interface has a Write method which will send the raw data as the response message body.

When looking at web application frameworks, it is important to keep in mind the bare essentials of the protocol. The HTTP protocol is a very simple text-based protocol, consisting of request and response messages that were designed to be stateless. However, some very interesting and complex systems and services have been built on top of this simple protocol. When we talk about web frameworks and Echo, we need to keep in mind how the features will affect our application, which in the end is a simple text-based protocol.

Go HTTP handlers

At some point in your web application, when you are given a request, you will need to perform actions and provide a response to the request. Within the net/http package, there is an http.Handler interface which is implementable, allowing a common entry point scheme for the web server to run your handler code. The Go web server implementation effectively takes a handler you specify, and for each request it runs your handler's ServeHTTP method, which is defined in the interface. As seen in the documentation (https://golang.org/pkg/net/http/#Handler) the handler's ServeHTTP method signature includes a parameter for a http.ResponseWriter interface as well as a pointer to the http.Request structure. The following is an example handler implementation, which can be found at https://github.com/PacktPublishing/Echo-Essentials/tree/master/chapter1/SimpleHandler.go:

package main

import "net/http"

func main() {
 http.Handle("/", new(myHandler))
 http.ListenAndServe(":8080", nil)
}

type myHandler struct{}

func (mh *myHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 w.WriteHeader(200)
 w.Write([]byte("hello!"))
}

Within the preceding example, we are creating a handler type called myHandler, which implements the handler interface. In the implementation of myHandler.ServeHTTP, we are writing hello! as the response body back to the caller. To exercise this handler, you can run this code with go run SimpleHandler.go and then do the same in another terminal running curl localhost:8080/, which will make a request to this service.

Though simple, this example outlines a number of concerns for web application developers. One primary concern is wondering how you can use URL path variables such as /resource/$ID, where $ID would be a variable within your handler. What happens if your code panics within your handler? Are you responsible for encoding all of your response body payload into a []byte type? How can you implement reusability if you need to run the same processes across multiple handlers?

Setting up the environment

In order to get started with Echo, you will need a working Go environment. To that end, installing Go in your environment is fairly trivial. Firstly, we need to download the Go source code and tool chain and install them. For this book, we will be using Go version 1.9.4, which is the latest release at the time of writing this book. As you can see at https://golang.org/doc/install, there are detailed instructions for setting up your environment correctly. The following are the typical steps needed to install Go for your system:

Linux/macOSX Go Installation

Download Go 1.9.4:

Linux:

      curl -O https://dl.google.com/go/go1.10.linux-amd64.tar.gz
      sudo tar -C /usr/local -xzf go1.10.linux-amd64.tar.gz

macOS using Homebrew:

      brew install go

Set up related environmental variables (assuming you use bash):

mkdir -p ~/go

echo "export GOPATH=$HOME/go" >> ~/.bashrc

echo "export PATH=$PATH:$HOME/go/bin:/usr/local/go/bin" >> ~/.bashrc 

source ~/.bashrc

At this point, the environment is set up. The preceding curl command downloads Go as a tarball, and the tar command will unpack the Go standard library source code and related binary tools into the /usr/local/go directory on your system. If you are running this as a non-root user, you might need to run the tar command listed in the preceding snippet with root privileges, potentially by changing users to root, or running the command with sudo.

The preceding environment variable setup commands will first set up a workspace as a go directory within your home directory. The command will also set your GOPATH environment variable to use this workspace for downloading go packages. Next, we need to set the PATH variable to include the bin directory that was installed in /usr/local/go/, which is where the Go compiler, as well as other tools such as gofmt that come with Go, reside. Finally, we add all of these environment variables into your .bashrc so that every time you log in, these environment variables are set appropriately.

Windows installation

You can download the MSI installer for Go: https://dl.google.com/go/go1.10.windows-amd64.msi.

The Windows installer will, by default, install Go in the c:\Go\ folder. The installer will take care of setting up all necessary environment variables for you.

Setting up Echo

After you have a working installation of Go, you will be able to install the Echo web application framework. To install third-party libraries and packages, you can use the go get command to get any package you need. The following is how you would go about installing Echo version 3.3.5, which is the latest version at the time of writing this book, and which is the version of Echo we will use for the remainder of this book:

go get github.com/labstack/echo

cd $GOPATH/src/github.com/labstack/echo 

git checkout tags/3.3.5

cd -

The preceding commands will clone the Echo web application framework into your GOPATH in the appropriate location, and then checkout the 3.3.5 release tag of the repository. With the preceding command, we can ensure that we are using the correct version of Echo. Cloning the repository directly and checking out the appropriate version might seem a little bit annoying, but unfortunately the go get tool, at the time of writing, does not support versioned package downloads. In future chapter examples, we will utilize a version pinning tool that is extremely handy called dep, which will eliminate the need to perform these steps on a per project basis.

In order to verify that we have your environment completely set up, we will now create a small Echo-based web application, and compile it to prove we have a working setup. You can download the following code using git with the following commands:

mkdir -p $GOPATH/src/github.com/PacktPublishing/

cd $GOPATH/src/github.com/PacktPublishing/

git clone https://github.com/PacktPublishing/Echo-Essentials

cd Echo-Essentials

Now that you have the code, you will be able to see the following code located in the ./chapter1/environment_setup.go: file:

package main
import (
 "net/http"
 "github.com/labstack/echo"
)

func main() {
 // create a new echo instance
 e := echo.New()
 // Route / to handler function
 e.GET("/", handler)
 // start the server, and log if it fails
 e.Logger.Fatal(e.Start(":8080"))
}

// handler - Simple handler to make sure environment is setup
func handler(c echo.Context) error {
 // return the string "Hello World" as the response body
 // with an http.StatusOK (200) status
 return c.String(http.StatusOK, "Hello World")
}

Within this project, you may notice that all of the dependencies are already downloaded to the ./chapter1/vendor directory. If you wanted to pull the dependencies yourself, this can be performed by running go get ./... from within the $GOPATH/src/github.com/PacktPublishing/Echo-Essentials/chapter1/ directory. The go get command will walk through your source code recursively and pull all of the dependencies needed from the internet, based on the import statements from within your code. At this point, you should attempt to run the code with go run environment_setup.go, which should produce the following output:

   go run $GOPATH/src/PacktPublishing/Echo-Essentials/chapter1/environment_setup.go 
   ____ __
  / __/___/ / ___
 / _// __/ _ \/ _ \
/___/\__/_//_/\___/ v3.3.5
High performance, minimalist Go web framework
https://echo.labstack.com
____________________________________O/_______
                                    O\
⇨ http server started on [::]:8080

This output proves that we are properly configured for our Echo-based web application, and that we are using the correct version of Echo, that is Version 3.3.5.