Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Go Cookbook
Go Cookbook

Go Cookbook: Build modular, readable, and testable applications in Go

eBook
£20.98 £29.99
Paperback
£36.99
Subscription
Free Trial
Renews at £16.99p/m

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Table of content icon View table of contents Preview book icon Preview Book

Go Cookbook

I/O and File Systems

In this chapter, the following recipes will be covered:

  • Using the common I/O interfaces
  • Using the bytes and strings packages
  • Working with directories and files
  • Working with the CSV format
  • Working with temporary files
  • Working with text/template and HTML/templates

Introduction

Go provides excellent support for both basic and complex I/O. The recipes in this chapter will explore common Go interfaces to deal with I/O and show how to make use of them. The Go standard library frequently uses these interfaces, and these interfaces will be used by recipes throughout the book.

You'll learn how to work with data in memory and in the form of streams. You'll see examples of working with files and directories and of working with the CSV format. The temporary files recipe discusses a mechanism to work with files without the overhead of dealing with name collision and more. Lastly, we'll explore Go standard templates for both plain text and HTML.

These recipes should lay the foundation for the use of interfaces to represent and modify data and should help you think about data in an abstract and flexible way.

Using the common I/O interfaces

Go provides a number of I/O interfaces used throughout the standard library. It is a best practice to make use of these interfaces wherever possible rather than passing structs or other types directly. Two powerful interfaces we explore in this recipe are the io.Reader and io.Writer interfaces. These interfaces are used throughout the standard library and understanding how to use them will make you a better Go developer.

The Reader and Writer interfaces look like this:

type Reader interface {
Read(p []byte) (n int, err error)
}

type Writer interface {
Write(p []byte) (n int, err error)
}

Go also makes it easy to combine interfaces. For example, take a look at the following code:

type Seeker interface {
Seek(offset int64, whence int) (int64, error)
}

type ReadSeeker interface {
Reader
Seeker
}

The recipe will also explore an io function called Pipe():

func Pipe() (*PipeReader, *PipeWriter)

The remainder of this book will make use of these interfaces.

Getting ready

Configure your environment according to these steps:

  1. Download and install Go on your operating system at https://golang.org/doc/install and configure your GOPATH environment variable.
  1. Open a terminal/console application, navigate to your GOPATH/src directory, and create a project directory such as $GOPATH/src/github.com/yourusername/customrepo.

All code will be run and modified from this directory.

  1. Optionally, install the latest tested version of the code using the following command:
      go get github.com/agtorre/go-cookbook/

How to do it...

These steps cover writing and running your application:

  1. From your terminal/console application, create a new directory called chapter1/interfaces.
  2. Navigate to that directory.

Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/interfaces, or use this as an exercise to write some of your own code.

  1. Create a file called interfaces.go with the following contents:
        package interfaces

import (
"fmt"
"io"
"os"
)

// Copy copies data from in to out first directly,
// then using a buffer. It also writes to stdout
func Copy(in io.ReadSeeker, out io.Writer) error {
// we write to out, but also Stdout
w := io.MultiWriter(out, os.Stdout)

// a standard copy, this can be dangerous if there's a
// lot of data in in
if _, err := io.Copy(w, in); err != nil {
return err
}

in.Seek(0, 0)

// buffered write using 64 byte chunks
buf := make([]byte, 64)
if _, err := io.CopyBuffer(w, in, buf); err != nil {
return err
}

// lets print a new line
fmt.Println()

return nil
}
  1. Create a file called pipes.go with the following contents:
        package interfaces

import (
"io"
"os"
)

// PipeExample helps give some more examples of using io
//interfaces
func PipeExample() error {
// the pipe reader and pipe writer implement
// io.Reader and io.Writer
r, w := io.Pipe()

// this needs to be run in a separate go routine
// as it will block waiting for the reader
// close at the end for cleanup
go func() {
// for now we'll write something basic,
// this could also be used to encode json
// base64 encode, etc.
w.Write([]byte("testn"))
w.Close()
}()

if _, err := io.Copy(os.Stdout, r); err != nil {
return err
}
return nil
}
  1. Create a new directory named example.
  2. Navigate to example.

  1. Create a main.go file with the following contents and ensure that you modify the interfaces imported to use the path you set up in step 2:
        package main

import (
"bytes"
"fmt"

"github.com/agtorre/go-cookbook/chapter1/interfaces"
)

func main() {
in := bytes.NewReader([]byte("example"))
out := &bytes.Buffer{}
fmt.Print("stdout on Copy = ")
if err := interfaces.Copy(in, out); err != nil {
panic(err)
}

fmt.Println("out bytes buffer =", out.String())

fmt.Print("stdout on PipeExample = ")
if err := interfaces.PipeExample(); err != nil {
panic(err)
}
}
  1. Run go run main.go.
  2. You may also run these:
      go build 
./example

You should see the following output:

        $ go run main.go
stdout on Copy = exampleexample
out bytes buffer = exampleexample
stdout on PipeExample = test
  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

The Copy() function copies between interfaces and treats them like streams. Thinking of data as streams has a lot of practical uses, especially when working with network traffic or filesystems. The Copy() function also creates a multi-writer that combines two writer streams and writes to them twice using ReadSeeker. If a Reader interface were used instead rather than seeing exampleexample, you would only see example despite copying to the MultiWriter interface twice. There's also an example of a buffered write that you might use if your stream is not fit into the memory.

The PipeReader and PipeWriter structs implement io.Reader and io.Writer interfaces. They're connected, creating an in-memory pipe. The primary purpose of a pipe is to read from a stream while simultaneously writing from the same stream to a different source. In essence, it combines the two streams into a pipe.

Go interfaces are a clean abstraction to wrap data that performs common operations. This is made apparent when doing I/O operations, and so the io package is a great resource for learning about interface composition. The pipe package is often underused but provides great flexibility with thread-safety when linking input and output streams.

Using the bytes and strings packages

The bytes and string packages have a number of useful helpers to work with and convert between strings and byte types. They allow the creation of buffers that work with a number of common I/O interfaces.

Getting ready

Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.

How to do it...

These steps cover writing and running your application:

  1. From your terminal/console application, create a new directory called chapter1/bytestrings.
  2. Navigate to this directory.

  1. Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/bytesstrings, or use this as an exercise to write some of your own code!
  2. Create a file called buffer.go with the following contents:
        package bytestrings

import (
"bytes"
"io"
"io/ioutil"
)

// Buffer demonstrates some tricks for initializing bytes
//Buffers
// These buffers implement an io.Reader interface
func Buffer(rawString string) *bytes.Buffer {

// we'll start with a string encoded into raw bytes
rawBytes := []byte(rawString)

// there are a number of ways to create a buffer from
// the raw bytes or from the original string
var b = new(bytes.Buffer)
b.Write(rawBytes)

// alternatively
b = bytes.NewBuffer(rawBytes)

// and avoiding the intial byte array altogether
b = bytes.NewBufferString(rawString)

return b
}

// ToString is an example of taking an io.Reader and consuming
// it all, then returning a string
func toString(r io.Reader) (string, error) {
b, err := ioutil.ReadAll(r)
if err != nil {
return "", err
}
return string(b), nil
}
  1. Create a file called bytes.go with the following contents:
        package bytestrings

import (
"bufio"
"bytes"
"fmt"
)

// WorkWithBuffer will make use of the buffer created by the
// Buffer function
func WorkWithBuffer() error {
rawString := "it's easy to encode unicode into a byte
array"

b := Buffer(rawString)

// we can quickly convert a buffer back into byes with
// b.Bytes() or a string with b.String()
fmt.Println(b.String())

// because this is an io Reader we can make use of
// generic io reader functions such as
s, err := toString(b)
if err != nil {
return err
}
fmt.Println(s)

// we can also take our bytes and create a bytes reader
// these readers implement io.Reader, io.ReaderAt,
// io.WriterTo, io.Seeker, io.ByteScanner, and
// io.RuneScanner interfaces
reader := bytes.NewReader([]byte(rawString))

// we can also plug it into a scanner that allows
// buffered reading and tokenzation
scanner := bufio.NewScanner(reader)
scanner.Split(bufio.ScanWords)

// iterate over all of the scan events
for scanner.Scan() {
fmt.Print(scanner.Text())
}

return nil
}
  1. Create a file called string.go with the following contents:
        package bytestrings

import (
"fmt"
"io"
"os"
"strings"
)

// SearchString shows a number of methods
// for searching a string
func SearchString() {
s := "this is a test"

// returns true because s contains
// the word this
fmt.Println(strings.Contains(s, "this"))

// returns true because s contains the letter a
// would also match if it contained b or c
fmt.Println(strings.ContainsAny(s, "abc"))

// returns true because s starts with this
fmt.Println(strings.HasPrefix(s, "this"))

// returns true because s ends with this
fmt.Println(strings.HasSuffix(s, "test"))
}

// ModifyString modifies a string in a number of ways
func ModifyString() {
s := "simple string"

// prints [simple string]
fmt.Println(strings.Split(s, " "))

// prints "Simple String"
fmt.Println(strings.Title(s))

// prints "simple string"; all trailing and
// leading white space is removed
s = " simple string "
fmt.Println(strings.TrimSpace(s))
}

// StringReader demonstrates how to create
// an io.Reader interface quickly with a string
func StringReader() {
s := "simple stringn"
r := strings.NewReader(s)

// prints s on Stdout
io.Copy(os.Stdout, r)
}
  1. Create a new directory named example.
  2. Navigate to example.
  3. Create a main.go file with the following contents and ensure that you modify the interfaces imported to use the path you set up in step 2:
        package main

import "github.com/agtorre/go-cookbook/chapter1/bytestrings"

func main() {
err := bytestrings.WorkWithBuffer()
if err != nil {
panic(err)
}

// each of these print to stdout
bytestrings.SearchString()
bytestrings.ModifyString()
bytestrings.StringReader()
}
  1. Run go run main.go.
  2. You may also run these:
      go build
./example

You should see the following output:

        $ go run main.go
it's easy to encode unicode into a byte array ??
it's easy to encode unicode into a byte array ??
it'seasytoencodeunicodeintoabytearray??true
true
true
true
[simple string]
Simple String
simple string
simple string
  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

The bytes library provides a number of convenience functions when working with data. A buffer, for example, is far more flexible than an array of bytes when working with stream processing libraries or methods. Once you've created a buffer, it can be used to satisfy an io.Reader interface so you can take advantage of ioutil functions to manipulate the data. For steaming applications, you'd probably want to use a buffer and a scanner. The bufio package comes in handy for these cases. Sometimes, using an array or slice is more appropriate for smaller datasets or when you have a lot of memory on your machine.

Go provides a lot of flexibility in converting between interfaces with these basic types--it's relatively simple to convert between strings and bytes. When working with strings, the strings package provides a number of convenience functions to work with, search, and manipulate strings. In some cases, a good regular expression may be appropriate, but most of the time, the strings and strconv packages are sufficient. The strings package allows you to make a string look like a title, split it into an array, or trim whitespace. It also provides a Reader interface of its own that can be used instead of the bytes package reader type.

Working with directories and files

Working with directories and files can be difficult when you switch between platforms (Windows and Linux, for example). Go provides cross-platform support to work with files and directories in the os and ioutils packages. We've already seen examples of ioutils, but now we'll explore how to use them in another way!

Getting ready

Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.

How to do it...

These steps cover writing and running your application:

  1. From your terminal/console application, create a new directory called chapter1/filedirs.
  2. Navigate to this directory.
  3. Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/filedirs, or use this as an exercise to write some of your own code!
  4. Create a file called dirs.go with the following contents:
        package filedirs

import (
"errors"
"io"
"os"
)

// Operate manipulates files and directories
func Operate() error {
// this 0777 is similar to what you'd see with chown
// on a command line this will create a director
// /tmp/example, you may also use an absolute path
// instead of a relative one
if err := os.Mkdir("example_dir", os.FileMode(0755));
err != nil {
return err
}

// go to the /tmp directory
if err := os.Chdir("example_dir"); err != nil {
return err
}

// f is a generic file object
// it also implements multiple interfaces
// and can be used as a reader or writer
// if the correct bits are set when opening
f, err := os.Create("test.txt")
if err != nil {
return err
}

// we write a known-length value to the file and
// validate that it wrote correctly
value := []byte("hellon")
count, err := f.Write(value)
if err != nil {
return err
}
if count != len(value) {
return errors.New("incorrect length returned
from write")
}

if err := f.Close(); err != nil {
return err
}

// read the file
f, err = os.Open("test.txt")
if err != nil {
return err
}

io.Copy(os.Stdout, f)

if err := f.Close(); err != nil {
return err
}

// go to the /tmp directory
if err := os.Chdir(".."); err != nil {
return err
}

// cleanup, os.RemoveAll can be dangerous if you
// point at the wrong directory, use user input,
// and especially if you run as root
if err := os.RemoveAll("example_dir"); err != nil {
return err
}

return nil
}
  1. Create a file called bytes.go with the following contents:
        package filedirs

import (
"bytes"
"io"
"os"
"strings"
)

// Capitalizer opens a file, reads the contents,
// then writes those contents to a second file
func Capitalizer(f1 *os.File, f2 *os.File) error {
if _, err := f1.Seek(0, 0); err != nil {
return err
}

var tmp = new(bytes.Buffer)

if _, err := io.Copy(tmp, f1); err != nil {
return err
}

s := strings.ToUpper(tmp.String())

if _, err := io.Copy(f2, strings.NewReader(s)); err !=
nil {
return err
}
return nil
}

// CapitalizerExample creates two files, writes to one
//then calls Capitalizer() on both
func CapitalizerExample() error {
f1, err := os.Create("file1.txt")
if err != nil {
return err
}

if _, err := f1.Write([]byte(`this file contains a
number of words and new lines`)); err != nil {
return err
}

f2, err := os.Create("file2.txt")
if err != nil {
return err
}

if err := Capitalizer(f1, f2); err != nil {
return err
}

if err := os.Remove("file1.txt"); err != nil {
return err
}

if err := os.Remove("file2.txt"); err != nil {
return err
}

return nil
}
  1. Create a new directory named example.
  2. Navigate to example.
  3. Create a main.go file with the following contents and ensure that you modify the filedirs package import to use the path you set up in step 2:
        package main

import "github.com/agtorre/go-cookbook/chapter1/filedirs"

func main() {
if err := filedirs.Operate(); err != nil {
panic(err)
}

if err := filedirs.CapitalizerExample(); err != nil {
panic(err)
}
}
  1. Run go run main.go.
  2. You may also run these:
      go build
./example

You should see the following output:

        $ go run main.go 
hello
  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

If you're familiar with files in Unix, the Go os library should feel very familiar. You can do basically all common operations--stat a file to collect attributes, collect a file with different permissions, and create and modify directories and files. We performed a number of manipulations to directories and files and then cleaned up after ourselves.

Working with file objects is very similar to in-memory streams. Files also provide a number of convenience functions directly, such as Chown, Stat, and Truncate. The easiest way to get comfortable with files is to make use of them. In all the previous recipes, we have to be careful to clean up after our programs.

Working with files is a very common operation when building backend applications. Files can be used for configuration, secret keys, as temporary storage, and more. Go wraps OS system calls using the os package and allows the same functions to operate regardless of whether you're using Windows or Unix.

Once your file is opened and stored in a File struct, it can easily be passed into a number of interfaces discussed earlier. All the earlier examples of working with buffers and in-memory data streams can be replaced directly with file objects. This may be useful for things such as writing all logs to stderr and the file at the same time with a single write call.

Working with the CSV format

CSV is a common format to manipulate data. It's common, for example, to import or export a CSV file into Excel. The Go CSV package operates on data interfaces, and as a result, it's easy to write data to a buffer, stdout, a file, or to a socket. The examples in this section will show some common ways to get data into and out of the CSV format.

Getting ready

Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.

How to do it...

These steps cover writing and running your application:

  1. From your terminal/console application, create a new directory called chapter1/csvformat.
  2. Navigate to this directory.
  3. Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/csvformat, or use this as an exercise to write some of your own code!

  1. Create a file called read_csv.go with the following contents:
        package csvformat

import (
"bytes"
"encoding/csv"
"fmt"
"io"
"strconv"
)

// Movie will hold our parsed CSV
type Movie struct {
Title string
Director string
Year int
}

// ReadCSV gives shows some examples of processing CSV
// that is passed in as an io.Reader
func ReadCSV(b io.Reader) ([]Movie, error) {

r := csv.NewReader(b)

// These are some optional configuration options
r.Comma = ';'
r.Comment = '-'

var movies []Movie

// grab and ignore the header for now
// we may also wanna use this for a dictionary key or
// some other form of lookup
_, err := r.Read()
if err != nil && err != io.EOF {
return nil, err
}

// loop until it's all processed
for {
record, err := r.Read()
if err == io.EOF {
break
} else if err != nil {
return nil, err
}

year, err := strconv.ParseInt(record[2], 10,
64)
if err != nil {
return nil, err
}

m := Movie{record[0], record[1], int(year)}
movies = append(movies, m)
}
return movies, nil
}

// AddMoviesFromText uses the CSV parser with a string
func AddMoviesFromText() error {
// this is an example of us taking a string, converting
// it into a buffer, and reading it
// with the csv package
in := `
- first our headers
movie title;director;year released

- then some data
Guardians of the Galaxy Vol. 2;James Gunn;2017
Star Wars: Episode VIII;Rian Johnson;2017
`

b := bytes.NewBufferString(in)
m, err := ReadCSV(b)
if err != nil {
return err
}
fmt.Printf("%#vn", m)
return nil
}
  1. Create a file called write_csv.go with the following contents:
        package csvformat

import (
"bytes"
"encoding/csv"
"io"
"os"
)

// A Book has an Author and Title
type Book struct {
Author string
Title string
}

// Books is a named type for an array of books
type Books []Book

// ToCSV takes a set of Books and writes to an io.Writer
// it returns any errors
func (books *Books) ToCSV(w io.Writer) error {
n := csv.NewWriter(w)
err := n.Write([]string{"Author", "Title"})
if err != nil {
return err
}
for _, book := range *books {
err := n.Write([]string{book.Author,
book.Title})
if err != nil {
return err
}
}

n.Flush()
return n.Error()
}

// WriteCSVOutput initializes a set of books
// and writes the to os.Stdout
func WriteCSVOutput() error {
b := Books{
Book{
Author: "F Scott Fitzgerald",
Title: "The Great Gatsby",
},
Book{
Author: "J D Salinger",
Title: "The Catcher in the Rye",
},
}

return b.ToCSV(os.Stdout)
}

// WriteCSVBuffer returns a buffer csv for
// a set of books
func WriteCSVBuffer() (*bytes.Buffer, error) {
b := Books{
Book{
Author: "F Scott Fitzgerald",
Title: "The Great Gatsby",
},
Book{
Author: "J D Salinger",
Title: "The Catcher in the Rye",
},
}

w := &bytes.Buffer{}
err := b.ToCSV(w)
return w, err
}
  1. Create a new directory named example.

  1. Navigate to example.
  2. Create a main.go file with the following contents and ensure that you modify the csvformat import to use the path you set up in step 2:
        package main

import (
"fmt"

"github.com/agtorre/go-cookbook/chapter1/csvformat"
)

func main() {
if err := csvformat.AddMoviesFromText(); err != nil {
panic(err)
}

if err := csvformat.WriteCSVOutput(); err != nil {
panic(err)
}

buffer, err := csvformat.WriteCSVBuffer()
if err != nil {
panic(err)
}

fmt.Println("Buffer = ", buffer.String())
}
  1. Run go run main.go.
  2. You may also run these:
      go build
./example

You should see the following output:

        $ go run main.go 
[]csvformat.Movie{csvformat.Movie{Title:"Guardians of the
Galaxy Vol. 2", Director:"James Gunn", Year:2017},
csvformat.Movie{Title:"Star Wars: Episode VIII", Director:"Rian
Johnson", Year:2017}}

Author,Title
F Scott Fitzgerald,The Great Gatsby
J D Salinger,The Catcher in the Rye
Buffer = Author,Title
F Scott Fitzgerald,The Great Gatsby
J D Salinger,The Catcher in the Rye
  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

In order to explore reading a CSV format, we first represent our data as a struct. It's very useful in Go to format data as a struct, as it makes things such as marshaling and encoding relatively simple. Our read example uses movies as our data type. The function takes an io.Reader interface that holds our CSV data as an input. This could be a file or a buffer. We then use that data to create and populate a Movie struct, including converting the year into an integer. We also add options to the CSV parser to use ; as the separator and - as a comment line.

Next, we explore the same idea, but in reverse. Novels are represented with a title and an author. We initialize an array of novels and then write specific novels in the CSV format to an io.Writer interface. Once again, this can be a file, stdout, or a buffer.

The CSV package is an excellent example of why you'd want to think of data flows in Go as implementing common interfaces. It's easy to change the source and destination of our data with small one-line tweaks, and we can easily manipulate CSV data without using an excessive amount of memory or time. For example, it would be possible to read from a stream of data one record at a time and write to a separate stream in a modified format one record at a time. Doing this would not incur significant memory or processor usage.

Later, when we explore data pipelines and worker pools, you'll see how these ideas can be combined and how to handle these streams in parallel.

Working with temporary files

We've created and made use of files for a number of examples so far. We've also had to manually deal with cleanup, name collision, and more. Temporary files and directories are a quicker, simpler way to handle these cases.

Getting ready

Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.

How to do it...

These steps cover writing and running your application:

  1. From your terminal/console application, create a new directory called chapter1/tempfiles.
  2. Navigate to this directory.
  3. Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/tempfiles, or use this as an exercise to write some of your own code!
  4. Create a file called temp_files.go with the following contents:
        package tempfiles

import (
"fmt"
"io/ioutil"
"os"
)

// WorkWithTemp will give some basic patterns for working
// with temporary files and directories
func WorkWithTemp() error {
// If you need a temporary place to store files with
// the same name ie. template1-10.html a temp directory
// is a good way to approach it, the first argument
// being blank means it will use create the directory
// in the location returned by
// os.TempDir()
t, err := ioutil.TempDir("", "tmp")
if err != nil {
return err
}

// This will delete everything inside the temp file
// when this function exits if you want to do this
// later, be sure to return the directory name to the
// calling function
defer os.RemoveAll(t)

// the directory must exist to create the tempfile
// created. t is an *os.File object.
tf, err := ioutil.TempFile(t, "tmp")
if err != nil {
return err
}

fmt.Println(tf.Name())

// normally we'd delete the temporary file here, but
// because we're placing it in a temp directory, it
// gets cleaned up by the earlier defer

return nil
}
  1. Create a new directory named example.
  2. Navigate to example.
  3. Create a main.go file with the following contents and ensure that you modify the tempfiles imported to use the path you set up in step 2:
        package main

import "github.com/agtorre/go-cookbook/chapter1/tempfiles"

func main() {
if err := tempfiles.WorkWithTemp(); err != nil {
panic(err)
}
}
  1. Run go run main.go.
  2. You may also run these:
      go build
./example

You should see (with a different path) the following output:

        $ go run main.go 
/var/folders/kd/ygq5l_0d1xq1lzk_c7htft900000gn/T
/tmp764135258/tmp588787953
  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

Creating temporary files and directories can be done using the ioutil package. Although you must still delete the files yourself, RemoveAll is the convention, and it will do that for you with only one extra line of code.

When writing tests, the use of temporary files is highly recommended. It's also useful for things such as build artifacts and more. The Go ioutil package will try and honor the OS preferences by default, but it allows you to fall back to other directories if required.

Working with text/template and HTML/templates

Go provides rich support for templates. It is simple to nest templates, import functions, represent variables, iterate over data, and so on. If you need something more sophisticated than a CSV writer, templates may be a great solution.

Another application for templates is for websites. When we want to render server-side data to the client, templates fit the bill nicely. At first, Go templates can appear confusing. This chapter will explore working with templates, collecting templates inside of a directory, and working with HTML templates.

Getting ready

Refer to the Getting ready section's steps in the Using the common I/O interfaces recipe.

How to do it...

These steps cover writing and running your application:

  1. From your terminal/console application, create a new directory called chapter1/templates.
  2. Navigate to this directory.
  3. Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/templates, or use this as an exercise to write some of your own!
  4. Create a file called templates.go with the following contents:
        package templates

import (
"os"
"strings"
"text/template"
)

const sampleTemplate = `
This template demonstrates printing a {{ .Variable |
printf "%#v" }}.

{{if .Condition}}
If condition is set, we'll print this
{{else}}
Otherwise, we'll print this instead
{{end}}

Next we'll iterate over an array of strings:
{{range $index, $item := .Items}}
{{$index}}: {{$item}}
{{end}}

We can also easily import other functions like
strings.Split
then immediately used the array created as a result:
{{ range $index, $item := split .Words ","}}
{{$index}}: {{$item}}
{{end}}

Blocks are a way to embed templates into one another
{{ block "block_example" .}}
No Block defined!
{{end}}


{{/*
This is a way
to insert a multi-line comment
*/}}
`

const secondTemplate = `
{{ define "block_example" }}
{{.OtherVariable}}
{{end}}
`

// RunTemplate initializes a template and demonstrates a
// variety of template helper functions
func RunTemplate() error {
data := struct {
Condition bool
Variable string
Items []string
Words string
OtherVariable string
}{
Condition: true,
Variable: "variable",
Items: []string{"item1", "item2", "item3"},
Words:
"another_item1,another_item2,another_item3",
OtherVariable: "I'm defined in a second
template!",
}

funcmap := template.FuncMap{
"split": strings.Split,
}

// these can also be chained
t := template.New("example")
t = t.Funcs(funcmap)

// We could use Must instead to panic on error
// template.Must(t.Parse(sampleTemplate))
t, err := t.Parse(sampleTemplate)
if err != nil {
return err
}

// to demonstrate blocks we'll create another template
// by cloning the first template, then parsing a second
t2, err := t.Clone()
if err != nil {
return err
}

t2, err = t2.Parse(secondTemplate)
if err != nil {
return err
}

// write the template to stdout and populate it
// with data
err = t2.Execute(os.Stdout, &data)
if err != nil {
return err
}

return nil
}
  1. Create a file called template_files.go with the following contents:
        package templates

import (
"io/ioutil"
"os"
"path/filepath"
"text/template"
)

//CreateTemplate will create a template file that contains data
func CreateTemplate(path string, data string) error {
return ioutil.WriteFile(path, []byte(data),
os.FileMode(0755))
}

// InitTemplates sets up templates from a directory
func InitTemplates() error {
tempdir, err := ioutil.TempDir("", "temp")
if err != nil {
return err
}
defer os.RemoveAll(tempdir)

err = CreateTemplate(filepath.Join(tempdir, "t1.tmpl"),
`Template 1! {{ .Var1 }}
{{ block "template2" .}} {{end}}
{{ block "template3" .}} {{end}}
`)
if err != nil {
return err
}

err = CreateTemplate(filepath.Join(tempdir, "t2.tmpl"),
`{{ define "template2"}}Template 2! {{ .Var2 }}{{end}}
`)
if err != nil {
return err
}

err = CreateTemplate(filepath.Join(tempdir, "t3.tmpl"),
`{{ define "template3"}}Template 3! {{ .Var3 }}{{end}}
`)
if err != nil {
return err
}

pattern := filepath.Join(tempdir, "*.tmpl")

// Parse glob will combine all the files that match
// glob and combine them into a single template
tmpl, err := template.ParseGlob(pattern)
if err != nil {
return err
}

// Execute can also work with a map instead
// of a struct
tmpl.Execute(os.Stdout, map[string]string{
"Var1": "Var1!!",
"Var2": "Var2!!",
"Var3": "Var3!!",
})

return nil
}
  1. Create a file called html_templates.go with the following content:
        package templates

import (
"fmt"
"html/template"
"os"
)

// HTMLDifferences highlights some of the differences
// between html/template and text/template
func HTMLDifferences() error {
t := template.New("html")
t, err := t.Parse("<h1>Hello! {{.Name}}</h1>n")
if err != nil {
return err
}

// html/template auto-escapes unsafe operations like
// javascript injection this is contextually aware and
// will behave differently
// depending on where a variable is rendered
err = t.Execute(os.Stdout, map[string]string{"Name": " <script>alert('Can you see me?')</script>"})
if err != nil {
return err
}

// you can also manually call the escapers
fmt.Println(template.JSEscaper(`example
<[email protected]>`))
fmt.Println(template.HTMLEscaper(`example
<[email protected]>`))
fmt.Println(template.URLQueryEscaper(`example
<[email protected]>`))

return nil
}
  1. Create a new directory named example.
  2. Navigate to example.
  3. Create a main.go file with the following contents and ensure that you modify the tempfiles imported to use the path you set up in step 2:
        package main

import "github.com/agtorre/go-cookbook/chapter1/templates"

func main() {
if err := templates.RunTemplate(); err != nil {
panic(err)
}

if err := templates.InitTemplates(); err != nil {
panic(err)
}

if err := templates.HTMLDifferences(); err != nil {
panic(err)
}
}
  1. Run go run main.go.
  2. You may also run these:
      go build
./example

You should see (with a different path) the following output:

        $ go run main.go 

This template demonstrates printing a "variable".


If condition is set, we'll print this


Next we'll iterate over an array of strings:

0: item1

1: item2

2: item3


We can also easily import other functions like strings.Split
then immediately used the array created as a result:

0: another_item1

1: another_item2

2: another_item3


Blocks are a way to embed templates into one another

I'm defined in a second template!





Template 1! Var1!!
Template 2! Var2!!
Template 3! Var3!!
<h1>Hello! &lt;script&gt;alert('Can you see
me?')&lt;/script&gt;</h1>

example [email protected]
example &lt;[email protected]&gt;
example+%3Cexample%40example.com%3E
  1. If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

Go has two template packages--text/template and html/template. These share functionality and a variety of functions. In general, use html/template to render websites and text/html for everything else. Templates are plain text, but variables and functions can be used inside of curly brace blocks.

The template packages also provide convenience methods to work with files. The example creates a number of templates in a temporary directory and then reads them all with a single line of code.

The html/template package is a wrapper around the text/template package. All of the template examples work with the html/template package directly, using no modification and only changing the import statement. HTML templates provide the added benefit of context-aware safety. This prevents things such as JavaScript injection.

The template packages provide what you'd expect out of a modern template library. It's easy to combine templates, add application logic, and ensure safety when emitting results to HTML and JavaScript.

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Discover a number of recipes and approaches to develop modern back-end applications
  • Put to use the best practices to combine the recipes for sophisticated parallel tools
  • This book is based on Go 1.8, which is the latest version

Description

Go (a.k.a. Golang) is a statically-typed programming language first developed at Google. It is derived from C with additional features such as garbage collection, type safety, dynamic-typing capabilities, additional built-in types, and a large standard library. This book takes off where basic tutorials on the language leave off. You can immediately put into practice some of the more advanced concepts and libraries offered by the language while avoiding some of the common mistakes for new Go developers. The book covers basic type and error handling. It explores applications that interact with users, such as websites, command-line tools, or via the file system. It demonstrates how to handle advanced topics such as parallelism, distributed systems, and performance tuning. Lastly, it finishes with reactive and serverless programming in Go.

Who is this book for?

This book is for web developers, programmers, and enterprise developers. Basic knowledge of the Go language is assumed. Experience with back-end application development is not necessary, but may help understand the motivation behind some of the recipes.

What you will learn

  • Test your application using advanced testing methodologies
  • Develop an awareness of application structures, interface design, and tooling
  • Create strategies for third-party packages, dependencies, and vendoring
  • Get to know tricks on treating data such as collections
  • Handle errors and cleanly pass them along to calling functions
  • Wrap dependencies in interfaces for ease of portability and testing
  • Explore reactive programming design patterns in Go
Estimated delivery fee Deliver to Great Britain

Standard delivery 1 - 4 business days

£4.95

Premium delivery 1 - 4 business days

£7.95
(Includes tracking information)

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Jun 28, 2017
Length: 400 pages
Edition : 1st
Language : English
ISBN-13 : 9781783286836
Vendor :
Google
Category :
Languages :

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Estimated delivery fee Deliver to Great Britain

Standard delivery 1 - 4 business days

£4.95

Premium delivery 1 - 4 business days

£7.95
(Includes tracking information)

Product Details

Publication date : Jun 28, 2017
Length: 400 pages
Edition : 1st
Language : English
ISBN-13 : 9781783286836
Vendor :
Google
Category :
Languages :

Packt Subscriptions

See our plans and pricing
Modal Close icon
£16.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
£169.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just £5 each
Feature tick icon Exclusive print discounts
£234.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just £5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total £ 148.97
Go: Design Patterns for Real-World Projects
£74.99
Building Microservices with Go
£36.99
Go Cookbook
£36.99
Total £ 148.97 Stars icon
Banner background image

Table of Contents

13 Chapters
I/O and File Systems Chevron down icon Chevron up icon
Command-Line Tools Chevron down icon Chevron up icon
Data Conversion and Composition Chevron down icon Chevron up icon
Error Handling in Go Chevron down icon Chevron up icon
All about Databases and Storage Chevron down icon Chevron up icon
Web Clients and APIs Chevron down icon Chevron up icon
Microservices for Applications in Go Chevron down icon Chevron up icon
Testing Chevron down icon Chevron up icon
Parallelism and Concurrency Chevron down icon Chevron up icon
Distributed Systems Chevron down icon Chevron up icon
Reactive Programming and Data Streams Chevron down icon Chevron up icon
Serverless Programming Chevron down icon Chevron up icon
Performance Improvements, Tips, and Tricks Chevron down icon Chevron up icon

Customer reviews

Rating distribution
Full star icon Full star icon Full star icon Full star icon Half star icon 4.3
(3 Ratings)
5 star 66.7%
4 star 0%
3 star 33.3%
2 star 0%
1 star 0%
Ben H. May 11, 2018
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I thought the book did a great job showcasing a broad number of solutions to very typical use cases for the Go language. There are several examples in the book that are extremely helpful and novel. The book is mostly code, as opposed to narrative, but the narrative included explains what the code samples do and how the work. I would recommend this book to anyone who has gotten started writing Go applications and needs a good base of examples to draw from.
Amazon Verified review Amazon
Mike J. Nov 11, 2017
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This book provides an excellent corpus of best practices for solving common problems in Go. As a Golang engineer, on topics familiar to me, I found myself affirming the techniques prescribed in this book. It is readily apparent that the author is an expert software engineer and Go developer. On unfamiliar topics, I was able to quickly discern the key points and high level ideas from the author's concise descriptions, consistent layout, and readable code. The book explores a broad set of topics and its layout is ideal for jumping around. The recipe code is well documented and links to additional information are provided for 3rd party libraries and resources. This book can easily serve as the go-to resource for quick introductions and refreshers to the majority of engineering problems that Golang developers encounter. Highly recommended!
Amazon Verified review Amazon
V. B. Rao Oct 02, 2018
Full star icon Full star icon Full star icon Empty star icon Empty star icon 3
Example, in "Getting and Setting Environment" variables : The author, instead of sticking with built-in Golang features, chooses to use a third-party library, now the code is full of references to this third-party function call. A poor way to describe the subject at hand, and additionally a poor design choice.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is the delivery time and cost of print book? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
What is custom duty/charge? Chevron down icon Chevron up icon

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order? Chevron down icon Chevron up icon

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges? Chevron down icon Chevron up icon

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

  • If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
  • Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.
How can I cancel my order? Chevron down icon Chevron up icon

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact [email protected] with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at [email protected] using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy? Chevron down icon Chevron up icon

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on [email protected] with the order number and issue details as explained below:

  1. If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on [email protected] within one hour of placing the order and we will replace/refund you the item cost.
  2. Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on [email protected] who will be able to resolve this issue for you.
  3. You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
  4. Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
  5. If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
  6. Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on [email protected] within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged? Chevron down icon Chevron up icon

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use? Chevron down icon Chevron up icon

You can pay with the following card types:

  1. Visa Debit
  2. Visa Credit
  3. MasterCard
  4. PayPal
What is the delivery time and cost of print books? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela