Working with binary data

If you need to send a piece of data over a network connection or store it in a file, you first have to encode it (or serialize it, or marshal it.) This is necessary because the system at the other end of the network connection or the application that will read the file you wrote may be running on a different platform. A portable, easy-to-debug but not necessarily efficient way to do this is to use text-based encodings such as JSON. If performance is paramount or when the use case demands it, you use binary encoding.

There are many high-level binary encoding schemes. Gob (https://pkg.go.dev/encoding/gob) is a Go-specific encoding scheme that can be used for networking applications. Protocol buffers (https://protobuf.dev) provide a language-neutral, extensible, schema-driven mechanism for encoding structured data. There are more. Here, we will look at the basics of binary encoding that every software engineer should know about.

Encoding data involves transforming...