Formal description and notation

We would like to introduce some notation and formal definitions for the terms used in supervised learning. We will follow this notation through the rest of the book when not specified and extend it as appropriate when new concepts are encountered. The notation will provide a precise and consistent language to describe the terms of art and enable a more rapid and efficient comprehension of the subject.

• Instance: Every observation is a data instance. Normally the variable X is used to represent the input space. Each data instance has many variables (also called features) and is referred to as x (vector representation with bold) of dimension d where d denotes the number of variables or features or attributes in each instance. The features are represented as x = (x₁,x₂,…x_d)^T, where each value is a scalar when it is numeric corresponding to the feature value.
• Label: The label (also called target) is the dependent variable of interest, generally denoted by...