What is a vector embedding?

At the most basic level, a vector is a list of numbers plus an implicit structure that determines how those numbers are defined and how you can compare them. The number of elements in a vector is the vector’s dimension.

Dimensions represent different aspects of the thing that they describe. You might think of a list of properties that describe a car and list them out in a structured way such that the order is always [year, make, model, color, mileage]. These properties form a vector space that can describe any car for which these properties hold. For example, you could describe a specific car with these values as [2000, Honda, Accord, Gold, 122000].

This is a useful model for building intuition on how vectors can encode information. However, each element may not always correspond to a concrete idea with a numerable set of possible values. The vectors used in AI applications are more abstract and have significantly more dimensions. In a way,...