Limitations of shallow encodings

Shallow embeddings have the advantage of being easy to understand and relatively easy to implement. However, they have several disadvantages, especially compared to deep encoding techniques:

• This method can’t incorporate node or link-level features. When using graph data, it’s common to have auxiliary information attached to every node or every edge, to describe further properties. By default, the random walk approaches aren’t capable of incorporating such information in their embeddings.
• They can be computationally expensive. How? If we’re interested in embeddings of some dimension, d, and the number of nodes is v, then we need to learn a total of v.d values. Deep approaches with hidden layers would likely have much lower parameters to learn, making the process more computationally efficient.
• Building from the previous point, since there are so many parameters to learn in this approach, we often can’...