Chapter 14 – LDA and BERTopic

1. BERT enhances the Transformer model by teaching the Transformer to learn from the words before and after each word so it knows the context and order better. This helps the Transformer understand tricky things such as jokes or words with multiple meanings, making it excellent at understanding all kinds of text, such as chatting or reading books. BERT removes the unidirectionality constraint in the Transformer and uses an MLM that randomly masks some of the input tokens. Since some tokens are masked, MLM has to predict the original vocabulary of the masked word based on its before and after context.
2. BERT consists of five modules: BERT, UMAP, HDBSCAN, c-TFIDF, and MMR.
3. UMAP stands for Uniform Manifold Approximation and Projection. It is a clever way to turn complex data into simpler pictures. Imagine you have a big puzzle with lots of pieces (data points), and you want to arrange them on a board so that similar pieces are close together...