Summary

In this chapter, we discussed how to fine-tune a pretrained model for any token classification task. Fine-tuning models on NER and QA problems was explored. Using pretrained and fine-tuned models on specific tasks with pipelines was detailed with examples. We also learned about various preprocessing steps for these two tasks. Saving pretrained models that are fine-tuned on specific tasks was another major learning point of this chapter. We also saw how it is possible to train models with a limited input size on tasks such as QA that have longer sequence sizes than the model input. Using tokenizers more efficiently to have document splitting with document stride was another important topic of this chapter too.

In the next chapter, we will discuss text representation methods using Transformers. By the end of the chapter, you will learn how to perform zero- or few-shot learning and semantic text clustering.