Open-domain question answering
Given a passage of text and a question related to that text, the idea of Question Answering (QA) is to identify the subset of the passage that answers the question. It is one of many tasks where Transformer architectures have been applied successfully. The Transformers library has a number of pretrained models for QA that can be applied even in the absence of a dataset to finetune on (a form of zero-shot learning).
However, different models might fail at different examples and it might be useful to examine the reasons. In this section, we'll demonstrate the TensorFlow 2.0 GradientTape functionality: it allows us to record operations on a set of variables we want to perform automatic differentiation on. To explain the model's output on a given input, we can:
- One-hot encode the input – unlike integer tokens (typically used in this context), a one-hot-encoding representation is differentiable
- Instantiate GradientTape...
