Let Your Data Do the Talking: Story, Questions, and Answers

Reading comprehension requires many skills. When we read a text, we notice the keywords and the main events and create mental representations of the content. We can then answer questions using our knowledge of the content and our representations. We also examine each question to avoid traps and making mistakes.

Transformers, no matter how powerful they have become, cannot answer open questions easily. An open environment means that somebody can ask any question on any topic, and a transformer would answer correctly. That is still impossible. Transformers often use general domain training datasets in a closed question-and-answer environment. For example, critical answers in medical care and law interpretation require additional NLP functionality.

However, transformers cannot answer any question correctly regardless of whether the training environment is closed with preprocessed question-answer sequences...