Issues when answering math problems

Although the results from LLMs are impressive, sometimes, models can get confused by seemingly simple questions. This tends to be more frequent when the question involves math. For example, I ran the following prompt in GPT-3.5 five times:

When I was 6 my sister was half my age.
Now I'm 70. How old is my sister?

The plugin I used to run this prompt is under plugins/prompt_engineering/solve_math_problem.

The correct answer is 67 because when I was 6, my sister was 3. Now, 64 years later, I’m 70, so she would be 67.

Here are the results of the five runs on GPT-3.5. The first result was incorrect, saying that “my sister is 64 years younger than her current age:”

If your sister was half your age when you were 6, that means she was 3 years younger than you at that time (6/2 = 3).
Since then, you have aged 64 years (70 - 6 = 64). Therefore, your sister would also be 64 years younger than her current age, which would...