GPT-4 Equipped with Numeric Calculation - Syme, Don (GitHub, 2023)
GPT-4 with Calc An exploration of using calculation generation to improve GPT-4's capabilities for numeric reasoning. GPT-4 is weak at calculating with numbers, compared to other resasoning capabilities. It makes mistakes that lead to problematic user experiences on texts involving basic numeric problems. We describe a simple, general technique to address this, by adding a step generating calculation code, apply it to some widely reported real-world failures of GPT-4-based AI, do some evaluation and discuss related issues.
https://github.com/githubnext/gpt4-with-calc/blob/main/docs/report.md