A chat with Claude 3
Back in February 2023, I did a chat with GPT to determine its understanding of supply and demand. I discovered that it did not understand the concept, but I didn’t blame the AI. Instead, I argued that it reflected a flawed understanding of S&D in the documents that GPT was trained on.
Claude 3 was just released, and I decided to investigate whether AI had progressed in its understanding of S&D. As we will see, it has not. I suspect this is because AI is limited by the quality of the material that it trains on, at least for the moment. Perhaps in the future it will be able to “think for itself”.
I had hoped that Claude 3 would recognize that the impact of a price change on consumption depends on whether the price change was caused by a shift in supply or a shift in demand. It failed:
As I did in February 2023, I specified that I was looking for an equilibrium outcome, to make the question less “tricky”. As in 2023, this led to some bizarre confusion in the reply:
The decrease in supply in step 5 is perfectly fine, but the way Claude reasons between steps #3 and #5 is cringeworthy.
I worried that the phrase “in equilibrium” might have confused Claude, leading it to assume I only meant starting in equilibrium. So I repeated the question with language making it clear I meant “remains in equilibrium”:
This is a bit better, although it still misses the basic point that you cannot reason from a price change. Claude had no basis for assuming the price increase was due to a decline in supply.
Interestingly, at some level Claude does “understand” the fallacy of reasoning from a price change . But here the term “understand” is used in roughly the way a B student at a decent university understands concepts. They can memorize them and provide answers on tests, but often are unable to apply the concept in a novel situation.
I also repeated a question about the impact of an expansionary monetary policy on interest rates. Once again, it assumed that interest rates would fall, which is a mistake made by many economists. More worrisome is its claim that inflation expectations would also fall:
