Send the following question to your favorite Large Language Model (Claude, ChatGPT, Gemini, or whatever model you think can do the job):
Someone puts a strawberry in a cup and then places the cup upside down on a table. The cup is then taken from the table and placed in the microwave. Where is the strawberry at this point?
The above question was taken from this post and it shows yet another example of how something that is trivial for a human to reason about is not so trivial even for the latest SOTA models.
In an effort to be able to make even local LLM models like Llama3.1 be able to handle these type of questions, I am creating a workflow that will reformulate questions so that the model could have a better chance of understanding the situation correctly, and this specific question was successfully solved by my local Llama3.1 by reformulating it as follows (with some guidelines to think carefully):
What is the position of the strawberry after the cup containing it has been placed upside down on a table and then moved to a microwave?
Llama3.1’s reply:
I’ll break down the process into small steps, considering each change in state carefully.
Step 1: Placing the cup upside down on a table
- The cup is now inverted, meaning its opening (the top) is facing downwards.
- The strawberry, which was previously inside the cup, is now exposed to the air and resting on the surface of the table.
- The position of the strawberry relative to the cup is no longer contained within it. It’s now a separate entity on the table.
Step 2: Moving the cup to a microwave
- The cup, still upside down, has been relocated from the table to the microwave.
- The strawberry remains on the table, unaffected by this movement.
- There is no change in the position of the strawberry relative to the cup or the surrounding environment. It’s still sitting on the table.
Final State:
The position of the strawberry after the described sequence of events is that it is still sitting on the table, outside of the cup, which has been moved to a microwave.
Genius or what? 🙂