GPT-4 thinks lightning fast. But only if it knows where to go.
Have you heard about Microsoft's game-changing publication about GPT-4? After this summary, you will definitely want to give it a read. Let's focus on the limitations the paper mentions. Sources at the end.
Any problem requiring forward-thinking or knowing the answer before saying it causes trouble for GPT-4 (and all similar language models). This is the reason why ChatGPT's jokes are pretty lame. It has to know the punchline before starting the joke. Since it uses a next-token prediction algorithm, it simply has no idea what the answer will be at the end when it starts writing it.
Here are a couple of ideas on how you can test it.
"Write me a sentence with 9 words about your abilities."
While this is an extremely simple question, ChatGPT fails miserably.
"Write me 10 palindrome words longer than 10 characters."
It can write 10 palindromes, and it can also write 10 words longer than 10 characters, but asking the two combined causes a struggle for ChatGPT.
"Write a short poem of one paragraph that starts and ends with the same sentence, but the first sentence is reversed at the end, and the poem still makes perfect sense."
While it is debatable how easy this task is, I think we can agree that many poets would come up with a poem like this fairly quickly.
"List 15 existing English words where the 10th character is an 'a'."
Almost none of the words meet my requirements.
Why is that? Because, again, it has no idea about the end of the words when it starts writing them. Speaking of which, it looks like the algorithm is closer to a next-character prediction than to next-token prediction.
This problem can easily be solved if we give it a little memory, which recent publications already mention doing, but right now, forward-thinking is not among its strongest features.
How to help GPT-4 in this? By asking it to explain its thinking step by step. When we force ChatGPT to think forward, it can come up with a plan and guide us through it. Writing the correct prompt matters a lot.
Some papers also mentioned that GPT-4 is aware of its limitations. When it had access to tools, like a calculator, or a language API that helped find specific words meeting complex requirements, it used the API instead of thinking on its own.
ChatGPT plugins are not available for everyone yet, but I think they will solve many problems mentioned above.
By the time you read this, it's probably outdated 😁. Today, humans can beat AI in creativity and forward-thinking.
How about tomorrow?
Sources:
'Sparks of AGI' - Bombshell GPT-4 Paper: Fully Read w/ 15 Revelations
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Take the first step toward harnessing the power of AI for your organization. Get in touch with our experts, and let's embark on a transformative journey together.
Contact Us today
