There’s no shortage of reports on Twitter on how ChatGPT isn’t quite as powerful as it was when it was first released, but OpenAI has refuted the charges.
OpenAI has said that it hasn’t been making ChatGPT dumber with time. “No, we haven’t made GPT-4 dumber. Quite the opposite: we make each new version smarter than the previous one,” said OpenAI VP of Product Peter Welinder.
He, however, acknowledged the reports of ChatGPT’s supposed loss in capabilities, but attributed them to users’ greater familiarity with the product. “Current hypothesis (of the reports of lower capabilities): When you use it more heavily, you start noticing issues you didn’t see before,” he said.
He encouraged users to share examples of decreased performance so that OpenAI could investigate. “If you have examples where you believe it’s regressed, please reply to this thread and we’ll investigate,” he said.
Over the last few months, there had been multiple reports that ChatGPT has become worse with time. Last month, we’d published an article documenting such complaints. A top post on Hacker news had said that GPT-4’s abilities had regressed. “It is much faster than before but the quality of its responses is more like a GPT-3.5++,” the user complained. “It generates more buggy code, the answers have less depth and analysis to them, and overall it feels much worse than before,” they added.
Yes. Before the update, when its avatar was still black, it solved pretty complex coding problems effortlessly and gave very nuanced, thoughtful answers to non-programming questions. Now it struggles with just changing two lines in a 10-line block of CSS and printing this modified 10-line block again. Some lines are missing, others are completely different for no reason. I’m sure scaling the model is hard, but they lobotomized it in the process. The original GPT-4 felt like magic to me, I had this sense of awe while interacting with it. Now it is just a dumb stochastic parrot,” another user had said.
OpenAI now appears to have said that it hadn’t intentionally degraded the quality of its model. But whether users are imagining the reduced performance, or it is real, the complaints aren’t the best look for OpenAI. There are now several credible alternatives out there which are approaching GPT-4 in capability, including Google’s Bard and Anthropic’s Claude-2, and users could conceivably switch if push comes to shove. OpenAI charges a lot of money for GPT-4, and it can ill-afford to be perceived as becoming a product that’s becoming worse with time.