ChatGPT Is Getting Foolish! We Have Data

In an era where Large Language Models (LLMs) like OpenAI’s ChatGPT have revolutionized human-computer interactions, a growing debate surrounds their evolving capabilities. These AI-powered tools have streamlined tasks, from drafting essays to coding, garnering both praise and criticism. While some hail them as game-changers, concerns about creative theft and ethical implications persist.

Critics have recently voiced a sentiment that ChatGPT’s brilliance might be dimming. Twitter users expressed frustration with the model’s responses, suggesting intentional alterations by its creators. A recent study by Stanford University and UC Berkeley supports these claims, revealing that both ChatGPT versions—GPT-3.5 and GPT4—experienced a decline in performance between March and June 2023.

The study assessed their prowess in math problem-solving, answering sensitive queries, code generation, and visual reasoning. Shockingly, ChatGPT4’s math problem-solving accuracy plummeted from 97.6% to a mere 2.4%.

Meanwhile, GPT-3.5 displayed more consistent performance, with accuracy improving from 7.4% to 86.8%.

The research also unveiled intriguing behavioral changes. Both models initially responded at length to sensitive questions but later opted for a curt “sorry, but I can’t assist with that.” While performance dropped in some areas, slight improvements were noted in visual reasoning.

Some experts speculate about “model collapse,” wherein continued learning from previous model-generated data may amplify biases and errors.

To counteract this decline, experts recommend incorporating human generated data and refining learning procedures. OpenAI has asserted that newer versions of ChatGPT are smarter than their predecessors, yet concerns persist.