ChatGPT’s capabilities are getting worse with age, new study claims

OpenAI’s artificial intelligence-powered chatbot ChatGPT seems to beryllium getting worse arsenic clip goes connected and researchers can’t look to fig retired the crushed why. 

In a July 18 study, researchers from Stanford and UC Berkeley recovered ChatGPT’s newest models had go acold little susceptible of providing close answers to an identical bid of questions wrong the span of a fewer months.

The study’s authors couldn’t supply a wide reply arsenic to wherefore the AI chatbot’s capabilities had deteriorated.

To trial however reliable the antithetic models of ChatGPT were, researchers Lingjiao Chen, Matei Zaharia and James Zou asked ChatGPT-3.5 and ChatGPT-4 models to lick a bid of mathematics problems, reply delicate questions, constitute caller lines of codification and behaviour spatial reasoning from prompts.

We evaluated #ChatGPT's behaviour implicit clip and recovered important diffs successful its responses to the *same questions* betwixt the June mentation of GPT4 and GPT3.5 and the March versions. The newer versions got worse connected immoderate tasks. w/ Lingjiao Chen @matei_zaharia https://t.co/TGeN4T18Fd https://t.co/36mjnejERy pic.twitter.com/FEiqrUVbg6

According to the research, successful March ChatGPT-4 was susceptible of identifying premier numbers with a 97.6% accuracy rate. In the aforesaid trial conducted successful June, GPT-4’s accuracy had plummeted to conscionable 2.4%.

In contrast, the earlier GPT-3.5 exemplary had improved connected premier fig recognition wrong the aforesaid clip frame.

Related: SEC’s Gary Gensler believes AI tin fortify its enforcement regime

When it came to generating lines of caller code, the abilities of some models deteriorated substantially betwixt March and June.

The survey besides recovered ChatGPT’s responses to delicate questions — with immoderate examples showing a absorption connected ethnicity and sex — aboriginal became much concise successful refusing to answer.

Earlier iterations of the chatbot provided extended reasoning for wherefore it couldn’t reply definite delicate questions. In June however, the models simply apologized to the idiosyncratic and refused to answer.

“The behaviour of the ‘same’ [large connection model] work tin alteration substantially successful a comparatively abbreviated magnitude of time,” the researchers wrote, noting the request for continuous monitoring of AI exemplary quality.

The researchers recommended users and companies who trust connected LLM services arsenic a constituent successful their workflows instrumentality immoderate signifier of monitoring investigation to guarantee the chatbot remains up to speed.

On June 6, OpenAI unveiled plans to create a squad that volition assistance negociate the risks that could look from a superintelligent AI system, thing it expects to get wrong the decade.

Collect this nonfiction arsenic an NFT to sphere this infinitesimal successful past and amusement your enactment for autarkic journalism successful the crypto space.

AI Eye: AI’s trained connected AI contented spell MAD, is Threads a nonaccomplishment person for AI data?