Latest overblown AI claim: GPT-4 achieves a perfect score on questions representing the entire MIT Math, EE, and CS curriculum

The paper, released on arXiv two days ago, is getting a lot of attention. Some issues with it were immediately apparent. Then three MIT students dug up the study’s test set, and discovered things were much worse than they initially appeared.