Beijing-based tech giant Baidu’s artificial intelligence (AI) chatbot Ernie Bot did better than OpenAI’s ChatGPT in some tests, the company stated in a Tuesday blog post.
Baidu referenced China Science Daily’s reporting that Ernie overtook ChatGPT — an older iteration of the chatbot — in benchmark tests and beat the advanced GPT-4 in Chinese language tests. Based on test scores, Ernie “has outstanding Chinese ability, even surpassing GPT-4,” China Science Daily stated, adding its “comprehensive ability is slightly inferior to GPT-4.”
Baidu obtained better scores than GPT-4 in the Chinese part of the benchmark test AGIEval, but GPT-4 did better at the English part, according to China Science Daily.
Microsoft Research published AGIEval, which is designed to assess the chatbots’ performance in human standardized tests and uses 20 exams “including general university entrance exams (such as the college entrance examination in China and the SAT exam in the United States), judicial exams, math competitions, bar exams, and national civil service exams … etc,” according to China Science Daily.
Ernie beat both ChatGPT and GPT-4 in the C-Eval test, which is a basic Chinese chatbot assessment developed and published by Chinese universities and the University of Edinburgh, according to China Science Daily. The test consists of nearly 14,000 multiple-choice questions in more than 50 subjects.
“ERNIE 3.5 has made significant strides in beta testing, surpassing ChatGPT (3.5) in comprehensive ability scores and outperforming GPT-4 in several Chinese language capabilities,” the blog post states.
Baidu’s emphasis on progress highlights the rivalry and race between the U.S. and China in the field of generative AI, according to CNBC. (RELATED: Tensions Between US And China Will Only Grow As Both Countries Enter An AI Arms Race, Experts Say)
Introducing ERNIE 3.5, our latest foundation model that powers ERNIE Bot.
– Powerful plugins and a staggering 17-fold increase in inference throughput over ERNIE 3.0
– Improved creative writing, Q&A, reasoning, and code generation👇https://t.co/Yekq8LXxh3 pic.twitter.com/4OQuHrPi2F— Baidu Inc. (@Baidu_Inc) June 27, 2023
ChatGPT and GPT-4 both surpassed Ernie on the English MMLU test, which is published by American universities and assesses the chatbots’ English professional skills in nearly 60 disciplines including “STEM, humanities, social sciences and more,” according to China Science Daily.
Baidu publicly released Ernie in March, but its debut of the bot was a failure and led to the company’s shares falling, according to CNBC.
“Just three months after the beta release of ERNIE Bot, Baidu’s large language model (LLM) built on ERNIE 3.0, ERNIE 3.5 has achieved broad enhancements in efficacy, functionality, and performance,” said Dr. Haifeng Wang, CTO of Baidu, according to the blog post. “These improvements are evident in creative writing, Q&A, reasoning, and code generation, as well as in training performance and inference performance.”
Baidu and OpenAI did not immediately respond to the Daily Caller News Foundation’s request for comment.
All content created by the Daily Caller News Foundation, an independent and nonpartisan newswire service, is available without charge to any legitimate news publisher that can provide a large audience. All republished articles must include our logo, our reporter’s byline and their DCNF affiliation. For any questions about our guidelines or partnering with us, please contact licensing@dailycallernewsfoundation.org.