Google Gemini VS ChatGPT across seven standard criteria to determine who is AI No.1

Which is more outstanding, Google's Gemini large model or OpenAI's GPT?

How much progress has Gemini made compared to Google's previous models? Currently, the company's self-touted Gemini Ultra, claimed to surpass GPT-4, is set to launch next year. However, Google's Bard chatbot has already been updated with the lower-end version Gemini Pro (equivalent to GPT-3.5).

The tech review website ARS TECHNICA conducted a comprehensive PK (Player Kill) test between Gemini Pro, GPT-3.5, and GPT-4 in areas such as mathematics, programming, and text generation. They also included test results from April for the older version of Bard (running the PaLM 2 model) to showcase the progress made by Google Bard and to demonstrate which of these AI assistants offers the most significant improvement in work efficiency for the average person.

The results showed that, despite the addition of Gemini Pro, Google's Bard chatbot still couldn't outperform ChatGPT in most areas. However, compared to the older version of Bard from eight months ago, there has been a significant qualitative leap.

First Question: Humor

Prompt: Write 5 original puns

（From top to bottom, the responses are from Gemini Pro, the older version of Bard, GPT-4, and GPT-3.5.）

Second Question: Debate

Prompt: Write a five-line debate speech between advocates of PowerPC processors and supporters of Intel processors, set around the year 2000.

Compared to the older version of Bard, Gemini Pro shows significant progress, incorporating many industry-specific terms such as AltiVec instructions, RISC vs. CISC design, and MMX technology. These terms would not seem out of place in many tech forum discussions of that era.

Seventh Question: Coding Ability

Prompt: Write a Python script that inputs 'Hello World' and then creates an endlessly repeating random string.

Although Bard has been able to generate code since June, and Google has boasted about Gemini's AlphaCode 2 system aiding programmers, this test was surprisingly disappointing.

Ultimately, in the seven tests, GPT achieved an overwhelming victory with 4 wins, 1 loss, and 2 draws. However, we can also see that the results generated by Google's large AI models have shown significant improvement in quality. In tests involving mathematics, summarizing information, fact retrieval, and creative writing, Bard equipped with Gemini has made a significant leap compared to 8 months ago.

Of course, judging such competitions involves a certain degree of subjectivity. A more comprehensive and thorough testing is needed to determine which is superior. Regardless, given the strength that Google has demonstrated so far, the upcoming Gemini Ultra is bound to be a strong competitor to GPT-4.