Code benchmark comparison between Gemini Ultra and GPT-4 in 2024

Published by
Bergur Thormundsson,
Feb 16, 2024
Google's Gemini Ultra and OpenAI's GPT-4 are the leading generative AI platforms worldwide, and they compare alike in the code creation benchmarks. Gemini Ultra is slightly ahead of GPT-4, beating it in both coding benchmarks as well as the general MMLU benchmarking. The lead is not so significant as to consider GPT-4 a lackluster model.

Comparison of code benchmarks between Google's Gemini Ultra and OpenAI's GPT-4 in 2024

© Statista 2024
Download
Show detailed source information?
Register for free
Already a member?
Log in
Source

Use Ask Statista Research Service

Release date

December 2023

Region

Worldwide

Survey time period

December 2023

Supplementary notes

Source notes: "HumanEval = Python code generation, Natural2Code = Python code generation. New held out dataset HumanEval-like, not leaked on the web"

Source notes: "MMLU = Representation of questions in 57 subjects (incl. STEM, humanities, and others)"

Citation formats
Access all statistics starting from $2,388 USD yearly *

* For commercial use only

Basic Account

For single users
$0 USD
Always free
Access limited to Free Statistics. Premium Statistics are not included.
  • Free Statistics
Based on your interests

Starter Account

For single users
$199 USD
per month, billed annually 1
Buy now
  • Free Statistics
  • Premium Statistics

    • The statistic on this page is a Premium Statistic and is included in this account.

Professional Account

For teams of up to 5 people
$959 USD
per month, billed annually 1
Buy now
  • Free + Premium Statistics
  • Reports
  • Market Insights

1 All prices do not include sales tax. The account requires an annual contract and will renew after one year to the regular list price.

Compare accounts

Statistics on " Generative artificial intelligence (AI) "

The most important statistics
The most important statistics
The most important statistics
The most important statistics
The most important statistics
The most important statistics

Other statistics that may interest you Generative artificial intelligence (AI)

Overview

3

Generative AI tools

3

Generative AI benchmarks

7

Generative AI developers

5

Generative AI use

3

Special focus: United States

5

Further Content: You might find this interesting as well

Topics

Generative artificial intelligence (AI)Artificial intelligence (AI) worldwideArtificial intelligence (AI) use in marketing ChatGPTBard
Access all statistics starting from $2,388 USD yearly *

* For commercial use only

Basic Account

For single users
$0 USD
Always free
Access limited to Free Statistics. Premium Statistics are not included.
  • Free Statistics
Based on your interests

Starter Account

For single users
$199 USD
per month, billed annually 1
Buy now
  • Free Statistics
  • Premium Statistics

    • The statistic on this page is a Premium Statistic and is included in this account.

Professional Account

For teams of up to 5 people
$959 USD
per month, billed annually 1
Buy now
  • Free + Premium Statistics
  • Reports
  • Market Insights

1 All prices do not include sales tax. The account requires an annual contract and will renew after one year to the regular list price.

Compare accounts
Learn more about how Statista can support your business.