Claude 2, developed by rising startup star Anthropic, is the most capable large language model generative AI on the current market. It reached a success ratio of 70 percent with the HumanEval benchmark. This is particularly noteworthy as it is a 0-shot evaluation, meaning all AI programs benchmarked against it had not had previous data of this sort nor previous training with the tasks. This means that Claude 2 was the quickest at absorbing and understanding the task given to it.
HumanEval benchmark comparison between major generative artificial intelligence (AI) programs in 2023
Characteristic
Benchmark success rate
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Exclusive Premium Statistic
To access all Premium Statistics, you need a paid Statista Account
Profit from the additional features of your individual account
Currently, you are using a shared account. To use individual functions (e.g., mark statistics as favourites, set
statistic alerts) please log in with your personal account.
If you are an admin, please authenticate by logging in again.
Learn more about how Statista can support your business.
xAI. (November 4, 2023). HumanEval benchmark comparison between major generative artificial intelligence (AI) programs in 2023 [Graph]. In Statista. Retrieved August 25, 2024, from https://www.statista.com/statistics/1447778/humaneval-benchmark-comparison-of-major-ai-programs/
xAI. "HumanEval benchmark comparison between major generative artificial intelligence (AI) programs in 2023." Chart. November 4, 2023. Statista. Accessed August 25, 2024. https://www.statista.com/statistics/1447778/humaneval-benchmark-comparison-of-major-ai-programs/
xAI. (2023). HumanEval benchmark comparison between major generative artificial intelligence (AI) programs in 2023. Statista. Statista Inc.. Accessed: August 25, 2024. https://www.statista.com/statistics/1447778/humaneval-benchmark-comparison-of-major-ai-programs/
xAI. "Humaneval Benchmark Comparison between Major Generative Artificial Intelligence (Ai) Programs in 2023." Statista, Statista Inc., 4 Nov 2023, https://www.statista.com/statistics/1447778/humaneval-benchmark-comparison-of-major-ai-programs/
xAI, HumanEval benchmark comparison between major generative artificial intelligence (AI) programs in 2023 Statista, https://www.statista.com/statistics/1447778/humaneval-benchmark-comparison-of-major-ai-programs/ (last visited August 25, 2024)
HumanEval benchmark comparison between major generative artificial intelligence (AI) programs in 2023 [Graph], xAI, November 4, 2023. [Online]. Available: https://www.statista.com/statistics/1447778/humaneval-benchmark-comparison-of-major-ai-programs/