On May 13, OpenAI unveiled its new GPT-4o model, sparking heated debate ever since. OpenAI's developers claim GPT-4o outperforms Claude Opus 3, the model Anthropic — founded by former OpenAI employees — presented on March 4.
In this article we'll cover the key features and strengths of each model, dig into where GPT-4o outperforms Claude Opus 3, and present expert reviews and benchmark rankings. Finally, we'll share the results of our own experiments to help you decide which model fits your tasks best.
GPT-4o: features and strengths
GPT-4o is OpenAI's newest model and a significant step forward for artificial intelligence.
GPT-4o's stated capabilities
- Multimodality: the model can process text, audio, and visual data in real time, analyzing and generating responses across different input types.
- High intelligence: GPT-4o delivers impressive results on complex content analysis and generation tasks.
- Improved performance: compared to GPT-4 Turbo, the model is more efficient at working with text, logical reasoning, and coding.
- Multilingual support: GPT-4o works with multiple languages, making it a versatile tool across different linguistic contexts.
GPT-4o's limitations
Despite its impressive capabilities, the model has a few limitations:
- Not all announced features are available via the API yet — some are still in development.
- Like GPT-4 Turbo, GPT-4o isn't directly accessible everywhere, and access, registration, and payment can be tricky depending on your region and local payment methods.
- The free version of GPT-4o has limits on request volume and functionality. Getting the most out of the model requires paid access, which comes with its own limits too.
Pricing
OpenAI priced GPT-4o at half the cost of GPT-4 Turbo for API use.
On GPTunneL, though, GPT-4o is available with no restrictions or limits at a very attractive price — just a fraction of a cent per 1,000 context tokens. That makes GPTunneL a great option for anyone who wants GPT-4o's capabilities at an affordable price with no limits.
Claude 3 Opus: strengths and standout features
Claude 3 Opus is the most powerful model in Anthropic's Claude 3 lineup. It delivers impressive results on complex tasks, making it one of the leading models on the AI market.
Claude Opus 3's key features
- High intelligence and creativity: the model excels at reasoning and generating creative content, making it ideal for tasks that call for deep analysis and out-of-the-box thinking.
- Versatility: Claude Opus 3 handles both text and visual tasks, opening up new possibilities in computer vision and image understanding — a major expansion of functionality compared to earlier versions.
- High accuracy: the model achieves near-perfect accuracy, exceeding 99% in some cases. That matters most for tasks demanding minimal error and high reliability.
- Near-human understanding: Claude Opus 3 handles open-ended queries and unfamiliar scenarios smoothly and naturally, making interactions with the model more intuitive and comfortable.
Model limitations
Despite its strengths, Claude Opus 3 also has some limitations:
- Claude 3 is officially available in 159 countries, but not everywhere — availability, registration, and local payment methods can be a barrier for some users depending on their region.
- Unlike some other models, Claude Opus 3 isn't available for free. The free tier only offers access to the smaller Claude Sonnet 3 model.
Pricing
Using Claude Opus 3 on GPTunneL costs a bit more per 1,000 context tokens than other models — a fair trade-off given its high accuracy and versatility.
You can check pricing for all Claude 3 models here.
Expert reviews and rankings
When it comes to leading AI models like OpenAI's GPT-4o and Anthropic's Claude Opus 3, it's worth weighing in expert opinions and independent benchmark results. This data helps objectively assess each model's strengths and weaknesses.
Benchmark results from OpenAI's official site
The image from OpenAI's official site shows benchmark results for several models, including GPT-4o, GPT-4 Turbo, GPT-4 (initial release 2023-03-14), Claude 3 Opus, Gemini Pro 1.5, Gemini Ultra 1.0, and Llama 3 400b.
Text evaluation
The benchmarks assess model performance across six criteria:
Criterion
Description
MMLU (%)
Tests the model's ability to understand natural language across multiple tasks. Includes 57 tasks spanning math, history, and computer science.
GPQA (%)
Measures the model's ability to answer a broad range of open-ended questions. A higher percentage indicates better performance.
MATH (%)
Evaluates the model's math skills using 12,500 challenging math problems.
HumanEval (%)
Evaluates the model's ability to generate code. A higher percentage indicates stronger coding ability.
MGSM (%)
Evaluates the model's ability to understand and analyze stories and narratives.
DROP (f1)
Measures the model's ability to answer questions that require discrete reasoning over paragraphs of text.
The key takeaways from these benchmarks:
- GPT-4o outperforms Claude 3 Opus on most standardized benchmarks, especially in multitask learning and math.
- Claude 3 Opus delivers comparable results on some benchmarks, showing it's highly capable and competitive.
These results confirm that both models represent state-of-the-art natural language processing solutions, though GPT-4o currently leads.
IQ evaluation
Journalist Maxim Lott tested the IQ of popular neural networks using the Norway Mensa test.
Claude 3's answers to the IQ test questions
According to his data, Claude Opus 3 became the first AI to exceed an IQ of 100. This significant achievement underscores how fast AI technology is advancing and its potential to handle complex tasks once reserved for humans.
LMSYS Chatbot Arena rankings
The LMSYS Chatbot Arena service compares different AI models. It's an open platform where people help rate language models.
Top LLM models on the LMSYS Chatbot Arena service
According to the latest rankings from May 30, 2024, the market leaders are models from OpenAI and Google:
- OpenAI's GPT-4o confidently holds the top spot in the rankings.
- Second place goes to Google's Gemini.
- Anthropic's Claude 3 Opus ranks third and has received the most votes despite its lower ranking, pointing to its broad popularity among users.
So, GPT-4o and Anthropic's Claude Opus 3 are the leading models in artificial intelligence today. GPT-4o delivers stronger performance on most benchmarks, but Claude Opus 3 also shows impressive results, especially on IQ tests and in user popularity. Both AIs have their own strengths and continue to evolve, offering users cutting-edge solutions for a wide range of tasks.
Our experiments and conclusions
To better understand what different AI models can do, we ran our own internal tests. Besides Claude 3 and GPT-4, we evaluated Meta's LlaMa 3, Google's Gemini Pro, Yandex GPT, and Cohere's Command R. You can check out our LLM benchmark results here.
The test consisted of 14 questions — about 10 math problems with a single correct answer, and 4 logic problems.
Rankings based on GPTunneL team's LLM benchmark tests
In our tests, Claude Opus 3 took first place, correctly answering 12 out of 14 questions. It made mistakes on one math problem and one logic problem — confirming the model's high accuracy and effectiveness at both math and logic tasks.
GPT-4 Omni also delivered a solid result, correctly answering 11 out of 14 questions. It made mistakes on two math problems and one logic problem, highlighting the model's strong competitiveness despite differences in reasoning approach compared to other models.
GPT-4 and GPT-4 Omni scored the same overall but made different mistakes. Their reasoning approaches also differed, pointing to distinct problem-solving strategies.
Wrapping up
In this article we looked at two leading AI models — OpenAI's GPT-4o and Anthropic's Claude Opus 3. Both models deliver outstanding results and come with their own unique strengths and weaknesses.
GPT-4o impresses with its multimodality, multilingual support, and improved performance, while Claude Opus 3 stands out for its high accuracy, deep analytical capability, and creativity. Both AIs represent cutting-edge solutions, and each deserves attention for its unique capabilities and high level of performance.
The choice between GPT-4o and Claude Opus 3 is yours. We recommend comparing them hands-on in our LLM Arena to see which model best fits your specific tasks and needs.