How Elon Musk’s New Grok AI Stacks Up Against Competitors

Almost two weeks after Elon Musk’s xAI startup opened up the AI mannequin behind Grok to the general public, its AI chatbot is about to get an improve.

The corporate announced Grok-1.5 on Thursday and claimed that its newest mannequin can perceive longer paperwork, deal with extra advanced prompts, and carry out extra superior reasoning.

Whereas Grok-1.5 seems to be a step up from the unique 1.0 with enhancements in coding and math expertise, its announcement submit reveals that it nonetheless lags behind Google’s Gemini Professional 1.5 AI, OpenAI’s GPT-4, and Anthropic’s Claude 3 Opus in some benchmark assessments, whereas outperforming OpenAI on one key HumanEval take a look at.

Associated: Meet Grok: Elon Musk Unveils ‘Spicy’ AI Chatbot Riddled With ‘Sarcasm’ and ‘Humor’

Grok-1.5 scored larger than GPT-4 on the HumanEval benchmark, which consists of 164 difficult programming issues not included within the AI mannequin’s coaching knowledge. GPT-4 had a rating of 67% and Gemini Professional 1.5 scored 71.9%, whereas Grok-1.5 acquired 74.1%.

^{Elon Musk’s xAI firm is about to launch a brand new model of the Grok AI chatbot, a ChatGPT competitor. Picture by Jaap Arriens/NurPhoto through Getty Photographs.}

With a rating of 81.3% on the MMLU take a look at, which covers data of 57 topics from an elementary to a sophisticated stage, Grok-1.5 carried out near Google Gemini’s rating (83.7%).

It additionally scored near GPT-4’s rating of 52.9% with a rating of fifty.6% on the MATH take a look at, a benchmark that covers grade faculty to highschool math competitors issues.

Associated: Elon Musk Sues ChatGPT-Maker OpenAI, Accuses the Company of Working to ‘Maximize Profits For Microsoft, Rather Than For the Benefit of Humanity’

Musk said in a Friday social media post that Grok 1.5 ought to be accessible on X, previously Twitter, by subsequent week.

The X proprietor has excessive expectations for the following era of Grok, writing that the following step after Grok-1.5 will outperform the AI presently accessible “on all metrics.” Grok 2 is “in coaching now,” he wrote within the submit.

Grok AI is presently only available to these with a $16 a month or larger Premium+ subscription on X.

Musk sued OpenAI, a competitor of xAI, earlier this month and requested for a court docket ruling that might power OpenAI to make the analysis and know-how behind its AI public.