Just now, Musk open-sourced Grok 2.5: Chinese companies are xAI's biggest competitors

Jin Lei from Wahu Temple
Quantum Bit | Official Account QbitAI

Just now, Musk's move to open-source caused a lot of attention —

xAI has officially open-sourced Grok 2.5, and Grok 3 will be open-sourced in six months.

In fact, as early as the beginning of this month, Musk had publicly stated:

It's time to open-source Grok, which will be next week.

Although the open-source time has exceeded what he said, as netizens said:

It's better to be late than nothing.

42 Files, 500 GB

Currently, Grok can be downloaded on HuggingFace (link attached at the end of the article):

xAI recommends using SGLang to run Grok 2, with the following steps:

First, download the weight files.

You can replace /local/grok-2 with any folder name you like:

hf download xai-org/grok-2 —local-dir /local/grok-2

The official says that some errors may occur during the download process. If an error occurs, try again until the download is successful.

After a successful download, there should be 42 files in the folder, approximately 500 GB in size.

Second, start the server.

xAI recommends installing the latest version of the SGLang inference engine (version number >= v0.5.1, address: https://github.com/sgl-project/sglang/).

Then use the following command to start the inference server:

python3 -m sglang.launch_server —model /local/grok-2 —tokenizer-path /local/grok-2/tokenizer.tok.json —tp 8 —quantization fp8 —attention-backend triton

It is worth noting that this model requires 8 GPUs (each must have more than 40GB of VRAM) to run.

The last step is to send the request.

This is a pre-trained model, so we need to ensure that we are using the correct chat template:

python3 -m sglang.test.send_one —prompt “Human: What is your name?<|separator|>\n\nAssistant:”

After sending the request, we should see the model reply its name, which is Grok.

So, what level is the newly open-sourced Grok 2 by xAI?

Although its capabilities are certainly not as good as various advanced mainstream models, we can get some idea from the technical blog published by xAI last year about the Grok 2-related model.

At that time, it had already surpassed Claude and GPT-4 in the overall Elo score on the LMSYS ranking.

Moreover, in a series of academic benchmark tests, the Grok 2 series achieved performance levels comparable to other cutting-edge models in areas such as graduate-level scientific knowledge (GPQA), general knowledge (MMLU, MMLU-Pro), and math competition problems (MATH).

However, to be honest, although netizens think that Musk's open-source action is quite good, there are also many criticisms.

For example, on HuggingFace, we did not see xAI clearly stating the parameter weights of the open-source model.

Therefore, netizens can only guess based on previous information that it is a 269 billion parameter MoE model.

Secondly, the issue of the open-source license, because xAI's statement on HuggingFace is as follows:

As netizens said, this is basically a non-commercial license:

Mistral, Qwen, DeepSeek, Microsoft, even OpenAI are using MIT or Apache 2.0 licenses to open-source models.

And, the most important point is the conditions for running this open-source model:

Thanks, I just need 8 GPUs with over 40GB of VRAM...

Two More Thing：

Aside from the open-source action, Musk also released some new features on the Grok APP.

This update (v1.1.58) mainly focuses on AI video generation, with the following effects:

Interested friends can experience it on the APP.

Also, Musk shared an interesting comment:

xAI will soon surpass Google, but Chinese companies are the biggest competitors.

References:

[1]https://x.com/elonmusk/status/1959379349322313920

[2]https://x.com/HuggingPapers/status/1959345658361475564

[3]https://x.com/elonmusk/status/1959384678768447976

[4]https://x.com/elonmusk/status/1959388879888302363

— End —

Quantum Bit QbitAI · Toutiao Account

Original: https://www.toutiao.com/article/7541954633686598159/

Disclaimer: The article represents the views of the author. Please express your opinion by clicking the [Up/Down] button below.