Tencent Hunyuan Large Model Ranks First in China in SuperCLUE Evaluation Report

On September 2, SuperCLUE, a Chinese large model evaluation benchmark, released the “Chinese Large Model Benchmark Evaluation August 2024 Report”. Tencent’s Hunyuan Large Model ranked first among domestic large models in total score due to its outstanding performance in multiple core tasks, becoming one of the fastest-improving models on the list.

According to the SuperCLUE report, in the evaluation of 11 capabilities, Tencent Hunyuan ranked first in China in 8 core tasks. “Tencent Hunyuan has good overall capabilities and is a very competitive general-purpose large model.”

What SuperCLUE is evaluating this time is Tencent Hunyuan’s new generation large language model preview version (Turbo-Preview). The model adopts a new hybrid expert model (MoE) structure, from training data, model architecture, training strategy, training framework, software and hardware system In other aspects, it has realized full-link self-research. On the one hand, the model has achieved a significant improvement in performance, and on the other hand, it has also achieved a significant reduction in reasoning costs, which has great application potential.

As an independent third-party Chinese large model benchmark evaluation organization, SuperCLUE’s August report focuses on general ability evaluation, and the evaluation plan consists of three dimensions: science, liberal arts, and hard. Specifically, science abilities include calculation, logical reasoning, and coding ability; liberal arts tasks cover seven dimensions: knowledge encyclopedia, language comprehension, long text, role-playing, generation and creation, security, and tool use; and hard tasks focus on precise instruction following and high-level reasoning of complex tasks.

As the best model in China, Tencent Hunyuan ranked first in both science and liberal arts. Tencent Hunyuan performed well in the Hard task, scoring 74.33 points, the only large model in China with a score of more than 70 points, only slightly behind ChatGPT-4o.

It is worth noting that with the vigorous development of the large model industry, domestic large models represented by Tencent Hunyuan are accelerating their evolution and upgrading their capabilities. The evaluation report data shows that in general, the gap between the general capabilities of the top domestic large model in the Chinese field and the leading foreign models continues to narrow, from 30.12% in May 2023 to 1.29% in August 2024, with a small gap of only about 1 point in the total score.

Since its official debut in September 2023, Tencent Hunyuan has been the first in China to adopt the MoE structure, and the model has expanded to a trillion-parameter scale. The overall performance has been continuously upgraded. In addition to general capabilities and text-to-text, it also has outstanding performance in multimodal capabilities such as text-to-image, image-to-text, and video generation. In the previously released Chinese multimodal large model SuperCLUE-V benchmark list, Tencent Hunyuan’s large model ranked first among domestic large models due to its outstanding performance in multimodal understanding, and has remained in the excellent leader quadrant.

Based on the accumulation of leading model capabilities, Tencent Hunyuan Big Model is actively promoting the implementation of applications to create more value for the big model. Currently, nearly 700 businesses and scenarios within Tencent have been connected, including Tencent Yuanbao, Tencent Cloud, QQ, WeChat Reading, Tencent News, Tencent Customer Service, etc. Previously, Tencent’s collaborative SaaS (software as a service) products were fully connected to the Tencent Hunyuan Big Model.

Tencent Hunyuan large model provides model services of various sizes on Tencent Cloud, and is fully open to enterprises and individual developers through access and use methods such as API, exclusive models, and fine-tuning models. Currently, the cloud versions of Tencent Hunyuan include Turbo-Preview, Pro, Standard, Lite and other versions; code generation, role-playing, Functioncall, etc. are open on the exclusive model; enterprises can also fine-tune Tencent Hunyuan through the Tencent Cloud TI platform.

Based on years of experience and accumulation in industrial Internet, Tencent Cloud has joined hands with leading companies in the industry to output more than 50 solutions for more than 20 industries, providing a complete set of model service tool chains to help companies create and deploy AI applications efficiently, with high quality and low cost.