Tencent Hy4 Preview: 770B MoE Weights Released Under Apache 2.0

On August 28, 2026, Tencent released the large language model Hy4 preview and published its weights to four hosts: Hugging Face, ModelScope, GitCode, and CNB ¹². The license is Apache License 2.0 ².

The model has 770B total parameters, 49B activated per token, and a context window of 1M tokens ¹². Two variants are available: the instruct model and an FP8-quantized version of it ².

Tencent calls this “open-source.” What can be verified as published are the weights and the model card; there is no confirmation that the training data or training code were released. This article treats it as a weights release.

77 of 78 layers are MoE

The model card discloses the architecture in some detail ². The backbone is 78 layers: only the first uses a standard dense FFN, and the remaining 77 replace it with MoE. Each MoE layer holds 256 routed experts plus 1 shared expert, and every token activates the top-8 routed experts along with the shared expert ².

Property	Value
Total / activated parameters	770B / 49B
Layers	78 (1 dense FFN, 77 MoE)
Experts	256 routed, 1 shared, top-8 routed activated per token
Hidden size	6144
Attention	Gated DSA (64 heads)
Context length	1M
Vocabulary size	120,832

(As the model card states, this table covers the backbone only and excludes the MTP layer ².)

Separate from the backbone, one MTP layer is built in for speculative decoding, with 10B total parameters and 0.7B activated ².

Attention uses Gated DeepSeek Sparse Attention (Gated DSA) combined with IndexCache, which reuses sparse indices across layers ². The residual pathway uses iHC (identity Hyper-Connections) ². The model card itself says the architecture was “inspired by DeepSeek and GLM” ².

The configuration closely resembles K-EXAONE 2.0, the 750B model LG AI Research released under Apache 2.0 on July 31. That model also runs 78 layers with 1 shared expert, 256 total, and top-8 activated per token — the only difference in this respect being the number of dense layers, which is two in LG’s case. Matching layer and expert counts at this level of detail suggests a settled pattern is emerging in large MoE design.

A blind evaluation with 163 in-house experts on 203 tasks

For evaluation results, Tencent offers figures from an internal blind evaluation. When 163 in-house experts scored outputs on 203 engineering tasks, Hy4 preview averaged 2.99 out of 4.00, slightly ahead of GLM-5.3 at 2.92 and Kimi K3 at 2.94 ¹. The model card also gives the win-loss breakdown: against GLM-5.3, 46.8% wins / 12.8% ties / 40.4% losses; against Kimi K3, 51.2% wins / 7.9% ties / 40.9% losses ².

All of these are evaluations Tencent ran in-house, not third-party reproductions. Kimi K3, the comparison model, is the one Moonshot AI published full weights for in July; GLM-5.3 is a Z.ai model. Note that GLM-5.3-Flash (320B/18B, MIT license), which this site covered on August 27, is a different model.

The model card includes benchmark charts, but they are images, and individual scores cannot be read off as text.

Tencent positions the model as being “among the top tier of open-source models” ¹. That is the company’s own framing, not an outside assessment.

The claim that the model optimized itself

The most striking part of the announcement is the passage saying the model joined its own development process. According to Tencent, Hy4 preview participated for the first time in the automated optimization of training methods, data strategies, evaluation frameworks, and low-level operators — proposing approaches, running experiments, and iterating on the results, with the resulting code, logs, and feedback feeding into subsequent rounds. This, the company says, “established an early-stage recursive self-improvement loop” ¹.

On the inference system, Tencent says the model autonomously analyzed bottlenecks and carried out multiple rounds of optimization on areas such as operator fusion and communication, raising end-to-end throughput by 31.8% over the baseline ¹.

This section warrants care. Both accounts come from Tencent, with no external verification. For the 31.8% figure in particular, the announcement does not define the baseline or state the measurement conditions. Treating it as the company’s claim that a model can optimize its own inference infrastructure is about as far as the source supports.

The cost of running it, and the option not to

The Apache 2.0 license is the practical difference for anyone weighing internal deployment. Restrictions on commercial use, modification, and redistribution are light, making it easier to plan self-hosting — much as when the 27B version of Qwen3.8 arrived under Apache 2.0.

That said, 770B is a reminder that obtaining weights and running them are separate problems. The model card points to vLLM or SGLang for deployment, after which you call an OpenAI-compatible API ². Even with the FP8 quantized build published alongside, this is not a scale you casually try on a laptop.

There are routes that skip the weights entirely. The model is available directly through Tencent products including WorkBuddy, CodeBuddy, Yuanbao, and ima, and through APIs via Tencent Cloud TokenHub and OpenRouter ¹. At launch, WorkBuddy and CodeBuddy offer two weeks of free use, and free access to the previous-generation Hy3 has been extended through September 30 ¹. The API pricing Tencent states is $0.834 per million input tokens, $2.501 per million output tokens, and $0.042 per million tokens for cache hits ¹.

Recommended parameters are temperature=0.9 and top_p=1.0, and reasoning mode defaults to “high” (deep chain-of-thought) ². For direct responses, you pass no_think as the reasoning_effort ².

What “preview” is doing in the name

The model card lists its own known issues. Real headroom remains in both pre-training and post-training, and the model ships while it still spends longer than necessary reasoning through complex tasks and tends to over-verify its own work ². As with Hy3 preview, the team writes, they would rather ship early and hear what breaks ².

For agent workloads, “spends longer than necessary reasoning” translates directly into token cost and response time. A low headline rate does not settle the bill if token consumption per task goes up. With spending data showing that top-tier models hold a small share of enterprise dollars, this is not a domain where the price sheet decides adoption. If you evaluate it, measure actual token consumption on your own representative tasks.

Tencent says it will keep shipping previews ahead of official releases, and that the next models in the Hy4 series are expected soon ¹. Given that what shipped here is the entry point to a series, there is little cost in waiting for the next release before running a serious evaluation.

Sources

1. Tencent Releases and Open-Sources Tencent Hy4 preview - Tencent official announcement (August 28, 2026)
2. tencent/Hy4-preview - Official model card on Hugging Face (created August 27, updated August 28, 2026)