RightSignal
1 of 6

Your RightSignal

AI

K2-Horizon: six fully open language models, with data and training code released

K2 Team, Zhengzhong Liu, Haonan Li, Xuezhe Ma, Alexander Moreno, Subham Sekhar Sahoo · Submitted 16 Nov 2026 · 60 upvotes yesterday
Preprint · not peer reviewedCode releasedTrending

The team released six language models from 0.9B to 375B parameters, along with checkpoints, pre-training data, logs and all training code. They also add new attention ideas and a way to speed up text generation.

Key finding: Their 36B model with 4B active parameters nearly matches a 32B dense model trained on the same data, with one-seventh the active parameters. About 10T tokens of their pre-training data were synthetic.

Why it matters: Few releases share everything needed to rebuild a model. This lets researchers study how scale and data choices really affect results.