Nanbeige4.3-33B, Simply Explained

Welcome back to AI, but simple, your AI research beacon with visual explanations.

Nanbeige4.3-33B was released this past week and is the new open-source flagship from Nanbeige LLM lab.

It is a representation of modern open-source LLM performance at the 30B weight class, and it shows a result that’s surprising in the midst of all the LLM architecture optimizations we’ve seen recently.

In this issue, we’ll unpack and visually explain Nanbeige’s choice to completely revert to a completely standard transformer architecture, despite its previous model being a looped transformer.