# Recursive Self-Improvement, Ternary 27B Model for Laptops, Jev Architecture

#120 Sep 21 Architecture

https://www.aibutsimple.com/p/recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture

For this week’s research update, the community was fixated on Typesafe’s system-1 thinking model (Jev) [[blog]](https://typesafe.ai/blog/introducing-system-one-models-and-jev?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture), recursive self-improvement, and a 27B ternary quantized model that fits on a 16GB laptop GPU [[model]](https://prismml.com/news/ternary-bonsai?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture).

We'll explain these developments and how they work visually, analyze some results, and watch key movements to keep you updated in no time.

The week, ranked

| [DeepMind's Dream-RSI tests new search policies on replays of old runs](https://huggingface.co/papers/2609.14858?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) | ▲ 286HF |
| [An open-weight Jev alternative that makes calibrated decisions in 33ms](https://huggingface.co/convaiinnovations/laya?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) | ▲ 735HF |
| [PrismML fits a 27B model in 5.9GB with 1.76-bit ternary weights](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) | ▲ 1.2KHF |
| [DeepSeek-V4.1-Flash pushes KV cache compression even further](https://huggingface.co/papers/2609.19969?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) | ▲ 124HF |
| [A curated list of tools, guides, and projects built around Jev](https://github.com/yibie/awesome-jev?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) | ▲ 462GH |

[All entries](https://www.aibutsimple.com/pro?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture)Ranked by [AIBUTSIMPLE PRO](https://www.aibutsimple.com/pro?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture)

The field

| Theory & math |  | 914 -1.2k |
| LLMs & fine-tuning |  | 887 -1.1k |
| Robotics & embodied |  | 743 -220 |
| Models & releases |  | 701 -2.3k |
| Agents |  | 675 -404 |
| Reinforcement learning |  | 671 -350 |
| Other |  | 611 -667 |
| Inference & serving |  | 609 -313 |
| Vision |  | 558 -1.5k |
| Safety & alignment |  | 557 -490 |

All mentions tracked with [AIBUTSIMPLE PRO](https://www.aibutsimple.com/pro?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture)

#### Self-improving agents now improve how they search

Recursive self-improvement (RSI) has greatly risen in popularity this past year with the rapid improvement of LLMs and agents.

Well first, what is it? RSI is when a model or system repeatedly makes changes to itself (e.g. its own code or architecture) to increase its efficacy.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/d2205f7e-4b7c-4bbf-87f2-9e4215d04be0/Recursive_Self_Improvement_Search.png)

We covered an example of an RSI model called the Darwin Gödel Machine (DGM) [[article]](https://newsletter.aibutsimple.com/p/self-improving-agents-simply-explained) a while back, you can check it out [[here]](https://newsletter.aibutsimple.com/p/self-improving-agents-simply-explained).

In the literature, a common way this is done is through a self-improvement loop. A coding agent writes a program, an evaluator scores it, and the agent uses the score to write the next one.

We use an exploration policy to determine how the agent will search, select, and test new modifications, which paths to build forward, and when to stop.

The issue here is that the policy is typically a fixed a set of rules determined by hand.

- Improving this policy is also expensive, since we would have to go through a full search run of hundreds of modifications and scorings for just a single reward.

This week, Google Deepmind’s new popular paper Dream-RSI [[arxiv]](https://arxiv.org/abs/2609.14858?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) now improves the exploration policy by testing new policies on old runs.

So instead of improving the model (which they hold fixed), we improve how we search for modifications.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/f169eef3-b659-456e-b67a-7c5afabdb5ba/Recursive_Self_Improvement_Search__1_.png)

Why would we want to do that? The reason is overall efficiency and cost.

In Dream-RSI, every finished search (or run) is saved as a tree, where each node is one attempt with its code and score.

A new policy is a trajectory through that tree (e.g., deep on one branch, shallow subtrees, or even stopping early).

The key here is that since all outcomes are already recorded, scoring a policy simply reads the saved results.

This means we can score thousands of strategies for little compute and keep the best one.

Their discovery loop then repeats three steps:

- A coding agent uses the current policy on a real run, and a new tree is saved.
- A second LLM rewrites the policy several times and replays every version on each saved tree. The score is the sum of the best result minus a cost per attempt plus a bonus for parallel attempts.
- The best version runs the next round. The new round never scores worse (on past runs) since the current policy is a candidate.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/7a2733fc-e468-4c47-8f28-ff1358c2b84d/image.png)

Dream-RSI’s process diagram

Only the policy code changes, while the model and the evaluator stay the same (saving a lot of resources).

In their testing, compared to a fixed policy, Dream-RSI wrote GPU kernels that ran as fast with up to 2.43× fewer attempts.

Although this sounds great, the scoring is based on a replay history, which are all past attempts. A new policy can't be tested on new exploration yet.

Dream-RSI is an interesting search optimization that keeps the model fixed. What about current research that modifies the model itself?

NeoHorse-1 [[arxiv]](https://arxiv.org/abs/2609.08183?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) (Alibaba, Tsinghua) is a fairly new paper that retrains the agent through a harness.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/70b1c046-57c4-4002-a8be-f17d5f79d07b/Recursive_Self_Improvement_Search__2_.png)

Similar to Dream-RSI, their approach also believes that agent modification trajectories are worth saving.

Their argument is that since the harness holds an agent's context, tools, and environment, it already produces the feedback RSI needs. Then:

- At each user turn, it predicts how much effort the query needs, picks a model from a pool, and records what happens.
- Those records become training examples, keeping reasoning, tool calls, and results attached to context.

After each round, they evaluate the model, find weak areas, and shift its next training mixture towards those areas.

Other recent papers with different angles:

- EvoX [[arxiv]](https://arxiv.org/abs/2602.23413?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) (UC Berkeley) evolves its search strategy during a run by testing each one.
- Hyperagents [[arxiv]](https://arxiv.org/abs/2603.19461?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) (Meta) lets the agent directly rewrite itself.

[▲ 200+ paper upvotes (not model likes) on Hugging Face](https://huggingface.co/papers/2609.14858?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture)

2.43×

fewer attempts on VGG16

1.79×

fewer attempts on LayerNorm

[Read The Dream-RSI Paper](https://arxiv.org/abs/2609.14858?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture)

| Presented by | ![ai, but simple pro](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/59bb099e-a38a-445e-8f0d-b6cd300aa85d/_UPDATED__AI__but_simple_Pro_banner__1_.png) |

### Discover The Techniques That Matter Before Everyone Else

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/f32673d5-e0d5-4903-ba1d-a0393f9d7a56/image.png)

With [PRO](https://www.aibutsimple.com/pro?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture), unlock access to our AI research layer. Discover emerging techniques and hot topics across thousands of recent papers.

- Perform smart searches across papers, code, jobs, topics, and even interview questions.
- Get personalized daily overviews and a feed just for you. Pick from 10 preset roles or customize your dashboard.

Get full access to this article, as well as:

- Signals. Adopt, test, or skip recent techniques and models. We test efficiency savings and benchmark impact on our own infrastructure.
- Code with every issue (with practice examples + solutions)
- Ad-free reading

[Get AI Research Intelligence](https://www.newsletter.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture)

#### A 27B model fits on a laptop (ternary quantization)

A typical 27B LLM (e.g. Qwen3.8 27B) takes over 50GB of VRAM to store in 16-bit precision (called half precision).

To reduce the model’s storage size, we use quantization to store each weight with fewer bits.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/9d0f00f1-0c42-459b-ac17-d6397152ddd6/Quantization_Modern_Transformers__1_.png)

For instance, 4-bit integer or mixed quantization seemingly preserves most of the performance while reducing the usage to around 18GB (in this case, 4-bit GGUF).

Why not use even fewer bits? Below 4 bits, regular quantization starts to break the model. A 2.2-bit version of the same Qwen3.8 takes 7.3 GB, but its AIME26 score drops from 94.58 to 78.6.

PrismML's new model, Bonsai 2 27B [[release]](https://prismml.com/news/bonsai-2-27b?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture), stores the same model at around 1.76 bits per weight using ternary quantization.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/4a6b6f9b-cf3a-4a38-bf77-11686465037d/image.png)

The model weights only take around 5.9GB and retains 98.2% of the original model performance, while 8GB is typically required for a minimal “full” setup and around 12GB for a default setup.

But how does ternary quantization even work?

First, each parameter takes one of three values: −1, 0, or +1.

Each group of 128 weights also shares one 16-bit scale, which has to do with their size.

A weight is then stored as the scale multiplied by the ternary value.

- For instance, the weights 0.21, −0.02, −0.19, and 0.18 could be stored with a scale of 0.15 and the ternary values +1, 0, −1, +1.

Let’s compute how many bits this uses. Three possible values carry log⁡2(3) (around 1.585) bits of information. The scale then adds 16 bits shared across 128 weights:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/279f1c2c-0a09-4a90-85b3-d4a256e816b3/image.png)

Keeping about 0.1% of parameters in full precision and packing for use in fast kernels brings it to 1.76 bits.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/38ff8a94-b465-4bc2-823d-04e00a054023/image__346___1___1_.png)

Anything else? There’s also a clever trick they perform where weights are rotated in blocks of 1,024 using a fixed Hadamard transform.

The rotation spreads a few very large weights out so they don’t vanish when we quantize to a blocks’ 3 values.

What about real world performance?

- Speed: 27.1 tokens per second on an M5 Pro (standard autoregressive)
- Benchmark scores: In a 20-benchmark suite, the ternary model averages 83.9, while full-precision Qwen3.8 scored 85.4. Long agentic tasks aren’t as good on Bonsai, however.

Can quantization go lower than 1.76 bits? Intel's BITCOS paper [[arxiv]](https://arxiv.org/abs/2609.16338?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) reaches a new floor of 1.485 bits.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/bd012444-464c-4c5a-bb3f-23222adf41a2/image.png)

The authors discovered that 30% to 51% of the weights in 29 ternary models are simply 0. So their model stores one "zero or not" bit per weight, plus a sign bit only for nonzeros.

Here’s some other recent work on sub-2-bit models:

- BITCOS [[arxiv]](https://arxiv.org/abs/2609.16338?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) (Intel) packs ternary weights by zero density and speeds up decoding by up to 1.27× on Intel GPUs.
- Bonsai (1) 27B [[blog]](https://prismml.com/news/bonsai-27b?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) (PrismML), released a 1-bit build at 3.9 GB for an iPhone 17 Pro.

[▲ 1.9M downloads on Hugging Face](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture)

1.76-bit

quantization

98%

performance kept

[Read The Bonsai 2 27B Blog](https://prismml.com/news/bonsai-2-27b?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture)

---

#### Jev (system 1 reasoning model) breaks cost barriers

There are two modes of thinking we refer to in AI reasoning systems.

The first is System 1, which is fast and automatic (e.g., recognizing a face), and the second is System 2, which is slow and deliberate (e.g., reasoning through a proof).

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/d28e5804-9478-45d9-86e0-17d327761fc2/System_1_vs_System_2_Energy_Based_Transformers__5_.png)

We covered an architecture built on a similar principle, Hierarchical Reasoning Models (HRMs) [[article]](https://newsletter.aibutsimple.com/p/hierarchical-reasoning-models-hrms-simply-explained), a while back, pairing a slow planning module with a fast one.

One thing to understand is that what software tasks LLMs to accomplish are mostly System 1 decisions.

TypeSafe, released Jev [[blog]](https://typesafe.ai/blog/introducing-system-one-models-and-jev?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) this week, a System 1 model that doesn’t actually generate text.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/ecea0d46-6ab4-4c07-be39-df43473ca050/image.png)

Jev takes two inputs:

1. State, the context to judge, such as the ticket above
2. Questions, typed questions about that state

It uses 3 core question types: choices, scores, and nouls.

A choice picks one of up to 255 options, a score rates the state on ordered levels, and a noul asks whether a statement is true.

Jev outputs a probability on accepted answers directly. For example, whether a payment failure causing lost sales is urgent gives 0.999 probability.

All questions are answered in one pass, while being in parallel and isolated from each other.

Because of this, adding questions barely changes response time, and one question can't take up another's context.

It’s a little like a more robust classifier. A classifier learns one fixed set of labels, while Jev's questions and options are written at inference.

TypeSafe recommends each question be a gut-check a knowledgeable person could make in a few seconds, with harder judgments split up and recombined in code.

Jev is trained with Reinforcement Learning for Calibrated Decisions (RLCD), which aims for calibrated probabilities. This means out of all answers given 80% probability, about 80% should be correct.

Code can then act alone on high probabilities and hand low ones to a human.

Jev responds in 70 to 500ms, and on TypeSafe's own workflow evals, it runs 193.6× faster and 444.6× cheaper than GPT-6 Astra and Fable 5.1 outputs.

Other recent writing on Jev:

- Building a harness with Jev [[blog]](https://www.langchain.com/blog/building-a-harness-with-jev?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) (LangChain) uses it to route between models and block risky tool calls.
- Turing Post's guide to Jev [[article]](https://www.turingpost.com/p/what-is-jev-rlcd?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture) argues it repackages older work on classification, calibration, and selective prediction.

[193.6× faster than LLMs](https://typesafe.ai/blog/introducing-system-one-models-and-jev?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture)

444.6×

cheaper than LLMs

70ms

fastest response time

[Read Jev Release Notes](https://prismml.com/news/bonsai-2-27b?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture)

#### Wrapping up

This week's three areas are all roughly in the area of compute efficiency, which is particularly relevant as we’re becoming increasingly compute-constrained.

Dream-RSI optimizes self-improvement search attempts by testing new policies on saved replay trees of attempts.

Bonsai 2 uses ternary quantization to fit a 27B model in a laptop GPU, keeping 98% of the full precision benchmark score. Jev removes text based reasoning when only system 1 reasoning is needed.

That's it for this week's free version of AI, but simple. Get deeper analysis and access to the whole with PRO. See you next week!

---

Here's a special thanks to our biggest supporters: Sushant Waidande, Sai Krishna Pashapu, Bob Boncella, Demet Yazici, Imran Khan

Want to reach 10000+ AI engineers, researchers, and technical founders? Advertising Inquiries:

[WORK WITH US 📈](https://tally.so/r/A7Dgdk?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=recursive-self-improvement-ternary-27b-model-for-laptops-jev-architecture)

Feedback or inquiries? Send us an email at [edwin@aibutsimple.com](mailto:edwin@aibutsimple.com).