# BDH-CQ: A 150M model just broke the ARC-AGI cost curve #115 Aug 17 Architecture https://www.aibutsimple.com/p/bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve Many of you will remember the [BDH architecture](https://newsletter.aibutsimple.com/p/the-dragon-hatchling-bdh-simply-explained) from a few weeks prior (if not, I’ve linked it above). The same research team at Pathway recently built a new reasoning system on top of it, and it’s been the top paper on Huggingface this past week. They call it [BDH-CQ (Engdahl et al., 2026)](https://arxiv.org/pdf/2608.09888?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve), and through combining in-context learning and recurrent latent reasoning, it obtains outstanding benchmark results… just not in the way you think. Instead of pushing for pure benchmark accuracy, BDH-CQ represents a potential paradigm shift in cost-efficiency scaling. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/687463e4-2af7-4ba0-9f35-e722cee3c184/BDH-CQ__a_Recurrent_Latent_Reasoning__ICL__Model__6_.png) One of the key results shows a 5-point benchmark decrease (-14%) on [ARC-AGI 1](https://arcprize.org/arc-agi/1?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) for an 11x cost reduction per task compared to GPT-5.6 Luna. It’s able to challenge the previously reported cost-accuracy Pareto frontier while holding just 150M parameters. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/4b1f0e50-0857-4ffd-994b-f891f1337369/BDH-CQ__a_Recurrent_Latent_Reasoning__ICL__Model__5_.png) The key contribution is a recurrent latent reasoning system. Typical reasoning models will generate intermediate thinking tokens in a scratchpad, consume them, and then continue to reason. BDH-CQ poses a pretty interesting question: do we really need to resolve reasoning into discrete words (tokens)? We’ll cover the surprising benchmark results, how BDH-CQ scales, get into how the architecture actually works, then discuss some key limitations from a detailed analysis from the paper. --- #### What You'll Learn 1. How BDH-CQ combines 2 well known fields of LLM research 2. The recurrent memory update that builds on previous examples 3. The 2 states of BDH-CQ and their updates, allowing for latent reasoning 4. What "breaking the cost-accuracy frontier" actually means, and the full results behind them 5. The limitations and analysis of BDH-CQ’s reasoning behavior #### What's Helpful to Know - In-Context Learning (ICL) - In-context learning allows a model to acquire a new skill from examples presented at inference time, usually carried out by [few-shot prompting (Brown et al., 2020)](https://arxiv.org/abs/2005.14165?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve). - ICL has seen a rapid shift in the field by using chain-of-thought (CoT) prompting in autoregressive language models by generating intermediate reasoning tokens. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/d75af02b-93e9-4d33-98ca-50ed079ebef3/In-Context_Learning_for_Multimodal_LLMs.png) - Latent reasoning - When a model reasons using its continuous hidden latent state instead of transforming every intermediate result back into vocabulary space, it decodes only the answer at the end. - [ARC-AGI](https://arcprize.org/arc-agi?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) - A family of reasoning and general intelligence benchmarks known as the “Abstraction and Reasoning Corpus.” - Each task specifies a new visual transformation (in a grid). Through a handful of examples, models must predict the exact output. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/99352a4d-be72-4485-863f-877ee3160862/image__293___1_.png) ARC-AGI sample tasks --- | Presented by | ![ai, but simple pro](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/59bb099e-a38a-445e-8f0d-b6cd300aa85d/_UPDATED__AI__but_simple_Pro_banner__1_.png) | ### You Build. We Verify, Then Send The Most Impactful Techniques. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/acd2356a-bb4c-4761-bff6-c70321592212/image.png) With PRO, unlock access to Signals. We tell you whether to adopt, test, or skip recent techniques and models. Learn whether latent reasoning can meaningfully cut costs for language models, how you can adopt them, and more, all verified by our research team. Get full access to this article, in addition to: - Basis, our AI research layer - Daily overviews - Models and code - Community topics - Interview questions and job trends - Code with every issue (with practice examples + solutions) - Ad-free reading We use a proprietary ranking system based on 50+ factors to ensure you get the most relevant entries for your interests and career. [AI Research Intelligence With PRO](https://www.aibutsimple.com/pro?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) #### Combining In-Context Learning With Latent Reasoning Latent reasoning and in-context learning have mostly seen separate development in the research landscape. In-context learning is common in reasoning language models, learning flexibly from context but allocating additional compute through generated tokens. Latent reasoning is commonly found in compact recursive solvers (HRM, TRM), but current solvers reason from past cases on specific evaluation tasks (e.g., ARC-AGI), instead of making general rules. They are transductive. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/c120ce6c-9ac5-4f01-8fef-3f62653037a8/BDH-CQ__a_Recurrent_Latent_Reasoning__ICL__Model__7_.png) One side adapts from examples and expends tokens. The other reasons efficiently, but largely in the literature, they are optimized on the task first. BDH-CQ bridges these capabilities together to form a capable and efficient model architecture. Carrying out a previously unseen task updates its recurrent memory, and all queries are then reasoned through in a high-dimensional latent workspace. - Here, intermediate reasoning states are not decoded into language. BDH-CQ aligns memory, adaptation, and inference in the same mathematical system. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/10266116-81ab-4c05-9dea-6c0a341b1a64/BDH-CQ__a_Recurrent_Latent_Reasoning__ICL__Model__16_.png) Task identifiers and other fixed-domain samples are excluded in training (enabling a full latent reasoning outside its domain), and in ICL fashion, no parameters are updated at inference time. --- #### The Recurrent Memory Update Contrary to the previous BDH paper, the implementation details are proprietary, but they leave the readers a high-level overview. Few shot examples (which they refer to as demonstrations, including inputs and outputs) are given one at a time, and they update a recurrent state. The recurrent memory update is governed by this simplification: ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/95f6c179-05f9-4028-8981-13ad39b8429f/image.png) Here, Dt​ is the content of the t-th demonstration, and Uθ is a parameterized update function, but the parameters θ stay fixed. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/5c9f72d8-207d-4e7f-bab1-6ea99e569af3/BDH-CQ__a_Recurrent_Latent_Reasoning__ICL__Model__9_.png) Since the examples are processed element by element, what is available to a later demonstration depends on the associations built from earlier samples. The authors don't derive this update exactly from attention (although related), but they point out several existing ideas that roughly model it. - [Fast-weight memories](https://arxiv.org/abs/1610.06258?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve), storing associations in matrices at inference - [Linear attention](https://arxiv.org/abs/2006.16236?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) is a similar kind of object as the state S - [Feed-forward layers](https://arxiv.org/abs/2012.14913?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) behave like key-value stores What these share is that the memory updates by addition. Linear attention is the conceptually simplest version (noted in the paper, to get an idea for how it can work), and it gives the below special case: ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/96e1a2f4-df9b-4dba-93bb-6f19fea8b7b8/image.png) Note that there is no growing key-value cache. Instead of “appending” to the memory, the memory is a state that gets corrected. The paper does not disclose the actual update rule for BDH-CQ, but they seem to be building on the BDH architecture, which previously used ReLU-low-rank transformations with linear attention in a large neuron space. --- #### Two States, Two Jobs BDH-CQ takes 2 inputs: - Demonstrations, D = {(xt, yt)}, from t=1 to K, with K being the number of demonstrations given in a specific task. - Query, x* ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/7c9d7aaa-0527-4267-8041-bafecc7aac90/BDH-CQ__a_Recurrent_Latent_Reasoning__ICL__Model__11_.png) After initial demonstration ingestion, BDH-CQ takes a query x* and performs iterative computation in a structured latent workspace Hr. The authors summarize this process through the following equations: ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/cd237a56-4d81-4e3a-928b-2ed7c4ec9ef3/image.png) There are 2 main states that reside in the equations: S and H. - S is the ICL portion that changes as evidence arrives, supporting learning from context. H is the latent workspace that carries the computation for the query. Above, H0 is the initial latent workspace, and it updates for all reasoning steps r between 0 and R - 1 as Hr. There are 3 learned functions: - Eθ represents an encoder - Fθ represents a parameterized update - Gθ represents a decoder SK​ is the same St memory after all K demonstrations have been read. The K subscript is fixed in all 3 equations, so it is frozen for the duration of the reasoning loop. At the end, the prediction ŷ is decoded (Gθ) from the last latent reasoning state HR. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/ec3af4eb-9a29-4924-973d-ef14a0b83349/BDH-CQ__a_Recurrent_Latent_Reasoning__ICL__Model__12_.png) Now, a quick note: Why bother keeping the thinking workspace continuous? A [recent paper](https://arxiv.org/abs/2510.25741?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) (Zhu et al. 2025) proved for directed graph reachability that continuous thoughts can encode several search frontiers at once and expand them in parallel. One state can hold multiple hypotheses, while a token stream has to commit to one (and write it down). --- #### Training and Scaling Reasoning Effort They trained a 150M-parameter model on a collection of ARC-style data across varying levels of latent reasoning (so we can select the effort level). The dataset combines privately gathered examples with publicly available data: - [ARC-AGI-1 training set](https://github.com/fchollet/ARC-AGI/tree/master/data/training?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) - [RE-ARC](https://github.com/michaelhodel/re-arc?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) - [ConceptARC](https://github.com/victorvikram/conceptarc?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) - [ARC-Heavy](https://github.com/neoneye/arc-dataset-collection/blob/main/dataset/ARC-Heavy/readme.md?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) - [ARC-GEN100K](https://github.com/google/arc-gen?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) They also augmented the data for more variability. BDH-CQ was evaluated on the 400-task public ARC-AGI-1 evaluation split and showed surprising results. Following the 2-attempt convention (pass@2), the default config reaches 29.5% pass@2 in ~0.85 H200 GPU-seconds per task. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/cad29e50-dd80-42aa-96c8-80d473f87a59/BDH-CQ__a_Recurrent_Latent_Reasoning__ICL__Model__14_.png) They assume a fixed $3 per H200-hour, resulting in a cost of $0.00070/task. Over effort levels, pass@2 ranges: - 21% at low (22% cheaper) - 27% at medium (11% cheaper) - 29.5% at high Using ARC Prize’s reported costs, BDH-CQ is ~11x cheaper (and only 14% worse) than GPT 5.6 Luna (Low), which scores 34.2% at $0.008. This even accounts for the 80% public API price reduction for Luna that took place on July 30. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/40e5e7d9-d0bf-4400-bfbc-fe1a8ccd1eb9/image.png) Full graph from paper BDH-CQ is also much cheaper than the other recursive latent solvers, with $1.48 per task for HRM and $1.76 for TRM. As the graph shows, BDH-CQ lies beyond the previously reported cost-accuracy Pareto frontier. No other model reaches at least this accuracy at equal or lower cost. --- #### Limitations of BDH-CQ The authors have an interesting analysis portion that acknowledges key limitations. They froze the model and built their own ARC-like tasks to test three aspects of generalization: - Contextual binding (pick up a rule from demonstrations) - Extrapolation (extrapolate past the demonstrated range) - Execution (output it onto a grid with supported context) It asks whether the demonstrations alone can teach generality to tasks it has never seen. Three tasks that they’ve built: - Color mapping (e.g., when blue becomes red, green becomes yellow, so on), where BDH-CQ gets all outputs right, working up to 8 swaps. - Sorting bars by height, near-perfect performance at 5 bars, degrading heavily up to 8 bars (1/24) - Resolving nested frames, near-perfect for 4 frames, degrading at 5 frames (29/36) ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/7247cc81-314b-4d4b-930a-085791f73c7b/image.png) Binding passes for all 3 of the above tasks; they’re able to pick up some sort of rule from examples. Extrapolation fails for all 3 tasks, especially sorting bars. Execution is interesting because it passes for only 1 task: the nested frames. In another they added one demonstration (example) at the hard setting, and nesting increases from 19/24 to 24/24. On [ConceptARC](https://arxiv.org/abs/2305.07141?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve), a separate benchmark that gives every task three test inputs, BDH-CQ gets 77.92% of individual inputs right but only 59.38% of tasks where all three must be correct. One additional thing: [ARC-AGI 1](https://arcprize.org/arc-agi/1?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) is pretty old at this point, being released in 2019, so it’s not the most current when it comes to frontier reasoning. [ARC-AGI 2](https://arcprize.org/arc-agi/2?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) and [ARC-AGI 3](https://arcprize.org/arc-agi/3?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) were released in 2025 and 2026, respectively, and those are the next targets. --- #### Conclusion BDH-CQ is an interesting challenge to the cost-accuracy frontier of reasoning models. It does not beat frontier models on ARC-AGI-1, but it reaches an acceptable score at a cost low enough that the comparison stops being about accuracy. It also gets there using in-context learning, using demonstrations to adapt to new tasks, which is an upgrade over compact recursive solvers. The authors are already pointing past ARC. Since BDH layers handle language modeling as well as latent reasoning, they are set up to tackle text generation as well. Future systems could also hold reasoning in latent space while decoding intermediate steps only when a tool call or a verification step actually requires words. It’s interesting to see where latent reasoning with ICL goes from here. That's it for this week's issue of AI, but simple. See you next week! --- Here's a special thanks to our biggest supporters: - Sushant Waidande - Sai Krishna Pashapu - Bob Boncella - Demet Yazici - Imran Khan --- Want to reach 10000+ AI researchers? Advertising Inquiries: [WORK WITH US 📈](https://tally.so/r/A7Dgdk?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=bdh-cq-a-150m-model-just-broke-the-arc-agi-cost-curve) Feedback or inquiries? Send us an email at [edwin@aibutsimple.com](mailto:edwin@aibutsimple.com).