# The Best NeurIPS Papers of 2025, Simply Explained #89 Feb 16 Papers https://www.aibutsimple.com/p/the-best-neurips-papers-of-2025-simply-explained 2025 was a defining and eventful year for deep learning (DL), machine learning (ML), and AI, defined by large-scale LLM research, validation, and new hybrid models. In 2026, there are no signs of slowing research progress. As we move forward in 2026, let’s take a look at some award-winning, highly cited, and highly influential NeurIPS papers from 2025, diving into the latest research to uncover some insights for the future. ### Reinforcement Learning Does Not Increase LLM Reasoning Capacity Paper: [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://arxiv.org/abs/2504.13837?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-best-neurips-papers-of-2025-simply-explained) Yue, Chen, et al., Tsinghua University - NeurIPS 2025 - Best Paper Runner-up Reinforcement Learning with Verifiable Rewards (RLVR) is one of the most popular and “effective” LLM fine-tuning techniques, used to train reasoning models like OpenAI o1 and DeepSeek-R1. At its core, Yue et al.’s paper is on the empirical analysis of RLVR and if it actually increases the model’s capability. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/cc51dfb0-fad9-4d48-bdfa-7c68c5e2868b/RLVR_Overview.png) The authors investigate whether RLVR actually teaches models how to reason or if it just makes them better at guessing what they already know. To test this theory, the researchers looked into the "reasoning boundary" of LLMs. Usually, models are evaluated on “pass@1,” which measures if the model gets the answer right on the first try. This paper instead used “pass@k” with very large values of k. For example, letting the model attempt a problem 256 or more times. If RLVR teaches new reasoning skills (e.g., "how to use a new math theorem"), the RL-trained model should be able to solve problems that the base model never could, given a sufficient and large number of guesses. The authors took a base LLM and an RLVR-trained version of it. They then sampled thousands of outputs for challenging math and coding problems to see the total set of solvable problems for each. It turned out that the RLVR fine-tuned model didn’t consistently score better (solve more problems) than the base LLM. RLVR did not expand the reasoning boundary, meaning that if the base model could not solve a problem in 10,000 tries, the RL model could not solve it either. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/6f2ba7db-5166-4d82-b225-019a609f601e/Pass___k_RLVR_fine-tuning__1920_x_1010_px___1920_x_1150_px___1920_x_1300_px_.png) What RLVR actually does is bias the probability distribution. It takes reasoning paths that already existed in the base model with low probability (e.g., 0.01%) and increases them to high probability (e.g., 90%). RLVR increases pass@1 (making the model more usable and fast), but what’s surprising is that it actually decreases pass@k at high k. The RL training collapses the model's diversity, making it more narrow than the base model, just slightly more specialized, in a sense. Well, what’s the solution? The paper found that [knowledge distillation](https://www.aibutsimple.com/p/llm-knowledge-distillation-simply-explained?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-best-neurips-papers-of-2025-simply-explained), training a student model on the outputs of a larger teacher, does expand the reasoning boundary. It introduces new patterns the student didn’t originally learn. --- ### [Level Up Your Career With AI, But Simple PRO](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-best-neurips-papers-of-2025-simply-explained) ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/281697a8-c915-435c-a68b-43374b671698/aibutsimple_academy.png) Supporting us by purchasing a PRO membership will get you: - Exclusive Hands-on Tutorials With Code - Full Math Explanations and Beautiful Visualizations - Ad-Free Reading for an Uninterrupted Learning Experience [Break into AI or sharpen your edge by joining today!](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-best-neurips-papers-of-2025-simply-explained) --- ### Gated Attention for LLMs Paper: [Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free](https://arxiv.org/pdf/2505.06708?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-best-neurips-papers-of-2025-simply-explained) Qiu, Wang, Zheng, et al., Qwen Team - NeurIPS 2025 - Best Paper In standard LLMs, the softmax function forces attention scores to sum to 1. If the current token is irrelevant, the model has nowhere to allocate the rest of the probability score, so it dumps it on the first few tokens. We call this an attention sink, and it is commonly observed in many LLMs. Mathematically, the attention weight for token i attending to token j is: ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/8259108c-9a5b-4e31-a4cf-6e6cbeead78d/image.png) Due to the softmax denominator, the sum of alphas must equal 1. The model must attend to something. If the current token i finds no relevant information in the context (all dot products si,j are low/negative), the softmax function forces it to move the probability mass somewhere. We can fix the attention sink problem, where models irrationally attend to the first few tokens, and improve long-context performance by simply adding a gate to the standard transformer attention mechanism. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/da71eb62-fa4c-4c2a-a381-0001711b9c3c/Attention_Sink_NeurIPS__Gated_Attention_2025.png) Regular transformers use scaled dot-product attention (SDPA) followed by a linear projection. Recall the standard SDPA equation for a singular head below. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/cb56af1c-9966-4aa4-99a0-7969c4723f69/image.png) It gets linearly projected by another matrix, WO to obtain the projected output (O). ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/1cd8f508-8e10-40d9-bad3-905e3a6e1636/image.png) This paper introduces a gate between the attention output and the final projection. Instead of just outputting Y (the attention result), the layer outputs Y' = Y ⊙ σ(XW). ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/aba0462d-acbe-4e0f-b919-3cdd79ae5552/image.png) Here, Y is the standard attention output, σ is the sigmoid activation function (non-linear), ⊙ is the element-wise or [Hadamard product](https://en.wikipedia.org/wiki/Hadamard_product_(matrices)?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-best-neurips-papers-of-2025-simply-explained), and XW is a learnable projection of the input X that determines how much of the attention output should actually pass through. While XW is the actual projection, W is the matrix that contains the tunable parameters to control the output. The final projected output is obtained using the same method (projection by another matrix WO). ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/93f7d619-0f22-415c-aefb-d43004c5d0b2/image.png) The gated attention mechanism allows the model to simply close the gate (output close to 0) for irrelevant tokens. It doesn't need a sink anymore because it can choose to ignore the attention head entirely. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/1ad000f7-e087-4141-9ecd-2a90a5425c8d/Gated_Attention_NeurIPS_2025.png) The gate is applied per attention head. This introduces dynamic sparsity; some heads can be active while others can be deactivated depending on the input. This architecture (used in [Qwen3-Next](https://qwen.ai/blog?from=research.latest-advancements-list&id=4074cca80393150c248e508aa62983f9cb7d27cd&utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-best-neurips-papers-of-2025-simply-explained)) stabilizes training at high learning rates and naturally improves performance on long-context tasks without requiring massive retraining. ### LLMs Are Slowly Losing Creativity Paper: [Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)](https://arxiv.org/pdf/2510.22954?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-best-neurips-papers-of-2025-simply-explained) Jiang, Chai, et al., University of Washington - NeurIPS 2025 - Best Paper Award (Datasets and Benchmarks) As we rely on different LLMs (Claude, GPT, Llama, DeepSeek) for creativity, we actually aren’t getting as diverse of outputs as we think. The paper tackles current LLM math and code benchmarks that have "correct" and single answers, highlighting that they can't measure diversity. The authors released INFINITY-CHAT, a dataset of 26,000 open-ended queries, including prompts like "Generate a joke about electric vehicles” or "Can happiness truly be measured?" ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/b9b8e60e-6443-42a3-bad9-61d82771e5bc/Copy_of_Pass___k_RLVR_fine-tuning__1920_x_1010_px___1920_x_1150_px___1920_x_1300_px___1_.png) They categorized prompts into creative generation, brainstorming, speculation, and more groupings seen above. The authors measured intra-model repetition, if the model says the same thing if asked twice, and inter-model homogeneity, if different models give the same answer. The study found a shockingly high level of similarity. When asked to write about time, most major LLMs choose the metaphor "Time is a river." When asked for marketing advice, they use the exact same adjectives, such as "sleek" and "streamlined." In the paper, homogeneity, or semantic similarity, is measured using cosine similarity (dot product). The paper treats model outputs as vectors in a high-dimensional semantic embedding space. Let vA be the embedding vector of the response from Model A, and let vB be the response from Model B. The similarity is calculated as: ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/a3d6af2d-eb67-4ee1-9b1a-8cc0a70d75bd/image.png) - If Sim = 1, the outputs are semantically identical. - If Sim = 0, they are unrelated (they are orthogonal to each other). It turns out that the homogeneity between models is shockingly similar. The semantic similarity between responses from competing companies ranges from 0.71 to 0.82. By using PCA to reduce the sentence embeddings to 2 dimensions, the models’ generated outputs lie very closely together in clusters for most prompts. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/d586eaa1-ac1e-4bae-87e7-8eb6c008d3f9/Hivemind_PCA_NeurIPS_2025__1_.png) This is the convergence of most models into the “hivemind” they refer to in the paper. How could this be happening? The authors hypothesize some possible root causes. The first is the use of RLHF normalization. Alignment training (RLHF) tends to punish outliers and reward safe and consensus-based answers. This pushes most LLMs to the mean of outputs. Additionally, models are increasingly trained on synthetic data generated by other models like GPT-4, leading to a decrease in diversity in word choice and problem-solving approaches. The paper suggests that relying on LLMs for brainstorming may reduce human cognitive diversity. Instead of expanding our creative ability, LLM outputs might be narrowing it to a statistically probable average thought. --- Here’s a special thanks to our [biggest supporters](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-best-neurips-papers-of-2025-simply-explained): - Sushant Waidande - Sai Krishna Pashapu - Bob Boncella - Demet Yazici --- If you enjoy our content, consider supporting us so we can keep doing what we do. Please share this with a friend! Want to reach 50000+ ML engineers? Let’s work together: [WORK WITH US 📈](https://www.passionfroot.me/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-best-neurips-papers-of-2025-simply-explained) Feedback, inquiries? Send us an email at [edwin@aibutsimple.com](mailto:edwin@aibutsimple.com). If you like, you can also donate to help our team push out better newsletters every week! [Support us ☕](https://ko-fi.com/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-best-neurips-papers-of-2025-simply-explained) --- That’s it for this week’s issue of AI, but simple. See you next week! —AI, but simple team