TL;DR: Open-ended discovery requires not only strong reasoning but also the intuition to quickly spot promising directions among countless possibilities. We argue that retrievers are the natural place to learn this intuition, and that the ceiling for learned intuition sits far above human level because a retriever can scale its search far beyond what any one person can read.

Discovery Begins with Deciding What to Explore

Exploring an open-ended space of possibilities requires quickly identifying promising connections. Experts often know which direction is promising before they can say why. A researcher hearing a half-formed idea may sense which old result could unlock it, a link that seems obvious only in hindsight. The clearest evidence comes from chess, where grandmasters search about as many moves as weaker experts yet consider better moves first1. In this view, intuition is “nothing more and nothing less than recognition”23. It first narrows the space to possibilities that deserve careful thought, and reasoning takes over from there.

Most recent progress in AI has come from scaling deliberate reasoning. Models that think longer or explore many paths in parallel now reach striking results on hard problems45. However, reasoning still depends on what it is given to reason about. For open-ended problems, much of the difficulty lies in deciding what is worth thinking about, especially when the space of possible connections far exceeds what a model can assess individually. Intuition begins exploration by narrowing a vast space to a few promising candidates for further investigation.

Research intuition acts as an upstream retrieval step in open-ended discovery, surfacing a few promising candidates (including distant connections) from a vast possibility space for subsequent reasoning and decision-making.

Rethinking Retrieval as Learned Intuition

Retrieval is usually framed as finding documents semantically similar to a given query. But a retriever can also play a more demanding role, closer to how an expert’s intuition picks out the few ideas worth pursuing among countless possibilities. Throughout this post, we use “retriever” broadly to mean any system that narrows a large corpus to a few candidates, including BM25, grep, dense embeddings, agentic search, and hybrid systems in which a model navigates the corpus directly6.

This role will become increasingly important in open-ended discovery. The literature is far too large to fit in any model’s context, and it keeps growing at an increasing rate7. Even if context kept expanding, reasoning carefully over every document would remain impractical, for both reasoning and decision models8. A model can only reason about what is in its context, so something must choose what goes in. The retriever, whatever its form, decides what reasoning can work with.

We argue for studying much harder retrieval problems, ones that test the intuition experts use to see subtle connections, and for explicitly training retrievers on that intuition. The retrieval community has developed efficient, scalable parameterizations for narrowing millions of documents to a few, and we can reuse them to learn intuition instead of similarity. By selecting which parts of an external corpus to bring into a model’s context, retrieval shapes what the model can consider without storing all that knowledge in its parameters. In this sense, a retriever with expert-level judgment can serve as a model’s learned intuition for choosing external knowledge to reason over.

Evaluating Research Taste with ScholarCatalyst

“If I have seen further, it is by standing on the shoulders of giants” — Isaac Newton.

In scientific research, this intuition (often called taste) helps researchers find the papers that can move an initial idea forward. Discovery builds on knowledge already recorded, much of it scattered across a vast literature and diverse fields. Great researchers can spot these less obvious connections from a vague starting idea, often seeing links that seem inevitable in hindsight even though one might never have thought to look for them. Their research taste helps them navigate this vast search space and identify which prior work is worth building on.

To study this capability, we built ScholarCatalyst, a benchmark that evaluates retrieval at the earliest stage of research. Given an initial research idea, systems must retrieve catalyst papers that project authors judge could help advance their work. Because published records rarely capture which papers could have helped a project, we collected these judgments directly from 184 lead authors across 207 recent computer science projects. The benchmark contains about 191k computer science papers, providing a constrained test of literature search9.

ScholarCatalyst asks whether a system can retrieve prior papers that inspire progress on a research project. Such a paper may share little semantic similarity with the question yet offer an insight that advances the project (right), while topically related work may not (left).

Our results reveal a gap between current retrieval systems and expert research intuition.

1. Retrieval Remains a Bottleneck Even for Strong Reasoning Models. Agentic search does not consistently outperform strong embedding retrieval. It achieves 0.42 Recall@20, compared with 0.48 for the embedding system, despite using the same retrieval tool. Even Claude Fable 5.1 reaches only 0.51, though it may have seen the completed papers during training. Recall@N measures the fraction of relevant papers that appear in a system’s top N results.

Agents use reasoning to formulate queries and guide later searches, but they access the corpus through embedding- or grep-based retrieval. At each step, they inspect only the small set of candidates those tools return, not the full literature. If a search misses a catalyst paper and later searches fail to recover it, the agent never gets a chance to reason about it. This makes candidate coverage, how systems search a vast corpus and surface a small set of candidates, a central bottleneck.

Current systems still struggle to find catalyst papers. Agentic search doesn’t beat embedding retrieval, even with the same retrieval tool underneath. Even Fable 5.1 finds only about half, despite potentially seeing the finished papers during training.

2. Recognition Could Help Us Hill Climb Toward Better Retrieval. Our results point to a potential signal for future progress. We also find a gap between finding a gold candidate and recognizing its value. Systems may miss catalyst papers, but stronger models can recognize their value once they see it. When we add these missed catalyst papers to the candidate list, stronger models are more likely to rank them among the top results. This suggests that their ability to recognize promising papers could provide feedback to improve retrieval over time.

Stronger models are better at recognizing relevant papers once they are shown. For Claude Fable, adding missed inspiration papers to the candidate pool raises Recall@5 from about 27% to 50%.

Toward Learning Intuition

Human intuition has limits, since it can only draw on what one person has experienced. In research, that is a small slice of a literature far larger than anyone can read10, and it keeps growing. Connections between published findings can stay unnoticed simply because no one has seen the relevant pieces together. A model faces no such limit, since it can draw on everything ever recorded. If it could also judge that record with an expert’s sense of what matters11, its intuition would be superhuman in breadth alone.

How humans develop expertise offers a clue for training intuition. Skilled intuition needs cues that reliably predict outcomes and prompt feedback on one’s judgments12. While open-ended problems offer little explicit feedback, models show a recognition asymmetry13: they recognize a useful candidate more reliably than they find one141516. The model can therefore act as its own critic, judging the value of candidates it could not have found. Applied across a whole corpus, these judgments could provide feedback at a scale no group of experts could match, as long as the critic is strong enough to be trusted.

So far, the field has mostly studied retrieval in relatively easy settings where relevance means similarity, and performance in this regime seems close to saturation. The harder setting remains largely open when the right document shares little with the question, and recognizing it requires deep insight into the problem itself. ScholarCatalyst studies one such hard setting, and many more surely exist. Problems like these call for a new generation of retrievers that learn intuition from the ground up, at expert level and beyond. Since retrievers already scale to millions of documents, intuition learned there can draw on the whole literature, which makes better retrievers perhaps the most direct path to superhuman intuition.

Reference

Acknowledgements

The ScholarCatalyst benchmark would not have been possible without the researchers who shared how their discoveries came about (see paper for full list), and we are deeply grateful for their contributions.

We’re also incredibly grateful to the entire ScholarCatalyst team: Bo Liu, Dayoon Ko, Rulin Shao, Seungone Kim, Graham Neubig, Pang Wei Koh, Aakanksha Chowdhery, Akari Asai, Omar Khattab, Yejin Choi, Gunhee Kim, and Chelsea Finn.

Special thanks to Yejin Choi, Akari Asai, Souradip Chakraborty, Jacob Li, Braden Hancock, and Rulin Shao for their thoughtful feedback that helped shape this blog post.

Finally, we’re grateful to the Laude Institute for supporting our continued exploration of this research direction through the Slingshot program.

Citation

@article{kim2026rethinking,
  title  = "Research Intuition as a Retrieval Task",
  author = "Kim, Sohyeon and Lee, Yoonho",
  year   = "2026",
  month  = "Nov",
  url    = "https://ai.stanford.edu/blog/scholarcatalyst/"
}

This post is based on our paper ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research (Paper, Data, Code)

@article{kim2026scholarcatalyst,
  title   = {ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research},
  author  = {Kim, Sohyeon and Lee, Yoonho and Liu, Bo and Ko, Dayoon and
             Shao, Rulin and Kim, Seungone and Neubig, Graham and Koh, Pang Wei and
             Chowdhery, Aakanksha and Asai, Akari and Khattab, Omar and Choi, Yejin and
             Kim, Gunhee and Finn, Chelsea},
  journal = {arXiv preprint arXiv:2610.02202},
  year    = {2026}
}
  1. Adriaan D. de Groot, “Thought and Choice in Chess,” Amsterdam University Press, 2008. ↩

  2. Herbert A. Simon, “What Is an ‘Explanation’ of Behavior?,” Psychological Science 3(3), 150–161, 1992. ↩

  3. Daniel Kahneman, “Thinking, Fast and Slow,” 2011. ↩

  4. OpenAI, “An OpenAI model has disproved a central conjecture in discrete geometry,” 2026. ↩

  5. OpenAI, “On the Navier–Stokes Millennium Prize Problem,” 2026. ↩

  6. Wan et al., “Retriever-Free Retrieval: Navigating Documents with Decision Models,” 2026. ↩

  7. The volume of scientific literature continues to grow with no sign of slowing. arXiv recently reported more than 3 million submissions in one year, which is over 40K in a single month, and a 6x increase in cs.AI submissions in two years. As the search space expands, quickly identifying papers that could advance a project becomes increasingly important. ↩

  8. Typesafeai, “Introducing System One Models & Jev,” 2026. ↩

  9. Sohyeon Kim and Yoonho Lee et al., “ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research,” 2026. ↩

  10. Don R. Swanson, “Undiscovered Public Knowledge,” The Library Quarterly 56(2), 103–118, 1986. ↩

  11. Jacob Xiaochen Li, “Machine Studying,” 2026. ↩

  12. Daniel Kahneman and Gary Klein, “Conditions for Intuitive Expertise,” 2009. ↩

  13. Recognition Asymmetry: Models may be better at recognizing (verifying) a candidate’s value once it is shown to them than at retrieving it from a large corpus. ↩

  14. Sohyeon Kim and Yoonho Lee et al., “ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research,” 2026. ↩

  15. Diane Tchuindjo et al., “OBLIQ-Bench: Exposing Overlooked Bottlenecks in Modern Retrievers with Latent and Implicit Queries,” 2026. ↩

  16. Jack Morris, “How to Train the Best Embedding Model in the World,” 2026. ↩