By any measure, I think large language models (LLMs) provide us with a very interesting picture of reality – the reality as encoded in language patterns. Language itself (multimodal, for sure – the vocabulary consisting not only of word tokens but also graphics and audio tokens, and also of natively multimodal tokens) is being used to paint the picture of reality, just like we would use a paint brush, or a photograph, or a spectrogram. And then this “LLM-o-gram” needs to be interpreted by humans (i.e., by means of a tailored harness) – along the lines of what OpenAI suggests.
Perhaps an LLM is just like a new kind of device for observing the world; just like we cannot observe the Higgs boson with our bare eye, we hardly can observe, say, half a billion works from OpenAlex with our bare eye. Sorry for this out-of-nowhere example – it’s just my professional field of evidence synthesis and meta-research that makes me think in these terms. In this field what we do, simply put, is pull together a lot of academic literature to figure what findings emerge across research studies.
So, because our bare eye is failing here, due to the data being too small or too big, we must develop a proxy. In the traditional systematic literature review (SLR) case, the proxy is, for example, a search strategy and the human “screening” process that the literature undergoes. Either way it’s not pure observation. Why cannot AI replace those proxies wholesale? I think it totally can.
Evidence synthesis was the human way to do things – relying on provenance, documentation, and ad hoc decisions. AI doesn’t inherently care about all of those things, nor it probably needs to. Clearly AI has its own way of seeing the world, a way beneficial and truthful in its own way. Truly harnessing AI power perhaps means yielding more and more autonomy to AI in letting it do its own thing.
The key here, therefore, would be epistemic deference to AI rather than trusting AI; accepting that we may not, feasibly, comprehend all the little things AI did in the process – and thus refrain from doing it by hand. Hand in hand with deference (pun not intended) comes instructing the LLM to fill out a human-interpretable harness in retrospect.
By no means am I the inventor of the term “auto-SLR,” but here is my vision: Perhaps what AI could do is go over these half a billion works, in its own LLM/agentic ways, and then secondarily build a verifiable evidence package (= harness) that will satisfy our human trust needs.
To qualify this even more specifically – this would mean replacing the study of “hard reality” (that is, of tangible physical objects and precise sensor data, such as academic titles/abstracts or PDFs that we would read using our own sensory organs) with the study of its LLM-generated representation. After all, who are we to judge that LLM representation of reality is false?
When there is a clear goal, such as, “Here is the (unambiguously!!) defined eligibility criteria and here is the population of academic works to review,” then from that point human attempts to prescribe specific steps to accomplish this may in fact hurt AI performance.
Alternatively, we could let AI do its own thing (while also letting it work for a while!! this is a very important remark, as this kind of performance scales with test-time), and it can figure out much better than me – honestly, much better than I can – how to best go about that half-a-bil snapshot and how to find eligible works there.
AI can do better because this kind of shell work is native for the agent, as it thinks in tool calls while my thinking is limited to a few worked patterns I could learn in the workplace in my professional 9–to-5 (ish) lifetime.
Compare this with LLMs’ training on the bulk of written human natural and programming languages, which led to their capturing myriads of subtle little patterns we never even knew existed.
So, after all, I’m not sure human tinkering necessarily adds performance. Fully autonomous AI work, where it “auto-engineers itself,” may be real, after all. In that case a human doesn’t necessarily add to the picture; by contrast, they may be subtracting from the picture due to micromanagement. Funnily enough, autonomous AI work is also one of the few things the Responsible use of AI in evidence SynthEsis (RAISE) framework, in its version 4 at the time of writing, specifically advises against.
Echoing OpenAI, I’d say that what AI generates is raw research data. Interpreting it (or standardizing it via a harness) would therefore be the prospective, human-led research process. This process, therefore, could and should, in principle, follow an ad hoc protocol! Developing such protocols in good detail would require a lot of experience doing this work, for sure, so perhaps the first protocols will end up accumulating a lot of amendments – which is not a bad thing necessarily and of course not a valid rationale to abandon the development of an ad hoc protocol.
Perhaps it would also be beneficial to let the LLM develop the harness itself, although that I’m decreasingly less certain about, as this is becoming more subjective and more “human,” which we care deeply about (while AI doesn’t).
When criteria of work acceptance are rather objective, less micromanagement vs. more micromanagement can actually be rather beneficial. Quitting prompt engineering in favour of raw test-time might be just the thing that’s needed for superhuman performance. But once criteria become subjective… That’s where this becomes a problem. Oh have I whined a lot about agents’ “employee behaviour”! Or about taking accountability for its actions or other human gibberish like that. Yeah, this stuff is hard for AI. After all, a lot of what humans care about is a highly subjective story, which is about implicit experiences we humans don’t talk so much about but rather all share by virtue of shared life and sensual experiences – which is specifically what today’s LLMs lack by training.
That said, if we assume that developing a harness or protocol is still a human-led process, then it would still perhaps be beneficial to develop that harness together with AI, in the sense that it may not be 100% prescriptive and ad hoc, after all, but rather the AI that’s been working on the autoresearch should be allowed to give inputs to the harness development process. This provision can be baked into the protocol beforehand. And then together with AI, perhaps we could come up with a custom, tailored to the topic verification approach that proves, in acceptable human ways, why we should trust the results of this auto-SLR. It could be some custom regular expression rules, or even perhaps some smarter algorithmic rules AI could come up with.
And that’s where the OpenAI’s post of late, which presents some reportedly great scientific discoveries made by AI, doesn’t look so contradictory anymore. It looks contradictory because there, they write:
In our view, these [AI-generated] manuscripts are best understood as a form of raw research data. To convert this raw research data into scientific understanding, [further steps are necessary]
This makes a lot of sense to me, and I could not agree more! But then they immediately add:
We do not seek academic credit for the discoveries produced by our internal models.
While also:
Inviting a researcher to help steward a collaboration also does not imply that they or their working group made the original discovery.
Which makes me wonder: Then who’s supposed to get academic credit for these “raw research data”? Who claims academic credit for these reported discoveries, if OpenAI disclaims it and the invited scientists are “officially discouraged” from doing it?
This controversy is resolved if, in light of the above, we consider that AI itself claims it – except it doesn’t because AI could not care less about academic credit. But the contribution itself, regardless of what we call it, does seem to emanate from the test-time (i.e., inference-time) manifestation, or a “ghost” – the term Andrej Karpathy once used in an interview, of the LLM weights.
In that sense, acceptance criteria in evidence synthesis that runs on auto-SLR would need to change from industry-wide to per project, because by nature AI’s work looks sort of chaotic to humans and it’s emergent from the data, and so acceptance criteria would need to be post-filled retrospectively. I don’t think the fact that it’s retrospective should necessarily break causality or rigour for us. After all, an ad hoc protocol is also a human way of seeing things.
Can AI produce garbage while doing its autoresearch? Absolutely, it can. But then we won’t find out until we take these “raw research data” and sit down to develop a harness that makes sense of it.
« Previous | The End
Original text written by Pavel Zhelnov on October 28, 2026 (thanks Martin Ho for the convos!); reworked into this post using ChatGPT-6-High (so badly that the author rewrote almost everything). Last revised October 28, 2026.