Insights · Guide
What agentic AI in private markets cannot do, and the cost of finding out
Each fund right now is being pitched an agentic AI investing tool. Most demo well. Most will not survive eighteen months in a real book. Here is why, with the numbers.
Agentic AI investing tools land 85-95% of answers correctly on a curated demo deal. On a real customer's first deployment, with documents the borrower actually sent, success rates drop to 30-50%, and by month eighteen fewer than 20% of deployments remain in their original shape; most get downgraded to a co-pilot the analyst double-checks. The failure mode is structural: conservation arithmetic, subject-identity mix-ups and lane-shape mismatches that prompt engineering alone does not fix.
The success rate looks fine in demo. It collapses in production.
On a curated demo deal, agentic AI investing tools land 85-95% of the answers correctly. The deal team rehearses the prompts, the documents are clean, the questions are scoped.
On a real customer's first deployment, a deal the team did not pick, with the documents the borrower actually sent, success rates drop to 30-50%. The variance is largest where lanes mix: a deal that is half-corporate, half-real-asset, with multiple legal entities and a multi-quarter reconciliation, breaks the agent in ways the demo did not test. These variance patterns are consistent across lenders and funds that have run a second deployment outside the vetted demo set.
By month eighteen at an institutional-grade customer, an LP, a regulated lender, anything that touches an audit, fewer than 20% of these deployments will still be in their original shape. Most get quietly downgraded to a co-pilot for the analyst: the human checks everything, and the productivity claim disappears.
Time to ship a demo: months. Time to be right at scale: years.
A reasonably staffed team can ship an agentic loop with retrieval over the data room in 4-8 months. The demo looks 80% done. It is closer to 15% done; the remaining 85% is the deterministic infrastructure underneath, the layer that catches the miscategorizations the agent makes silently.
4-8 months
Ship an agentic loop with retrieval over the data room. The demo looks mostly done.
18-30 months
Pass a customer's first regression test: a known-bad deal where the right answer must surface a specific blocker.
Ongoing
Discover conservation-arithmetic bugs, subject-identity bugs and lane-shape bugs, each requiring part of the substrate the LLM-first build skipped.
5-8 years
Ship a system that holds up at scale, across deal lanes and audit cycles, for a well-funded team that knows what it is building.
The token economics are quietly large
A single agentic AI pass over a real deal, reading the documents, extracting KPIs, running covenants, drafting a memo, raising flags, runs 200K-500K input tokens and 100K-300K output tokens. At current API rates, that is $5-15 per pass. Real deals get re-analyzed: quarterly cycles, evidence updates and continuous monitoring typically run 10-60 passes per deal per year, which is $250-1,500 per deal annually before retries.
| Book size | Example | Annual inference cost |
|---|---|---|
| 30 portfolio companies | A mid-size PE fund | $7K-45K per year |
| 200 borrowers in monitoring | A direct lender | $50K-300K per year |
| 1,000 deals | A multi-strategy alts manager | $250K-1.5M per year |
Five things prompt engineering will not fix
- Conservation arithmetic. A portfolio DSCR is the sum of NOI over the sum of debt service, not the average of per-property DSCRs. An LLM not wrapped by a deterministic policy layer will at some point compute the wrong one.
- Subject-identity guarantees. Multi-document deal rooms have multiple subjects: properties, entities, sponsors, customers. Without a deterministic guard layer, an agent will at some point match a document to the wrong subject.
- Lane-shape correctness. A corporate credit deal needs corporate stress packs and covenant axes; a real-estate asset needs DSCR and rent-roll analytics. An LLM without lane-aware projection will silently apply the wrong analysis to the wrong lane.
- Cross-source reconciliation that stays transparent. The management pack, the audited financials and the QoE normalization can disagree on the same figure. A useful platform shows the candidates with reasoning, picks one explicitly and logs the conflict, instead of picking one silently.
- Saying 'I don't know.' Confabulation is the LLM's default mode. A useful platform for IC work has the opposite default: it states what is missing and what would sharpen the call, instead of rendering a memo against insufficient data.
This is the shape of the problem Capital Refinery is built around: when two sources disagree on a figure, the disagreement becomes a review item with both values as choices, and the resolution, who chose what and why, is written to an audit log, not lost in a chat transcript.
How to evaluate the difference, in fifteen minutes
- 01Ask for a downloadable deal pack where each KPI cites its source document and page. A screenshot of a UI claiming traceability is not the same as a file you can inspect on your own.
- 02Ask to run the same deal twice. If the platform's own output is not consistent on identical evidence, no citation can mean what the traceability claim says.
- 03Ask the platform to recommend a deal where it should not. A platform built to say what evidence is missing will refuse to render an IC-grade memo against insufficient data. An LLM-first product will produce one regardless.
If you are evaluating an agentic AI investing tool, run these three tests against the alternative before you open the first PDF.
None of these three tests requires trusting a vendor's marketing page. A deal pack you can download and inspect on your own laptop, a repeat run you can diff yourself, and a refusal you can watch happen in front of you are all things a buyer can verify in the room, during the pitch, without waiting for a production deployment eighteen months out to find out the hard way.
Don't trust the dashboard. Trace the numbers yourself.
The fastest way to feel the gap between a demo and a production deployment is to bring a real deal and check whether the numbers trace. Capital Refinery traces figures to their source documents and cites them where a locator resolves, so the trace is something you can follow, not something you take on faith.
Questions
Do agentic AI tools work for private equity investing?
They work well in a demo: 85-95% of answers land correctly on a curated deal. In production, on a real customer's first deployment, success rates drop to 30-50%, and the gap is structural, not a prompt problem.
Why do agentic AI investing tools fail in production?
Mainly structural reasons: conservation arithmetic, subject-identity mix-ups, and lane-shape mismatches that a probabilistic model will get wrong at some point unless a deterministic layer catches it first.
How much does an agentic AI pass over a deal cost?
At current API rates, a single pass runs about $5-15 in raw inference. Deals get re-analyzed 10-60 times a year, which is roughly $250-1,500 per deal annually before retries.
What three questions should I ask an agentic AI vendor?
Ask for a downloadable deal pack with source-cited KPIs, ask to run the same deal twice and compare the output, and ask the platform to recommend a deal it should decline.
Does Capital Refinery use agentic AI to make investment decisions?
No. Numbers are read by deterministic parsers; where candidates conflict, a local model chooses among the named candidates and never invents a value, and the deal team resolves disagreements on a review surface. Some written outputs, such as memo polish and narrative summaries, are drafted by a hosted model from Anthropic, which receives deal details and summary figures, not the decision.