What Makes ChatGPT a Finance Product? Joseph Kim on Data, Judgment, and Evidence

1. Guest Background

Joseph Kim leads financial services at OpenAI, with responsibility for product and engineering work on its ChatGPT finance offering. In this Latent Space interview, his perspective sits between model research and the practical demands of delivering software to financial institutions. The interviewer describes him as coming from the research side; Kim discusses roughly six months of work on the product and later emphasizes the substantial time he spends on evaluations. Those are the relevant experiences established here, rather than a complete account of his career.

Kim introduces ChatGPT for financial services as an enterprise product combining ChatGPT’s tools with firm-specific templates and financial data sources. He names Morgan Stanley and Evercore as design partners. Their involvement explains part of the product’s development context, but the interview does not establish deployment scale, customer economics, or measured productivity gains.

The conversation asks what must happen after a general model acquires useful financial capabilities. Its subjects include data integration, plugin development, review responsibilities, presentation quality, financial modeling, and the difficulty of evaluating professional work. Kim speaks both as a product builder reporting decisions and as a research-oriented leader forecasting future capabilities. Keeping those forms of evidence separate is essential to reading the episode.

2. What the Episode Covers

Kim begins by distinguishing financial intelligence from the product that delivers it. Training for financial reasoning and analysis, he says, goes into the main general model. The resulting capabilities are intended to be available to outside developers, startups, and financial firms integrating models directly. In his account, the development sequence starts with research into what models can do and where they are heading, followed by product features and interfaces that make those capabilities accessible.

An internal adoption story gives that distinction practical force. Kim describes steady Codex adoption among OpenAI software engineers, alongside a much sharper change among finance, legal, and sales staff: usage moved from near zero to roughly 90 percent. He associates that jump with the February release of the desktop application. The interview supplies neither a definition of adoption nor evidence isolating the application’s causal contribution. Its useful implication is narrower: having a capable model does not guarantee that the intended audience has an effective way to use it.

Data delivery provides the most concrete product decision. Bundling information such as quarterly earnings-call transcripts initially faced an obvious objection. Customers might already hold licenses, and MCP connectors might already provide access. Why should they pay again? Kim says the team tested connectors and found error rates above 40 percent for some prominent providers. It responded by bringing data into a delivery path whose experience it could control end to end. He also describes data as the feature customers most often praised.

That is an argument for taking responsibility for the delivered experience, but its evidentiary scope matters. The reported error rate comes from Kim’s account of particular tests. The episode does not disclose vendors, sample sizes, error definitions, or comparison conditions. Nor does customer praise establish a financial return. The story explains why bundling could add value despite existing access; it does not establish a failure rate for the MCP ecosystem.

Finally, Kim says most of the product was built using Codex or ChatGPT plugin architecture and primitives available to outside builders. That makes the interview relevant beyond OpenAI: significant product work concerns how capabilities, data, and workflows are assembled, rather than privileged access to model modification.

3. Core Views: Reasoning, Examples, and Limits

The central product argument is that extensive general capability can coexist with decisive industry-specific gaps. Kim estimates that another company could have done 95 percent of what his team built. He also describes Codex as supplying about 80 percent of the foundation for a successful product, while arguing that simply launching it as a finance product would fail. These percentages are his practical estimates, not measured production ratios. The underlying reasoning is that a relatively small set of unmet requirements can prevent adoption even when the underlying system already does a great deal. Data, interface changes, or an integration may supply that missing value. His prediction of a billion-dollar business built on a ChatGPT plugin expresses confidence in the opportunity; the interview offers no realized example establishing it.

The choice of initial tasks follows the same logic. Kim says models are good at executing concrete instructions, while many people he speaks with dislike the repetitive detail of building spreadsheets or positioning elements in slides. Junior analyst tasks therefore offer an intersection between present capability and work users want to delegate. That explains a product focus without establishing that the whole occupation is ready for automation. Producing an artifact is only part of a role that also includes judgment and responsibility.

Responsibility is explicit in Kim’s description of the released product. It assumes a human remains in the loop, reviewing work that might be partially completed to different degrees depending on the task. His examples of getting 30 or 80 percent of the way through a task illustrate variability, rather than a measured completion rate. For more autonomous agents, he proposes borrowing familiar arrangements for employee permissions and oversight. Banking already has a review chain from junior analysts through associates and senior staff. Such structures offer an understandable organizational model, but the interview does not demonstrate an implementation or show that existing controls can absorb greater autonomy without modification. Human supervision is a condition of the current design, not proof of its effectiveness.

Kim also separates factual correctness from professional quality. A factual error that changes a conclusion is a serious failure; a slide with awkward colors or sloppy formatting creates a different problem even if its content is correct. He believes factual reliability is improving and expects progress in judgment and taste. The episode provides no benchmark supporting the size of the reliability improvement or his confidence about severe errors reaching senior reviewers. Attention to presentation quality should therefore not be read as evidence that factual risk has disappeared.

His training example is more specific. Kim says the team used human comparisons between pairs of slides to train a reward model. People could choose the better-looking slide within seconds, whereas a model grader might reason extensively and still choose incorrectly. The example shows how human preferences can supply a learning signal where there is no single correct answer. It does not establish that collecting representative preferences is easy, or that one preference model will capture every firm’s standards.

The financial-modeling analogy gives taste a broader meaning than appearance. Code can compile and return the expected output while remaining poorly structured or difficult to maintain. Similarly, a spreadsheet can sum correctly and contain no reference errors without being a well-designed financial model. Kim describes experienced practitioners perturbing inputs and inspecting outputs to judge whether data flows through the model sensibly. Professional judgment here concerns behavior and structure, not merely visual polish.

Investment ideas introduce a further obstacle: delayed feedback. Kim anticipates movement toward more senior tasks in 2027, but notes that a concrete assessment of an investment thesis might require three years of waiting. Fuzzier objectives and slower feedback make success on junior execution tasks an insufficient demonstration of investment judgment. His proposed ingredients—research, strong data, and expert feedback—describe a research direction rather than an already validated capability.

Evaluations determine whether any claimed progress can be recognized. Kim stresses that an evaluation is a proxy for a dimension of ability, and that flawed grading can reward high scores without corresponding competence. He reports saturation in some financial evaluations, with remaining failures attributable to answer keys, underspecified prompts, or other defects, and says his team stopped tracking and reporting GDP eval. The interview does not include the underlying item audits. His broader warning is persuasive on its own terms: researchers may investigate disappointing scores while accepting rising scores as progress. The team’s proposed occupational evaluations aim to predict actual automation of work such as junior investment-banking tasks. That correspondence remains a development goal in this account.

4. Learning and Application

A practical application of the interview is to select tasks at the intersection of clear instructions, inspectable outputs, and genuine user willingness to delegate. Producing a first draft in an approved template or handling repetitive spreadsheet work fits Kim’s explanation of the initial product focus. Repetition alone is insufficient, however. If an output is difficult to review, faster production may simply relocate effort to correction. A pilot should examine what reviewers can actually use and how much revision remains, alongside generation speed.

Data decisions should be tested through representative work. Where a customer already has licenses and connectors, a team can compare the existing path with a bundled alternative on the same questions. Separating connection failures, missing data, and reasoning mistakes helps identify what additional control would improve. This is an application of the reasoning behind Kim’s example, not a reproduction of his experiment. His reported error rate should not become a default assumption for another project, and customer enthusiasm should not substitute for evidence that additional costs are justified.

Quality review can then distinguish deterministic checks from expert assessment. Numbers, formulas, references, and totals belong in the first layer. Presentation suitability and model structure require another. Kim’s input-perturbation example suggests reviewing how a financial model behaves when assumptions change, rather than inspecting only a static output. For presentation improvement, comparing pairs of artifacts can produce useful feedback, especially when reviewers explain why one is preferable. The boundary is that attractive outputs may still be wrong. Preference collection should preserve that distinction instead of rewarding polish indiscriminately.

Domain expertise should participate before final acceptance. Kim describes recruiting financial experts to work with researchers, engineers, and product managers because the team lacks the same intuitive understanding of good finance work that supported its coding efforts. Applied locally, that means involving practitioners in task definition, examples, and review criteria. It does not require treating every expert preference as an objective rule; disagreement can identify where standards depend on context.

Human review also needs named responsibilities. Teams can specify who checks a draft, approves a write operation, and authorizes the next stage of delivery. Moving from information retrieval to system modification warrants a fresh authorization and review design. Employee permissions and banking hierarchies provide useful starting points, but Kim’s analogy does not prove that existing controls transfer unchanged. Errors capable of changing a conclusion still deserve priority, regardless of confidence in general model improvement.

Evaluation work should connect scores to reviewed job outputs. Check whether prompts contain the information needed, whether answer keys are defensible, and whether scoring permits shortcuts that fail the underlying task. Apply that scrutiny when scores rise as well as when they stall. Saturation may justify more discriminating tasks, but disappointing results alone do not establish a broken test.

Finally, distinguish a roadmap from a delivered capability. Investment ideas with years-long feedback should remain material for scrutinized analysis, not gain credibility from the speed with which they were generated. Kim’s 2027 expectations and effort to extend enterprise access to Pro users are prospective statements. Adoption can expand with evidence of present performance; research aspirations should not inherit the same level of trust.

Source

• Original episode: Building ChatGPT for Finance — Joseph Kim, OpenAI's Head of Financial Services