From Episodic Memory to Clinical Expertise: Joint Optimization in Self-Learning Diagnostic Agents

The Gap Between Medical Knowledge and Clinical Experience

Medical education produces a familiar paradox. Students emerge from training with vast declarative knowledge, yet true diagnostic competence develops only through years of practice, through exposure to edge cases, rare presentations, and the subtle pattern recognition that distinguishes textbook descriptions from living pathology. This distinction between knowing and recognizing, between semantic memory and experiential learning, has remained stubbornly difficult to replicate in artificial intelligence systems.

In their recent preprint, "Joint Optimization of Reasoning and Dual-Memory for Self-Learning Diagnostic Agent," Bingxuan Li, Simo Du, and Yue Guo from the University of Illinois Urbana-Champaign and Jacobi Medical Center introduce SEA, a diagnostic agent architecture that explicitly models this developmental trajectory. Rather than treating each clinical encounter as an isolated prediction task, SEA maintains a dynamic memory system that consolidates experience into reusable abstractions, mimicking the cognitive mechanisms that allow human clinicians to improve over time.

The Architecture of Clinical Memory

The cognitive science literature has long distinguished between episodic memory, concrete records of specific experiences, and semantic memory, generalized knowledge abstracted from those experiences. SEA operationalizes this distinction through a dual-memory module comprising a bounded short-term memory and an evolving long-term memory.

The short-term component functions as an append-only buffer with fixed capacity K, storing raw case trajectories including patient presentations, reasoning chains, and diagnostic outcomes. When this buffer reaches capacity, the system faces a critical decision: which experiences to retain, which to discard, and critically, how to compress discarded cases into abstract rules that migrate to long-term memory. This compression step is where SEA diverges fundamentally from standard retrieval-augmented generation systems. Rather than storing and retrieving raw cases, the agent distills diagnostic heuristics, effectively learning how to generalize from specific encounters.

This design addresses a specific pathology of medical AI: the information density and noise characteristics of clinical data make verbatim storage computationally expensive and diagnostically unreliable. Rare diseases present atypically, and the signal that distinguishes a zebra from a horse often lies in subtle combinations of findings that raw case retrieval might miss. By forcing the system to abstract patterns into compact rules, SEA encourages the formation of mental representations that Schmidt, Dornan, and others have identified as central to clinical expertise development.

Joint Optimization: The Technical Core

The architectural insight alone, however, solves only half the problem. The more formidable challenge lies in optimization. Standard supervised fine-tuning or outcome-only reinforcement learning treats memory management as a fixed heuristic or ignores it entirely. SEA instead proposes a reinforcement learning framework that jointly optimizes both diagnostic reasoning accuracy and memory management decisions.

The technical implementation involves structured rollouts that generate traceable reasoning trajectories leading to final diagnoses. These rollouts serve dual purposes: they produce the diagnostic prediction, and they generate the memory update decisions including which cases to evict and how to formulate consolidation rules. The authors introduce adaptive rewards that combine diagnostic correctness with measures of effective memory management, creating a credit assignment problem that spans multiple timescales.

This joint optimization is computationally demanding but conceptually necessary. If the system optimizes only for immediate diagnostic accuracy, it might overfit to common presentations and fail to preserve rare but critical cases in memory. Conversely, if it optimizes only for memory efficiency, it might overcompress and lose diagnostically decisive signals. The adaptive reward structure explicitly balances these competing objectives, encouraging the agent to develop a memory policy that supports continual improvement rather than static competence.

Empirical Validation and the Long Horizon

The authors evaluate SEA across two complementary benchmarks that test different aspects of clinical capability. On the MedCaseReasoning dataset, which measures static diagnostic competence without cross-case adaptation, SEA achieves 92.46% accuracy, outperforming the strongest baseline by +19.6%. This result demonstrates that even without the benefit of accumulated experience, the joint optimization of reasoning and memory architecture yields superior diagnostic reasoning compared to outcome-only supervision.

More revealing are the results on the ER-Reason dataset, which simulates streaming deployment with feedback over long horizons. Here, SEA achieves a final accuracy of 0.7214 with an improvement metric of +0.35 (∆Acc@100), while baseline methods show limited gains (around +0.03) or even degradation (negative accuracy changes at early stages). This divergence is significant. It suggests that standard training paradigms, even those using reinforcement learning, fail to achieve continual learning without explicit mechanisms for experience accumulation and reuse.

Perhaps most compelling is the expert evaluation of the consolidated rules themselves. Rather than merely assessing diagnostic accuracy, the authors had clinicians rate the extracted rules on clinical correctness, usefulness, and trustworthiness, yielding scores of 3.865, 3.946, and 4.216 respectively on a 5-point Likert scale. This validation matters because it addresses a critical deployment concern: if the agent's internal abstractions are clinically nonsensical, the system becomes an opaque liability, regardless of its accuracy metrics. The strong expert ratings suggest that SEA's compression mechanism preserves medically meaningful patterns rather than spurious correlations.

Limitations and Broader Implications

Despite these promising results, several challenges warrant consideration. The medical domain imposes unique constraints on continual learning systems that this work touches upon but does not fully resolve. First, the liability and safety implications of autonomously evolving rule sets remain unclear. If a clinician acts on a recommendation derived from a consolidated rule that originated from a misdiagnosed case three months prior, assigning responsibility becomes legally and ethically complex.

Second, the evaluation assumes a relatively stable distribution of diseases. In real clinical environments, distributional shift occurs through seasonal variations, emerging pathogens, and changing patient demographics. The current framework does not explicitly account for concept drift or catastrophic forgetting in the semantic memory component, though the eviction mechanism provides some natural buffer against obsolete patterns.

Third, the abstraction bottleneck poses inherent limitations. While compressing cases into rules improves scalability, it also creates an information bottleneck that might discard contextually critical details. The expert evaluation provides reassurance, but larger scale deployment would require continuous monitoring of rule quality.

Beyond medicine, the dual-memory architecture suggests broader lessons for agentic AI. Legal reasoning, engineering diagnostics, and scientific discovery all involve domains where expertise develops through the accumulation of case-specific patterns that resist formalization as static knowledge bases. The principle that joint optimization of reasoning and memory management outperforms independent inference applies across these domains. The technical contribution of structured rollouts with adaptive rewards for memory operations offers a template for continual learning in any information-dense, high-stakes environment.

Open Questions and Future Directions

Several research directions emerge from this work. Most pressing is the question of multimodal integration. Clinical reasoning relies not only on text but on imaging, time-series data, and laboratory values. How should a dual-memory system compress and consolidate multimodal experiences into abstract rules? The current formulation handles primarily textual reasoning chains, but medical practice is inherently multimodal.

Another open question concerns human-agent collaboration. The current SEA framework operates as a autonomous learning system, but clinical deployment likely requires a human-in-the-loop mechanism for memory curation. Clinicians might need to validate consolidated rules before they enter long-term memory, creating a mixed-initiative system that combines the scale of automated learning with the safety of expert oversight.

Finally, the field requires better benchmarks for continual learning in specialized domains. The ER-Reason dataset represents progress, but longer horizons, rare disease emphasis, and explicit tests for catastrophic forgetting remain necessary to validate these systems for real deployment.

SEA demonstrates that diagnostic AI need not remain static. By treating clinical reasoning as a continual learning problem and memory management as a learnable policy rather than a fixed heuristic, the system captures something essential about how expertise actually develops. The path from medical student to master clinician is paved not with memorized facts but with compressed, reusable wisdom. Building machines that walk this same path requires architectures that respect the distinction between knowing and learning.