ComBodied Agents: a New Paradigm of Human-Centric Agentic AI

1. Introduction: The Unseen Failure Mode of Modern AI Agents

Current developments in Agentic AI overwhelmingly prioritize external task completion over human state understanding. Ding et al. illustrate this systemic gap through a common healthcare scenario: after an older adult misses a medication dose, a software agent might issue a repeat notification, while an embodied robotic agent might physically deliver the pill. However, neither system determines whether the missed dose stems from memory loss, confusion, adverse side effects, or a deliberate refusal to take the medication, nor do they determine what support is appropriate. This failure highlights a structural limitation across modern AI architectures: systems optimize for isolated, short-term task execution while remaining blind to the user’s underlying physiological, cognitive, and emotional condition.

To resolve this limitation, Ding et al. argue that Agentic AI must undergo a reorientation—shifting from maximizing immediate, external task completion toward supporting beneficial long-term trajectories of human state and preserving human agency over time. Central to this approach is the principle of Co-agency, wherein the agent dynamically adapts the division of labor with the human user rather than treating total automation or human substitution as the default state.

---

2. Unpacking the Action Substrates: Digital, Embodied, and ComBodied

To classify agentic systems, the authors introduce the concept of an action substrate—the primary class of target states that organizes an agent’s modeling, decision-making, intervention, and evaluation criteria. Based on these target states, the paper categorizes agents into three distinct substrates: Digital, Embodied, and ComBodied.

The term ComBodied represents an explicit portmanteau of Companion and Body (Definition 1 in Ding et al.). The authors provide a crucial clarifying caveat regarding paradigm scope: ComBodied does not imply a third-party conversational companion or emotional chatbot; rather, it denotes an agent that treats human bodily, cognitive, emotional, behavioral, and social states as its primary perceptual and action domain. Software tools, sensors, wearables, robots, and human services serve merely as action channels rather than end goals.

Ding et al. establish five jointly necessary properties that define ComBodied systems:

1. Human-centric state modeling: The person, rather than an external task alone, is the primary object of modeling and support.
2. Longitudinality: The agent reasons across repeated interactions, evolving life contexts, and extended time horizons.
3. Intervention: The agent provides structured support (e.g., reminding, coaching, reflecting, escalating) rather than merely generating passive responses.
4. Co-agency: The agent works collaboratively with the human, dynamically adjusting autonomy boundaries instead of replacing human decision-making.
5. Agency preservation: Interventions must preserve or strengthen user autonomy, control, dignity, relationships, and long-term capability.

Agent Category	Primary Target / Action Substrate	Core Evaluation Criteria & Limitations
Digital Agents	Digital states and artifacts (GUI navigation, code modification, API execution, workflow coordination).	• Criteria: Task correctness, code execution efficiency, workflow completion, and API policy compliance. • Limitations: Insufficient awareness of human cognitive load, fatigue, stress, values, or obligations; prone to automating tasks in ways that cause agency loss.
Embodied Agents	Physical or simulated physical states (robotic manipulation, physical navigation, autonomous driving).	• Criteria: Physical task success rate, kinematic safety, control reliability, and environmental generalization. • Limitations: Weak modeling of long-term human history, internal cognitive/emotional states, social ties, and personal vulnerabilities.
ComBodied Agents	Human state trajectories (cognition, emotion, physiology, behavior, social ties, goals) and agency preservation over time. Uses digital and physical tools merely as action channels.	• Criteria: Longitudinal human capability growth, retained user judgment, calibrated reliance, wellbeing, dignity, and agency preservation. • Limitations: Dependent on uncertainty-bounded sensing, purpose-scoped data governance, and user-correctable personal models.

---

3. The Closed-Loop Architecture of ComBodied Systems

To unify capabilities across personal assistants, health trackers, and adaptive tools, the authors formalize a closed-loop technical framework. The operational mechanics of this longitudinal loop are specified by the following formal equations governing state estimation, memory retrieval, human transitions, observation processes, and memory updates:

1. Latent Human State Estimation: The agent maintains an uncertainty-bearing posterior representation $Z_t$ of the unobservable human state $H_t$ given governed event-evidence history $D_{\le t}$, longitudinal memory $M_t$, and context $C_t$: $Z_t \sim q_\phi(\cdot \mid D_{\le t}, M_t, C_t)$
2. Memory Retrieval: Decision-relevant temporal context $R_t$ is extracted from memory based on current state estimate $Z_t$, context $C_t$, and user goals $G_t$: $R_t = \text{Read}(M_t; Z_t, C_t, G_t)$
3. Human-State Transition Process: The true human state evolves based on previous state $H_t$, agent intervention $a^{\text{agent}}_t$, explicit user actions $a^{\text{user}}_t$, and exogenous environmental/social influences $\Xi_t$: $H_{t+1} \sim T_H(\cdot \mid H_t, a^{\text{agent}}_t, a^{\text{user}}_t, \Xi_t)$
4. Observation Process: New multimodal observations $O_{t+1}$ are generated from the updated human state and context: $O_{t+1} \sim \Omega(\cdot \mid H_{t+1}, C_{t+1})$
5. Memory Update: Longitudinal memory is updated with new observations, executed agent actions, observed outcomes $Y_{t+1}$, and explicit or implicit feedback $F_{t+1}$: $M_{t+1} = \text{Update}(M_t, O_{t+1}, a^{\text{agent}}_t, Y_{t+1}, F_{t+1})$

The architecture organizes these mechanics into four functional pillars:

Pillar 1: Event-Based Multimodal Perception

Rather than ingesting unconstrained raw data streams, the perceptual subsystem converts multi-source personal evidence into governed, uncertainty-bearing event-evidence records. The framework enforces a strict 5-stage abstraction chain to bridge raw sensing to authorized policy execution:

$\text{Observation} \longrightarrow \text{Event} \longrightarrow \text{Inferred State} \longrightarrow \text{Predicted Trajectory} \longrightarrow \text{Authorized Intervention}$

Concrete Abstraction Example: An optical heart-rate sensor measures a spike to 135 bpm (Observation). Coupled with motion data, the system identifies elevated physical exertion followed by an abnormally slow heart-rate recovery rate post-exercise (Event). The system estimates a posterior of acute physical fatigue (Inferred State). The internal world model projects a trajectory of severe muscular exhaustion and prolonged recovery if strenuous activity continues (Predicted Trajectory). The policy selects a non-coercive rest recommendation (Authorized Intervention).

Multimodal perception operates across five distinct acquisition modes: user-reported, device-mediated, on-body/contact, ambient/contactless, and institutional. These signals follow six temporal sampling patterns: continuous, periodic, episodic, event-triggered, context-triggered, and query-on-demand. To preserve uncertainty awareness, event-evidence records must retain explicit provenance metadata, including sensor placement, signal-to-noise indicators, clock drift, device calibration, processing pipeline versioning, and alternative interpretations.

Pillar 2: Longitudinal Memory

Longitudinal memory acts as the temporal evidence layer tracking human state trajectories across extended horizons. The paper details seven core memory components, structured with explicit schema and governance requirements:

Memory Component	Schema Content & Function	Required Metadata & Governance Constraints
Episodic Memory	Concrete events, past interactions, and specific user experiences.	Timestamp, sensor source, modality, confidence score, raw data deletion status.
Semantic Person Memory	Relatively stable user facts, core values, preferences, and explicit boundaries.	User-verification status, source provenance, last review date, user edit history.
Trajectory Memory	Longitudinal changes in physical health, emotion, cognition, behavior, and routines.	Baseline reference values, temporal trend vectors, variance thresholds, drift markers.
Goal & Commitment Memory	User goals, active plans, personal obligations, and operational promises.	Goal origin (user vs. inferred), target completion horizon, priority score, active status.
Relationship Memory	Social, caregiver, clinical, family, or work ties and boundary permissions.	Authorized role scopes, shared memory visibility flags, consent boundaries per participant.
Intervention-Response Memory	Record linking past agent support interventions to observed user responses and outcomes.	Action type, user response (accept/reject/ignore), observed benefit/harm ($Y_{t+1}$), causal confidence.
User-Control Memory	Explicit user restrictions, do-not-remember rules, corrections, and deletion logs.	Non-modifiable override authority, deletion confirmation logs, scope restrictions.

All memory components must maintain strict provenance, remaining inspectable, correctable, and deletable by the user to prevent the accumulation of unverified system inferences.

Pillar 3: Personal World Model (PWM)

A purpose-bounded, individual-specific dynamics predictor that processes governed event evidence ($D_{\le t}$) and context ($C_t$) to estimate calibrated distributions over future personal states, observable events, and outcomes under candidate scenarios.

Pillar 4: Admissible Intervention Policy

A decision engine that constrains candidate support actions against explicit safety boundaries, uncertainty thresholds, consent models, and reversibility requirements before delivery.

Transition to Section 4: While Pillar 4 governs decision boundaries, the predictive accuracy of the entire closed loop depends fundamentally on Pillar 3. Section 4 examines the mathematical formalization, structural contracts, and causal limitations of Personal World Models.

---

4. Personal World Models (PWMs): Predicting Human Trajectories Without Digital Twins

PWM Contract vs. Human Digital Twins

Ding et al. draw a technical distinction between Personal World Models (PWMs) and Human Digital Twins (HDTs). A full Human Digital Twin aims to construct an exhaustive, high-fidelity digital replica of a human through continuous multi-scale data synchronization. The authors emphasize that an exhaustive HDT remains practically unfeasible due to the extreme complexity of human physiology, cognition, and social contexts, along with privacy, computation, and governance limits.

In contrast, a PWM does not attempt to construct a complete digital twin. Instead, a PWM is defined as a purpose-bounded, individual-specific event-dynamics model. It maintains uncertainty-aware, user-correctable representations restricted strictly to an agreed-upon support context (such as medication adherence or skill acquisition) and focuses on predicting intervention-conditioned state trajectories rather than achieving total representational fidelity.

Action-Conditioned Trajectory Prediction

Formally, let $D_{\le t}$ denote the history of governed event-evidence records up to time $t$, $Z_t$ denote the latent human state representation, $C_t$ denote current spatiotemporal context, and $G_t$ capture user goals. Given a candidate scenario $s_{t:t+\Delta} = (a^{\text{user}}_{t:t+\Delta}, a^{\text{agent}}_{t:t+\Delta}, \Xi_{t:t+\Delta})$ specifying candidate user decisions, agent interventions, and environmental changes $\Xi$, the PWM estimates a joint distribution over future latent states, observable personal events $E$, and outcomes $Y$:

$p_\theta(Z_{t+1:t+\Delta}, E_{t+1:t+\Delta}, Y_{t+1:t+\Delta} \mid D_{\le t}, Z_t, C_t, G_t, s_{t:t+\Delta})$

The outcome vector $Y$ tracks metrics such as goal progress, health status, capability growth, calibrated reliance, and agency preservation. By simulating candidate scenario rollouts, the PWM allows the system to compare outcomes across alternative actions (e.g., comparing an immediate reminder delivery against delaying action or requesting clarification).

Causal Reasoning Limits

The authors explicitly distinguish predictive scenario rollouts from strict causal interventional distributions, formally expressed using Pearl’s $do$-calculus:

$p_\theta(Y_{t+1:t+\Delta} \mid do(a^{\text{agent}}_t = a), Z_t, R_t, C_t)$

Ding et al. emphasize that observed human behavior is deeply confounded by unobserved user motivation, hidden context, fluctuating health status, prior interactions, and selective user engagement. Simply inserting a $do(\cdot)$ operator into a model architecture does not resolve underlying confounding.

To achieve valid causal identification, the system requires defensible Structural Causal Models (SCMs) and strict adherence to formal causal assumptions:

• Positivity: Ensuring all candidate interventions have a non-zero probability of occurrence across all context states.
• Consistency: Ensuring that the observed outcome under a given intervention matches the potential outcome for that action.
• Time-Varying Confounding Control: Correcting for intermediate variables that act simultaneously as consequences of past actions and confounders of future actions.

To validate causal claims without introducing harm, the authors recommend empirical estimation techniques such as N-of-1 trial designs and micro-randomized trials (MRTs). Crucially, the paper establishes a firm safety constraint for red-teaming and system design: high-risk agents must not execute unconstrained exploratory actions solely to gather data for causal model learning.

---

5. Agency Preservation and Admissible Intervention Policies

Multi-Objective Pareto Policy Optimization

To prevent safety failures resulting from scalar reward exploitation (such as maximizing engagement at the expense of user autonomy), policy execution in ComBodied systems is framed as multi-objective Pareto optimization over an admissible action space.

Formally, the optimal intervention action $a^{\text{agent}, *}_t$ is selected according to Equation 8 from the paper:

$a^{\text{agent}, *}_t \in \operatorname{ParetoArgmax}_{a \in \mathcal{A}^{\text{adm}}_t} \mathbb{E}_{p_\theta} [\mathbf{U}(Y_{t+1:t+\Delta}, G_t)]$

Optimization occurs over a multi-objective vector $\mathbf{U}$, which explicitly balances capability growth, autonomy retention, physical/psychological safety, and relationship quality across non-dominated actions within the admissible space $\mathcal{A}^{\text{adm}}_t$. The policy structure strictly forbids scalar reward functions that trade off user safety or consent to maximize system usage or engagement.

Safety and Governance Principles

The operational execution of intervention policies relies on three core safety principles:

• The Admissible Action Space ($\mathcal{A}^{\text{adm}}_t$): The policy restricts action selection to a constrained set of admissible interventions. An action $a^{\text{agent}}$ is admissible at time $t$ only if it satisfies explicit user consent, operates within authorized domain boundaries, remains below maximum uncertainty thresholds, maintains action reversibility, and adheres to mandatory escalation paths.

• Intervention Semantics: The paper defines an action spectrum varying in target, intensity, required authority, and reversibility:

  • Inform: Explaining, summarizing, or interpreting data for cognitive clarity.
  • Remind: Directing user attention to upcoming tasks or schedules.
  • Recommend: Proposing non-binding options, plans, or next steps.
  • Coach: Providing structured feedback to build capability and habits over time.
  • Nudge: Steering behavioral tendencies gently toward safer choices.
  • Reflect: Prompting the user to review emotional patterns, choices, and trade-offs.
  • Coordinate: Managing communication with authorized social, care, or work networks.
  • Protect: Intervening against immediate hazards (e.g., detecting financial scams).
  • Escalate: Transferring control to external human experts, clinicians, or emergency services.
  • Execute: Performing direct, user-delegated software or digital actions.

  Crucially, non-intervention (remaining silent), requesting clarification, or referring to human care are valid, often optimal choices within the admissible space.

• Agency Preservation Metrics: The authors argue that policy optimization must explicitly avoid optimizing for maximum user engagement, usage time, or total automation. Instead, evaluation formulations must measure long-term user capability growth, retained independent judgment, calibrated reliance, dignity, and preserved autonomy.

---

6. Key Takeaways and Future Directions

Ding et al. frame ComBodied systems as a structural evolution required for safe, human-centric AI deployment. The primary takeaways for researchers, red-teamers, and AI practitioners include:

1. Shift in Optimization Target: System evaluation must move beyond immediate, task-oriented benchmarks. True safety and efficacy in human-centric agents require measuring longitudinal human capability gain, retained user judgment, dynamic co-agency, and preserved autonomy alongside standard execution metrics.
2. Purpose-Bounded Data Governance: Mitigating surveillance risks requires event-based multimodal perception. Systems must prioritize edge-native local processing, explicit record provenance, uncertainty tracking, and user-driven inspection, correction, and deletion of personal memory stores.
3. Proportional, Uncertainty-Aware Intervention: System safety depends on intervention policies bounded by explicit admissibility constraints. Agents must possess calibrated uncertainty models that inform when to act, when to reduce support intensity, when to seek clarification, and when to remain silent or escalate to human caregivers.

Read the full paper on arXiv · PDF