The Machine as Multiplier, Not Substitute

1. From a telemedicine project to an institutional system

TutelAge belongs to the national AGENAS programme for telemedicine solutions serving “grandi anziani”: people over 80 with at least one chronic condition. The national initiative selected 58 projects from 146 applications, with PNRR funding of EUR 150 million and an aggregate target of approximately 60,000 beneficiaries over an 18-month experimental period.[1] The programme focuses on three entangled risks: cognitive deterioration, poor therapeutic adherence and social isolation. In the TutelAge working design, approximately 2,000 older citizens in Veneto are expected to be supported through regular contacts, cognitive stimulation, group activities and smartphone-mediated monitoring.[2]

These numbers describe scale, not meaning. The essential object being built is a socio-technical institution. It joins a patient-facing application, an operational Data Clinical Viewer, the OCTAVIA healthcare data lakehouse, external sources such as the Electronic Health Record, physiological and behavioural acquisition services, AI models, a control room and escalation pathways towards local health authorities. Each connection changes who knows what, when they know it and what they are expected to do next.

A useful abstraction is a partially observed dynamical system. Let the latent condition of patient $i$ at time $t$ be $x_{i,t}$, including clinical, cognitive and social components. The platform receives heterogeneous observations $y_{i,t}$ from questionnaires, interaction logs, medication information, smartphone sensors and professional assessments:

Here $a_{i,t}$ represents care actions, $u_{i,t}$ environmental and social influences, and $w_{i,t}$ and $v_{i,t}$ process and measurement noise. The platform never observes “the patient” directly; it observes a lossy, biased and strategically produced representation. That distinction is fundamental. A longitudinal profile is not a digital person. It is an epistemic instrument whose uncertainty must remain visible.

2. The ethical objective function

Conventional platform design tends to optimise measurable proxies: prediction accuracy, alert throughput, operator productivity, participation rate or reduction in service costs. Elder care requires a broader welfare function. Let social value $W$ be composed of health benefit $H$, autonomy $A$, relational continuity $R$, equity $E$ and operational efficiency $Q$, diminished by privacy intrusion $P$, clinical harm $C$, exclusion $X$ and workforce deskilling $D$:

The coefficients are normative weights, not parameters that can be inferred from clickstream data. Choosing them is a political and institutional act. A procurement document that rewards only throughput implicitly sets $\epsilon$ high and the remaining coefficients close to zero. A model that maximises sensitivity without accounting for alarm burden may increase $H$ for some patients while increasing $C$ through alert fatigue and reducing $R$ by converting care into continuous triage.

The safer formulation is constrained optimisation rather than a single unconstrained score:

Policy $\pi$ maps available information to recommendations, alerts and workflow assignments. Ethical safeguards become non-negotiable constraints: a minimum level of autonomy and equity, maximum acceptable privacy and clinical risk, and mandatory human review for consequential actions. This formalisation captures the central claim of the original post: machines may multiply professional capacity, but they must not acquire unbounded authority.

3. Human-in-the-loop is an organisational property

“Human-in-the-loop” is frequently used as a comforting label. It is meaningless if the human lacks time, information, competence or authority to challenge the system. A nominal reviewer who receives hundreds of poorly prioritised alerts is not exercising oversight; the platform is delegating liability while retaining practical control.

Effective oversight requires at least four properties: epistemic access, meaningful time, reversal power and accountability clarity. The operator must understand why an alert exists, have enough time to examine relevant evidence, be able to override or defer the recommendation, and know who owns the final decision. WHO guidance on AI for health places protection of human autonomy and retention of human control over healthcare decisions at the centre of ethical governance.[3] NIST similarly treats role differentiation and oversight in human–AI configurations as a governance requirement, not a user-interface feature.[4]

A simple model shows why. Let $q$ be the quality of an algorithmic recommendation, $s$ the professional's independent signal quality, and $\rho$ the correlation between their errors. If errors were independent, combination would create substantial value. When both rely on the same incomplete data, $\rho$ approaches one and the apparent second opinion becomes redundant:

The architecture should therefore preserve independent human evidence rather than forcing the professional to see only the algorithmically preselected narrative. Otherwise, automation bias is built into the information topology.

4. Observability without surveillance

Smartphone-based, sensor-less monitoring can reduce cost and adoption barriers, but it also expands the perimeter of observation into domestic life. Accelerometer traces, voice, facial video, participation patterns and therapeutic behaviour may reveal more than their declared purpose: frailty, mood, daily routines, social networks or household presence. The ethical question is not whether the data are useful. Almost all intimate data can become useful under a sufficiently broad model. The question is whether usefulness is necessary, proportionate and governable.

GDPR principles require purpose limitation, data minimisation, storage limitation, security and data protection by design and by default.[5] In mathematical terms, acquisition should solve a minimum-information problem. For target inference $Z$ and candidate data set $S$, collect the smallest subset $S^{\ast }$ that reaches an agreed performance floor $\tau$ while respecting an intrusion budget $\kappa$:

Mutual information $I(S;Z)$ expresses how informative the data are about the legitimate target; $I(S;\text{PrivateLife})$ represents collateral inference. This makes clear why “collect now, decide later” is not neutral. A lakehouse increases future analytic optionality, but unrestricted optionality conflicts with purpose limitation. OCTAVIA therefore needs purpose-tagged data products, retention policies by domain, lineage, access segmentation, auditable secondary-use approvals and revocation-aware processing.

Consent presents a further difficulty. An over-80 population is heterogeneous; cognitive vulnerability, dependency on caregivers and digital asymmetry can weaken the practical freedom to refuse. Consent cannot carry the entire ethical burden. Where processing relies on other lawful bases, transparency, proportionality, rights management and public accountability become even more important. Where consent is appropriate, it should be granular and renewable, not a one-time gateway to indefinite inference.

5. Game I — The principal–agent problem of public digital health

The public authority, the provider and frontline professionals do not share identical objectives. Consider a principal $P$ (the health authority) and an agent $V$ (the technology/service consortium). The principal values health impact $h(e)$, where $e$ is costly implementation effort. The agent receives payment $T$ but bears cost $c(e)$. True effort and system quality are only partially observable:

Here $r$ is residual risk. If payment is tied primarily to enrolment or delivered features, the agent's equilibrium effort shifts towards what is easy to evidence. Data quality, explainability, operator training and long-term maintainability may be underprovided because their benefits are delayed and difficult to attribute. This is a classic moral-hazard problem.

A better contract is multi-dimensional and delayed:

Yet every metric can be gamed. Goodhart's law becomes a strategic prediction: once a measure determines payment, actors optimise the measure rather than the underlying social objective. The solution is not an ever-larger KPI catalogue. It is a balanced mechanism combining outcome measures, process guarantees, independent audit, qualitative review and the right to inspect raw evidence. Indicators must be difficult to optimise without producing genuine value.

6. Game II — Therapeutic adherence as signalling, not mere compliance

Adherence is often modelled as a binary fact: the patient either followed the plan or did not. In reality, the platform observes signals—questionnaire answers, reminders acknowledged, refill events, self-reports or physiological patterns. Patients may strategically report adherence to avoid judgement, additional contacts or loss of autonomy. Professionals may under-record deviations to reduce escalation workload. The resulting data are endogenous.

Let patient type $\theta \in \{A,N\}$ denote adherent or non-adherent behaviour. The patient sends message $m\in \{\text{report adherence},\text{report difficulty}\}$; the platform and professional choose action $a\in \{\text{routine},\text{support},\text{escalation}\}$. A truthful separating equilibrium exists only if the benefit of tailored support exceeds the perceived stigma and intervention cost for the non-adherent patient:

This yields a direct design consequence: punitive alerting destroys information quality. If disclosure reliably produces loss of control or moral judgement, rational patients conceal difficulty. A supportive system should reduce $C_{\text{disclosure}}$, avoid language of blame and make early assistance less burdensome than late escalation. The data architecture cannot repair incentives created by the care model.

7. Game III — Alert thresholds, congestion and the tragedy of attention

Every model competes for a scarce shared resource: professional attention. Teams developing separate indicators may each prefer high sensitivity because the cost of missed cases is visible within their domain, while the cost of false positives is externalised to the control room. The aggregate result is an attention commons depleted by alerts.

For threshold $\tau$, let $FN(\tau )$ and $FP(\tau )$ be false negatives and false positives, with clinical cost $c_{FN}$, review cost $c_{FP}$ and congestion cost $g(N_{\text{alerts}})$:

The optimal threshold is not the one maximising AUROC, sensitivity or F1 score. It minimises total expected harm within actual operational capacity. Because $g$ is typically convex, the marginal cost of the thousandth alert is much greater than that of the tenth. Alert design must therefore include prioritisation, deduplication, temporal persistence, suppression rules, ownership, service-level objectives and explicit closure states.

A practical escalation score could be expressed as:

But missingness deserves special attention. A silent smartphone may indicate technical failure, non-use, hospitalisation, cognitive decline or a conscious refusal to participate. Treating absence of data as evidence of deterioration is both statistically naïve and ethically coercive. Missingness must be classified, investigated and represented with uncertainty.

8. Repeated games and the production of trust

TutelAge is an 18-month programme, not a single encounter. Patients and operators repeatedly decide whether to cooperate: disclose information, respond to calls, review alerts, explain decisions and respect preferences. Trust is therefore an equilibrium of a repeated game, not a communications slogan.

Let the one-period gain from opportunistic behaviour be $G$, the cooperative payoff be $R$, the punishment payoff after detection be $P$, and $\delta$ the discount factor reflecting the value placed on future relationships. Cooperation can be sustained when:

For patients, opportunism may mean withholding data; for providers, expanding data use beyond expectations; for operators, mechanically closing alerts; for institutions, using pilot data for performance surveillance. High turnover, opaque algorithms and frequent policy changes reduce $\delta$: actors have less reason to invest in the future relationship. Stable teams, transparent rules, reversible consent and visible responses to reported problems raise $\delta$ and make cooperation rational.

9. Fairness beyond equal error rates

Sensor-less approaches are not automatically inclusive. Camera-based remote photoplethysmography can perform differently across lighting conditions, devices and skin characteristics; voice analytics can encode language, dialect, education and socioeconomic status; interaction metrics can confuse low digital literacy with cognitive decline. Older adults who most need support may generate the least technically convenient data.

Group fairness metrics are useful diagnostics but cannot settle the normative question. Equalising false-negative rates across groups $g$ can be written as:

However, enforcing one parity criterion may worsen calibration or false-positive parity. The ethical choice depends on the intervention. If an alert triggers supportive contact, some false positives may be tolerable. If it triggers restriction, clinical labelling or coercive escalation, the harm structure changes. Fairness is therefore action-relative. OCTAVIA should record model version, subgroup performance, device context, language context, uncertainty and downstream action, enabling outcome audits rather than abstract benchmark claims.

10. Interoperability is semantic governance

Integrating the Electronic Health Record, care plans, questionnaires, application events and AI outputs is not merely an API problem. Identical labels can encode different clinical or administrative meanings; different sources can measure the same concept at incompatible timescales. A data lakehouse that achieves syntactic ingestion without semantic governance produces high-volume ambiguity.

Every clinically relevant data product should therefore declare provenance $p$, event time $t_e$, ingestion time $t_i$, author or device $a$, coding system $c$, units $u$, confidence $q$, lawful purpose $l$ and transformation lineage $\ell$:

This is particularly important for AI-derived indicators. A “cognitive risk score” without the task performed, language, model version, confidence interval, reference population and intended use is not interoperable knowledge; it is an orphan number. The platform must distinguish observations, derived features, predictions, recommendations, professional assessments and final decisions as separate semantic classes.

11. Regulatory classification and responsibility

The legal status of each component depends on intended purpose and actual use. Under the EU AI Act, certain systems are classified as high risk because of their relationship to regulated products or their potential impact on health, safety and fundamental rights.[6] If a model becomes part of a medical device or drives consequential clinical action, additional conformity, quality-management, risk-management, logging, data-governance and human-oversight obligations may apply. The correct strategy is not to label every component “decision support” and assume risk disappears. Intended purpose, user instructions, workflow integration and foreseeable misuse must align.

Responsibility must be allocated across the chain. The data platform owns lineage, access control, reliability and reproducibility; the model provider owns documented performance and limitations; the deploying organisation owns workflow design and monitoring; professionals retain authority for clinical interpretation; public authorities own the legitimacy of objectives and acceptable risk. “The algorithm decided” must remain an invalid sentence.

12. The value of information—and its limits

The strongest justification for OCTAVIA is its ability to correlate information that no individual can continuously synthesise. Decision theory provides a disciplined way to express this advantage. If a decision maker chooses action $a$ under uncertain state $\theta$, the expected value of information from signal $Y$ is:

Data collection is justified when EVI exceeds acquisition, privacy, interpretation and action costs. Crucially, information without action capacity may have zero or negative value. Detecting social withdrawal is not beneficial if no operator can make contact; it may merely create recorded knowledge of unmet need. A platform should therefore be evaluated on closed-loop value: detection, review, intervention, outcome and learning—not on prediction alone.

This formula also disciplines the temptation to add experimental modalities. Facial or vocal features may be scientifically promising, but their incremental value must be evaluated against consent complexity, demographic bias, storage risk and clinical actionability. Innovation is not the number of sensors or models integrated. It is the amount of justified, equitable and actionable uncertainty reduced.

13. A technical-ethical architecture for OCTAVIA

The foregoing analysis implies a concrete architecture. Ethical principles should be encoded in platform capabilities and operating procedures rather than left in policy documents.

• A canonical longitudinal record that separates source observations, transformations, predictions, recommendations and human decisions.

• Purpose-bound data products, attribute-based access control, encryption, auditable lineage, retention automation and revocation-aware downstream processing.

• A model registry containing intended use, prohibited use, training and validation populations, subgroup metrics, uncertainty, version history and rollback procedures.

• An alert orchestration layer with persistence rules, deduplication, severity, capacity-aware thresholds, ownership, escalation and closure evidence.

• Human-review interfaces that expose relevant raw context and counter-evidence, not only a risk score or explanation generated by the same model.

• Operational telemetry linking alerts to response times, interventions, outcomes, overrides, complaints and safety events.

• A governance board able to suspend a model, change thresholds, restrict secondary use and commission independent evaluation.

The minimum viable product should privilege reliable workflows over speculative inference: enrolment, assignment, scheduling, questionnaires, therapeutic plans, contact outcomes, participation, interoperable acquisition, longitudinal profiles, dashboards and controlled alerts. Voice, facial and multimodal experimentation should remain in a separately governed research track until protocols, datasets, consent, subgroup validity and clinical utility are demonstrated.

14. A compact mechanism-design agenda

Game theory is most useful here not as mathematical ornament, but as a warning that actors respond to rules. The platform and contract should be designed so that socially desirable behaviour is also individually rational.

Strategic risk	Likely equilibrium	Mechanism response
Metric gaming	Optimise enrolment, alerts or usage rather than welfare	Mixed metrics, audit sampling, delayed outcome payments
Patient concealment	Under-report difficulties to avoid stigma or escalation	Support-first responses; proportionate escalation; contestability
Alert externality	Each model maximises sensitivity and overloads staff	Shared attention budget; congestion-aware thresholds
Automation bias	Operators accept recommendations to save time or shift liability	Independent evidence, override rights, review sampling
Data expansion	Reuse grows because marginal analytic benefit appears cheap	Purpose binding, expiry, approval and deletion controls
Vendor lock-in	Switching costs weaken public bargaining power	Open standards, portable models/data products, exit testing

Conclusion: engineering the right asymmetry

Machines can observe continuously, compare thousands of trajectories and detect weak correlations. Humans can understand biography, ambiguity, refusal, dignity and the moral weight of intervention. Neither statement should be romanticised. Humans are inconsistent, overloaded and biased; models are scalable, brittle and indifferent to meaning. The goal is not symmetry between them, but a carefully governed asymmetry: machines should have broad computational reach and narrow institutional authority; professionals should have bounded workloads but final interpretive and decision rights; patients should remain more than the objects of observation, with rights to understand, refuse, correct and contest.

The social promise of TutelAge lies precisely here. A missed therapy, shrinking social network or subtle cognitive change can become visible before it becomes a crisis. But visibility is not care. Care begins when information reaches a competent person, is interpreted in context and produces a proportionate response. OCTAVIA can multiply that chain; it cannot morally replace it.

The success criterion should therefore be neither the number of integrated data sources nor the sophistication of the AI layer. It should be the creation of a stable cooperative equilibrium in which older people gain safety without losing autonomy, professionals gain information without losing judgement, public institutions gain efficiency without abandoning responsibility, and technology providers gain room to innovate without acquiring unaccountable power. That is a harder engineering problem than prediction. It is also the only one worth solving.