The security review is where a hospital AI deal either survives or quietly dies, and most of what gets asked in that room is aimed at the wrong risk. A CISO spends forty minutes on where the servers sit and four minutes on who at the vendor can read a patient record at two in the morning. The first question feels weighty. The second is the one that actually costs you.
This piece is about that gap. What large hospital groups say they are afraid of, what they should be afraid of, and how to tell the difference before you sign.
Start with what a breach actually costs
IBM's Cost of a Data Breach Report 2025 puts the average healthcare breach at 7.42 million dollars, the most expensive of any industry for the fifteenth consecutive year. The same report finds healthcare takes 279 days on average to detect and contain an incident. Those are global figures weighted heavily toward United States regulatory penalties, so the rupee number for an Indian hospital group is smaller, but the shape holds: healthcare data is expensive to lose and slow to notice missing.
India already has the worst case on record. The ICMR breach disclosed in late 2023 exposed data tied to roughly 815 million people, including Aadhaar and passport detail. Nothing in that incident involved AI. It is worth remembering that before the next AI security review, because the instinct in that room is to treat the AI vendor as the novel threat when the hospital's existing attack surface is almost always larger.
On-premise is a liability decision, not a security one
The most common demand in a hospital AI procurement is full on-premise deployment. No data leaves the building. It sounds like the safe answer, and for a specific class of buyer it can be. For most it is theatre.
On-premise reduces risk only if the hospital can run it properly: network segmentation, patch discipline, key management, physical access control, monitored logs. Walk into the average Indian hospital data centre and you find a flat network, servers three kernel versions behind, and a shared admin password that four vendors already know. Moving an AI workload into that environment does not make it safer. It moves the blast radius closer to the systems that matter and hands the operational burden to a team that is already short-staffed.
What on-premise genuinely does is move liability. When the data never leaves your perimeter, the vendor's exposure shrinks and yours grows. That is a legitimate reason to choose it, and it should be named as such in the contract. A well-run cloud deployment with proper tenant isolation, encryption in transit and at rest, and a vendor who cannot silently exfiltrate your data is, for most hospital groups, the lower-risk option. The honest conversation is about who carries the liability, not about which model is inherently secure.
PHI residency: what DPDP actually requires, and what it does not
A large share of the residency anxiety in hospital security reviews rests on a belief that Indian health data is legally required to stay on Indian soil. As of mid-2026 that is not what the law says.
The Digital Personal Data Protection Rules 2025 were notified in November 2025, with full compliance required by May 2027. The Act uses a negative-list model for cross-border transfer: data may move to any country by default unless the Central Government specifically restricts that destination. There is no general health-data localisation mandate in the DPDP framework today. What does exist is the Significant Data Fiduciary designation, where an organisation processing large volumes of sensitive data can be held to stricter conditions, including possible restrictions on offshore transfer of certain categories. Sector-specific rules and future government notifications can tighten this further.
So the practitioner position is this. You are probably not legally barred from cloud inference outside India, but you may become subject to localisation obligations if you are designated an SDF, and the direction of regulatory travel is toward more restriction. Design for residency you can prove and relocate. Know exactly which country your model inference runs in, get it in writing, and make sure you can move it inside India within a defined window if the rules change. Residency you cannot document is the real gap, not residency in the wrong postcode.
Vendor access is where the real exposure lives
Here is the question that should take forty minutes. When your AI vendor's engineers need to debug a production issue at 2 am, what can they see, under whose credentials, and where is that access recorded?
In most AI deployments this is governed by nothing more than trust and a shared VPN. A support engineer assumes a role that can read raw patient records, does the fix, and logs off. No per-record justification, no time-boxed access, no immutable log the hospital can read without asking the vendor for it. That standing access is a far larger exposure than the choice of hosting model, and it is almost never examined with the same intensity.
What good looks like is specific. Vendor access should be just-in-time and approved per session, not standing. It should run through the hospital's own identity provider so access can be revoked in seconds without a vendor's cooperation. Production PHI should be masked or tokenised for anyone doing routine support, with break-glass access to raw data logged separately and reviewed. And the credentials that the AI system itself uses to reach the HIS or the EMR should be scoped to exactly the fields it needs, not a blanket read grant that every future feature inherits.
There is a second access question specific to language-model systems, and it is the one most security reviews miss entirely: what happens to a patient record after it goes into a prompt. If the model provider retains inputs or trains on them, every consultation you send becomes training data on someone else's servers. The control here is contractual before it is technical. A usable enterprise model agreement commits, in writing, to not training on customer inputs and to a bounded retention window. That commitment is what lets PHI touch a hosted model at all. Without it, no amount of on-premise networking around the edges matters, because the sensitive payload is the prompt itself.
The audit trail most hospitals think they have
Ask a hospital for a complete audit trail of who accessed a given patient's record through the AI system over the last ninety days, and watch what happens. Most cannot produce it at the row level. They have application logs that show a service account made a query, not which human or agent triggered it, on whose authority, and what came back.
For a regulated environment that is the gap that turns a manageable incident into an unmanageable one. With a 279-day average detection window, the audit trail is often the only evidence you will have of what an attacker or a misconfigured agent actually touched. It needs to be immutable, retained on the hospital side and not only the vendor side, and query-linked so that every model call carries the identity that authorised it. An AI system that cannot tell you, months later, exactly which records a given agent read is not production-grade in a hospital, whatever the demo looked like.
What is theatre, plainly
Some of the security ritual in hospital AI procurement protects no one. An ISO 27001 certificate proves a vendor documented a management system, not that your data is safe. A blanket "we do not store your data" claim is meaningless until you see where the prompts and the logs actually go. Penetration test summaries with the findings redacted tell you nothing. And full air-gapping demanded on top of a hospital network that has no internal segmentation is effort spent on the strong wall of a house with open windows.
The distinction is not hard once you name it. Real controls change what an attacker or an insider can actually do: least-privilege field access, just-in-time vendor sessions, tenant isolation, contractual no-training terms, immutable hospital-side logs. Theatre changes how the deal feels in the committee room. A CISO worth the title spends the budget on the first list and refuses to be charged for the second.
The wall worth building
The data security wall large hospital groups need is real, and most of them are building it in the wrong place. The perimeter question, on-premise or cloud, is mostly a liability decision dressed as a security one. The residency question is manageable if you can document and relocate. The risks that actually lose patient data are quieter: standing vendor access, unscoped credentials, prompts that leave a trail on someone else's servers, and audit logs that cannot answer the one question an investigator will ask.
Build for those. Put vendor access under your own identity system, scope every credential to the field, get the no-training term in the contract, and keep an immutable log you control. Do that and the hosting diagram matters far less than the committee thinks it does. Skip it, and no server room in the building will save you.