Updated:
Before staff use an AI chatbot: the DPC’s questions for hospitals
Is a clinician pasting notes into a chatbot? The Data Protection Commission’s AI guidance lists what to understand first, and the risks it names, in plain language.
Hospitals now face two kinds of AI use: the tools the organisation buys, and the chatbots staff open on their own. The Data Protection Commission (DPC) addressed both in “AI, Large Language Models and Data Protection”, published on 18 July 2024. It is not about hospitals specifically, but nearly every question in it applies to them.
It starts with a warning
The DPC says there may be a tendency to use large amounts of personal data in training, sometimes unnecessarily and without people’s knowledge. It also flags accuracy and retention problems, models shared with others for purposes people do not agree with, biased outputs, and fine-tuning that exposes people to risks where there were none before. Where personal data is involved, the GDPR applies to organisations using AI systems as well as to the providers.
What to understand before using an AI product
The guidance says an organisation using an AI product could be a data controller, and that it should consider a formal risk assessment. Before starting, it should understand:
- what personal data the system uses and how it uses it;
- where the data goes when a third party is involved;
- whether the provider retains it or re-uses it; and
- how the product lets the organisation meet its GDPR obligations. The documentation should tell you in an understandable form.
Risks the DPC names
- Memorisation: some models can regurgitate passages of training data unintentionally.
- Filters can be circumvented, so data you thought was blocked may appear.
- Inaccurate or biased output relied upon without critical human review creates automated decision-making risks.
- Data subject rights: if someone asks for access to or deletion of their data in the AI system, can you do it?
- Storage limitation: without a retention schedule, the organisation risks non-compliance.
- Third-party products can raise further security risks from what employees input.
What to do on Monday
- Say in writing which AI tools staff may use with patient information, and which they may not.
- For each approved tool, record the four things above.
- Make sure someone can handle access and deletion requests that touch the tool.
- Add a retention rule.
If your hospital builds its own AI product using personal data, the guidance also asks you to consider whether there is a less risky non-AI route, to check that the purpose fits the legal basis you already have, and to think about an impact assessment.
Key takeaways
- The DPC’s July 2024 guidance applies to organisations that use AI products and to those that build them.
- Understand what data the tool uses, where it goes, whether it is kept or re-used, and how you meet your GDPR duties.
- Watch for memorisation, circumvented filters, unreviewed outputs, rights requests and retention. This is general information, not legal advice.
Sources: Data Protection Commission, AI, Large Language Models and Data Protection (18 July 2024). Read on 15 November 2026.