How AI Can Be Used Safely in Human Services

The question that actually matters is whether it's being used in a way that protects the people you serve and holds up if DHS or a privacy officer asks how it works. Here's how we think about it, and how we build it into the systems we set up for providers.

Start with where the data lives

Before turning on any AI tool, map exactly where protected health information would touch it. If a tool sees service notes, incident reports, or anything tied to a specific person, it needs to be treated like any other system handling PHI. It doesn't get a shortcut around your usual controls just because it's new.

Get a business associate agreement that actually says what you need

A generic AI tool isn't automatically HIPAA compliant just because a vendor says so. You need a signed business associate agreement, and it needs to explicitly prohibit the vendor from using your data to train its models. If a vendor can't or won't sign that, that tells you everything you need to know.

Know when 42 CFR Part 2 applies

If your program touches substance use disorder treatment records, there's a stricter federal layer on top of HIPAA. 42 CFR Part 2 protects even the fact that someone received SUD treatment, and it comes with its own consent and redisclosure rules. An AI tool that's fine for general case notes may not be fine for anything touching SUD-related records without additional safeguards layered on top.

Keep a person in the loop, every time

The providers getting this right treat AI as a drafting assistant, never as a decision maker. A tool can draft a note. It shouldn't finalize one. Every AI-assisted note needs a qualified staff member to review it before it becomes part of the official record, and the workflow itself should make that step impossible to skip.

Log who reviewed what

If a note was AI-assisted, your system should be able to show, on demand, who reviewed it and when. That audit trail is what separates a defensible use of AI from a liability, and it's usually the first thing a reviewer or a privacy officer will ask to see.

Train staff on the boundaries

Staff need to know, specifically, what the tool can help with and where it stops. An AI system that answers compliance questions based on your own policies is genuinely useful. One that staff start trusting to make judgment calls about a person's care plan is a different, riskier thing entirely, and the line between the two isn't always obvious unless someone draws it explicitly.

Where this leaves providers

Used carefully, AI can meaningfully cut down the hours staff spend on documentation and give coordinators faster answers to compliance questions. Used carelessly, it introduces exactly the kind of risk 245D providers can least afford. The difference comes down to whether the tool was set up with these guardrails from day one, not added after something went wrong.

The AI layer we build into Karebase engagements is trained on 245D and HCBS documentation, augmented with a program's own policies, and built around the human-in-the-loop and audit logging principles above from the start.