How to Conduct a HIPAA Risk Assessment for AI-Powered Healthcare Applications

Introduction

Artificial intelligence is transforming healthcare through automated clinical documentation, medical data analysis, patient communication, and clinical decision-support applications. However, AI systems that process protected health information (PHI) introduce additional privacy and cybersecurity risks.

A HIPAA risk assessment helps healthcare organizations identify potential threats to electronic protected health information (ePHI), evaluate security vulnerabilities, and determine appropriate safeguards.

Unlike traditional software, AI-powered healthcare applications may introduce additional risks involving model training, external APIs, prompts, generated outputs, cloud infrastructure, and third-party data processing.

Understanding these risks is essential before deploying AI solutions in healthcare environments.

1. Identify Where AI Applications Process PHI

Start by mapping how patient information enters, moves through, and leaves the AI application.

Identify all systems that create, receive, maintain, or transmit ePHI, including:

• Electronic health record (EHR) integrations.

• AI transcription and documentation tools.

• Patient-facing chatbots.

• Cloud storage and databases.

• External AI APIs.

• Application logs and monitoring systems.

• Backup and recovery environments.

For example, an AI medical documentation application may process patient conversations, generate clinical summaries, and transfer information into an EHR. Each processing point should be evaluated for potential exposure.

HHS identifies understanding the location and movement of ePHI as a fundamental part of HIPAA risk analysis.

2. Identify AI-Specific Security and Privacy Risks

After mapping data flows, identify threats and vulnerabilities that could compromise patient information.

Common AI-related risks include:

• Unauthorized access: Weak authentication or excessive permissions expose patient records.

• Insecure APIs: Poorly protected integrations expose sensitive information.

• Prompt leakage: PHI entered into AI prompts may appear in logs or unintended locations.

• Uncontrolled data retention: AI providers may retain prompts, outputs, or recordings longer than expected.

• Model training exposure: Patient information may be used for model training without appropriate contractual and technical restrictions.

• Inaccurate AI outputs: Generated clinical summaries may contain incomplete or misleading information.

Evaluate each risk based on its potential effect on the confidentiality, integrity, and availability of ePHI.

3. Evaluate Existing Security Controls

Review the administrative, physical, and technical safeguards protecting the AI application.

Important security controls include:

• Role-based access control.

• Multi-factor authentication.

• Encryption during transmission and storage.

• Audit logging and activity monitoring.

• Secure API authentication.

• Backup and recovery procedures.

• Workforce security training.

• Incident response policies.

Also assess whether developers, administrators, and healthcare employees have access only to the information necessary for their responsibilities.

Remember that encryption is an addressable implementation specification under the HIPAA Security Rule. Organizations must evaluate appropriate safeguards based on their risk analysis and document their decisions.

4. Assess AI Vendors and Third-Party Services

AI applications frequently depend on external cloud providers, model providers, analytics services, and software vendors.

Identify every provider that may create, receive, maintain, or transmit PHI.

Evaluate:

• Whether a Business Associate Agreement (BAA) is required and available.

• Which services and configurations the BAA covers.

• Whether subcontractors process PHI.

• How prompts and generated outputs are stored.

• Whether submitted information is used for model training.

• Data retention and deletion policies.

• Security incident reporting responsibilities.

A BAA alone does not guarantee HIPAA compliance. Organizations must also evaluate their own application architecture, configurations, policies, and security practices.

5. Evaluate and Prioritize Identified Risks

Not every vulnerability creates the same level of risk. Evaluate each identified threat according to its likelihood and potential impact.

These are illustrative ratings, not universal HIPAA classifications. Actual priorities should reflect the organization's systems, environment, existing controls, and documented risk methodology.

6. Develop a Risk Management Plan

After prioritizing risks, create a documented plan to reduce them to reasonable and appropriate levels.

Assign responsibilities and timelines for corrective actions.

Possible improvements include strengthening authentication, restricting API permissions, improving encryption practices, reviewing vendor agreements, limiting unnecessary data retention, and conducting security testing.

Document identified risks, existing safeguards, remediation decisions, and follow-up activities. HIPAA compliance requires more than completing an assessment; organizations must also implement appropriate risk management measures.

7. Use Updated HIPAA and AI Risk Assessment Resources

Healthcare organizations should use current regulatory guidance when evaluating AI applications.

As of November 2026, the existing HIPAA Security Rule remains in effect. HHS published proposed Security Rule cybersecurity changes on January 6, 2025, but these should not be treated as finalized requirements. Organizations should monitor the rulemaking process while continuing to follow the current requirements.

Another useful resource is the HHS Security Risk Assessment Tool version 3.7, updated September 18, 2026. It includes improvements addressing assessment scope, remote access, technology asset inventories, system activity logging, and potential assessment review triggers.

Organizations can also consult the NIST AI Risk Management Framework and its Generative AI Profile to evaluate AI-specific concerns such as governance, third-party dependencies, pre-deployment testing, and generated content risks. These are voluntary resources, not HIPAA certifications.

Conclusion

Conducting a HIPAA risk assessment for AI-powered healthcare applications requires more than reviewing basic software security. Organizations must understand how PHI moves through AI systems, evaluate third-party providers, identify technology-specific vulnerabilities, and implement appropriate safeguards.

Regular assessments and documented risk management help healthcare organizations make informed technology decisions while protecting sensitive patient information.

Building a secure healthcare AI application?

Barqsol helps organizations explore HIPAA-focused AI solutions with attention to application architecture, data protection, and healthcare technology requirements.

Explore HIPAA-Compliant AI Solutions to learn more about developing healthcare AI applications with privacy and security considerations in mind.