Privacy and Data Protection in AI Applications¶

What You'll Learn¶

Objective	Time	Difficulty
Detect and handle PII in AI pipelines	35 min	Intermediate
Implement anonymization and redaction
Apply data minimization principles
Understand key privacy regulations

Privacy Risks in AI Applications¶

When users interact with AI systems, sensitive data flows through multiple points:

User Input (may contain PII)
    |
    v
Your Application Server (logs?)
    |
    v
API Provider (OpenAI, Anthropic, etc.)
    |  - May use data for training?
    |  - Stored in logs?
    |  - Accessible to provider employees?
    v
Model Response (may reflect PII)
    |
    v
Your Storage (conversation history, analytics)

Key Risks¶

Risk	Description	Impact
Data leakage via API	PII sent to third-party model providers	Privacy violation, regulatory fines
Training data memorization	Models may memorize and repeat training data	Exposure of personal information
Conversation logging	Storing conversations with sensitive content	Data breach liability
Context window exposure	Other users' data in shared context	Cross-user data leaks
Model outputs containing PII	Model generates real personal information	Privacy violation

PII Detection and Redaction¶

Basic PII Detection with Regex¶

import re
from typing import Dict, List, Tuple

def detect_pii(text: str) -> List[Dict]:
    """Detect common PII patterns in text."""
    patterns = {
        "email": r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
        "phone_us": r'\b(?:\+1[-.\s]?)?\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}\b',
        "ssn": r'\b\d{3}-\d{2}-\d{4}\b',
        "credit_card": r'\b(?:\d{4}[-\s]?){3}\d{4}\b',
        "ip_address": r'\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b',
        "date_of_birth": r'\b(?:DOB|born|birthday)[:\s]+\d{1,2}[/\-]\d{1,2}[/\-]\d{2,4}\b',
    }

    found_pii = []
    for pii_type, pattern in patterns.items():
        matches = re.finditer(pattern, text, re.IGNORECASE)
        for match in matches:
            found_pii.append({
                "type": pii_type,
                "value": match.group(),
                "start": match.start(),
                "end": match.end()
            })

    return found_pii

def redact_pii(text: str) -> Tuple[str, List[Dict]]:
    """Detect and redact PII from text."""
    pii_items = detect_pii(text)

    # Sort by position (reverse) to maintain correct indices during replacement
    pii_items.sort(key=lambda x: x["start"], reverse=True)

    redacted = text
    for item in pii_items:
        placeholder = f"[{item['type'].upper()}_REDACTED]"
        redacted = redacted[:item["start"]] + placeholder + redacted[item["end"]:]

    return redacted, pii_items

# Example usage
text = "Please help John Smith at john.smith@email.com, phone 555-123-4567, SSN 123-45-6789"
redacted, found = redact_pii(text)
print(f"Redacted: {redacted}")
print(f"Found {len(found)} PII items")

LLM-Based PII Detection (More Comprehensive)¶

from openai import OpenAI
import json

client = OpenAI()

def detect_pii_with_llm(text, model="gpt-4.1-mini"):
    """Use an LLM to detect PII that regex might miss."""
    response = client.chat.completions.create(
        model=model,
        messages=[{
            "role": "system",
            "content": """Identify all personally identifiable information (PII) in the text.
Include: names, emails, phone numbers, addresses, SSNs, credit cards, 
dates of birth, medical info, financial info, and any other identifying data.

Return JSON: {"pii_items": [{"type": "...", "value": "...", "risk": "high|medium|low"}]}"""
        }, {
            "role": "user",
            "content": text
        }],
        temperature=0,
        response_format={"type": "json_object"}
    )

    return json.loads(response.choices[0].message.content)

Data Minimization for LLM Calls¶

Send only the data the model needs — nothing more.

def minimize_data_for_llm(user_message, task_type):
    """Strip unnecessary PII before sending to LLM."""
    # Step 1: Detect PII
    redacted_text, pii_items = redact_pii(user_message)

    # Step 2: Determine which PII is needed for the task
    needed_pii_types = {
        "general_question": [],  # No PII needed
        "account_lookup": ["email"],  # Only email needed
        "shipping_update": ["name"],  # Only name needed
        "billing_issue": ["name", "email"],  # Name and email
    }

    allowed = needed_pii_types.get(task_type, [])

    # Step 3: Only restore PII that is needed
    minimized = redacted_text
    for item in pii_items:
        if item["type"] in allowed:
            minimized = minimized.replace(
                f"[{item['type'].upper()}_REDACTED]",
                item["value"]
            )

    return minimized

Privacy Regulations Overview¶

Requirement	Implication for AI Apps
Lawful basis	Need consent or legitimate interest to process data
Data minimization	Only collect/send what's necessary
Right to erasure	Must be able to delete user data on request
Right to explanation	Users can ask why an AI made a decision
Data Processing Agreements	Need DPAs with API providers (OpenAI, etc.)
Cross-border transfers	Restrictions on sending data outside EU

CCPA (California Consumer Privacy Act)¶

Requirement	Implication
Disclosure	Tell users what data you collect and how it's used
Opt-out	Allow users to opt out of data "sale" (includes sharing with AI providers)
Deletion	Delete personal data on request

AI-Specific Regulations¶

Regulation	Region	Key Requirements
EU AI Act	EU	Risk-based framework, transparency for high-risk AI
Executive Order on AI	US	Safety testing, red-teaming for powerful models
PIPL	China	Strict data localization, consent requirements

Privacy-First Architecture¶

class PrivacyAwareAIClient:
    """Wrapper that enforces privacy controls before sending data to an LLM."""

    def __init__(self, client, log_pii_detections=True):
        self.client = client
        self.log_pii_detections = log_pii_detections

    def chat(self, messages, model="gpt-4.1-mini", redact=True, **kwargs):
        """Send a chat request with automatic PII handling."""
        processed_messages = []

        for msg in messages:
            if msg["role"] == "user" and redact:
                redacted_content, pii_found = redact_pii(msg["content"])

                if pii_found and self.log_pii_detections:
                    print(f"WARNING: Redacted {len(pii_found)} PII items from user message")

                processed_messages.append({
                    "role": msg["role"],
                    "content": redacted_content
                })
            else:
                processed_messages.append(msg)

        return self.client.chat.completions.create(
            model=model,
            messages=processed_messages,
            **kwargs
        )

# Usage
privacy_client = PrivacyAwareAIClient(OpenAI())
response = privacy_client.chat([
    {"role": "user", "content": "Help me with my account, email is john@example.com"}
])

Resources¶

OWASP LLM Privacy Guide: Best practices for LLM data handling
GDPR.eu: Comprehensive guide to GDPR compliance
Microsoft Presidio: Open-source PII detection and anonymization library
NIST AI Risk Management Framework: Government framework for AI risk assessment

Key Takeaways¶

Always assume user inputs contain sensitive data — detect and redact PII before sending to LLM APIs
Apply data minimization: only send what the model needs for the specific task
Understand your obligations under GDPR, CCPA, and emerging AI regulations
Use a layered approach: regex detection + LLM-based detection + data minimization
Review your API provider's data policies — understand if/how they use your data
Build privacy controls into your architecture from day one, not as an afterthought

Next Lesson¶

Lesson 6: AI Alignment and RLHF Safety — Explore the deeper challenges of aligning AI systems with human values and intentions.