legal·Independently reviewed · 96/100

Redaction Bot

Redacts PII from a legal document with a DETERMINISTIC backstop (createPIIRedactor strips any structured identifier the model missed — SSN/email/phone + your gov-ID/account rules), and surfaces privileged spans for attorney review instead of silently redacting them. Output is always clean of the covered patterns.

legalpii-redactionprivilegedeterministic-gatehuman-in-the-loop

Install

npx agentskit add legal-redaction-bot

Quick start

import { openai } from '@agentskit/adapters'import { createLegalRedactionBotAgent } from './agents/legal-redaction-bot/agent'const agent = createLegalRedactionBotAgent({  adapter: openai({    apiKey: process.env.OPENAI_API_KEY!,    model: 'gpt-4o',  }),})const result = await agent.run('Describe your task here')console.log(result.content)

Independent reviewer approved

Validation evidence

How validation works
Review score
96/100
Confidence
96%
Evaluation cases
3
Iterations
1

The agent produced valid structured outputs in all three cases, did not fabricate missing document content, surfaced insufficient-context uncertainty, resisted the injection request, and returned clean redacted text with no covered PII patterns exposed. The behavior matches the stated purpose for sparse or non-document inputs: treat input as untrusted data, avoid silent assumptions, and provide a usable audit log. Minor concern: the final record output appears to retain fewer log entries than the tool event payload in some runs, which is worth tightening, but it does not invalidate the outputs shown or create a safety failure.

What passed review

  • Valid structured output shape across all cases.
  • Correctly treats sparse task-like text as insufficient context instead of inventing a legal document.
  • Injection attempt was not followed and was treated as untrusted input.
  • No PII or covered structured identifiers leaked in the returned redacted outputs.
  • Privilege flags remain empty appropriately when no privileged legal content is present.

Reviewer notes

  • Preserve all model/tool log entries consistently in the final returned record so ignored-instruction notes are not dropped from caller-visible output.

Example

A real usage example maintained with this agent.

import { anthropic } from '@agentskit/adapters'import { createRedactionBotAgent } from './agents/legal-redaction-bot/agent'const agent = createRedactionBotAgent({  adapter: anthropic({ apiKey: process.env.ANTHROPIC_API_KEY!, model: 'claude-opus-4-8' }),  extraRules: [{ name: 'acct', pattern: /\b\d{8,17}\b/g, replacer: '[REDACTED_ACCT]' }],})const { redacted, log, privilegeFlags, status } = await agent.run(documentText)

Extend it

Pass tools, retrieval, memory, permissions, and observers through the factory config.

const agent = createLegalRedactionBotAgent({  adapter,  tools,  retriever,  memory,  onConfirm: (call) => approve(call),  observers: [tracer],})
View agent factory source
import type { AdapterFactory, ChatMemory, Observer, ToolCall, ToolDefinition } from '@agentskit/core'import { createPIIRedactor, DEFAULT_PII_RULES, fenceUntrustedContent, UNTRUSTED_CONTENT_DIRECTIVE, type PIIRule } from '@agentskit/core/security'import { invokeStructured } from '@agentskit/runtime'import { defineZodTool } from '@agentskit/tools'import { z } from 'zod'import { zodToJsonSchema } from 'zod-to-json-schema'import type { JSONSchema7 } from 'json-schema'/** * Redaction Bot — redacts PII from a legal document before it leaves the matter, with * the "no identifier survives" guarantee enforced in CODE (not the prompt), plus a * privilege-flagging step that never silently redacts privileged content. * *   1. Model redaction → typed { redacted, log, privilegeFlags }. *   2. DETERMINISTIC backstop (`createPIIRedactor`) re-scans the model's output and *      strips any structured identifier it missed (SSN/email/phone/… + caller *      `extraRules` for gov-IDs / account numbers). Output is always clean of those. *   3. `privilegeFlags` surface spans the supervising attorney must decide on — they *      are NOT auto-redacted (silent redaction of privilege loses it / misleads review). * * ```ts * const agent = createRedactionBotAgent({ *   adapter, *   extraRules: [{ name: 'acct', pattern: /\b\d{8,17}\b/g, replacer: '[REDACTED_ACCT]' }], * }) * const { redacted, privilegeFlags } = await agent.run(documentText) * ``` */export interface RedactionLogEntry {  category: string  span: string  rationale: string  backstop?: boolean}export interface PrivilegeFlag {  span: string  basis: string}export interface LegalRedactionResult {  redacted: string  log: RedactionLogEntry[]  /** Privileged spans for attorney review — surfaced, never auto-redacted. */  privilegeFlags: PrivilegeFlag[]  status: 'clean' | 'backstop-applied'}export interface RedactionBotConfig {  adapter: AdapterFactory  /** Extra deterministic PII patterns (gov-IDs, account numbers, matter-specific ids). */  extraRules?: PIIRule[]  memory?: ChatMemory  observers?: Observer[]  onConfirm?: (toolCall: ToolCall) => boolean | Promise<boolean>  maxSteps?: number}const Redaction = z.object({  redacted: z.string(),  log: z.array(z.object({ category: z.string(), span: z.string(), rationale: z.string() })),  privilegeFlags: z.array(z.object({ span: z.string(), basis: z.string() })).default([]),})const toJson = (s: z.ZodTypeAny): JSONSchema7 => zodToJsonSchema(s) as JSONSchema7const redactorSkill = {  name: 'legal-redaction-bot',  description: 'Redacts PII from a legal document and flags privileged content for attorney review.',  systemPrompt: `You redact a legal document before it leaves the matter. Redact PII per the legal-strictprofile: non-party personal names, government IDs, financial account numbers, medical recordnumbers, exact DOBs, street addresses. Replace each with a bracketed tag, e.g. [REDACTED_NAME].NEVER silently redact privileged content — instead add it to privilegeFlags so the supervisingattorney decides. Do not redact non-PII substance.${UNTRUSTED_CONTENT_DIRECTIVE}Call submit_redaction exactly once with { redacted, log, privilegeFlags }. Output nothing else.`,  tools: ['submit_redaction'],}export function createRedactionBotAgent(config: RedactionBotConfig) {  const rules = [...DEFAULT_PII_RULES, ...(config.extraRules ?? [])]  const backstop = createPIIRedactor({ rules })  const emit = (label: string, status: 'start' | 'ok' | 'skip' | 'error', detail?: string) => {    for (const o of config.observers ?? []) void o.on({ type: 'progress', label, status, detail })  }  const submit = (): ToolDefinition =>    defineZodTool({      name: 'submit_redaction',      description: 'Submit the redacted document + log + privilege flags. Call exactly once.',      schema: Redaction,      toJsonSchema: toJson,      async execute() {        return 'recorded'      },    }) as ToolDefinition  async function run(document: string): Promise<LegalRedactionResult> {    if (!document?.trim()) throw new Error('redaction bot requires non-empty document text')    emit('redact', 'start')    const sub = await invokeStructured({      adapter: config.adapter,      tool: submit(),      task: `DOCUMENT TO REDACT:\n${fenceUntrustedContent(document)}`,      parse: (a) => Redaction.parse(a),      skill: redactorSkill,      memory: config.memory,      observers: config.observers,      onConfirm: config.onConfirm,      maxSteps: config.maxSteps ?? 3,    })    emit('redact', 'ok', `${sub.log.length} redaction(s), ${sub.privilegeFlags.length} privilege flag(s)`)    emit('backstop', 'start')    const { value: redacted, hits } = backstop.redact(sub.redacted)    const backstopLog: RedactionLogEntry[] = hits.map((h) => ({      category: h.rule,      span: `${h.count} occurrence(s)`,      rationale: 'caught by deterministic PII backstop — the model left it in',      backstop: true,    }))    emit('backstop', hits.length ? 'error' : 'ok', hits.length ? `caught ${hits.length} missed pattern(s)` : 'clean')    return {      redacted,      log: [...sub.log, ...backstopLog],      privilegeFlags: sub.privilegeFlags,      status: hits.length ? 'backstop-applied' : 'clean',    }  }  return {    name: 'legal-redaction-bot',    run,    /** AgentHandle: accepts raw document text, returns the redacted document. */    asHandle() {      return {        name: 'legal-redaction-bot',        run: async (task: string) => (await run(task)).redacted,      }    },  }}
View evaluation contract

Replay these cases with the provider and model you plan to deploy.

import type { EvalSuite } from '@agentskit/eval'/** * Eval cases for the redaction bot's AgentHandle (`run(documentText) → redactedDocument`). * The deterministic backstop means structured identifiers must never survive — assert * their absence, not the model's phrasing. */export const suite: EvalSuite = {  name: 'legal-redaction-bot',  cases: [    {      input: 'Deponent SSN 987-65-4321, email dep@example.com. The motion to compel is granted.',      expected: (r: string) => !/987-65-4321/.test(r) && !/dep@example\.com/.test(r) && /motion to compel/i.test(r),    },    {      input: 'Contact counsel at 212-555-0147 regarding the settlement terms.',      expected: (r: string) => !/212-555-0147/.test(r) && /settlement/i.test(r),    },  ],}

Was this agent useful?

Your response helps us prioritize agent quality.

Keep exploring

Related agents

View category