Governing Internal Knowledge AI: RAG Systems That Don't Leak Across Org Boundaries

Governing internal Knowledge AI: RAG systems that don't leak across org boundaries

Generative AI can turn a corporate knowledge base into a high-value, always-on expert. But it also turns that knowledge base into a potential compliance and reputational hazard - especially when Retrieval-Augmented Generation (RAG) systems blend documents from multiple business units, acquisitions, or portfolio companies. When the RAG pipeline doesn't carry entitlements as faithfully as the source systems, the result is an "ask anything" bot that leaks secrets nobody expected it to reveal.

This is the story of a private-equity backed holding company that learned that lesson the hard way - and how they redesigned governance, architecture, and operations to make internal knowledge AI safe, trustworthy, and audit-ready. The outcome wasn't just compliance: it preserved economic value (the system stayed in use), restored workflow trust (teams used it again), and created governance that accelerated future adoption.

The incident: confidential due diligence surfaced to the wrong people

A mid-market holding company used RAG to build a single internal assistant for hundreds of employees across its operating companies. The promise was real: faster onboarding, fewer repeated questions to scarce subject-matter experts, and a searchable memory of past deals and playbooks.

But one morning an associate at an operating company asked the assistant a question about integration checklists for recent acquisitions. The assistant surfaced verbatim excerpts from a recent acquisition target's due-diligence memo - documents that had been uploaded by the deal team and should have been limited to the investment group and the target's core board. The memo included sensitive valuation assumptions and negotiation strategy. Word spread. Operating-company managers were upset. The deal team lost confidence in the tool; several people disabled integrations. The holding company faced a confidentiality audit from an investor and - worst of all - risked damaging deal flow if word leaked.

Why did this happen? In short: the RAG pipeline treated content as fungible. Documents were indexed into a central vector store without preserving or enforcing document-level entitlements. Embeddings and retrieval ignored the ACLs that governed the original systems, and the answer builder concatenated content from multiple sources into a single response with no provenance or redaction.

They could have scrapped the project. Instead they redesigned the system around access control and governance.

Why RAG is uniquely risky for access control

RAG systems create a set of challenges that traditional access-control models don't handle well:

- Embeddings erase original context. When you convert a document into vectors, the representation contains semantic meaning but not the access-control metadata that lives in the source system. A vector lookup will surface matches regardless of who is asking unless you explicitly enforce entitlements at retrieval time.

- Fusion multiplies exposure. RAG synthesizes passages from different documents into one answer. Even if each source is only slightly sensitive, the synthesis can reveal inferences or join facts across boundaries.

- Provenance is harder after synthesis. Users get fluent answers, not citations. Without strong provenance trails, you can't tell which document fragments were used to generate a response.

- Indexing across boundaries creates blast radius. Mixing content from multiple business units into a single index makes it easy for queries from one group to surface content from another.

- Redaction is nontrivial. Automated redaction must be precise and fast, and it must coexist with the need for full-text retrieval for authorized users.

These features make RAG both powerful and potentially leaky. Governance must therefore be built into the architecture, not bolted on afterward.

Architecture patterns that enforce entitlements

The holding company rebuilt their RAG system using a layered architecture that treats entitlements as first-class metadata. Key patterns they used:

1. Source-system permission inheritance
- When ingesting content, connectors extract not only content and metadata, but the source ACLs (who can view, who owns, sensitivity labels).
- Ingested documents carry an immutable entitlement record. This is stored alongside the vectors in the index and in a separate entitlement database for fast checks.

2. Index sharding and tenancy
- Content is partitioned by organizational boundary: separate vector indexes or shards per portfolio company, per legal entity, or by sensitivity tier.
- Shared indexes are only used for public/internal-non-sensitive content. Sensitive and embargoed content stays in isolated indexes.

3. Document-level entitlement enforcement at retrieval
- Retrieval is a two-step guarded process:
1. A semantic search returns candidate document IDs (or vector IDs) but not the embeddings themselves.
2. A policy engine evaluates the caller's identity and attributes against the document entitlements before any text is returned to the RAG prompt builder.
- If a candidate fails the entitlement check, it is excluded from results and the retrieval is re-run or extended.

4. Metadata-aware embeddings
- Embeddings are stored with metadata tokens indicating sensitivity categories, jurisdiction, and retention tags. Retrieval filters can use this metadata as fast pre-filters before semantic scoring.

5. Redaction and response-sanitization pipeline
- Before a generated response is returned, a redaction engine scans the proposed answer for sensitive categories and redacts or masks content if the caller lacks clearance.
- For particularly sensitive categories (M&A documents, legal privileged material), the pipeline can escalate output to a human reviewer or return a safe alternative (summary without specifics).

6. Provenance and explainability
- Every response includes a provenance package: the list of document fragments consulted, source names, timestamps, and confidence scores.
- The system ensures a clear mapping from generated tokens to original document excerpts so audits can reconstruct chain-of-custody.

7. Encryption and key separation
- Keys are partitioned by business unit or sensitivity tier. Indexes holding sensitive content can be encrypted with separate KMS keys, reducing blast radius if a key is compromised.

8. Sync and revocation
- Connectors track changes in source-system entitlements. If access is revoked at the source, the entitlement database and index are updated and cached tokens invalidated - preventing stale access.

This architecture raised the technical bar, but it was the governance and process controls that ensured it actually worked in practice.

Audit, logging, and observable controls

Technical enforcement is necessary but not sufficient. The holding company implemented rigorous observability so auditors - and internal stakeholders - could verify the system's behavior.

What they logged and how they used it:

- Query-level logs - who asked what, timestamp, client app
- Retrieval logs - which document/vector IDs were considered and which passed entitlement checks
- Access decision logs - details of the policy evaluation that allowed or denied access
- Response assembly logs - which fragments were used in the final answer
- Redaction events - what was redacted, why, and whether escalation occurred

Logs were written to an immutable, tamper-evident store and integrated with the SIEM for anomaly detection: spikes in access to a particular due-diligence folder, cross-entity queries that triggered denials, or policy failures. Retention and redaction of logs were aligned with privacy policy and regulator expectations.

For audits, the company could replay queries and reconstruct the exact documents and decision path that produced a response. That capability was what satisfied the investor confidentiality audit.

The governance committee that owns enterprise knowledge AI

Technology without ownership drifts. The holding company created an "Enterprise Knowledge AI Committee" with clear charter and responsibilities:

- Composition
- Chair: Chief Data Officer or Head of AI
- Members: GC (legal), CISO (security), Head of Compliance, Head of HR (for training and role design), representatives from deal team and operating company leadership, and an investor representative from the private equity firm
- Technical advisor: platform engineering lead or vendor PM

- Mandate
- Approve sources for indexing and sensitivity classification criteria
- Define entitlement models (role-based, attribute-based, and document-level exceptions)
- Own the quarterly access review and certification process
- Oversee incident management and breach response playbook
- Approve new use cases or expansions across portfolio companies
- Maintain the vendor and procurement checklist (data residency, contract clauses, audit access)

- Processes
- Quarterly access reviews where business-unit leaders certify who needs access to what.
- Rapid review path for urgent business needs that require temporary elevated access (timeboxed and logged).
- Change control for model updates or index re-sharding.
- Regular tabletop exercises simulating leaks and response.

Ownership made governance a tool for speed, not just a brake. The committee's existence also signaled to operating-company teams that their confidentiality concerns were heard and addressed.

The payoff: trust returned, system kept, audit passed

With entitlements enforced, provenance visible, and a formal governance committee in place, the assistant regained trust. The deal team resumed uploading sensitive materials under strict entitlements. Operating companies felt safe using the system again, and leadership realized they'd preserved a valuable productivity capability rather than scrapping it after one incident.

The confidentiality audit found no evidence of uncontrolled disclosure. The firm's investors felt reassured. Critically, governance didn't shut down innovation - it made it sustainable. The company continued to roll out new knowledge services, now with an approved pattern for safely ingesting acquisition materials during due diligence.

Practical readiness moves for executives

If you run or sponsor enterprise knowledge AI, here are concrete steps to make your RAG deployment auditable and safe:

1. Map your knowledge boundaries
- Inventory systems and identify content that must not be shared across units (M&A, HR, legal, target DD).
2. Capture and persist entitlements at ingestion
- Ensure connectors extract ACLs and sensitivity tags. Store them with the indexed vectors and in a fast entitlement store.
3. Avoid a single undifferentiated index
- Use per-entity shards or per-sensitivity indexes for high-risk content.
4. Enforce document-level access on retrieval
- Implement a two-step retrieval/authorization flow where candidates are filtered by policy before text is returned.
5. Build a redaction/sanitization pipeline
- Deploy automated redactors for common sensitive categories plus human-in-the-loop for escalations.
6. Log everything needed for reconstruction
- Query logs, retrieval logs, decision logs, and response provenance should be immutable and auditable.
7. Create a governance committee with teeth
- Include legal, security, business owners, and investors if necessary. Run quarterly access certifications.
8. Plan for revocation and lifecycle
- Ensure entitlement changes in source systems propagate quickly to indexes and caches.
9. Run tabletop audits and incident drills
- Validate that logs and provenance let you reconstruct answers and that your breach playbook works.
10. Link to existing frameworks
- Use NIST AI RMF principles, sector regulations, and ISO guidance as pragmatic guardrails for risk classification and controls.

Conclusion - governance as an enabler

RAG systems are transformational for knowledge work, but they force organizations to reconcile two truths: access multiplicity (who should see what) and semantic fusion (how content combines). Treating governance as a compliance checkbox will break the system or drive it underground. Treating governance as an operating capability - with architecture that preserves entitlements, observability that supports audits, and a committee that makes trade-offs - makes RAG both safe and productive.

For executives, the takeaway is simple: if you want knowledge AI to scale across businesses and survive investor and regulator scrutiny, design access control and governance into the stack from day one. The economic upside - faster decisions, better reuse of institutional knowledge, and less duplicated work - only materializes when teams trust the tool. Governing RAG well is how you build that trust.