BR /Private AI
Private AI: what stays inside your environment?
A practical checklist for model hosting, document access, external connections, logs, and backups before a private AI pilot.
A local model is only one part of the system
Running a language model on a company server answers one question: where that model processes a request. It does not establish where documents are indexed, where conversations are logged, or which external services an assistant can call.
Before choosing hardware, write down what “private” must mean for your workload. Is the requirement that documents remain on premises, that only approved processors handle them, or that the whole application works without an internet connection? These are different architectures, with different operating responsibilities.
Our starting point is an explicit boundary: approved data, approved users, approved processing locations, and approved outbound connections. A deployment label is not a security guarantee.
Map the whole journey of a request
For each component, record what it receives, where it runs, who can access it, how long it keeps data, and what it can send elsewhere. Use the following checklist during an architecture review; it is not a description of a completed BELL RIVER deployment.
| Component | What to establish |
|---|---|
| Model inference | Where prompts, retrieved passages, and responses are processed; whether any fallback uses a hosted model. |
| Document processing and embeddings | Where files are parsed and converted into searchable representations; whether OCR or embedding requests use another provider. |
| Search and document stores | Where source files and indexes live; how permissions, deletions, and updates propagate. |
| Tools and connectors | Which systems can receive data or be changed, using whose credentials, and with what approval. |
| Logs and telemetry | Whether prompts, outputs, document excerpts, or identifiers appear in diagnostics or support exports. |
| Backups and administration | Backup locations, restoration access, retention, software updates, and any remote support access. |
A hybrid arrangement can be appropriate when its data transfers are intentional and approved. If the requirement is no outbound processing, verify that requirement across every component and test it with outbound access restricted. Do not infer it from the model’s hosting location.
Do not give the assistant everyone’s access
A knowledge assistant should only retrieve material the requesting user is permitted to see. Indexing a restricted folder into a shared search store must not make its contents available to the entire company. Check both the original sources and the application’s retrieval controls, including what happens when access is revoked.
Reading a record and changing it are separate permissions. Start an operational assistant with the narrowest necessary tools. Require a person to approve consequential actions, and enforce that approval in the application—not just in instructions to the model.
These controls address recognized risks, not hypothetical perfection: OWASP describes sensitive-information disclosure and excessive agency as distinct application risks. See its guidance on sensitive information disclosure and excessive agency.
Test the workflow, not just the answer
For a first pilot, select one team, a limited document set, and a small set of useful questions. Begin with approved, non-sensitive examples where possible. Have subject-matter reviewers check answers and their supporting sources, including situations where the assistant should say it does not have enough information.
- Attempt retrieval as users with different permissions, including a user whose access has been removed.
- Test unavailable sources, stale documents, and requests outside the approved scope.
- Inspect application logs and configured network destinations for unexpected data transfers.
- Test a blocked tool action and confirm that it cannot bypass the required approval.
- Record useful-answer quality, correction effort, response time, and operating cost on representative tasks.
Set acceptance criteria before evaluation. A convincing demonstration alone does not establish production readiness, regulatory compliance, or a security certification.
Five questions to ask before commissioning a pilot
- Can you show every place our prompts, documents, and outputs may be processed or stored?
- Which external connections are required, optional, or disabled?
- How will source permissions and access changes be enforced?
- Which actions require approval, and where is that requirement enforced?
- Who owns updates, backups, monitoring, access reviews, and incident response after handover?
The useful outcome is a documented boundary and a tested operating workflow. Model selection comes inside that decision, not instead of it.