Technical whitepaper

1. Threat model

Most "private AI" claims skip this step. We'd rather tell you exactly what we defend against and what we don't, so you can make an informed decision instead of trusting a slogan.

In short: the product's job is to stop your own software from leaking client data to the internet, and to stop personal identifiers from reaching the AI model unnecessarily. It is not a replacement for disk encryption, endpoint security, or professional judgment.

2. Architecture

Legal Box is a desktop application with three local processes and no remote ones:

There is no account system, no license server, no telemetry endpoint, and no analytics SDK anywhere in the stack. The only egress that ever occurs is the one-time model download during first-launch setup, which the user explicitly initiates and which stops permanently once the model is on disk.

3. The PII shield — masking before the model, restoring after

Every piece of text bound for the model passes through a two-layer detector built on Presidio (MIT — the open-source PII framework originally created by Microsoft) plus Singapore-specific recognizers written for this product. Detection runs as a single pass over the original text — every regex and the NER model see the same untouched string — and overlapping matches are resolved by a fixed category priority (structured identifiers like NRIC beat looser matches like a bare name) before any replacement happens. That ordering matters: running detectors sequentially on text that's already partly masked is a common source of missed or double-masked entities, and we specifically designed against it.

Detected values are replaced with indexed placeholders ([PERSON_1], [NRIC_1]…) before the prompt is assembled. The model only ever sees placeholders — it has no way to know a real name was ever there. After the model responds, the placeholders are swapped back using a reversible token map that lives only in the backend process's memory for the duration of the request.

What the lawyer types:
  "Advise on Tan Ah Kow (NRIC S1234567A), owes Acme Pte Ltd S$45,000, call +65 9123 4567"

What the AI model receives:
  "Advise on [PERSON_1] (NRIC [NRIC_1]), owes [ORG_1] S$45,000, call [PHONE_1]"

What the lawyer gets back:
  Real names, restored automatically. The model never had them.

A bug we found and fixed — told straight, because that's the point of this page

An earlier version restored placeholders with a naive find-and-replace. Once a document had ten or more entities of the same type, [PERSON_1] would match as a substring of [PERSON_10], [PERSON_11], etc., corrupting the restore for every double-digit entity. We rewrote restoration as a single regex pass over whole tokens (\[[A-Z][A-Z_]*_\d+\]), added a regression test that plants twelve NRICs in one fixture and asserts every one restores to exactly its own value, and now count any leftover placeholder-shaped text in the audit log so a mangled restore is detectable, not silent. We're disclosing this because a security page that only lists what's perfect isn't credible — a page that shows what broke and how it's tested against now is.

Three things reviewers usually probe

• The search index is masked too. Uploaded case documents are shielded before chunking and embedding, so the vector database only ever stores placeholder text — even the search index itself never holds a raw NRIC or name. The per-case token map lives in a local SQLite column.
• Degradation is loud, not silent. If the NER model fails to load, the app shows a red "PII shield degraded — names not masked" banner and reports it in /api/health. The regex layer (NRIC, phones, emails, UEN…) keeps working regardless — only free-text name/organisation detection depends on the NER model.
• You can inspect it live, not just read about it. Settings → Confidentiality has a "paste anything — see exactly what the AI sees" panel. Bring your own test data and check coverage before you trust it with anything real.

4. The egress lock — incapable, not just configured

Most "your data stays local" claims are a configuration choice — a setting that could be flipped, a call that could be added in the next release without anyone noticing. Legal Box makes it a structural property of the running process instead.

At startup the backend patches its own socket layer: any attempt to open a connection to a non-loopback address raises an error before any network I/O happens. The only sanctioned peer is the local Ollama runtime. The Electron shell has an equivalent allowlist on the UI side. This means a bug, a misbehaving dependency, or a malicious document can't quietly exfiltrate anything — the connection is refused at the socket, not filtered after the fact.

# backend/egress_guard.py — actual behaviour, not a mock-up
socket.connect(("8.8.8.8", 53))
PermissionError: Egress blocked by Legal Box confidentiality guard:
refusing outbound connection to '8.8.8.8'. All processing is local-only by design.

socket.connect(("127.0.0.1", 11434))     # Ollama — allowed
socket.connect(("192.168.1.1", 443))     # anything else — blocked, no exceptions

5. Data at rest

• All user data lives in one folder (~/Library/Application Support/OTG Legal Box/ on macOS; %APPDATA%\OTG Legal Box\ on Windows).
• The search index stores masked text only; original PDFs are retained for provenance and re-indexing.
• The token maps sit beside the data, so this is AI-exposure hygiene, not encryption — we say this plainly rather than overselling it. For loss-or-theft protection, enable FileVault (macOS) or BitLocker (Windows) full-disk encryption — this is a hard requirement in our own deployment checklist, not an optional nicety.
• The audit log records metadata only: action, timestamp, duration, and PII counts — never document content or the masked values. Every entry carries "network": "local_only".
• One-click erasure: Settings → About → "Remove all data…" wipes the entire data folder (type-ERASE confirmation), for clean offboarding or machine handover — verified by a test that plants files and asserts the folder is empty afterward.

6. Testing & CI — the test suite is the actual contract

Marketing claims are cheap. Here is what runs on every push, and fails the build if it doesn't hold:

• No-leak: synthetic Singapore legal documents (letters of demand, affidavits, sale-and-purchase agreements, chronology memos) with manifests of every planted PII value — none may ever appear in a prompt captured on its way to the model, across all nine AI-facing endpoints.
• Round-trip identity: mask-then-restore must reproduce the original text byte-for-byte, including a regression fixture with fourteen entities of the same type (the exact scenario the bug in §3 broke).
• Collision guards: a Singapore postal code must never be misread as a bank account; a bare ISO date must never be masked; a Luhn-invalid number that merely looks like a card must be left alone.
• Egress enforcement: connections to public and private non-loopback addresses must raise a PermissionError; loopback connections must pass through untouched.
• Endpoint round trips: a fake model that echoes back placeholder tokens verifies every one of the nine LLM-facing endpoints — including the bundle-authorities tool, the one gap our own review found and closed.
• Benchmark: a per-entity precision/recall harness against a gold-labelled corpus; recall must not regress on any entity type between releases.

Current status: 47 tests passing, 4 skipped only in sandboxes that can't download the optional large NER model (they run fully in CI). Reproduce it yourself: cd backend && pytest tests/ -q.

7. Supply chain

Every third-party component, its exact license, and how it's used is recorded in the project's NOTICE.md — including which projects were adapted for content (Anthropic's claude-for-legal, Apache 2.0) and which were used only to prepare data during development and never shipped in the product (worldwidelaw/legal-sources, AGPL — the boundary is documented explicitly). There is no AGPL code in the shipped application; an earlier PDF library was replaced specifically to keep that true.

8. vs. cloud AI tools

This isn't a claim that local models match frontier cloud models on every task — they don't, yet. It's a statement about where the tradeoff sits for confidential legal work: for firms where "did our data leave the building" is the first question a client asks, the architecture matters more than the last few points of benchmark quality.

9. Self-hosting

The core engine — the PII shield, the egress guard, the document tools, the desktop app — is MIT-licensed (see LICENSE). We're in the process of publishing the source repository; until it's public, email us and we'll send access directly — no sales call required, and no cost to run it yourself once you have it:

git clone <repository URL — provided on request>
cd legal-box
./setup.sh        # installs Python + Node dependencies, downloads the NER model
./start.sh         # runs the app in developer mode

# or build a distributable installer:
./scripts/build-app.sh

Full build and packaging instructions are in BUILD-GUIDE.md; the developer install path is in README.md. If your firm has an engineer who can run a shell script, this is a legitimate, fully-supported way to run Legal Box for free, forever, with no contract required.

10. Verify it yourself — the five-minute script

1. Turn Wi-Fi off first. Then run Chat, Contract Review, Summarize, Chronology. Everything works.
2. Open Settings → Confidentiality. Paste your own test text; confirm the masked version is what you'd accept a third party seeing (there is no third party — but the point stands).
3. Check the sockets. lsof -i -P | grep -iE 'legalbox|ollama' — every connection is 127.0.0.1. Run Little Snitch or GlassWire for the same conclusion with a GUI.
4. Read the audit log. Metadata only, PII counts only, local_only on every line.
5. Read the source. backend/pii_shield.py and backend/egress_guard.py are under 400 lines combined, deliberately kept small enough to actually review — request access (§9) and read them yourself rather than taking this page's word for it.

11. Limitations — stated plainly

• PII detection is very strong on structured identifiers (NRIC, UEN, phones — 100% recall on our benchmark corpus) and good-but-not-perfect on free-text names, especially non-Western names where the underlying NER model has weaker training coverage; the live preview exists precisely so you can check coverage on your own documents, and a human lawyer reviews all output regardless.
• Data at rest relies on OS full-disk encryption rather than app-level cryptography (see §5) — an honest tradeoff for a desktop appliance, not a gap we're hiding.
• Server mode (one Mac Mini serving a firm over office Wi-Fi) currently trusts the office network; per-user authentication is on the roadmap, and the recommended pilot configuration is single-machine desktop mode until that ships.
• AI output quality is bounded by the local model, which trails the largest cloud models on some tasks; every result is labelled a draft for a qualified lawyer's review.
• Installers are not yet code-signed, so first install requires a one-time Gatekeeper/SmartScreen bypass (documented in INSTALL.md) — signing is in progress.

12. FAQ

Why not just use ChatGPT Enterprise or Azure OpenAI with a DPA?

A Data Processing Agreement is a legal promise about how a vendor will handle data that still leaves your building. Legal Box's answer to "does it leave the building" is architectural, not contractual — there's no vendor to sign an agreement with because there's no vendor in the data path. Both approaches can be legitimate depending on a firm's risk posture; we simply don't ask a client to trust a policy when a socket-level guard is available instead.

Is the local model as good as GPT-4-class models?

Not on every benchmark, no — that gap has been narrowing every quarter and is real to acknowledge. The product lets you pick the largest model your hardware can run, and every output is explicitly a draft for a qualified lawyer's review, not a final answer.

What happens if the AI model itself is malicious or compromised?

Ollama and the model weights are pulled from their official sources at install time and never fetched again without an explicit user action. This is supply-chain trust, not a technical guarantee — it's listed honestly in §1 as something the architecture reduces but doesn't eliminate.

Can I run this on a shared office machine for multiple lawyers?

Yes — Server mode does this today over office Wi-Fi — but per-user authentication isn't built yet (§11), so we currently recommend it only within a firm's existing trusted network, and desktop mode for anything beyond that.

Found something in this page that doesn't hold up, or want to inspect deeper? haojun@ontheground.agency — including security disclosures.