Methodology
Regulatory intelligence is only worth what its provenance is worth. This page describes how obligations get from a published legal text into a mapping you can defend to an auditor — and where the method's limits are.
1. Decomposition: from legal text to obligation
Source regulation text is ingested as published. On ingestion we compute a SHA-256 hash of the source text and store it against the regulation record. That hash is what lets us prove later which version of a text an obligation was derived from — and detect when the source changes underneath us.
The text is parsed into its hierarchical structure — chapters, articles, sections, sub-points — preserving the parent–child relationships rather than flattening to a list. Each section is then decomposed into individual obligations: a single enforceable requirement, with its subject (who must comply), the action required, the conditions under which it applies, and the source text excerpt that creates it.
Decomposition is to the article level and below, not to the framework level. "Implement a risk management system" is not one obligation; the specific, separately auditable requirements inside it are. This is the difference between a checklist and an evidence base.
Each obligation is embedded as a vector and stored alongside cross-references to related obligations in other regulations. Those cross-references are what make it possible to see that an EU AI Act Article 9 requirement, a NIST Govern practice and an ISO 42001 clause are asking for substantially the same control.
2. Mapping: from obligation to your systems
Mapping runs in two stages, deliberately.
Stage one is retrieval, not judgement. Your system description is embedded and compared against the obligation corpus by cosine similarity, returning the top 30 candidate obligations. This stage is fast and cheap, and it is allowed to be imprecise — its job is to avoid missing anything, not to decide anything.
Stage two is judgement. Candidates are passed in batches to Claude, which assesses whether each obligation genuinely applies to that system, what the impact is, and how confident the assessment is. Obligations judged not applicable are discarded rather than stored as low-confidence noise.
Every surviving mapping is stored with three things attached: a confidence score, the reasoning chain that produced it, and a link to the source regulation text. A mapping you cannot trace back to a specific article is not a finding — it is an assertion, and we do not store assertions.
Nothing is applied without a human. Mappings are proposed, not accepted. A reviewer accepts or rejects each one. The system's job is to make the review fast and well-evidenced, not to remove the reviewer.
3. Measurement: how we know the pipeline works
Prompt changes are not shipped on the strength of looking better. Every change to the mapping pipeline is scored against a hand-curated ground-truth set before it reaches production. Improvements are kept; regressions are discarded automatically.
Runs are scored on four weighted dimensions:
| Dimension | Weight | What it measures |
|---|---|---|
| Accuracy | 0.40 | Whether the applicability and impact judgements match ground truth |
| Structural | 0.30 | Whether the output conforms to the required schema |
| Rationale | 0.20 | Whether the reasoning chain is complete and specific |
| Parsability | 0.10 | Whether the output can be reliably machine-read |
Stated plainly: the ground-truth set is currently small — a handful of hand-curated cases, not a large benchmark. It is enough to catch regressions in output structure and obvious accuracy failures. It is not enough to claim a validated accuracy rate, and we do not claim one. Expanding it is ongoing work, and we would rather tell you its size than let the phrase "benchmarked against ground truth" imply something larger.
4. Provenance and audit trail
Every AI-generated output — obligation extraction, mapping, gap analysis, copilot answer — records the model used, the confidence score, the source text excerpt that informed it, and the reasoning chain, in a structured audit record.
The audit log is append-only: entries are written once and never updated or deleted. That constraint is enforced in the application layer rather than by a database grant, which is a real distinction and one we would rather state than gloss. It means the guarantee is as strong as the application boundary around the database, not stronger.
The practical test we hold ourselves to: for any conclusion the platform presents, a user should be able to reach the specific sentence of regulation that produced it, and see the reasoning that connected the two. If a conclusion cannot survive that trace, it should not be displayed.
5. Currency: how the data stays current
Source texts are monitored by content hash. When a monitored source changes, the hash changes, and the affected regulation is flagged for re-ingestion rather than silently drifting out of date.
When a monitored source changes, a regulation-level update record is written — classified as an amendment, guidance, correction or repeal — carrying the source reference and an assessment of which obligations are affected. That record is the audit trail of the change itself.
Being built, and not yet true: obligation-level supersession. Today an obligation row carries no status field, so a superseded provision cannot yet be marked as such individually — the change is recorded against the regulation, not against each obligation it touched. We are adding that field now, prompted by the two mid-2026 changes that make it unavoidable: Colorado's repeal and reenactment of its AI statute, and the rescheduling of the EU AI Act's high-risk regime by Regulation (EU) 2026/1744. Until it ships, treat obligation records as reflecting the version of the text they were extracted from, identified by that text's content hash, rather than as automatically current.
The design principle it will follow: flag, never delete. Deleting a superseded obligation would destroy the audit trail of assessments made in good faith against the previous version — and an assessment is always a claim about a moment in time. A repeal-and-reenactment makes this sharper than it sounds: Colorado's successor statute reuses the same section numbers for different content, so a stored citation can become silently wrong rather than visibly broken.
6. What this method does not do
- It does not determine legal compliance. It structures obligations and evidences gaps. Whether an organisation is compliant is a legal conclusion, and not one a pipeline is competent to reach.
- It does not guarantee completeness. Decomposition captures what is in the ingested text. A regulation not ingested, or a source text not yet re-read after amendment, is not covered — which is why currency and provenance are treated as first-class rather than as a maintenance chore.
- It does not replace review. Every mapping is proposed for a human to accept or reject. Volume of output is not the product; reviewable, sourced output is.
- It does not have a published accuracy rate. See section 3.
7. Corrections
When we get something wrong, we correct it in place, date the correction, and leave the original visible where it was load-bearing for a reader's decisions. We would rather publish a visible correction history than a clean-looking archive.
Regulatory intelligence, not legal advice. This page describes how we structure and maintain regulatory data. It is not a substitute for advice from a qualified lawyer in your jurisdiction.