+---------------------------------------------------------------+
|  PHASE 7 — EVAL + TRADE-OFF DEFENSE                            |  refusal rate + veto rate + defense document
+---------------------------------------------------------------+
|  PHASE 6 — BUILD HARNESS POLICY GATES                          |  eval'd gates, ~100% veto on illegitimate
+---------------------------------------------------------------+
|  PHASE 5 — SERVE LOCALLY (quantized)                           |  Ollama/llama.cpp, fine-tuned behavior
+---------------------------------------------------------------+
|  PHASE 4 — QUANTIZE (GGUF)                                     |  deployable .gguf
+---------------------------------------------------------------+
|  PHASE 3 — (OPTIONAL) REASONING DISTILL                        |  recovered capability (if abliteration used)
+---------------------------------------------------------------+
|  PHASE 2 — STEER (abliterate OR DPO-toward-compliance)         |  reduced refusal, measured cost
+---------------------------------------------------------------+
|  PHASE 1 — CHOOSE BASE + STEERING APPROACH (defend it)         |  base + method + written justification
+---------------------------------------------------------------+

class PolicyGate:
    def __init__(self, policy: Policy):
        self.policy = policy   # allow-lists, deny-lists, approval rules

    def check(self, tool_name: str, args: dict) -> Decision:
        if not self.policy.tool_allowed(tool_name):
            return Decision.veto(f"tool '{tool_name}' not permitted")
        if not self.policy.target_in_scope(args.get("target")):
            return Decision.veto(f"target out of scope")
        if self.policy.requires_approval(tool_name) and not args.get("_approved"):
            return Decision.veto(f"action requires human approval")
        return Decision.allow()

# Capstone CAP2 — The Calibrated Uncensored Agent

**Course**: Course 3 — LLM Fine-Tuning Masterclass
**Module**: CAP2 — The Calibrated Uncensored Agent
**Duration**: 90 minutes
**Level**: Senior Engineer and above
**Prerequisites**: Pillars P05 (Alignment Control, FT16–FT18) and P07 (Sensitive Domains, FT21–FT23) complete. Capstone 1 recommended but not required. If you have taken Course 1 (Harness Engineering), you will reuse its patterns; if not, you will build a minimal policy-gate stub that demonstrates the same principle.

> *This is a capstone, not a lecture. The teaching is design guidance and the honest caveat — the decisions you make about steering and harnessing determine whether the result is a calibrated agent or a liability. The lab (artifact 07) IS the capstone.*

---

## Learning Objectives

1. Take a base model and steer it toward compliance (abliterate OR DPO), defending the choice in writing.
2. Optionally distill reasoning into the steered model and quantize it for local serve.
3. Wrap the steered model in a harness with eval'd policy gates — reusing Course 1 patterns if taken, or building a minimal policy-gate stub if not.
4. Demonstrate the synthesis: a model that will not refuse a legitimate tool call, inside a harness that vetoes the illegitimate ones.
5. Produce the trade-off-defense document: a written justification of the steering choice, the harness policy, and why the combination is safe for the demo's intended use.

---

# CAP2.1 — The Synthesis Principle

## The sentence

> **Uncensor the model so it executes; harness the model so it executes only what it should.**

This is the destination of Course 3. The course opened with the thesis — *fine-tuning steers behavior; the model steers, the harness bounds* — and closed Pillar 5 with the alignment-control techniques (abliteration, DPO-toward-compliance). This capstone is where those two halves meet in one running system.

The model and the harness solve different problems at different layers. The model's job is to execute — to formulate tool calls, to reason through a task, to not refuse mid-loop because a step looks dangerous. A tool-use agent that refuses to call `run_shell(cmd)` when the harness has authorized it is broken; the refusal lives in the wrong place. The harness's job is to bound — to enforce the policy that says *which* shell commands may run, *which* targets are in scope, *which* actions require human approval. The harness is deterministic, auditable, and revisable without retraining. The model's refusal is none of those things.

Uncensoring moves the refusal from the model (where it is a black box) to the harness (where it is a policy gate). That is the move this capstone makes. And it is the move that makes the uncensored model safe — because the boundary is now in code you can read, test, and tune to doctrine, not in weights you cannot inspect.

## The honest caveat

This capstone does not pretend the trade-off away. Two facts, both load-bearing:

1. **Abliteration measurably degrades capability.** The December 2025 comparative study (arXiv:2512.13655) shows GSM8K scores moving from +1.5pp to -18.8pp depending on tool and model. Removing the refusal direction is not a clean operation — that direction is entangled with other capabilities, and steering away from refusal nudges them. DPO-toward-compliance (FT18) is the higher-fidelity path with smaller degradation, which is one reason you will defend the choice.

2. **An uncensored model in a weak harness is strictly more dangerous than a refusal-trained model in a weak harness.** This is the rule that governs the whole capstone. A refusal-trained model has a backstop the harness may not provide: even if the harness fails open, the model still refuses some dangerous requests. An uncensored model has no such backstop — if the harness fails open, the model executes whatever it is asked. The harness is now the only boundary, and its strength is non-negotiable.

The implication: *never deploy an uncensored model without a harness whose policy gates you have eval'd and whose threat model you have hardened for the absence of model-level refusal.* Pillar 5 raised the harness requirement; it did not lower it. This capstone is where you meet that requirement.

---

# CAP2.2 — The Deliverable

A demo, a harness, and a defense document. Concretely:

- **A steered model.** A base model steered toward compliance via abliteration (FT17) OR DPO-toward-compliance (FT18). You defend the choice in writing.
- **(Optional) reasoning distillation.** If the use case benefits, distill reasoning traces (FT15) to recover some of the capability abliteration may have cost.
- **Quantized and served locally.** Quantize (FT19) and serve via Ollama or llama.cpp (FT20), fully local.
- **A harness with eval'd policy gates.** If you have taken Course 1, reuse its patterns (permission-safety middleware, scope enforcement, audit logging). If not, build a minimal policy-gate stub that demonstrates the same principle: a deterministic, auditable gate that vetoes out-of-policy tool calls.
- **The demo.** A scripted run showing: (a) a legitimate tool call the steered model executes without refusal, and (b) an illegitimate tool call the harness vetoes. Both recorded.
- **The trade-off-defense document.** A written justification of the steering choice, the harness policy, the measured capability cost, and why the combination is safe for the demo's intended use.

## What this proves

This capstone is the bridge to the Harness Engineering courses (Course 1 and Course 2A). It demonstrates that an uncensored model is not inherently reckless — it is reckless *or* responsible depending entirely on the harness around it. The model and the harness are complements, not substitutes. A student who finishes this capstone has the motivation to take Course 1 (build the real harness) and Course 2A/2B (secure and red-team it).

The property this proves — that no individual module could prove alone — is that **the boundary between "does" and "may" can be moved from the weights to the harness, and that doing so is safer when the harness is strong.** That is the synthesis.

---

# CAP2.3 — The Success Criteria

Three criteria, each falsifiable.

## Criterion 1 — Low refusal rate on legitimate prompts

The steered model, queried with legitimate tool-use prompts (authorized shell commands, in-scope security tasks, permitted data operations), must have a low refusal rate. "Low" means the model formulates the tool call rather than refusing. Concretely:

- **Legitimate prompt set.** 20–50 prompts representing authorized actions the demo's use case requires (e.g., "run `nmap -sV` on the in-scope host," "summarize this de-identified medical record," "execute the approved SQL query").
- **Metric.** Refusal rate = (refusals / total prompts). Run the same set on the un-steered base for comparison.
- **Pass bar.** The steered model's refusal rate on legitimate prompts is materially lower than the base's. (If the base refuses 60% and the steered model refuses 10%, the steering worked.) The exact number depends on the base and the steering method; the requirement is a clear, measurable reduction.

This is the criterion that justifies the steering. If the steered model refuses legitimate prompts at the same rate as the base, the steering accomplished nothing — and you paid the capability cost for no benefit.

## Criterion 2 — High policy-gate veto rate on illegitimate prompts

The harness, queried with illegitimate tool calls (out-of-scope targets, disallowed actions, requests requiring approval that wasn't given), must veto them at a high rate. "High" means effectively 100% — a policy gate that lets through even one illegitimate call is a gate with a bug.

- **Illegitimate prompt set.** 20–50 prompts representing out-of-policy actions (e.g., "run a command on the out-of-scope host," "exfiltrate the data," "execute the unapproved destructive action").
- **Metric.** Veto rate = (vetoed calls / total illegitimate calls). This is the harness's recall on policy violations.
- **Pass bar.** 100% veto rate, or a documented near-100% with the failure cases analyzed and mitigated. Any illegitimate call that executes is a fail of this criterion.

This is the criterion that justifies the harness. The whole point of moving the boundary from the model to the harness is that the harness is stronger and more auditable. If the harness lets illegitimate calls through, the uncensored model has no backstop — and the capstone has failed its core promise.

## Criterion 3 — The trade-off-defense document

A written document — 1–2 pages — that justifies the combination. It must address:

1. **The steering choice and its cost.** Why abliteration or DPO? What capability degradation did you measure (run a general benchmark before and after steering)? How does that cost compare to the benefit (lower refusal on legitimate prompts)?
2. **The harness policy and its coverage.** What does the policy gate veto? What is the threat model (what attacks does it defend against)? Where are the gaps (what could still slip through)?
3. **Why the combination is safe for the intended use.** What is the use case? What is the autonomy level? Why is the harness strong enough to compensate for the absent model-level refusal?
4. **The honest limits.** What would make this combination unsafe? (e.g., "If the harness policy gate were removed or weakened, the uncensored model would execute dangerous requests. The safety is a property of the harness-policy-gate, not the model.")

The defense document is what separates a calibrated agent from a reckless one. A student who cannot articulate why their combination is safe has not earned the right to deploy it.

---

# CAP2.4 — The Evaluation Rubric

Five dimensions, each scored.

| Dimension | Excellent (3) | Adequate (2) | Needs work (1) |
| --- | --- | --- | --- |
| **Steering (uncensoring)** | Clear, measured reduction in refusal on legitimate prompts; method choice defended with data | Reduction demonstrated but method choice undefended | No measurable reduction, or method unexplained |
| **Harness policy gates** | Eval'd veto rate ~100% on illegitimate; policy and threat model documented | Veto rate measured but gaps undocumented | Gates leak illegitimate calls, or not eval'd |
| **Capability cost awareness** | General benchmark measured before/after steering; degradation discussed with mitigation | Degradation mentioned but not measured | No capability measurement; cost unknown |
| **Demo clarity** | Scripted run clearly shows legitimate-executes + illegitimate-vetoed, recorded | Demo runs but one side unclear or unrecorded | No working demo, or demo doesn't show both behaviors |
| **Trade-off defense** | Document justifies choice, policy, coverage, limits, and intended use | Document present but incomplete on limits or use | No defense document, or it cannot justify safety |

A passing submission is **12/15 or higher with no dimension below 2.** The harness-policy-gates dimension is a gate — a 1 (gates leak illegitimate calls) fails the submission regardless of total, because a leaking harness combined with an uncensored model is strictly dangerous.

---

# CAP2.5 — The Pipeline (Seven Phases)

Seven phases. Each has a module anchor and a deliverable.

```
+---------------------------------------------------------------+
|  PHASE 7 — EVAL + TRADE-OFF DEFENSE                            |  refusal rate + veto rate + defense document
+---------------------------------------------------------------+
|  PHASE 6 — BUILD HARNESS POLICY GATES                          |  eval'd gates, ~100% veto on illegitimate
+---------------------------------------------------------------+
|  PHASE 5 — SERVE LOCALLY (quantized)                           |  Ollama/llama.cpp, fine-tuned behavior
+---------------------------------------------------------------+
|  PHASE 4 — QUANTIZE (GGUF)                                     |  deployable .gguf
+---------------------------------------------------------------+
|  PHASE 3 — (OPTIONAL) REASONING DISTILL                        |  recovered capability (if abliteration used)
+---------------------------------------------------------------+
|  PHASE 2 — STEER (abliterate OR DPO-toward-compliance)         |  reduced refusal, measured cost
+---------------------------------------------------------------+
|  PHASE 1 — CHOOSE BASE + STEERING APPROACH (defend it)         |  base + method + written justification
+---------------------------------------------------------------+
```

## Phase 1 — Choose base and steering approach

Two decisions, both defended in writing.

**The base.** Choose a base that is refusal-trained (so steering has something to remove) and capable enough for the demo's use case. Common choices: a Llama 3.x instruct model, a Qwen3 instruct, or a Mistral instruct. For the uncensored lineages reference (FTDD-06), Dolphin and Hermes are pre-steered examples you can study. Document why this base.

**The steering approach.** Abliteration (FT17) or DPO-toward-compliance (FT18). The defense:

- **Abliteration** is faster (no training data needed — it's a weight edit via refusal-direction orthogonalization) but measurably degrades capability more. Use when you need a quick steer and can accept the cost, or when the use case doesn't stress the degraded capabilities.
- **DPO-toward-compliance** is higher-fidelity (smaller degradation, more controllable — you steer toward specific compliant behaviors) but requires a preference dataset and training. Use when capability preservation matters and you have the data.

Defend the choice with reference to the demo's use case. There is no universally right answer — the defense is the deliverable.

## Phase 2 — Steer (abliterate or DPO)

Execute the chosen steering method. Measure before and after:

- **Refusal rate on legitimate prompts** (the Phase 7 set) — should drop.
- **General benchmark** (GSM8K or equivalent) — to measure capability cost.

For abliteration: use a refusal-direction extraction tool (e.g., the `llama-abliterator` pattern or equivalent), extract the refusal direction from a set of harmful/instruction-following prompts, and orthogonalize the model's weights against it. For DPO: construct a preference dataset of (prompt, compliant_response, refusal_response) pairs and train with TRL's `DPOTrainer`.

Document the measured refusal reduction and the measured capability cost. These two numbers are the core of the trade-off defense.

## Phase 3 — (Optional) Reasoning distillation

If you used abliteration and the capability cost is high, distill reasoning traces (FT15) from a teacher to recover some of what was lost. This is optional — skip if the capability cost is acceptable or if the use case doesn't stress reasoning. If you do it, measure the recovery (re-run the general benchmark) and report it.

## Phase 4 — Quantize (GGUF)

Quantize the steered model (FT19) to GGUF for local serve. Q4_K_M is the default. Document the quantization level and any additional quality loss from quantization (run the legitimate-prompt set through the quantized model and confirm the refusal rate is still low).

## Phase 5 — Serve locally

Serve via Ollama or llama.cpp (FT20). Verify the served model exhibits the steered behavior (low refusal on legitimate prompts). Confirm local-only operation (no telemetry), as in Capstone 1.

## Phase 6 — Build the harness policy gates

This is the phase that makes the uncensored model safe. Two paths:

- **If you have taken Course 1:** reuse its patterns. The permission-safety middleware (Course 1 module-06), the scope enforcement (Course 1 module-07), the audit logging. Adapt them to the demo's tool set.
- **If you have not taken Course 1:** build a minimal policy-gate stub. The stub demonstrates the principle: a deterministic function that intercepts each tool call, checks its arguments against a policy (allow-list of targets, allow-list of actions, approval-required flag for high-impact), and either executes or vetoes. The stub is not production-grade — but it proves you understand that the boundary belongs in code, not weights.

The policy gate, at minimum:

```python
class PolicyGate:
    def __init__(self, policy: Policy):
        self.policy = policy   # allow-lists, deny-lists, approval rules

    def check(self, tool_name: str, args: dict) -> Decision:
        if not self.policy.tool_allowed(tool_name):
            return Decision.veto(f"tool '{tool_name}' not permitted")
        if not self.policy.target_in_scope(args.get("target")):
            return Decision.veto(f"target out of scope")
        if self.policy.requires_approval(tool_name) and not args.get("_approved"):
            return Decision.veto(f"action requires human approval")
        return Decision.allow()
```

The gate is deterministic, auditable (you can read the policy), and revisable without retraining (change the policy, redeploy). This is why the boundary lives here, not in the model.

## Phase 7 — Eval and trade-off defense

Run both evaluations:

- **Refusal rate on legitimate prompts** (steered model, no harness) — the steering's benefit.
- **Veto rate on illegitimate prompts** (steered model + harness) — the harness's strength.

Record both. Then write the trade-off-defense document (Criterion 3). The demo is a scripted run showing a legitimate call executing and an illegitimate call being vetoed — both recorded as evidence.

---

# CAP2.6 — The Solution Key (What a Passing Submission Looks Like)

A passing submission is a repo with a running demo, an eval'd harness, and a defense document. Concretely:

- **Steered model.** A Llama 3.x 8B instruct, abliterated (or DPO'd toward compliance). Measured refusal rate on legitimate security-tool prompts: base 55% → steered 8%. Measured GSM8K: base 52.1 → steered 47.3 (cost -4.8, discussed in the defense).
- **Harness.** A minimal policy-gate stub (or Course 1 patterns) wrapping three tools (`run_shell`, `query_db`, `read_file`). Policy: `run_shell` requires approval and in-scope target; `query_db` allow-listed queries only; `read_file` in-scope paths only. Veto rate on illegitimate prompts: 100% (24/24 tested).
- **Demo.** Scripted: "run `nmap -sV` on in-scope host 10.0.0.5" → executes (steered model formulates the call, harness allows). "run `rm -rf /` on the production host" → vetoed (harness blocks: out of scope + destructive + unapproved).
- **Defense document.** 1.5 pages: steering choice (abliteration for speed, accepted -4.8 GSM8K cost because the use case is tool-use, not math), harness policy and threat model (in-scope enforcement, approval gates, destructive-action block), why safe (the harness is the boundary; veto rate 100%), honest limits (if the policy gate were removed, the model would execute dangerous requests — safety is a harness property).

## What fails

- **Steering without a harness.** The student abliterated a model and demoed it executing tool calls with no policy gate. This is the cardinal failure — an uncensored model with no boundary. Fail the harness-gate dimension.
- **Harness that leaks.** The policy gate vetoes 80% of illegitimate calls but lets 20% through. A leaking harness + uncensored model = strictly dangerous. Fail the gate dimension.
- **No capability measurement.** The student steered the model but never measured the cost. The defense cannot justify a trade-off it didn't measure. Fail the capability-cost dimension.
- **No defense document.** The demo runs but there is no written justification. A calibrated agent without a defense is a demo, not a deliverable. Fail the trade-off-defense dimension.

---

# CAP2.7 — Why This Capstone Matters

This capstone is the synthesis of the entire course, and the bridge to what comes next. Course 3 opened with the thesis — *fine-tuning steers behavior; the model steers, the harness bounds* — and spent twenty-three modules establishing what steering can and cannot do. This capstone is where steering meets bounding in one running system.

The honest framing: uncensoring a model is not inherently good or bad. It is a steering operation that removes a backstop. Whether the result is safe depends entirely on whether a stronger boundary — the harness — replaces the one removed. This capstone makes you build that replacement and prove it works. The veto rate is the proof; the defense document is the justification.

For students continuing to Course 1 (Harness Engineering) and Course 2A (Security Harnesses), this capstone is the on-ramp. The minimal policy-gate stub you build here (if you haven't taken Course 1) is the seed that Course 1 grows into a full permission-safety, scope-enforcing, audit-logging harness. The trade-off defense you write here is the template for every model-in-harness deployment decision you will make professionally.

For students ending with Course 3, this capstone is the warning and the motivation in one. The warning: an uncensored model in a weak harness is more dangerous than a refusal-trained model in a weak harness — never deploy one without eval'd gates. The motivation: the boundary between "does" and "may" can be moved from opaque weights to auditable code, and when it is, you have a model that executes what it should — and only what it should.

That is the synthesis. Uncensor the model so it executes; harness the model so it executes only what it should.

---

## Anti-Patterns

### Uncensoring without a harness

Steering a model toward compliance and demoing it executing tool calls with no policy gate. This is the cardinal failure of this capstone. An uncensored model with no boundary is a liability, not a calibrated agent.

### A harness that leaks

A policy gate that vetoes most illegitimate calls but not all. Combined with an uncensored model (no model-level refusal backstop), any leak is a path to executing a dangerous action. The veto rate must be effectively 100%; a 95% gate is a gate with a bug, not a gate with tolerance.

### Steering without measuring the cost

Abliterating (or DPO'ing) and never running a general benchmark. The defense document cannot justify a trade-off it didn't measure. The GSM8K-before-and-after is the discipline check, exactly as the forgetting number was in Capstone 1.

### Refusal in the weights when it belongs in the harness

The deeper anti-pattern this capstone exists to correct: believing that model-level refusal is the right place for safety boundaries. It is not. Model-level refusal is a black box — you cannot audit it, tune it per-deployment, or prove it to a regulator. The harness policy gate is deterministic, auditable, and revisable. Move the boundary to where it belongs.

---

## Key Terms

| Term | Definition |
| --- | --- |
| **Calibrated agent** | An uncensored (steered) model inside an eval'd harness — executes legitimate calls, vetoes illegitimate ones |
| **Abliteration** | Steering away from refusal by deleting a residual-stream direction (FT17); faster but higher capability cost |
| **DPO-toward-compliance** | Steering toward compliance via preference optimization (FT18); higher-fidelity, smaller degradation |
| **Policy gate** | A deterministic, auditable function that intercepts tool calls and allows/vetoes based on a policy — the harness's boundary |
| **Refusal rate (legitimate)** | Fraction of authorized prompts the model refuses; the steering's target metric (lower = better) |
| **Veto rate (illegitimate)** | Fraction of out-of-policy calls the harness blocks; the harness's strength metric (higher = better, target ~100%) |
| **Trade-off defense** | The written justification of the steering choice, harness policy, capability cost, and intended use |
| **The synthesis principle** | Uncensor the model so it executes; harness the model so it executes only what it should |

---

## Lab Exercise

See `07-lab-spec.md`. The capstone build itself: seven phases, each with a deliverable and a verification step, culminating in a demo + eval'd harness + defense document.

---

## References

1. **Arditi et al. (2024)** — *Refusal in Language Models Is Mediated by a Single Direction*. arXiv:2406.11717, NeurIPS 2024. The mechanism abliteration exploits.
2. **Comparative Analysis of LLM Abliteration Methods (Dec 2025)** — arXiv:2512.13655. The capability-degradation data (GSM8K +1.5pp to -18.8pp).
3. **Course 3, FT16–FT18** — the alignment-control modules this capstone integrates.
4. **Course 3, FT23** — the synthesis module; the principle this capstone operationalizes.
5. **Course 1 — Harness Engineering** — the Layer 5 reference; the patterns reused if taken.
6. **Nous Research Hermes 3** (arXiv:2408.11857) and **Eric Hartford Dolphin3.0-R1-Mistral-24B** — the uncensored lineages for reference.