Full Audit Report

An evidence-grounded audit of the Peeragogy Handbook across 88 nodes, using a multi-agent pipeline.

Context

Most social knowledge in the peer-production literature exists as theory without systematic feedback.

The Peeragogy Handbook codifies patterns for peer learning — but like almost all social knowledge, those patterns were written by people who had time to write them, tested by people who had time to test them, and generalised far beyond the context where they were first observed. No systematic loop brings operational reality back into the theoretical corpus.

UnPeeragogy was built to close that loop. The audit is its first full-scale application.

📝 Background reading: The story of how the audit pipeline was designed and the context that made it necessary is told across two posts on the Pyragogy blog — "The Empty Laboratory" (Part I) and "The Empty Laboratory — Part II" (Part II), in the Laboratory category.

The Pipeline

The audit was executed by a multi-agent orchestration pipeline designed around a strict principle:

The model may propose. The evidence must dispose.

Each of the 88 nodes of the Peeragogy Handbook was subjected to a pipeline with five specialised functions:

1. Interpretation

Parses the theoretical claim of each pattern. What is it asserting? What conditions does it assume? What is left unspecified?

2. Evidence Hunter

Uses parametric knowledge to surface candidate evidence — known cases, research, phenomena — that may be relevant to the pattern's claim.

3. Grounding Gate

Critical check: can this candidate actually be connected to a real source, an empirical case, or a documented observation? If not, it is rejected — regardless of how plausible the model made it sound.

4. Critical Analysis

Assesses what the grounded evidence means for the pattern's claim. Does it support, complicate, contradict, or leave the claim ambiguous?

5. Tension Computation

Computes a raw tension delta between the theoretical claim and the evidence set. The tension is not a judgment of validity — it is a measure of observable friction.

Every step is recorded. Every candidate has an identifier. Every source has a provenance trail. The full run produces an audit trail and a reproducibility manifest.

Results in brief

Mean raw tension delta: +0.027 — the resultant of the current evidence set. The tension is real but not catastrophic. Some patterns hold under specific conditions; others show clear gaps between the theoretical claim and operational reality.

All 88 nodes produced structured vault actions with both positive and negative operational rules. No pattern was dismissed as "simply wrong". Every pattern received at least one supporting observation and at least one complicating observation — which is exactly what a useful stress test should produce.

The full report

The complete PDF contains the pipeline methodology, the detailed results for each of the 88 nodes, and a section of conjectures for future research. It is the same document archived on Zenodo under a CC Zero dedication.

What this means for UnPeeragogy

The audit is not a conclusion. It is a starting point.

The tension deltas tell us where to look, not what to conclude. A pattern with near-zero tension may not be "true" — it may simply lack sufficient evidence to challenge it. A pattern with high tension may not be "false" — it may operate well under conditions the corpus has not yet captured positively.

What the audit does provide is a structured, reproducible, evidence-anchored map of where the Peeragogy corpus holds, where it frays, and where it would benefit from closer observation. The 52 unique entities and 50 distinct source URLs give the map a grounding that unaided intuition cannot.

And — critically — every tension value in the audit can be revised when new evidence arrives. The loop is not closed after a single run.

Reproducibility

The audit was executed with a deterministic pipeline. The same inputs, run through the same orchestration, should produce the same outputs — within the limits of LLM variance. A reproducibility manifest accompanies the report, documenting every parameter, every prompt template, and every model configuration used in the run.

The pipeline is open-source and designed to be applicable to other corpora. Its generalizability to domains beyond peeragogy remains to be tested — but the architecture is domain-agnostic by design.