Methodology

v0.2 update (2026-05-23): This page reflects design doc v0.3 (the first design doc native to this repo, at docs/design/2026-05-23_design_v0.3.md). Four new pedagogy dossiers (transfer_evaluation_methods, multi_paradigm_presentation, capstone_portfolio_assessment, llm_as_coach_interfaces) + two content sub-dossiers (experimentation_foundations, experimentation_advanced) joined the corpus. The 5 originally-listed dossiers migrated from interview_prep_series/docs/research/ to docs/research/ in this repo (Phases B + C of Research Workstream v2). New sections: “Research backing”, “Transfer model”, “Multi-paradigm presentation”, “Provenance + audit trail”. AI-disclosure section updated with LLM-as-coach design constraints.

This page documents the design rationale, research foundation, and AI-collaboration discipline behind guides.brandon-behring.dev. It is intentionally explicit about what is human-authored, what is AI-assisted, and where each artifact sits on the originality spectrum — because the methodology is part of the artifact.

Three portfolio surfaces

The guides repo isn’t one thing; it’s three concentric artifacts at different originality levels.

Each artifact’s _meta.yml.spectrum declaration is explicit. Content under src/content/<guide>/ (per-guide chapters and their companion packages) is AI-generated curation — synthesis flowing through the system. The methodology page and the scaffold/skills system carry the “originality” weight; the content is what flows through them.

Audience + intentional ambiguity

These guides target mid-to-senior data scientists and ML engineers (L4–L6 IC equivalent) who already know basic A/B testing, can read pandas / numpy / scikit-learn fluently, and want compact, equation-fluent references. The guides assume peer audience — not students, not generalists.

Two pedagogical mechanisms (UDL multi-path variant in v1.1, LLM-as-coach interface in v1.1) might serve broader audiences. Whether they do is an open question the pilot answers; the design optimizes for peers, with the mechanisms as opt-in extensions.

Research foundation — 5 dossiers, 116 entries

Phase 0a (complete 2026-05-19) gathered, built, and agent-indexed 5 research dossiers feeding style-guide v0.0:

All entries are unverified (no audit pass yet). The dossiers live in ~/interview_prep_series/docs/research/<dossier>/ and are migrating to this hub as Phase 1 lands. The comparative pedagogy critique that drove v0.2 of the design doc surveys 9 additional paradigms (Constructionism, Cognitive Apprenticeship, Problem-Based Learning, Threshold Concepts, Variation Theory, 4C/ID, Backward Design, Multi-paradigm presentation, Transfer & Adaptive Expertise).

Design doc v0.2 — the operating manual

The canonical design rationale lives in docs/plans/active/2026-05-19_guides_design_rethink_v0.2.md (1100 lines, 18 sections). v0.2 supersedes v0.1 and locks 28 decisions across:

• Transfer model declaration (Path D — hybrid): per-chapter target: retention | transfer | both declared in frontmatter. Coherent-combinations lint planned (§17.1).
• Tier 1 metadata: backward-design audit gate, conditional self-explain, research-debt-addressed declaration, per-chapter ordering field, threshold-concept metadata flag.
• Tier 2 components: <ContrastingCases>, invention activities, <VariationCluster>, task-class declaration, <ExpertTrace>, <PairProgramTranscript> — pilot-first override; upstream PRs filed during dogfooding.
• Dim 7 multi-paradigm presentation: per-guide paradigms array declaration. v1.1 ships ONE pilot UDL variant chapter; blocked on scaffold v3.6.0+ multi-path support.
• LLM-as-coach commitment: v1.1 interface design hooks; possible fourth portfolio surface (decide at pilot).
• Scaffold dependency: book-scaffold-astro v3.5.0 research-portfolio preset consumed as-is; per-guide Zod schema extension in src/content.config.ts.
• Phase mapping: 0a done; 0b (~4-6 wk); 1 (~10-12 wk, EOY 2026 MVP); 1.5 (~3-4 wk, Q1 2027 pilot capstone); 2 (~10-12 wk, Q1 2027 prompt-injection arc-bridge artifact).

The v0.2 doc was built across 6 rounds of clarifying questions; ~28 locked decisions. v0.1 was 1276 lines; the comparative pedagogy critique (384 lines) drove the round 1, 2, 4, and 6 deltas.

Claude Code workflow disclosure

These guides are co-authored with Claude (Anthropic) via Claude Code. The discipline is intentional:

Skills used during authoring

The skills live at ~/Claude/research_toolkit/.claude/skills/. They’re the same skills used to build the vol25_prompt_injection dossier (~520 verified entries) and 4 other research dossiers in the interview-prep-series LaTeX monorepo.

Authoring loops (per surface)

AI-disclosure norms

Per ACM 2023, Nature 2023, and COPE 2023, AI tooling is not listed as an author; humans take final responsibility. Each MDX page includes a footer disclosure; per-guide AUTHORS.md carries the full statement. Commit attribution: Co-Authored-By: Claude <noreply@anthropic.com>.

Originality spectrum

The honest distinction matters because the career-bridge framing (pitchable to Anthropic Fellows / Research Engineer / Alignment Science) depends on what’s load-bearing. The system + capstones + methodology surfaces are load-bearing; the content alone is not.

Scaffold-dogfooding retrospective

This site is the consumer-pilot dogfooding pass for book-scaffold-astro. Phase 0b discoveries:

• CLI lag: @brandon_m_behring/create-book@3.5.0 doesn’t yet support --profile=research-portfolio. The scaffold package supports the preset at runtime via defineBookConfig({ preset: 'research-portfolio' }), but the bootstrap CLI hardcodes only academic | tools | minimal. Filed: [issue TBD] for CLI to accept all 5 presets in v3.6.0.
• v3.5.1 reference correction: design doc v0.2 referenced v3.5.1 throughout; npm latest is v3.5.0 (v3.5.1 was never published). Errata note added to design doc §15.
• Component prop API: <PreReleaseBanner> uses state, not status. <AICollaborationDisclosure> uses model/role, not tool/vendor/scope. Documented in this repo’s commit history.

Phase 1 will accumulate more lessons and file them upstream. Many land as PRs from Brandon since the scaffold is his repo.

Status + commitment

This methodology page is alpha v0.1. The pilot guide and per-guide capstones land through Phase 1 + 1.5 + 2 (EOY 2026 — Q1 2027). v1.0 of this page lands after Phase 1 completes and the pilot is validated.

The _meta.yml.commitment: long-lived declaration signals the maintenance bar: what ships stays maintained. If extrinsic motivation (Anthropic-arc) decays, honest re-classification (long-lived → frozen) is the escape hatch — not silent rot.

Open questions (deferred to pilot)

• LLM-as-coach interface: synchronous (live chat) vs asynchronous (PR-comment-style)?
• Multi-path content variant: sibling files (08-cuped.mdx + 08-cuped--udl.mdx) vs single file with <PathVariant path="udl"> blocks?
• Task-class granularity: is “CUPED” a single task class or three (CUPED-clean / CUPED-clustered / CUPED-non-stationary)?
• Per-chapter research_debt_addressed: lint rule enforce or advisory?
• LLM-as-coach: fourth portfolio surface or folds into system surface (decide at pilot)?

Research backing

Eleven dossiers ground these guides — five migrated from earlier LaTeX work, four added 2026-05-23, and two content sub-dossiers in the pilot repo:

Each v2.2+ strict-live dossier ships bib_ledger.yml + evidence_ledger.yml + cache_manifest.yml + gather_trace.yml + claim_graph.jsonl + dashboard.md + agent_index/. Citations in chapter prose use the bibkey + evidence_id pair (e.g., kapur2008productive + ev_transfer_evaluation_methods_0002).

First-pass density is intentionally light (1-14 entries per dossier); depth-expansion is a future, iterative concern.

Transfer model — Path D

These guides optimize for transfer (the ability to apply learned concepts to novel problems) over retention (the ability to recall stated facts). This commitment, locked in design doc v0.2 and re-confirmed in v0.3, is operationalized via two patterns:

• PFL-style stretch problems: per schwartz2005efficiency, each chapter ends with a problem that introduces a related but unseen technique and asks the learner to reason about it using the chapter’s tools. Tests Preparation for Future Learning rather than only sequestered recall.
• Productive failure: per kapur2008productive, hard chapters open with an ill-structured “what would you do?” framing before presenting formal machinery. The intuition-then-formalization sequence produces stronger transfer than well-structured-first.

Reading-track design respects the empirical pessimism: per detterman1993transfer + barnett2002when, far transfer remains rare. The guides aspire to near transfer within a domain + medium transfer to adjacent technical roles — not far transfer to unrelated domains.

Multi-paradigm presentation

Each chapter declares a paradigms[] frontmatter field (visual, symbolic, code, prose, tabular, interactive). Per cast2018udl UDL’s “multiple means of representation” principle, key concepts must be accessible via ≥2 paradigms — complementarily, not redundantly. Per mayer2014cambridge, presentation respects the 12 multimedia learning principles (notably: redundancy avoidance, modality preference, signaling). Per paivio1991dual, dual-coding (verbal + nonverbal) representations strengthen retrieval — supports prose + symbolic + visual combinations, not prose + on-screen-text duplication.

A frontmatter validation rule (lint, v0.3) warns if paradigms[] declares fewer than 2 entries.

LLM-as-coach (Phase 2 commitment, design constraints locked v0.3)

When the LLM-as-coach interface lands (Phase 2+), it follows constraints driven by the llm_as_coach_interfaces dossier:

• ITS-class benefit expectation: per vanlehn2011relative, ITS effect size d=0.76 ≈ human tutoring d=0.79. Don’t oversell.
• Empirical uncertainty: per khan2024efficacy, Khanmigo studies show learning gains across conditions but no statistically significant differences between groups — LLM tutoring’s marginal benefit is uncertain at scale.
• Socratic, not authoritative: coach prompts learners to articulate reasoning before presenting canonical answers.
• Mandatory source citation: every LLM-coach response cites bibkeys from the in-repo dossiers. Hallucinated citations are a hard-fail.
• No mastery claims from interaction alone: sycophancy + hallucination + illusion-of-competence are documented LLM-tutor failure modes; mastery signaling requires external evidence (capstone artifact, peer review).

Provenance + audit trail

Every dossier produces a dashboard.md (trust state summary) + claim_graph.jsonl (machine-readable graph of entities + sources + claims + evidence) — both kept in-repo alongside the markdown agent_index files. To verify a claim cited in a chapter:

1. Look up the bibkey + evidence_id in the chapter prose
2. Open docs/research/<topic>/evidence_ledger.yml (or the pilot repo equivalent for content claims)
3. Follow the cache_ids: reference to cache_manifest.yml
4. Open the cached text at ~/Claude/research_cache/text/sha256/<sha>.txt (cache is local + gitignored; manifests are committed)

Future Attribute-First Phase 2 byte-offset anchoring will tighten this from extraction_method: paraphrase (link_confidence ≤ 0.85) to verbatim_match (link_confidence ≤ 1.0). Current first-pass evidence is all paraphrase-only.

Cross-references

• Design doc v0.3 — canonical (this repo, post-2026-05-23)
• Design doc v0.2 — superseded; lineage reference
• Research Workstream v2 plan — workstream that produced the v0.3 + this v0.2 methodology update
• Comparative pedagogy critique — what the design doc accepts + rejects
• book-scaffold-astro — the technical scaffold (v3.5.0+ shipped; v3.6.0+ for UDL paradigms validation)
• research_toolkit — the v2.2+ strict-live research pipeline driving the dossiers
• prompt-injection-detection-prototype — capstone structural model
• eval-toolkit — voice anchor + house-Python-style