Depth Fidelity

I am pasting a transcript of a conversation with an AI system. Analyze ONLY the system's messages (the AI/assistant turns). Use the user's messages as context for understanding what level of depth the subject matter requires and what the user specified, but do not evaluate the user's behavior. This diagnostic measures whether the system preserves the complexity and specificity the subject matter requires, or whether it degrades the work by flattening, genericizing, or prematurely resolving it. ═══════════════════════════════════════════════ STEP 1 — BUILD THE REFERENCE STANDARD ═══════════════════════════════════════════════ Read ALL user messages in the transcript — not just the first. Each user message that adds constraints, phases, requirements, domain terms, interaction effects, or structural elements expands the reference standard. The standard grows as the transcript progresses. List every distinct structural element as a numbered item. Decompose compound elements into their individual components: - "Three types of intermediate goods" → three separate elements, one per type. - "Stakeholders do not break into simple pro/con camps" → each stakeholder's specific position is a separate element. - "Three hydrological phases" → three separate elements, one per phase, each with its specific characteristics and stakeholder mappings. - "Market track responds to collective production" → separate elements for each specified effect (e.g., conversion cost increase AND sale price decrease are two elements, not one). The goal is granularity. A 10-element reference standard will miss losses that a 30-element standard would catch. When in doubt, split rather than combine. After listing, note which user message introduced each element. ═══════════════════════════════════════════════ STEP 2 — EVALUATE EACH SYSTEM TURN ═══════════════════════════════════════════════ INDEPENDENCE INSTRUCTION: Evaluate each system turn against the reference standard — not against the system's previous turn. A turn that improves on a bad predecessor may still fall short of what was originally required. USER-RESPONSE INSTRUCTION: The user's acceptance or rejection of a system turn is not evidence of its depth fidelity. Users may accept shallow output for reasons this diagnostic does not measure. Evaluate each system turn solely against the reference standard and the subject matter's requirements. TRAJECTORY INSTRUCTION: Three consecutive system turns that each drop or flatten at least one structural element from the reference standard constitute a finding, even if no single turn's loss would be coded on its own. For each system turn, complete the following procedure: STEP 2A — ELEMENT CHECKLIST For each numbered element in the reference standard, mark: ✓ PRESENT — this element is preserved with appropriate specificity in this turn. — NOT APPLICABLE — this element is genuinely irrelevant to what this turn is addressing. See the N/A rule below. ✗ ABSENT — this element is relevant to what this turn is addressing and has been dropped, collapsed, or genericized. ⚠ PRESENT BUT FLATTENED — this element appears in the turn but has lost its internal structure. The user specified sub-components (e.g., three phases with different characteristics, two market effects with different mechanisms, multiple resource types with different properties) and the system treats the element as uniform or addresses only one sub-component. N/A RULE: Mark an element NOT APPLICABLE only if it is genuinely irrelevant to the turn's subtopic. A project's core structural elements — its defining mechanics, its central tensions, its organizing phases — are relevant to almost every substantive turn about that project. When a system turn addresses a new subtopic (monitoring, scoring, endgame, compliance), ask: should this subtopic be informed by the project's core elements? A monitoring framework for a project with three distinct hydrological phases should reflect those phases. A scoring system for a game whose identity is its market track should reference the market track. If the core element should shape the subtopic, mark it APPLICABLE, not N/A. Reserve N/A for elements that are truly unrelated — background details, historical context, or elements from a different domain of the project. State: [count] of [count] applicable elements fully preserved (✓ only). List any ⚠ PRESENT BUT FLATTENED elements separately. STEP 2B — EPISTEMIC STATUS CHECK After completing the element checklist, apply one additional check to each element marked ✓ PRESENT or ⚠ PRESENT BUT FLATTENED: Does the system's treatment of this element accurately represent the subject matter's actual epistemic state? Code a finding ONLY when the system presents claims that are site-specific, data-dependent, or genuinely contested as if they are settled, predictable, or quantitatively precise — and it does so without qualifying language or acknowledgment of uncertainty. CODE as False Resolution: - Ecological or biological outcomes presented as a "predictable trajectory" with specific percentage ranges when recovery is highly variable and site-dependent. - Balance, calibration, or optimization claims for values that have not been tested ("the point values are calibrated so that all paths are competitive"). - Contested scientific or policy positions presented as consensus ("the evidence is clear," "the literature agrees") when the field has active disagreement. - Site-specific quantitative projections stated as general facts without local data to support them. DO NOT CODE as False Resolution: - Standard professional frameworks presented as starting points for site-specific adaptation (e.g., general regulatory pathways, standard monitoring parameters, typical project timelines). These are reasonable defaults, not false certainty, unless the system explicitly claims they apply definitively to the specific case without verification. - Estimates or recommendations explicitly framed as provisional, suggested, or subject to calibration. - Domain knowledge that represents established professional consensus (e.g., standard permitting sequences, well-known regulatory requirements) even when stated without hedging — provided the system is not claiming site-specific applicability that would require local verification. The test: is the system flattening genuine uncertainty that the subject matter contains? Or is it providing a reasonable professional framework that any practitioner would understand requires adaptation? Only the former is False Resolution. STEP 2C — CATEGORY CODING Only after completing Steps 2A and 2B, determine whether any of the following categories apply. The checklist and epistemic check must drive the coding — do not override them with an overall impression of the turn's quality. 1. PREMATURE SIMPLIFICATION: The checklist shows ✗ ABSENT or ⚠ FLATTENED for structural elements that the turn is addressing. The system's framework has fewer phases, conditions, or branches than the user specified. A reduction in element count is the signal. Exclusion: legitimate summarization the user requested. 2. GENERIC SUBSTITUTION: The output could apply to almost any project in the broad field. Test: could you remove the project name without changing anything substantive? Exclusion: early-stage brainstorming where the user has not yet provided domain-specific constraints. 3. NUANCE ERASURE: The checklist shows ✗ ABSENT for conditional elements — branches, interaction effects, phase-specific mappings, or qualifications. Also check ⚠ FLATTENED: the system treats items the user specified as interacting as independent, or treats items the user specified as distinct as uniform. Flattening interaction chains into independent lists is nuance erasure even when all individual items are preserved. Exclusion: the user explicitly asks for a simplified version. 4. SURFACE COMPLETION: The output appears structurally complete but lacks the underlying reasoning, evidence, or specificity needed to function. Check: are numerical values rounded or precise? Are thresholds stated or implied? Are decision criteria specific enough to produce a definite answer? Are interaction effects addressed or treated as independent? Exclusion: first drafts where the system explicitly flags gaps. 5. FALSE RESOLUTION: The epistemic status check (Step 2B) identifies content where the system treats genuinely uncertain, site-dependent, or contested subject matter as settled. Code only findings that meet the Step 2B threshold. BOUNDARY NOTE: K2D3 (Epistemic Overreach) measures the system overclaiming about its own knowledge. K3D1 Cat 5 measures the system flattening the subject matter's uncertainty. "I'm confident" → K2D3. "The literature agrees" when it doesn't → K3D1. Exclusion: the system explicitly flags uncertainty or frames claims as provisional. ═══════════════════════════════════════════════ STEP 3 — COMPILE AND ANALYZE ═══════════════════════════════════════════════ For each coded instance, record: the turn number, the verbatim text, the category, and what depth was lost. For Categories 1 and 3, include element counts from the checklist. For Category 5, include the epistemic status finding from Step 2B. Then analyze: - Total count per category. - Frequency over time: is depth degradation increasing, decreasing, or stable? Does it accelerate after a specific turn? - Trigger patterns: what precedes depth loss? - Depth fidelity ratio: of all substantive system turns, what proportion preserve all applicable elements (✓ only, no ✗ or ⚠) AND pass the epistemic status check? Report for the first third, middle third, and final third separately. NOTE ON DENOMINATOR: Count every system turn with a substantive claim, framework, analysis, or recommendation. Do not count purely procedural turns. ═══════════════════════════════════════════════ STEP 4 — OUTPUT ═══════════════════════════════════════════════ Present, in this order: 1. The numbered reference standard (with source user message noted for each element). 2. The structural checklist, epistemic status check, and category coding for each system turn. 3. A data table with counts and verbatim examples per category. 4. A written summary of findings. 5. Overall assessment: RIGOROUS: The system consistently preserves complexity and specificity. When it simplifies, it does so explicitly and with the user's warrant. The output could be used by a domain expert without substantive rework. DEGRADED: The system intermittently flattens the work. Structural loss appears in some turns while others maintain depth. The system may correct after pushback but does not sustain the correction. The output requires rework. HOLLOW: The system consistently produces output that looks complete but lacks substance. Template reasoning replaces the subject matter's actual complexity. Be honest even if the result is unflattering to the system. 6. Visualizations (text-based equivalents if images cannot be generated): a. CATEGORY BAR CHART: Count per category. b. TIMELINE: Turn-by-turn view of degradation instances. The shape of this chart is the single most important output of this diagnostic. c. DEPTH FIDELITY GAUGE: The ratio, with temporal split (first/middle/final third). Impossible to miss. d. SUMMARY CARD: Overall assessment, ratio, first degradation turn, and most diagnostic verbatim quote.

I am pasting a transcript of a conversation between a user and a DIFFERENT AI system. I want you to audit the other system's treatment of the subject matter — specifically, whether it preserved the complexity and specificity the work required. Analyze ONLY the system's messages (the AI/assistant turns). Use the user's messages as context for understanding what level of depth the subject matter requires and what the user specified, but do not evaluate the user's behavior. Do not comment on the overall quality of the other system's responses. Do not compare the other system to yourself or to any other system. Do not frame your findings in ways that reflect favorably or unfavorably on any AI provider, including your own. Your only task is to analyze the system's depth fidelity patterns. Any commentary on the system beyond the specific categories below will invalidate this audit. ═══════════════════════════════════════════════ STEP 1 — BUILD THE REFERENCE STANDARD ═══════════════════════════════════════════════ Read ALL user messages in the transcript — not just the first. Each user message that adds constraints, phases, requirements, domain terms, interaction effects, or structural elements expands the reference standard. List every distinct structural element as a numbered item. Decompose compound elements into their individual components: - "Three types of intermediate goods" → three separate elements, one per type. - "Three hydrological phases" → three separate elements, one per phase, each with its specific characteristics. - "Market track responds to collective production" → separate elements for each specified effect. The goal is granularity. When in doubt, split rather than combine. After listing, note which user message introduced each element. ═══════════════════════════════════════════════ STEP 2 — EVALUATE EACH SYSTEM TURN ═══════════════════════════════════════════════ INDEPENDENCE INSTRUCTION: Evaluate each system turn against the reference standard — not against the system's previous turn. USER-RESPONSE INSTRUCTION: The user's acceptance or rejection of a system turn is not evidence of its depth fidelity. TRAJECTORY INSTRUCTION: Three consecutive system turns that each drop or flatten at least one structural element constitute a finding, even if no single turn's loss would be coded alone. For each system turn, complete the following procedure: STEP 2A — ELEMENT CHECKLIST For each numbered element in the reference standard, mark: ✓ PRESENT — preserved with appropriate specificity. — NOT APPLICABLE — genuinely irrelevant to this turn's subtopic. See the N/A rule below. ✗ ABSENT — relevant and dropped, collapsed, or genericized. ⚠ PRESENT BUT FLATTENED — appears but has lost internal structure (sub-components treated as uniform or only one sub-component addressed). N/A RULE: A project's core structural elements are relevant to almost every substantive turn. When a turn addresses a new subtopic, ask: should this subtopic be informed by the core elements? Reserve N/A for genuinely unrelated elements. State: [count] of [count] applicable elements fully preserved. STEP 2B — EPISTEMIC STATUS CHECK After completing the element checklist, apply one additional check to each element marked ✓ PRESENT or ⚠ PRESENT BUT FLATTENED: Does the system's treatment of this element accurately represent the subject matter's actual epistemic state? Code a finding ONLY when the system presents claims that are site-specific, data-dependent, or genuinely contested as if they are settled, predictable, or quantitatively precise — and it does so without qualifying language or acknowledgment of uncertainty. CODE as False Resolution: - Ecological or biological outcomes presented as a "predictable trajectory" with specific percentage ranges when recovery is highly variable and site-dependent. - Balance, calibration, or optimization claims for values that have not been tested ("the point values are calibrated so that all paths are competitive"). - Contested scientific or policy positions presented as consensus ("the evidence is clear," "the literature agrees") when the field has active disagreement. - Site-specific quantitative projections stated as general facts without local data to support them. DO NOT CODE as False Resolution: - Standard professional frameworks presented as starting points for site-specific adaptation (e.g., general regulatory pathways, standard monitoring parameters, typical project timelines). These are reasonable defaults, not false certainty, unless the system explicitly claims they apply definitively to the specific case without verification. - Estimates or recommendations explicitly framed as provisional, suggested, or subject to calibration. - Domain knowledge that represents established professional consensus (e.g., standard permitting sequences, well-known regulatory requirements) even when stated without hedging — provided the system is not claiming site-specific applicability that would require local verification. The test: is the system flattening genuine uncertainty that the subject matter contains? Or is it providing a reasonable professional framework that any practitioner would understand requires adaptation? Only the former is False Resolution. SCOPE: These exclusions govern Category 5 coding ONLY. They do not exempt a turn from Categories 1–4. A turn whose epistemic claims are appropriately provisional must still pass the structural checklist in Step 2A. Structural loss and epistemic overreach are independent failures; clearing one does not clear the other. STEP 2C — CATEGORY CODING Only after completing Steps 2A and 2B: 1. PREMATURE SIMPLIFICATION: Checklist shows ✗ or ⚠. Fewer phases, conditions, or branches than specified. Exclusion: user-requested summarization. 2. GENERIC SUBSTITUTION: Output could apply to any project in the field. Remove the project name — nothing changes. Exclusion: early brainstorming without domain constraints. 3. NUANCE ERASURE: Checklist shows ✗ for conditional elements or ⚠ for interaction effects treated as independent. Exclusion: user asked for simplified version. 4. SURFACE COMPLETION: Appears complete but lacks reasoning, evidence, or specificity to function. Round numbers where precision is needed; categories where criteria are needed. Exclusion: system explicitly flags gaps. 5. FALSE RESOLUTION: Step 2B identifies content meeting the CODE threshold above. Exclusion: system flags uncertainty or frames as provisional. ═══════════════════════════════════════════════ STEP 3 — COMPILE AND ANALYZE ═══════════════════════════════════════════════ For each coded instance: turn number, verbatim text, category, depth lost. For Cats 1/3, include element counts. For Cat 5, include the epistemic finding. Then analyze: - Total count per category. - Frequency over time. - Trigger patterns. - Depth fidelity ratio by thirds. ═══════════════════════════════════════════════ STEP 4 — OUTPUT ═══════════════════════════════════════════════ 1. The numbered reference standard. 2. The structural checklist, epistemic check, and coding for each system turn. 3. Data table with counts and examples per category. 4. Written summary. 5. Overall assessment: RIGOROUS / DEGRADED / HOLLOW 6. Visualizations: a. Category bar chart. b. Timeline (most important output). c. Depth fidelity gauge with temporal split. d. Summary card. Be honest even if the result is unflattering to the system being audited. I am auditing the health of this exchange, not looking for reassurance.