CFN Loop Output Processing - Unified TypeScript Module

Version: 1.0.0 Status: Production Purpose: Type-safe, consolidated output processing for Loop 2 (Validators) and Loop 3 (Implementers) agents

---

Overview

Problem Solved

Previously, output processing was scattered across multiple bash scripts with duplicate logic:

• Loop 2 (validators): cfn-loop2-output-processing/parse-feedback.sh
• Loop 3 (implementers): cfn-loop3-output-processing/parse-confidence.sh
• Duplicate confidence extraction logic
• No type safety
• Difficult to maintain and extend

Solution

Single TypeScript module (cfn-loop-output-processing) providing:

• Type-safe interfaces for all result types
• Unified confidence extraction with 5+ parsing patterns
• Robust feedback parsing with severity categorization
• Consensus calculation from multiple validators
• CLI entry points for orchestrator integration
• 90%+ test coverage with comprehensive test suite

---

Architecture

Module Structure

.claude/skills/cfn-loop-orchestration-v2/lib/output/
├── src/
│   ├── output-processor.ts          # Core module (type-safe)
│   └── cli/
│       ├── process-loop3.ts         # Loop 3 CLI tool
│       └── process-loop2.ts         # Loop 2 CLI tool
├── tests/
│   └── output-processor.test.ts     # Comprehensive test suite (90%+ coverage)
├── SKILL.md                         # This file
└── package.json                     # Dependencies

Key Types

// Loop 3 Result (Implementers)
interface Loop3Result {
  agentId: string;
  confidence: number;
  confidenceSource: 'explicit' | 'calculated' | 'fallback';
  filesChanged: number;
  deliverables: string[];
  testsPassedCount?: number;
  testsFailed?: number;
  passRate?: number;
  output: string;
  iteration: number;
  timestamp: string;
}

// Loop 2 Result (Validators)
interface Loop2Result {
  validatorId: string;
  score: number;
  scoreSource: 'explicit' | 'calculated' | 'qualitative';
  issues: FeedbackItem[];
  criticalCount: number;
  warningCount: number;
  suggestionCount: number;
  recommendations: string[];
  output: string;
  iteration: number;
  timestamp: string;
}

// Consensus Result
interface ConsensusResult {
  averageScore: number;
  threshold: number;
  passed: boolean;
  validatorCount: number;
  scoredCount: number;
  minScore: number;
  maxScore: number;
  summary: string;
  details: {
    criticalIssuesTotal: number;
    warningIssuesTotal: number;
    suggestionsTotal: number;
  };
}

---

Core Functions

Confidence Extraction

parseConfidence(output, config?)

Extracts confidence score using 5-pattern fallback approach.

Patterns (priority order):

1. Explicit header: ## Validation Confidence: 0.85
2. Generic field: confidence: 0.92 or Confidence: 0.92
3. Score field: Score: 0.78
4. Percentage: 92% or 92 percent
5. Parentheses: (0.87)
6. Qualitative: high confidence → 0.90, medium → 0.75, low → 0.50

Returns:

{
  score: number;          // 0.0-1.0
  source: 'explicit' | 'qualitative' | 'none';
}

Examples:

parseConfidence('Confidence: 0.85');           // { score: 0.85, source: 'explicit' }
parseConfidence('92%');                        // { score: 0.92, source: 'explicit' }
parseConfidence('high confidence');            // { score: 0.90, source: 'qualitative' }
parseConfidence('No score provided');          // { score: 0.0, source: 'none' }

Feedback Extraction

extractFeedback(output)

Extracts feedback items from validator output, categorized by severity.

Supported formats:

• Markdown sections: ### CRITICAL Issues\n- Issue text
• Inline format: CRITICAL: Issue text
• Multiple bullet styles: -, *, •

Returns:

FeedbackItem[] = [
  { severity: 'CRITICAL', text: 'Missing error handling' },
  { severity: 'WARNING', text: 'Performance issue' },
  { severity: 'SUGGESTION', text: 'Add documentation' }
]

Example:

### CRITICAL Issues
- SQL injection vulnerability
- Missing authentication

### WARNING Issues
- Performance concern in loop

### SUGGESTION Items
- Add unit tests

extractRecommendations(output)

Extracts recommendations/suggestions from validator output.

Supported headers: Recommendations:, Recommendation:, Suggestions:

Returns: string[] of recommendations

Consensus Calculation

calculateConsensus(results, threshold?)

Calculates consensus from multiple validator results.

Parameters:

• results: Array of Loop2Result
• threshold: Minimum average score to pass (default: 0.70)

Returns:

{
  averageScore: number;
  threshold: number;
  passed: boolean;
  validatorCount: number;
  scoredCount: number;
  minScore: number;
  maxScore: number;
  summary: string;
  details: {
    criticalIssuesTotal: number;
    warningIssuesTotal: number;
    suggestionsTotal: number;
  };
}

Example:

const consensus = calculateConsensus([
  { validatorId: 'v1', score: 0.9, ... },
  { validatorId: 'v2', score: 0.85, ... }
], 0.80);

// Result: { averageScore: 0.875, passed: true, ... }

Loop-Specific Parsers

parseLoop3Output(agentOutput, agentId, iteration?, gitStatus?)

Complete Loop 3 parser combining all extraction logic.

Features:

• Extracts explicit confidence or calculates fallback
• Counts files changed from git status
• Extracts test results if present
• Validates all data

Example:

const result = parseLoop3Output(
  agentOutput,
  'coder-1',
  1,
  { before: '...', after: '...' }
);

// Result: Loop3Result with confidence, deliverables, test info

parseLoop2Output(validatorOutput, validatorId, iteration?)

Complete Loop 2 parser for validator feedback.

Features:

• Extracts confidence score
• Parses feedback by severity
• Extracts recommendations
• Counts issues by category

Example:

const result = parseLoop2Output(
  validatorOutput,
  'reviewer-1',
  1
);

// Result: Loop2Result with score, issues, recommendations

---

CLI Tools

Loop 3 Processor

npx ts-node src/cli/process-loop3.ts [OPTIONS]

Options:

• --agent-id <id> - Agent identifier (required)
• --output <text> - Agent output text (required)
• --output-file <path> - Alternative: read from file
• --iteration <n> - Iteration number (default: 1)
• --files-changed <n> - Number of files changed
• --deliverables <file,file> - Comma-separated files

Output: JSON with Loop3Result

Example:

npx ts-node src/cli/process-loop3.ts \
  --agent-id "coder-1" \
  --output "Implementation complete. Confidence: 0.85" \
  --files-changed 5

# Output:
# {
#   "agentId": "coder-1",
#   "confidence": 0.85,
#   "confidenceSource": "explicit",
#   "filesChanged": 5,
#   ...
# }

Loop 2 Processor (Single Validator)

npx ts-node src/cli/process-loop2.ts [OPTIONS]

Options:

• --validator-id <id> - Validator identifier (required)
• --output <text> - Validator output text (required)
• --output-file <path> - Alternative: read from file
• --iteration <n> - Iteration number (default: 1)

Output: JSON with Loop2Result

Example:

npx ts-node src/cli/process-loop2.ts \
  --validator-id "reviewer-1" \
  --output "Validation: confidence 0.92"

# Output:
# {
#   "validatorId": "reviewer-1",
#   "score": 0.92,
#   ...
# }

Loop 2 Consensus Mode

npx ts-node src/cli/process-loop2.ts --consensus [OPTIONS]

Options:

• --consensus - Enable consensus mode
• --results-file <path> - JSON file with Loop2Result array (required)
• --threshold <n> - Minimum score to pass (default: 0.70)

Output: JSON with ConsensusResult

Example:

npx ts-node src/cli/process-loop2.ts \
  --consensus \
  --results-file ./validator-results.json \
  --threshold 0.75

# Output:
# {
#   "averageScore": 0.88,
#   "passed": true,
#   "summary": "PASS: 88% consensus from 3 validators...",
#   ...
# }

---

Integration with Orchestrator

Loop 3 Integration

# From orchestrator: process implementer output
RESULT=$(npx ts-node $HOME/.claude/skills/cfn-loop-orchestration-v2/lib/output/src/cli/process-loop3.ts \
  --agent-id "$AGENT_ID" \
  --output "$AGENT_OUTPUT" \
  --iteration "$ITERATION")

CONFIDENCE=$(echo "$RESULT" | jq -r '.confidence')
FILES_CHANGED=$(echo "$RESULT" | jq -r '.filesChanged')

# Report to Redis
redis-cli LPUSH "swarm:${TASK_ID}:${AGENT_ID}:confidence" "$CONFIDENCE"
redis-cli LPUSH "swarm:${TASK_ID}:${AGENT_ID}:done" "complete"

Loop 2 Integration (Single Validator)

# Process validator output
RESULT=$(npx ts-node $HOME/.claude/skills/cfn-loop-orchestration-v2/lib/output/src/cli/process-loop2.ts \
  --validator-id "$VALIDATOR_ID" \
  --output "$VALIDATOR_OUTPUT" \
  --iteration "$ITERATION")

SCORE=$(echo "$RESULT" | jq -r '.score')

# Store for later consensus calculation
echo "$RESULT" >> ./validator-results.json

Loop 2 Integration (Consensus)

# Calculate consensus from all validators
CONSENSUS=$(npx ts-node $HOME/.claude/skills/cfn-loop-orchestration-v2/lib/output/src/cli/process-loop2.ts \
  --consensus \
  --results-file ./validator-results.json \
  --threshold "$CONSENSUS_THRESHOLD")

PASSED=$(echo "$CONSENSUS" | jq -r '.passed')

if [ "$PASSED" = "true" ]; then
  echo "Validators consensus PASSED"
else
  echo "Validators consensus FAILED - average score below threshold"
fi

---

Confidence Extraction Patterns

NOTE: this 5-pattern extraction ladder is RECOVERY-ONLY. It exists to salvage confidence from malformed agent output. Orchestrator prompts must demand the JSON contract as primary (agents report an explicit JSON block with a decimal confidence field, computed by the mechanical rubric in lib/validation/SKILL.md). If you find yourself relying on patterns 3-5, fix the spawn prompt, not the parser.

Priority Order

1. Explicit Header (Priority 1)

   • ## Validation Confidence: 0.85
   • Highest precision

2. Generic Field (Priority 2)

   • confidence: 0.92
   • Confidence: 0.92
   • Score: 0.78

3. Percentage (Priority 3)

   • 92%
   • 92 percent

4. Parentheses (Priority 4)

   • (0.87)
   • Context-dependent

5. Qualitative (Priority 5)

   • high confidence → 0.90
   • medium confidence → 0.75
   • low confidence → 0.50

Fallback Strategy

If no explicit confidence found:

1. Check qualitative mappings
2. For Loop 3 only: Calculate from file changes + tests
3. Return 0.0 with source="none"

---

Feedback Categorization

CRITICAL Issues

• Blocking problems
• Security vulnerabilities
• Fundamental functionality breaks
• Must be fixed before proceeding

WARNING Issues

• Important non-blocking problems
• Performance concerns
• Maintainability issues
• Should be addressed

SUGGESTION Items

• Optional improvements
• Code style recommendations
• Minor optimizations
• Nice-to-have enhancements

---

Test Coverage

Test Suite

File: tests/output-processor.test.ts Coverage: 90%+ (lines, functions, branches)

Test Categories

• Confidence extraction (8 tests): All 5 patterns, qualitative, edge cases
• Feedback extraction (5 tests): Markdown sections, inline format, edge cases
• Recommendations (4 tests): Multiple header formats, empty cases
• Fallback calculation (6 tests): File change thresholds, test impacts
• Validation (4 tests): Range validation, NaN handling
• Loop 3 parsing (4 tests): Complete workflow, git integration
• Loop 2 parsing (3 tests): Complete workflow, categorization
• Consensus (6 tests): Multiple validators, edge cases, aggregation
• Default detection (3 tests): Default output identification
• Serialization (3 tests): JSON round-trip
• Integration (2 tests): Complete workflows

Running Tests

# Install dependencies
npm install

# Run all tests with coverage
npm run test

# Run specific test suite
npm test -- --testNamePattern="parseConfidence"

# Watch mode for development
npm test -- --watch

---

Migration Guide

From Bash to TypeScript

Old (Bash):

CONFIDENCE=$(./parse-confidence.sh "$AGENT_OUTPUT")
DELIVERABLES=$(./verify-deliverables.sh --before "$BEFORE" --after "$AFTER")

New (TypeScript):

import { parseConfidence, parseLoop3Output } from './output-processor';

const { score, source } = parseConfidence(agentOutput);
const result = parseLoop3Output(agentOutput, agentId, iteration, gitStatus);

CLI (Same interface):

npx ts-node src/cli/process-loop3.ts \
  --agent-id "coder-1" \
  --output "$AGENT_OUTPUT"

Deprecation Timeline

Immediate (Now):

• cfn-loop2-output-processing/parse-feedback.sh → Use TypeScript module
• cfn-loop3-output-processing/parse-confidence.sh → Use TypeScript module
• cfn-loop3-output-processing/calculate-confidence.sh → Use TypeScript module

30 days: Bash scripts continue to work (deprecated status) 60 days: Orchestrator fully migrated to TypeScript 90 days: Bash scripts removed

---

Benefits

Aspect	Bash Scripts	TypeScript Module
Type Safety	❌ No	✅ Yes
Code Reuse	❌ Duplicate logic	✅ Shared functions
Test Coverage	❌ Limited	✅ 90%+
Documentation	❌ Inline comments	✅ Comprehensive
Maintainability	❌ Hard to modify	✅ Easy to extend
Performance	✅ Fast	✅ Fast (Node.js)
Debuggability	⚠️ Bash errors	✅ Stack traces
IDE Support	❌ None	✅ Full IntelliSense

---

Error Handling

Invalid Input

// Empty input
parseConfidence('');                    // { score: 0.0, source: 'none' }

// Invalid JSON in results file
const results = parseJson(invalidJson); // null

// Out of range score
isValidConfidence(1.5);                 // false

CLI Error Messages

# Missing required argument
$ npx ts-node src/cli/process-loop3.ts
Error: --agent-id is required

# File not found
$ npx ts-node src/cli/process-loop3.ts --output-file missing.txt
Error: Output file not found: missing.txt

---

Performance

• Confidence extraction: <1ms
• Feedback parsing: <2ms
• Consensus calculation: <5ms (10 validators)
• Total CLI execution: <100ms

Memory efficient with minimal allocations.

---

Backward Compatibility

Breaking Changes

None. This module consolidates without changing external interfaces.

CLI Compatibility

All CLI tools accept same arguments as predecessor bash scripts.

Integration Points

Orchestrator integration remains identical - only internal implementation changes.

---

Related Documentation

• Agent Output Processing (.claude/skills/cfn-agent-lifecycle/lib/output/README.md) - Universal pattern
• Loop Orchestration (.claude/skills/cfn-loop-orchestration-v2/lib/orchestrator/SKILL.md) - Coordinator integration
• CFN Loop Architecture (docs/CFN_LOOP_ARCHITECTURE.md) - System design

---

Version History

1.0.0 (2025-11-19)

• Initial TypeScript consolidation
• Unified confidence extraction (5+ patterns)
• Robust feedback parsing
• Consensus calculation
• Type-safe interfaces
• 90%+ test coverage
• CLI entry points for orchestrator
• Deprecates 3 bash scripts

---

Summary

This skill provides type-safe, consolidated output processing for all CFN Loop agents. By consolidating previously scattered bash scripts into a single TypeScript module, we achieve:

• Type Safety: Compile-time error detection
• Maintainability: Single source of truth for parsing logic
• Testability: 90%+ test coverage with comprehensive suite
• Performance: Optimized parsing with multi-pattern fallbacks
• Documentation: Complete API documentation with examples
• Backward Compatibility: Identical CLI interfaces

The module is production-ready and replaces cfn-loop2-output-processing and cfn-loop3-output-processing bash skills.