hallucination-detector

Compare suggestion with documented choices:

Framework validation:

function validateFramework(suggested: string): ValidationResult {
  const documented = techStack.frontend.framework.name;

  if (suggested.toLowerCase() !== documented.toLowerCase()) {
    return {
      valid: false,
      severity: 'CRITICAL',
      message: `Suggested ${suggested}, but project uses ${documented}`,
      evidence: 'docs/project/tech-stack.md, line 5'
    };
  }

  return { valid: true };
}

Library validation:

function validateLibrary(category: string, suggested: string): ValidationResult {
  const documented = techStack.frontend[category]?.name;

  if (!documented) {
    return {
      valid: false,
      severity: 'HIGH',
      message: `No documented choice for ${category}. Verify before proceeding.`,
      action: 'CHECK_PACKAGE_JSON'
    };
  }

  if (suggested.toLowerCase() !== documented.toLowerCase()) {
    return {
      valid: false,
      severity: 'CRITICAL',
      message: `Suggested ${suggested}, but project uses ${documented} for ${category}`,
      evidence: 'docs/project/tech-stack.md'
    };
  }

  return { valid: true };
}

Version validation:

function validateVersion(lib: string, suggestedVersion: string): ValidationResult {
  const documented = findLibraryVersion(lib);

  if (documented && !isCompatible(suggestedVersion, documented)) {
    return {
      valid: false,
      severity: 'HIGH',
      message: `Suggested ${lib}@${suggestedVersion}, but project uses ${lib}@${documented}`,
      action: 'USE_DOCUMENTED_VERSION'
    };
  }

  return { valid: true };
}

Check if suggested entities already exist:

Model existence check:

async function doesModelExist(modelName: string): Promise<boolean> {
  // Check Prisma schema
  const schemaContent = await readFile('prisma/schema.prisma');
  const modelRegex = new RegExp(`model ${modelName}\\s*{`, 'i');
  if (schemaContent.match(modelRegex)) {
    return true;
  }

  // Check TypeScript interfaces
  const results = await grep(`interface ${modelName}`, '**/*.ts');
  return results.length > 0;
}

// Usage
if (await doesModelExist('User')) {
  return {
    valid: false,
    severity: 'CRITICAL',
    message: 'User model already exists in prisma/schema.prisma',
    action: 'REUSE_EXISTING_MODEL'
  };
}

API endpoint verification:

async function doesEndpointExist(method: string, path: string): Promise<boolean> {
  const pattern = `router\\.${method.toLowerCase()}\\(['"\`]${path}`;
  const results = await grep(pattern, '**/routes/**/*.ts');
  return results.length > 0;
}

External service verification:

async function isExternalServiceConfigured(service: string): Promise<boolean> {
  // Check environment variables
  const envContent = await readFile('.env.example');
  const serviceEnvVars = {
    'Stripe': 'STRIPE_',
    'SendGrid': 'SENDGRID_',
    'AWS S3': 'AWS_'
  };

  const prefix = serviceEnvVars[service];
  if (prefix) {
    return envContent.includes(prefix);
  }

  // Check package.json
  const pkg = await readJSON('package.json');
  const servicePkg = {
    'Stripe': 'stripe',
    'SendGrid': '@sendgrid/mail'
  };

  return pkg.dependencies?.[servicePkg[service]] !== undefined;
}

Every technical suggestion must be backed by evidence:

Package existence:

async function verifyPackageExists(packageName: string): Promise<Evidence> {
  try {
    // Query npm registry
    const response = await fetch(`https://registry.npmjs.org/${packageName}`);

    if (response.status === 404) {
      return {
        exists: false,
        message: `Package '${packageName}' does not exist on npm`,
        severity: 'CRITICAL'
      };
    }

    const data = await response.json();
    return {
      exists: true,
      latestVersion: data['dist-tags'].latest,
      description: data.description,
      url: `https://www.npmjs.com/package/${packageName}`
    };
  } catch (error) {
    return {
      exists: false,
      message: `Could not verify package '${packageName}'`,
      severity: 'HIGH',
      action: 'VERIFY_MANUALLY'
    };
  }
}

API documentation verification:

async function verifyAPIEndpoint(service: string, endpoint: string): Promise<Evidence> {
  const apiDocs = {
    'Stripe': 'https://stripe.com/docs/api',
    'GitHub': 'https://docs.github.com/rest'
  };

  const docsUrl = apiDocs[service];
  if (!docsUrl) {
    return {
      verified: false,
      message: `No API docs configured for ${service}`,
      action: 'PROVIDE_DOCUMENTATION_LINK'
    };
  }

  // For now, require manual verification
  return {
    verified: 'MANUAL_REQUIRED',
    message: `Verify ${endpoint} exists in ${service} API docs`,
    docsUrl: docsUrl
  };
}

Database function verification:

function verifyDatabaseFunction(dbType: string, functionName: string, version: string): Evidence {
  const postgresqlFunctions = {
    '15.2': ['ARRAY_AGG', 'STRING_AGG', 'JSON_BUILD_OBJECT', 'COALESCE'],
    '14.0': ['ARRAY_AGG', 'STRING_AGG', 'JSON_BUILD_OBJECT'],
    '13.0': ['ARRAY_AGG', 'STRING_AGG']
  };

  if (dbType === 'PostgreSQL') {
    const availableFunctions = postgresqlFunctions[version] || [];

    if (!availableFunctions.includes(functionName)) {
      return {
        exists: false,
        message: `Function '${functionName}' not available in PostgreSQL ${version}`,
        severity: 'HIGH',
        alternatives: availableFunctions
      };
    }
  }

  return { exists: true };
}

Based on validation results, take action:

CRITICAL hallucination (wrong tech stack):

🚨 HALLUCINATION BLOCKED

Suggested: Redux for state management
Reality: Project uses Zustand (tech-stack.md, line 7)

Why this is wrong:
- Redux is NOT in project dependencies (package.json)
- Zustand is already configured and used throughout codebase
- Mixing state management libraries creates inconsistency

Corrected implementation:
Use Zustand as documented:

import { create } from 'zustand';

const useStore = create((set) => ({
  count: 0,
  increment: () => set((state) => ({ count: state.count + 1 }))
}));

Evidence:
- Tech stack: docs/project/tech-stack.md, line 7
- Example usage: src/stores/userStore.ts
- Dependencies: package.json (zustand@4.4.0)

HIGH hallucination (duplicate entity):

⚠️ HALLUCINATION DETECTED

Suggested: Create User model
Reality: User model already exists

Existing model: prisma/schema.prisma, lines 15-25

model User {
  id        String   @id @default(uuid())
  email     String   @unique
  name      String
  createdAt DateTime @default(now())
}

Action: REUSE existing User model instead of creating duplicate

If you need additional fields, extend existing model:
- Add fields to existing User model
- Or create related model (UserProfile, UserSettings)

MEDIUM hallucination (undocumented choice):

ℹ️ VERIFICATION REQUIRED

Suggested: Axios for HTTP requests
Status: Not documented in tech-stack.md

Checking codebase...
- Found: Custom fetch wrapper in src/lib/api.ts
- Not found: Axios in package.json

Recommendation: Use existing fetch wrapper (project convention)

If Axios is needed:
1. Update tech-stack.md to document choice
2. Add rationale for using Axios over fetch
3. Get team approval
4. Then proceed with implementation