Data Readiness Assessor

Overview

Evaluate whether data is ready for AI/ML projects before committing resources. Assess quality, availability, labeling needs, and identify gaps that require remediation.

Core principle: Data is the foundation. A thorough assessment prevents "garbage in, garbage out" and project delays.

When to Use

• Starting new AI/ML project
• Evaluating feasibility of AI use case
• Diagnosing model performance issues
• Planning data infrastructure investments

Output Format

data_assessment:
  project: "[Project name]"
  assessment_date: "[YYYY-MM-DD]"
  assessor: "[Name]"
  
  overall_readiness:
    score: "[1-5]"
    verdict: "[Ready | Ready with caveats | Not ready]"
    summary: "[Brief assessment]"
  
  data_sources:
    - source: "[Data source name]"
      type: "[Structured | Unstructured | Semi-structured]"
      location: "[Where stored]"
      owner: "[Data owner]"
      access: "[How to access]"
      
      volume:
        records: "[Count]"
        size: "[GB/TB]"
        time_range: "[Date range covered]"
        sufficient: "[Yes | No | Borderline]"
      
      quality:
        completeness:
          score: "[1-5]"
          missing_rate: "[%]"
          critical_fields_missing: ["[Field]"]
        
        accuracy:
          score: "[1-5]"
          known_issues: ["[Issue]"]
          validation_method: "[How verified]"
        
        consistency:
          score: "[1-5]"
          duplicates: "[% or count]"
          format_issues: ["[Issue]"]
        
        timeliness:
          score: "[1-5]"
          freshness: "[How recent]"
          update_frequency: "[How often updated]"
      
      relevance:
        features_available: ["[Feature 1]", "[Feature 2]"]
        features_missing: ["[Needed but not present]"]
        target_variable: "[Available | Derivable | Missing]"
  
  labeling_assessment:
    required: [true | false]
    current_state:
      labeled_volume: "[Count or %]"
      label_quality: "[High | Medium | Low | Unknown]"
      labeling_consistency: "[Assessment]"
    
    gap:
      additional_labels_needed: "[Count]"
      estimated_effort: "[Hours/days]"
      labeling_approach: "[Manual | Semi-automated | Crowdsourced]"
  
  integration:
    accessibility:
      api_available: [true | false]
      export_options: ["[Format options]"]
      real_time_possible: [true | false]
    
    legal_compliance:
      pii_present: [true | false]
      consent_status: "[Covered | Needs review | Not covered]"
      retention_policies: "[Compliant | Needs review]"
      cross_border: "[Applicable | Not applicable]"
  
  gaps:
    critical:
      - gap: "[Gap description]"
        impact: "[How it affects project]"
        remediation: "[How to fix]"
        effort: "[Time/cost estimate]"
    
    important:
      - gap: "[Gap description]"
        impact: "[How it affects project]"
        remediation: "[How to fix]"
  
  recommendations:
    proceed_if:
      - "[Condition for proceeding]"
    
    actions_required:
      - action: "[Required action]"
        owner: "[Who]"
        timeline: "[When]"
        blocking: [true | false]

Quality Dimensions

The Five V's Assessment

Sample Size Guidelines

Data Quality Scorecard

quality_scorecard:
  dimension: "Completeness"
  scoring:
    5: "<1% missing values in critical fields"
    4: "1-5% missing values, no critical gaps"
    3: "5-15% missing values, some critical gaps"
    2: "15-30% missing values, significant gaps"
    1: ">30% missing or critical fields unavailable"
  
  dimension: "Accuracy"
  scoring:
    5: "Validated against ground truth, <1% error"
    4: "Spot-checked, <5% error rate"
    3: "Some validation, known issues documented"
    2: "Limited validation, suspected issues"
    1: "No validation, reliability unknown"
  
  dimension: "Consistency"
  scoring:
    5: "Standardized formats, no duplicates"
    4: "Minor format variations, <1% duplicates"
    3: "Multiple formats, 1-5% duplicates"
    2: "Significant format issues, 5-10% duplicates"
    1: "Major inconsistencies, >10% duplicates"

Labeling Assessment

Labeling Quality Checklist

labeling_quality:
  guidelines:
    - "Clear labeling instructions exist"
    - "Edge cases documented"
    - "Examples provided for each class"
  
  process:
    - "Multiple labelers for quality"
    - "Inter-annotator agreement measured"
    - "Disagreements have resolution process"
  
  coverage:
    - "All classes represented"
    - "Class distribution acceptable"
    - "Edge cases labeled"

Labeling Effort Estimation

Red Flags

Readiness Levels

Assessment Checklist

• All data sources identified
• Access verified for each source
• Volume sufficiency assessed
• Quality dimensions scored
• Labeling needs determined
• Legal/compliance reviewed
• Gaps documented with remediation
• Readiness verdict provided