Troubleshooting

Airflow DAG Parsing Errors

Symptom: DAG not appearing in Airflow UI

# Test DAG parsing locally
python dags/multi_sport_betting_workflow.py

# Check Airflow logs
docker logs $(docker ps -qf "name=scheduler") 2>&1 | grep -i error

Common causes:

• Import errors in plugins (missing dependencies)
• Syntax errors in DAG file
• Circular imports

Fix: Ensure plugins directory is in Python path:

plugins_dir = Path(__file__).parent.parent / "plugins"
if str(plugins_dir) not in sys.path:
    sys.path.insert(0, str(plugins_dir))

Database Connection Issues

Symptom: connection refused or authentication failed

# Check PostgreSQL is running
docker ps | grep postgres

# Test connection
psql -h localhost -U airflow -d airflow -c "SELECT 1"

Environment variables:

export POSTGRES_HOST=localhost
export POSTGRES_PORT=5432
export POSTGRES_USER=airflow
export POSTGRES_PASSWORD=airflow
export POSTGRES_DB=airflow

Kalshi API Authentication

Symptom: 401 Unauthorized or Invalid API key

Check credentials:

with open("kalshkey") as f:
    api_key = f.readline().strip()
    private_key = f.read()

print(f"API Key length: {len(api_key)}")
print(f"Private key starts with: {private_key[:50]}")

Common issues:

• Extra whitespace in kalshkey file
• Expired API key
• Wrong environment (demo vs production)

Team Name Mismatches

Symptom: Bets not matching, empty results

from naming_resolver import NamingResolver

resolver = NamingResolver()

# Debug team name resolution
print(resolver.resolve("LAL", "nba"))
print(resolver.resolve("Lakers", "nba"))
print(resolver.resolve("Los Angeles Lakers", "nba"))

Fix: Add missing mappings to naming_resolver.py:

NBA_ALIASES = {
    "LA Lakers": "Los Angeles Lakers",
    "LAL": "Los Angeles Lakers",
    # Add missing aliases
}

Data Pipeline Failures

Symptom: Missing games, stale data

# Check last successful run
docker exec $(docker ps -qf "name=scheduler") \
  airflow dags list-runs -d multi_sport_betting_workflow --limit 5

Debug data freshness:

from db_manager import default_db

df = default_db.fetch_df("""
    SELECT sport, MAX(game_date) as latest
    FROM unified_games
    GROUP BY sport
""")
print(df)

Task Failures

Get task logs:

docker exec $(docker ps -qf "name=scheduler") \
  airflow tasks logs multi_sport_betting_workflow download_games_nba 2024-01-15

Clear failed task:

docker exec $(docker ps -qf "name=scheduler") \
  airflow tasks clear multi_sport_betting_workflow -t download_games_nba -s 2024-01-15 -e 2024-01-15

Memory Issues

Symptom: OOM errors, slow performance

# Process data in chunks
for chunk in pd.read_sql(query, engine, chunksize=10000):
    process(chunk)

Import Errors

Symptom: ModuleNotFoundError

# Check if module exists
ls plugins/ | grep module_name

# Check Python path
python -c "import sys; print('\n'.join(sys.path))"

Files to Reference

• TROUBLESHOOTING.md - Additional troubleshooting
• docker-compose.yaml - Service configuration
• logs/ - Airflow logs directory