AWS WTF Skill for Opsy

Explains every charge on your AWS bill — what it is, why you're paying, and what resource caused it.

⚠️ Mandatory Output

You MUST automatically generate and save a CSV file at the end of every analysis. Do not wait for the user to ask. The analysis is incomplete until the CSV exists.

Step 1: Cost Explorer (Two Queries Required)

Run BOTH queries to detect credit coverage:

# Query 1: Normal (shows $0 if credits cover costs)
aws ce get-cost-and-usage \
  --time-period Start=$(date -v-30d +%Y-%m-%d),End=$(date +%Y-%m-%d) \
  --granularity MONTHLY \
  --metrics "UnblendedCost" "UsageQuantity" \
  --group-by Type=DIMENSION,Key=SERVICE

# Query 2: Exclude credits (shows ACTUAL usage cost)
aws ce get-cost-and-usage \
  --time-period Start=$(date -v-30d +%Y-%m-%d),End=$(date +%Y-%m-%d) \
  --granularity MONTHLY \
  --metrics "UnblendedCost" "UsageQuantity" \
  --group-by Type=DIMENSION,Key=SERVICE \
  --filter '{"Not": {"Dimensions": {"Key": "RECORD_TYPE", "Values": ["Credit", "Refund"]}}}'

Interpretation:

• Query 1 = $0, Query 2 = $X → Credits covering $X actual usage
• Query 1 = Query 2 = $0 → Real free tier
• Query 1 = Query 2 = $X → Normal billing

If credits detected, warn user: "Your bill shows $0 but actual usage is $X/month. When credits run out, you WILL be charged."

Step 2: Identify ALL Regions

aws ce get-cost-and-usage \
  --time-period Start=$(date -v-30d +%Y-%m-%d),End=$(date +%Y-%m-%d) \
  --granularity MONTHLY \
  --metrics "UnblendedCost" \
  --group-by Type=DIMENSION,Key=REGION \
  --filter '{"Not": {"Dimensions": {"Key": "RECORD_TYPE", "Values": ["Credit", "Refund"]}}}'

You MUST enumerate resources in EVERY region showing charges > $0.01.

Step 3: Enumerate ALL Resources

For each region with charges, query ALL applicable services using --region $REGION:

• aws ec2 describe-instances
• aws ec2 describe-volumes
• aws ec2 describe-snapshots --owner-ids self
• aws ec2 describe-addresses
• aws ec2 describe-nat-gateways
• aws rds describe-db-instances
• aws elbv2 describe-load-balancers
• aws ecs list-clusters → describe-clusters → list-services → list-tasks
• aws eks list-clusters → describe-cluster
• aws lambda list-functions
• aws s3api list-buckets (global) → get-bucket-location per bucket
• aws ecr describe-repositories
• aws secretsmanager list-secrets
• aws logs describe-log-groups
• aws kms list-keys
• aws route53 list-hosted-zones (global)

ARN Construction

For resources without ARN in response, construct: arn:aws:{service}:{region}:{account}:{resource-type}/{id}

Examples:

• EC2: arn:aws:ec2:us-east-1:123456789012:instance/i-abc123
• EBS: arn:aws:ec2:us-east-1:123456789012:volume/vol-abc123
• S3: arn:aws:s3:::bucket-name

⚠️ CRITICAL: Every Charge Must Have Identification

Every row in the CSV MUST have a resource identifier (ARN + resource_id) UNLESS it is truly untraceable.

What CAN Be Traced (MUST have ARN)

ANY charge from these services MUST be traced to a specific resource:

Service	Has ARN	Example
EC2	✅ Always	Instance, Volume, Snapshot, EIP, NAT Gateway
S3	✅ Always	Bucket
RDS	✅ Always	Instance
ECS/EKS	✅ Always	Cluster, Service, Task
Lambda	✅ Always	Function
ALB/NLB	✅ Always	Load Balancer
CloudWatch Logs	✅ Always	Log Group
Secrets Manager	✅ Always	Secret
KMS	✅ Always	Key
ECR	✅ Always	Repository
Route 53	✅ Always	Hosted Zone

If Cost Explorer shows charges for these but you can't find the resource → it was deleted mid-period. Put ARN as DELETED - {service} and note in description.

What CANNOT Be Traced (N/A allowed)

Only these charges are truly untraceable to a single resource:

Charge Type	Why Untraceable
Data Transfer Out	Aggregated from multiple sources
Data Transfer Inter-Region	No single source
Data Transfer Inter-AZ	No single source
Support Plan	Account-level
Tax	Account-level
CloudWatch Custom Metrics (aggregated)	No single dimension

For these only: use arn: N/A - Service-level charge or N/A - Account-level charge

Verification Rule

Before marking ANY charge as N/A, ask: "Is there a specific AWS resource that caused this?"

• If YES → find it, get its ARN
• If NO (only data transfer, support, tax) → N/A is acceptable

Elastic IP Verification

Check AssociationId before calling an IP "unattached":

• AssociationId present → attached (even if InstanceId is empty)
• NetworkInterfaceOwnerId = "amazon-..." → service-managed (ALB, RDS, NAT)

Public IPv4 Charges

AWS charges $0.005/hr ($3.60/mo) per public IPv4. Find all sources:

• EC2 public IPs, Elastic IPs, internet-facing ALBs, NAT Gateways

CSV Output (Mandatory)

Filename: aws-wtf-{account-id}-{date}.csv

account_id,resource_name,charge_category,charge_explanation,monthly_cost_usd,status,resource_id,arn,region,resource_type,tags,description

Column Definitions

Column	Description
account_id	AWS account ID
resource_name	Name tag (empty if untagged)
charge_category	Compute, Storage, Database, Networking, Container, Serverless, Monitoring, DNS, Security, Data Transfer, Support, Tax
charge_explanation	What you're paying for: EC2 t3.small, EBS gp3 20GB, ALB hourly
monthly_cost_usd	Actual cost (not $0 even if credit-covered)
status	Billed, Free-Tier, Credit-Offset
resource_id	AWS resource ID or N/A for non-resource charges
arn	Full ARN or N/A - Service-level charge
region	AWS region or global
resource_type	EC2, EBS, RDS, S3, Lambda, etc.
tags	key=value,key=value
description	Cost breakdown: 720 hrs × $0.10/hr, details

One Row Per Charge Type

A resource can have multiple rows:

• EC2: Compute hours + Public IPv4
• RDS: Instance hours + Storage
• ALB: Hourly + LCU + Public IPv4
• ECS Fargate: vCPU + Memory

Example

account_id,resource_name,charge_category,charge_explanation,monthly_cost_usd,status,resource_id,arn,region,resource_type,tags,description
550435500798,api-server,Compute,EC2 t3.small,15.18,Credit-Offset,i-abc123,arn:aws:ec2:us-east-1:550435500798:instance/i-abc123,us-east-1,EC2,"Env=prod","720 hrs × $0.0211/hr"
550435500798,api-server,Networking,Public IPv4,3.60,Credit-Offset,i-abc123,arn:aws:ec2:us-east-1:550435500798:instance/i-abc123,us-east-1,EC2-IPv4,,"720 hrs × $0.005/hr"
550435500798,/aws/lambda/func,Monitoring,CloudWatch Logs,0.45,Credit-Offset,/aws/lambda/func,arn:aws:logs:us-east-1:550435500798:log-group:/aws/lambda/func,us-east-1,CloudWatch-Logs,,"15GB × $0.03/GB"
550435500798,,Data Transfer,Egress to Internet,12.00,Billed,N/A,N/A - Service-level charge,us-east-1,DataTransfer,,"133GB × $0.09/GB"

Summary Report

## AWS Bill Breakdown
**Account:** {id} | **Period:** {start} to {end}

### Cost Summary
| Metric | Amount |
|--------|--------|
| Actual Usage | $XXX |
| Credits Applied | -$XXX |
| **You Pay** | **$X.XX** |

### By Category
| Category | Amount | % |
|----------|--------|---|
| Compute | $XX | X% |
| Storage | $XX | X% |
...

### Top Charges
| Resource | Type | Cost | Description |
|----------|------|------|-------------|
| {name} | {type} | $XX | {explanation} |

Completion Checklist

• Cost Explorer: both queries (with/without credits)
• All regions identified and enumerated
• All resources listed with ARNs
• CSV file saved
• Summary shown to user

Analysis is INCOMPLETE until CSV file exists.