Gemini Batch API Skill

What this skill carries — grep references/ for any subject the names below miss: !d=${CLAUDE_SKILL_DIR}; command -v skill-toc >/dev/null 2>&1 && exec skill-toc "$d"; s=$HOME/.claude/skills/plugin-utils/bin/skill-toc; [ -x "$s" ] && exec "$s" "$d"; echo "(skill-toc unavailable: references and scripts are NOT listed here — install the plugin-utils plugin, or start a new session so its bin/ reaches PATH)"

Large-scale asynchronous document processing using Google's Gemini models.

When to Use

• Process thousands of documents with the same prompt
• Cost-effective bulk extraction (50% cheaper than synchronous API)
• Jobs that can tolerate 24-hour completion windows

IRON LAW: Use Examples First, Never Guess API

READ EXAMPLES BEFORE WRITING ANY CODE. NO EXCEPTIONS.

The Rule

User asks for batch API work
    ↓
MANDATORY: Read examples/batch_processor.py or examples/icon_batch_vision.py
    ↓
Copy the pattern exactly
    ↓
DO NOT guess parameter names
DO NOT try wrapper types
DO NOT improvise API calls

Why This Matters

The Batch API has non-obvious requirements that will fail silently:

1. Metadata must be flat primitives - Nested objects cause cryptic errors
2. dest is a config field, not a kwarg - Pass via config={"dest": "gs://..."}. Older SDKs accepted dest= directly; newer ones raise TypeError.
3. Config is plain dict - Not a wrapper type
4. Examples are authoritative - Working code beats assumptions

Rationale: Previous agents wasted hours debugging API errors that the examples would have prevented. The patterns in examples/ are battle-tested production code.

Red Flags

• About to pass dest= as a kwarg → STOP. That works on older SDKs only; the current SDK puts dest inside config={}. Read the examples.
• About to instantiate a CreateBatchJobConfig object → STOP. The config is a plain dict, not a wrapper type.
• About to nest metadata like a normal API → STOP. Nested objects trigger BigQuery type errors; flatten the data.
• About to assume this works like other Google APIs → STOP. This API is different; the examples are authoritative.
• About to improvise the JSONL format → STOP. Copy the structure from the examples instead.

MANDATORY Checklist Before ANY Batch API Code

• Read examples/batch_processor.py OR examples/icon_batch_vision.py
• Identify which example matches the use case (Standard API vs Vertex AI)
• Copy the example's API call pattern exactly
• Copy the example's JSONL structure exactly
• Copy the example's metadata structure exactly
• Adapt for specific needs only after copying base pattern

Enforcement: Writing batch API code without reading examples first violates this IRON LAW and will result in preventable errors.

Prerequisites

Install gcloud SDK

# macOS: Install via nix-darwin (add to ~/nix/ configuration)
# Or if already available: gcloud --version

# Linux: Install Google Cloud SDK from official sources
curl https://sdk.cloud.google.com | bash

Authentication Setup

# Authenticate with Google Cloud Platform
gcloud auth login

# Set up Application Default Credentials for Python libraries
gcloud auth application-default login

# Enable Vertex AI API in your project
gcloud services enable aiplatform.googleapis.com

Why both auth methods?

• gcloud auth login: For gsutil and gcloud CLI commands
• gcloud auth application-default login: For google-generativeai Python library
• CRITICAL: Vertex AI requires ADC (step 2), not just API key

Create GCS Bucket

# Create bucket in us-central1 (required region)
gsutil mb -l us-central1 gs://your-batch-bucket

# Verify bucket location is us-central1
gsutil ls -L -b gs://your-batch-bucket | grep "Location"

See references/gcs-setup.md for complete setup guide.

Quick Start

Standard Gemini API (API Key)

Uses the Gemini File API for input. Results returned via batch_job.dest.file_name.

from google import genai

client = genai.Client()  # Uses GOOGLE_API_KEY env var

# Upload JSONL to File API
uploaded = client.files.upload(
    file="requests.jsonl",
    config={"mime_type": "application/jsonl"}
)

# Submit batch job
job = client.batches.create(
    model="gemini-2.5-flash-lite",
    src=uploaded.name,  # "files/..." URI
    config={"display_name": "my-batch-job"}
)

# Results available at job.dest.file_name after completion

Vertex AI (Recommended for GCS workflows)

Uses GCS URIs directly. dest is a field of the config dict in the current SDK (older SDKs accepted dest= as a kwarg — that now raises TypeError: Batches.create() got an unexpected keyword argument 'dest').

from google import genai

# Use Vertex AI with ADC (not API key)
client = genai.Client(
    vertexai=True,
    project="your-project-id",
    location="us-central1"
)

# Submit batch job with GCS paths.
# Current SDK signature: create(*, model, src, config)
job = client.batches.create(
    model="gemini-2.5-flash-lite",
    src="gs://bucket/requests.jsonl",     # GCS input
    config={
        "display_name": "my-job",
        "dest": "gs://bucket/outputs/",   # GCS output (Vertex AI only!)
    },
)

Verify your SDK before changing: inspect.signature(client.batches.create). If dest is in the kwargs, the kwarg form works; otherwise use config.

Key difference: Standard API uses File API (files/...), Vertex AI uses GCS (gs://...) with dest (now a config field).

Core Workflow

Standard API:

1. Create JSONL request file with prompts
2. Upload JSONL to File API via client.files.upload()
3. Submit batch job via client.batches.create(src=uploaded.name)
4. Monitor for completion — use Monitor tool (jobs expire after 24 hours)
5. Download results from job.dest.file_name

Vertex AI:

1. Upload files to GCS bucket (us-central1 region required)
2. Create JSONL request file with document URIs and prompts
3. Submit batch job via client.batches.create(src=..., config={"dest": ...})
4. Monitor for completion — use Monitor tool (jobs expire after 24 hours)
5. Download and parse results from GCS output URI
6. Handle failures gracefully (partial failures are common)

Monitoring Batch Jobs with Monitor Tool

After submitting a batch job, use Monitor instead of sleep-polling in Python:

Monitor(
  description="Gemini batch job progress",
  persistent=true,
  timeout_ms=3600000,
  command="while true; do uv run python3 -c \"import google.genai as genai; j=genai.batches.get(name='$JOB_NAME'); print(f'{j.state} | {j.name}'); exit(0 if j.state in ('JOB_STATE_SUCCEEDED','JOB_STATE_FAILED','JOB_STATE_CANCELLED') else 1)\" && break; sleep 60; done"
)

This frees the conversation to continue working while the batch runs. You get notified when the job completes or fails — no polling loop blocking your context.

Key Gotchas (API Structure)

Metadata must be flat primitives (no nested objects — BigQuery-backed storage). dest is a config field, not a top-level kwarg in the current SDK (Vertex AI only). Config is a plain dict (not a wrapper type).

See the Red Flags in the first Iron Law section above — the same gotchas apply here. The Key Gotchas table below summarizes all critical issues.

Key Gotchas

Issue	Solution
Nested metadata fails	Use flat primitives or json.dumps() for complex data
TypeError: unexpected keyword dest	Move dest inside config={} (Vertex AI; current SDK)
Mixing API patterns	Standard API: File API + no dest. Vertex AI: GCS + dest
Auth errors with Vertex AI	Run gcloud auth application-default login
vertexai=True requires ADC	API key is ignored with vertexai=True
Missing aiplatform API	Run gcloud services enable aiplatform.googleapis.com
Region mismatch (Vertex)	Use us-central1 bucket only
Wrong URI format (Vertex)	Use gs:// not https://
Invalid JSONL	Use scripts/validate_jsonl.py
Image batch: inline data	Use fileData.fileUri for batch, not inline
Duplicate IDs	Hash file content + prompt for unique IDs
Large PDFs fail	Split at 1,000 pages / 50MB max (the per-file limit)
JSON parsing fails	Use robust extraction (see gotchas.md)
Output not found (Vertex)	Output URI is prefix, not file path
uploadToFileSearchStore 503 for files >10KB	Use two-step: files.upload() then fileSearchStores.importFile()
File stuck in PROCESSING state	Poll files.get() until state is ACTIVE before importing
SDK Pager stops after first page	Use pager.hasNextPage() + pager.nextPage(), NOT for await
Batch inlinedResponse.response.text is undefined	Response is raw JSON, not hydrated class. Use candidates[0].content.parts[0].text
Store document displayName is random ID after importFile	Read bibkey from customMetadata, not displayName
responseMimeType + tools in batch = error code 3	Omit responseMimeType when using tools; use prompt-based JSON instructions
RuntimeError: Cannot send a request, as the client has been closed	Hold ONE genai.Client for the process; an inline/per-call client is GC'd mid-request
Vertex batch: 404 The PublisherModel <id> does not exist	Qualify it: publishers/google/models/<id> — the bare id works only on the Standard API
Vertex batch 404 on a model models.list() SHOWS in the region	Listing ≠ batch-servable. Fix the LOCATION, not the model: location="global" for 3.x — it accepts a us-central1 src/dest. Downgrading a tier silently changes output

Top 3 mistakes (bolded above):

1. Using nested objects in metadata instead of flat primitives
2. Mixing Standard API and Vertex AI patterns
3. Passing dest= as a kwarg instead of inside config={} (Vertex AI; current SDK)

See references/gotchas.md for detailed solutions (now with Gotchas 10-20; 18-20 are Vertex-batch specific).

Red Flags — STOP If You Catch Yourself:

About to	Why Wrong	Do Instead
Write genai.Client().batches.get(...) inline, or a def client(): return genai.Client(...) factory	The client owns an httpx pool and closes it on __del__; the request dies mid-flight with Cannot send a request, as the client has been closed. Fatal in polling loops.	Hold a module-level singleton for the life of the process
Run pdftotext and send the string because "text is cheaper"	It is not: 258 tokens/page vs ~4 chars/token, and native PDF text is unbilled. Measured 22% more expensive, plus truncation and manual OCR	Send the PDF via fileData.fileUri
Pass a bare model id to batches.create on a Vertex client	Batch needs the publisher path; the bare id 404s even though generate_content accepts it	publishers/google/models/<id>
Conclude a model is unavailable — or available — from models.list()	Listing is not a batch-availability check: us-central1 lists 3.x models that batch then 404s, because 3.x is location: global	Submit a probe job (~20s). Fix the LOCATION; never downgrade a tier to clear a 404 — that swaps the model your evals were run on
Reuse one generationConfig across model tiers	3.x needs thinkingConfig pinned or it returns empty on MAX_TOKENS; 2.5 rejects the field outright	Set thinkingConfig only for gemini-3*, per Gotcha 17/20
Debug a Vertex 404 by reading docs instead of listing + submitting	The error text names the model, never the region or the availability rule — it is the same string for three different causes	Check Gotchas 19 and 20 before assuming the id is wrong

Send the PDF, not extracted text

**When the source is a PDF, send the PDF. Do NOT run `pdftotext` and send the string.**

Gemini bills a document at 258 tokens per page, and native text extracted from the PDF is not charged at all (Google's document-processing docs, verified 2026-08-31). Extracted text is billed as ordinary input at roughly 4 chars/token, so for text-heavy documents the string is the more expensive representation. Measured on a 1,313-document legal corpus — 21,393 pages, 28.2M extracted characters:

representation	tokens
the PDFs	5,519,394
pdftotext output	7,050,563

Extracting first cost 22% more and bought nothing. It also created three problems that do not exist when you send the document:

• You will truncate. A char cap drops whatever sits at the end — in that corpus, counsel's signature block, which silently blanked a required field on every long document until it was caught by hand.
• You will OCR scans yourself. 99 image-only PDFs were run through tesseract at real wall-clock cost; Gemini reads scanned pages natively at the same 258/page.
• You lose layout. Tables, exhibits and signature blocks arrive as flattened text, and the model can no longer see the structure it would use to disambiguate them.

Reach for pdftotext only to triage locally (is this file a scan? how long is it?) — never as the transport into the model.

Limits: 50 MB or 1,000 pages per file, for both inline data and Files API uploads. Batch requests reference the document with fileData.fileUri (a GCS URI on Vertex); never inline the bytes in a batch JSONL.

Rate Limits

Limit	Value
Max requests per JSONL	10,000
Max concurrent jobs	10
Max job size	100MB
Job expiration	24 hours

Recommended Models

ALWAYS verify model IDs and pricing against the live docs

Never recall a model ID or a price from training data — it is always stale. Fetch the .md.txt variants (LLM-optimized, far easier to parse than the HTML):

• Models: https://ai.google.dev/gemini-api/docs/models.md.txt
• Pricing: https://ai.google.dev/gemini-api/docs/pricing.md.txt

Real failures this prevents (encountered 2026-08-03):

• A plan specified gemini-3-pro — that ID does not exist.
• A config carried gemini-3.1-flash-lite priced at {input 0.125, output 0.75}; the current lineup has gemini-3.5-flash-lite at {input 0.30, output 2.50} standard, {0.15, 1.25} batch.
• Version numbers do not stay in parity across lines. As of 2026-08-03 there is a gemini-3.6-flash but no gemini-3.6-flash-lite; the newest Flash-Lite is gemini-3.5-flash-lite.

Batch API pricing is 50% of standard across models.

Model selection: default to Flash / Flash-Lite for extraction

For structured information extraction — schema-constrained JSON pulled out of documents — default to Flash or Flash-Lite. Reserve Pro for tasks needing genuine reasoning. Do not reach for Pro by default just because the task feels important.

Measured 2026-08-03 on the realpage project (SEC IPO prospectus extraction; ~16,700 input tokens/doc, ~350-600 output; identical prompt, identical 100 documents):

model	finds the target provision	quote-verification	judge	cost/doc	1,926-doc run
gemini-3.1-pro-preview	47%	97.9%	1.00	$0.0187	~$36
gemini-3.6-flash	62%	95.2%	0.85	$0.0151	~$29
gemini-3.5-flash	70%	94.3%	1.00	$0.0149	~$29

Pro was the most conservative extractor, not the best one. It found the target provision in 47% of documents where Flash found 62-70% of the same documents. On an extraction task Pro's extra reasoning showed up as under-extraction — the failure mode that silently biases a research dataset. Scored against held-out human hand-coding (20 rows the research team coded before the pipeline existed, never having seen a machine output), all four models were identical — 85.0% exact agreement, 90% recall on real entitlements, 80% exact on those — and they failed on the same three rows. So Pro's extra reasoning bought nothing measurable, while its conservatism cost 15-23 points of detection.

Two honest caveats. The human sample was small (20 rows, 11 companies), and identical failures on identical rows says the residual errors were structural — a provision filed in an exhibit rather than the prospectus, a right held through a GP entity — not model quality. And the detection gap itself stayed unresolved: on the documents where models disagreed there was no ground truth, so which model is right on that 23-point spread was still open. Do not read this table as "Flash is more accurate"; read it as "Pro was not measurably better, and was measurably quieter."

Cost savings from Pro → Flash are smaller than people expect when the task is input-dominated. Here it was only ~20%, because Flash input is $0.75/1M against Pro's $1.00, while output — where Flash is much cheaper — was a rounding error at ~350 tokens. Flash-Lite is the only tier that cuts input price materially ($0.15/1M, ~85% saving). Work out whether the job is input- or output-dominated before assuming a Flash switch saves real money: compute mean_input_tokens * input_price vs mean_output_tokens * output_price from a Stage 2 sample (see references/scale-up-testing.md).

Model	Use Case	Cost	Location	Thinking default
gemini-2.5-flash-lite	Most batch jobs	Lowest	us-central1	OFF
gemini-2.5-flash	Complex extraction	Medium	us-central1	OFF
gemini-2.5-pro	Highest accuracy	Highest	us-central1	ON (cannot disable)
gemini-3-flash-preview	New gen, larger context	5× flash-lite	global	HIGH (set MINIMAL!)
gemini-3.1-flash-lite-preview	Cheapest gen-3	~2× 2.5 flash-lite	global	HIGH (set MINIMAL!)
gemini-embedding-001	Default for text-only (short titles, classification, retrieval over text)	Low	Standard API	n/a
gemini-embedding-2	Multimodal (text+image) inputs	Low	Standard API	n/a
text-embedding-005	Need Vertex Batch console visibility (legacy)	Low	us-central1	n/a

Critical for Gemini 3.x: Always pin thinkingConfig: {thinkingLevel: ...} in generationConfig or batch responses will silently fail with MAX_TOKENS and empty content. The level is not the same across tiers: Flash and Flash-Lite accept MINIMAL, but Pro rejects it ("Thinking level MINIMAL is not supported for this model", verified 2026-08-03) and needs LOW. Use a helper that picks the level per model — a single hardcoded constant breaks when you switch tiers. See references/gotchas.md Gotcha 17.

Critical for embedding batches: Embedding work has its own rules and failure modes — use file-based JSONL with per-row key on the Standard API; never inlined_requests (scrambles order at scale). Default to gemini-embedding-001 for text-only tasks. See references/embeddings.md and examples/embeddings_batch.py.

Additional Resources

References

• references/embeddings.md - NEW: Dedicated reference for embedding batches (model choice, file-based + keyed pattern, sentinel verification)
• references/gcs-setup.md - Complete GCS and Vertex AI setup guide
• references/gotchas.md - 20 critical production gotchas (Gemini 3.x thinking_level per tier, location='global'; embedding gotcha now lives in embeddings.md)
• references/best-practices.md - Idempotent IDs, state tracking, validation
• references/scale-up-testing.md - Incremental scale-up testing (LangExtract prototyping, LLM-as-judge, Vertex AI batch, gate design, input- vs output-dominated cost)
• references/troubleshooting.md - Common errors and debugging
• references/vertex-ai.md - Enterprise alternative with comparison
• references/cli-reference.md - gsutil and gcloud commands
• references/files-api.md - Files API: upload, poll-until-ACTIVE, 48h expiry, size limits
• references/file-search.md - File Search (managed RAG): store creation, metadata filtering, grounding metadata
• references/structured-output.md - responseJsonSchema / responseSchema: the supported schema subset, enums

Examples

• examples/icon_batch_vision.py - NEW: Batch vision analysis with Vertex AI
• examples/batch_processor.py - Complete GeminiBatchProcessor class
• examples/embeddings_batch.py - NEW: gemini-embedding-2 via client.batches.create_embeddings() (the only supported production path; Vertex Batch rejects this model)
• examples/pipeline_template.py - Customizable pipeline template

Scripts

• scripts/validate_jsonl.py - Validate JSONL before submission
• scripts/test_single.py - Test single request before batch

External Documentation

• Gemini Batch API Guide
• Google Cloud Storage
• Vertex AI Batch Prediction

Date Awareness

Gemini API evolves rapidly. For API features or model names with uncertainty, verify against current documentation.