Creating Pipeline Templates

Templates are YAML definitions + seed files in lib/templates/. Auto-discovered on startup by TemplateRegistry (lib/templates/__init__.py).

• Template ID = filename without .yaml
• Seed file: seed_<template_id>.json or seed_<template_id>.md

Template YAML Format

name: Template Display Name
description: What this template generates
blocks:
  - type: BlockClassName       # must match class name exactly
    config:
      param1: value1           # must match __init__ parameter names exactly
      user_prompt: "{{ var }}" # Jinja2 references to seed metadata
  - type: AnotherBlock
    config:
      field_name: generated

Seed File Format

JSON (most templates):

[
  {"repetitions": 3, "metadata": {"content": "input text here"}}
]

Markdown (only for MarkdownMultiplierBlock as first block):

• File: seed_<template_id>.md
• Registry auto-wraps as [{"repetitions": 1, "metadata": {"file_content": "<content>"}}]

Available Blocks

Common Pipeline Patterns

# simple generation + validation
StructuredGenerator → JSONValidatorBlock

# document processing (multiplier first)
MarkdownMultiplierBlock → TextGenerator → StructuredGenerator → JSONValidatorBlock

# data augmentation
StructureSampler → SemanticInfiller → DuplicateRemover

# generation + metrics
StructuredGenerator → FieldMapper → RagasMetrics

# generation + review-friendly output
StructuredGenerator → FieldMapper (flatten for review)

Adding a FieldMapper for Review

The Review page displays records from the last block's accumulated_state. Only first-level keys are shown as primary/secondary fields. Nested objects (e.g. generated.confirmed_dependencies) appear as raw JSON strings and can't be configured as separate review fields.

Always add a FieldMapper as the last block to surface the fields reviewers need at the top level.

Why it matters

Without a FieldMapper, the accumulated_state after a StructuredGenerator looks like:

{
  "input_field": "...",
  "generated": {
    "question": "...",
    "answer": "...",
    "contexts": ["..."]
  }
}

The review UI sees input_field and generated (a blob). Reviewers can't configure question or answer as primary fields.

How to add it

Add a FieldMapper as the last block (or last before metrics/observability blocks):

  - type: FieldMapper
    config:
      mappings:
        # Flatten nested fields to top level
        question: "{{ generated.question }}"
        answer: "{{ generated.answer }}"
        # tojson is safe only for structured data (IDs, numbers, short labels)
        # avoid tojson on arrays/objects with free-text — newlines/quotes break JSON parsing
        context_count: "{{ generated.contexts | length }}"
        # Carry forward useful seed metadata
        source: "{{ source_document }}"

Rules

1. Map every field the reviewer needs — if it's not a first-level key after the last block, it won't be configurable in the review field settings
2. Use | tojson for arrays/objects — FieldMapper auto-parses JSON strings back to objects, so the review UI can display them properly. Exception: tojson on arrays/objects whose values contain unescaped quotes or newlines (e.g. free-text descriptions) will break FieldMapper JSON parsing. In that case, map only scalar summaries (counts, IDs) and let the array flow through as an existing first-level key.
3. Use | length for counts — gives reviewers a quick numeric summary without expanding lists
4. Use | default('') for optional fields — prevents Jinja2 errors when a field is missing
5. Don't map internal/noisy fields — skip folder_path, _usage, _seed_samples etc. Only map what's useful for human review
6. Order matters — FieldMapper outputs merge into accumulated_state, so its keys become the available fields in the Review "Configure Fields" modal

Step-by-Step Workflow

1. Define use case — what data to generate, what fields in output, what seed input needed
2. Choose blocks — pick from table above, wire outputs to inputs
3. Write YAML — lib/templates/<template_id>.yaml
4. Write seed file — match {{ variables }} in YAML to metadata keys
5. Validate template loads:

   uv run python -c "
   from lib.templates import template_registry
   for t in template_registry.list_templates():
       print(f'{t[\"id\"]}: {t[\"name\"]}')
   "
   

6. Check block params (if unsure about config keys):

   uv run python -c "
   from lib.blocks.registry import BlockRegistry
   registry = BlockRegistry()
   for name, cls in registry._blocks.items():
       schema = cls.get_schema()
       print(f'{name}: {list(schema.get(\"config_schema\", {}).get(\"properties\", {}).keys())}')
   "
   

7. Test single execution:

   # create pipeline from template
   curl -s -X POST http://localhost:8000/api/pipelines/from_template/<template_id> | python -m json.tool
   # execute with seed
   curl -s -X POST http://localhost:8000/api/pipelines/<id>/execute \
     -H 'Content-Type: application/json' \
     -d '{"content": "test input"}' | python -m json.tool
   

Reference Templates

Common Mistakes

Checklist

• YAML in lib/templates/ with correct block types and config keys
• Seed file matching template ID with all referenced variables
• Template loads via TemplateRegistry
• Single execution produces expected output fields
• Trace shows all blocks executed successfully
• Seed file has 2-3 diverse examples
• FieldMapper as last block flattens outputs for Review UI (all reviewer-relevant fields are top-level keys)

Related Skills

• implementing-datagenflow-blocks — creating new block types
• debugging-pipelines — troubleshooting template execution
• testing-pipeline-templates — thorough end-to-end testing