to-be-continuous Component Creation Guide

Guide users through creating production-ready to-be-continuous components that follow ecosystem conventions, architectural principles, and best practices.

Overview

to-be-continuous components follow standardized patterns enabling composability, reusability, and maintainability across projects. Understanding component structure, naming conventions, and design principles is essential for creating components compatible with the ecosystem.

Key concepts:

• Single-responsibility principle per component
• Component-based architecture with spec.inputs
• Variant patterns for different authentication methods
• Base job inheritance for extensibility
• Convention over configuration
• Adaptive pipeline strategy

Core Philosophy

Single Responsibility & Modularity

Each component handles ONE specific pipeline stage (build, test, package, deploy, analyze). Components cooperate gracefully through:

• Shared artifact formats (Maven artifacts → Docker builds)
• Standardized dotenv variable exports
• Common stage naming conventions
• Reusable tool outputs (SonarQube reusing test reports)

Adaptive Pipeline Strategy

"Prioritize speed in early development stages, gradually introduce Quality & Security tasks as you get closer to production."

Implementation:

• Development branches: Manual quality checks, allowed to fail
• Draft MRs: Auto-run quality checks, allowed to fail
• Ready MRs: Auto-run quality checks, must succeed
• Production/integration branches: All checks required

Composability Over Integration

Components integrate through:

• Standardized stage sequence (11 stages)
• Dotenv artifact propagation
• Common variable naming patterns
• Shared authentication mechanisms

Component Directory Structure

Every to-be-continuous component follows this organizational pattern:

component-name/
├── templates/
│   ├── gitlab-ci-component.yml        # Standard variant (required)
│   ├── gitlab-ci-component-vault.yml  # Vault variant (optional)
│   ├── gitlab-ci-component-oidc.yml   # OIDC variant (optional)
│   ├── gitlab-ci-component-gcp.yml    # GCP variant (optional)
│   ├── gitlab-ci-component-aws.yml    # AWS/EKS variant (optional)
│   └── gitlab-ci-component-ecr.yml    # ECR variant (optional)
├── README.md                          # Component documentation (required)
├── CHANGELOG.md                       # Version history (required)
├── .gitlab-ci.yml                     # Component's own CI/CD (required)
└── LICENSE                            # MIT license (recommended)

Critical rules:

1. templates/ directory: MUST contain all component YAML files
2. Standard variant: MUST exist (gitlab-ci-component.yml)
3. Variant suffixes: Use established patterns (-vault, -oidc, -gcp, -aws, -eks, -ecr)
4. README.md: MUST document inputs, examples, variants
5. CHANGELOG.md: MUST track version changes

Component YAML Structure

Required Sections

Every component YAML file MUST contain:

1. spec: Input/output specification
2. Base job: Hidden job (.component-base) extended by all jobs
3. Jobs: Concrete jobs that execute in pipeline stages
4. Scripts: Reusable bash functions (via YAML anchors)

Standard Template Structure

# Component: component-name
# Version: X.Y.Z
# Description: [What this component does]
# Documentation: https://to-be-continuous.gitlab.io/doc/ref/component-name

spec:
  inputs:
    # Required inputs (no default)
    required-input:
      description: "Description of required input"
      type: string

    # Optional inputs (with defaults)
    optional-input:
      description: "Description of optional input"
      type: string
      default: "default-value"

    # Boolean inputs
    feature-enabled:
      description: "Enable specific feature"
      type: boolean
      default: false

    # Enumerated inputs
    build-tool:
      description: "Tool to use for building"
      type: string
      options:
        - tool1
        - tool2
        - default
      default: "default"

# Reusable script library
.scripts: &component-scripts
  - |
    # Logging functions
    log_info() { echo -e "\033[1;34m[INFO]\033[0m $*"; }
    log_warn() { echo -e "\033[1;33m[WARN]\033[0m $*"; }
    log_error() { echo -e "\033[1;31m[ERROR]\033[0m $*"; }

    # Component-specific functions
    component_function() {
      # Function implementation
    }

# Hidden base job - extended by all component jobs
.component-base:
  image: appropriate/image:tag
  variables:
    # Map inputs to variables for backward compatibility
    COMPONENT_VAR: $[[ inputs.required-input ]]
    COMPONENT_FEATURE: $[[ inputs.feature-enabled ]]
  before_script:
    - *component-scripts
  rules:
    - !reference [.tbc-workflow-rules, skip-back-merge]
    - !reference [.tbc-workflow-rules, prefer-mr-pipeline]
    - !reference [.tbc-workflow-rules, extended-skip-ci]

# Concrete jobs
component-build:
  stage: build
  extends: .component-base
  script:
    - log_info "Building with $COMPONENT_VAR"
    - component_function
  artifacts:
    reports:
      dotenv: component.env
  rules:
    - when: on_success

component-test:
  stage: test
  extends: .component-base
  script:
    - log_info "Testing component"
  needs:
    - component-build
  rules:
    - !reference [.test-policy, rules]

component-publish:
  stage: publish
  extends: .component-base
  script:
    - log_info "Publishing component"
  needs:
    - component-build
    - component-test
  rules:
    - !reference [.delivery-policy, rules]

Input Specification Best Practices

Dual Syntax Support

Components MUST support BOTH input syntaxes for backward compatibility:

Modern (Component Inputs):

include:
  - component: $CI_SERVER_FQDN/to-be-continuous/component/gitlab-ci-component@1.0.0
    inputs:
      build-tool: "maven"
      enable-tests: true

Legacy (Variables):

include:
  - project: 'to-be-continuous/component'
    ref: '1.0.0'
    file: '/templates/gitlab-ci-component.yml'

variables:
  COMPONENT_BUILD_TOOL: "maven"
  COMPONENT_ENABLE_TESTS: "true"

Implementation pattern:

spec:
  inputs:
    build-tool:
      type: string
      default: "default"

.component-base:
  variables:
    # Input maps to variable
    COMPONENT_BUILD_TOOL: $[[ inputs.build-tool ]]

Naming Conventions

Inputs (kebab-case):

• build-tool, snapshot-image, registry-mirror
• Use hyphens to separate words
• All lowercase

Variables (SCREAMING_SNAKE_CASE):

• COMPONENT_BUILD_TOOL, COMPONENT_SNAPSHOT_IMAGE, COMPONENT_REGISTRY_MIRROR
• Use COMPONENT_ prefix
• All uppercase with underscores

Input Types

spec:
  inputs:
    # String input
    string-input:
      type: string
      description: "Single value input"
      default: "default-value"

    # Number input
    number-input:
      type: number
      description: "Numeric value"
      default: 3

    # Boolean input
    boolean-input:
      type: boolean
      description: "True/false flag"
      default: false

    # Enumerated input
    choice-input:
      type: string
      description: "Select from options"
      options:
        - option1
        - option2
        - option3
      default: "option1"

Required vs Optional Inputs

Required inputs (no default):

• Critical for component functionality
• Component fails if not provided
• Example: deployment target, required credentials

Optional inputs (with default):

• Customization and tuning
• Sensible defaults for common use cases
• Example: image versions, feature flags, timeouts

Best practice: Minimize required inputs. Provide defaults whenever possible.

Base Job Patterns

Hidden Base Job

Purpose: Centralize configuration affecting all component jobs

Pattern:

.component-base:
  image: $[[ inputs.image ]]
  tags:
    - docker
  variables:
    VAR1: $[[ inputs.var1 ]]
    VAR2: "default-value"
  before_script:
    - *component-scripts
    - setup_function
  rules:
    - !reference [.tbc-workflow-rules, skip-back-merge]
    - !reference [.tbc-workflow-rules, prefer-mr-pipeline]
    - !reference [.tbc-workflow-rules, extended-skip-ci]

Users can override:

.component-base:
  tags:
    - kubernetes
  variables:
    http_proxy: "http://proxy:8080"

Tool-Specific Base Jobs

For components supporting multiple tools:

.component-base:
  # Common configuration

.component-tool1-base:
  extends: .component-base
  image: tool1/image:latest
  variables:
    TOOL: "tool1"

.component-tool2-base:
  extends: .component-base
  image: tool2/image:latest
  variables:
    TOOL: "tool2"

component-build-tool1:
  extends: .component-tool1-base
  script:
    - tool1 build

component-build-tool2:
  extends: .component-tool2-base
  script:
    - tool2 build

Job Naming Conventions

Format

[component]-[action] or [component]-[tool]-[action]

Examples:

• docker-build, docker-publish, docker-trivy
• maven-compile, maven-test, maven-deploy
• k8s-review, k8s-staging, k8s-production

Action Verbs

Use consistent verbs:

• build: Compile, assemble artifacts
• test: Unit tests, integration tests
• scan: Security scanning, vulnerability analysis
• lint: Code linting, static analysis
• package: Create distributable format
• publish: Upload to registry
• deploy: Deploy to environment
• cleanup: Remove resources

Environment Jobs

For deployment components:

• component-review (ephemeral review environments)
• component-integration (integration testing environment)
• component-staging (pre-production environment)
• component-production (production environment)

Pipeline Stages

Standard 11-Stage Sequence

stages:
  - .pre
  - build
  - test
  - package-build
  - package-test
  - infra
  - deploy
  - acceptance
  - publish
  - infra-prod
  - production
  - .post

Stage assignments:

• build: Compile code, run unit tests
• test: Additional testing (integration, contract)
• package-build: Build containers, packages
• package-test: Test packages (security scans, healthchecks)
• infra: Provision non-prod infrastructure
• deploy: Deploy to non-prod environments
• acceptance: Functional, performance, security tests
• publish: Promote artifacts to release registries
• infra-prod: Provision production infrastructure
• production: Deploy to production

Custom stages: Can be inserted (e.g., code-analysis between test and package-build)

Variant Patterns

Common Variants

Variant	Suffix	Authentication	Use Case	Components
Standard	(none)	CI/CD variables	Simple deployments	All (62)
Vault	-vault	HashiCorp Vault JWT/AppRole	Enterprise secrets management	~20
OIDC	-oidc	OpenID Connect	Temporary cloud credentials	AWS, Azure, GCloud
GCP	-gcp	Workload Identity Federation	GKE deployments, Artifact Registry	Docker, K8s, Helm, S3, Terraform
AWS/EKS	-aws/-eks	OIDC with IAM roles	EKS deployments, ECR registry	K8s, Docker
ECR	-ecr	OIDC with IAM	ECR-specific registry auth	Docker

Consult ../template-discovery/references/variantes.md for complete catalog (70+ variants documented).

When to Create a Variant

Create a variant when:

• Authentication method differs: Vault vs static credentials
• Platform integration required: GCP Workload Identity, AWS OIDC
• Security requirements change: OIDC temporary credentials vs long-lived tokens
• Deployment strategy differs: Different cloud provider SDKs

Don't create variant for:

• Configuration differences (use inputs/variables)
• Optional features (use boolean inputs)
• Minor behavior changes (use conditional logic)

Variant Implementation Pattern

Step 1: Copy standard variant

cp gitlab-ci-component.yml gitlab-ci-component-vault.yml

Step 2: Modify authentication mechanism

# Standard variant uses CI/CD variables
variables:
  REGISTRY_TOKEN: $[[ inputs.registry-token ]]

# Vault variant fetches from Vault
variables:
  REGISTRY_TOKEN: '@url@http://vault-secrets-provider/api/secrets/registry?field=token'

Step 3: Add variant-specific inputs

spec:
  inputs:
    vault-addr:
      description: "Vault server address"
      type: string
    vault-role:
      description: "Vault role for authentication"
      type: string
      default: "gitlab-ci"

Step 4: Update README with variant documentation

Step 5: Add to variantes.md catalog

Variable Conventions

Variable Scoping

Three levels of variables:

1. Global Component Variables

variables:
  COMPONENT_VERSION: "1.0.0"
  COMPONENT_DEFAULT_IMAGE: "alpine:latest"

2. Job-Level Variables

.component-base:
  variables:
    BASE_VAR: "value"

component-build:
  variables:
    BUILD_SPECIFIC_VAR: "value"

3. Scoped Variables (conditional)

variables:
  S3_BUCKET: "nonprod-bucket"
  scoped__S3_BUCKET__if__CI_ENVIRONMENT_NAME__equals__production: "prod-bucket"

Secret Handling

Standard secrets (CI/CD variables):

variables:
  SECRET_VAR: $SECRET_FROM_CICD_VARS

Unmaskable secrets (Base64 encoding):

variables:
  SECRET_WITH_SPECIAL_CHARS: '@b64@eyJvcGVuIjoiJOKCrDVAbWUifQ=='

External secrets (Vault, URL):

variables:
  VAULT_SECRET: '@url@http://vault-secrets-provider/api/secrets/path?field=name'

Secret evaluation (in scripts):

eval_secret() {
  local var_name="$1"
  local var_value="${!var_name}"

  # Decode @b64@ prefix
  if [[ "$var_value" =~ ^@b64@(.+)$ ]]; then
    echo "${BASH_REMATCH[1]}" | base64 -d
    return
  fi

  # Fetch @url@ prefix
  if [[ "$var_value" =~ ^@url@(.+)$ ]]; then
    curl -sSf "${BASH_REMATCH[1]}"
    return
  fi

  # Return as-is
  echo "$var_value"
}

Artifact Export Patterns

DotEnv Artifacts

Export variables for downstream jobs:

component-build:
  script:
    - build_command
    - |
      {
        echo "component_version=${VERSION}"
        echo "component_artifact=${ARTIFACT_PATH}"
        echo "component_digest=$(sha256sum artifact | cut -d' ' -f1)"
      } > component.env
  artifacts:
    reports:
      dotenv: component.env

Downstream consumption:

component-deploy:
  needs:
    - component-build
  script:
    - echo "Deploying version ${component_version}"
    - echo "Artifact digest: ${component_digest}"

Standard Output Variables

Naming convention: component_attribute

Examples:

• docker_image, docker_tag, docker_digest
• maven_artifact, maven_version
• k8s_namespace, k8s_service_url

Benefits:

• Declarative deployment pipelines
• Version propagation across stages
• Artifact traceability

Rules and Conditions

Standard Rule References

Components MUST include standard workflow rules:

.component-base:
  rules:
    - !reference [.tbc-workflow-rules, skip-back-merge]
    - !reference [.tbc-workflow-rules, prefer-mr-pipeline]
    - !reference [.tbc-workflow-rules, extended-skip-ci]

Explanation:

• skip-back-merge: Prevent pipelines on automated back-merges
• prefer-mr-pipeline: Use MR pipelines over branch pipelines when both exist
• extended-skip-ci: Support [skip ci on tag], [skip ci on mr], etc.

Adaptive Pipeline Policies

Test jobs (quality/security):

component-test:
  rules:
    - !reference [.test-policy, rules]

Behavior:

• Tag/protected branches: Auto-run, must succeed
• Dev branch (no MR): Manual, allowed to fail
• Draft MR: Auto-run, allowed to fail
• Ready MR: Auto-run, must succeed

Delivery jobs (publish/production):

component-publish:
  rules:
    - !reference [.delivery-policy, rules]

Behavior:

• Execute on release tags (SemVer)
• Execute on production/integration branches
• Skip elsewhere

Custom Rules

component-manual-job:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
      when: never
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
      when: manual
    - when: never

Script Best Practices

Logging Functions

.scripts: &component-scripts
  - |
    # ANSI color codes
    log_info() { echo -e "\033[1;34m[INFO]\033[0m $*"; }
    log_warn() { echo -e "\033[1;33m[WARN]\033[0m $*"; }
    log_error() { echo -e "\033[1;31m[ERROR]\033[0m $*"; }
    log_success() { echo -e "\033[1;32m[SUCCESS]\033[0m $*"; }

Tool Installation Helpers

maybe_install_tool() {
  if ! command -v tool &> /dev/null; then
    log_info "Installing tool..."
    # Installation logic
  fi
}

Multi-Distro Compatibility

detect_package_manager() {
  if command -v apt-get &> /dev/null; then
    echo "apt-get"
  elif command -v yum &> /dev/null; then
    echo "yum"
  elif command -v apk &> /dev/null; then
    echo "apk"
  else
    log_error "No supported package manager found"
    return 1
  fi
}

Error Handling

set -euo pipefail  # Exit on error, undefined vars, pipe failures

# Function with error handling
safe_operation() {
  if ! risky_command; then
    log_error "Command failed"
    return 1
  fi
  log_success "Command succeeded"
}

Component Naming

Component Names

• kebab-case: docker, semantic-release, kubernetes, cloud-native-buildpacks
• Descriptive: Clearly indicate technology/purpose
• Match technology: maven for Maven, terraform for Terraform
• No abbreviations: kubernetes not k8s (exception: widely known like s3)

Template Files

Pattern: gitlab-ci-[component][-variant].yml

Examples:

• gitlab-ci-docker.yml (standard)
• gitlab-ci-docker-vault.yml (Vault variant)
• gitlab-ci-docker-gcp.yml (GCP variant)
• gitlab-ci-kubernetes-aws.yml (AWS variant)

Documentation Requirements

README.md Structure

# Component Name

Brief description of what component does.

## Inputs

| Input | Type | Default | Description |
|-------|------|---------|-------------|
| `input-name` | string | `default` | Description |

## Variants

- **Standard**: Description
- **Vault**: Description
- **OIDC**: Description

## Example

\`\`\`yaml
include:
  - component: $CI_SERVER_FQDN/to-be-continuous/component/gitlab-ci-component@1.0.0
    inputs:
      example-input: "value"
\`\`\`

## Variables

| Variable | Description | Default |
|----------|-------------|---------|
| `COMPONENT_VAR` | Description | `value` |

## Jobs

- **component-build**: Description
- **component-test**: Description

## Integration

How component integrates with other components.

## License

MIT

CHANGELOG.md

Follow Keep a Changelog:

# Changelog

## [1.1.0] - 2024-01-15
### Added
- New input for feature X
- Support for variant Y

### Changed
- Updated default image to v2.0

### Fixed
- Bug in authentication flow

## [1.0.0] - 2023-12-01
### Added
- Initial release

Versioning Strategy

Semantic Versioning

Components MUST follow SemVer:

• MAJOR: Breaking changes (incompatible inputs, removed jobs)
• MINOR: New features (new inputs, new jobs, backward compatible)
• PATCH: Bug fixes (no new functionality)

Version Tags

Create Git tags for releases:

git tag -a v1.2.3 -m "Release version 1.2.3"
git push origin v1.2.3

Version Aliases

GitLab automatically creates aliases:

• 1 → latest 1.x.x
• 1.2 → latest 1.2.x
• 1.2.3 → exact version

Users can pin:

# Always get latest patches
component: .../component@1.2

# Pin exact version
component: .../component@1.2.3

Testing Components

Sample Project

Create sample project demonstrating component usage:

samples/component-sample/
├── .gitlab-ci.yml          # Uses component
├── README.md               # Explains sample
├── src/                    # Sample source code
└── tests/                  # Sample tests

Sample .gitlab-ci.yml:

include:
  - component: $CI_SERVER_FQDN/to-be-continuous/component/gitlab-ci-component@1.0.0
    inputs:
      build-tool: "maven"
      enable-tests: true

Validation Checklist

Before releasing component:

• All variants tested
• README documentation complete
• CHANGELOG updated
• Input specifications validated
• Sample project created
• Semantic versioning applied
• Security scan passed
• Component CI/CD passes

Reference Files

For detailed component patterns and examples:

• ../template-discovery/references/catalog.md: 62 existing components to study
• ../template-discovery/references/variantes.md: 70+ variant implementations across components
• ../template-discovery/references/usage-guide.md: Component syntax, scoped variables, secrets management
• ../template-discovery/references/best-practices.md: Architectural patterns (Review Apps, GitOps, repo structure)

Common Patterns

Minimal Component

Single job, single stage:

spec:
  inputs:
    target:
      type: string

.component-base:
  image: alpine:latest
  rules:
    - !reference [.tbc-workflow-rules, skip-back-merge]
    - !reference [.tbc-workflow-rules, prefer-mr-pipeline]

component-job:
  stage: deploy
  extends: .component-base
  script:
    - echo "Deploying to $[[ inputs.target ]]"

Multi-Tool Component

Support multiple build tools:

spec:
  inputs:
    build-tool:
      type: string
      options: [tool1, tool2, default]
      default: "default"

.component-base:
  variables:
    TOOL: $[[ inputs.build-tool ]]

component-build-tool1:
  extends: .component-base
  rules:
    - if: '$TOOL == "tool1"'
  script:
    - tool1 build

component-build-tool2:
  extends: .component-base
  rules:
    - if: '$TOOL == "tool2"'
  script:
    - tool2 build

Environment Deployment Component

Deploy to multiple environments:

.component-base:
  image: kubectl:latest
  script:
    - kubectl apply -f manifests/

component-review:
  extends: .component-base
  stage: deploy
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    on_stop: component-review-cleanup

component-staging:
  extends: .component-base
  stage: deploy
  environment:
    name: staging

component-production:
  extends: .component-base
  stage: production
  environment:
    name: production
  rules:
    - !reference [.delivery-policy, rules]

Contributing Components

Workflow

1. Discuss on Discord: Post message with topic New template: component-name
2. Core team creates project: From template skeleton repository
3. Fork and branch: Work from initial-component branch
4. Implement component: Follow patterns in this guide
5. Create MR: Mark as Draft until ready
6. Review: At least 2 core team approvals required
7. Merge: Automatic release on merge

Commit Standards

• Atomic commits: One logical change per commit
• Semantic commits: Follow Conventional Commits
• Signed-off: Include Signed-off-by: line (DCO)

Examples:

feat: add OIDC authentication variant
fix: correct environment variable mapping
docs: update README with new inputs

Review Criteria

• Follows naming conventions
• Includes all required files
• Documentation complete
• Sample project provided
• All variants tested
• Follows to-be-continuous philosophy

Summary: Key Behaviors

STRUCTURED:

• Follow standard directory layout
• Use established naming conventions
• Include all required files (README, CHANGELOG)

COMPATIBLE:

• Support dual syntax (inputs + variables)
• Use standard workflow rules
• Export dotenv artifacts for downstream jobs
• Follow 11-stage pipeline sequence

MODULAR:

• Single responsibility per component
• Hidden base job for extensibility
• Variant pattern for authentication methods
• Composable with other components

DOCUMENTED:

• Clear input specifications
• Usage examples in README
• Variant documentation
• Version changelog

TESTED:

• Sample project demonstrating usage
• All variants validated
• Component CI/CD passing
• Security scanning enabled

Remember: Study existing components in catalog.md and variantes.md for proven patterns. When in doubt, reference similar existing components for guidance.