ytt: YAML Templating with Starlark

ytt Deep Dive

ytt (pronounced “why-tee-tee”) is a YAML-aware templating tool from the Carvel project (VMware Tanzu). Unlike text-based templaters, ytt understands YAML structure, enabling safer and more powerful configuration management.

Note

ytt uses a customized fork of starlark-go (carvel-dev/starlark-go) with block-based syntax support. This enables clean YAML-embedded templates without Python-style whitespace sensitivity.

The Problem with Text Templating

Traditional templating (like Helm’s Go templates) treats YAML as plain text:

# Helm-style text templating - fragile!
args:
- "-listen=:{{ .Values.port }}"     # Manual quoting
- {{ if .Values.debug }}-debug{{ end }}  # Indentation issues
env:
{{- range $k, $v := .Values.env }}  # Complex syntax
  - name: {{ $k }}
    value: {{ $v | quote }}
{{- end }}

Common issues:

• Manual quoting and escaping
• Indentation errors break YAML structure
• Template errors caught only at render time
• Hard to compose and test

ytt’s Structural Approach

ytt understands YAML structure natively:

#@ load("@ytt:data", "data")

args:
- #@ "-listen=:" + str(data.values.port)
- #@ "-debug" if data.values.debug else None
env:
#@ for k, v in data.values.env.items():
- name: #@ k
  value: #@ v
#@ end

Benefits:

• YAML structure preserved automatically
• No quoting/escaping needed
• Type-safe values
• Errors caught at template evaluation time

---

Core Concepts

Annotations (#@ Directives)

ytt uses comment annotations to embed Starlark:

Annotation	Purpose
#@ expr	Inline Starlark expression
#@ load(...)	Import modules
#@ def fn(): ... #@ end	Define functions
#@ if cond: ... #@ end	Conditionals
#@ for x in items: ... #@ end	Loops
#@data/values-schema	Mark as schema file
#@overlay/...	Overlay operations
#! comment	Preserved comment

The @ytt: Module Namespace

ytt provides domain-specific modules:

#@ load("@ytt:data", "data")        # Data values
#@ load("@ytt:overlay", "overlay")  # Patching
#@ load("@ytt:template", "template") # Template utils
#@ load("@ytt:library", "library")  # Library loading
#@ load("@ytt:assert", "assert")    # Validation
#@ load("@ytt:struct", "struct")    # Typed structs
#@ load("@ytt:yaml", "yaml")        # YAML encoding
#@ load("@ytt:json", "json")        # JSON encoding
#@ load("@ytt:base64", "base64")    # Base64 encoding
#@ load("@ytt:md5", "md5")          # MD5 hashing
#@ load("@ytt:sha256", "sha256")    # SHA256 hashing
#@ load("@ytt:url", "url")          # URL handling
#@ load("@ytt:regexp", "regexp")    # Regular expressions
#@ load("@ytt:ip", "ip")            # IP address parsing
#@ load("@ytt:math", "math")        # Math functions

---

Data Values System

Schema Definition

Define your configuration schema:

#@data/values-schema
---
#@schema/title "Application Configuration"
app:
  name: ""
  #@schema/desc "Number of replicas"
  replicas: 1
  #@schema/nullable
  image: ""

  resources:
    cpu: "100m"
    memory: "128Mi"

  #@schema/type any=True
  extra_config: {}

debug: false

Using Data Values

#@ load("@ytt:data", "data")

apiVersion: apps/v1
kind: Deployment
metadata:
  name: #@ data.values.app.name
spec:
  replicas: #@ data.values.app.replicas
  template:
    spec:
      containers:
      - name: app
        image: #@ data.values.app.image
        resources:
          requests:
            cpu: #@ data.values.app.resources.cpu
            memory: #@ data.values.app.resources.memory

Overriding Values

# Via CLI flags
ytt -f config/ -v app.name=myapp -v app.replicas=3

# Via values file
ytt -f config/ -f values-prod.yml

# Via environment
ytt -f config/ --data-values-env APP_

# Multiple sources (later overrides earlier)
ytt -f config/ -f base.yml -f prod.yml -v debug=true

---

Overlays: Surgical YAML Patching

Overlays modify existing YAML without changing the original files.

Basic Overlay Pattern

#@ load("@ytt:overlay", "overlay")

#! Match all documents
#@overlay/match by=overlay.all
---
metadata:
  #@overlay/match missing_ok=True
  annotations:
    managed-by: ytt

Overlay Annotations

Annotation	Purpose
#@overlay/match	Select nodes to modify
#@overlay/replace	Replace node value
#@overlay/insert	Insert new items
#@overlay/remove	Delete nodes
#@overlay/assert	Validate matches
#@overlay/match-child-defaults	Set defaults for children

Matching Strategies

#@ load("@ytt:overlay", "overlay")

#! Match by document subset
#@overlay/match by=overlay.subset({"kind": "Deployment"})
---
spec:
  template:
    spec:
      #@overlay/match missing_ok=True
      nodeSelector:
        env: production

#! Match by custom function
#@overlay/match by=lambda i,l,r: l["metadata"]["name"].endswith("-api")
---
metadata:
  labels:
    tier: api

#! Match with expectations
#@overlay/match by=overlay.subset({"kind": "Service"}), expects="1+"
---
spec:
  type: LoadBalancer

Array Operations

#@ load("@ytt:overlay", "overlay")

#@overlay/match by=overlay.subset({"kind": "Deployment"})
---
spec:
  template:
    spec:
      containers:
      #! Match container by name
      #@overlay/match by="name"
      - name: app
        #@overlay/match missing_ok=True
        env:
        #@overlay/append
        - name: NEW_VAR
          value: "added"

      #! Insert new container
      #@overlay/append
      - name: sidecar
        image: sidecar:latest

Remove Resources

#@ load("@ytt:overlay", "overlay")

#! Remove all ServiceAccounts
#@overlay/match by=overlay.subset({"kind": "ServiceAccount"}), expects="0+"
#@overlay/remove
---

---

Functions and Modules

Defining Functions

#@ def labels(name, component="app"):
app.kubernetes.io/name: #@ name
app.kubernetes.io/component: #@ component
app.kubernetes.io/managed-by: ytt
#@ end

---
apiVersion: v1
kind: Service
metadata:
  name: api
  labels: #@ labels("api", "backend")

Multi-Document Functions

#@ def namespace(name):
apiVersion: v1
kind: Namespace
metadata:
  name: #@ name
---
apiVersion: v1
kind: ResourceQuota
metadata:
  name: default
  namespace: #@ name
spec:
  hard:
    pods: "10"
#@ end

--- #@ template.replace(namespace("prod"))
--- #@ template.replace(namespace("staging"))

Library Modules

Create reusable libraries in _ytt_lib/:

config/
├── config.yml
└── _ytt_lib/
    └── @k8s/
        ├── deployment.lib.yml
        └── service.lib.yml

#! _ytt_lib/@k8s/deployment.lib.yml
#@ load("@ytt:struct", "struct")

#@ def deployment(name, image, replicas=1):
apiVersion: apps/v1
kind: Deployment
metadata:
  name: #@ name
spec:
  replicas: #@ replicas
  selector:
    matchLabels:
      app: #@ name
  template:
    metadata:
      labels:
        app: #@ name
    spec:
      containers:
      - name: main
        image: #@ image
#@ end

#@ k8s = struct.make(deployment=deployment)

#! config.yml
#@ load("@k8s:deployment.lib.yml", "k8s")

--- #@ k8s.deployment("api", "myapp/api:v1", replicas=3)
--- #@ k8s.deployment("worker", "myapp/worker:v1")

---

Assertions and Validation

Basic Assertions

#@ load("@ytt:assert", "assert")
#@ load("@ytt:data", "data")

#! Validate data values
#@ assert.equals(data.values.env, "prod", "staging", "dev")
#@ assert.min(data.values.replicas, 1)
#@ assert.max(data.values.replicas, 100)

#! Inline assertions
replicas: #@ assert.min(data.values.replicas, 1)
name: #@ data.values.name or assert.fail("name is required")

Schema Validation

#@data/values-schema
---
#@schema/validation min=1, max=100
replicas: 1

#@schema/validation one_of=["debug", "info", "warn", "error"]
log_level: "info"

#@schema/validation format="ip"
bind_address: "0.0.0.0"

Custom Validation Functions

#@ load("@ytt:assert", "assert")

#@ def validate_port(val):
#@   if val < 1 or val > 65535:
#@     assert.fail("port must be 1-65535")
#@   end
#@   return val
#@ end

port: #@ validate_port(data.values.port)

---

Real-World Examples

Multi-Environment Kubernetes

config/
├── schema.yml
├── base/
│   ├── deployment.yml
│   └── service.yml
├── overlays/
│   ├── prod.yml
│   └── dev.yml
└── values/
    ├── prod.yml
    └── dev.yml

#! schema.yml
#@data/values-schema
---
env: ""
app:
  name: myapp
  replicas: 1
  image: ""
  resources:
    cpu: "100m"
    memory: "128Mi"

#! values/prod.yml
#@data/values
---
env: prod
app:
  replicas: 3
  image: registry.prod/myapp:v1.2.3
  resources:
    cpu: "500m"
    memory: "512Mi"

#! overlays/prod.yml
#@ load("@ytt:overlay", "overlay")

#@overlay/match by=overlay.subset({"kind": "Deployment"})
---
spec:
  template:
    spec:
      #@overlay/match missing_ok=True
      nodeSelector:
        env: production
      #@overlay/match missing_ok=True
      tolerations:
      - key: "production"
        operator: "Exists"

# Deploy to production
ytt -f config/schema.yml \
    -f config/base/ \
    -f config/values/prod.yml \
    -f config/overlays/prod.yml | kubectl apply -f -

Helm Post-Rendering

ytt can post-process Helm output:

#!/bin/bash
# ytt-post-renderer
ytt -f - -f overlays/

helm template myrelease mychart/ --post-renderer ./ytt-post-renderer

ConfigMap from Files

#@ load("@ytt:data", "data")

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  #@ for path in data.list("configs/"):
  #@ filename = path.split("/")[-1]
  #@ content = data.read(path)
  _: #@ { filename: content }
  #@ end

Secret Generation

#@ load("@ytt:data", "data")
#@ load("@ytt:base64", "base64")
#@ load("@ytt:sha256", "sha256")

apiVersion: v1
kind: Secret
metadata:
  name: app-secrets
  annotations:
    #! Hash for change detection
    checksum: #@ sha256.sum(data.read("secrets.json"))
type: Opaque
data:
  config.json: #@ base64.encode(data.read("secrets.json"))

---

Starlark Fork: Why and How

ytt uses a customized fork of starlark-go (carvel-dev/starlark-go) rather than upstream. This wasn’t a casual decision - it solves a fundamental incompatibility between Starlark and YAML templating.

The Problem: Whitespace Conflicts

Standard Starlark uses Python-style significant whitespace:

# Standard Starlark - whitespace determines block structure
def greet(name):
    if name:
        return "Hello, " + name  # indentation required
    return "Hello"               # dedent closes block

But YAML also uses significant whitespace:

# YAML - indentation determines structure
spec:
  containers:    # child of spec
  - name: app    # list item

The conflict: If you embed Starlark in YAML templates, which whitespace rules win? This was raised in ytt Issue #8 - developers couldn’t indent YAML inside Starlark functions because the indentation would be interpreted as Starlark block structure.

The Solution: Block Scanner

In July 2019, ytt’s creator Dmitriy Kalinin added a blockScanner to their vendored starlark-go. This scanner:

1. Ignores whitespace for block detection - Doesn’t use INDENT/OUTDENT tokens based on spaces
2. Uses explicit end keyword - Blocks close with end, not dedentation
3. Handles else/elif specially - These implicitly close prior blocks
4. Forbids pass - Only end is allowed to close blocks

// From syntax/block_scanner.go
// blockScanner changes INDENT/OUTDENT to be
// based on nesting depth (start->end) instead of whitespace
type blockScanner struct {
    scanner     *scanner
    indentStack []blockScannerToken
    // ...
}

How It Works

The parser can be invoked with a BlockScanner mode flag:

// In parse.go
const (
    RetainComments Mode = 1 << iota
    BlockScanner   Mode = 1 << iota // use if/end syntax instead of indent
)

func Parse(filename string, src interface{}, mode Mode) (*File, error) {
    in, _ := newScanner(filename, src, mode&RetainComments != 0)
    var inScanner scannerInterface = in
    if (mode & BlockScanner) == BlockScanner {
        inScanner = newBlockScanner(in)  // Use block-based parsing
    }
    // ...
}

Block-Based Syntax in Practice

#@ if condition:
some_yaml: here
  nested:
    data: works
#@ end

#@ for item in items:
- name: #@ item
  value: #@ item.upper()
#@ end

#@ def my_function():
key: value
list:
- one
- two
#@ end

Notice how YAML indentation inside blocks is now purely YAML - not Starlark block structure.

Other Fork Changes

Beyond the block scanner, the fork:

1. Removed unused modules - Stripped lib/proto/, lib/json/, lib/math/, lib/time/ (~2000+ lines) since ytt provides its own @ytt:* modules
2. Simplified Thread model - Removed step counting, cancellation APIs (ytt doesn’t need execution limits)
3. Added Module type - starlarkstruct/module.go for creating named module collections
4. Enhanced Struct - Binary operations support for struct merging

Why Not Upstream?

The block scanner is ytt-specific. Standard Starlark needs whitespace sensitivity for:

• Bazel BUILD files (pure Starlark, no YAML embedding)
• General Python-like behavior expectations

The fork is kept minimal and synced with upstream where possible. The key commits:

fd88429 - moved over from ytt repo (add blockScanner, improve reserved keyword errors)
3e0f0a9 - Make ytt flavoured starlark embeddable into starlark applications
8a7b203 - fix handling of unbalanced end and if/for/...

Note

The fork decision reflects a pragmatic trade-off: modify Starlark’s parser to solve a real problem (YAML+code whitespace conflicts) rather than force awkward workarounds on users. The changes are isolated to parsing, not the language semantics.

---

Comparison with Alternatives

vs. Helm

Aspect	ytt	Helm
Templating	YAML-aware (structural)	Text-based (Go templates)
Package Management	None (use with imgpkg)	Built-in charts
Composition	Overlays + libraries	Values + subcharts
Type Safety	Schema validation	Limited
Learning Curve	Medium (Starlark)	Low-Medium

When to use ytt over Helm:

• Need structural YAML manipulation
• Want type-safe configuration
• Complex multi-environment setups
• Post-processing Helm output

vs. Kustomize

Aspect	ytt	Kustomize
Language	Starlark (full language)	YAML patches only
Complexity	Can be complex	Simpler, limited
Flexibility	Very high	Medium
Kubectl Integration	Separate	Built-in

When to use ytt over Kustomize:

• Need conditional logic
• Complex transformations
• Reusable functions
• Cross-cutting concerns

vs. Jsonnet

Aspect	ytt	Jsonnet
Target Format	YAML-first	JSON-first
Syntax	Python-like	Unique JSON-like
Learning Curve	Lower	Higher
Kubernetes Focus	Primary	General

---

Best Practices

1. Use Schemas

Always define data value schemas:

#@data/values-schema
---
#@schema/desc "Application name (required)"
name: ""

#@schema/desc "Deployment replicas"
#@schema/validation min=1
replicas: 1

2. Separate Concerns

config/
├── schema.yml          # Data value schema
├── base/               # Base configurations
├── overlays/           # Environment-specific patches
├── values/             # Environment values
└── _ytt_lib/           # Reusable libraries

3. Validate Early

#@ load("@ytt:assert", "assert")
#@ load("@ytt:data", "data")

#@ if not data.values.name:
#@   assert.fail("name is required")
#@ end

4. Use Libraries for Reuse

#@ load("@lib:k8s.lib.yml", "k8s")

--- #@ k8s.deployment(data.values)
--- #@ k8s.service(data.values)

5. Document with Comments

#! This overlay adds production-specific settings
#@ load("@ytt:overlay", "overlay")

#@overlay/match by=overlay.all, expects="1+"
---
#! Add resource limits for production
spec:
  #@overlay/match missing_ok=True
  resources:
    limits:
      memory: "1Gi"

---

Resources

Official

• Carvel ytt Documentation
• GitHub Repository
• Starlark Fork
• Carvel Project

Examples

• ytt Examples
• Carvel Examples

Related Tools

• kapp - Kubernetes application deployment
• kbld - Image building and resolution
• imgpkg - Bundle packaging
• vendir - Dependency vendoring