Schema migration strategies for evolving applications including adding new fields, backfilling data, removing deprecated fields, index migrations, and zero-downtime migration patterns
Produces a schema change plus a batched backfill that keeps every deploy green. The one rule: the schema must describe the documents that exist right now, not the documents you want. Change the data first, then tighten the validator.
Convex has no migration files or migrate command. npx convex dev pushes the schema and validates every existing document against it. Existing data is never transformed for you.
npx convex dev fails with Schema validation failed after a schema editFor step by step recipes (rename, change type, split, merge, add required, remove) open references/migration-patterns.md. For installing and running @convex-dev/migrations open references/migrations-component.md.
Every migration follows the same six steps. Skipping one is how deploys break.
convex/schema.ts (or add the new field as v.optional).npx convex dev. Update readers to handle undefined and writers to set the new shape.v.string() instead of v.optional(v.string()), or drop the old field).Steps 1 and 2 must ship before step 3 starts. Otherwise new writes keep producing old shaped documents while the backfill runs.
One internalMutation that pages through the table with paginate, patches only documents that still need it, then reschedules itself with the continue cursor. Each batch is its own transaction, so a large table never hits the per function limit.
// convex/migrations.ts
import { internalMutation } from "./_generated/server";
import { internal } from "./_generated/api";
import { v } from "convex/values";
const BATCH_SIZE = 100;
export const backfillAvatarUrl = internalMutation({
args: { cursor: v.union(v.string(), v.null()) },
returns: v.null(),
handler: async (ctx, args) => {
const result = await ctx.db
.query("users")
.paginate({ numItems: BATCH_SIZE, cursor: args.cursor });
for (const user of result.page) {
// Idempotent: skip documents that already have the field
if (user.avatarUrl === undefined) {
await ctx.db.patch(user._id, {
avatarUrl: defaultAvatar(user.name),
});
}
}
// Reschedule with the cursor until the table is exhausted
if (!result.isDone) {
await ctx.scheduler.runAfter(0, internal.migrations.backfillAvatarUrl, {
cursor: result.continueCursor,
});
}
return null;
},
});
function defaultAvatar(name: string): string {
return `https://api.dicebear.com/7.x/initials/svg?seed=${encodeURIComponent(name)}`;
}
Start it from the CLI against the dev deployment:
npx convex run migrations:backfillAvatarUrl '{"cursor": null}'
Keep the mutation idempotent. Re running it after a partial failure must be safe. Use internalMutation, never a public mutation, so it cannot be called from a client. Use ctx.db.patch for adding or changing fields and ctx.db.patch(id, { field: undefined }) to remove one.
@convex-dev/migrations wraps the pattern above and adds what the hand rolled version lacks: persisted progress per migration, status checks, dry runs, cancel, and ordered runs of several migrations. Reach for it when:
Minimal setup:
// convex/convex.config.ts
import { defineApp } from "convex/server";
import migrations from "@convex-dev/migrations/convex.config.js";
const app = defineApp();
app.use(migrations);
export default app;
// convex/migrations.ts
import { Migrations } from "@convex-dev/migrations";
import { components } from "./_generated/api";
import { DataModel } from "./_generated/dataModel";
export const migrations = new Migrations<DataModel>(components.migrations);
export const addDefaultRole = migrations.define({
table: "users",
migrateOne: async (ctx, user) => {
if (user.role === undefined) {
await ctx.db.patch(user._id, { role: "user" });
}
},
});
npx convex run migrations:addDefaultRole '{"dryRun": true}'
npx convex run migrations:addDefaultRole
A one off backfill on a small table does not need the component. The hand rolled mutation is fine.
When a schema push fails, npx convex dev prints the table, one offending document id, and the field that does not match:
Schema validation failed
Document with ID "j57abc..." in table "users" does not match the schema:
Object is missing the required field `avatarUrl`.
Consider wrapping the field validator in `v.optional(...)` if this is expected.
Read it as: existing data in users predates the field. The fix is never to delete the document. Wrap the field in v.optional, deploy, backfill, then remove the v.optional. If the error says the field has the wrong type (for example string where number is expected), widen the validator to v.union(v.string(), v.number()), backfill the conversion, then narrow it.
Do not pass { schemaValidation: false } to defineSchema to make the push succeed. It hides the mismatch and moves the failure into your queries.
| Mistake | Why it breaks | Do instead |
|---|---|---|
| Add a required field in one deploy | Existing documents fail validation, push is rejected | Add as v.optional, backfill, then require |
| Start the backfill before deploying the new writers | New rows keep arriving in the old shape | Deploy schema and code first, backfill second |
One mutation that .collect()s the whole table |
Hits the transaction size and time limits | Page with paginate and reschedule per batch |
Backfill with a public mutation |
Anyone can call it and re run it | internalMutation or migrations.define |
| Non idempotent patch | Re running after a failure double applies | Check the field before patching |
| Removing the old field before readers stop using it | Fallback code reads undefined |
Switch readers, then remove |
Using .filter() to find unmigrated rows |
Full scan on every batch | Paginate the whole table or use customRange with an index |
Turning off schemaValidation to pass the deploy |
Bad data reaches queries at runtime | Fix the data, keep validation on |
| Running a backfill against production first | No way to catch a wrong conversion | Run on dev, then --prod |
v.optional (or a widened v.union) in the first deployundefined and writers produce the new shape before the backfill startsinternalMutation or migrations.define, never publicpaginate and reschedules via ctx.scheduler.runAfter(0, internal....)