Skip to main content
The benchmark harness compares supaschema with Supabase CLI diff engines on identical fixtures. Each generated migration is:
  1. applied once;
  2. applied again;
  3. compared against the target catalog.
Reference run: 2026-06-12 on Apple Silicon, Node 24, PostgreSQL 17.6, and Supabase CLI 2.106.0.
Short version: supaschema stays fast at large schema sizes, keeps F1 at 1.000, and produces migrations that survive a second apply.

​
Speed and accuracy

At 1,000 tables (~7,000 objects), supaschema against each of the five Supabase CLI engines — median diff latency, accuracy (F1 vs a ground-truth change manifest), and whether the migration survives a second apply:
supaschema vs diff engines at 1,000 tables: latency bars, F1 accuracy, and replay-safety
The gap widens with scale. At 2,500 tables (~17,500 objects) the engines cross three minutes while supaschema stays around three seconds — still scoring F1 1.000, while every engine still drops the same policy and still fails the second apply:
supaschema vs diff engines at 2,500 tables: latency bars, F1 accuracy, and replay-safety

​
Full workflow

The diff is only half the loop. Once type outputs exist, getting a migration and refreshed types is one supaschema diff — against the CLI it takes three commands (db diff, apply, gen types) and a database that has already caught up. End to end at 1,000 tables:
full workflow vs diff engines at 1,000 tables: supaschema migration plus refreshed type outputs in one command versus db diff, apply, and gen types per engine

​
Accuracy

Diff output is scored two ways:
  • F1 against a ground-truth change manifest by object identity.
  • Catalog fingerprint after applying the generated migration to a throwaway database.
supaschema scores F1 1.000 on every manifest-carrying fixture in file, live-database, and full-workflow modes. The compared Supabase engines score 0.982-0.999 and miss the same RLS policy change. On that fixture, each generated migration also fails the fingerprint check.

​
Replay safety

migration verification at 1,000 tables across supaschema and Supabase CLI engines
Every compared engine reaches the target catalog after one apply. Only supaschema also succeeds on the second apply for every fixture; the other engines emit unguarded column and index changes that fail when a migration is retried.

​
Reproduce

Run the benchmark harness from this repository: 
npm run benchmark
The harness and comparison utilities live in the repository’s benchmarks/ directory.
Last modified on June 18, 2026