Architecture overview

For the API/accounts/media architecture and security boundaries examined on 2026-10-03, see Backend, API, images and access control. The historical figures and scan paths below are not a live deployment audit.

One-page view of the whole stack — for a colleague who's never seen the project. Read this first; the deeper docs (scan-architecture.md, the per-TCG …-ingest pages, the scan-lab-plan phase log) drill into specific decisions.

Snapshot dated 2026-05-08.

TL;DR (the 30-second pitch)

• We let users scan a card from their phone and get instant identity + price + collection management. Five TCGs supported today: Pokémon, One Piece, Magic the Gathering, Lorcana, Flesh and Blood.
• Everything runs on our own VPS. No external API call during a scan. No service-tier dependency on a friend's repo, even though one of our containers was originally cloned from one.
• Recognition uses DINOv2-small image embeddings at 518×518 input with OpenCV perspective correction before the model (same preprocessing as Tormknd's reference app). Two parallel implementations exist (FAISS sidecar vs pgvector inside our Postgres) — see The two scan paths below for which TCG goes through which.
• Total catalog as of 2026-05-08: 143 452 vectors across 5 TCGs and ~1 200 sets, ~26 GB of card images on disk, all embedded at 518 + warp.

Big picture

┌──── Phone (React Native) ─────────────────────────────────┐
│  ScannerScreen → backendScan.ts                           │
│  Parallel fan-out (Promise.allSettled) to 4 endpoints:    │
│   • POST :8080/scan-lab    (Lab,      single-signal)      │
│   • POST :8000/scan-compat (Compat,   single-signal)      │
│   • POST :8002/scan        (Tormknd,  single-signal)      │
│   • POST :8080/scan        (Combined, MULTI-signal)       │
└────────────────────┬──────────────────────────────────────┘
                     │
                     ▼
┌──── Our VPS — docker compose ─────────────────────────────┐
│                                                           │
│  ┌─ tcg-vision   :8000 ─┐  FAISS index, baked at build:   │
│  │  /scan-compat        │  ~30 k Pokémon + OP + Lorcana   │
│  │  DINOv2 + FAISS      │  (our fork of tgc-app)          │
│  └────────┬─────────────┘                                 │
│           │ enrich via /cards/:id                         │
│           ▼                                               │
│  ┌─ api Fastify :8080 ──┐  /cards, /sets,                 │
│  │  /scan-lab           │  /scan-lab  (pgvector single)   │
│  │  /scan               │  /scan      (multi-signal:      │
│  │                      │   pHash + OCR + DINOv2)         │
│  └────────┬─────────────┘                                 │
│           ▼                                               │
│  ┌─ postgres (pgvector) ┐  cards (~145 k), sets,          │
│  │  card_embeddings_lab │  card_embeddings_lab (HNSW)     │
│  │                      │  — shared by Lab + Combined,    │
│  └──────────────────────┘  cards.phash + cards.printed_*  │
│                                                           │
│  ┌─ embeddings sidecar ─┐  DINOv2-small @ 518×518,        │
│  │  internal :8000      │  + OpenCV warp (CPU),           │
│  │                      │  ~2 cards/s @ 3 cores           │
│  └──────────────────────┘                                 │
│                                                           │
│  ┌─ tormknd-api  :8002 ─┐  Unmodified tgc-app code,       │
│  │  /scan               │  frozen A/B reference,          │
│  │  (--profile tormknd) │  off by default                 │
│  └──────────────────────┘                                 │
└───────────────────────────────────────────────────────────┘

Plus an ingest one-shot worker (also in compose) that pulls TCGdex / OPTCG API / Scryfall and writes to cards + data/images/.

TCG status

TCG	Source	Cards	Sets	Scan path	FAISS	pgvector	Where the index is
Pokémon	TCGdex (multi-lang)	~23 k	204	/scan-lab	✅ baked-in (rollback)	✅	card_embeddings_lab (us)
One Piece	OPTCG API	~3 k	20	/scan-lab	✅ baked-in (rollback)	✅	card_embeddings_lab (us)
Magic	Scryfall bulk-data	~107 k	976	/scan-lab	❌ not in FAISS	✅	card_embeddings_lab (us)
Lorcana	Lorcast API	~2.6 k	—	/scan-lab	❌ not in FAISS	✅	card_embeddings_lab (us)
Flesh and Blood	the-fab-cube GitHub bulk	~9.3 k	—	/scan-lab	❌ not in FAISS	✅	card_embeddings_lab (us)

As of 2026-05-08, all five TCGs scan via the pgvector path (/scan-lab), all embedded at 518×518 + Tormknd-style perspective correction. Tormknd's FAISS sidecar is still running for rollback purposes (covers Pokémon + One Piece + sliver of Lorcana) but no longer in the live scan path. The original asymmetry is gone — we ingest, embed, and query everything ourselves. See catalog-updates.md for how a new set lands.

The four scan paths

Update 2026-05-10: a fourth path was added to the mobile fan-out — Combined, the historical /scan endpoint on the Node API. Multi-signal (pHash + Tesseract OCR + DINOv2 inside a unified decideMatch), it complements the three earlier paths which are single-signal embedding. See scan-comp for the detailed comparison and tcg-collector-backend/docs/scanner/SCAN_PATHS.md for the full technical matrix.

All four go through DINOv2 to produce a 384-d vector; three of them rely on only that signal (single-signal embedding), the fourth combines it with pHash and OCR (multi-signal).

	/scan-lab (Fastify)	/scan-compat (tcg-vision)	/scan (tormknd-api)	/scan Combined (Fastify)
Port	8080	8000	8002	8080
Pipeline	Single-signal embedding	Single-signal embedding	Single-signal embedding	Multi-signal: pHash + OCR + embedding
Vector store	pgvector HNSW (Postgres)	FAISS IndexFlatIP (in-memory)	FAISS IndexFlatIP (in-memory)	pgvector + phash / printed_* columns
Index source	card_embeddings_lab table, populated by embedCatalogLab.ts	services/tcg-vision/data/faiss_index.bin baked at build (fork that has diverged)	tgc-app/data/faiss_index.bin from Tormknd's repo, unmodified	card_embeddings_lab (since 2026-05-10, shared with Lab) + cards.phash + cards.printed_*
TCGs covered	Pokémon + OP + Magic + Lorcana + FAB (143 452 vectors)	Pokémon + One Piece + sliver of Lorcana (~30 370)	Same as /scan-compat (same upstream dataset)	Pokémon + OP + Magic + Lorcana + FAB
Metadata enrichment	Native SQL JOIN against cards	Adapter that calls our api for non-Pokémon	None — raw upstream response	SQL JOIN + signals exposed (match.phash, ocr.*, embedding.*)
Used by the RN app	✅ Scan Lab button (default)	Scan Compat button (rollback / regression-bisect)	Scan Tormknd button (A/B against unmodified upstream)	Scan Combined button (4th cell, multi-signal)
Started by docker compose	✅ by default	✅ by default	⚠ profile --profile tormknd (off by default)	✅ by default (same container as Lab)

Phase F shipped on 2026-05-07: the RN app's default path flipped to /scan-lab. Since 2026-05-09 ScannerScreen.tsx fans out in parallel via Promise.allSettled to every active path; since 2026-05-10 it's 4 cells side-by-side instead of 3 (Combined multi-signal added). The user picks the winner — no auto-rerank app-side, the goal of the fan-out is comparison.

When each path decides differently: Combined can return no-match reason low-confidence when its decideMatch cascade can't get pHash, OCR, and embedding to converge on the same answer — by design (see scan-comp for the cascade detail). Before 2026-05-10, Combined returned low-confidence almost universally because it consulted a legacy card_embeddings table that turned out to be empty (the ingest job populating it had been abandoned long ago). Fix applied: Combined now points at the same card_embeddings_lab table as Lab.

Re-embed caveat: when embedCatalogLab.ts FORCE=true regenerates the vectors (e.g. after the warp-fix from #135), it updates card_embeddings_lab — the table shared by Lab AND Combined since 2026-05-10. The FAISS .bin files for compat and tormknd stay frozen (the warp-fix was ported into the Python code via #138, without rebuilding the bins because build_index.py doesn't call correct_perspective).

Why local

Every piece of the stack lives on our VPS. The only inbound data sources are build-time (TCGdex / OPTCG / Scryfall when ingesting) — never runtime.

• A user scan in 2030 with TCGdex offline: still works. Index + images are local.
• A user scan with Tormknd's repo deleted: still works. The container we built is locally pinned.
• Our DB lost: rebuild from data/images/ (still on disk) by re-running embedCatalogLab.ts (~3 h for the full catalog).

Disk storage breakdown:

Path	Content	Size
data/images/	All card JPEGs, organised by set/collector_number	~26 GB
data/postgres/	cards, sets, card_images, card_embeddings_lab	~500 MB
data/cache/scryfall/	Daily-refreshed Scryfall bulk JSON (kept across re-runs)	~514 MB

Roadmap

Phase	Status
A — card_embeddings_lab schema + /scan-lab endpoint	✅ shipped 2026-05-06 (#103)
B — variant-aware cards schema	✅ shipped 2026-05-06 (#105)
C — variant-aware OP ingest	✅ shipped 2026-05-06 (#107)
D — full catalog embed	✅ shipped 2026-05-07 (all 3 TCGs, 131 k+ vectors)
E — cross-recall vs phone photos	✅ shipped 2026-05-07 (5/5 SV01 photo agreement, surfaced + fixed OP variant disk corruption)
F — flip RN app to /scan-lab	✅ shipped 2026-05-07 (poc-tcg-collector#127)
Magic ingest end-to-end	✅ shipped 2026-05-07 (#111)

All three TCGs (Pokémon, One Piece, Magic) now resolve via /scan-lab. Next deferred items: Lorcana ingest (Lorcast API + AVIF→JPEG conversion needed, ~2-3 h), Flesh and Blood (no clean source as of 2026-05-07), retire the tcg-vision container after a few weeks of production validation.

See also

• Card scan architecture — same diagram, more detail on each component
• Plan — /scan-lab pgvector validation — full phase log with incident history
• Magic the Gathering ingest — the precedent: how a new TCG joins the pgvector path
• Rollback — One Piece integration — how to unwind a TCG addition if it goes sideways
• tcg-collector-backend/docs/INGEST.md — backend-side technical reference (pipeline, per-TCG sources, multi-language)