Documentation

Appearance

Scan paths comparison — Lab / Compat / Tormknd / Combined ​

The mobile app sends every capture in parallel to the four raw-photo scan paths below, plus the separate Path #5 Frame cell documented in the scan-frame series. Results come back side-by-side on the Scanner screen and in the Scan Debug history. This document details the strengths, weaknesses, and differences of the four raw-photo paths.

Canonical technical reference: the full operational matrix lives in the backend repo → tcg-collector-backend/docs/scanner/SCAN_PATHS.md. This document is the comparison-oriented summary.

TL;DR ​

  • Lab: our main path (Node + pgvector + HNSW). 5 TCGs covered, the fastest, the most up-to-date pre-processing-wise. Single-signal embedding.
  • Compat: a fork of tgc-app we modified and embedded in the stack. Limited coverage (Pokémon + One Piece + a bit of Lorcana). Exists for fast rollback and regression bisecting. Single-signal embedding.
  • Tormknd: the unmodified code of the friend's tgc-app (Tormknd). A/B reference to check whether our fork helped or broke something. Single-signal embedding.
  • Combined: the historical /scan endpoint on the Node API. Multi-signal: combines pHash + Tesseract OCR + DINOv2 in a unified decideMatch. Stricter — can return no-match when the other three see a card.

Single-signal vs multi-signal — why it matters ​

A signal is a way of looking at the card. Each signal has its strengths and blind spots.

SignalWhat it isStrengthWeakness
pHash (perceptual hash)64-bit hash that summarises the global visual structureUltra-fast (< 1 ms), excellent on duplicatesBroken by angle, flash, crop, colour shifts
OCR (Tesseract)Reads the printed text: set code + collector number + HP + nameNear-infallible when it reads cleanly (numbers don't lie)Fails on blurry / mis-oriented / foreign-language photos
Embedding (DINOv2)384-dim neural-network vector measuring learned visual similarityRobust to angle / flash / cropPrecision ceiling on visually-close cards (e.g. Pawmi vs Panpour)

Single-signal (Lab / Compat / Tormknd) only relies on the DINOv2 embedding. More forgiving but can accept a card on visual similarity alone, so more prone to false-positives on layout-similar cards.

Multi-signal (Combined) demands consensus across signals via decideMatch:

  1. If pHash finds a near-exact copy → match immediately (high confidence; same image).
  2. Else, if OCR returns a set + collector matching exactly one DB candidate → match (high confidence; the printed identifier decides).
  3. Else, the embedding's top-1 must clear a similarity threshold AND not have a too-close runner-up → match.
  4. Else → no-match reason low-confidence.

That's why Combined can reject a card the three single-signal paths all accept: if the embedding is confident but neither pHash nor OCR confirm, decideMatch distrusts. See the sv01-003 example below.

Comparison table ​

Scan LabScan CompatScan TormkndScan Combined
Endpoint/scan-lab/scan-compat/scan/scan
Port8080800080028080
Containertcg-collector-api-1tcg-vision-apitormknd-apitcg-collector-api-1
Started by default✅ yes✅ yes❌ no (--profile tormknd)✅ yes
CodeNode + Fastify (OURS)Python — fork of tgc-app that has divergedPython — friend's tgc-app, unmodifiedNode + Fastify (OURS)
Pipeline typeSingle-signal embeddingSingle-signal embeddingSingle-signal embeddingMulti-signal: pHash + OCR + embedding
Vector storePostgreSQL + pgvector HNSWFAISS IndexFlatIP (in-memory)FAISS IndexFlatIP (in-memory)PostgreSQL + pgvector (reuses Lab's store)
Distancecosine (embedding <=> $1::vector)inner product (cosine via L2-norm)inner product (cosine via L2-norm)cosine + Hamming pHash + exact OCR match
EmbeddingsDINOv2 ViT-S/14 (384-d)DINOv2 ViT-S/14 (384-d)DINOv2 ViT-S/14 (384-d)DINOv2 ViT-S/14 (384-d)
Index sourcecard_embeddings_lab table, populated by embedCatalogLab.tsservices/tcg-vision/data/faiss_index.bintgc-app/data/faiss_index.bincard_embeddings_lab (since 2026-05-10) + cards.phash + cards.printed_*
Cardinality143,452 vectors (5 TCGs)~30,370 vectors~30,370 vectors (same upstream dataset)~143k cards with pHash + metadata
TCGs coveredPokémon, One Piece, Magic, Lorcana, FABPokémon (FR/EN/JA), One Piece, Lorcanaidentical to CompatPokémon, One Piece, Magic, Lorcana, FAB
Pre-ANN filter by TCG/set/lang✅ native SQL JOIN❌ post-fetch Python filter❌ post-fetch Python filter✅ native SQL JOIN
Rich metadata in response✅ SQL JOIN to cards✅ adapter calling our api for non-Pokémon❌ raw upstream response✅ SQL JOIN to cards + OCR/pHash signals exposed
Behaviour on ambiguous photoaccepts if embedding > thresholdaccepts if embedding > thresholdaccepts if embedding > thresholdrejects if pHash/OCR don't confirm

Scan Lab ​

What it is ​

Our home-grown path. When you take a photo, the Node API:

  1. POSTs the image to the embeddings (Python) sidecar → receives a 384-d DINOv2 vector.
  2. Runs SELECT … ORDER BY embedding <=> $1::vector LIMIT 5 filtered by TCG/set if requested.
  3. Re-ranks the top-5 and returns enriched JSON (TCGdex metadata, Cardmarket prices, etc.).

Strengths ​

  • Widest coverage: 5 TCGs (Pokémon + OP + Magic + Lorcana + FAB), 143,452 vectors.
  • SQL pre-filter: scope the search to tcg=pokemon AND lang=fr before the ANN — something FAISS can't do natively.
  • Reproducible index: rebuild with embedCatalogLab.ts FORCE=true, idempotent, transactional.
  • Always-fresh metadata: SQL JOIN to cards → image, rarity, price, etc.
  • Fastest server-side: ~30-60 ms.

Weaknesses ​

  • Depends on Postgres + pgvector. If the DB goes down, the path is dead.
  • Indexed only with what we've embedded: a freshly-ingested card not yet embedded won't appear until the next re-embed.
  • HNSW is approximate: top-1 isn't always the exact best match. Recall is very good but not 100%-guaranteed (negligible at 144k vectors in practice).
  • Single-signal: the embedding alone can be tricked on visually-close cards.

Scan Compat ​

What it is ​

A fork of the friend's tgc-app we embedded in our Docker stack and let diverge. The sidecar runs in Python (FastAPI) and loads a prebuilt faiss_index.bin into RAM.

Strengths ​

  • Battle-tested code: it's the historical path before the flip to Lab on 2026-05-07. Stable.
  • Zero Postgres dependency: if pgvector explodes, this path keeps serving results.
  • Frozen index: reproducible, no "what changed between two scans?" question.
  • Bonus metadata: we added an adapter that calls our API to enrich non-Pokémon TCGs (One Piece especially) from our Postgres.

Weaknesses ​

  • Limited coverage: only Pokémon (FR/EN/JA) + One Piece + a bit of Lorcana. No Magic, no Flesh and Blood. An MTG scan returns a Pokémon false-positive.
  • .bin frozen in time: reflects the pre-processing pipeline at the time of the last rebuild. The warp-fix #135 (2026-05-08) was ported into the code path via #138, so it's active at scan time, but the bin itself wasn't rebuilt (not necessary — build_index.py doesn't call correct_perspective).
  • No SQL pre-ANN filter: TCG/set filters are applied post-fetch in Python — slower and less precise than a SQL JOIN.
  • Single-signal: same false-positive limit as Lab.
  • Fork divergence: we added three things over vanilla Tormknd:
    1. /scan-compat adapter that re-shapes the response to match the /scan-lab contract.
    2. LOCAL_IMAGE_BASE_URL that rewrites image URLs to our nginx instead of the TCGdex CDN.
    3. BACKEND_API_URL that calls our API to enrich non-Pokémon TCGs (otherwise just returns a card_id).

Scan Tormknd ​

What it is ​

The friend's tgc-app (Tormknd), built as-is from ../../tgc-app/ (the local repo checkout). Unmodified code, original index, original mapping.

Strengths ​

  • Perfect A/B reference: it's exactly what the friend's app would return. Better result on Compat → our fork improved. Worse → we broke. Tormknd same as Compat → fork divergence isn't the cause.
  • Zero contamination from our stack: no Postgres, no warp-fix, no re-embed, no nginx. It's the friend's code and data, period.
  • Reproducible upstream: if the friend rebuilds his index with a new backbone or a new fix, we integrate by re-checking out his repo and docker compose build tormknd.

Weaknesses ​

  • Off by default: gated behind --profile tormknd to keep the daily up light (first build = 5-10 min: torch hub + faiss-cpu install).
  • Limited coverage: identical to Compat (Pokémon + OP + Lorcana). No MTG, no FAB.
  • No enriched metadata: the response is whatever the friend decided to return. No price, no rarity, no image rewriter.
  • Single-signal: only the embedding drives the decision upstream.
  • Evolves independently of our stack: if the friend pushes a fix into tgc-app, we need a local git pull + container rebuild to pick it up.

Scan Combined ​

What it is ​

The historical /scan endpoint on the same Node API as Lab (port 8080). Before the flip to Lab on 2026-05-07, it was the mobile app's default path. It combines several independent signals through a unified decideMatch function:

  1. pHash (64-bit perceptual hash) computed on the photo, compared via Hamming distance to the pre-computed pHashes of catalog cards. If an indexed card is below a Hamming threshold → immediate match (the photo and catalog image are near-identical).
  2. Tesseract OCR reads known text regions: collector number (bottom-right), set code (top-right or top-left depending on the TCG), name, HP. If OCR returns a set + collector matching exactly one card_id in the DB → match (high confidence; the printed identifier decides).
  3. DINOv2 embedding proposes its top-1 + runner-up. To be accepted, similarity must clear a threshold AND the runner-up must be sufficiently far away (else it's ambiguous).
  4. If no signal clears its threshold → no-match with reason: "low-confidence".

Strengths ​

  • Most precise on easy cases: an exact duplicate or readable photo passes pHash or OCR with confidence ~1.0. Faster AND more precise than the embedding alone.
  • Most robust on tricky cases: on two visually-close Pokémon (Pawmi vs Panpour), embedding alone can be wrong, but OCR of the collector number decides.
  • Coverage identical to Lab: 5 TCGs, 143,452 vectors. Since 2026-05-10 Combined queries the same card_embeddings_lab table as Lab — both paths see the exact same vectors, including the warp-fix #135 re-embed. Combined adds cards.phash and cards.printed_* on top for the pHash and OCR signals.
  • Signals exposed in the response: match.phash, ocr.collectorNumber, ocr.totalNumber, ocr.hp, embedding.cardId, embedding.similarity, runnerUp. It's the only path that gives you everything it saw, signal by signal.
  • Ideal for debug: a Combined cell returning no-match while Lab/Compat/Tormknd match tells you the non-visual signals disagree — often a hint that the photo itself is bad.

Weaknesses ​

  • The strictest: can return no-match on photos where the three single-signal paths see a card. The decideMatch cascade requires at least one signal to clear its threshold; if pHash (Hamming distance), OCR (no exact set+collector match), and embedding (similarity below threshold) all fail, the cascade falls through to low-confidence.
  • Slower than Lab alone: Tesseract OCR costs ~150-300 ms on CPU, even when pHash has already decided. Optimisation possible (early-return) not implemented.
  • pHash threshold tuning still pending: post-2026-05-10 fix smoke surfaced cases where pHash short-circuits the cascade with similarity ~0.75 to a wrong card (typically a same-set neighbour), preventing the embedding from voting. The cascade is "first match wins"; a future migration to a weighted rerank (top-50 ANN → fused score across signals) would let the embedding correct pHash misfires.
  • Depends on Postgres + pgvector like Lab.

Historical note — the broken wiring before 2026-05-10 ​

Before 2026-05-10, /scan (Combined) read from a separate card_embeddings (legacy) table populated by embedCatalog.ts — distinct from embedCatalogLab.ts. The 4-button fan-out smoke revealed that table was empty (0 rows) — the legacy ingest job hadn't run for a long time, since embedCatalogLab.ts had taken over for the Lab A/B phase. Consequence: Combined was running with no embedding signal at all — pHash + OCR only — and was silently returning low-confidence no-match on most photos.

1-line fix in src/api/src/scan/embeddings.ts: point the query at card_embeddings_lab. No re-embed needed (the target table was already populated), no waiting. The legacy card_embeddings table stays in the schema but unused — can be dropped in a separate migration.

An earlier version of this doc described Combined as using "pre-warp-fix vectors" — that was inaccurate. Combined was using non-existent vectors. The fix is now applied, Combined consults the same vectors as Lab.

Which path to use when ​

Use caseRecommended path
Production scan / end-user appLab (max coverage, fastest) or Combined (stricter, fewer false-positives)
Verify a card hasn't regressed after a re-embedLab + compare with Compat
Bisect a fork-specific regressionCompat vs Tormknd side-by-side
Test a Magic, Lorcana or Flesh and Blood cardLab or Combined (the other two don't cover them)
Diagnose a sketchy Python sidecar responseTormknd (eliminates our fork)
Understand why a photo is ambiguousCombined — its ocr.* and match.phash fields tell you which signal did (or didn't) confirm
Demo / presentation / project sharingLab or Combined (both cover all 5 TCGs)

Current state and known degradations ​

This section drifts over time — at best it's a snapshot. For live state, look at the container dashboards and the re-embed position in /tmp/embed_overnight2.log.

  • Lab: ✅ re-embed completed 2026-05-10, 143,452 vectors across 5 TCGs with warp-fix #135. ?tcg=lorcana and ?tcg=fab honoured since #138.
  • Compat: ✅ warp-fix #135 ported into services/tcg-vision/src/vision_service.py via #138 — /scan-compat now rejects sub-region quads (artwork window, text box) at scan time. No FAISS rebuild needed (build_index.py doesn't call correct_perspective, so the bin stays consistent with the unchanged index-time pipeline). The sv01-003 → xy1-37 regression is resolved: sv01-003 → sv01-003 sim 0.876.
  • Tormknd: 🟡 state depends on what the friend last built. Intentionally not affected by our warp-fixes — it's the immutable reference. To start: docker compose --env-file .env -f docker/compose.yml --profile tormknd up -d tormknd.
  • Combined: ✅ uses card_embeddings_lab since the 2026-05-10 fix in src/api/src/scan/embeddings.ts. It now consults the same populated vector table as Lab; the legacy card_embeddings table remains unused and can be dropped in a separate migration. pHash and OCR remain independent signals on top of the shared embedding.

See also ​