Scan Algorithm Inventory

Last updated: 2026-05-23.

This page maps the mobile scanner choices to the actual recognition engines behind them. It also clarifies where the POC fixture photos live and whether they participate in production matching.

Executive Summary

The app currently has four user-facing scan families:

Mobile source	Item type	Runtime path	What it is for
SKN	Card	lab, compat, tormknd, combined, frame fan-out	Compare every card recognition path side by side.
SKN	Slab	Same fan-out as Card	Temporary behavior. There is no slab-specific recognition yet.
SKN	Sealed	sealed only	Generic sealed product recognition in the SKN backend.
Booster Lab	Booster	tcgapp2-booster only	Tormknd-style booster scanner isolated from the SKN sealed path.

Photo capture and image import are only input sources. Once an image is selected, both go through the same active scan plan for the selected item type.

Current Algorithms

Lab

• Mobile mode: lab
• Endpoint: POST /scan-lab on the SKN Node/Fastify API, port 8080
• Object class: raw trading card photo
• Pre-processing: server-side JPEG decode, OpenCV card-bound detection, crop/warp when a card-shaped region is found
• Embedding: DINOv2 small, 384 dimensions, from the embeddings sidecar
• Index: PostgreSQL pgvector, table card_embeddings_lab, HNSW cosine index
• Catalog scope: Pokémon, One Piece, Magic, Lorcana, Flesh and Blood
• Decision model: single visual embedding signal
• Strength: fastest SKN card path with wide catalog coverage and SQL filters before nearest-neighbor search
• Weakness: can false-positive on visually similar cards because it does not use OCR or pHash as confirmation

This is the main SKN card path. It is the cleanest test of our own DINOv2 plus pgvector stack.

Compat

• Mobile mode: compat
• Endpoint: POST /scan-compat on the tcg-vision Python sidecar, port 8000
• Object class: raw trading card photo
• Pre-processing: Tormknd-style OpenCV perspective correction
• Embedding: DINOv2 small, 384 dimensions
• Index: in-memory FAISS IndexFlatIP with a prebuilt faiss_index.bin
• Catalog scope: the catalog baked into the sidecar index, mainly Pokémon, One Piece, and some Lorcana
• Decision model: single visual embedding signal
• Strength: useful rollback and A/B path because it does not depend on pgvector
• Weakness: narrower coverage, frozen FAISS artifact, weaker filtering than SQL

Compat is our forked/in-stack version of the friend's tgc-app path. It adapts the response to the SKN mobile contract and can enrich results with SKN metadata.

Tormknd / tgc-app

• Mobile mode: tormknd
• Endpoint: POST /scan on the unmodified Tormknd sidecar, port 8002
• Object class: raw trading card photo
• Pre-processing: upstream tgc-app OpenCV perspective correction
• Embedding: DINOv2 small, 384 dimensions
• Index: upstream FAISS IndexFlatIP and mapping
• Decision model: single visual embedding signal
• Strength: immutable reference path for comparing our fork and our backend changes
• Weakness: not production-enriched, narrower catalog, off by default behind the Docker tormknd profile

In conversation this is sometimes called tcg-app, but the local reference repo is tgc-app. We treat it as a reference repo and do not modify it for SKN behavior.

Combined

• Mobile mode: combined
• Endpoint: POST /scan on the SKN Node/Fastify API, port 8080
• Object class: raw trading card photo
• Pre-processing: server-side crop, then several independent signals
• Signals: pHash, OCR, DINOv2 embedding
• Embedding index: same card_embeddings_lab pgvector table as Lab
• Decision model: multi-signal cascade through decideMatch
• Strength: best debug path for understanding why a photo is accepted or rejected
• Weakness: stricter and slower than Lab; may reject an image that an embedding-only path accepts

Combined is the historical production path. It can match via near-duplicate pHash, exact printed identifiers from OCR, or embedding rescue. It is especially useful when the app needs to explain ambiguity.

Frame

• Mobile mode: frame
• Endpoint: POST /scan-frame on the SKN Node/Fastify API, port 8080
• Object class: card photo after client-side rectification
• Pre-processing: Android native OpenCV detectQuad plus warpQuad, then upload the warped crop
• Embedding: DINOv2 small, 384 dimensions
• Index: same card_embeddings_lab pgvector table as Lab
• Decision model: single visual embedding signal
• Strength: isolates the impact of doing the crop/warp on the phone
• Weakness: slow or fragile when the client-side quadrilateral detector misses the card or crops the wrong region

Frame is Path #5. It is a lab path for measuring whether the phone can produce a better canonical crop than the backend.

Sealed

• Mobile mode: sealed
• Endpoint: POST /scan-sealed-lab on the SKN Node/Fastify API, port 8080
• Object class: generic Pokémon sealed products: ETB, display, booster bundle, collection box, blister, booster pack
• Pre-processing: raw full image by default; optional sealed crop when the app has a booster-like hint
• Embedding: DINOv2 base, 768 dimensions, from the dedicated sealed-embeddings sidecar
• Index: PostgreSQL pgvector, table sealed_product_embeddings_vitb_lab
• Reference sources: internal ptcg-assets images by default; SKN-owned sealed reference photos are paused for normal scans and must be requested explicitly
• Extra signals: optional OCR text extraction, product-type and language filters, candidate reranking
• Optional detector: YOLO segmentation when models/yolo/booster_seg.pt exists; OpenCV foreground crop fallback otherwise
• Strength: broad ptcg-assets sealed coverage can be queried in the same backend stack as cards, while SKN-owned references remain available for controlled debug and benchmarks
• Weakness: real-world accuracy is still limited by reference coverage, photo quality, product variants, language variants, and detector quality

Generic sealed scans intentionally do not force booster_pack, because a visible booster inside a box can otherwise dominate the match and hide the real product.

tcg-app2 Booster

• Mobile mode: tcgapp2-booster
• Endpoint: POST /scan/booster on the local tcg-app2 FastAPI service, port 8003
• Object class: booster packs
• Pre-processing: blur check, YOLOv11-Seg booster segmentation when a model is present, OpenCV contour fallback otherwise
• Embedding: DINOv2 ViT-B/14, 768 dimensions, 518-pixel input
• OCR: EasyOCR, currently focused on useful booster metadata such as language, publisher, edition, year, and TCG hint
• Vector store: Qdrant, either local embedded storage or server mode
• Reference source: tcg-app2/data/boosters_metadata.json plus booster images indexed by scripts/build_booster_index.py
• Decision model: vector search with metadata filters from OCR
• Strength: closer to the friend's booster-specific architecture and good for booster-only A/B tests
• Weakness: separate stack, separate index, booster-only scope, quality depends heavily on the booster segmentation model and indexed booster catalog

This path is intentionally isolated from the SKN generic sealed path. It is a lab comparator, not a replacement for all sealed products.

On-device fallback

• Runtime: React Native app, no backend required
• Inputs: local src/scan/cardIndex.json and scanner strategies
• Signals: pHash, color histogram, local OCR/MLKit where available, debug strategy ensemble
• Strength: works when backend endpoints fail
• Weakness: less accurate and less complete than backend recognition

This is a fallback and diagnostic layer, not the primary production scanner.

Slab recognition

• Current status: the app has a Slab item type, but it currently uses the same five card paths as Card
• What is missing: slab detection, label crop, grading-company recognition, grade OCR, cert-number OCR, and a slab-specific verification policy

So today a slab photo can still identify the visible card if the crop is good enough, but it does not truly recognize PSA 10, BGS, CGC, or cert data yet.

What Is Trained vs Indexed

Component	Trained by us today?	What changes when we improve it?
DINOv2 card embeddings	No. Frozen pretrained model.	Rebuild card embeddings when model, crop, or input geometry changes.
DINOv2 sealed embeddings	No. Frozen pretrained model.	Rebuild sealed embeddings when model, crop, or input geometry changes.
FAISS / pgvector / Qdrant	No. These are vector indexes, not recognition models.	Re-index vectors and metadata.
YOLO sealed/booster detector	Yes, once we provide weights.	Better object crop before embedding, especially on real phone photos.
OCR	No. Off-the-shelf OCR plus rules.	Improve text regions, dictionaries, filters, and reranking.
ptcg-assets / SKN-owned images	Not models. They are references. Normal scans default to ptcg-assets; SKN-owned is paused unless explicitly requested.	Add cleaner, licensed, better-labeled reference coverage.

Where the POC Photos Intervene

There are two different concepts that must not be mixed: POC test fixtures and backend sealed references.

In poc-tcg-collector

The POC repository contains image fixtures for tests and local fallback behavior:

Path	Count observed	Role
__fixtures__/cards/real/*.jpg	5	Real-card integration fixtures for tests around demoScanDb.
__fixtures__/tcgdex/images/*.jpg	13	Reproducibility fixtures for the historical TCGdex ingest tests.
__fixtures__/cards/card-a-stripes.jpg, card-b-checker.jpg, card-c-gradient.jpg	3	Synthetic unit-test fixtures for pHash and histogram determinism.
src/scan/demoScanDb.ts	generated rows	Small legacy demo DB used by tests. The production scanner screen no longer reads it directly.
src/scan/cardIndex.json	JSON catalog	Metro-bundled fallback/debug card catalog generated from backend data.
root Screenshot_*.jpg files	untracked	Visual debugging screenshots only. They are not reference images.

These POC images do not feed /scan-lab, /scan-compat, /scan, /scan-frame, /scan-sealed-lab, or tcg-app2 production matching at runtime. They are mostly fixtures for tests, reproducibility, and local fallback diagnostics.

The "10 Photos" From the Last Sealed Work

The 10 SKN-owned sealed photos are in the backend, not the POC:

tcg-collector-backend/src/api/tests/fixtures/sealed-samples/images/labeled/

They are listed in:

tcg-collector-backend/src/api/tests/fixtures/sealed-samples/manifest.json

They intervene in two places:

1. Benchmark samples: npm run benchmark:sealed posts them to /scan-sealed-lab and measures exact/top-K accuracy.
2. Private SKN-owned sealed references: the Docker ingest mounts sealed-samples as SKN_SEALED_REFERENCES_SOURCE, but as of 2026-05-23 the compose default is SEALED_INCLUDE_SKN_REFERENCES=false. If enabled manually, they can be embedded into sealed_product_embeddings_vitb_lab as skn-owned:* references.

That second point is useful, but it is also dangerous to overinterpret: matching those same 10 photos after indexing them validates wiring and duplicate recall. It does not prove the scanner generalizes to new phone photos, angles, lighting, languages, or unseen variants.

Normal /scan-sealed-lab requests now default to source=ptcg-assets, even if old skn-owned rows are still present in Postgres. Use source=skn-owned only to debug owned bootstrap references, or source=all for controlled A/B comparisons.

For unseen-field validation, use the separate evaluation-only folder:

tcg-collector-backend/src/api/tests/fixtures/sealed-eval/

Those images must not be ingested as references. They are for measuring generalization.

Practical Reading Guide

• If Lab is good and Combined rejects: the visual neighbor is probably good, but OCR/pHash did not confirm.
• If Frame is worse than Lab: the client crop is probably wrong or too slow.
• If Compat or Tormknd is better than Lab: compare preprocessing and index coverage before changing thresholds.
• If tcg-app2 Booster is better than Sealed on boosters: its booster-specific segmentation/Qdrant/OCR pipeline is helping.
• If Sealed returns wrong nearby products: inspect top-K, OCR filters, product type, language, whether the expected product exists as a reference, and whether a debug request explicitly enabled source=skn-owned or source=all.
• If a photo was taken from a PC screen: expect lower accuracy, glare and moire can dominate the embedding.

Related Pages

• Scan paths comparison
• Sealed scanner lab
• Recognition optimization strategy
• Sealed field photo collection format