Documentation

Appearance

TCG Collector — Improvement priorities (2026-05) ​

Current state ​

The current engine relies on:

  • DINOv2 ViT-S/14
  • 1024-d global embeddings
  • pgvector + HNSW
  • ANN cosine retrieval
  • warp perspective + normalization
  • top-k SQL retrieval

The system is already solid and ahead of most hobby scanners.

The current ceiling is no longer infrastructure, but:

  • fine-grained disambiguation
  • visual variants
  • reprints
  • weak differences between cards

Top priority: targeted OCR ​

Current problem ​

Pure visual retrieval still confuses:

  • reprints
  • reverse holo
  • close editions
  • promo variants
  • languages
  • close alternate arts

Global embeddings alone are no longer enough.

Solution ​

Add an OCR pipeline targeted at:

  • collector number
  • set code
  • rarity
  • sometimes language

OCR must NOT be global on the whole image.

It must be:

  • localized
  • ultra-targeted
  • run only after ANN retrieval

Step 1 ​

ANN retrieval:

  • top-50 candidates

Step 2 ​

Localized OCR:

  • footer
  • collector number
  • set symbol
  • specific text zones

Step 3 ​

Hybrid rerank:

  • visual score
  • OCR score
  • set symbol score

Second priority: reranking ​

Current problem ​

ANN cosine similarity does not always match the best real candidate.

The ANN top-1 is often:

  • very close
  • but not exact

Solution ​

Add a secondary reranker:

  • lightweight model
  • pairwise comparison
  • visual cross-encoder
  • fine-tuned siamese

Goal ​

Use:

  • ANN only to retrieve candidates
  • reranker to pick the real top-1

Multi-crop embeddings ​

Current problem ​

The embedding is computed on:

  • the full card

But many discriminative cues are local:

  • collector number
  • footer
  • set symbol
  • rarity
  • promo stamps

Solution ​

Build several embeddings:

  • full card
  • footer
  • artwork center
  • set area

Then:

  • weighted fusion
  • or multi-view rerank

Perceptual hashing ​

Perceptual hashing must not replace ANN.

It should be used for:

  • holo vs reverse
  • exact reprints
  • very close scans
  • false-positive elimination

ANN:

  • top-20

Then:

  • pHash/dHash rerank

DINOv2 fine-tuning ​

Current state ​

The model is used frozen.

That is great for bootstrapping.

But the major future gain will come from:

  • metric learning
  • triplet loss
  • hard negative mining
  • supervised contrastive learning

Dataset already available ​

The project already has:

  • 140k+ cards
  • natural classes
  • implicit hard negatives
  • multi-TCG

The dataset is enough for:

  • a real retrieval fine-tuning

Geometric detection ​

Current state ​

The warp/perspective pipeline is already good.

But hard cases remain:

  • glare
  • sleeves
  • reflections
  • fingers
  • occlusions
  • aggressive angles

Recommendation ​

Evolve toward:

  • card segmentation
  • ML corner regression
  • confidence scoring

FAISS ​

Observation ​

The pgvector + HNSW choice is good.

Advantages:

  • SQL filtering
  • transactionality
  • operational simplicity
  • TCG/language/set filters before ANN

Current problem ​

The FAISS sidecars:

  • use IndexFlatIP
  • do not scale well
  • are now technically inferior to the pgvector pipeline

Recommendation ​

If FAISS is kept:

  • migrate to HNSWFlat
  • or IVF-PQ

Otherwise:

  • favor pgvector only

Variants: the real future problem ​

The current system will struggle on:

  • reverse holo
  • foil
  • textured
  • promo stamp
  • first edition
  • language variants

Cause ​

Global embeddings capture:

  • the general identity

but not enough of:

  • the micro-variations

Recommended roadmap ​

Sprint 1 (max ROI) ​

  • OCR collector number
  • OCR set code
  • top-50 hybrid rerank

Sprint 2 ​

  • multi-crop embeddings
  • pHash rerank
  • rebuild FAISS post-warp

Sprint 3 ​

  • retrieval fine-tuning
  • hard negative mining
  • metric learning

Target architecture ​

capture ↓ card detection + rectify ↓ global ANN retrieval ↓ localized OCR ↓ hybrid rerank ↓ perceptual rerank ↓ confidence scoring ↓ final match

Conclusion ​

The system is already at a serious level.

Future gains will no longer come from:

  • the backend
  • nor the infrastructure

but from:

  • hybrid retrieval
  • reranking
  • targeted OCR
  • metric learning
  • local disambiguation