Documentation

Appearance

2. Mobile warpQuad — native OpenCV perspective warp ​

Published on 2026-05-12

Companion to Backend /scan-frame. The backend expects an already-rectified crop; this page documents the client-side piece that produces it.

Context ​

Path #5 « Frame » on mobile ships across several PRs:

PRWhatStatus
2aNative detectQuad(photoUri) stub returning nullmerged 2026-05-12 (#152)
2bReal OpenCV pipeline behind detectQuadmerged 2026-05-12 (#153)
3Backend /scan-framemerged 2026-05-12 (tcg-collector-backend#148)
4awarpQuad(photoUri, corners) native + JS wrapperthis PR
4b5th "Frame" cell in the mobile fan-outupcoming

PR 4a fills the remaining native gap: given the corners returned by detectQuad, warp the card out of the captured JPEG and save the result as a JPEG in the app cache. The Frame cell (PR 4b) will chain detectQuad → warpQuad and upload the resulting crop to /scan-frame.

Why split 4a / 4b ​

PR 4b focuses on wiring the new mode into the existing fan-out machinery — 90 % of the diff is plumbing, low native risk. Isolating warpQuad in PR 4a lets the native surface (memory hygiene, error contract, output file format) get reviewed on its own before integration.

Native pipeline ​

The Kotlin module QuadDetectorModule (Android) gains a new @ReactMethod warpQuad(photoUri, cornersMap, promise) that runs:

  1. Validate cornersMap shape — expects { topLeft: [x,y], topRight: [x,y], bottomRight: [x,y], bottomLeft: [x,y] }. Missing or malformed field → resolve null.
  2. Decode the JPEG at photoUri (file://, content://, bare path — same loader as detectQuad).
  3. Utils.bitmapToMat → OpenCV Mat.
  4. Imgproc.getPerspectiveTransform(srcCorners, dstCorners) with dstCorners = [(0,0), (W-1,0), (W-1,H-1), (0,H-1)], W=600, H=825.
  5. Imgproc.warpPerspective using the 3×3 matrix.
  6. Utils.matToBitmap → Bitmap.compress(JPEG, q=85) → FileOutputStream into reactApplicationContext.cacheDir/scan-frame-warp-<timestamp>.jpg.
  7. Resolve "file://<absolute path>".

Why 600 × 825 ​

Pokémon TCG card ratio is 63 × 88 mm = 0.716. The pair (600, 825) gives 0.7273, the closest integer-sized canvas. The backend resizes to 518 × 518 before DINOv2 embedding — what matters here is the aspect ratio, sized to avoid stretching artefacts that would degrade similarity scores.

JPEG quality 85 ​

Empirical sweet spot for card crops: upload ≤ 100 KB, no visible block artefacts in the A/B similarity numbers vs /scan-lab.

Memory hygiene ​

Every Mat, MatOfPoint2f, transform Mat, output Bitmap is released/recycled in finally. Otherwise the OpenCV native heap is never reclaimed by the JVM GC — the device OOMs after a few dozen scans.

kotlin
} finally {
  outBitmap?.recycle()
  transform?.release()
  srcPoints?.release()
  dstPoints?.release()
  warped.release()
  src.release()
}

JS wrapper ​

src/scan/quadDetector/warpQuad.ts is a thin async function over NativeModules.QuadDetector.warpQuad. Signature:

ts
export async function warpQuad(
  photoUri: string,
  corners: QuadCorners,
): Promise<string | null>

Defensive on three layers:

  1. Native module absent → null (older APK / iOS).
  2. Native rejection → null (unexpected OpenCV throwable).
  3. Native resolves anything that isn't a non-empty string → null (malformed payload from a future native change).

No as, no !, no any — pure type-guard chain. Corners ({x, y} on the JS side) are mapped to the [x, y] pairs the native side reads.

TDD ​

7 unit tests in __tests__/scan/quadDetector/warpQuad.test.ts:

  • Module absent
  • Native resolves null
  • Native resolves a file URI (happy path)
  • Native resolves an empty string
  • Native resolves a non-string object
  • Native rejects
  • Payload forwarding shape

Result: 331 / 331 jest + tsc --noEmit clean.

Known limitations ​

  • Not validated on device yet: PR 4a ships the mechanism but nothing calls it yet. PR 4b will add the cell that exercises warpQuad chained with detectQuad. Visual validation (look at the resulting crop) lands then.
  • iOS not covered: the project is Android-only (CLAUDE.md). The TS wrapper degrades gracefully (module absent → null) if iOS is ever ported.

Further reading ​