Handcrafted features, deliberately
Eight computable statistics keep the benchmark a testable hypothesis about classical features — not a showcase for a learned model that would blur what is being measured.
The Grinding Lens · room 03 of 11
CAPTCHA Quality & Robustness Analyzer
How much visual damage can a model actually handle?
A frozen, preregistered research benchmark: can eight handcrafted image features predict the perceived similarity (SSIM) of a degraded synthetic CAPTCHA-style image against its exact clean reference?
It is a measurement instrument, not a security tool — the full pipeline (corruptions, features, splits, replay) is deterministic, versioned and publicly inspectable through a results-only Streamlit dashboard.
K7X-2Mseverity 0 — clean reference
K7X-2Mblur σ1
K7X-2Mblur σ3 + noise
K7X-2Mthin-line + noise
K7X-2Melastic warp (illustrative)
illustrative CSS degradation of one specimen — the benchmark itself measures SSIM against the exact clean reference
The actual problem
Measuring degradation objectively is harder than it looks: 'the image looks damaged' is not a number. The benchmark asks whether simple, handcrafted image statistics — computable without any learned model — can track a principled similarity metric under controlled corruption.
The methodological trap is leakage and irreproducibility: degrade an image, measure it, and tune until the numbers look good, and the result describes the tuning rather than the features. The project was designed to make that trap impossible.
The exploded machine
Synthetic CAPTCHA-style images are generated across a cross-font bench (10 fonts, 300 sources), then passed through a controlled corruption ladder — blur, noise, thin-line, arc-stroke and elastic warp — at graded severities.
the measuring bench — 8 handcrafted features → SSIM
- edge density
- contrast energy
- entropy
- sharpness
- stroke width
- ink coverage
- connectivity
- interference
SSIM vs exact clean reference
Evaluated on source-grouped splits across a cross-font bench (10 fonts / 300 sources) — and the honest headline result is included: the interference feature's R² goes negative.
Eight handcrafted image features are computed on every degraded image and evaluated as predictors of SSIM against the exact clean reference, on source-grouped splits so no source's images appear in both sides of the evaluation.
The benchmark is frozen as a v1.0.0 closed release with provenance audits and deterministic replay — every published number can be regenerated bit-for-bit.
Engineering decisions
Source-grouped splits
Grouping by image source prevents the evaluation from memorising sources — the split honours the way the data was generated, not just the row boundaries.
Frozen v1.0.0 release
The benchmark closed at v1.0.0: no post-hoc adjustments, no silent re-runs — results either reproduce or the claim is withdrawn.
Negative results published as-is
The interference feature's R² goes negative in the published results — I kept it in, because a finding is a finding.
Proof — verified results
- Benchmark status
- v1.0.0 frozen, preregisteredrepo
- Test suite
- 866 passing testsresume-sourced
- Cross-font bench
- 10 fonts / 300 sourcesrepo
- Corruption ladder
- blur · noise · thin-line · arc-stroke · elastic warprepo
- Features evaluated
- 8 handcrafted image features → SSIMrepo
- Interference feature R²
- negative — reported as foundrepo
every number here carries its source — my resume's figures and the synthetic-fixture results are marked for what they are