A defect detector for optical surface inspection

Trained on DAGM 2007, a real, published, CC BY 4.0 industrial optical inspection benchmark: 512×512 greyscale texture images with weak elliptical defect annotations. Classes 1 and 2 were used for training, 180 defective images among 1,380. Class 3 was held out of training entirely and we will come back to it. Classes 7 to 10 are the competition classes; the authors ask that they are not used during development, so they were never downloaded.

COCO AP @ 50:95

0.341

the standard primary metric

AP @ 50

0.818

the number in most vendor decks

AP @ 75

0.239

same model, tighter box

Recall at best F1

85.5%

precision 86.7%, 76 defects

Judge it before we go on 0.341 mean average precision. That is a mediocre score and nothing has been sabotaged to produce it — no crippled backbone, no withheld epochs. Would you deploy this? What would you do next to improve it? Answer before you scroll.

Which number did you look at?

The same model on the same 576 images scores 0.818 at IoU 0.50 and 0.239 at IoU 0.75. The detector is largely right about whether there is a defect and much less right about exactly where. Quoting AP@50 is not fraud, it is convention — and it is also the most common way an inspection benchmark flatters itself.

DAGM ships weak ellipse masks. The boxes this model was trained against are approximations drawn around approximations. A large part of the gap above is not the model failing to learn; it is the model faithfully reproducing labels that were never tight. Hold that thought until screen 3.

What the room says to do about it

Record the instincts now. Screen 10 comes back to this list with measurements against every one of them.

Training record

14 epochs in 181 s on Metal GPU via PyTorch MPS on AMD Radeon Pro 5300M, float32. Validation average precision at IoU 0.50, evaluated on the CPU in numpy — metric computation is kept off the Metal backend because it has no float64 and pycocotools is float64 throughout.

The model, and its licence

Architecture: torchvision ResNet-18 (BSD-3-Clause) truncated at layer3, stride-8 fusion, CenterNet-style anchor-free head
Parameters: 2.88 M
Input: 256×256 greyscale, stride 8
Backbone licence: BSD-3-Clause — torchvision ResNet-18, ImageNet weights
Dataset licence: CC-BY-4.0 — DAGM 2007, DOI 10.5281/zenodo.12750201
Trained on: Metal GPU via PyTorch MPS on AMD Radeon Pro 5300M, float32
Hardware: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (x86_64), 12 logical cores, macOS 14.7.6, GPU AMD Radeon Pro 5300M

No Ultralytics, no AGPL, no GPL. Backbone weights are torchvision ImageNet, BSD-3-Clause. See screen 9 for why that sentence is the point of this demonstration rather than a footnote.

Try it on your own image → Real inference, on this server's CPU, through ONNX Runtime.

precomputed Weights and per-image predictions built by scripts/train_baseline.py at 2026-08-09T19:23:35+00:00 on Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (x86_64), 12 logical cores, macOS 14.7.6, GPU AMD Radeon Pro 5300M. computed just now Every average-precision figure on this page, including the IoU sweep, was recomputed from those predictions during this request.