The aggregate score was hiding where the failures are

One number over a mixed population is an average of things that behave nothing alike. Below, the same predictions are partitioned four ways — by defect class, by defect size, by local contrast, and by scene brightness. Size, contrast and brightness were measured per image at build time from the pixels, so these are real partitions of real data, not an illustration of what slicing would look like.

Include Class 3, which was held out of training –

Watch the aggregate move Tick and untick that box. The headline score changes by more than any modelling change in this entire demonstration, and nothing about the model changed at all — only which parts it was asked about.

By defect class

Three texture families. One of them was never in the training set.

By defect size

Quartiles of the labelled box area, in pixels squared.

By local contrast

Mean grey level inside the defect box against an annulus of its own background. This is the number that decides whether a defect is findable at all.

By scene brightness

Mean grey level of the whole frame — the lighting proxy.

Every slice is confounded, and pretending otherwise is how teams waste a quarter

The brightness slice looks like a devastating finding about lighting. Check the composition column below before you act on it.

Per-image, not per-bucket

Each point is one defective image: its measured local contrast against the best IoU the detector achieved on it. Buckets hide the shape; this does not. Click a legend entry to isolate a class.

What this rules out

precomputed Per-image predictions from scripts/train_baseline.py; per-image contrast, brightness and defect area measured from the pixels by scripts/prepare_data.py. computed just now Every slice, every average precision and every composition count on this page was computed during this request, which is why the toggle above works at all.