Blog - Image recognition — from pixels to a decision

Image recognition is not “magic AI” — it is a pipeline: representation → features → model → decision. Here is what actually happens under the hood.

From image to vector

An $H \times W$ RGB image is a tensor $x \in \mathbb{R}^{H \times W \times 3}$. A CNN builds feature maps $f_\theta(x)$. A classifier then outputs logits $z \in \mathbb{R}^{C}$ for $C$ classes.

Probabilities come from softmax:

$$p_i = \frac{e^{z_i}}{\sum_{j=1}^{C} e^{z_j}}$$

The naive decision is $\hat{y} = \arg\max_i p_i$, but production systems usually apply a threshold $\tau$: accept only if $\max_i p_i \ge \tau$.

Pipeline

Detection vs classification

• Classification — one image → one label.
• Detection — object localization (bbox + class), e.g. YOLO / DETR.
• Segmentation — pixel masks (semantic or instance).

For detection, mAP depends on an IoU threshold such as $0.5$:

$$\mathrm{IoU}(A,B) = \frac{|A \cap B|}{|A \cup B|}$$

Common pitfalls

1. Domain shift — studio-trained models fail on factory floors.
2. Calibration — high $p_i$ is not true confidence; measure ECE.
3. Error cost — false positives ≠ false negatives; tune $\tau$ for the business, not for accuracy alone.

Takeaway

Good image recognition is mostly a data-quality + decision-threshold contract, and only then a network architecture.