yolocomputer-visionpythonobject-detectionmachine-learning

How I Solved YOLO Bounding Box False Positives with a Dual-Model Detection Strategy

·6 min read
How I Solved YOLO Bounding Box False Positives with a Dual-Model Detection Strategy

How I Solved YOLO Bounding Box False Positives with a Dual-Model Detection Strategy

When you're building a production background removal service that crops people from photos, getting the bounding box right is everything. A missed detection means you lose part of someone's head. A false positive means you crop around a mannequin in the background instead of the actual person. I spent weeks chasing bounding box bugs across different YOLO models before landing on a solution that none of them could deliver alone: running two models in parallel and merging their detections intelligently.

The Problem

Our service removes backgrounds from product photos — specifically, photos of people wearing clothing for an e-commerce platform. The pipeline is straightforward: detect the person with YOLO, remove the background with BiRefNet, crop to the bounding box, and return the result. Simple in theory.

In practice, every single YOLO model I tried had a different failure mode.

YOLOv8n (the small, fast model) was our first choice. It ran quickly and detected people reliably — most of the time. But it had a nasty habit of hallucinating small person detections in cluttered backgrounds. A texture on a wall, a coat hanging on a rack, a reflection in glass — yolov8n would confidently report these as people. When you union all "person" bounding boxes together (because sometimes models are photographed in groups), one false positive 200 pixels away from the real subject would stretch the crop box way out, ruining the output.

YOLOv8m (the medium model) was more accurate. It didn't hallucinate as often. But it had its own blind spots — it would occasionally miss a person's extended arm, or fail to detect a second person standing close to the first. YOLOv8n would catch those cases that yolov8m missed.

I also tried YOLO11m (yolo26m.pt) hoping a newer architecture would solve both problems. It didn't. Different model, different failure modes, same category of problem.

The frustrating part: for any given photo, at least one model would get it right. But no single model got it right for all photos.

Single model failure modes — YOLOv8n catches ghosts, YOLOv8m misses detections

The Insight

After switching between models three times and looking at hundreds of failure cases, a pattern emerged:

  • yolov8m rarely hallucinates, but sometimes misses real detections
  • yolov8n rarely misses, but sometimes hallucinates

These failure modes are complementary. What if I ran both and used yolov8m as the "anchor" to filter yolov8n's false positives?

The Dual-Model Architecture

Here's the approach I landed on:

Dual-model detection pipeline — parallel inference through trust zone filtering to merged output

1. Run Both Models in Parallel

with ThreadPoolExecutor(max_workers=2) as pool:
    future_acc = pool.submit(run_accurate)   # yolov8m @ 1280px
    future_fast = pool.submit(run_fast)      # yolov8n @ 640px
    acc_dets = future_acc.result()
    fast_dets = future_fast.result()

yolov8m runs at 1280px input resolution with a high confidence threshold (0.5+). yolov8n runs at 640px with a low confidence threshold (0.25) — intentionally permissive so it catches everything. The ThreadPoolExecutor means the latency cost is roughly the slower of the two, not the sum.

2. Build a Trust Zone from yolov8m

yolov8m's detections define the "trust zone" — the region of the image where we expect people to actually be:

if acc_dets:
    xs = [b[0] for b, _ in acc_dets] + [b[2] for b, _ in acc_dets]
    ys = [b[1] for b, _ in acc_dets] + [b[3] for b, _ in acc_dets]
    gw = max(xs) - min(xs)
    gh = max(ys) - min(ys)
    mx, my = gw * YOLO_PROXIMITY_MARGIN, gh * YOLO_PROXIMITY_MARGIN
    bounds = (min(xs) - mx, min(ys) - my, max(xs) + mx, max(ys) + my)

The trust zone is the bounding hull of all yolov8m detections, expanded by 1.5x in each direction. This margin is generous enough to catch nearby detections that yolov8m missed, but tight enough to reject phantom detections on the other side of the image.

Trust zone filtering — v8m anchors the zone, v8n detections outside it get rejected

3. Filter yolov8n Detections

Every yolov8n detection must pass two filters to survive:

Size filter — reject tiny boxes that are less than 3% of the image area. Real people in product photos take up a meaningful portion of the frame. A 30x40 pixel "person" detection on a 2000px image is almost certainly noise.

if (bw * bh) / (img_w * img_h) < YOLO_MIN_BOX_RATIO:
    continue

Proximity filter — the center of the yolov8n box must fall inside the trust zone. If yolov8m says the people are in the left half of the image, a yolov8n detection in the far right is rejected.

cx, cy = (box[0] + box[2]) / 2, (box[1] + box[3]) / 2
if not (bounds[0] <= cx <= bounds[2] and bounds[1] <= cy <= bounds[3]):
    continue

4. NMS Merge

After filtering, we have yolov8m's detections (all trusted) plus yolov8n's surviving detections. Many of these overlap — both models detected the same person. Standard NMS (Non-Maximum Suppression) with IoU > 0.7 deduplicates them, keeping the higher-confidence box:

def _nms_merge(detections):
    dets = sorted(detections, key=lambda d: d[1], reverse=True)
    keep = []
    for box, conf in dets:
        if not any(_iou(box, kb) > YOLO_NMS_IOU for kb, _ in keep):
            keep.append((box, conf))
    return keep

5. Union + Asymmetric Padding

The surviving boxes are unioned into a single bounding rectangle, then padded asymmetrically — 5% on top (to catch hair and hats), 2% on sides and bottom:

pad_x = int((x2 - x1) * 0.02)
pad_y_top = int((y2 - y1) * 0.05)
pad_y_bottom = int((y2 - y1) * 0.02)

The top padding is deliberately larger because the most common crop failure in product photography is cutting off the top of someone's head. Losing 2% of empty space below someone's feet is invisible; losing the top of their hair is immediately noticeable.

Asymmetric padding — 5% top to protect hair, 2% on sides and bottom

Configuration

The whole system is tuned by five constants:

Parameter Value Purpose
YOLO_FAST_CONF 0.25 Low threshold so yolov8n catches everything
YOLO_NMS_IOU 0.7 How much overlap before two boxes are "the same"
YOLO_MIN_BOX_RATIO 0.03 Reject detections smaller than 3% of image area
YOLO_PROXIMITY_MARGIN 1.5 How far from yolov8m's zone to accept yolov8n detections
imgsz 1280 / 640 Resolution for accurate / fast model

These values came from testing against a few hundred real product photos. The 1280px input for yolov8m was a key upgrade from the default — it dramatically improved detection of people's extremities (hands, feet) that were being missed at lower resolutions.

Results

The dual-model approach eliminated the two categories of failure we were seeing:

  • False positives gone: yolov8n's phantom detections in backgrounds, reflections, and textures are filtered out because they fall outside yolov8m's trust zone or are too small.
  • Missing detections recovered: Cases where yolov8m missed an extended arm or a second person are caught by yolov8n's lower confidence threshold and broader detection.

The latency cost is minimal because both models run in parallel. On our c6a.2xlarge instance (8 vCPU), the dual detection adds roughly 100-200ms over a single model — a small price for eliminating crop failures that required manual correction.

What I Learned

No single model is universally best. I went through yolov8n → yolov8m → YOLO11m → back to yolov8m, each time thinking the next model would solve everything. The breakthrough came from accepting that different models have different strengths and engineering a system that combines them.

Use the accurate model as the anchor, not the fast one. It's tempting to use the fast model as primary and the accurate model as a check. But in practice, building the trust zone from the more reliable model and filtering the noisier model's output works much better than the reverse.

Asymmetric padding matters. Equal padding on all sides sounds reasonable but doesn't match how product photos are composed. People's heads are at the top of the frame and are the most visually important part — they need more room.

Run in parallel, not sequentially. The ThreadPoolExecutor approach means you pay the latency of the slower model, not the sum of both. On CPU, YOLO inference releases the GIL during the C++ forward pass, so true parallelism works here even in Python.

© 2026 thearifoffice · Made with careBangladesh ↔ GCC · Singapore · Malaysia · Worldwide