Pet Detection Under Occlusion

A computer-vision pipeline that generates synthetic obstructions, trains pet detectors, and measures how their results transfer to naturally occluded animals.

Computer Vision
PyTorch
Object Detection
Data Augmentation
COCO Evaluation

Building an occlusion experiment

I built an experiment around a practical detection problem: recognizing pets when furniture and other household objects hide part of the animal.

My pipeline generates annotated training composites, integrates two detector architectures, and compares clean-image performance with a separately annotated real-occlusion collection.

From masks to measured detections

Oxford-IIIT PetPet images + segmentation masksConvert foreground to whole-pet boxes

ADE20KSegmented household objectsFilter, scale, and composite over pets

3,312 clean3,000 synthetic

6,312 training imagesOriginal pet boxes retained under occlusion

YOLOv8nEfficientDet-D0

Clean test3,669 images

Natural occlusion74 images / 90 pet boxes

Each architecture is trained on clean data and on the combined set. The same held-out evaluations measure transfer to real occlusion.

Generating annotated training images

The first challenge was creating useful training examples without labeling every composite by hand. I converted Oxford-IIIT Pet segmentation masks into bounding boxes around the whole animal, with a small amount of padding. Cats and dogs share a single detection class, pet.

I then built a pipeline that extracts household objects from ADE20K scene masks and places them over the training images. Category, size, and shape filters reject unsuitable objects; scaling and placement vary the obstruction, and softened mask edges blend the composite. Each generated image keeps the original whole-pet box, including the hidden part of the animal.

The resulting dataset contains 3,000 synthetic examples alongside 3,312 clean training images. Validation and clean-test images stay separate. Object-to-pet area ratios guide synthesis, although they do not measure how much pet foreground is actually covered after placement.

Training and comparing detectors

I set up clean-only and augmented training runs for YOLOv8n and EfficientDet-D0. Using the same data preparation and evaluation sets made it possible to compare how each architecture responded to the synthetic images.

YOLOv8n used Ultralytics training tools. EfficientDet required custom PyTorch data adapters and a training loop: images and boxes are resized together, batches accommodate different numbers of boxes, and validation loss determines which weights to save. Evaluation maps predictions back to the original image coordinates before computing COCO metrics.

I also tested whether combining the clean and augmented EfficientDet checkpoints helped. The ensemble merges their predictions and removes overlapping lower-confidence boxes with non-maximum suppression at an IoU threshold of 0.55. DETR dataset and batching integration was explored separately; its incomplete training record keeps it outside the final comparison.

Original report figure: training and validation box loss generally decrease across 50 epochs of clean YOLOv8 training. — Original project report, Figure 5. Clean YOLOv8 training and validation box loss across 50 epochs. Open full-size figure in a new tab

What the experiments revealed

The evaluation separates clean-image detection from transfer to 74 naturally occluded photographs containing 90 pet boxes. Mean average precision (mAP) measures detection quality, with higher values indicating better results. The overlap threshold, IoU, sets how closely a predicted box must match its annotation.

Augmented YOLOv8n improved clean-test mAP across IoU 0.50–0.95 from 0.885 to 0.893. On natural occlusion, YOLOv8n performed better with clean training, while EfficientDet-D0 improved modestly with augmentation. The EfficientDet ensemble reached 0.097 mAP at IoU 0.50, below the augmented checkpoint alone.

These comparisons exposed a transfer gap that clean-image evaluation alone would have missed. The real-image collection is small, and synthetic labels cover the whole animal while the real annotations describe visible regions. The experiments did not isolate the effect of that annotation difference.

Real-occlusion evaluationmAP@0.50

Recorded mAP at 0.50 on real-occlusion images
Detector	Clean training	Clean + synthetic training
YOLOv8n	0.471	0.345
EfficientDet-D0	0.098	0.112

Recorded evaluation on 74 real-occlusion images with 90 pet boxes. Best-checkpoint predictions; higher values indicate better detection quality.

Stricter localization metric on the same real-image set: mAP averaged across IoU 0.50–0.95.
Detector	Clean training	Augmented training
YOLOv8n	0.229	0.152
EfficientDet-D0	0.036	0.043

Tools and technologies

Python
PyTorch
Ultralytics YOLOv8n
EfficientDet-D0
effdet / timm
Transformers / DETR
NumPy
OpenCV
Pillow
pycocotools
Jupyter
Matplotlib
Kaggle GPU

Python, OpenCV, NumPy, and Pillow handle image preparation and compositing. PyTorch supports the custom EfficientDet training workflow; Ultralytics provides YOLOv8n, and pycocotools provides the common evaluation metrics. Notebook analysis and Matplotlib figures supported the report.