Copy-Pasting Coherent Depth Regions Improves Contrastive Learning for Urban-Scene Segmentation

November 25, 2022 · Entered Twilight · 🏛 British Machine Vision Conference

Repo contents: MScThesis.pdf, README.md, __init__.py, dataloader, detectron2_model, eval_unsupervised.py, fig, model.py, pixel_grouping, run.py, trainer.py, utils.py

Authors Liang Zeng, Attila Lengyel, Nergis Tömen, Jan van Gemert arXiv ID 2211.14074 Category cs.CV: Computer Vision Citations 0 Venue British Machine Vision Conference Repository https://github.com/LeungTsang/CPCDR ⭐ 2 Last Checked 1 month ago

Abstract

In this work, we leverage estimated depth to boost self-supervised contrastive learning for segmentation of urban scenes, where unlabeled videos are readily available for training self-supervised depth estimation. We argue that the semantics of a coherent group of pixels in 3D space is self-contained and invariant to the contexts in which they appear. We group coherent, semantically related pixels into coherent depth regions given their estimated depth and use copy-paste to synthetically vary their contexts. In this way, cross-context correspondences are built in contrastive learning and a context-invariant representation is learned. For unsupervised semantic segmentation of urban scenes, our method surpasses the previous state-of-the-art baseline by +7.14% in mIoU on Cityscapes and +6.65% on KITTI. For fine-tuning on Cityscapes and KITTI segmentation, our method is competitive with existing models, yet, we do not need to pre-train on ImageNet or COCO, and we are also more computationally efficient. Our code is available on https://github.com/LeungTsang/CPCDR