Skip to content

Commit da43065

Browse files
youshyeeyoushyee
committed
update CoSA model
1 parent 9a202d5 commit da43065

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

87 files changed

+159718
-2
lines changed

‎.gitignore‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# create by https://github.com/iamcco/coc-gitignore (Wed Jul 10 2024 00:54:23 GMT+0100 (British Summer Time))
2+
# Python.gitignore:
3+
# Byte-compiled / optimized / DLL files
4+
__pycache__/
5+
*.py[cod]
6+
*$py.class
7+
8+
# C extensions
9+
*.so
10+
11+
# Distribution / packaging
12+
.Python
13+
build/
14+
develop-eggs/
15+
dist/
16+
downloads/
17+
eggs/
18+
.eggs/
19+
lib/
20+
lib64/
21+
parts/
22+
sdist/
23+
var/
24+
wheels/
25+
pip-wheel-metadata/
26+
share/python-wheels/
27+
*.egg-info/
28+
.installed.cfg
29+
*.egg
30+
MANIFEST
31+
32+
# PyInstaller
33+
# Usually these files are written by a python script from a template
34+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
35+
*.manifest
36+
*.spec
37+
38+
# Installer logs
39+
pip-log.txt
40+
pip-delete-this-directory.txt
41+
42+
# Unit test / coverage reports
43+
htmlcov/
44+
.tox/
45+
.nox/
46+
.coverage
47+
.coverage.*
48+
.cache
49+
nosetests.xml
50+
coverage.xml
51+
*.cover
52+
.hypothesis/
53+
.pytest_cache/
54+
55+
# Translations
56+
*.mo
57+
*.pot
58+
59+
# Django stuff:
60+
*.log
61+
local_settings.py
62+
db.sqlite3
63+
db.sqlite3-journal
64+
65+
# Flask stuff:
66+
instance/
67+
.webassets-cache
68+
69+
# Scrapy stuff:
70+
.scrapy
71+
72+
# Sphinx documentation
73+
docs/_build/
74+
75+
# PyBuilder
76+
target/
77+
78+
# Jupyter Notebook
79+
.ipynb_checkpoints
80+
81+
# IPython
82+
profile_default/
83+
ipython_config.py
84+
85+
# pyenv
86+
.python-version
87+
88+
# pipenv
89+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
90+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
91+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
92+
# install all needed dependencies.
93+
#Pipfile.lock
94+
95+
# celery beat schedule file
96+
celerybeat-schedule
97+
98+
# SageMath parsed files
99+
*.sage.py
100+
101+
# Environments
102+
.env
103+
.venv
104+
env/
105+
venv/
106+
ENV/
107+
env.bak/
108+
venv.bak/
109+
110+
# Spyder project settings
111+
.spyderproject
112+
.spyproject
113+
114+
# Rope project settings
115+
.ropeproject
116+
117+
# mkdocs documentation
118+
/site
119+
120+
# mypy
121+
.mypy_cache/
122+
.dmypy.json
123+
dmypy.json
124+
125+
# Pyre type checker
126+
.pyre/
127+
weights/

‎README.md‎

Lines changed: 159 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,159 @@
1-
# CoSA (Coming soon)
2-
code for paper: Weakly Supervised Co-training with Swapping Assignments for Semantic Segmentation
1+
# CoSA                                     [![](https://img.shields.io/badge/ECCV-2024-orange)](https://eccv2024.ecva.net/)  [![arXiv](https://img.shields.io/badge/arXiv-2402.17891-b31b1b.svg)](http://arxiv.org/abs/2402.17891)
2+
3+
**Weakly Supervised Co-training with Swapping Assignments for Semantic Segmentation**
4+
5+
Xinyu Yang, Hossein Rahmani, Sue Black, Bryan M. Williams
6+
7+
8+
9+
[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/weakly-supervised-co-training-with-swapping/weakly-supervised-semantic-segmentation-on)](https://paperswithcode.com/sota/weakly-supervised-semantic-segmentation-on?p=weakly-supervised-co-training-with-swapping)
10+
[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/weakly-supervised-co-training-with-swapping/weakly-supervised-semantic-segmentation-on-4)](https://paperswithcode.com/sota/weakly-supervised-semantic-segmentation-on-4?p=weakly-supervised-co-training-with-swapping)
11+
12+
## Overview
13+
We propose an end-to-end framework for WSSS: Co-training with Swapping Assignments (**CoSA**),
14+
<p align="middle">
15+
<img src="./assets/overview.png" alt="CoSA pipeline" width="1200px">
16+
</p>
17+
18+
---
19+
## Usage
20+
21+
### 1. Data Preparation
22+
23+
<details>
24+
<summary>
25+
COCO dataset
26+
</summary>
27+
28+
#### 1. Download and Extract COCO 2014
29+
``` bash
30+
mkdir coco
31+
cd coco
32+
wget http://images.cocodataset.org/zips/train2014.zip
33+
wget http://images.cocodataset.org/zips/val2014.zip
34+
wget http://images.cocodataset.org/zips/test2014.zip
35+
unzip ./train2014.zip
36+
unzip ./val2014.zip
37+
unzip ./test2014.zip
38+
```
39+
40+
#### 2. Download Segmentation Labels
41+
42+
[Here](https://drive.google.com/file/d/147kbmwiXUnd2dW9_j8L5L0qwFYHUcP9I/view?usp=share_link) to download the COCO segmentation labels `coco_anno.tar` and move it to `coco` dir.
43+
After that you should extract it by running:
44+
```bash
45+
tar -xvf coco_anno.tar
46+
```
47+
then you should have a directory structure like this (the number in the brackets is the number of images):
48+
``` bash
49+
coco/
50+
├── SegmentationClass
51+
│   ├── train2014 (82081)
52+
│   └── val2014 (40137)
53+
├── test2014 (40775)
54+
├── train2014 (82783)
55+
└── val2014 (40504)
56+
```
57+
</details>
58+
59+
<details>
60+
<summary>
61+
VOC dataset
62+
</summary>
63+
64+
#### 1. Download PASCAL VOC 2012 and Extract
65+
66+
``` bash
67+
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
68+
tar –xvf VOCtrainval_11-May-2012.tar
69+
```
70+
#### 2. Download the augmented annotations
71+
[Here](https://www.dropbox.com/s/oeu149j8qtbs1x0/SegmentationClassAug.zip?dl=0) is a download link of the augmented annotations. After downloading ` SegmentationClassAug.zip `, you should unzip it and move it to `VOCdevkit/VOC2012`. The directory structure should be like this (the number in the brackets is the number of images):
72+
73+
``` bash
74+
VOCdevkit/
75+
└── VOC2012
76+
├── Annotations
77+
├── ImageSets
78+
├── JPEGImages (17125)
79+
├── SegmentationClass
80+
├── SegmentationClassAug (12031)
81+
└── SegmentationObject (2913)
82+
```
83+
</details>
84+
85+
86+
### 2. Setup Python Environment
87+
We recommend using Anaconda to create a virtual environment.
88+
``` bash
89+
conda create -yn cosa python=3.10 pip wheel
90+
conda activate cosa
91+
pip install -r requirements.txt
92+
```
93+
after that, you can install some extension packages: `mmcv`, `bilateralfilter` and `pydensecrf` by running:
94+
``` bash
95+
mim install mmcv-lite
96+
pip install git+https://github.com/lucasb-eyer/pydensecrf.git
97+
cd utils/bilateralfilter
98+
#sudo apt install swig
99+
swig -python -c++ bilateralfilter.i
100+
python setup.py install
101+
```
102+
## 3. Train and Evaluate
103+
``` bash
104+
105+
### train and eval coco, you may need to modify the `coco_root` in `run_coco.sh` to the path of your COCO dataset.
106+
sh run_coco.sh
107+
108+
### train and eval voc, you may need to modify the `voc12_root` in `run_voc.sh` to the path of your VOC dataset.
109+
sh run_voc.sh
110+
111+
```
112+
113+
## Tested Environment
114+
- Ubuntu 20.04 LTS x86_64
115+
- CUDA 12.1
116+
- NVIDIA GeForce RTX 3090 x2
117+
- Python 3.10
118+
119+
## Our Results
120+
Semantic performance on VOC and COCO. Logs and weights are available now.
121+
| Dataset | Backbone | Val | Test | Log | Weight |
122+
| :-------: | :--------: | :-----: | :----: | :---: | :------: |
123+
| COCO | ViT-B | 51.0 | - | [log](assets/logs/coco_log.txt) | [weight]() |
124+
| VOC | ViT-B | 76.2 | [75.1](http://host.robots.ox.ac.uk:8080/anonymous/GOZOHI.html) | [log](assets/logs/voc_log.txt) | [weight]() |
125+
126+
Visualization results for CoSA comparing with [MCT](https://github.com/xulianuwa/MCTformer), [ToCo](https://github.com/rulixiang/ToCo) and [BECO](https://github.com/ShenghaiRong/BECO) on COCO:
127+
128+
<p align="middle">
129+
<img src="./assets/coco1.png" alt="COCO Visual1" width="1200px">
130+
</p>
131+
132+
<p align="middle">
133+
<img src="./assets/coco2.png" alt="COCO Visual2" width="1200px">
134+
</p>
135+
136+
Visualization results for CoSA comparing with [MCT](https://github.com/xulianuwa/MCTformer), [ToCo](https://github.com/rulixiang/ToCo) and [BECO](https://github.com/ShenghaiRong/BECO) on VOC:
137+
138+
<p align="middle">
139+
<img src="./assets/voc1.png" alt="VOC Visual1" width="1200px">
140+
</p>
141+
142+
143+
The code and weights for CoSA-MS are coming soon.
144+
145+
## Citation
146+
Please cite our work if you find it helpful:
147+
```bash
148+
@article{yang2024weakly,
149+
title={Weakly supervised co-training with swapping assignments for semantic segmentation},
150+
author={Yang, Xinyu and Rahmani, Hossein and Black, Sue and Williams, Bryan M},
151+
journal={arXiv preprint arXiv:2402.17891},
152+
year={2024}
153+
}
154+
155+
```
156+
157+
158+
## Acknowledgement
159+
This repo is heavily built upon [ToCo](https://github.com/rulixiang/ToCo) and [MCT](https://github.com/xulianuwa/MCTformer). Please consider citing their works if you find this repo helpful.

0 commit comments

Comments
 (0)