Skip to content

Conversation

chunhanl
Copy link

  1. The test.py example mentioned at
    segmentation.rst
    does not match the arguments mentioned in test.py
    should probably be change to
    python test.py --dataset ade20k --pretrained --model fcn --backbone resnet50 --eval

  2. The dimension doesn't seem right when calculating the pixACC and mIoU
    The output is 3D and the target is 2D
    The safest way I could think of is to add a new axis if batch dim is missing.
    Might have a better way to deal with.

Result after modification (without training)
Screenshot from 2019-10-17 20-26-19

@chunhanl chunhanl changed the title Add dim check for segmentation Add dim check while calculating mIoU&pixAcc for segmentation test Oct 18, 2019
@chunhanl
Copy link
Author

Apologies for the new spaces, it was automatically modified by Pycharm IDE

@mli
Copy link
Member

mli commented Oct 18, 2019

Job PR-991-1 is done.
Docs are uploaded to http://gluon-vision-staging.s3-website-us-west-2.amazonaws.com/PR-991/1/index.html
Code coverage of this PR: pr.svg vs. Master: master.svg

evaluating the models (VOC results are evaluated using the official server). For example ``fcn_resnet50_ade``::

python test.py --dataset ade20k --model-zoo fcn_resnet50_ade --eval
python test.py --dataset ade20k --pretrained --model fcn --backbone resnet50 --eval
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why make this change?

Copy link
Author

@chunhanl chunhanl Oct 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello,
According in test.py

def parse_args():

The args defined in parse_args() no longer has args 'model-zoo'

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add that back instead. Not sure who removed that.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether this may have conflicts with these settings.

model_prefix = args.model + '_' + args.backbone
if 'pascal' in args.dataset:
model_prefix += '_voc'
withQuantization = True if (args.backbone in ['resnet101']) else withQuantization
elif args.dataset == 'coco':
model_prefix += '_coco'
withQuantization = True if (args.backbone in ['resnet101']) else withQuantization
elif args.dataset == 'ade20k':
model_prefix += '_ade'
elif args.dataset == 'citys':
model_prefix += '_citys'
else:
raise ValueError('Unsupported dataset {} used'.format(args.dataset))
if args.ngpus > 0:
withQuantization = False
if withQuantization and args.quantized:
model_prefix += '_int8'
if not args.deploy:
if args.calibration:
args.pretrained = True
# create network
if args.pretrained:
model = get_model(model_prefix, pretrained=True)
model.collect_params().reset_ctx(ctx=args.ctx)

# inputs are NDarray, output 4D, target 3D
# add new axis if missing batch dimension
# the category -1 is ignored class, typically for background / boundary
if len(output.shape) == 3:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

batch_intersection_union is supposed to work for a batch with 4D.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I'll try to modify from the dataset

@chunhanl
Copy link
Author

python test.py --dataset ade20k --pretrained --model fcn --backbone resnet50 --eval
Screenshot from 2019-10-18 16-35-31

python test.py --dataset pascal_voc --pretrained --model fcn --backbone resnet101 --eval
Screenshot from 2019-10-18 16-31-43

@mli
Copy link
Member

mli commented Oct 18, 2019

Job PR-991-2 is done.
Docs are uploaded to http://gluon-vision-staging.s3-website-us-west-2.amazonaws.com/PR-991/2/index.html
Code coverage of this PR: pr.svg vs. Master: master.svg

for i, (data, dsts) in enumerate(tbar):
if args.eval:
predicts = [pred[0] for pred in evaluator.parallel_forward(data)]
predicts = [pred for pred in evaluator.parallel_forward(data)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect. The output of gluoncv segmentation network is always a list

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants