-
Notifications
You must be signed in to change notification settings - Fork 4
Eval 结果
limiao edited this page Nov 26, 2017
·
44 revisions
B4 = BLEU_4
C = CIDEr
M = METEOR
R = ROUGE_L
| model(config) | step | B4 | C | M | R | notes_on_training_etc |
|---|---|---|---|---|---|---|
| ShowAndTell (train) | 193009 | 0.410 | 1.142 | 0.349 | 0.599 | 原基线程序 |
| ShowAndTell (train) | 387562 | 0.417 | 1.198 | 0.350 | 0.604 | 原基线程序 |
| ShowAndTell (train) | 658160 | 0.416 | 1.200 | 0.351 | 0.603 | 原基线程序 |
| ShowAndTell (lr=0.0005 finetune) | 837155 | 0.442 | 1.292 | 0.361 | 0.620 | 原基线程序，从 420000 开始 finetune |
| ShowAndTell (lr=0.001 finetune) | 840000 | 0.453 | 1.334 | 0.367 | 0.628 | 从 420000 开始 finetune |
| ShowAndTell (finetune with decay) | 840000 | 0.472 | 1.396 | 0.374 | 0.639 | 从 420000 开始 finetune, 学习率接续训练时学习率继续 decay |
| ShowAndTell (from scratch) | 420000 | 0.528 | 1.589 | 0.394 | 0.667 | 载入 Inception 后就开始按照原定学习率和 decay 训练整个网络 |
| ShowAndTell (from scratch) | 580411 | 0.530 | 1.601 | 0.395 | 0.668 | 载入 Inception 后就开始按照原定学习率和 decay 训练整个网络 |
| ShowAndTell Data Augmentation | 361163 | 0.413 | 1.191 | 0.351 | 0.603 | |
| ShowAndTell Advanced (Visual Attention, finetune with decay, Data Aug) | 640713 | 0.5732 | 1.8031 | 0.4126 | 0.6896 | 从 105000 开始 finetune, 学习率接续训练时学习率继续 decay, initlr=1.0, decay=0.6 |
| ShowAttendTell | 415284 | 0.513 | 1.521 | 0.384 | 0.657 | 同scratch训练, word embedding为随机初始化常数 |
| ShowAndTell Advanced (Visual Attention, finetune with decay) | 580699 | 0.565 | 1.763 | 0.408 | 0.684 | 从 105000 开始 finetune, 学习率接续训练时学习率继续 decay, initlr=1.0, decay=0.6 |
| ShowAndTell (finetune with decay) | 581226 | 0.535 | 1.643 | 0.397 | 0.672 | 从 105000 开始 finetune, 学习率接续训练时学习率继续 decay, initlr=1.0, decay=0.6 |
| ShowAndTell (finetune with decay) | 581181 | 0.541 | 1.662 | 0.399 | 0.674 | 从 105000 开始 finetune, 学习率接续训练时学习率继续 decay, initlr=1.0, decay=0.9 every 2.0 epochs |
| ShowAndTell (finetune with decay) | 580699 | 0.530 | 1.620 | 0.396 | 0.669 | 从 105000 开始 finetune, 学习率接续训练时学习率继续 decay, initlr=1.0, decay=0.9 every 2.0 epochs, lstm=128 |
| ShowAndTell (finetune with decay) | 600000 | 0.514 | 1.545 | 0.390 | 0.661 | 从 105000 开始 finetune, 学习率接续训练时学习率继续 decay, initlr=2.0, decay=0.5 |
| ShowAndTell Advanced (Scheduled Sampling, finetune with decay) | 581226 | 0.536 | 1.639 | 0.398 | 0.674 | 从 105000 开始 finetune, 学习率接续训练时学习率继续 decay, initlr=1.0, decay=0.6, linear sample rate (train 0 -> 0.25, finetune 0 -> 0.5) |
| Semantic-Attention | 741392 | 0.5546 | 1.7216 | 0.4048 | 0.6805 | 先只训练attribute loss，从180973 开始，训练caption loss + attribute loss, 训练参数包括inception，initlr=4.6，decay=0.6，使得在一起训练时的学习率大概为1.0左右 |
| Semantic-Attention Luong | 740834 | 0.5555 | 1.7393 | 0.4059 | 0.6806 | 先只训练attribute loss，从180000 开始，训练caption loss + attribute loss, 训练参数包括inception，initlr=4.6，decay=0.6，使得在一起训练时的学习率大概为1.0左右 |
| Semantic-Attention idf-weighted | 740684 | 0.5556 | 1.7261 | 0.4053 | 0.6813 | 先只训练attribute loss，从200000 开始，训练caption loss + attribute loss, 训练参数包括inception，initlr=4.6，decay=0.6，使得在一起训练时的学习率大概为1.0左右 |