Add support to print multi system results as JSON

[me-manikanta]

Issue #207

Added logic to convert the multi-system output to a dictionary and print the dictionary as JSON

Verified the changes for the following:

BLEU and CHRF for Multiple Systems

python3 sacrebleu/sacrebleu.py --input data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.KIT.4951.de-en data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.LIUM-NMT.4733.de-en --language-pair hi-en  -lc  --encoding utf-8-sig data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.C-3MA.4958.de-en -m bleu chrf  -sh --format json 

Output:

[
    {
        "System": "data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.KIT.4951.de-en",
        "BLEU": "55.1",
        "chrF2": "73.8"
    },
    {
        "System": "data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.LIUM-NMT.4733.de-en",
        "BLEU": "52.2",
        "chrF2": "72.1"
    }
]

Paired test using bootstrap resampling

python3 sacrebleu/sacrebleu.py --input data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.KIT.4951.de-en data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.LIUM-NMT.4733.de-en --language-pair hi-en  -lc  --encoding utf-8-sig data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.C-3MA.4958.de-en -m bleu chrf  -sh --format json --paired-bs

Output:

[
    {
        "System": "Baseline: data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.KIT.4951.de-en",
        "BLEU": {
            "score": 55.05078088066641,
            "p_value": null,
            "mean": 55.04298847040095,
            "ci": 0.7779333388823026
        },
        "chrF2": {
            "score": 73.81214722094455,
            "p_value": null,
            "mean": 73.79934401771284,
            "ci": 0.48946591806097217
        }
    },
    {
        "System": "data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.LIUM-NMT.4733.de-en",
        "BLEU": {
            "score": 52.18305014543905,
            "p_value": 0.000999000999000999,
            "mean": 52.18573885163444,
            "ci": 0.7636661706325505
        },
        "chrF2": {
            "score": 72.10939066073507,
            "p_value": 0.000999000999000999,
            "mean": 72.10817046815828,
            "ci": 0.5031330617618934
        }
    }
]

Paired test using approximate randomization

python3 sacrebleu/sacrebleu.py --input data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.KIT.4951.de-en data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.LIUM-NMT.4733.de-en --language-pair hi-en  -lc  --encoding utf-8-sig data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.C-3MA.4958.de-en -m bleu chrf  -sh --format json --paired-ar

Output:

[
    {
        "System": "Baseline: data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.KIT.4951.de-en",
        "BLEU": {
            "score": 55.05078088066641,
            "p_value": null,
            "mean": null,
            "ci": null
        },
        "chrF2": {
            "score": 73.81214722094455,
            "p_value": null,
            "mean": null,
            "ci": null
        }
    },
    {
        "System": "data/wmt17-submitted-data/txt/system-outputs/newstest2017/de-en/newstest2017.LIUM-NMT.4733.de-en",
        "BLEU": {
            "score": 52.18305014543905,
            "p_value": 9.999000099990002e-05,
            "mean": null,
            "ci": null
        },
        "chrF2": {
            "score": 72.10939066073507,
            "p_value": 9.999000099990002e-05,
            "mean": null,
            "ci": null
        }
    }
]

Other tests

Similar results are generated when using --paired-bs-n and --paired-ar-n

[mjpost]

These seem to be randomly failing due to downloading timeouts.