[BiDAF-QANet] Common data processing logic for BiDAF and QANet

[Ishitori]

Description

This is the first PR from the "Merge QANet & BiDAF" project - https://github.com/dmlc/gluon-nlp/projects/7 In this commit I extract common data preprocessing code and put it as a separate module.

Checklist

Essentials

• PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
• Changes are complete (i.e. I finished coding on this PR)
• Code is well-documented

[codecov]

Codecov Report

❗ No coverage uploaded for pull request head (bidaf_qanet_merge_data_process@cc8ecd3). Click here to learn what that means.
The diff coverage is n/a.

[codecov]

Codecov Report

Merging #739 into master will increase coverage by 0.14%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master    #739      +/-   ##
=========================================
+ Coverage   90.56%   90.7%   +0.14%     
=========================================
  Files          65      64       -1     
  Lines        6071    6295     +224     
=========================================
+ Hits         5498    5710     +212     
- Misses        573     585      +12

Impacted Files	Coverage Δ
src/gluonnlp/data/question_answering.py	100% <100%> (ø)	⬆️
src/gluonnlp/data/corpora/google_billion_word.py	66.66% <0%> (-8.34%)	⬇️
src/gluonnlp/data/utils.py	74.14% <0%> (-4.21%)	⬇️
src/gluonnlp/utils/parameter.py	81.35% <0%> (-3.83%)	⬇️
src/gluonnlp/data/word_embedding_evaluation.py	95.45% <0%> (-1.52%)	⬇️
src/gluonnlp/vocab/vocab.py	97.29% <0%> (-0.65%)	⬇️
src/gluonnlp/model/sequence_sampler.py	91.69% <0%> (-0.54%)	⬇️
src/gluonnlp/model/transformer.py	90.05% <0%> (-0.22%)	⬇️
src/gluonnlp/data/glue.py	98.64% <0%> (-0.14%)	⬇️
src/gluonnlp/optimizer/__init__.py	100% <0%> (ø)	⬆️
... and 15 more

[mli]

Job PR-739/1 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/1/index.html

[mli]

Job PR-739/2 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/2/index.html

[mli]

Job PR-739/3 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/3/index.html

[mli]

Job PR-739/4 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/4/index.html

[mli]

Job PR-739/5 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/5/index.html

[mli]

Job PR-739/6 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/6/index.html

[mli]

Job PR-739/7 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/7/index.html

[mli]

Job PR-739/8 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/8/index.html

[mli]

Job PR-739/9 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/9/index.html

[mli]

Job PR-739/10 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/10/index.html

[eric-haibin-lin]

Some minor comments. I'm totally fine if they're addressed in later PRs

[mli]

Job PR-739/12 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/12/index.html

[mli]

Job PR-739/13 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/13/index.html

[mli]

Job PR-739/14 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-739/14/index.html