[DATA] Remove repeated chars during preprocessing

[DrownFish19]

PR types

Function optimization

PR changes

Others

Description

Remove repeated chars during preprocessing.

1. add remove_repeated_chars() to remove the repeated chars.
2. with max_repeated_len to control the max length.
3. add max_repeated_len in README.md.

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 56.56%. Comparing base (37e85e6) to head (b8baf6f).
Report is 1 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #7739   +/-   ##
========================================
  Coverage    56.56%   56.56%           
========================================
  Files          589      589           
  Lines        89964    89964           
========================================
  Hits         50889    50889           
  Misses       39075    39075           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ZHUI]

LGTM

[wawltor]

LGTM