Arabic text is extracted in the wrong order

[baaziznasser]

There is a big problem with arabic text extraction.

If we have a string that says (مرحبا هذه تجربة) the PyPDF2 extract_text function returned it like : (ةبرجت هذه ابحرم).

Environment

$ python -m platform
Windows-10-10.0.19044-SP0

$ python -c "import PyPDF2;print(PyPDF2.__version__)"
2.10.3

Code + PDF

This is a minimal, complete example that shows the issue with file.pdf:

from PyPDF2 import PdfReader

reader = PdfReader("file.pdf")

text = ""
for page in reader.pages:
    text += page.extract_text()
    break
print(text)

It gives:

Ang-L1+sociology-globalisation: 
  
 :مﻗر ة رﻀﺎحمﻟا1              : ﺔمﻟوﻌﻟاglobalization : 
 
 : ﺔمﻟوﻌﻠﻟ ﺔ�خ�رﺎتﻟا ﺔ�فﻠخﻟا 
 : ﺢﻠطصمﻟا  ﺢﻠطصمﻟا ﺔمﻟوﻌﻟاmondialisation  ﺔمﻠكﻟا نﻤmonde  ﺔ�نیﺘﻼﻟا ﺔمﻠكﻟا نﻤ ةدمتسﻤmundus 
 
 ﻲنﻌﺘ ﻲتﻟاوunivers.  و ،globe  ﺔ�نیﺘﻼﻟا ﺔمﻠكﻟا نﻤglobus   .ءﻲشﻟا م�مﻌﺘ ﻲنﻌﺘو 
: ﺎﻬﻔ�رﻌﺘ 
  نودﺒ وأ دصﻘ� ﻰﻌسﺘ ﻲتﻟا تا روطتﻟا و تادجتسمﻟا " ﺎﻬﻨﺄ� ﺔمﻟوﻌﻟا بﺎت� ﻒﻟؤﻤ :زرﺘاو موكﻟﺎﻤ ﺎﻬﻓ رﻌ�.دﺤاو ﻲمﻟﺎﻋ ﻊمتجﻤ ﻲﻓ مﻟﺎﻌﻟا نﺎكﺴ ﺞﻤد ﻰﻟإ دصﻗ 
Globalization  ﻞكﻟا ﻞمش�ﻟ ﻪﺘ رﺌاد ﻊ�ﺴوﺘ و ءﻲشﻟا م�مﻌﺘ : ﻲنﻌﺘ ﻲﻬﻓ."ادﺤاو ﺎمﻟﺎﻋ مﻟﺎﻌﻟا ﻞﻌﺠ "يأ ﻞﺴا ر زیﺘ زﻟ رﺎشﺘ ﻲك� رﻤﻷا وﻫ ﺔمﻟوﻌﻠﻟ ﻰﻟوﻻا ئدﺎ�مﻟا ﻊﻀوﺒ مﺎﻗ نﻤ لوأ charles taze russell  
  " ﺢﻠطصﻤ ﻰﻟإ ﻞﺼوﺘ نﻤ لوأ دﻌ� ﻪﻨا ﺎم� ،تﺎ� رﺸ بﺤﺎﺼ و ﺎ�ﻟﺎمﺴأ ر نﺎ� نأ دﻌ� ﺎسﻗ ﺢ�ﺼأ يذﻟا  مﺎﻋ ﻲﻓ"ﺔﻗﻼمﻌﻟا تﺎ� رشﻟا1897 . 
ا ﻲﻓ كﻟذ� ﺢﻠطصمﻟا ﻞمﻌتﺴأ  نﺎﺘ ری�و� رﺎ�ﺒ ف رط نﻤ ة رﻤ لوﻷ ﺔ�سﻨ رﻔﻟpierre de coubertin  ﻲ ﻓ  ة د �ر ﺠle figaro
  م ﺎ ﻋ13  ربمس�د1904 . 
ﻲسﻨ رﻔﻟا ﻲﻓا رﻐجﻟا خ رؤمﻟا بﺎت� ﻲﻓ رﻬظ مﺜ    vincent copdepny  ت� رﺘأ لوﺒ ل بﺎت� ﻲﻓ رشﻨ ، paul otlet   ﺔنﺴ1916 "ﻲنیﺠ نﺎﻓ دﻟوﻨ رأ" دﯿ ﻰﻠﻋ مﺜ.arnold von gennep ,1933   ظ و ﻲﻓا رﻐجﻟا بﺎت� ﻲﻓ كﻟذ� ت رﻬle géographe laurent carroué    ﻰﻟإ ﺔﻓﺎﻀﻹﺎ� ،يوورﺎ� نا روﻟ   ﻪ�ﺸور ﻲﻏguy rocher     نﺎﻫوﻟ كﺎﻤ لﺎﺸ رﺎﻤ دﯿ ﻰﻠﻋ و ، ﺎﻬﻔ� رﻌﺘ لوﺎﺤ يذﻟاmarshall Mc luhan   ﻪﻔﻟؤﻤ ﻲﻓvillage Global "   ﻲﻨﺎ�ﺴﻹا ز� رﯿ ﺎمﻛmanuel castell’s    و ﺔ�دﺎصتﻗﻻا ﻞﻤاوﻌﻟا ﻰﻠﻋ ﻻا ز� ر�و ، ﺔ�عﺎمتﺠjohn urry
   ﺦﻟإ...،ﺔ�ﺴﺎ�سﻟا و ﺔ�فﺎﻘثﻟا ،ﺔ�دﺎصتﻗﻻا ، ﺔ�ﻨﺎسﻨﻹا تا رییﻐتﻟا ﻰﻠﻋ 
: ةﺄشنﻟا : ﺔمﻟوﻌﻟا روﻬظ بﺎ�ﺴأ 
: ﺔ�دﺎصتﻗﻹا ﻞﻤاوﻌﻟا 
 ﺔ�ﻨﺎط� ربﻟا ﺔ�ق رشﻟا دنﻬﻟا ﺔ� رﺸ س�ﺴﺄﺘ . 
 : ﺔ�ﻟﺎﻤو ﺔ� رﺎجﺘ ، ﺔ�عﺎنﺼ ، ﺔ�ﺠﺎتﻨإ ﻞﻤاوﻋ 
- ة روثﻟا و ﺔ�عﺎنصﻟا ة روثﻟا.ﺔ�ﺠوﻟونكتﻟا 
- .ﺔ�ﻟﺎمﻟا تﺎسﺴؤمﻟا لﻼﺨ نﻤ ﻲﻟﺎﻤ دﺎصتﻗا روﻠبﺘ ق� رط نﻋ لاوﻤﻷا سوؤر 
-.لاوﻤﻷا سوؤر لﺎﻘتﻨاو ﺔ� رﺎجتﻟا ﺔ� رحﻟا ةدﺎ� ز 

but it's partially reversed, e.g. the beginning

should be

‫‪:‬‬ ‫‪globalization‬‬ ‫‪:‬‬ ‫العولمة   ‫‪1:‬‬ ‫رقم‪:‬‬ ‫المحاضرة‬
‬

[MartinThoma]

Hi @baaziznasser ,

Thank you for letting us know that there is an issue. It would have been way more helpful if you used the standard bug template. I've added the relevant parts to your post. Please fill out the TODO

[baaziznasser]

@MartinThoma done
as i said on the example the arabic text returned reversed
please try to solve that as soon as you can, because i realy need it before the school come.
am working to make a text reader for the blind users.
help please

[baaziznasser]

i made this function to try to solve the problem
but if there is numbers within the word or none arabic chars it also reversed

def arabic_reverse(string):
pattern = re.compile(r'[ا-يئءؤأإآةًٌٍَُِّْ]')
if not pattern.findall(string):
return string
ar = ""
for line in string.split("\n"):
pattern = re.compile(r'[ا-يئءؤأإآةًٌٍَُِّْ]')
if pattern.findall(line):
line = line[::-1]
arline = []
for word in line.split():
pattern = re.compile(r'[ا-يئءؤأإآةًٌٍَُِّْ]')
if not pattern.findall(word):
arline.append(word[::-1])
else:
arline.append(word)
ar = f"{ar}\r\n{' '.join(arline)}"

[MartinThoma]

Do you have an example pdf which you can upload that shows the issue?

[baaziznasser]

@MartinThoma
here is a pdf test am sorry i wasn't able to upload it to the main post i don't know why

file.pdf

[pubpub-zz]

under analysis.
@baaziznasser,
to confirm my understanding:
a) the arabic characters are inversed : am I right?
I've tried to produce a very simple example using the 3 keys on left of the keyboard
arab.pdf
in the file the code character (purely arbitrary codes)are
<034e> <0321> <032d>
those are then translated into
"\u0626","\u0634","\u0636" = ('ئ', 'ش', 'ض')
it could be reversed, compared to print, but I do not know how to detect RTL(I would like to have the general rule)Any Idea ?

[baaziznasser]

@pubpub-zz thanks for your reply
i don't fully understand what you said but if you mean how to detect the rtl text
i was using this you can get an idea from it

import unicodedata as UD
texts = ['مرحبا'.encode('utf-8').decode('utf-8'), 'Hello']
for text in texts:
				x = len([None for ch in text if UD.bidirectional(ch) in ('R', 'AL')])/float(len(text))
				print('{t} => {c}'.format(t=text.encode('utf-8'), c='RTL' if x>0.5 else 'LTR'))