Infinite request loop in listObjects() if last item returned is directory

[mboudet]

Hello,

After testing, the issue might be unrelated to delimiters.

When making requests to list objects in containers, there are at least two requests :

The first one is a simple get (with additional parameters if present)

The second one take the last returned item's name and set it as a marker, to make sure all items were returned. The requests stop if the response is either a 204 response code or the body is empty.

If for some reason, the last item does not have the name attribute (I.E, it's a directory)
(Might happen only with directories created with the Swift web interface), the marker is empty, and thus the same request is sent over and over again (since the response is never empty).

Seems related to #214

Example of a response causing issue (via HTTP requests) :

[{u'bytes': 0, u'last_modified': u'2018-02-20T10:12:34.698650', u'hash': u'd41d8cd98f00b204e9800998ecf8427e', u'name': u'test/', u'content_type': u'application/octet-stream'}, {u'subdir': u'test/folder2/'}, {u'subdir': u'test/folder3/'}, {u'subdir': u'test/folder4/'}, {u'bytes': 7424, u'last_modified': u'2018-02-20T14:03:44.385580', u'hash': u'949493fe0b3520e7937d87e0690ba01b', u'name': u'test/virtual-machine-scale-sets.png', u'content_type': u'image/png'}, {u'subdir': u'test/zolder5/'}]

I'm guessing the script tries to set marker to the last item "name", and thus is empty

[haphan]

@mboudet Did you encounter the issue before #214 merged?

Appreciate if you can provide minimum re-producible code.

[mboudet]

The issue was actually fixed by #214 , I still had 3.0.3 locally. It works perfectly now.

Issue actually stems from Horizon with the Swift dashboard : when asked to create a folder, instead of creating a normal object, it create a "folder" object, without the "name" key.

Folder item : {'subdir': 'test/'} instead of {'name': 'test/'}

I'm guessing that the marker sent to the API is the "name" of the last item, thus the issue occurring when a folder is the last one listed.

While the issue is already fixed, it might be interesting to check if a new marker was actually set before re-sending the request, to avoid infinite loops.

[haphan]

Great to hear that. I am going to release 3.0.4 now.