b2sdk with InMemoryAccountInfo not successfully reauthenticating after authorizationToken expires

[anderoonies]

I'm using https://github.com/ehossack/django-backblaze-b2 as a storage backend, which uses b2sdk to store files in a backblaze bucket.

Around 24 hours after deployment, I see frequent errors:
unauthorized for application key with capabilities 'readFiles,listBuckets,shareFiles,listFiles,readBuckets,writeFiles,deleteFiles', restricted to bucket <bucket-name>. My understanding is the authorizationToken has expired and isn't being refreshed.

django-backblaze-b2 instantiates a B2Api, authorizes the account, then doesn't interact with the authentication lifecycle at all, leaving that up to b2sdk. I believe these errors are related to b2sdk, and not django-backblaze-b2.

With these errors, I see the following two requests:

POST https://api.backblazeb2.com/b2api/v2/b2_authorize_account 200
HEAD https://<realm>/file/<path> 401

This leads me to believe that the _reauthorization_loop method is running to attempt to update the authorizationToken, then failing.

Unfortunately I can't replicate this in an environment where I can step through the error, since it requires waiting 24 hours for the authorizationToken to expire.

[mjurbanski-reef]

This is most likely related to the AccountInfo class in use. Which in this case, is in all likeliness `django-backblaze-b2 specific

https://github.com/ehossack/django-backblaze-b2/blob/eb4ecb99e19d0721f234ed268cc6adcd9a9aa448/django_backblaze_b2/cache_account_info.py

This AccountInfo uses django cache (as it should).
My guess is this is a misconfiguration error, or less likely, bug in django-backblaze-b2 . Either way you will have more luck reporting it there.
My suggestion would be to double check settings of django cache used in that account info backend.

Alternatively, if you don't trust neither b2sdk nor django-backblaze-b2 package you can use S3 compatibility and for example https://django-storages.readthedocs.io/en/latest/backends/s3_compatible/backblaze-B2.html as django storage backend if you don't mind a slightly less efficient non-native API implementation. Please note though, that if your caching is badly configured - for example, Dummy or LocMemCache cache is used with multiple django serving processes - you are going to encounter more unexpected errors in your app, so I strongly suggest double checking that first.

[anderoonies]

Thanks for the reply. When we first started seeing this error, my guess was that django-backblaze-b2's caching was causing the issues as well. So I removed caching, and I've configured django-backblaze-b2 to use the SDK's InMemoryAccountInfo, so it should be per-process.

I can confirm that b2sdk is using that class:

>>> from django.core.files.storage import default_storage
>>> default_storage.b2_api.account_info
<b2sdk._internal.account_info.in_memory.InMemoryAccountInfo object at *********>

Something I am seeing now is the AuthInfoCache, not the AccountInfoCache. django-backblaze-b2 initializes the B2Api with cache=AuthInfoCache, which is this. It seems like that cache is only ever cleared here, if the API authenticates with a different account info. Shouldn't that cached AuthInfo also be invalidated when the authorizationToken expires?

[mjurbanski-reef]

AccountInfo class is the one that is important here. AuthInfoCache doesn't matter much, since only thing it does it extends AccountInfo to also cache bucket info. There are no *AccountInfoCache classes. InMemoryAccountInfo is in-memory cache implementation of AccountInfo.

My suspicion was that set_auth_data method of AccountInfo implemented in django-backblaze-b2 was not working correctly when it gets called during reauthorization.
But if you can recreate that error using b2sdk InMemoryAccountInfo that should not be the case.

Please share full traceback of an error, b2sdk version used, ideally with debug logs as well.

The b2sdk version used should not matter much since I don't recall last time when we had to fix the reauth code, but as always I highly recommend using the latest version.