Download thumbnail and media at the same priority #650
Conversation
|
No real objection to this but can you detail why this is preferable? Tasks are still going to be processed in queue order I imagine. |
|
It means some videos download before every thumbnail has finished. |
|
Yeah obviously I got that. I meant this is only really ever an issue when adding a large source and the initial scan and sync and this patch will delay thumbnail downloads which are pretty fast, potentially, until all media items are downloaded. |
|
As I understand it, the thumbnails won't all be delayed. At the same priority, I'm hoping to have these tasks interleaved. I see huge wait times for the current code because every channel I add has to do a metadata task for every video on the channel, then every thumbnail downloads, and as soon as that's done another channel can be indexed. It takes a really long time to get to the first video. I'm looking at increasing the workers also to address this.
The queue names are fairly useless since the worker isn't filtering on them. Or did I misunderstand something? |
|
Yeah I see. It's probably safe to increase the number of workers for thumbnails, they're direct requests to the CDN without really making requests to YouTube. The calls to
No you've not missed anything, they're just indexed by time. I meant if a scan detects a thousand missing thumbnails and queues up a thousand thumbnail tasks they'll still be processed in a big block before any downloads. I imagine if you're adding a channel the thumbnails and downloads would be interleaved, yes. |
They are only processed as a big block because the thumbnail tasks are sorted before any video downloads. The scheduling happens one video at a time, so the thumbnail followed very quickly by the video. At the same priority, I expect completed videos much faster. |
|
Yep fair enough. I'll merge this but comment back on here if this does improve your download times just because I'm curious. |
No description provided.