[enhancement] "What's New" sorting improvement

[M86xKC]

So now the "What's New" page shows the latest video from each channel you subscribed to, but that's not how YouTube subscriptions work, youtubes' subscription list sorts videos by the publish date.

Now in NewPipe each channel have only one video in the "What's New" feed even if they published 10 videos recently "say 1 day ago" at the same time only one will show up in "What's New", and some other channels that you may subscribed to, haven't published in months and you have watched their videos it still shows them every time you go to "What's New".

So it will be nice to sort the "What's New" videos by the publish date.

[theScrabi]

It would be nice if we could have a proper feed manager like typical rss managers that shows how much videos of a channel you didn't see, and which one they are.

[karyogamy]

This problem is noted and discussed in more detail in #739, but may take some time before a good implementation surfaces. In short, the timestamp available on the webpage is insufficient in determining the time of publishing, so the latest video is used as a replacement for getting the feed. Also, in order to implement a proper feed, we have to consider how to deal with other services in the future.

[M86xKC]

@theScrabi @karyogamy @epitron @Generator @timofonic
I found this "https://github.com/ram-on/SkyTube" and it is not feature-rich as NewPipe but its "feed tab" is awesome, (It sorts the videos by time published) I hope you can fork it and get it workin' in NewPipe

[karyogamy]

@pupyc SkyTube uses Youtube API for retrieving video's publish date, something NewPipe will not support.

[jfwerner]

@karyogamy It still knows when the video was published though....

Just make it like YouTube. Just a normal feed. I don't want to click on "What's new" to see the videos. Wouldn't it be much more logical to make that "My subscribed channels" and show the Feed directly?

[epistularum]

In my opinion a subscription feed would be a major addition for anyone wishing to use newpipe as a replacement of the YouTube app.

[theScrabi]

Yes I see it like you. I wished NewPipe had this. Just someone had to do it :)

[wojcik-online]

I've been using NewPipe as a replacement for the YouTube app, and this issue has been bothering me the most. So I've started to work on an implementation, but before adding a PR, I would like to get some input.

How I imagine the feed to work:

• When the feed is first opened, it loads the ChannelInfos of all subscribed channels. This takes some time.
• In the meantime it could display cached items (saved in the db?).
• When loading is finished, it discards all videos older then a month (?) and sorts the remaining ones.
• The textual publication date is parsed. The translated resource strings would have to provide format patterns.

@theScrabi What do you think about it?

[theScrabi]

Sounds good. But isn't the date already parsed in the extractor? If not than the parsing would belong there anyway.

[karyogamy]

@wojcik-online Looks good to me too, though like @theScrabi said, we do have to take into account that different services may have different format patterns on the upload date strings. Also, I'd like request you to not change the database at the moment as it is undergoing a schema overhaul in #1004. By the way, great job on the change you made on feed fragment, I didn't know you can use Rx buffer like that.

[mauriciocolli]

@theScrabi Take a look here: YoutubeStreamInfoItemExtractor.java#L118, we are simply taking what the streaming service is displaying.

---

I think all translation stuff should happen in the front end (NewPipe) but here's some ups and downs on downloading the page in only one language:

Ups:

• We have to handle several languages and changes that the streaming service may be doing, so it would make things easier to parse
• Less testing to check support of another languages (the translation would be handled by the front end)

Downs:

• The content will be in only one language, even if the original content is in another
  • For example, the title and description that the author may be translating (and it is rather common to multiple uploaders translate it to english)
  • I don't know if there's a way to get the original when we are downloading the page accepting only one language (except the feed page, but that would only work in YouTube)

[wojcik-online]

@mauriciocolli I think the page could be downloaded in the language selected by the user and the parsing could take place in the front end. To parse every supported language, format patterns would have to be added to the resource strings.

<?xml version='1.0' encoding='UTF-8'?> 
<resources>
    <string name="service_yt_http_accept_language">en-US</string>
    <string name="service_yt_date_format_month">(\d)+ months ago</string>
</resources>

[mauriciocolli]

@wojcik-online that's the problem, it's just too many and who knows for how long they will be valid.

Even if we did, people would have to actively maintain the multitude of languages (someone who knows, at least, regexp). And then, even if the last point wasn't a problem, there's multiple services that could be using different words/phrases that would amplify the maintenance.

For these reasons, I think the cost of maintenance it's just too high.