[YouTube] Mark members-only videos #1280
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Stypox
approved these changes
Feb 21, 2025
ShareASmile
added
enhancement
FineFindus
commented
Feb 21, 2025
There was a problem hiding this comment.
Thinking about it a bit more, maybe it makes sense to rename requiresMembership to isPaidContent (or something similar)? That would keep the naming consistent with the PaidContentException that is already thrown for member only content, and it would be generic enough for other services to use. But I'm not sure if that might conflict with something else, e.g. premium videos?
FineFindus
added a commit
to FineFindus/NewPipe
that referenced
this pull request
Mar 2, 2025
YouTube has started to mix members-only videos into the normal videos in the Videos tab. This filters them, as they cannot be watched without an account, leading to clutter in the feed. Requires TeamNewPipe/NewPipeExtractor#1280. Closes: TeamNewPipe#12011 Closes: TeamNewPipe#12040
FineFindus
added a commit
to FineFindus/NewPipe
that referenced
this pull request
Mar 2, 2025
YouTube has started to mix members-only videos into the normal videos in the Videos tab. This filters them, as they cannot be watched without an account, leading to clutter in the feed. Requires TeamNewPipe/NewPipeExtractor#1280. Closes: TeamNewPipe#12011 Closes: TeamNewPipe#12040
5 tasks
|
What about tests? Is it possible to create tests for the new extractions? |
|
I'm honestly a bit overwhelmed with how to create tests for static class MembersOnlyVideos extends DefaultListExtractorTest<ChannelTabExtractor> {
private static YoutubeChannelTabExtractor extractor;
@BeforeAll
static void setUp() throws IOException, ExtractionException {
YoutubeTestsUtils.ensureStateless();
NewPipe.init(DownloaderFactory.getDownloader(RESOURCE_PATH + "membersOnlyVideos"));
extractor = (YoutubeChannelTabExtractor) YouTube.getChannelTabExtractorFromId(
"@TheLinuxEXP", ChannelTabs.VIDEOS);
extractor.fetchPage();
}
@Override public ChannelTabExtractor extractor() throws Exception { return extractor; }
@Override public StreamingService expectedService() throws Exception { return YouTube; }
@Override public String expectedName() throws Exception { return ChannelTabs.VIDEOS; }
@Override public String expectedId() throws Exception { return "UC5UAwBUum7CPN5buc-_N1Fw"; }
@Override public String expectedUrlContains() throws Exception { return "https://www.youtube.com/channel/UC5UAwBUum7CPN5buc-_N1Fw/videos"; }
@Override public String expectedOriginalUrlContains() throws Exception { return "https://www.youtube.com/@TheLinuxEXP/videos"; }
@Override public boolean expectedHasMoreItems() { return true; }
@Test
@Override
public void testRelatedItems() throws Exception {
final List<StreamInfoItem> allItems = extractor.getInitialPage().getItems()
.stream()
.filter(StreamInfoItem.class::isInstance)
.map(StreamInfoItem.class::cast)
.collect(Collectors.toUnmodifiableList());
final List<StreamInfoItem> membershipVideos = allItems.stream()
.filter(item -> !item.requiresMembership())
.collect(Collectors.toUnmodifiableList());
assertTrue(membershipVideos.size() <= allItems.size());
}
}logAnd just checking if there are |
|
Note about tests: When writing a channel extractor test the channel should be chosen very carefully. I think it's best to chose a channel here that either
Maybe as an alternative a playlist can also be considered. |
|
Added a test using a playlist with only members only videos. |
TobiGr
reviewed
Mar 15, 2025
extractor/src/main/java/org/schabi/newpipe/extractor/stream/StreamInfoItem.java
Outdated
Show resolved
Hide resolved
TobiGr
reviewed
Mar 15, 2025
extractor/src/main/java/org/schabi/newpipe/extractor/stream/StreamInfoItem.java
Outdated
Show resolved
Hide resolved
|
For #1280 (review), you're right, a better approach would be to create an enum that would return availability type (creator membership required, service paid content, upcoming content, available content, geo-restricted, ...), it could be applied to info items and items. This would solve several issues of detecting some content unavailability type in a good way in the extractor. |
litetex
reviewed
Mar 15, 2025
...rc/test/java/org/schabi/newpipe/extractor/services/youtube/YoutubePlaylistExtractorTest.java
Outdated
Show resolved
Hide resolved
AudricV
changed the title
|
As suggested in #1280 (comment), I rewrote the code to return an enum with different availabilities instead. I left out the test for the other types for now. Also, if I remember correctly, for upcoming videos an exception is thrown, so it wouldn't be possible to use the enum, but I would gladly leave this to someone with more experience in the code base :) |
|
@AudricV One general question: should the |
6 tasks
AudricV
previously requested changes
Jul 9, 2025
extractor/src/main/java/org/schabi/newpipe/extractor/stream/StreamInfoItemExtractor.java
Show resolved
Hide resolved
extractor/src/main/java/org/schabi/newpipe/extractor/stream/StreamInfoItem.java
Outdated
Show resolved
Hide resolved
extractor/src/main/java/org/schabi/newpipe/extractor/stream/StreamInfoItem.java
Outdated
Show resolved
Hide resolved
litetex
reviewed
Jul 10, 2025
...org/schabi/newpipe/extractor/services/youtube/extractors/YoutubeStreamInfoItemExtractor.java
Outdated
Show resolved
Hide resolved
FineFindus
force-pushed
the
feat/member-channeltab-info
branch
from
2041b3c to
32cdee8
Compare
I don't think so. For me, availability and privacy (or better for this term visibility), should be different values.
Except my very nitpicking change request which has been not applied, it could make sense to add a new value describing we don't have any info for the availability like Changes LGTM otherwise (unless we want better docs), we need to apply these changes to Soundcloud (and Bandcamp too I think) tracks :) |
FineFindus
force-pushed
the
feat/member-channeltab-info
branch
2 times, most recently
from
9a0f4a8 to
291946d
Compare
FineFindus
force-pushed
the
feat/member-channeltab-info
branch
from
291946d to
7684059
Compare
|
Regenerated the mocks and squashed them into ef0da2b. |
YouTube inserts members-only videos (i.e., videos that require channel membership to watch) into the Videos tab. Because the extractor unable to distinguish between them and "normal" videos, they may appear in subscriptions feeds. This enables the extractor to check if videos require membership, allowing clients to filter them. Ref: TeamNewPipe/NewPipe#12040 Ref: TeamNewPipe/NewPipe#12011
Adds a new field to `StreamInfoItem` to denote the availability of the stream. A stream may be restricted to only certain user groups or times. This allows users to ignore the stream based on their availability, e.g. a app may choose to hide all streams that require payment.
Stypox
force-pushed
the
feat/member-channeltab-info
branch
from
7684059 to
9b98978
Compare
Stypox
approved these changes
Oct 4, 2025
There was a problem hiding this comment.
Looks good to me, thanks! I pushed two commit, one to add the UNKNOWN field as suggested by Audric, and another to add a base test for StreamInfo that currently does nothing useful (since StreamInfo.getContentAvailability() always returns UNKNOWN), but that will force any future contributors that want to implement overrides for StreamInfo.getContentAvailability() to also implement tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
YouTube inserts members-only videos (i.e., videos that require channel membership to watch) into the Videos tab. Because the extractor unable to distinguish between them and "normal" videos, they may appear in subscriptions feeds.
This enables the extractor to check if videos require membership, allowing clients to filter them.
Ref: TeamNewPipe/NewPipe#12040
Ref: TeamNewPipe/NewPipe#12011