Skip to content

Conversation

mileskrell
Copy link
Contributor

@mileskrell mileskrell commented Mar 12, 2025

What is it?

  • Bugfix (user facing)
  • Feature (user facing)
  • Codebase improvement (dev facing)
  • Meta improvement to the project (dev facing)

Description of the changes in your PR

Enables automatic per-app language support.

On Android 13+, the preference in Settings -> Content shows any locale that has been set by the user through Android's per-app language settings. When tapped, it opens NewPipe's per-app language settings screen.

Also added a method to migrate any existing app language preference from SharedPreferences to the Android per-app language settings when we run the app on Android 13+.

Before/After Screenshots/Screen Record

  • Before:
  • After:
per-app.language.settings.integrated.with.existing.ui.without.toast.mp4

APK testing

The APK can be found by going to the "Checks" tab below the title. On the left pane, click on "CI", scroll down to "artifacts" and click "app" to download the zip file which contains the debug APK of this PR. You can find more info and a video demonstration on this wiki page.

Due diligence

@github-actions github-actions bot added the size/small PRs with less than 50 changed lines label Mar 12, 2025
@TobiGr
Copy link
Contributor

TobiGr commented Mar 12, 2025

I'll paste my comment from #10994 here:

NewPipe has an in-app language chooser (settings > content > app language). This one conflicts with the language chosen in the Android app settings when a language is set. The language chosen inside the app is used. Everything works fine when the "system default" option is chosen. Not sure if this is an unwanted behaviour in terms of accessibility.

Is it possible to check whether the per-app language is set? If it is, we might want to disable the in-app setting and display a hint that the per-app language preference is used instead.

@mileskrell
Copy link
Contributor Author

thanks @TobiGr! I looked briefly for an existing in-app language picker but didn't check in the "content" settings.

Yeah, I'll need to make some more changes so they don't conflict. I should be able to keep the in-app language picker and migrate it to use the new APIs.

@mileskrell mileskrell marked this pull request as draft March 12, 2025 20:41
@TobiGr
Copy link
Contributor

TobiGr commented Mar 12, 2025

@Stypox @AudricV Should these changes be done on the dev branch or the refactor branch?

@Stypox
Copy link
Member

Stypox commented Mar 13, 2025

If it's just a matter of hiding the settings screen language toggle and showing a message instead, in case the language has been changed from outside, then it's ok on the dev branch. If it involves heavy changes to the way the app sets the language then it's better on the refactor branch. Thanks @mileskrell !

@github-actions github-actions bot added size/medium PRs with less than 250 changed lines and removed size/small PRs with less than 50 changed lines labels Mar 16, 2025
@mileskrell mileskrell marked this pull request as ready for review March 16, 2025 02:46
Comment on lines +16 to +21
<Preference
android:key="@string/app_language_android_13_and_up_key"
android:title="@string/app_language_title"
app:isPreferenceVisible="false"
app:singleLineTitle="false"
app:iconSpaceReserved="false" />
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a separate Preference for the Android 13+ behavior only because for some reason when I set an OnPreferenceClickListener on the existing preference that returns true (indicating "the click was handled"), it still opens the existing dialog.

@mileskrell
Copy link
Contributor Author

@TobiGr @Stypox I've updated this PR so it doesn't conflict with the existing in-app language picker. Let me know what you think!

@ShareASmile ShareASmile added the feature request Issue is related to a feature in the app label Mar 16, 2025
Copy link
Contributor

@TobiGr TobiGr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. There is still a toast displayed that the language change is going to take effect with the next start of the app when changing the language on SDK 33+ which is obviously false.

Comment on lines 128 to 150

if (Build.VERSION.SDK_INT >= 33) {
ensureAppLanguagePreferenceIsMigrated(prefs);
}
}

private void ensureAppLanguagePreferenceIsMigrated(final SharedPreferences prefs) {
final String appLanguageDefaultValue = getString(R.string.default_localization_key);
final String appLanguageKey = getString(R.string.app_language_key);
final String appLanguageCurrentValue = prefs.getString(appLanguageKey, null);
if (appLanguageCurrentValue != null) {
// Migrate to Android per-app language settings
prefs.edit().remove(appLanguageKey).apply();
if (!appLanguageCurrentValue.equals(appLanguageDefaultValue)) {
try {
AppCompatDelegate.setApplicationLocales(
LocaleListCompat.forLanguageTags(appLanguageCurrentValue)
);
} catch (final RuntimeException e) {
Log.e(TAG, "Error migrating to Android 13+ per-app language settings");
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks like it should be moved and converted to a Migration in org.schabi.newpipe.settings.SettingMigrations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only concern is that if we convert this to a Migration, my understanding is that it will only run once. So in theory, the following scenario is possible:

  1. Someone running NewPipe on Android <13 sets a custom app language
  2. They update to the new version of NewPipe, where this migration is run but doesn't do anything because they're on Android <13
  3. They update to Android 13+
  4. Their custom app language pref isn't used (because it's running the Android 13+ code path) and it's not migrated from their old preference (because there's nothing triggering the migration to run again)

I recognize this will probably be a pretty rare scenario, just wanted to flag it. But whether or not we make it a Migration, I do recognize that this code belongs somewhere outside of the Application subclass for sure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've moved it to a Migration. If we want to prevent the hypothetical app language setting loss I described above, maybe we could add a new migrations file for "migrations that should always be run on app startup because they depend on the current Android version".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmmh you are right, this shouldn't be a migration as it needs to run every time the app notices it has been updated to Android 13+ from a lower Android version. What do you think @TobiGr?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you are right. My bad. I'd move it to initSettings in NewPipeSettings then. Does that make sense @Stypox ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@mileskrell
Copy link
Contributor Author

mileskrell commented Mar 16, 2025

My bad, totally forgot to fix the toast!

So prior to this PR, we've been showing this toast whenever the app language, content language, or content country is changed. But are we sure that changes to content language/country have ever required a restart? When building from the latest commit on dev, after I set the content language, I'm shown the toast. But if I then open a video that has an audio track for the new content language, that audio track is automatically used without having to restart first:

content.language.changing.without.restart.mp4

If changes to the content language and country are reflected without having to restart, then the only situation where we need to show the toast is when app language is changed on a device running Android <13. Let me know if that sounds correct!

@mileskrell
Copy link
Contributor Author

mileskrell commented Mar 17, 2025

If changes to the content language and country are reflected without having to restart, then the only situation where we need to show the toast is when app language is changed on a device running Android <13. Let me know if that sounds correct!

The most recent commit I pushed assumes this is the case, but we can revert it if not.

Video showing new behavior (no toast when changing only content language):

dont.show.toast.when.changing.content.language.or.country.mp4

@mileskrell mileskrell requested a review from TobiGr March 20, 2025 04:33
Copy link
Member

@Stypox Stypox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good to me, thank you! I have a comment here that could possibly be solved in a separate PR if you prefer. Also, see my comment about the migration.

Also, could you take a video of the updated code on pre-Android-13, so we can see if it still works as expected there?

Comment on lines +100 to +103
androidResources {
generateLocaleConfig = true
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't pay attention to new languages being added from Weblate, this might lead to half-translated being proposed to the user. Currently the list of languages is manually kept in settings_keys.xml, so when new languages are added through Weblate they don't automatically appear in the app menu. However, I'm fine with switching to the new behavior, as it would avoid work to keep the language list up to date. Is it possible to access the locale config from within the app, so we can delete the list of languages in settings_keys.xml and use the autogenerated locale config even for the pre-Android-13 language picker?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found LocaleManager#getOverrideLocaleConfig but it sounds like that method is only to get a LocaleConfig that's already been set with setOverrideLocaleConfig(). As far as I can tell, we would have to manually generate a LocaleConfig instead of using the auto-generated one if we wanted to use it for this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could find the data under app/build/generated/res/localeConfig/debug/xml/_generated_res_locale_config.xml after running the build, with this data inside:

<locale-config xmlns:android="http://schemas.android.com/apk/res/android">
    <locale android:name="en-US"/>
    <locale android:name="ace"/>
    ...

So in theory the data is there, but in practice I think we better not access it by manually opening the xml since it's not officially supported. So yeah let's keep the previous behavior.

@mileskrell
Copy link
Contributor Author

Also, could you take a video of the updated code on pre-Android-13, so we can see if it still works as expected there?

@Stypox I tried it on an emulator running Android 12 and it seems fine:

changing.language.on.android.12.mp4

@Stypox
Copy link
Member

Stypox commented Mar 24, 2025

Thank you! I tested to import an export from a previous version of NewPipe into this PR's build on Android 13 and unfortunately it did not work: the app language remained the one that was there previously before importing settings. This is the database I imported (it should set the language to italian): NewPipeData-20250324_172806.zip

@mileskrell
Copy link
Contributor Author

thanks for catching that and for sharing the settings export @Stypox!

It turns out AppCompatDelegate.setApplicationLocales wasn't working - I missed the part of the documentation that says

Note: This API should always be called after Activity.onCreate(), apart from any exceptions explicitly mentioned in this documentation.

I've moved the migration method to Localization.migrateAppLanguageSettingIfNecessary and I'm calling it at the end of MainActivity.onCreate. Here's a video of the working migration:

fixed.migration.mp4

@mileskrell mileskrell requested a review from Stypox March 27, 2025 23:19
Copy link

Copy link
Member

@Stypox Stypox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested again and it all seems to work as expected. Thank you for making the requested changes!

@Stypox Stypox merged commit 756327d into TeamNewPipe:dev Apr 8, 2025
8 checks passed
@mileskrell mileskrell deleted the mileskrell/support-per-app-language-preferences branch April 8, 2025 21:26
@AudricV AudricV changed the title Support per-app language preferences [Android 13+] Support per-app language preferences May 25, 2025
@AudricV AudricV added device/software specific Issues that only happen on some devices or with some specific hardware/software localisation / translation Everything that has to do with translations or Weblate labels May 25, 2025
@AudricV AudricV removed the request for review from TobiGr May 25, 2025 08:20
Comment on lines +459 to +461
Log.e(TAG, "Failed to migrate previous custom app language "
+ "setting to public per-app language APIs"
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the error be logged?

Comment on lines +52 to +54
final Intent intent = new Intent(Settings.ACTION_APP_LOCALE_SETTINGS)
.setData(Uri.fromParts("package", requireContext().getPackageName(), null));
startActivity(intent);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a very edge case, but if the user has disabled settings or if some ROMs are broken (hello weird Android TV devices), this would crash the app due to no app being able to handle the intent.

@TobiGr
Copy link
Contributor

TobiGr commented Jul 28, 2025

This PR introduced a regression: locals which are not available in Android are not supported by the per-app language preference. E.g. #12476 adds Latin / Latina as language. It's a dead language and thus not available in Android and cannot be selected in the per-app language preferences. However, it can be selected by users with Android 12 and earlier.

This might be related to the fact that not all locals have an ISO 639-1 code. Possible workaround: Set generateLocaleConfig = false and provide our own locale config

@Stypox
Copy link
Member

Stypox commented Jul 29, 2025

So I did a quick comparison of the languages available in the per-app languages vs those available in the in-app chooser. The below lists are not comprehensive of non-latin scripts (there are only a few such languages) because I can't neither write nor compare non-latin scripts 😅. In any case, I think it's pretty clear Android is filtering out a whole bunch of languages. I would say we revert this PR.

Our in-app picker additionally has
  • Bahasa Indonesia
  • Bahasa Melayu
  • Basa Acèh
  • Esperanto
  • 한국어 (korean)
  • Kurmancî
  • Latina
  • Nनेपाली (nepalese)
  • Occitan
  • Toki Pona
  • Босански (bosnian)
  • български език (bulgarian)

I did try setting generateLocaleConfig = false and then explicitly listing locales in locales_config.xml, and despite it listing English, Italian and Latin, only English and Italian would be shown in the system per-app language chooser. So I think there is nothing we can do to make Android show the languages we want in the per-app chooser, it seems like it only supports the most popular languages. @Isira-Seneviratne do you have any insight on this?

locales_config.xml
<?xml version="1.0" encoding="utf-8"?>
<locale-config xmlns:android="http://schemas.android.com/apk/res/android">
    <locale android:name="en-US"/>
    <locale android:name="it-IT"/>
    <locale android:name="la-Latn"/>
</locale-config>

I took a look at the current automatically generated locales_config and it indeed does list all of the languages we support. Actually there are even some we don't support but that have a strings.xml with only a couple strings, and this is kind of a problem as we don't want to make them choosable since they're not really translated:

Android per-app picker additionally has
  • Afrikaans
  • Bamanakan (Mali)
  • Bosanski (latinica)
  • Cymraeg
  • Indonesia
  • Kiswahili
  • Melayu
  • қазақ тілі (kazak)

Stypox added a commit to Stypox/NewPipe that referenced this pull request Jul 29, 2025
@litetex
Copy link
Member

litetex commented Jul 29, 2025

@Stypox

I think some parts of the list are incorrect. Here is the full list of available languages on Android 16.

There is e.g. korean, bulgarian, indonesian and nepalese (random sampling) inside Android's list and I could also locate these languages on my phone (Android 15; CalyxOs on FP5) for NewPipe (nightly) and apply them successfully. Maybe they have slightly different names?

However I couldn't find latin - most likely because it's dead for ~1200 years and not supported (at this point this stuff is just like technical debt).

Anyway I don't think that we should revert this because of 1 dead language, that is used by maybe 2-5? users.

@Stypox
Copy link
Member

Stypox commented Jul 29, 2025

There is e.g. korean, bulgarian, indonesian and nepalese
random sampling

Seems like you randomly sampled exactly those that are indeed in a different place. But these are definitely unavailable (I searched for them): Esperanto, Acehnese, Occitan, Toki Pona.

at this point this stuff is just like technical debt

Yeah but per-app languages are already a more complex system than the simple setting we used to have, and if it's not even on par with previous behavior for the stupid reason it lacks lesser-known languages then it's not so worth it... Why does Android just not include all available languages?!? If they don't have a name for "Latin" they could just show the locale name, "la", and call it a day, instead of hiding the locale completely...

@Stypox
Copy link
Member

Stypox commented Jul 29, 2025

Oh wait:

  • Esperanto and Toki Pona are constructed languages (though Esperanto is widely known so it's strange it's not included in Android)
  • Occitan is basically a dialect
  • Acehnese should exist though, it's spoken by 3 million people: https://en.wikipedia.org/wiki/Acehnese_language

@ShareASmile
Copy link
Collaborator

Esperanto and Toki Pona are constructed languages (though Esperanto is widely known so it's strange it's not included in Android)
Occitan is basically a dialect
Acehnese should exist though, it's spoken by 3 million people

Yes, definitely more than one reason to revert this PR.

@litetex
Copy link
Member

litetex commented Jul 29, 2025

at this point this stuff is just like technical debt

I was sarcastic about the latin language being present in Java and not our code ;)


So I checked again on my phone because the emulator is unreliable:

Esperanto exists ✔️ grafik

Toki Pona

  • has no ISO 639-1 entry (parent language)
  • not present in Android
  • has 3k native speakers, how the heck did we get this language in the first place? There is likely 0 people using the app with this language.
Occitan exists ✔️ grafik

Acehnese

  • has no ISO 639-1 entry (parent language)
  • not present in Android
  • all people that speak it also speak Indonesian because that's the official language of Aceh

it lacks lesser-known languages

I also crunched some numbers:

  • We now have 3 languages that are missing: Acehnese, Latin and Toki Pona - out of 641 that Android supports.
  • These languages have approved translations (that are in our translation files) of 1% each on Weblate and total translations of 2%, 8% and 10%.
  • They were last update: 4 years ago, 4 years ago (beside a single line 5 months ago) and 2 years ago.

So I think we should draw a line here and simply ignore, block and remove these languages:

  • They aren't translated properly in the first place
  • People that use these languages can to 100% also speak another language? Why? Because they couldn't use Android otherwise.
  • They just bloat the app size and maintenance effort
  • They block the adoption of a new Android feature (new Android feature > languages used by like 3 users) and cause the revert.

PS: All this translation stuff wouldn't be required if the Tower of Babel was still standing :P

@Isira-Seneviratne
Copy link
Member

@Isira-Seneviratne do you have any insight on this?

Unfortunately, no. If setting the locales manually doesn't work either, then it would indeed be best to undo the change and use the in-app picker.

@Stypox
Copy link
Member

Stypox commented Jul 30, 2025

Thank you for your analysis! Strangely enough neither Esperanto nor Occitan exist on the emulator I was using and on my FP3 with /e/OS. Maybe it's an OS issue?

Supporting languages with only 1% of translated strings is indeed not very useful, and all other languages seem to be present, so we should be fine. I will close #12478 but still add a warning in the release notes so users can report any issue they have.

@Stypox
Copy link
Member

Stypox commented Jul 30, 2025

I just tested on an emulator with Google Play Services and Esperanto and Occitan are available there. Neither of them are available on my phone (with MicroG) nor on emulators without Google Play Services. So my guess is that Google Play Services somehow adds support for more languages (?).

litetex added a commit to litetex/NewPipe that referenced this pull request Aug 3, 2025
Acehnese, Latin and Toki Pona are not supported by Android:
https://developer.android.com/guide/topics/resources/app-languages#locale-names

They are also completely outdated and have a low amount of translated keys

See TeamNewPipe#12093 (comment) for further details
@enolp
Copy link
Contributor

enolp commented Aug 14, 2025

Be careful with locale support on Android,. As I already said in another issue, some manifacturers or ROMs can expand or reduce locale compatibility. On the other hand, most OEM ROMs are only compatible with the locales they are translated to, but using apps like "MoreLocale2" can show magically those hidden locales in different ways.

EDITED 2025-08-14:

Occitan is basically a dialect

Talking from the linguistic scope, some of you don't know the difference between a language, a variant and a dialect. Spoiler, they are all the same.
Distinctions made between language and dialect come from a very old political prospective. I mean, every contry which has a colonial/nationalist past have a record of "linguistic cleanse" and one of the ways they had to impose their language was degrading linguistic classification of the languages they "conquered" or wanted to erradicate.

They aren't translated properly in the first place

From my point of view, I would get mad if I see my contributions getting removed because NewPipe is unable to find people who can continue my work. Someone spends time for free translating NewPipe for getting their tranlsations removed? No, thanks!

People that use these languages can to 100% also speak another language? Why? Because they couldn't use Android otherwise.

The right question is: What is the NewPipe we want? A NewPipe for the 4 most spoken worldwide languages or an accessible NewPipe for all languages, big, endangered or minorized? My choice is clear, I bet for an inclusive and truly open NewPipe which gives visibility to the real linguistic diversity. You cannot force a user to speak or select the languages you want, that would break the spirit of any free (as freedom) license. The user needs free of choice and right to chose, even more with endangered languages.

all people that speak it also speak Indonesian because that's the official language of Aceh

"Why would you speak your ancient and ugly language instead of speaking a GrEaT, uNiVeRsAl aNd MoDeRn language like (insert language)??"
"You are not allowed to use/speak your language because is unofficial here. Speak the only official language!!!!11!!1!"
"Now we conquered you, we refuse any comumication made in your language. Speak our language now"

They just bloat the app size and maintenance effort

Let me tell you NewPipe suffers from worse issues other than localization...

They block the adoption of a new Android feature (new Android feature > languages used by like 3 users) and cause the revert.

Why not to make an hybrid system? NewPipe could honor system locale while having a proper language selector for all "unsupported" Android locales.

litetex added a commit to litetex/NewPipe that referenced this pull request Aug 16, 2025
Acehnese, Latin and Toki Pona are not supported by Android:
https://developer.android.com/guide/topics/resources/app-languages#locale-names

They are also completely outdated and have a low amount of translated keys

See TeamNewPipe#12093 (comment) for further details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

device/software specific Issues that only happen on some devices or with some specific hardware/software feature request Issue is related to a feature in the app localisation / translation Everything that has to do with translations or Weblate size/medium PRs with less than 250 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants