Skip to content

Conversation

@IsaMorphic
Copy link
Contributor

What does the pull request do?

This pull request adds a much-needed implementation of UI automation / screen reader accessibility features to the Android implementation of Avalonia. It does so by implementing Android's ExploreByTouchHelper built-in as a compatibility shim between Avalonia's own AutomationPeer API and Android's AccessibilityNodeInfo API.

What is the current behavior?

The current behavior of Avalonia on Android is to essentially operate as a dynamically rendered raster SurfaceView which is completely opaque to screen readers such as Android's TalkBack or HumanWare's KeySoft. This is completely unsuitable for users who have visual and/or cognitive impairments and need a screen reader to help them understand what's on the screen.

What is the updated/expected behavior with this PR?

The new behavior that this PR implements is a fully functional prototype integration between Avalonia and Android's various accessibility APIs. This integration includes support for all relevant specialized "providers" within the AutomationPeer API and creates a performant, responsive, and verbose user experience for visually impaired users of Avalonia apps on Android.
To test, enable TalkBack on your Android mobile device and run the ControlCatalog sample. You will observe that the "explore by touch" feature now responds appropriately to user input and allows them to hover over interactive elements like buttons and text fields in order to focus them and have their attached labels read aloud.

How was the solution implemented (if it's not obvious)?

The solution was implemented by implementing a specialized AccessibilityDelegateCompat via Android's ExploreByTouch abstract. This delegate acts as a plug-in for Avalonia's existing automation features into Android's accessibility APIs designed for custom controls.

Checklist

Breaking changes

This PR only modifies internal Avalonia.Android functionality and as such does not expose any new APIs or make changes to the existing API surface.

Fixed issues

@IsaMorphic IsaMorphic force-pushed the feature/android-touch branch from cf694f6 to 89c4fff Compare December 5, 2024 23:18
@IsaMorphic IsaMorphic force-pushed the feature/android-touch branch from 89c4fff to d64e513 Compare December 5, 2024 23:29
@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053729-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@cla-avalonia
Copy link
Collaborator

cla-avalonia commented Dec 6, 2024

  • All contributors have signed the CLA.

@IsaMorphic
Copy link
Contributor Author

@cla-avalonia agree

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053735-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053745-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053747-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053749-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053751-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053865-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053891-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053983-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054029-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054087-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054155-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054190-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054200-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054211-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054295-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@maxkatz6
Copy link
Member

Thank you!

@maxkatz6 maxkatz6 added this pull request to the merge queue Jan 20, 2025
@IsaMorphic
Copy link
Contributor Author

You betcha! :3

Merged via the queue into AvaloniaUI:master with commit 57c8595 Jan 20, 2025
10 checks passed
@IsaMorphic IsaMorphic deleted the feature/android-touch branch September 4, 2025 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Accessibility support for targeted platforms needs to be considered

6 participants