Support webusb as a backend #99

Yatekii · 2024-12-22T19:29:30Z

Very rough first implementation of a webusb backend.
Flashing in https://github.com/probe-rs/probe-rs works.

While this works, it's really rough around the edges and straight up does not fully compile. Nevertheless I am already looking for feedback as some of the unsafe primitives of nusb made me struggle a bit so I am sure I did this the wrong way.

Closes #83

kevinmehall

A very nice start!

I haven't carefully reviewed the unsafe, but it's not doing anything as tricky with pointers as the native backends do. You have the ability to keep data in a closure all the way through instead of needing to put everything behind a *mut c_void.

src/platform/webusb/device.rs

kevinmehall · 2024-12-22T21:37:01Z

src/platform/webusb/device.rs

+    //     todo!()
+    // }
+
+    pub(crate) async fn claim_interface(


And that also means making some of the blocking methods async...

Is that a problem? Or do you think we can get away with it? I can maybe move the future to a thread if it blocks too long (tokio does not like that but I also don't want to assume tokio).

This applies to Device::open, Device::claim_interface, Device::set_configuration, Device::reset, Interface::set_alt_setting, Interface::clear_halt.

Some of these operations like set_configuration, set_alt_setting, and clear_halt involve control transfers to the device, so can wait arbitrarily long and really should be async. The problem is that the OS APIs for all of these operations are blocking. As you mention, the "right" way to handle that is using something that runs the blocking operations as a threadpool, like smol's blocking or tokio spawn_blocking. Even without wasm, there could be nusb users that want that to avoid blocking their event loops.

However, the question then is which runtime, and I would like to make it possible to use nusb without needing to pull in either big async runtime for (non-wasm) users who only want blocking operations anyway. Could do Cargo feature flags, but that's a maintenance burden.

The other case is WebUSB operations like release_interface and close that are triggered by dropping nusb types, since Rust doesn't have async drop yet. Since Interface and Device are ref-counted and already wait for transfers to cancel, dropping one already doesn't guarantee that anything is closed right away, so I don't expect this to be a problem. Could offer an async release method consuming self that waits for drop or errors if other references exist, if you for some reason want to be able to release an interface and then claim it again right away.

Trying this out in #100. The idea is that IoAction::wait() would be cfg'd out on wasm, and then you could trivially impl<T> IoAction for T where T: Future.

kevinmehall · 2024-12-22T21:37:09Z

src/platform/webusb/enumeration.rs

+                manufacturer_string: device.manufacturer_name(),
+                product_string: device.product_name(),
+                serial_number: device.serial_number(),
+                interfaces: {


It looks like you should be able to get this info from device.configuration.interfaces[].alternate

https://developer.mozilla.org/en-US/docs/Web/API/USBAlternateInterface

The problem is that this call returns empty alternates, hence the manual shenanigans with reading the descriptors.

Hmm. I try to avoid device IO in list_devices because making requests to random devices can be slow, or even crash them. Less bad in WebUSB, though, because it's only devices selected at the browser prompt.

Hmm I understand that concern. The question is how I get the respective webusb primitives to work then though :/

I just tried it in Chrome on Linux from the JS console and get class/subclass/protocol but not the interface string descriptor. Is that what you're seeing, or is the interface entirely missing?

> d = await navigator.usb.requestDevice({filters: [{}]}) USBDevice {usbVersionMajor: 2, usbVersionMinor: 1, usbVersionSubminor: 0, deviceClass: 0, deviceSubclass: 0, …} > d.configuration.interfaces[0].alternate USBAlternateInterface {alternateSetting: 0, interfaceClass: 255, interfaceSubclass: 0, interfaceProtocol: 0, interfaceName: null, …}

That interface string descriptor is also missing on some driver installation configurations on Windows. Though you probably care about that string descriptor more than most because of CMSIS-DAP.

Correct, that interface string descriptor is missing. For J-Link for example it's not relevant but the CMSIS-DAP spec requires us to check it.

src/platform/webusb/device.rs

src/platform/webusb/transfer.rs

kevinmehall · 2024-12-23T17:41:25Z

src/platform/webusb/transfer.rs

+        let ep_type = self.ep_type;
+        let endpoint_number = self.endpoint & (!0x80);
+        let (mut data, len) = data.into_vec();
+        spawn_local(async move {


I need to look at how Rust async and the JS event loop interoperate, but turning the WebUSB promise into a future and spawning a task to await that future seems like extra steps. I think it should be possible to make a JS closure wrapping a Rust function that that copies the data into linear memory, marks the transfer as complete, and notifies the transfer's waker, basically in the place of the event thread that does that on native platforms.

You will have to go through the future though to make this transfer happen. I am not sure which layer exactly you want to shed?

So spawn_local creates a task with a waker, and JsFuture::from creates a JS callback that arranges for that waker to be woken when the JS promise completes. That waker then queues a poll of the task, where your code handles the transfer completion, and calls notify_completion. The common code wakes another waker stored on the transfer (the waker of the task polling the future returned by nusb), which then queues the user's task.

The TransferInner / PlatformTransfer internals are kind of a mismatch here, because other nusb platforms implement their own event loop but JS comes with one. But if you ignore the event loop itself then notify_completion is effectively a completion callback invoked by "platform code". You could use promise.then() directly with a callback that calls notify_completion instead of converting the promise to a JsFuture. One wrinkle is that it needs both success and error callbacks, where JsFuture uses Rc to share state between the two.

Alternatively, I wonder if there would be a clean way to make the nusb returned future somehow poll the JsFuture, instead of reimplementing what JsFuture does, but I think that would be an invasive change to the cross-platform code.

Not essential and not even sure how much of a performance hit it is to have an extra queueMicrotask hop back to JS, but just a thought.

Ah, yeah, I did not consider the double spawn/waker. I will fix that :) it's bad enough to have so many microticks :D

src/platform/webusb/device.rs

Yatekii · 2024-12-24T01:29:05Z

Cargo.toml

-[target.'cfg(any(target_os="linux", target_os="android"))'.dependencies]
-rustix = { version = "0.38.17", features = ["fs", "event", "net"] }
-libc = "0.2.155"
+# [target.'cfg(all(any(target_os="linux", target_os="android"), not(target_arch="wasm32")))'.dependencies]


Even with the conditional it fails to compile. Not sure how to do this one.

Ok consider me confused. Having this last time I tried made it break. It works fine now.

src/platform/webusb/device.rs

Yatekii · 2024-12-27T12:10:40Z

src/platform/webusb/device.rs

kevinmehall · 2024-12-30T05:28:49Z

src/transfer/mod.rs

+        web_sys::UsbTransferStatus::Ok => Ok(()),
+        web_sys::UsbTransferStatus::Stall => Err(TransferError::Stall),
+        web_sys::UsbTransferStatus::Babble => Err(TransferError::Unknown),
+        _ => unreachable!(),


This should go in platform/webusb, not here.

Weird that these are the only values defined. Specifically, what happens if you disconnect the device?

Done :)

I am not sure what happens. I would expect it to stall.

kevinmehall · 2024-12-30T05:43:29Z

src/platform/webusb/hotplug.rs

+        self.waker.register(cx.waker());
+        match self.events.try_recv() {


Nothing ever wakes the waker, and the channel can't block, so how does this ever get notified of an event?

Seems like instead of the channel, you should be sharing an Arc of the AtomicWaker and a RefCell<VecDeque<HotplugEvent>>.

Ah yes indeed! I fixed the usage of the waker.
Why do you think the channel blocks? try_recv and send should both be nonblocking?

src/platform/webusb/enumeration.rs

kevinmehall · 2024-12-30T06:11:56Z

src/platform/webusb/device.rs

+    //     todo!()
+    // }
+
+    pub(crate) async fn claim_interface(


Trying this out in #100. The idea is that IoAction::wait() would be cfg'd out on wasm, and then you could trivially impl<T> IoAction for T where T: Future.

Yatekii · 2025-05-19T18:19:45Z

I finally rebased this badboi. Probably highly buggy. I will test it the coming days :) I am still terrible at all the unsafe stuff. I need a way to make the WebUsb device handle Send/Sync. It has an internal *mut u8 ptr. I guess just warping it in a uncloneable newtype struct that is Send is enough? And then I wrap that in an Arc<Mutex<>>. How I sync creation of the device wrapper is also an open question but I guess a global list with open devices works :) I am not sure if WebUSB even remembers open devices and only lets you open once. I'll check that too.

I am pretty sure I did most of the things wrong in one or the other way (for example returning the device descriptor) but I'll hopefully clear that up :)

kevinmehall · 2025-05-20T14:46:24Z

I need a way to make the WebUsb device handle Send/Sync

I think the correct way is that the types that wrap WebUsb objects just aren't Send/Sync. That is, we should cfg out the tests checking for those traits when compiling for wasm. There's a good reason that JsValue and related are not Send: they're indexes into a JS-side table of object references, and while the wasm linear memory is shared with web workers, that table is not. So you simply can't create a JsValue on one thread and use it on another.

If that becomes painful for cross-platform use we could consider something like wgpu's fragile-send-sync-non-atomic-wasm feature that makes them Send/Sync but only when atomics are disabled, meaning it can't be used in multi-threaded wasm and those traits are meaningless. But I'd expect wasm atomics and multithreading to become increasingly popular.

kevinmehall · 2025-05-20T14:22:12Z

src/platform/webusb/device.rs

+    /// A queue of pending transfers, expected to complete in order
+    pending: VecDeque<Pending<super::TransferData>>,
+
+    idle_transfer: Option<Idle<super::TransferData>>,


There's no requirement to use crate::transfer::internal and it might be easier and safer not to for webusb. That module is common code for platforms where a transfer is handed off to the OS APIs with a *mut c_void context pointer and a callback that needs to trigger the waker. web_sys already gives you a Future, so I don't think there's any need for that.

I'd just make pending a VecDeque of a new struct containing the JsFuture and the buffer, and poll_next_complete could poll the future from the head of the list. Hopefully no unsafe code needed, except maybe to copy to the uninitialized buffer for IN transfers.

kevinmehall · 2025-05-20T14:31:46Z

src/descriptors.rs

-            warn!("invalid device descriptor bLength");
+            warn!("invalid config descriptor bLength. expected {DESCRIPTOR_LEN_CONFIGURATION}, got {}", buf[0]);


This is validating a device descriptor, not a config descriptor.

Yatekii · 2025-05-20T21:25:07Z

I think we could make a MaybeSend and a MaybeSync but I must say, being Send/Sync is definitely convenient for users of nusb. I see why it's dangerous to assume Send/Sync ofc, but I thought we might be able to make it work with proper synchronization. The UsbDevice pointer is guaranteed to always point to the same object according to spec, so it seems like we could use that.

kevinmehall · 2025-05-21T05:51:24Z

It's not a synchronization issue, but a limitation of how Wasm to JS interop works. If I understand correctly, it will either panic or call the method on the wrong JS object if used from a different thread. I think even in pure JS it's not possible because the WebUSB types are not a transferable object so they exist only in one JS heap.

One option would be to proxy all the calls through a dedicated web worker or the main thread, but that would get complicated and probably require the user to do additional work to set things up.

kevinmehall reviewed Dec 23, 2024

View reviewed changes

Yatekii commented Dec 24, 2024

View reviewed changes

src/platform/webusb/device.rs Outdated Show resolved Hide resolved

Yatekii commented Dec 24, 2024

View reviewed changes

Yatekii commented Dec 26, 2024

View reviewed changes

src/platform/webusb/device.rs Show resolved Hide resolved

kevinmehall reviewed Dec 30, 2024

View reviewed changes

Yatekii force-pushed the task/webusb branch from 1693504 to d7b95cb Compare February 17, 2025 21:48

Yatekii force-pushed the task/webusb branch 2 times, most recently from dad0884 to 97a49a6 Compare May 19, 2025 18:11

Yatekii force-pushed the task/webusb branch 4 times, most recently from 367769c to 1701b7f Compare May 19, 2025 20:45

Add a WebUSB backend

c9b2c93

Yatekii force-pushed the task/webusb branch from 1701b7f to c9b2c93 Compare May 19, 2025 20:49

kevinmehall reviewed May 20, 2025

View reviewed changes

kevinmehall force-pushed the main branch from b0b4a54 to 4009138 Compare July 26, 2025 23:21

		self.waker.register(cx.waker());
		match self.events.try_recv() {

		warn!("invalid device descriptor bLength");
		warn!("invalid config descriptor bLength. expected {DESCRIPTOR_LEN_CONFIGURATION}, got {}", buf[0]);

Support webusb as a backend #99

Are you sure you want to change the base?

Support webusb as a backend #99

Uh oh!

Conversation

Yatekii commented Dec 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kevinmehall left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kevinmehall Dec 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Yatekii commented Dec 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Yatekii commented May 19, 2025

Uh oh!

kevinmehall commented May 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Yatekii commented May 20, 2025

Uh oh!

kevinmehall commented May 21, 2025

Uh oh!

Uh oh!

Yatekii commented Dec 22, 2024 •

edited

Loading

kevinmehall Dec 23, 2024 •

edited

Loading

Yatekii commented Dec 27, 2024 •

edited

Loading