stutters every so often on macOS

[kujohn]

Describe the bug
Stream is perfect but every so often, maybe around 5-10min, video/audio stutters for few seconds. The controls still work during this time, but stutters and skips until it's over. Then it's flawless streaming until the next stutter.

Moonlight settings (please complete the following information)
I have tested with all variant of settings, changing Video bitrates from 20 ~ 150, v-sync, frame pacing, etc.. Occasional stutter still remains.

Client PC details (please complete the following information)

• OS: Mac OS X 10.14.2
• Moonlight Version: v0.8.0

Server PC details (please complete the following information)

• OS: Windows 10 1803
• GeForce Experience version: 3.16.0.140
• Nvidia GPU driver: 417.35

Moonlight Logs (please attach)
Moonlight-1546797780.log

[cgutman]

I've also seen this periodically on my MacBook, but it doesn't happen on either of my other two non-Mac computers streaming over the same WiFi network. Mine is the Early 2016 MacBook. What model is yours?

My guess is that it's something macOS is doing in the background that's eating up either CPU or network resources. Moonlight doesn't do any periodic work while streaming like that, so I can't imagine what it could be inside Moonlight. I'm going to try to catch the stutter in the Instruments profiler and see what's going on with the system at the time.

[kujohn]

Thanks, I did some testing before your post. I had run the stream and concurrently ping against the host computer and surprisingly the ping did not loose packet at all during the stutter.

369 packets transmitted, 369 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.522/8.366/140.319/12.945 ms

This just reconfirms that it might be a macOS thing and not network inefficiency. Thanks!

[kujohn]

My model is MacBook Pro (Retina, 15-inch, Late 2013

[cgutman]

I wouldn't totally trust the ping results. Ping is sending/receiving about 1 packet per second, while Moonlight (depending on how much is happening on screen) can be receiving 1,000 packets per second or more. If the network was only dropping a fraction of the total packets, the odds that one of the dropped packets would be your ping packet in all of that traffic is very low.

[kujohn]

Ah, good insight. 👍

[cgutman]

I captured a trace of it in Instruments System Profiler and found some interesting stuff:

Timestamp	Narrative	
00:05.286.573	Called "recvfrom(fd:45, buf:0x3ab2200, len:1.37 KiB, flags:0x0,...) = 0 Bytes" for 100.39 ms, end-of-file was encountered.
00:05.286.583	Blocked for 100.30 ms (99.9% of recvfrom's duration) starting at priority 47.  Thread is waiting for event/lock with id 0xa33ba8a859006e9f.	__recvfrom ← (6 other frames)	
00:05.386.880	The thread was made runnable by a timer expiration (handled by CPU 0's timer interrupt handler).  It waited for an available CPU for 66.54 µs (0.1% of recvfrom's duration) before running again at a priority of 47.
00:05.386.947	Ran for 51.42 µs on CPU 0 at priority 47
00:05.386.976	Called "recvfrom(fd:45, buf:0x3ab2200, len:1.37 KiB, flags:0x0,...) = 323 Bytes" for 52.55 ms
00:05.386.998	Blocked for 52.51 ms (99.9% of recvfrom's duration) starting at priority 47.  Thread is waiting for event/lock with id 0xa33ba8a859006e9f.

So it really appears that recvfrom() is blocking for over 100 ms without receiving a packet. The Moonlight receive thread is not being preempted during that time or anything, since it's able to run and call recvfrom() again when the first call times out.

This confirms the issue is not our fault and lies with the network or OS network stack.

[kujohn]

Wow interesting. I wonder if other real-time streaming platforms (PS4 remote play) would have the same problem or if there is a workaround.

[kujohn]

Just want to report some of the things I've tried to resolve the issue that didn't work:

• reset NVRAM
• disable spotlight indexing (adding disk to privacy)
• disable most settings in 'energy saver'
• removing 'mediaanalysisd' daemon from startup source

[euroclydon37]

This whole thread is very insightful. And I've had the same issue for what it's worth. But, it's also worth noting that I get this issue on iOS as well. Both on my 2016 iPad Pro and iPhone 6s Plus.

[kujohn]

During the stutter, there is still interim packets of video/audio that was received. So it seems like recvfrom isn't entirely blocked..

If @euroclydon37 has problems on iOS too, does it mean the mac version and ios version utilize the same framework?

[cgutman]

In the trace I recorded, I saw recvfrom() take over 150 ms to return a single packet at one point. This is consistent with the stutter observed, because we are getting a bunch of data dropped and the rest delayed significantly.

I'm highly suspicious of jumping to conclusions that iOS may be affected. Moonlight iOS and PC have totally different codebases and I haven't personally seen iOS affected myself.

We need to remember that 99.9% of the time these issues really are transient network problems that have nothing to do with Moonlight. It takes pretty extraordinary evidence to prove that it's not the network's fault when we can prove that packets are being delayed or dropped before reaching the client. I'm not even totally convinced this isn't what I'm seeing on my own network.

Very short network congestion conditions arise all the time, especially on WiFi with multiple clients and shared airspace, and real-time remote gameplay is about as intolerant of application to an imperfect network as you can find. In many ways, you are really seeing and hearing your network's performance. When packets are delayed, you see the video run in slow motion. When packets get dropped, you hear the audio crackle as it runs out of audio to play and see the video freeze until the congestion passes.

[kujohn]

So it looks like the packet drop from the recvfrom blockage has something to do with the network layer for OS X, right? I was just wondering if OSX and iOS share the same network layer framework, thus creating similar issues. It's possible that the internal implementation is different though. Sorry, pure speculation on my part since I don't know moonlight codebase.🥉

However, I am going to test it out further on the same machine:

• PS4 remote play
• Windows 10 bootcamp

I'll report back the findings later this week.

[kujohn]

Did some snooping around with packet capture on the server and client, found that during the hiccup, the packets were unable to reach port:

src             dest
192.168.1.20	192.168.1.100	ICMP	70	Destination unreachable (Port unreachable)

And server tries to initiate a new TLS handshake:

src             dest
192.168.1.100	192.168.1.20	TLSv1.2	916	Server Hello, Certificate, Certificate Request, Server Hello Done
192.168.1.20	192.168.1.100	TLSv1.2	1363	Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message
192.168.1.100	192.168.1.20	TLSv1.2	105	Change Cipher Spec, Encrypted Handshake Message

[cgutman]

@kujohn do you have the full packet capture of those packets (including port numbers and the original packet that the ICMP message was referring to)?

There should be no HTTP/HTTPS activity from Moonlight after the stream is established until the session ends.