VAAPI on N100 Stops Working After 2 Hours, Requires Reboot #16438
Replies: 13 comments 35 replies
-
|
This is likely a bug in the intel_gpu_top tool, I don't think there is any issue with the GPU itself, qsv should be used on this hardware |
Beta Was this translation helpful? Give feedback.
-
|
Thanks. now happens something very strange, when i use VAAPI both camera works fine. when i use qsv, only one camera working. my back camera works good. my front camera doesn't work at all. i added the logs when i try to use only the front camera. 2025-02-14 09:56:59.502341557 [2025-02-14 11:56:59] ffmpeg.entrancezoom.detect ERROR : [vist#0:0/h264 @ 0x564f90aff8c0] [dec:h264_qsv @ 0x564f90ab3e80] Decoding error: Invalid argument |
Beta Was this translation helpful? Give feedback.
-
|
Something has changed Now the VAAPI works with 0.4%. This is a small change but it's still something. Just for the demonstration I did another reboot To see how it's should be. I was thinking about running Frigate directly through Ubuntu in this case i need a new pc for that use case. |
Beta Was this translation helpful? Give feedback.
-
|
I'm having issues with VAAPI on N100 aswell (see: #16609). Mine is totally not working (0% GPU), just wondering how you got yours to work. |
Beta Was this translation helpful? Give feedback.
-
|
Any solutions? Works few hours and after this time GPU 0% |
Beta Was this translation helpful? Give feedback.
-
|
Adding my own experience here. Did a fresh Debian on N100 bare metal install with docker, frigate, coral USB. See ~5% GPU for the first few hours. Then later it is ~0.37%. CPU stays roughly the same FWIW so that may support this simply being a reporting bug but would be good to know for sure. Restarting the container does not change the GPU reporting but rebooting the whole system does. I didn't install any Coral or Intel GPU drivers on Debian assuming the docker container takes care of both. Not sure if that is the correct approach but it seems to work. |
Beta Was this translation helpful? Give feedback.
-
|
have the same issue on my N100 running home assistant with Frigate as an addon. (have tested both Beta and main release). Everything works for a few hours, then suddenly Frigate stops using GPU, and CPU skyrockets. Rebooting the Home Assisstant machine resolves the issue until it repeats itself. I'm using a identical hw on a secondary site, however with Proxmox instead, and Frigate as a docker and a seperate VM. Same config otherwise, but at this location no issue. If I we're to guess there seems the 'res some sort of bug in the HAOS Docker environment causing frigate addon with GPU passthrough not working properly/or stop working |
Beta Was this translation helpful? Give feedback.
-
|
I have the same issue, N100 on Debian, Frigate 0.15 (vaapi) hardware acceleration works for a few hours, then it ssuddenly stops working. needs reboot. always the same. |
Beta Was this translation helpful? Give feedback.
-
|
I don't know what is the fix exactly, but it looks like vaapi is finally working properly in my N100 with Debian 12. I tried to make QSV work, but I could not get it working, althought vaapi got fixed in the process. It was a hell, I cannot say how many things I tried, but resuming: I installed/updated too many things, drivers, intel-media-sdk, backports of the kernel, and anything related to graphics that could cause a problem. That looks like it fixed the problem of vaapi for me. (by now, after 5 days it keeps running) |
Beta Was this translation helpful? Give feedback.
-
|
Hello @Wasted-Engineer, I would like to understand, did you install the custom ffmpeg and solve it? Here I also tried everything, updated several colors such as Linux Debian 12 with Kernel 6.12.32+bpo-amd64 via bookworm-backports repository, updated iHD driver intel-media-va-driver-non-free 23.3.4 and Google Coral updated with the official components of Github. Other things I also did was set memory and resource limits for the GPU, through the GRUB, which also did not solve. Thank you if you can share in detail what you did to reproduce here on my N100. |
Beta Was this translation helpful? Give feedback.
-
|
I'm sorry, I'm Brazilian and I don't think the translation was very good. |
Beta Was this translation helpful? Give feedback.
-
|
@Wasted-Engineer can you tell me the kernel version you are running currently |
Beta Was this translation helpful? Give feedback.
-
|
I'm currently running kernel version: Let me know if you'd like me to share any additional system details. |
Beta Was this translation helpful? Give feedback.
-
|
I never was able to find an officialish source that this is a bug in intel_gpu_top but do we think this is the same thing/the right place to maybe have some more activity? https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/issues/175 I tried |
Beta Was this translation helpful? Give feedback.
-
|
After updating everything possible — kernel (via backports), Intel iHD video drivers, and even Google Coral drivers — unfortunately, none of these steps permanently resolved the issue where However, I recently decided to follow the Frigate documentation and replaced the built-in I mapped this custom
I’ll keep monitoring and will share more findings as I go. |
Beta Was this translation helpful? Give feedback.
-
|
I am running my compiled kernel with the patch applied and will report back if it fixes the problem or is unrelated. @fernandolcardoso the patch I was looking at was not in the kernel (via backports) that you were running. Please report back if your |
Beta Was this translation helpful? Give feedback.
-
|
Despite several adjustments, the runtime was longer this time — around 8 hours — but unfortunately, the issue wasn't resolved. So I’ve decided to run a series of targeted tests now, aiming to pinpoint something more specific. |
Beta Was this translation helpful? Give feedback.
-
|
I would run the mainline kernel or at least 6.14 as a test if you are eager to try something. I am at 24 hrs with my patched kernel so far. I would change no other variables. |
Beta Was this translation helpful? Give feedback.
-
|
My testing is still inconclusive. The behavior seems changed but still not quite right, I might revisit it. |
Beta Was this translation helpful? Give feedback.
-
|
"Here are my current specs for VA-API on Debian 12 with Intel N100:* Key fixes I applied: Added user to render and video groups. And installed the things I say in my first post, I cannot remember if I updated something else... But I confirm is working like a charm, no stops anymore. |
Beta Was this translation helpful? Give feedback.
-
|
So, I continue monitoring, now with all the kernel, gpu + firmware updates, just like what was reported here, but I still suspect that something is wrong.
|
Beta Was this translation helpful? Give feedback.
-
System Configuration and FFmpeg Testing Plan (Frigate 0.16.0-beta3)I'm participating in this discussion because I’m facing exactly the same issue: after about 1 to 2 hours of continuous usage, the GPU (Intel N100 / Alder Lake-N) stops reporting any activity. Both It appears that VAAPI acceleration or the iHD driver itself silently stops working. Several other users have reported the same behavior, which suggests the issue may lie somewhere between the video driver, FFmpeg integration, or kernel-level handling. To ensure a controlled and clean test environment, I fully reformatted the system and reinstalled Debian 12 (Bookworm) from scratch, leaving it in its most basic form, without graphical interface and without third-party or nonfree repositories. My current setup:
Testing PlanI'm starting a new round of diagnostics with the following goals:
If anyone is conducting similar testing or has ideas for additional diagnostic commands or kernel tweaks, I’m happy to collaborate. Once I identify a reproducible behavior or narrow down the root cause, I’ll update this thread with findings. |
Beta Was this translation helpful? Give feedback.
-
|
My test is still going strong...I would wait another day to hear back from me before putting in more effort to be honest What was your result from yesterday #16438 (reply in thread) |
Beta Was this translation helpful? Give feedback.
-
Update on my testingAlthough I initially started with a clean, minimal environment using the most basic versions of drivers and firmware — including a custom static FFmpeg build — the issue persisted. After confirming that the problem was not resolved by downgrading components, I proceeded to update everything: kernel, Intel GPU drivers, firmware, and other dependencies to their latest stable or backports versions. Unfortunately, the behavior remains unchanged. After approximately 1 to 2 hours of continuous usage, the GPU performance drops dramatically, especially when using advanced features like:
In this degraded state:
However, when I disable the features mentioned above and run Frigate with only basic object detection, the GPU remains stable, maintaining an average of 3% usage for multiple days with no signs of failure. This strongly suggests a potential issue related to:
I'll continue investigating, and I’m open to collaboration. If anyone has suggestions (e.g., kernel/sysctl tweaks, FFmpeg parameters, i915 module options), feel free to share.
|
Beta Was this translation helpful? Give feedback.
-
|
My setup
My config is as follows |
Beta Was this translation helpful? Give feedback.
-
|
Hello everyone, I'd like to share the solution I found for the common problem of the Intel N100 GPU stopping after 2 hours of use with Frigate. After several tests, I was able to resolve the situation, and my system has been running stably for over a day now. Here are the adjustments I made:
It's also worth noting that the video driver used by Frigate 0.16 is different from the one on the host system, so updating the system driver doesn't bring any benefits. I only observed this after several tests. I hope this detailed breakdown can help others who are facing the same issue. I believe with these adjustments, the GPU crashing problem is solved for good. |
Beta Was this translation helpful? Give feedback.
-
|
To add to my previous answer, some users might need to update their kernel, which requires the correct repositories. I am using the following sources in my After adding the |
Beta Was this translation helpful? Give feedback.
-
|
This morning/last night I was going to reply that I tried everything you said essentially and still saw the issue For (1) I did After a few hours/overnight, no luck, was nearing reporting 0% again. HOWEVER, I then added I will give it more time, but |
Beta Was this translation helpful? Give feedback.
-
|
I spoke too soon, ~8 hours into it, basically all reporting 0% So I am not sure, I am running on proxmox so I will look into trying to limit the C state on the host (which makes sense of course) |
Beta Was this translation helpful? Give feedback.
-
|
Ok after setting Edit: Only 2 hours later and it is basically 0% again, so I am calling this inconclusive and still have no idea how people are getting to the point where they dont experience this anymore |
Beta Was this translation helpful? Give feedback.
-
|
I share the output of my minipc, just in case... |
Beta Was this translation helpful? Give feedback.
-
|
Hello, |
Beta Was this translation helpful? Give feedback.
-
|
It's just the GPU stats that stop not the actual acceleration |
Beta Was this translation helpful? Give feedback.
-
|
Ah, thank you, I was not aware of this. Will the upcoming release of version 0.16 fix this issue, so that the GPU stats always shows correct figures? |
Beta Was this translation helpful? Give feedback.
-
|
I don't know if this is the case. When my GPU stats stop showing anything, CPU usage spikes from a couple of % to 98-100%, causing the system to be extremely slow. This leaves me to belive that GPU acceleration is actually broken with N100 in some setups |
Beta Was this translation helpful? Give feedback.
-
It's not a frigate issue, it's due to host packages changing |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the details, Nick. Well, now I wonder to whom I should address this issue or perhaps it is already addressed. Or could I change something on my N100 host or within the Home Assistant OS there? Honestly, I am somewhat lost now. |
Beta Was this translation helpful? Give feedback.
-
|
Hello everyone, I've been following the discussion and would like to share my experience, which seems to be quite similar to yours. I'm currently using Frigate with an Intel N100 processor and am also facing the issue of the GPU stopping responding after a period of time, which requires a system reboot. Initially, I had a stability of approximately 2 hours, but after some adjustments and tests, I was able to extend the uptime to something between 12 and 24 hours. However, the problem persists, and we still don't have a definitive solution. To help identify the issue, I installed 
In my setup, I'm using Frigate 0.16 RC4 with 13 IMOU cameras. For detection, I use nHD streams at 5 FPS, and for recording, FHD streams at 25 FPS. To offload the CPU, I'm using a Google Coral M.2. When the system is rebooted, the GPU usage is around 1% to 2% in I've researched the topic extensively and, like many of you, have come to the conclusion that the problem may be related to the Intel Kernel, drivers, or firmware. I've found several discussions on Intel forums and GitHub repositories, where I appreciate everyone's support and collaboration in the search for a solution. For now, I will maintain my current setup, but I'm already considering investing in more reliable hardware in the future that can ensure GPU stability. I sincerely hope we can find a solution to this problem soon. |
Beta Was this translation helpful? Give feedback.
-
|
The GPU also can't just stop being used midway through, the process would have to stop and start over or ffmpeg would throw a memory error |
Beta Was this translation helpful? Give feedback.
-
|
Hello, @NickM-27, Thank you for your response and for your perspective. To try and force the GPU to be used more actively, I enabled some of Frigate's more intensive features, such as Facial Recognition, License Plate Recognition, and Semantic Search. All were configured in the YAML file to use the GPU. In this scenario, I used only one high-quality stream (FHD @ 25 FPS) for recording and another low-quality stream (720p @ 5 FPS) for motion and object detection. The reason for this configuration is that my cameras do not offer a native 720p stream. With this setup, I managed to use the GPU at about 50% of its capacity. However, after some time, the GPU usage dropped drastically to 0%, while CPU usage rose to 20-30%. This behavior is what leads me to believe that the GPU, in fact, stops working. As I mentioned in previous comments, I made several adjustments that managed to extend the GPU statistics' stability time, from about 2 hours to somewhere between 12 and 24 hours. Considering this scenario of more intensive use and the subsequent behavior, how would you explain this abrupt drop in GPU usage and the corresponding increase in the CPU? I appreciate your help in the analysis. |
Beta Was this translation helpful? Give feedback.
-
|
This is still somewhat anecdotal, CPU usage can rise for any number of reasons, even if the GPU just enters a different power state, or any other number of things. Your report is also not consistent with other reports we have seen of the issue. To claim that the GPU is no longer being used in any way we would need more explicit evidence, as this would change the problem or even perhaps mean there are two separate problems with similar behavior. Like I said, when you start ffmpeg it is using the GPU driver to move the frames to and from memory usage, if the GPU suddenly stopped being usable by ffmpeg then it would crash as it was no longer able to move that memory, it won't just silently move to using the CPU for decoding (and if it restarted, it would presumably never start up again since it either works on the GPU or it doesn't work at all when GPU args are set) |
Beta Was this translation helpful? Give feedback.
-
|
Hello, @NickM-27, I completely understand your point of view. The discussion I'm proposing here is more related to the analysis of telemetry data than to failures in Frigate or FFmpeg. What is catching my attention is the discrepancy in the usage data. My perception is that the problem lies in a deeper layer, possibly in the kernel, drivers, or firmware. This behavior is reflected in the usage graphs, which show the GPU stopping its response while Frigate continues to function in an apparently stable manner. If my N100 processor were in fact decoding 13 cameras simultaneously using only the CPU, the resource consumption would be significantly higher. Therefore, I believe the issue is not a usage failure but an anomaly in the data we are observing. This is the core of my question and our discussion. |
Beta Was this translation helpful? Give feedback.
-
|
Right, I believe that has been the understanding in this thread as well, but attempts at updating all pieces you mentioned have not resolved it |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Describe the problem you are having
I am using Frigate as an add-on in Home Assistant. Additionally, I have integrated Double Take, Deepstack, CompreFace, and Plate Recognizer. The system is set up to detect humans as well as vehicles and license plates. Currently, I have only two Reolink cameras connected.
I have been trying for a while to get the GPU to work. I have also tried VAAPI and QSV, but neither worked. It constantly shows 0% usage. The only way I can get it to work is by performing a full reboot of the computer, after which it functions for about two hours only.
I have repeated this reboot process three times, and each time the behavior is the same—it works for about two to three hours and then stays at 0% usage without recovering.
I saw that I can try using the i965 environment variable, but I got an error stating that i965 couldn't initialize. When I disable it, the system works for the two hours I mentioned earlier.
Version
0.15.0-cea210d
Frigate config file
docker-compose file or Docker CLI command
Relevant Frigate log output
Relevant go2rtc log output
FFprobe output from your camera
Install method
HassOS Addon
Object Detector
Coral
Network connection
Wired
Camera make and model
Reolink Tracemix wifi
Screenshots of the Frigate UI's System metrics pages
Any other information that may be helpful
{"return_code":0,"stderr":"","stdout":"vainfo: VA-API version: 1.22 (libva 2.10.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 24.3.3 ()
vainfo: Supported profile and entrypoints
VAProfileNone : VAEntrypointVideoProc
VAProfileNone : VAEntrypointStats
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Simple : VAEntrypointEncSlice
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointFEI
VAProfileH264Main : VAEntrypointEncSliceLP
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointFEI
VAProfileH264High : VAEntrypointEncSliceLP
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointEncPicture
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: VAEntrypointFEI
VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
VAProfileVP8Version0_3 : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointFEI
VAProfileHEVCMain : VAEntrypointEncSliceLP
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileHEVCMain10 : VAEntrypointEncSlice
VAProfileHEVCMain10 : VAEntrypointEncSliceLP
VAProfileVP9Profile0 : VAEntrypointVLD
VAProfileVP9Profile0 : VAEntrypointEncSliceLP
VAProfileVP9Profile1 : VAEntrypointVLD
VAProfileVP9Profile1 : VAEntrypointEncSliceLP
VAProfileVP9Profile2 : VAEntrypointVLD
VAProfileVP9Profile2 : VAEntrypointEncSliceLP
VAProfileVP9Profile3 : VAEntrypointVLD
VAProfileVP9Profile3 : VAEntrypointEncSliceLP
VAProfileHEVCMain12 : VAEntrypointVLD
VAProfileHEVCMain12 : VAEntrypointEncSlice
VAProfileHEVCMain422_10 : VAEntrypointVLD
VAProfileHEVCMain422_10 : VAEntrypointEncSlice
VAProfileHEVCMain422_12 : VAEntrypointVLD
VAProfileHEVCMain422_12 : VAEntrypointEncSlice
VAProfileHEVCMain444 : VAEntrypointVLD
VAProfileHEVCMain444 : VAEntrypointEncSliceLP
VAProfileHEVCMain444_10 : VAEntrypointVLD
VAProfileHEVCMain444_10 : VAEntrypointEncSliceLP
VAProfileHEVCMain444_12 : VAEntrypointVLD
VAProfileHEVCSccMain : VAEntrypointVLD
VAProfileHEVCSccMain : VAEntrypointEncSliceLP
VAProfileHEVCSccMain10 : VAEntrypointVLD
VAProfileHEVCSccMain10 : VAEntrypointEncSliceLP
VAProfileHEVCSccMain444 : VAEntrypointVLD
VAProfileHEVCSccMain444 : VAEntrypointEncSliceLP
VAProfileAV1Profile0 : VAEntrypointVLD
VAProfileHEVCSccMain444_10 : VAEntrypointVLD
VAProfileHEVCSccMain444_10 : VAEntrypointEncSliceLP"}
No response
Beta Was this translation helpful? Give feedback.
All reactions