[Build 2024111416-4.2] openQA test fails in upload_packages

[marmarek]

Observation

openQA test in scenario qubesos-4.2-securedrop-x86_64-securedrop_install@64bit fails in
upload_packages

curl: (28) Operation timed out after 10003 miliseconds with 0 bytes received

Test suite description

Install SecureDrop

Reproducible

Fails since (at least) Build 2024111416-4.2 (current job)

Expected result

Last good: 2024111309-4.2 (or more recent)

Further details

Always latest result in this scenario: latest

See more info at QubesOS/openqa-tests-qubesos#25 (comment)

[deeplow]

Adding some context captured on an old commit that won't make it into main:

In a specific set of circumstances dropped packets were causing the log uploads to take a long time and eventually fail. Even though the client would send retries, they would not reach the host.

A failok flag was added to the logs-uploading method, as a a temporary measure to let the tests proceed even though the log uploading failed.

[marmarek]

@deeplow here is the issue I mentioned - if you hit such issue, add the issue URL as the job comment

[marmarek]

Recent related failures:

?column?	t_finished	state	result	test	host
https://openqa.qubes-os.org/tests/159898	2025-11-19 10:08:45	done	failed	system_tests_basic_vm_qrexec_gui_xfs	ramen
https://openqa.qubes-os.org/tests/159885	2025-11-19 07:25:53	done	failed	system_tests_basic_vm_qrexec_gui_xfs	ramen
https://openqa.qubes-os.org/tests/159851	2025-11-19 06:24:35	done	failed	system_tests_guivm_gui_interactive	herring
https://openqa.qubes-os.org/tests/159763	2025-11-18 13:25:23	done	failed	system_tests_basic_vm_qrexec_gui_ext4	herring
https://openqa.qubes-os.org/tests/159739	2025-11-18 12:51:48	done	failed	system_tests_basic_vm_qrexec_gui	herring
https://openqa.qubes-os.org/tests/159757	2025-11-18 12:31:32	done	failed	system_tests_guivm_vnc_gui_interactive	onigiri
https://openqa.qubes-os.org/tests/159670	2025-11-18 07:14:30	done	failed	system_tests_qrexec	herring
https://openqa.qubes-os.org/tests/159681	2025-11-18 05:59:27	done	failed	system_tests_update	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/159532	2025-11-16 04:43:17	done	failed	system_tests_network_ipv6	herring
https://openqa.qubes-os.org/tests/159561	2025-11-16 04:12:40	done	failed	system_tests_basic_vm_qrexec_gui_btrfs	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/159562	2025-11-16 03:06:23	done	failed	system_tests_basic_vm_qrexec_gui_ext4	ramen
https://openqa.qubes-os.org/tests/159469	2025-11-14 17:16:55	done	failed	system_tests_basic_vm_qrexec_gui_xfs	onigiri
https://openqa.qubes-os.org/tests/159468	2025-11-14 17:14:38	done	failed	system_tests_basic_vm_qrexec_gui_ext4	ramen
https://openqa.qubes-os.org/tests/159467	2025-11-14 17:11:09	done	failed	system_tests_basic_vm_qrexec_gui_btrfs	ramen
https://openqa.qubes-os.org/tests/159350	2025-11-13 05:48:57	done	failed	system_tests_basic_vm_qrexec_gui_ext4	herring
https://openqa.qubes-os.org/tests/159160	2025-11-11 16:02:53	done	failed	system_tests_basic_vm_qrexec_gui_ext4	ramen
https://openqa.qubes-os.org/tests/159153	2025-11-11 13:46:03	done	failed	system_tests_basic_vm_qrexec_gui_ext4	herring
https://openqa.qubes-os.org/tests/159140	2025-11-11 12:15:19	done	failed	system_tests_basic_vm_qrexec_gui_btrfs	herring
https://openqa.qubes-os.org/tests/159141	2025-11-11 12:05:07	done	failed	system_tests_basic_vm_qrexec_gui_ext4	ramen
https://openqa.qubes-os.org/tests/159152	2025-11-11 11:46:07	done	failed	system_tests_guivm_vnc_gui_interactive	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/159130	2025-11-11 11:45:36	done	failed	system_tests_backupdispvm	onigiri
https://openqa.qubes-os.org/tests/159021	2025-11-11 06:08:12	done	failed	system_tests_basic_vm_qrexec_gui_ext4	herring
https://openqa.qubes-os.org/tests/159060	2025-11-11 06:07:31	done	failed	system_tests_update	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/158845	2025-11-10 14:32:03	done	failed	system_tests_network	herring
https://openqa.qubes-os.org/tests/158811	2025-11-10 11:57:38	done	failed	system_tests_update	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/158796	2025-11-09 13:25:30	done	failed	system_tests_splitgpg	ramen
https://openqa.qubes-os.org/tests/158674	2025-11-05 16:59:43	done	failed	system_tests_dispvm	herring
https://openqa.qubes-os.org/tests/158605	2025-11-05 05:35:32	done	failed	system_tests_dispvm	herring
https://openqa.qubes-os.org/tests/158632	2025-11-05 04:42:32	done	failed	system_tests_basic_vm_qrexec_gui_xfs	ramen
https://openqa.qubes-os.org/tests/158494	2025-11-04 05:29:51	done	failed	system_tests_whonix	ramen
https://openqa.qubes-os.org/tests/158489	2025-11-03 22:28:56	done	failed	system_tests_dispvm	onigiri
https://openqa.qubes-os.org/tests/158490	2025-11-03 22:15:46	done	failed	system_tests_basic_vm_qrexec_gui_zfs	herring
https://openqa.qubes-os.org/tests/158456	2025-11-03 21:24:03	done	failed	system_tests_dispvm	herring
https://openqa.qubes-os.org/tests/158483	2025-11-03 20:28:16	done	failed	system_tests_basic_vm_qrexec_gui_xfs	onigiri
https://openqa.qubes-os.org/tests/158481	2025-11-03 20:23:46	done	failed	system_tests_basic_vm_qrexec_gui_btrfs	ramen
https://openqa.qubes-os.org/tests/158443	2025-11-03 20:11:58	done	failed	system_tests_network	ramen
https://openqa.qubes-os.org/tests/158004	2025-10-30 07:04:33	done	passed	system_tests_gui_interactive_preloaded	onigiri
https://openqa.qubes-os.org/tests/157976	2025-10-30 06:59:25	done	passed	system_tests_gui_interactive	ramen
https://openqa.qubes-os.org/tests/157787	2025-10-28 12:09:56	done	failed	system_tests_extra	ramen
https://openqa.qubes-os.org/tests/157762	2025-10-28 07:08:39	done	failed	system_tests_extra	ramen
https://openqa.qubes-os.org/tests/157755	2025-10-28 06:20:42	done	failed	system_tests_basic_vm_qrexec_gui	ramen
https://openqa.qubes-os.org/tests/157757	2025-10-28 06:04:18	done	failed	system_tests_basic_vm_qrexec_gui_ext4	ramen
https://openqa.qubes-os.org/tests/157756	2025-10-28 05:58:34	done	failed	system_tests_basic_vm_qrexec_gui_btrfs	ramen
https://openqa.qubes-os.org/tests/157737	2025-10-28 05:26:09	done	failed	system_tests_whonix	onigiri
https://openqa.qubes-os.org/tests/157685	2025-10-27 08:05:05	done	failed	system_tests_basic_vm_qrexec_gui_zfs	ramen
https://openqa.qubes-os.org/tests/157674	2025-10-27 06:01:33	done	failed	system_tests_guivm_gui_interactive	ramen
https://openqa.qubes-os.org/tests/157379	2025-10-24 09:57:06	done	failed	system_tests_basic_vm_qrexec_gui	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/157308	2025-10-23 15:16:53	done	failed	system_tests_network_updates	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/157195	2025-10-22 20:53:59	done	failed	system_tests_guivm_gui_interactive	ramen
https://openqa.qubes-os.org/tests/156736	2025-10-21 08:37:34	done	failed	system_tests_network_updates	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/156730	2025-10-21 08:25:51	done	failed	system_tests_extra	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/156734	2025-10-21 07:07:23	done	failed	system_tests_network	onigiri
https://openqa.qubes-os.org/tests/156725	2025-10-21 06:36:55	done	failed	system_tests_basic_vm_qrexec_gui_ext4	ramen
https://openqa.qubes-os.org/tests/156427	2025-10-16 22:17:16	done	failed	system_tests_network	ramen
https://openqa.qubes-os.org/tests/156227	2025-10-14 19:27:09	done	failed	system_tests_basic_vm_qrexec_gui_zfs	ramen
https://openqa.qubes-os.org/tests/156228	2025-10-14 17:32:03	done	failed	system_tests_guivm_vnc_gui_interactive	onigiri
https://openqa.qubes-os.org/tests/156170	2025-10-14 07:40:39	done	failed	system_tests_network_updates	ramen
https://openqa.qubes-os.org/tests/156157	2025-10-14 06:31:44	done	failed	system_tests_basic_vm_qrexec_gui	ramen
https://openqa.qubes-os.org/tests/155873	2025-10-11 10:19:25	done	failed	system_tests_network	ramen
https://openqa.qubes-os.org/tests/155571	2025-10-07 17:34:21	done	failed	system_tests_guivm_vnc_gui_interactive	ramen
https://openqa.qubes-os.org/tests/155519	2025-10-07 09:06:53	done	failed	system_tests_guivm_vnc_gui_interactive	onigiri
https://openqa.qubes-os.org/tests/155192	2025-10-05 01:24:30	done	failed	system_tests_splitgpg	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/155193	2025-10-05 01:14:46	done	failed	system_tests_extra	onigiri
https://openqa.qubes-os.org/tests/154818	2025-10-02 20:41:32	done	failed	system_tests_basic_vm_qrexec_gui_ext4	onigiri
https://openqa.qubes-os.org/tests/154792	2025-10-02 20:17:36	done	failed	system_tests_dispvm	onigiri
https://openqa.qubes-os.org/tests/154503	2025-09-30 06:48:46	done	failed	system_tests_basic_vm_qrexec_gui_ext4	onigiri
https://openqa.qubes-os.org/tests/154175	2025-09-23 06:31:14	done	failed	system_tests_basic_vm_qrexec_gui_btrfs	ramen
https://openqa.qubes-os.org/tests/154176	2025-09-23 06:10:21	done	failed	system_tests_basic_vm_qrexec_gui_ext4	ramen
https://openqa.qubes-os.org/tests/153865	2025-09-19 01:59:23	done	failed	system_tests_basic_vm_qrexec_gui_xfs	onigiri
https://openqa.qubes-os.org/tests/153729	2025-09-18 08:00:15	done	failed	system_tests_basic_vm_qrexec_gui_btrfs	onigiri
https://openqa.qubes-os.org/tests/153696	2025-09-18 07:34:39	done	failed	system_tests_extra	onigiri
https://openqa.qubes-os.org/tests/153589	2025-09-17 21:53:16	done	failed	system_tests_basic_vm_qrexec_gui_ext4	onigiri
https://openqa.qubes-os.org/tests/153581	2025-09-17 19:10:07	done	failed	system_tests_basic_vm_qrexec_gui_ext4	ramen
https://openqa.qubes-os.org/tests/153555	2025-09-17 18:31:17	done	failed	system_tests_dispvm	ramen
https://openqa.qubes-os.org/tests/153547	2025-09-17 18:18:36	done	failed	system_tests_extra	ramen
https://openqa.qubes-os.org/tests/153362	2025-09-16 07:51:41	done	failed	system_tests_network_updates	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/153364	2025-09-16 07:17:08	done	failed	system_tests_qrexec	ramen
https://openqa.qubes-os.org/tests/153351	2025-09-16 06:39:40	done	failed	system_tests_basic_vm_qrexec_gui_ext4	onigiri
https://openqa.qubes-os.org/tests/153350	2025-09-16 06:23:58	done	failed	system_tests_basic_vm_qrexec_gui_btrfs	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/153340	2025-09-16 05:35:07	done	failed	system_tests_guivm_vnc_gui_interactive	ramen
https://openqa.qubes-os.org/tests/153286	2025-09-16 03:54:16	done	failed	system_tests_network	ramen
https://openqa.qubes-os.org/tests/153266	2025-09-15 19:16:16	done	failed	system_tests_basic_vm_qrexec_gui_xfs	onigiri
https://openqa.qubes-os.org/tests/153236	2025-09-15 18:27:09	done	failed	system_tests_qrexec	ramen
https://openqa.qubes-os.org/tests/152831	2025-09-12 08:28:13	done	failed	system_tests_network_updates	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/152833	2025-09-12 07:45:15	done	failed	system_tests_dispvm	ramen
https://openqa.qubes-os.org/tests/152828	2025-09-12 07:10:09	done	failed	system_tests_usbproxy	ramen
https://openqa.qubes-os.org/tests/152238	2025-09-07 02:52:05	done	failed	system_tests_basic_vm_qrexec_gui_ext4	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/152212	2025-09-07 01:35:18	done	failed	system_tests_backupdispvm	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/152218	2025-09-07 01:28:12	done	failed	system_tests_basic_vm_qrexec_gui_zfs	onigiri
https://openqa.qubes-os.org/tests/152199	2025-09-07 00:15:42	done	failed	system_tests_basic_vm_qrexec_gui	salmon.qubes-os.org
https://openqa.qubes-os.org/tests/152150	2025-09-06 04:51:24	done	failed	system_tests_basic_vm_qrexec_gui_ext4	onigiri
https://openqa.qubes-os.org/tests/152130	2025-09-06 04:35:24	done	failed	system_tests_basic_vm_qrexec_gui_zfs	ramen
https://openqa.qubes-os.org/tests/152111	2025-09-06 03:38:04	done	failed	system_tests_basic_vm_qrexec_gui	ramen

[marmarek]

In a very limited testing, it appears as ethtool -K ens6 sg off in sys-net helps.