Jobs hang trying to use AF_XDP interfaces in 20260323.65.1 image on Ubuntu 24.04

[igsilya]

Description

After hte recent update to 20260323.65.1 images, any time our jobs are trying to send traffic through a veth interfaces opened via AF_XDP socket, the job hangs and times out. We also can't get any useful logs from these jobs to identify the problem any closer.

Runs hang and fail 100% of a time. Some examples:
https://github.com/openvswitch/ovs/actions/runs/23649403693/job/68892543795
https://github.com/openvswitch/ovs/actions/runs/23649033087/job/68888916065
https://github.com/openvswitch/ovs/actions/runs/23649123362/job/68890888183

This breaks all CI runs for Open vSwitch project.

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 22.04
• Ubuntu 24.04
• Ubuntu Slim
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• macOS 26
• macOS 26 Arm64
• Windows Server 2022
• Windows Server 2025
• Windows Server 2025 with Visual Studio 2026

Image version and build link

Version: 20260323.65.1

Is it regression?

Yes, worked correctly for years. The known good build is 20260309.50.1. Successful job: https://github.com/openvswitch/ovs/actions/runs/23551887876/job/68567896852

Expected behavior

Jobs should not hang.

Actual behavior

Job hangs on trying to run tests that send traffic via AF_XDP sockets.

Repro steps

Run this job: https://github.com/openvswitch/ovs/actions/runs/23649403693/job/68892543984

[igsilya]

Can't even cancel these jobs, they keep spinning until the timeout. But on some of them even after timeout we can see the yellow spinner spinning. Feels like a kernel-level lockup.

[igsilya]

Some potentially relevant fixes available in the newer azure kernel 6.17.0-1009.9:

  * Questing update: v6.17.10 upstream stable release (LP: #2137723) //
    CVE-2025-68232
    - veth: more robust handing of race to avoid txq getting stuck
  * Questing update: v6.17.11 upstream stable release (LP: #2138824) //
    CVE-2025-68341
    - veth: reduce XDP no_direct return section to fix race

https://changelogs.ubuntu.com/changelogs/pool/main/l/linux-azure/linux-azure_6.17.0-1009.9/changelog

[v-GeorgyPuzakov]

Hi @igsilya,
Could you clarify or share the specific part of your setup that relates to our repository? From what I can see, you’re using a custom workflow.

As you mentioned, the error may be caused by a kernel update or another application outside the scope of what we support.

Once a version containing a fix is officially released, we will most likely include it in a subsequent image release.