Skip to content

[BUG] fw_chain_egress missing ct state established,related accept causes inbound TCP to fail with default egress deny on routed networks #12962

@khumps

Description

@khumps

problem

Problem

When a routed network is configured with default egress policy Deny, inbound TCP connections to guest VMs fail even when the correct ingress firewall rules are present. The Virtual Router's nftables fw_chain_egress chain is missing a ct state established,related accept rule, causing return packets (e.g. TCP SYN-ACK) from the VM to be dropped by the final drop rule in fw_chain_egress.

The FORWARD chain correctly jumps traffic sourced from the guest subnet (ip saddr jump fw_chain_egress). However, fw_chain_egress only contains explicit port/protocol allow rules followed by a default drop — there is no connection tracking rule to permit return traffic for inbound connections that were allowed by fw_chain_ingress.

Note: This issue only occurs when the network's default egress policy is Deny. Networks with default egress policy Allow are not affected — inbound TCP works correctly in that configuration.

versions

CloudStack Management Server: 4.22.0.0-shapeblue0 (package: cloudstack-management)
Virtual Router: 4.22.0 (Debian GNU/Linux 12 bookworm, kernel 6.1.0-40-arm64 aarch64)
Hypervisor: KVM
Network type: Routed isolated network, dynamic BGP routing mode
Default egress policy: Deny
Steps to reproduce

The steps to reproduce the bug

Observed fw_chain_egress on VR (clean state, post-reboot):

chain fw_chain_egress {
    ip saddr 0.0.0.0/0 ip daddr 0.0.0.0/0 icmp type { echo-reply, destination-unreachable, source-quench, redirect, echo-request, router-advertisement, router-solicitation, time-exceeded, parameter-problem, timestamp-request, timestamp-reply, info-request, info-reply, address-mask-request, address-mask-reply } accept
    ip saddr 0.0.0.0/0 ip daddr 0.0.0.0/0 tcp dport 22 accept
    counter packets 115 bytes 6900 drop
}

Expected fw_chain_egress:

chain fw_chain_egress {
    ct state established,related accept   ← missing
    ip saddr 0.0.0.0/0 ip daddr 0.0.0.0/0 icmp type { ... } accept
    ip saddr 0.0.0.0/0 ip daddr 0.0.0.0/0 tcp dport 22 accept
    counter drop
}

This is the IPv4 analog of the issue fixed for IPv6 in PR #10970.

Create a zone with dynamic routing enabled and an IP subnet pool configured
Create a network offering with routing mode = Dynamic and default egress policy = Deny
Deploy an isolated routed network using that offering
In the network's IPv4 Routing Firewall, add:
Ingress rule: TCP port 22 from 0.0.0.0/0
Egress rule: TCP port 22 to 0.0.0.0/0
Deploy a VM in the network
Attempt to SSH to the VM from an external host
Connection hangs
Diagnosis: tcpdump on the VR confirms the SYN arrives on eth2, is forwarded out eth0 to the VM, the VM responds with SYN-ACK on eth0, but the SYN-ACK is dropped by the counter drop rule in fw_chain_egress because no ct state established,related accept rule exists.

Workaround

Manually insert the missing rule on the VR (lost on VR restart):

nft insert rule ip ip4_firewall fw_chain_egress ct state established,related accept
What to do about it?

Add ct state established,related accept as the first rule in fw_chain_egress in the VR nftables generation code. This should always be present regardless of user-defined egress rules, as it is required to allow return traffic for inbound connections permitted by fw_chain_ingress. This mirrors the fix applied for IPv6 in PR #10970.

What to do about it?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions