Routing & Failover
This section covers how ololo-relay’s built-in L7 DPI classifier selects an egress path for each session, how per-client and per-subnet split routing (client_defaults) works, how automatic gateway failover behaves under packet loss, and how a gateway node captures LAN or VPN interfaces without breaking port forwards.
Evaluation Order (Passive First-Packet L7 DPI)
For every new TCP connection or UDP flow, the ingress node (edge) performs lightweight Deep Packet Inspection on the initial packet without TLS termination — extracting the target hostname from TLS ClientHello SNI on :443, HTTP Host on :80, or QUIC Initial on udp/443, alongside L7 P2P protocol signatures — and evaluates rules top-down:
- Drop rules (
drop) — destination domain or IP matches a route targeting the built-indroppseudo-peer → immediate connection reset. - Destination routes (
routes) — longest domain suffix match, falling back to longest prefix match (LPM) on CIDR → ordered failover list of exit peers. - Local bypass (
localhost) — match against alocalhostroute (RFC 1918, CGNAT100.64.0.0/10,.local,.lan) → direct local egress bypassing the mesh. - Per-client / subnet split routing (
client_defaults) — LPM lookup by client source IP (SrcIP) and optionaledge_id(/32overrides/24, explicitedge_idbeats wildcard):- with custom route table (
custom_routes: true), steps 1–3 run exclusively against this client rule’s ownrouteslist (cluster-wide routes are isolated), and unmatched traffic uses this rule’s default gateway list; - with shared cluster routes (
custom_routes: false), cluster-wide routes in steps 1–3 apply first, while unmatched traffic uses this client’s personal default gateway list.
- with custom route table (
- Cluster default (
default) — governs all remaining traffic (relay,direct, ordrop).
Per-Client & Per-Subnet Split Routing (client_defaults)
Assigns both a dedicated default gateway failover chain and an optional isolated domain/CIDR routing table to an individual device (10.8.0.15/32) or entire VLAN/subnet (172.20.10.0/24).
Common scenarios:
- pin CI build agents or a QA subnet to a specific regional exit without changing the office-wide
default; - clone the standard cluster route table for a department and customize two or three rules in isolation;
- let a media box or storage server exit directly (
mode: direct) or restrict an IoT subnet strictly to allowlisted routes (mode: drop+custom_routes: true).
In the desktop GUI, Copy from standard populates a custom route table in one click without retyping rules or local-bypass from scratch.
Example in policy.yaml / initial-policy.yaml:
client_defaults:
- id: "1"
name: "build-agents"
sources: ["172.20.10.0/24", "10.8.0.15/32"]
edge_id: "gw-office-1" # empty = match on any edge
mode: relay
peers: ["exit-de-1", "exit-nl-1"]
custom_routes: true
routes:
- id: "local-bypass"
peers: ["localhost"]
domains: [".local", ".lan"]
cidrs: ["127.0.0.0/8", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "100.64.0.0/10"]
- id: "1"
peers: ["exit-fi-1"]
domains: [".registry.example.com"]
Gateway Failover on > 20% Loss & Lazy Recovery
Wherever an ordered peer list is specified (default, client_defaults, or routes), peer order defines failover priority (#1 primary, #2 secondary, etc.):
- Sliding-window loss detection.
edgetracks a 20-sample window combiningTrafficRelay.Pingprobes (every 5s) and live dial outcomes. Both hard timeouts and single-packet drops causing TCP retransmission spikes (RTT > max(400ms, 3 × minRTT)) count as loss. - Immediate failover above 20% loss. Once loss exceeds
20%(or session dial fails), traffic switches immediately to the next healthy peer (#2,#3, …). - Lazy primary probing & anti-flap hold-down. While running on a backup gateway,
edgeprobes primary#1every30s. Once#1recovers (loss <= 20%), traffic returns to#1— enforced by afailover_holddowntimer (3mdefault) to prevent rapid flapping on unstable links (if the backup itself fails during hold-down, failover to the next live peer is immediate).
Tunables in /etc/ololo-relay/node.yaml:
edge:
loss_threshold: 0.20 # switch away when loss > 20%
lazy_ping_interval: 30s # primary probe interval while on backup
failover_holddown: 3m # minimum hold time before returning to #1
max_conn_age: 6m # mean HTTP/2 pool connection lifetime before rotation
max_conn_age_jitter: 60s # ±60s jitter (5..7m window, zero stream drops)
Exit Node BitTorrent Guard (block_bittorrent)
Cluster policy enables BitTorrent / P2P filtering by default (block_bittorrent: true) so a background torrent client on a user device cannot trigger abuse suspensions on cloud exit VMs:
- On
edge, torrent sessions are dropped pre-dial by port (6881–6889,6969,51413), protocol signatures (BitTorrent handshake, DHT, uTP, trackers), and MSE/PE swarm heuristics — without ever affecting legitimate TLS, QUIC, WebSocket, SSH, STUN/WebRTC, or WireGuard traffic. - On
exit, lightweight stateless first-packet inspection provides defense in depth.
Toggle live in the desktop GUI or declaratively via block_bittorrent in policy YAML (per-node overrides are also available in node.yaml via edge.block_bittorrent / exit.block_bittorrent).
Transit Interface Capture (capture_iifs)
Redirects transit traffic arriving on specified interfaces (such as an office LAN NIC or an OpenVPN / Pritunl / WireGuard tun0 interface) into ololo0 via policy routing:
edge:
tun:
name: ololo0
capture_iifs:
- tun0
RFC 1918 (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) and CGNAT (100.64.0.0/10) destinations automatically stay in the main routing table.
Preserving DNAT Port Forwards (capture_wan_iface)
If your gateway hosts inbound port forwards (WAN_IP:port → LAN server) alongside capture_iifs, specify the uplink interface so conntrack marks inbound WAN flows and routes return packets back out the WAN instead of into the mesh:
edge:
capture_iifs:
- eth1 # LAN interface captured into relay
capture_wan_iface: eth0 # WAN uplink interface for symmetric DNAT replies
Destination Exclusions (capture_exclude_dst)
Keep specific public IPs or subnets (such as the gateway’s own public WAN IP, VRRP/keepalived VIPs, or peer gateways) in the main routing table:
edge:
capture_exclude_dst:
- 203.0.113.10/32
Gateway Kill-Switch (fail_closed)
By default (fail_closed: false), stopping ololo-relay removes capture rules and falls back to normal gateway routing. Setting fail_closed: true installs a blackhole fallback in routing table 100 that persists across service stops or crashes so non-local client traffic never leaks directly from the gateway’s WAN IP:
edge:
fail_closed: true
Caution: When
fail_closed: trueis active,systemctl stop ololo-relayintentionally leaves the blackhole rule in place untilololo-relaystarts again or you flush it manually:ip route flush table 100 ip rule del iif <iif> lookup 100 priority 99 ip rule del to 10.0.0.0/8 lookup main priority 98Never enable
fail_closedon an interface carrying your own remote SSH management session.