Skip to content

Routing & Failover

This section covers how ololo-relay’s built-in L7 DPI classifier selects an egress path for each session, how per-client and per-subnet split routing (client_defaults) works, how automatic gateway failover behaves under packet loss, and how a gateway node captures LAN or VPN interfaces without breaking port forwards.

Evaluation Order (Passive First-Packet L7 DPI)

For every new TCP connection or UDP flow, the ingress node (edge) performs lightweight Deep Packet Inspection on the initial packet without TLS termination — extracting the target hostname from TLS ClientHello SNI on :443, HTTP Host on :80, or QUIC Initial on udp/443, alongside L7 P2P protocol signatures — and evaluates rules top-down:

  1. Drop rules (drop) — destination domain or IP matches a route targeting the built-in drop pseudo-peer → immediate connection reset.
  2. Destination routes (routes) — longest domain suffix match, falling back to longest prefix match (LPM) on CIDR → ordered failover list of exit peers.
  3. Local bypass (localhost) — match against a localhost route (RFC 1918, CGNAT 100.64.0.0/10, .local, .lan) → direct local egress bypassing the mesh.
  4. Per-client / subnet split routing (client_defaults) — LPM lookup by client source IP (SrcIP) and optional edge_id (/32 overrides /24, explicit edge_id beats wildcard):
    • with custom route table (custom_routes: true), steps 1–3 run exclusively against this client rule’s own routes list (cluster-wide routes are isolated), and unmatched traffic uses this rule’s default gateway list;
    • with shared cluster routes (custom_routes: false), cluster-wide routes in steps 1–3 apply first, while unmatched traffic uses this client’s personal default gateway list.
  5. Cluster default (default) — governs all remaining traffic (relay, direct, or drop).

Per-Client & Per-Subnet Split Routing (client_defaults)

Assigns both a dedicated default gateway failover chain and an optional isolated domain/CIDR routing table to an individual device (10.8.0.15/32) or entire VLAN/subnet (172.20.10.0/24).

Common scenarios:

  • pin CI build agents or a QA subnet to a specific regional exit without changing the office-wide default;
  • clone the standard cluster route table for a department and customize two or three rules in isolation;
  • let a media box or storage server exit directly (mode: direct) or restrict an IoT subnet strictly to allowlisted routes (mode: drop + custom_routes: true).

In the desktop GUI, Copy from standard populates a custom route table in one click without retyping rules or local-bypass from scratch.

Example in policy.yaml / initial-policy.yaml:

client_defaults:
  - id: "1"
    name: "build-agents"
    sources: ["172.20.10.0/24", "10.8.0.15/32"]
    edge_id: "gw-office-1"       # empty = match on any edge
    mode: relay
    peers: ["exit-de-1", "exit-nl-1"]
    custom_routes: true
    routes:
      - id: "local-bypass"
        peers: ["localhost"]
        domains: [".local", ".lan"]
        cidrs: ["127.0.0.0/8", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "100.64.0.0/10"]
      - id: "1"
        peers: ["exit-fi-1"]
        domains: [".registry.example.com"]

Gateway Failover on > 20% Loss & Lazy Recovery

Wherever an ordered peer list is specified (default, client_defaults, or routes), peer order defines failover priority (#1 primary, #2 secondary, etc.):

  1. Sliding-window loss detection. edge tracks a 20-sample window combining TrafficRelay.Ping probes (every 5s) and live dial outcomes. Both hard timeouts and single-packet drops causing TCP retransmission spikes (RTT > max(400ms, 3 × minRTT)) count as loss.
  2. Immediate failover above 20% loss. Once loss exceeds 20% (or session dial fails), traffic switches immediately to the next healthy peer (#2, #3, …).
  3. Lazy primary probing & anti-flap hold-down. While running on a backup gateway, edge probes primary #1 every 30s. Once #1 recovers (loss <= 20%), traffic returns to #1 — enforced by a failover_holddown timer (3m default) to prevent rapid flapping on unstable links (if the backup itself fails during hold-down, failover to the next live peer is immediate).

Tunables in /etc/ololo-relay/node.yaml:

edge:
  loss_threshold: 0.20       # switch away when loss > 20%
  lazy_ping_interval: 30s    # primary probe interval while on backup
  failover_holddown: 3m      # minimum hold time before returning to #1
  max_conn_age: 6m           # mean HTTP/2 pool connection lifetime before rotation
  max_conn_age_jitter: 60s   # ±60s jitter (5..7m window, zero stream drops)

Exit Node BitTorrent Guard (block_bittorrent)

Cluster policy enables BitTorrent / P2P filtering by default (block_bittorrent: true) so a background torrent client on a user device cannot trigger abuse suspensions on cloud exit VMs:

  • On edge, torrent sessions are dropped pre-dial by port (6881–6889, 6969, 51413), protocol signatures (BitTorrent handshake, DHT, uTP, trackers), and MSE/PE swarm heuristics — without ever affecting legitimate TLS, QUIC, WebSocket, SSH, STUN/WebRTC, or WireGuard traffic.
  • On exit, lightweight stateless first-packet inspection provides defense in depth.

Toggle live in the desktop GUI or declaratively via block_bittorrent in policy YAML (per-node overrides are also available in node.yaml via edge.block_bittorrent / exit.block_bittorrent).

Transit Interface Capture (capture_iifs)

Redirects transit traffic arriving on specified interfaces (such as an office LAN NIC or an OpenVPN / Pritunl / WireGuard tun0 interface) into ololo0 via policy routing:

edge:
  tun:
    name: ololo0
  capture_iifs:
    - tun0

RFC 1918 (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) and CGNAT (100.64.0.0/10) destinations automatically stay in the main routing table.

Preserving DNAT Port Forwards (capture_wan_iface)

If your gateway hosts inbound port forwards (WAN_IP:port → LAN server) alongside capture_iifs, specify the uplink interface so conntrack marks inbound WAN flows and routes return packets back out the WAN instead of into the mesh:

edge:
  capture_iifs:
    - eth1                     # LAN interface captured into relay
  capture_wan_iface: eth0      # WAN uplink interface for symmetric DNAT replies

Destination Exclusions (capture_exclude_dst)

Keep specific public IPs or subnets (such as the gateway’s own public WAN IP, VRRP/keepalived VIPs, or peer gateways) in the main routing table:

edge:
  capture_exclude_dst:
    - 203.0.113.10/32

Gateway Kill-Switch (fail_closed)

By default (fail_closed: false), stopping ololo-relay removes capture rules and falls back to normal gateway routing. Setting fail_closed: true installs a blackhole fallback in routing table 100 that persists across service stops or crashes so non-local client traffic never leaks directly from the gateway’s WAN IP:

edge:
  fail_closed: true

Caution: When fail_closed: true is active, systemctl stop ololo-relay intentionally leaves the blackhole rule in place until ololo-relay starts again or you flush it manually:

ip route flush table 100
ip rule del iif <iif> lookup 100 priority 99
ip rule del to 10.0.0.0/8 lookup main priority 98

Never enable fail_closed on an interface carrying your own remote SSH management session.