IPVS (keepalived) doesn’t balance UDP connections

Posted on

A server stack is the collection of software that forms the operational infrastructure on a given machine. In a computing context, a stack is an ordered pile. A server stack is one type of solution stack — an ordered selection of software that makes it possible to complete a particular task. Like in this post about IPVS (keepalived) doesn’t balance UDP connections was one problem in server stack that need for a solution. Below are some tips in manage your linux server when you find problem about linux, rsyslog, udp, keepalived, lvs.

I have two load balancer with Debian 8 and three Graylog server with Debian 9.

Every server in my network sends logs via rsyslog to a virtual server configured on the LB. The connection is UDP.

The problem is that the packets are not balanced. (all connections goes on the first real server on the list)

In case of failover the packets are correctly sent to the others real servers.

The only way I found to re-balance the connection is to remove all real server from the LB and the restart keepalived service.

I already tied to set:

ipvsadm --set 0 0 1
Timeout (tcp tcpfin udp): 900 120 1

I already set these two variables:

echo 1 > /proc/sys/net/ipv4/vs/expire_nodest_conn
echo 1 > /proc/sys/net/ipv4/vs/expire_quiescent_template

IPVS is configure as follow:

vrrp_instance logserver {
    state MASTER
    interface eth0
    virtual_router_id 195
    priority 200
    advert_int 1
    authentication {
        auth_type keepalived
        auth_pass xxxxxx
    }
    virtual_ipaddress {
        10.20.20.195/22
    }
}


virtual_server 10.20.20.195 0 {
    delay_loop 60
    protocol UDP
    lb_algo wrr
    lb_kind DR
    persistence_timeout 30

    real_server 10.20.20.196 0 {
        weight 100
        MISC_CHECK {
                connect_timeout 3
                misc_path "/etc/keepalived/checkgraylog 10.20.20.196"
        }
    }

    real_server 10.20.20.197 0 {
       weight 100
        MISC_CHECK {
                connect_timeout 3
                misc_path "/etc/keepalived/checkgraylog 10.20.20.197"
        }
    }

    real_server 10.20.20.198 0 {
       weight 100
        MISC_CHECK {
                connect_timeout 3
                misc_path "/etc/keepalived/checkgraylog 10.20.20.198"
        }    } }

Is there a way to effective balance UDP connection with Direct Routing?
Thank you

virtual_server 10.20.20.195 12333 {
    delay_loop 60
    protocol UDP
    lb_algo wrr
    lb_kind DR
    ops # <<< - Try this. Works for me (Ubuntu 18.04, Keepalived v1.3.9, ipvsadm v1.28) 

    real_server 10.20.20.196 12333 {

Option ops for me works only if either:

  • Virtual server port is explicitly defined.
  • fwmark is used together with in virtual_server definition.

Does not work for virtual_server_IP 0 form – in that case ipvsadm -Ln shows that persistent option is used as well.

Leave a Reply

Your email address will not be published. Required fields are marked *