What causes “SYN to LISTEN sockets dropped”?

Posted on

What causes “SYN to LISTEN sockets dropped”? – A server stack is the collection of software that forms the operational infrastructure on a given machine. In a computing context, a stack is an ordered pile. A server stack is one type of solution stack — an ordered selection of software that makes it possible to complete a particular task. Like in this post about What causes “SYN to LISTEN sockets dropped”? was one problem in server stack that need for a solution. Below are some tips in manage your linux server when you find problem about linux, ubuntu, networking, netstat, .

A quite busy proxy server has lots of “SYNs to LISTEN sockets dropped”.

I learned one cause could be a too small backlog size. But in that case the “times the listen queue of a socket overflowed” value should be equal (which it is not).

So what could be a cause for this behaviour? Maybe a broken nic?

We have 5 proxies, in 2 of which the two numbers are not equal, so this problem seems to be happening there.

Here the output from netstat:

$ netstat -s | grep -i list
238627 times the listen queue of a socket overflowed
8610307 SYNs to LISTEN sockets dropped

the servers have ipv4 and ipv6 traffic, maybe that helps?

These counters ultimately come from the kernel and map to the LINUX_MIB_LISTENOVERFLOWS and LINUX_MIB_LISTENDROPS counters. You can see from the source of net/ipv4/tcp_ipv4.c(tcp_v4_syn_recv_sock) around line #1392 that when LINUX_MIB_LISTENOVERFLOWS is incremented, LINUX_MIB_LISTENDROPS will also be incremented but there are exit conditions where only the latter can be incremented so it’s not a bug that they don’t match.

In the same file you can see there’s this code:

1291 int tcp_v4_conn_request(struct sock *sk, struct sk_buff *skb)
1292 {
1293         /* Never answer to SYNs send to broadcast or multicast */
1294         if (skb_rtable(skb)->rt_flags & (RTCF_BROADCAST | RTCF_MULTICAST))
1295                 goto drop;
1297         return tcp_conn_request(&tcp_request_sock_ops,
1298                                 &tcp_request_sock_ipv4_ops, sk, skb);
1300 drop:
1301         NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENDROPS);
1302         return 0;
1303 }

So you can see at least one cause is a SYN to a broadcast or multicast address.

Usually wmem and rmem defaults are 212992 bytes. Apparently not enough on busy server. Raised to 8MB and the problem disappeared.

sysctl -w net.core.wmem_default=8388608
sysctl -w net.core.rmem_default=8388608

Leave a Reply

Your email address will not be published. Required fields are marked *