ip_conntrack: table full, dropping packet

linux shell

From time to time on busy servers with high volume of network connections it is possible to find that your kernel conntrack table is full, webs starts going slow, same as images and all related content. If you look at the logs you may find this kernel error:

ip_conntrack: table full, dropping packet 

The linux kernel uses ip_conntrack to keep a tracking of the state of each network connection, and if your network activity is big, you may end with the table full of entries, that’s when this error happens.

The error can be found in many ways:

Searching dmesg output

dmesg | grep conntrack

Tailing the messages log file in real time

tail -f /var/log/messages | grep conntrack

Doing a grep all over

grep conntrack /var/log/messages

Anyway, once you have found the error, you need to do run some commands in order to understand why this happening and how to fix it. Let’s begin:

wc -l /proc/net/ip_conntrack

Will let you know how many connections are open right now.

sysctl -a | grep conntrack_max

Will output what’s the conntrack limit, for example:

net.ipv4.netfilter.ip_conntrack_max = 65536

How can I raise the ip_conntrack value?

nano -w /etc/sysctl.conf

On CentOS 6.x you may need to add this variable:

net.nf_conntrack_max = 165536

On CentOS 5.x and previous try with this:

net.ipv4.netfilter.ip_conntrack_max = 165536

Save the file and run this command to apply changes:

sysctl -p

If you get this kind of errors:

error: "net.nf_conntrack_max" is an unknown key error: "net.ipv4.ip_conntrack_max" is an unknown key

That means the conntrack module isn’t loaded in your kernel, try to load it manually using modprobe, for example:

modprobe nf_conntrack

Then apply the changes:

sysctl -p

Remember to keep an eye on the system logs and watch it from time to time to see if there are new ip_conntrack: table full, dropping packet errors, you may need to put the value even higher in some cases.