Module: sip-router Branch: master Commit: 46e13e4093f1b8f61dd34adb05e83adebff4ead1 URL: http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=46e13e4… Author: oej &lt;oej(a)edvina.net&gt; Committer: oej &lt;oej(a)edvina.net&gt; Date: Fri Oct 9 22:38:48 2009 +0200 Typos and formatting --- doc/tcp_tunning.txt | 52 +++++++++++++++++++++++++++++--------------------- 1 files changed, 30 insertions(+), 22 deletions(-) diff --git a/doc/tcp_tunning.txt b/doc/tcp_tunning.txt index b383a42..db85289 100644 --- a/doc/tcp_tunning.txt +++ b/doc/tcp_tunning.txt @@ -10,15 +10,16 @@ TCP Tunning/monitoring for lots of open connections ---------------- This document describes very briefly various settings that should improve - ser+tcp performance for sites handling a lot of tcp traffic (> 1000 open - connections or very high connection/disconnection rates). -For now it deals only with linux specific optimizations. +ser+tcp performance for sites handling a lot of tcp traffic (> 1000 open +connections or very high connection/disconnection rates). +For now it deals only with Linux specific optimizations. -1. Usefull kernel settings ---------------------------- -1.1 connection rate/pending connections: by default the connection rate is +1. Useful kernel settings +------------------------- + +1.1 Connection rate/pending connections: by default the connection rate is too small net.core.somaxconn - limit of the listen() backlog, default 128 @@ -27,7 +28,9 @@ net.ipv4.tcp_timestamps - default on., should be on (along with tcp_tw_recycle and timestamp supporting peers allows for fast connections rates) -1.2 connection in close_wait: connection should stay as little as possible +1.2 connection in close_wait + + Connection should stay as little as possible in close_wait to quickly free the fd/resources for new connections attempts WARNING: this could break normal tcp use, use it only if you know what you are doing @@ -44,10 +47,11 @@ net.ipv4.tcp_tw_reuse - allows reusing of time-wait sockets (default off) net.ipv4.tcp_syncookies - default off, in this case it's probably better to keep it off -1.3 port range +1.3 Port range net.ipv4.ip_local_port_range - should be increased (e.g. 4096-65534) -1.4 open fds +1.4 Open file descriptors + fs.file-max - maximum number of fds that will be allocated (you probably need to increase it, default depends on installed memory) @@ -59,20 +63,23 @@ fs.epoll.max_user_instances - maximum number of devices - per user (2.6.27.8+) fs.epoll.max_user_watches - maximum number of "watched" fds - per user (2.6.27.8+) -iptables - remove the ip_conntrack module (it limits the maximum tcp - connections, adds extra overhead (slow)). It's probably better to remove - all the iptables modules. - net.ipv4.tcp_max_orphans - might be worth a look if things go wrong net.core.netdev_max_backlog - maximum device backlog +Related applications +-------------------- ifconfig <dev> txqueuelen <val> - set device transmission queue len +iptables - remove the ip_conntrack module (it limits the maximum tcp + connections, adds extra overhead (slow)). It's probably better to remove + all the iptables modules. + + 2. Monitoring (values to watch for) ----------------------------------- -2.1 FDs +2.1 File descriptors fs.dentry-state - format: nr. dentries, nr. unused, age_limit, want_pages fs.file-nr - format: allocated, unused, max (==fs.file-max) @@ -87,14 +94,15 @@ fs.inode-state - format: nr. allocated, nr. free, preshrink /proc/net/sockstat -3. Ser settings +3. Sip-router settings +---------------------- -- don't forget to increase tcp_max_connections and the amount of shared memory -- you should increase the number of ser "tcp_children" processes (-N no) -As a rule of thumb, (maximum simultaneous connections)/2000 should be ok -- you might have to decrease TCP_BUF_SIZE to a smaller value (e.g 8K) -- you might want to increase PKG_MEM_POOL_SIZE (for large queues) +- Don't forget to increase tcp_max_connections and the amount of shared memory +- You should increase the number of ser "tcp_children" processes (-N no) + As a rule of thumb, (maximum simultaneous connections)/2000 should be ok +- You might have to decrease TCP_BUF_SIZE to a smaller value (e.g 8K) +- You might want to increase PKG_MEM_POOL_SIZE (for large queues) -- you might need to increase the maximum open fds limit before starting ser - (e.g. ulimit -n 1000000) +- You might need to increase the maximum open fds limit before starting ser + (e.g. ulimit -n 1000000)