Hello,
I'm using several modules that access a database. I've noticed that when I start ser in forked mode with "children=8", it generates 62 connections to database. Does every child open a separate set of database handles (one for every module)? I remember I saw on the mailing list advises to run ser with hundreds of children to avoid problems when no workers are available to process a message. If my guess is true it would generate hundreds (or even thousands) of database connections. I'm really interested to hear how people out there deal with it.
On a separate note I'm curious whether number of children in ser is limited to a given startup value or ser is able to fork additional children as needed? Thank you,
Michael
On 18-07-2005 15:43, Michael Ulitskiy wrote:
Hello,
I'm using several modules that access a database. I've noticed that when I start ser in forked mode with "children=8", it generates 62 connections to database. Does every child open a separate set of database handles (one for every module)?
Yes.
I remember I saw on the mailing list advises to run ser with hundreds of children to avoid problems when no workers are available to process a message. If my guess is true it would generate hundreds (or even thousands) of database connections. I'm really interested to hear how people out there deal with it.
No, you don't need hundreds of children, usualy 16 is maximu what you need. Newer ser versions contain connection pool, so each child will open exactly one database connection and it will be reused across modules.
On a separate note I'm curious whether number of children in ser is limited to a given startup value or ser is able to fork additional children as needed? Thank you,
It is limited by the value configured in the config file.
Jan.
On Saturday 23 July 2005 06:20 pm, Jan Janak wrote:
On 18-07-2005 15:43, Michael Ulitskiy wrote:
Hello,
I'm using several modules that access a database. I've noticed that when I start ser in forked mode with "children=8", it generates 62 connections to database. Does every child open a separate set of database handles (one for every module)?
Yes.
I remember I saw on the mailing list advises to run ser with hundreds of children to avoid problems when no workers are available to process a message. If my guess is true it would generate hundreds (or even thousands) of database connections. I'm really interested to hear how people out there deal with it.
No, you don't need hundreds of children, usualy 16 is maximu what you need. Newer ser versions contain connection pool, so each child will open exactly one database connection and it will be reused across modules.
What about processing that involves slow DNS queries? I thought this would eat up available workers quite quickly and further processing will be blocked? Please correct me if I'm wrong. Also has connection pool been implemented (or are there plans to implement it) for postgres? This is very important for me as I need to use it.
On a separate note I'm curious whether number of children in ser is limited to a given startup value or ser is able to fork additional children as needed? Thank you,
It is limited by the value configured in the config file.
Thanks for the info.
Jan.
On 24-07-2005 17:05, Michael Ulitskiy wrote:
On Saturday 23 July 2005 06:20 pm, Jan Janak wrote:
On 18-07-2005 15:43, Michael Ulitskiy wrote:
Hello,
I'm using several modules that access a database. I've noticed that when I start ser in forked mode with "children=8", it generates 62 connections to database. Does every child open a separate set of database handles (one for every module)?
Yes.
I remember I saw on the mailing list advises to run ser with hundreds of children to avoid problems when no workers are available to process a message. If my guess is true it would generate hundreds (or even thousands) of database connections. I'm really interested to hear how people out there deal with it.
No, you don't need hundreds of children, usualy 16 is maximu what you need. Newer ser versions contain connection pool, so each child will open exactly one database connection and it will be reused across modules.
What about processing that involves slow DNS queries? I thought this would eat up available workers quite quickly and further processing will be blocked? Please correct me if I'm wrong.
You can install local (running on the same host as the proxy) DNS cache that can also cache negative entries (those that do not exist or cannot be looked up), such as dnsmasq. This way the slow query would be performed once and subsequent queries will be answered from the cache.
Moreover, kernel maintains a queue of incoming SIP requests and it will continue receiving SIP messages even if all processes are blocked. The kernel starts to drop incoming SIP messages once the queue is full.
Also has connection pool been implemented (or are there plans to implement it) for postgres? This is very important for me as I need to use it.
Yes, there is a plan to use the connection pool in postgres as well.
Jan.
On Monday 25 July 2005 08:21 am, Jan Janak wrote:
No, you don't need hundreds of children, usualy 16 is maximu what you need. Newer ser versions contain connection pool, so each child will open exactly one database connection and it will be reused across modules.
What about processing that involves slow DNS queries? I thought this would eat up available workers quite quickly and further processing will be blocked? Please correct me if I'm wrong.
You can install local (running on the same host as the proxy) DNS cache that can also cache negative entries (those that do not exist or cannot be looked up), such as dnsmasq. This way the slow query would be performed once and subsequent queries will be answered from the cache. Moreover, kernel maintains a queue of incoming SIP requests and it will continue receiving SIP messages even if all processes are blocked. The kernel starts to drop incoming SIP messages once the queue is full.
Well, I guess it's not a solution. It's a workaround at best. I'd vote for something apache-like. I.e. master process monitoring the number of idle children and forking additional as needed. Thanks anyway. Also could you (or someone) give me a rough estimation on what is the optimal number of children to serve let's say 10k sip clients as registrar and proxy with moderately complex config file, but not involving DNS queries? INVITEs may involve several database queries to a dedicated database server. I understand that the question is rather vague, I'm just interested in a rough estimation and some real-world numbers people use.
Also has connection pool been implemented (or are there plans to implement it) for postgres? This is very important for me as I need to use it.
Yes, there is a plan to use the connection pool in postgres as well.
Great! Thanks Jan.
Jan.
Michael
On Jul 25, 2005 at 12:07, Michael Ulitskiy mdu113@acedsl.com wrote:
On Monday 25 July 2005 08:21 am, Jan Janak wrote:
No, you don't need hundreds of children, usualy 16 is maximu what you need. Newer ser versions contain connection pool, so each child will open exactly one database connection and it will be reused across modules.
What about processing that involves slow DNS queries? I thought this would eat up available workers quite quickly and further processing will be blocked? Please correct me if I'm wrong.
You can install local (running on the same host as the proxy) DNS cache that can also cache negative entries (those that do not exist or cannot be looked up), such as dnsmasq. This way the slow query would be performed once and subsequent queries will be answered from the cache. Moreover, kernel maintains a queue of incoming SIP requests and it will continue receiving SIP messages even if all processes are blocked. The kernel starts to drop incoming SIP messages once the queue is full.
Well, I guess it's not a solution. It's a workaround at best. I'd vote for something apache-like. I.e. master process monitoring the number of idle children and forking additional as needed. Thanks anyway.
Why not start them all from the beginning? Is not like they will eat a lot of resources...
Also could you (or someone) give me a rough estimation on what is the optimal number of children to serve let's say 10k sip clients as registrar and proxy with moderately complex config file, but not involving DNS queries? INVITEs may involve several database queries to a dedicated database server. I understand that the question is rather vague, I'm just interested in a rough estimation and some real-world numbers people use.
Try it and if you can see packets in the socket buffers (netstat) increase the number of children. I would use 20-50 children. The "lots of db connections" problem is solved when using connection pools (or by increasing the maximum connections in mysql's config :-)). You could use also tune the dns timeout (limit the maximum time a dns request can take). For this you would need either unstable (grep NEWS for dns_) or to edit your /etc/resolv.conf and set the timeout and attempts options and remove the search or domain lines.
Andrei
On 25-07-2005 19:28, Andrei Pelinescu-Onciul wrote:
On Jul 25, 2005 at 12:07, Michael Ulitskiy mdu113@acedsl.com wrote:
On Monday 25 July 2005 08:21 am, Jan Janak wrote:
No, you don't need hundreds of children, usualy 16 is maximu what you need. Newer ser versions contain connection pool, so each child will open exactly one database connection and it will be reused across modules.
What about processing that involves slow DNS queries? I thought this would eat up available workers quite quickly and further processing will be blocked? Please correct me if I'm wrong.
You can install local (running on the same host as the proxy) DNS cache that can also cache negative entries (those that do not exist or cannot be looked up), such as dnsmasq. This way the slow query would be performed once and subsequent queries will be answered from the cache. Moreover, kernel maintains a queue of incoming SIP requests and it will continue receiving SIP messages even if all processes are blocked. The kernel starts to drop incoming SIP messages once the queue is full.
Well, I guess it's not a solution. It's a workaround at best. I'd vote for something apache-like. I.e. master process monitoring the number of idle children and forking additional as needed. Thanks anyway.
Why not start them all from the beginning? Is not like they will eat a lot of resources...
Also could you (or someone) give me a rough estimation on what is the optimal number of children to serve let's say 10k sip clients as registrar and proxy with moderately complex config file, but not involving DNS queries? INVITEs may involve several database queries to a dedicated database server. I understand that the question is rather vague, I'm just interested in a rough estimation and some real-world numbers people use.
Try it and if you can see packets in the socket buffers (netstat) increase the number of children. I would use 20-50 children.
This it to be on the extra-safe side. I was using 16 children on a similar system (in terms of number of subscribers) with DNS turned on.
Jan.
-
Andrei Pelinescu-Onciul -
Jan Janak -
Michael Ulitskiy