17 Jan
2005
17 Jan
'05
4:43 p.m.
Hi,
Is there a simple way to lookup contacts directly from a DB table
instead of memory? A quick look into the register module doesn't show
anything like that.
The idea behind that is to build a redundant SER cluster. All nodes
write their registrations directly into a MySQL master database, which
replicates the contacts to the slave databases. The SERs then lookup
contacts from their local slave databases. The master database will be
secured by a MySQL cluster.
This is because if I replicate registers on SIP layer (for example with
t_replicate()), I have to synchronize the location databases manually
from time to time, because register replications get lost on heavy SIP
load, and there is actually no way to automatically load contacts which
are stored on other nodes while one SER was down.
Their is also no need to send replicated registers over the net which
reduces SIP messages.
I don't really know how much impact this approach will have regarding
performance (going around the internal location memory), but I think it
will not hurt as much as having out-of-synch location tables on the SER
nodes.
Any comments on this approach? Anybody who is also interested in such
approach?
Andy
17 Jan
17 Jan
9:38 p.m.
Have a look at:
http://www.iptel.org/ser/doc/modules/html/usrloc.html#AEN153
However, this probably does not do all you need, as you still need lookup to
synchronize with the db. This could maybe be done with the timer_interval
parameter (lower it). I haven't had a look at the code implementing the
flush/synchronization, but that could be a start.
I haven't really validated the idea of using database replication/cluster,
so there are probably other things to consider as well. I wouldn't worry
about heavy SIP load and lost registers because you should a long time
before that have taken measures to keep your systems at a lower load...
However, synchronization of servers that went down is an issue. Haven't
really seen any discussions on the list on replication in itself as a
redundancy issue. Anyone?
Locations are not really that critical, are they? If you use
registration intervals of ex. 10 minutes, how do you even manage to
synchronize the databases manually?
Any more thoughts on your usage scenario and why you would put the effort
into moving t_replicate to SQL replication?
g-)
Andreas Granig wrote:
Hi,
Is there a simple way to lookup contacts directly from a DB table
instead of memory? A quick look into the register module doesn't show
anything like that.
The idea behind that is to build a redundant SER cluster. All nodes
write their registrations directly into a MySQL master database, which
replicates the contacts to the slave databases. The SERs then lookup
contacts from their local slave databases. The master database will be
secured by a MySQL cluster.
This is because if I replicate registers on SIP layer (for example
with t_replicate()), I have to synchronize the location databases
manually from time to time, because register replications get lost on
heavy SIP load, and there is actually no way to automatically load
contacts which are stored on other nodes while one SER was down.
Their is also no need to send replicated registers over the net which
reduces SIP messages.
I don't really know how much impact this approach will have regarding
performance (going around the internal location memory), but I think
it will not hurt as much as having out-of-synch location tables on
the SER nodes.
Any comments on this approach? Anybody who is also interested in such
approach?
Andy
_______________________________________________
Serusers mailing list
serusers(a)lists.iptel.org
http://lists.iptel.org/mailman/listinfo/serusers
18 Jan
18 Jan
12:18 p.m.
See comments inline...
Greger V. Teigre wrote:
Have a look at:
http://www.iptel.org/ser/doc/modules/html/usrloc.html#AEN153
Synchronization *to* a DB works of course out of the box (needed for
saving locations).
But for clustering you will need to synchronize your memory *from* the
DB or from the memory of the other nodes on the fly to get locations
from other nodes so you can lookup them again.
Locations are not really that critical, are they?
If you use
registration intervals of ex. 10 minutes, how do you even manage to
synchronize the databases manually?
We've about 20.000 users, which would result in ~2.900.000 successful
registrations per day or ~35 successful registerations per second, and
that means ~70 register messages per second (because of the www_challenge).
Replicating on SIP layer means that every successful REGISTER has to be
sent to every server, so IMHO this just will not scale.
Cheers,
Andy
5:51 p.m.
:-) You never know where people are when they ask questions. Better start
with the basics... See inline comments.
Yes, I did a quick look at the usrloc module code and can see that resync
from DB should be possible, but will probably take some time and I don't
think that is a good path.
Actually, Java Rocxx (Paul) and I tried to get a carrier-grade platform
discussion going in December, I think, but the reponse was not really there.
We don't use sql clusters, but have distributed data centers with IPsec
VPN connecting the centers. Each center is as self-contained as possible.
I think Paul mentioned that his approach was to use the FIFO queue with a
TCP/IP wrapper. An ul_add FIFO command to each server should probably do
the trick. I assume that the sql synchronization will not take less
bandwidth?! Unfortunately, Paul's company probably would like to keep their
solution secret, it's part of their commercial product. However, I would be
interested in a joint effort that could be contributed to the project. A
SOAP wrapper comes to mind as a solution to the network exchange, but maybe
even the SOAP exhange will add too much overhead? Well, probably not, but
just thinking out aloud here.
What would be the best replication implementation in ser? A new module?
The code would trigger a network exchange and would have to be forked.
Honestly, I haven't looked at the inner workings of ser, so I don't know.
Any pointers, anybody?
Excellent! Don't make it a problem until it is :-) Do you do load balancing
with DNS SRV or Vovida load balancer or similar? Is the MySQL cluster
networked or in the same physical location? (I don't know much about mySQL
clusters...)
Paul claimed voicemail to be their single, biggest problem, citing sems
to be the most promising. We don't even offer voicemail, but have looked at
voicemail solutions from the PSTN domain :-( How have you solved
voicemail?
And of course RTP proxy :-) I like rtpproxy because its simple, you
have good control and it's written in C. However, it has no load balancing
and you don't really have control over sessions (stats). It looks like most
large installation have chosen python-based mediaproxy due to these two
things (as well as video-stream support). Any thoughts?
g-)
Have a look
at:
http://www.iptel.org/ser/doc/modules/html/usrloc.html#AEN153
Synchronization *to* a DB works of course out of the box (needed for
saving locations).
But for clustering you will need to synchronize your memory *from* the
DB or from the memory of the other nodes on the fly to get locations
from other nodes so you can lookup them again. Locations
are not really that critical, are they? If you use
registration intervals of ex. 10 minutes, how do you even manage to
synchronize the databases manually?
We've about 20.000 users, which would result in ~2.900.000 successful
registrations per day or ~35 successful registerations per second, and
that means ~70 register messages per second (because of the
www_challenge).
Replicating on SIP layer means that every successful REGISTER has to
be sent to every server, so IMHO this just will not scale.
6:51 p.m.
Greger V. Teigre wrote:
Yes, I did a quick look at the usrloc module code and
can see that
resync from DB should be possible, but will probably take some time and
I don't think that is a good path.
Yes, either you'll lose the fast memory access if always looking up the
DB or you synch back from the DB periodically which would mean that it
could be the case that a location isn't up to date or has expired on a node.
possible. I think Paul mentioned that his approach was
to use the FIFO
queue with a TCP/IP wrapper. An ul_add FIFO command to each server
should probably do the trick.
Also a possible approach, maybe we should think about something like that.
I assume that the sql synchronization
will not take less bandwidth?!
A rough estimate shows that it's at least less then sending a SIP
message. In fact it's more or less the sql statement that has to be
transmitted.
But basically in my case I don't see the network as bottleneck since all
our equipment is located in a 100MBit LAN.
A SOAP wrapper comes to mind as a solution
to the network exchange, but maybe even the SOAP exhange will add too
much overhead?
We currently handle call forwards (conditional and unconditional) with
location databases too, and here a cronjob runs every minute for
inserting the CFC and CFUC locations by calling serctl (so it's related
to Paul's approach of location handling, we just don't use the FIFO
directly).
But it's planned to use SOAP for realtime provisioning of this stuff, so
thanks for the hint, maybe this could be done with the location database
too. For this a kind of soap module would be great to prevent exec_msg
calls for each registration.
Of course also just thinking loud ;o)
Do you do load
balancing with DNS SRV or Vovida load balancer or similar?
It's currently DNS SRV, but I'd like to set up some kind of load
balancer (maybe based on
http://www.linuxvirtualserver.org/software/ipvs.html), just because so
much UACs don't get the SRV handling right, and it's always a fight to
get potential new UAC vendors to fix it.
Is the MySQL
cluster networked or in the same physical location? (I don't know much
about mySQL clusters...)
It will be in the same VLAN. MySQL clustering is still beta, so a first
step would a simple replication (let the SERs write their locations into
the master database and lookup the location in the local slave
databases), because replication works well with MySQL. The master
database could be secured by a heartbeat cluster.
The advantage of the MySQL cluster would be that the SERs could write
also into their local database. But maybe it would generally be a good
idea to move the databases to own dedicated machines. We'll see.
Paul claimed voicemail to be their single, biggest
problem, citing
sems to be the most promising. We don't even offer voicemail, but have
looked at voicemail solutions from the PSTN domain :-( How have you
solved voicemail?
We use Asterisk for voicemail. The asterisk instances run on the same
machine as the SERs, and a SER always redirects voicemail calls to the
local Asterisk, with a failure route to a second one. Not that redundant
and scaleable, but I don't see voicemail as operation critical as normal
call routing.
The voicemail storage is shared via NFS by the way.
And of course RTP proxy :-) I like rtpproxy
because its simple, you
have good control and it's written in C. However, it has no load
balancing and you don't really have control over sessions (stats). It
looks like most large installation have chosen python-based mediaproxy
due to these two things (as well as video-stream support). Any thoughts?
We use mediaproxy here, because it's scalable via DNS SRV and it
supports T.38 (I sent a patch to the serdev list some months ago to
handle reinvites, and the guys at AG Projects thought about implementing
something like that, but I haven't heard anything about that since then).
Cheers,
Andy
7251
days inactive
7252
days old
4 comments
3 participants
participants (3)
-
Andreas Granig -
Andreas Granig -
Greger V. Teigre