Hi,
I've uploaded a new module "path" to the tracker on sourceforge, which allows a intermediate proxy (an openser instance working as loadbalancer) to add it's URI into a Path-HF in various ways:
- add_path(): Adds URI including the address of the outgoing interface as Path-HF in the form of "Path: sip:1.2.3.4:5060"
- add_path_received(): like the above, but adds a parameter "received=sip:2.3.4.5:1234" to the URI to pass the received-address of an UAC to a registrar
- both of the above functions also allow to pass a userpart for the Path-URI, so add_path("loadbalancer") results in a Path-HF "Path: sip:loadbalancer@1.2.3.4:5060"
- If the module parameter "use_received" is set to 1, the module hooks into rr-processsing and sets the "received"-parameter of a Route-URI as destination-uri if available. This is used in combination with add_path_received(), where subsequent requests are sent to the received-address of the initial registration (aka the URI given in the received-parameter of the Path-HF).
Also included is a patch to the registrar module, which allows to store the URI in the received-parameter of the first Path-HF as received-value of usrloc and set the NAT flag for this contact. This feature can be enabled by setting the parameter "path_use_received" of the registrar module to 1. One should be aware that nat-pinging of such a contact isn't supported by nathelper/mediaproxy yet, but the values are in place (since I use an external application for nat-pinging).
Some feedback is very much appreciated.
Cheers, Andy
Hi,
To show a possible scenario for the new Path features, consider the following setup:
[UAC] <--> (NAT) <-+-> [LB] <--+--> [REGn] | | [Peer-Proxies] <---+ +--> [PRXn] <-- DNS-SRV --> [GWn]
Where UAC are the clients (optionally behind NAT), LB is a SIP loadbalancer based on OpenSER, REGn is a federation of Registrars and PRXn a federation of Proxies, all using the new cacheless usrloc feature of OpenSER. GWn is a federation of SIP GWs for PSTN termination, and Peer-Proxies are other SIP service providers.
LB is the outbound proxy of the UACs, and the registrars and proxies behind it are in a private LAN. LB uses the dispatcher module for load balancing and failover handling of these nodes. By using the cacheless usrloc feature in combination with a mysql cluster, registrars and proxies can scale up to an arbitrary number of nodes. Since only one public IP can be advertised to the clients for proper NAT traversal, LB has to be clustered in an active/passive way using IP-takeover for high-availability. If the LB becomes the bottleneck, a second LB pair can be set up and been advertised as outbound proxy to new UACs, using the same proxy/registrar federation as backend. But since LB doesn't need to access a DB or perform much processing, a quite high CPS rate can be expected (not measured yet though).
An example-config for the LB may look like that (only relevant parts shown):
#+ ... loadmodule "dispatcher.so" loadmodule "path.so"
modparam("dispatcher", "list_file", "/etc/dispatcher.lst") modparam("path", "use_received", 1) ... route { ... if(method == "REGISTER") { if( /* NAT check here */ ) { ... add_path_received("loadbalancer"); ... } else { ... add_path("loadbalancer"); ... } ds_select_dst( /* select a registrar here */ ); } else if(method == "INVITE") { ... ds_select_dst( /* select a proxy here */ ); record_route(); ... } ... }
/* * failure routes for failover, reply-routes for NAT-handling etc. * go here */ #-
add_path() ensures, that subsequent requests (INVITES etc.) from the proxy are routed via the same loadbalancer, which is used by the UAC as outbound proxy, and use_received=1 lets requests from backend proxies route to the NATed address if available.
The config for the registrars is straight-forward, except for the registrar-module parameters:
#+ ... modparam("registrar", "use_path", 1) modparam("registrar", "path_mode", 0) modparam("registrar", "path_use_received", 1) ... #-
This ensures that the Path header is recognized (use_path=1), the Path header is never included in the reply (path_mode=0), and the received-parameter of the first Path URI is saved as received-value in usrloc, if available.
Note, that nat-pinging is not properly supported by this setup (as noted before), because UACs have to be pinged with the source-address set to the address of the LB. This could either be accomplished by using OPTION for nat-ping and route that request according to the stored Path header (will put a lot of unnecessary load to the LB), or let the LB natping the UACs (also unnecessary load for the LB, and LB would require access to usrloc), or use an external application which spoofs the LB address and directly fetches the received-address from the location table (used in our setup and seems to be the most flexible and scalable way for me).
Hope, this sheds some light on the usefulness of the new Path features. Please be aware that this is currently a proof-of-concept implementation for a highly scalable SIP system using openser, and is not in production yet. So comments are highly welcome.
Cheers, Andy
Andreas Granig writes:
Since only one public IP can be advertised to the clients for proper NAT traversal, LB has to be clustered in an active/passive way using IP-takeover for high-availability.
andy,
how does passive LB know that active LB is not anymore functional? do you assume a third monitor process that sends SIP requests through active LB and if not successful, make passive active or do LBs somehow watch each other?
-- juha
Juha Heinanen wrote:
how does passive LB know that active LB is not anymore functional? do you assume a third monitor process that sends SIP requests through active LB and if not successful, make passive active or do LBs somehow watch each other?
We currently use heartbeat for LB failover, and the passive host routes OPTION requests (one per second) through the active one using sipsak and a selfmade responder. If the host itself or the uplink is down, heartbeat takes over automatically, and if three consecutive OPTION requests are lost, hb_takeover is executed on the standby node. LBs also have no local harddisk, but use PXE-boot and two NICs in active-backup bonding-mode on two seperate switches to minimize sources of failures.
The setup is quite experimental right now, but especially LB failover undergoes intensive testing right now, since it's the most critical part in the system.
Andy
Hi Andreas,
look pretty cool this feature :)...
I will overview the patch that need to be applied on register module and if so, I will commit it. then feel free to upload the module on CVS.
regarding the natping stuff....I will try to brainstorm a little bit to find some built-in solutions....for the moment: 1) if use OPTIONS ping, we can use path to route the ping via the balancer as OBP 2) a light usrloc on OBP just for natping purposes.... 3) open for suggestions...
regards, bodgan
Andreas Granig wrote:
Hi,
I've uploaded a new module "path" to the tracker on sourceforge, which allows a intermediate proxy (an openser instance working as loadbalancer) to add it's URI into a Path-HF in various ways:
- add_path(): Adds URI including the address of the outgoing interface
as Path-HF in the form of "Path: sip:1.2.3.4:5060"
- add_path_received(): like the above, but adds a parameter
"received=sip:2.3.4.5:1234" to the URI to pass the received-address of an UAC to a registrar
- both of the above functions also allow to pass a userpart for the
Path-URI, so add_path("loadbalancer") results in a Path-HF "Path: sip:loadbalancer@1.2.3.4:5060"
- If the module parameter "use_received" is set to 1, the module hooks
into rr-processsing and sets the "received"-parameter of a Route-URI as destination-uri if available. This is used in combination with add_path_received(), where subsequent requests are sent to the received-address of the initial registration (aka the URI given in the received-parameter of the Path-HF).
Also included is a patch to the registrar module, which allows to store the URI in the received-parameter of the first Path-HF as received-value of usrloc and set the NAT flag for this contact. This feature can be enabled by setting the parameter "path_use_received" of the registrar module to 1. One should be aware that nat-pinging of such a contact isn't supported by nathelper/mediaproxy yet, but the values are in place (since I use an external application for nat-pinging).
Some feedback is very much appreciated.
Cheers, Andy
Devel mailing list Devel@openser.org http://openser.org/cgi-bin/mailman/listinfo/devel
Bogdan-Andrei Iancu wrote:
I will overview the patch that need to be applied on register module and if so, I will commit it. then feel free to upload the module on CVS.
Could you please also review the module code? I'm not really a pro programmer, so there are for sure some things which may be solved more efficiently...
regarding the natping stuff....I will try to brainstorm a little bit to find some built-in solutions....for the moment:
- if use OPTIONS ping, we can use path to route the ping via the
balancer as OBP
I've looked at that too, but it would require quite some changes to nathelper/mediaproxy/usrloc, so I left it alone for now... But the main problem I see here is, that one probably won't like to route 20.000 OPTION requests per minute via the OBP, because it's the only part, which is quite hard to scale (in economical terms, since you always need a pair of it for availability reasons).
- a light usrloc on OBP just for natping purposes....
Maybe by passing back the Path HF in 200-OK reply from registrar to OBP, extracting the address of the received-param of the first Path URI and plainly save that in memory? This would also require synchronization between active and passive node, which is quite subobtimal for availability, IMHO.
- open for suggestions...
No other ideas for now from me, except contributing our external tool to the openser project (but it's limited to mysql, so maybe to specific).
Andy