Hi guys,
I'd like to propose another possibility for a highly-available and scalable system design based on SER:
The problems I've encountered for scalable systems are: - Distribution of the user location and alias location among the nodes (user location is based on registrations, alias location comes from web interfaces and is used for call forwarding). - Reloading up to date location tables after breakdown and recovery of a node
So I'm just thinking loud about the following provisioning system:
- Write a client which fulfills the this demands:
- Receive one or more locations from SER via a SER module or from a web application and distribute them to other known clients. Take care of retransmissions if a client isn't reachable or reports a temporary failure.
- Receive one or more locations from other clients and write them into the SER FIFO. If writing into the FIFO fails, try to write directly into the database (location-table, alias-table etc.). Report a temporary failure if this also fails.
Maybe a centralized server should be used which receives the locations from the clients and distributes them to other clients, so that the nodes just know about the server and nothing about other nodes. This would make integration of new nodes easy. On the other hand, it's another single point of failure, so a decentralized solution should be considered. But that would mean that you've to inform every node about the existence of a new node.
The protocol used between the nodes should be simple and fast. So I think SOAP drops out here. Maybe XMLRPC or ICE (http://www.zeroc.com/ice.html) could be used.
One might think now why not just use replication on SIP layer, but t_replicate only supports one peer and you've no possibility to get locations on a node while it's down. Replication of other location tables like the alias-table is also not possible.
I'd be willing to release these parts as GPL for creating an open framework for carrier-grade SER integration, so any feedback, improvements or flames are highly welcome.
Cheers, Andy
Dear Andreas,
As I indicated in an earlier email, I would be interested in taking part in a joint effort to further develop ser's high-availability and scalability (HAS). I would probably have to do some development anyway, and I would prefer to see such support in the public domain. In Nov/Dec I called for responses on a SOAP-based provisioning interface, but heard nothing, so here is an overlap of interests. Please see inline comments.
Andreas Granig wrote:
Hi guys,
I'd like to propose another possibility for a highly-available and scalable system design based on SER:
The problems I've encountered for scalable systems are:
- Distribution of the user location and alias location among the nodes (user location is based on registrations, alias location comes from web interfaces and is used for call forwarding).
- Reloading up to date location tables after breakdown and recovery of a node
If we are to define the elements required as part of a HAS-effort, I think I would also include load balancing across servers in a single data center, not only distribution of usrloc/alias replication across centers. You mentioned yourself the problem of clients not supporting DNS SRV. I think some kind of reference design using Linux HA or virtual server would be useful for the community. I have seen several people asking questions, but I haven't seen documentation of a successful implementation. The reference design must of course handle NAT traversal and the possibility of RTP proxying. Anybody with experiences or thoughts?
So I'm just thinking loud about the following provisioning system:
Write a client which fulfills the this demands:
Receive one or more locations from SER via a SER module or from a web application and distribute them to other known clients. Take care of retransmissions if a client isn't reachable or reports a temporary failure.
Receive one or more locations from other clients and write them into the SER FIFO. If writing into the FIFO fails, try to write directly into the database (location-table, alias-table etc.). Report a temporary failure if this also fails.
yes, I see you here define a more general network wrapper to the FIFO. Why not extend the scope right now to support any FIFO implemented command, also future? If FIFO writing is not possible, I believe an alert should be made and the request left in a transaction queue. There is probably a reason for the FIFO error and also writing directly to the database will mess up the synch between DB and server.
Maybe a centralized server should be used which receives the locations from the clients and distributes them to other clients, so that the nodes just know about the server and nothing about other nodes. This would make integration of new nodes easy. On the other hand, it's another single point of failure, so a decentralized solution should be considered. But that would mean that you've to inform every node about the existence of a new node.
I agree that that each node should be independent. I once implemented a token ring-like system where each node would report its existance. Something similar would be appropriate. We could even use SIP REGISTER from each client using a specifically crafted username and use NOTIFYs to update everybody about changes :-) Each client could use its local ser server to lookup contacts. Well, this could create a dependency loop, but it was sort of a cool idea ... Bringing up new clients will of course need some special thoughts in terms of their status and at what point in replication they should enter. Probably each transaction should use a unique transaction id...
The protocol used between the nodes should be simple and fast. So I think SOAP drops out here. Maybe XMLRPC or ICE (http://www.zeroc.com/ice.html) could be used.
Why do you think SOAP should be dropped? If this is to be a general module for both provisioning and replication across servers, we must handle many different usage scenarios, ex. provisioning over the Internet, and we should use a framework where we can get as much as possible for "free", ex. SSL/TLS, authentication etc. In my experience, SOAP can be both simple and fast and there are far more auto-generation tools/frameworks etc for SOAP. A simple RPC-type SOAP message is not really far from XML-RPC, basically an envelope extra. The ability to auto-generate a client based on a WSDL file is pretty nice, as well. With regards to ICE, we should probably avoid using toolkits where commercial licenses must be bought for commercial usage. I have used gSOAP (http://www.cs.fsu.edu/~engelen/soap.html), which is a powerful, but very simple to use C/C++.
I'd be willing to release these parts as GPL for creating an open framework for carrier-grade SER integration, so any feedback, improvements or flames are highly welcome.
Ditto.
I have here contributed to increasing the scope... But I think the correct approach would be to make something simple in the beginning, maybe only the provisioning and the queue and a text-based config file for clients to replicate to. We could then add the ser module for forwarding usrloc to the local client.
Andreas: What's your time scope?
Core developers of ser: I think we need your opinion(s) before we start anything?!
Regards, Greger
You might allso want to check the LVS mailinglist, there is some discussion going on for a SIP module for the LVS with call-id persistence, that would solve distributing request among the real servers.
Greger V. Teigre wrote:
Dear Andreas,
As I indicated in an earlier email, I would be interested in taking part in a joint effort to further develop ser's high-availability and scalability (HAS). I would probably have to do some development anyway, and I would prefer to see such support in the public domain. In Nov/Dec I called for responses on a SOAP-based provisioning interface, but heard nothing, so here is an overlap of interests. Please see inline comments.
Andreas Granig wrote:
Hi guys,
I'd like to propose another possibility for a highly-available and scalable system design based on SER:
The problems I've encountered for scalable systems are:
- Distribution of the user location and alias location among the nodes (user location is based on registrations, alias location comes from web interfaces and is used for call forwarding).
- Reloading up to date location tables after breakdown and recovery of a node
If we are to define the elements required as part of a HAS-effort, I think I would also include load balancing across servers in a single data center, not only distribution of usrloc/alias replication across centers. You mentioned yourself the problem of clients not supporting DNS SRV. I think some kind of reference design using Linux HA or virtual server would be useful for the community. I have seen several people asking questions, but I haven't seen documentation of a successful implementation. The reference design must of course handle NAT traversal and the possibility of RTP proxying. Anybody with experiences or thoughts?
So I'm just thinking loud about the following provisioning system:
Write a client which fulfills the this demands:
Receive one or more locations from SER via a SER module or from a web application and distribute them to other known clients. Take care of retransmissions if a client isn't reachable or reports a temporary failure.
Receive one or more locations from other clients and write them into the SER FIFO. If writing into the FIFO fails, try to write directly into the database (location-table, alias-table etc.). Report a temporary failure if this also fails.
yes, I see you here define a more general network wrapper to the FIFO. Why not extend the scope right now to support any FIFO implemented command, also future? If FIFO writing is not possible, I believe an alert should be made and the request left in a transaction queue. There is probably a reason for the FIFO error and also writing directly to the database will mess up the synch between DB and server.
Maybe a centralized server should be used which receives the locations from the clients and distributes them to other clients, so that the nodes just know about the server and nothing about other nodes. This would make integration of new nodes easy. On the other hand, it's another single point of failure, so a decentralized solution should be considered. But that would mean that you've to inform every node about the existence of a new node.
I agree that that each node should be independent. I once implemented a token ring-like system where each node would report its existance. Something similar would be appropriate. We could even use SIP REGISTER from each client using a specifically crafted username and use NOTIFYs to update everybody about changes :-) Each client could use its local ser server to lookup contacts. Well, this could create a dependency loop, but it was sort of a cool idea ... Bringing up new clients will of course need some special thoughts in terms of their status and at what point in replication they should enter. Probably each transaction should use a unique transaction id...
The protocol used between the nodes should be simple and fast. So I think SOAP drops out here. Maybe XMLRPC or ICE (http://www.zeroc.com/ice.html) could be used.
Why do you think SOAP should be dropped? If this is to be a general module for both provisioning and replication across servers, we must handle many different usage scenarios, ex. provisioning over the Internet, and we should use a framework where we can get as much as possible for "free", ex. SSL/TLS, authentication etc. In my experience, SOAP can be both simple and fast and there are far more auto-generation tools/frameworks etc for SOAP. A simple RPC-type SOAP message is not really far from XML-RPC, basically an envelope extra. The ability to auto-generate a client based on a WSDL file is pretty nice, as well. With regards to ICE, we should probably avoid using toolkits where commercial licenses must be bought for commercial usage. I have used gSOAP (http://www.cs.fsu.edu/~engelen/soap.html), which is a powerful, but very simple to use C/C++.
I'd be willing to release these parts as GPL for creating an open framework for carrier-grade SER integration, so any feedback, improvements or flames are highly welcome.
Ditto.
I have here contributed to increasing the scope... But I think the correct approach would be to make something simple in the beginning, maybe only the provisioning and the queue and a text-based config file for clients to replicate to. We could then add the ser module for forwarding usrloc to the local client.
Andreas: What's your time scope?
Core developers of ser: I think we need your opinion(s) before we start anything?!
Regards, Greger _______________________________________________ Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
I've changed to the serdev list, I think it's ontopic there.
Greger V. Teigre wrote:
DNS SRV. I think some kind of reference design using Linux HA or virtual server would be useful for the community. I have seen several people asking questions, but I haven't seen documentation of a successful implementation. The reference design must of course handle NAT traversal and the possibility of RTP proxying.
Ack. I heard about people testing http://www.linuxvirtualserver.org/software/ipvs.html with SER, but the system design and the results are, well, closed.
yes, I see you here define a more general network wrapper to the FIFO. Why not extend the scope right now to support any FIFO implemented command, also future?
Of course this should be possible.
If FIFO writing is not possible, I believe an alert should be made and the request left in a transaction queue. There is probably a reason for the FIFO error and also writing directly to the database will mess up the synch between DB and server.
I meant that when you definitely know that the SER is currently down you could write directly to the database. On the other hand writing to the DB would mean that you've to know the usrloc backend, so I agree that this is not a good idea.
Bringing up new clients will of course need some special thoughts in terms of their status and at what point in replication they should enter. Probably each transaction should use a unique transaction id...
Adding new SERs to a node isn't the common case I think. So I'd keep it rather simple in the beginning and hardcode the peers in a file or something like this. New contacts are automatically pushed to new nodes, and the overall location table has to be pulled by the new node. Just my opinion.
Why do you think SOAP should be dropped? If this is to be a general module for both provisioning and replication across servers, we must handle many different usage scenarios, ex. provisioning over the Internet, and we should use a framework where we can get as much as possible for "free", ex. SSL/TLS, authentication etc. In my experience, SOAP can be both simple and fast and there are far more auto-generation tools/frameworks etc for SOAP. A simple RPC-type SOAP message is not really far from XML-RPC, basically an envelope extra. The ability to auto-generate a client based on a WSDL file is pretty nice, as well.
I didn't perform a benchmark, just asked the mailserver guys here who use SOAP for provisioning, and they say it's ok for a few requests per second, but it will not perform for, say, 100 requests per second.
I have here contributed to increasing the scope... But I think the correct approach would be to make something simple in the beginning, maybe only the provisioning and the queue and a text-based config file for clients to replicate to. We could then add the ser module for forwarding usrloc to the local client.
My current status is:
I've written a SER module that hooks into the usrloc module by using the existing callback mechanism. On each change of a contact (insert, update, delete, expire) the module connects via UNIX socket to a external client and serializes the contact to the client via XML:
<?xml version="1.0" standalone="yes"?> <contact> <reptype>update</reptype> <domain>bG9jYXRpb24=</domain> <aor>MDE4OTA5MDAy</aor> <c>c2lwOjAxODkwOTAwMkA4My42NS40MC44Nzo1MDYwO3VzZXI9cGhvbmU7dHJhbnNwb3J0PXVkcA==</c> <expires>1106312247</expires> <q>-1</q> <callid>MTkzNDY0MjcxN0A4My42NS40MC44Nw==</callid> <cseq>15</cseq> <replicate>0</replicate> <state>1</state> <flags>0</flags> <user_agent>Q2lzY28gQVRBIDE4NiAgdjMuMi4wIGF0YXNpcCAoMDQxMTExQSk=</user_agent> </contact>
Note that strings except reptype are base64 encoded. The structure of the message will maybe change alot when we've defined the details (which protocol, which messages etc), the current module version is just a proof of concept, how contacts can be passed out of the SER. So please continue contributing ideas ;o)
Btw, for serializing the ucontact_t structure to XML I use the structs mechanism of libpdel (http://www.dellroad.org/pdel/index).
The external client (most of it has still to be implemented) has a list of peers where this message has to be send to. If sending fails, it will be put in a queue, and a timer triggers a retransmission. Clients also listen for such replication messages from other peers and write them to the SER fifo (or maybe directly via the module, don't know yet ;o)
The external client can also be reused for provisioning, because you just have to build the XML string in your web application and send it via UNIX socket to the client like the SER module does.
Btw, as you can see: My time scope is ASAP ;o)
Andy
Greger V. Teigre writes:
As I indicated in an earlier email, I would be interested in taking part in a joint effort to further develop ser's high-availability and scalability (HAS). I would probably have to do some development anyway, and I would prefer to see such support in the public domain. In Nov/Dec I called for responses on a SOAP-based provisioning interface, but heard nothing, so here is an overlap of interests.
greger,
we have done some work on xmlprc based provisioning and it looks promising. xmlrpc spec is three pages long and even i can understand it. soap spec, on the other hand, is far too thick and goes way above my head.
-- juha
Juha, Yes, I completely agree with you. However, I don't need to read the spec and far from understand it before I use it... ;-) So I did start to look at SOAP and have very good experiences, both in terms of usability and scalability. But, I don't have strong opinions, if the people who are going to use the interface are all against SOAP, XMLRPC is the right choice.
The xmlrpc-provisioning work you have done, can it be coordinated with Andreas' effort? g-)
Juha Heinanen wrote:
Greger V. Teigre writes:
As I indicated in an earlier email, I would be interested in taking part in a joint effort to further develop ser's high-availability and scalability (HAS). I would probably have to do some development anyway, and I would prefer to see such support in the public domain. In Nov/Dec I called for responses on a SOAP-based provisioning interface, but heard nothing, so here is an overlap of interests.
greger,
we have done some work on xmlprc based provisioning and it looks promising. xmlrpc spec is three pages long and even i can understand it. soap spec, on the other hand, is far too thick and goes way above my head.
-- juha
Greger V. Teigre writes:
The xmlrpc-provisioning work you have done, can it be coordinated with Andreas' effort?
i don't know about andrea's efforts, but i used xmlrpc for registering ser users' phone numbers with tier-1 enum database in finland. finnish authorities had specified xmlrpc interface for that purpose.
-- juha
Juha Heinanen wrote:
i don't know about andrea's efforts, but i used xmlrpc for registering ser users' phone numbers with tier-1 enum database in finland. finnish authorities had specified xmlrpc interface for that purpose.
In short, I plan to write an xmlrpc-based provisioning system for distributing FIFO commands. The main focus is sharing registration contacts and aliases among different SERs (all the data that can't be replicated on a lower (database) layer due to memory caching).
There's an external application on each SER host receiving contacts from a SER module via a local socket. These contacts are wrapped into xmlrpc (in fact the contact is converted into a FIFO command which is wrapped into xmlrpc) and are distributed to the same application on other SER hosts, which writes them into the SER FIFO. If an application is unreachable or reports an error, the distributed FIFO command is queued.
The application can receive arbitrary FIFO commands, and a flag indicates if the command should be redistributed to the other hosts.
Andy
Andreas Granig writes:
In short, I plan to write an xmlrpc-based provisioning system for distributing FIFO commands. The main focus is sharing registration contacts and aliases among different SERs (all the data that can't be replicated on a lower (database) layer due to memory caching).
is replication of register requests to other sers too slow for you?
regarding aliases, i don't use aliases table at all. i return the possible aliases called user as reply items to radius_does_uri_exist call.
There's an external application on each SER host receiving contacts from a SER module via a local socket. These contacts are wrapped into xmlrpc (in fact the contact is converted into a FIFO command which is wrapped into xmlrpc) and are distributed to the same application on other SER hosts, which writes them into the SER FIFO. If an application is unreachable or reports an error, the distributed FIFO command is queued.
this is close to what i did when i wrote fifo server. separate fifo server was supposed to be a temporary thing that can go away when ser learns to receive fifo commands over tcp. i still think that it is a bad idea to introduce a new process to the system that can go down requiring queuing and other complexities. so why just not introduce tcp support for ser fifo?
-- juha
Juha Heinanen wrote:
is replication of register requests to other sers too slow for you?
With t_replicate() only two SERs can replicate back and forth, and with stateless forward()ing packets may get lost because of missing retransmission. Beside that the domain tables (location etc) get out of synch if one of the SERs is down for a moment, because retransmission is only tried a few times.
Or is somebody aware of other possibilities for replication?
regarding aliases, i don't use aliases table at all. i return the possible aliases called user as reply items to radius_does_uri_exist call.
I use different domains for call forwarding, like cfuc for call forward unconditional, cfb for call forward on busy and so on:
In main route I have a routing logic like this if(lookup("cfuc")) { /* cfuc active */ } if(lookup("location") { /* a voip call */ } else { /* a pstn call */ } t_relay();
and in a failure route for example: if(t_check_status("486") && lookup("cfb")) {/*call forward on busy, relay*/}
So I register call forwarding destinations like aliases, like: serctl cfuc add <real-user-name> sip:<forwarded-uri>
These domains also have to be replicated among the SERs like the location domain.
this is close to what i did when i wrote fifo server. separate fifo server was supposed to be a temporary thing that can go away when ser learns to receive fifo commands over tcp. i still think that it is a bad idea to introduce a new process to the system that can go down requiring queuing and other complexities. so why just not introduce tcp support for ser fifo?
Queueing has to be done anyway, because if one SER is down for, say, ten minutes, the registrations received in that time on other SERs have to be replicated to that SER.
So you could do the queueing in a SER replication module, or just pass the modified contacts to an external application that handles this. If the external application isn't reachable, the module could write the contact messages to a DB table instead, which is read by the application periodically.
Andy
Andreas Granig writes:
With t_replicate() only two SERs can replicate back and forth, and with stateless forward()ing packets may get lost because of missing retransmission.
you can have any number of proxies participating in replication. you just check that you don't replicate a request that came from another proxy. what comes to packets getting lost, that is again udp vs. tcp issue.
Beside that the domain tables (location etc) get out of synch if one of the SERs is down for a moment, because retransmission is only tried a few times.
i don't see why this needs to be the case with db mode 2. when ser comes back up, it updates its location table from database.
In main route I have a routing logic like this if(lookup("cfuc")) { /* cfuc active */ } if(lookup("location") { /* a voip call */ } else { /* a pstn call */ } t_relay();
and in a failure route for example: if(t_check_status("486") && lookup("cfb")) {/*call forward on busy, relay*/}
So I register call forwarding destinations like aliases, like: serctl cfuc add <real-user-name> sip:<forwarded-uri>
i can't say anything about that, but you could also put forwarding info into avps and not use location table at all for them. thne you would have one less availability thing to worry about.
Queueing has to be done anyway, because if one SER is down for, say, ten minutes, the registrations received in that time on other SERs have to be replicated to that SER.
when ser comes up, it reads initializes its location info from database. so i don't see your point here.
-- juha
Juha Heinanen wrote:
you can have any number of proxies participating in replication.
What method are you thinking of? t_replicate() reports
ERROR: t_newtran: transaction already in process 0x4054d5ec
if you call it twice, like
t_replicate("foohost", "5060"); t_replicate("barhost", "5060");
Or do you mean something like
forward_tcp("foohost", "5060"); forward_tcp("barhost", "5060");
and on the receiving hosts
if(/* register from replicating host */) save_noreply("location");
which would be a possibility, indeed...
Beside that the domain tables (location etc) get out of synch if one of the SERs is down for a moment, because retransmission is only tried a few times.
i don't see why this needs to be the case with db mode 2. when ser comes back up, it updates its location table from database.
I think mode 1 (Write-Through) should be used because the SER could start up while some of the contacts aren't flushed to DB yet.
However, how would you set up your database connections here? Using a common usrloc database for all hosts (-> single point of failure)? This is the main point. _How_ do you share the contacts as reliable as possible so that a host can go down for a while without getting out of synch regarding the contacts?
Andy
Let me try to sort out the issues we are discussing here, so we at least can see if we agree to the goals:
1. Reliability and scalability issues ----------- Scenario: Tens of thousands or hundreds of thousands of users require a reliable and scalable infrastructure Goal: Find a good reference scenario for building a reliable and scalable infrastructure of ser servers. Problems: Everybody tries to solve this their own way and most keep their solutions as a secret because it is a competitive advantage not to tell anybody.
**** I think that your solution to #1 will dominate the discussions on the issues below. Using RADIUS (and possibly LDAP back-ends) for everything but usrloc is one solution that seems to be Juha's scenario (and mine). Andreas uses mysql for subscriber info as well. Do you have one server center with load balancing or geographically-distributed server centers? It will influence your needs. So, let's sort out our scenarios before we discuss what is the "best " solution.
2. Usrloc replication across standalone ser servers. ------------ Scenario: Independent servers with independent databases run either with some sort of load balancing or DNS SRV. Goal: Make sure that all ser servers have updated usrloc information, so each can handle any SIP message. Problems: Distribute REGISTER messages to all servers; Make sure that server unavailability does not corrupt the usrloc DB state
*** We all have this issue. It is my understanding that t_replicate: a) uses SIP messages b) uses a best-effort algorithm (haven't looked at the code...) c) can be used between several servers, but when you introduce a new server, you need to change each server's ser.cfg My suggestion for a simple solution based on the discussion so far: Extend t_replicate with a guaranteed mode of replication. mysql can be used as a queue with replication states (or even a text-file for that sake). Whether SIP messages are used or TCP/IP-based FIFO is really based on an estimation of network traffic. Result: The least work and the code is an integrated part of ser.
3. Network-based provisioning of new users, aliases, etc ------------ Scenario: One server need to be provisioned from a web server or process running on a remote server Goal: Allow ser to receive TCP/IP based provisioning messages Problems: ser's FIFO does not have a TCP/IP interface
*** I think this is an extension to ser that would benefit many people. I also believe that a provisioning interface should be SOAP based due to share number of projects that probably will use the interface for provisioning.
4. Replication of user database, aliases, etc across standalone ser servers. ------------ Scenario: Independent servers with independent databases run either with some sort of load balancing or DNS SRV and subscriber information is stored in sql tables Goal: Make sure that each server recognizes all subscribers, aliases, etc Problems: Make sure that all servers have updated database tables
*** RADIUS/LDAP solutions do not need to do this as RADIUS servers, LDAP replication etc take care of both reliability and scalability. However, I think ser support more than one RADIUS server. A defined secondary server would be useful. With SQL-based scenarios however, I see three natural solutions: a) Rely on sql-based replication. Without checking this, I believe ser always write such FIFO commands directly to the DB, so sql-level replication should work b) Extend ser's FIFO to also have a replication configuration, i.e. in ser.cfg you define the peer servers that need replication. If the extension to t_replicate uses TCP/IP based FIFO, the code can be re-used. c) Implement provisioning systems so that each ser server is updated through the TCP/IP-based FIFO
To be honest, I'm not sure if I see the value of such an effort (b). Also, as usage of sql for storage is just one of several modes, it is probably not right to integrate such code into FIFO. a) and b) are more natural choices.
--------------------------------------------------------------------------
My summary and conclusions: - I believe a TCP/IP-based FIFO (#3) is a core feature that we all can agree would be useful and natural to implement; - I don't know the details of how t_replicate functions, but Juha's opinion is that it takes care of all the issues Andreas points out except one: The amount of traffic SIP messages create. I will not interfere with this discussion, of course, if t_replicate can handle unavailable servers etc, that would be great. Anyway, a reliable replication of usrloc is essential to a carrier-grade architecture - After this discussion, I now believe we should keep provisioning (#3) and the two types of replication (#2 and #4) separate also in implementation.
Well, my attempt at sorting out issues. Any succes, you think? ;-)
g-)
Andreas Granig wrote:
Juha Heinanen wrote:
you can have any number of proxies participating in replication.
What method are you thinking of? t_replicate() reports
ERROR: t_newtran: transaction already in process 0x4054d5ec
if you call it twice, like
t_replicate("foohost", "5060"); t_replicate("barhost", "5060");
Or do you mean something like
forward_tcp("foohost", "5060"); forward_tcp("barhost", "5060");
and on the receiving hosts
if(/* register from replicating host */) save_noreply("location");
which would be a possibility, indeed...
Beside that the domain tables (location etc) get out of synch if
one of > the SERs is down for a moment, because retransmission is only tried a > few times.
i don't see why this needs to be the case with db mode 2. when ser comes back up, it updates its location table from database.
I think mode 1 (Write-Through) should be used because the SER could start up while some of the contacts aren't flushed to DB yet.
However, how would you set up your database connections here? Using a common usrloc database for all hosts (-> single point of failure)? This is the main point. _How_ do you share the contacts as reliable as possible so that a host can go down for a while without getting out of synch regarding the contacts?
Andy
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Hi all!
Well, maybe I don't know what I am talking about, I just saw now this message, don't know the entire discussion, so anyway, here it goes.
I am working on implementing 3GPP's IMS using ser and by doing this I am implementing the 3GPP AAA scenarios, which, as I understood, will evolve into IETF's "Diameter SIP authentication" (now is a Draft) http://www.potaroo.net/ietf/idref/draft-ietf-aaa-diameter-sip-app/ .
Usrloc, although I use it locally is not anymore needed on a large distributed scale because the AAA servers keep track of where the users are registered and you can query them by standard request (Location Information Request, User Authorization Request) to find out where to forward messages.
1. more users, more servers. I think this should be the way to do it. I never liked replication because it's kind of a too complicated solution to work reliably. I really think that the Diameter SIP auth will scale very well. Keep in mind that it was designed to work in large mobile networks (3G, right?). The network should be splitted into different functional sip proxies, and in this case: - interogating(finds where a user is/should be registered and forwards the msg there) - serving (actually services the user).
2.the replication problem has just moved from ser to the AAA server. But the AAA server can be distributed so that each one will hold a limited number of users, and a special AAA routing node can be used to route requests toward the right AAA, which will respond to them. The AAA server holding users could be a point of failure so maybe you will still need to duplicate it's functionality but I think it's better to do a replication just to ensure availability/fail-safety then to do it in order to implement functionality. At SIP level the distribution can be done using the interogating nodes, first to redirect REGISTERs in load balancing if you want, to different ser servers; and then the users can be found by interogating the AAA servers which will track the users registration status.
3. the provisioning needs to be done in the AAA server only and all the ser servers in the farm will obey. I would suggest a "farm" architecture for ser, where all nodes are configured the same and then you could dinamically introduce or take back some such servers depending on load. All this would be done by provisioning on the AAA.
4. Actually 1. responded to this. Load balancing is controled in interogating sip-proxies and in AAA servers and not by "patches" like DNS SRV (please don't flame me on this one ;-) ).
I, personaly consider that ser should be limited to what it does best - SIP - and other components should do provisioning, replication and user tracking. Anyway, user location is more of a AAA job, don't you think? This is my personal opinion and I might be wrong :) . It is based on the patterns of the 3GPP and diameter-sip-app. Although I have reached the testing phase and it works, I have not yet performed large stress tests to see how it works because my target is to build a working 3GPP IMS, not a diameter-sip application as the draft says. So if there is interest for this, please let me know.
Dragos
Greger V. Teigre wrote:
Let me try to sort out the issues we are discussing here, so we at least can see if we agree to the goals:
- Reliability and scalability issues
Scenario: Tens of thousands or hundreds of thousands of users require a reliable and scalable infrastructure Goal: Find a good reference scenario for building a reliable and scalable infrastructure of ser servers. Problems: Everybody tries to solve this their own way and most keep their solutions as a secret because it is a competitive advantage not to tell anybody.
**** I think that your solution to #1 will dominate the discussions on the issues below. Using RADIUS (and possibly LDAP back-ends) for everything but usrloc is one solution that seems to be Juha's scenario (and mine). Andreas uses mysql for subscriber info as well. Do you have one server center with load balancing or geographically-distributed server centers? It will influence your needs. So, let's sort out our scenarios before we discuss what is the "best " solution.
- Usrloc replication across standalone ser servers.
Scenario: Independent servers with independent databases run either with some sort of load balancing or DNS SRV. Goal: Make sure that all ser servers have updated usrloc information, so each can handle any SIP message. Problems: Distribute REGISTER messages to all servers; Make sure that server unavailability does not corrupt the usrloc DB state
*** We all have this issue. It is my understanding that t_replicate: a) uses SIP messages b) uses a best-effort algorithm (haven't looked at the code...) c) can be used between several servers, but when you introduce a new server, you need to change each server's ser.cfg My suggestion for a simple solution based on the discussion so far: Extend t_replicate with a guaranteed mode of replication. mysql can be used as a queue with replication states (or even a text-file for that sake). Whether SIP messages are used or TCP/IP-based FIFO is really based on an estimation of network traffic. Result: The least work and the code is an integrated part of ser.
- Network-based provisioning of new users, aliases, etc
Scenario: One server need to be provisioned from a web server or process running on a remote server Goal: Allow ser to receive TCP/IP based provisioning messages Problems: ser's FIFO does not have a TCP/IP interface
*** I think this is an extension to ser that would benefit many people. I also believe that a provisioning interface should be SOAP based due to share number of projects that probably will use the interface for provisioning.
- Replication of user database, aliases, etc across standalone ser
servers.
Scenario: Independent servers with independent databases run either with some sort of load balancing or DNS SRV and subscriber information is stored in sql tables Goal: Make sure that each server recognizes all subscribers, aliases, etc Problems: Make sure that all servers have updated database tables
*** RADIUS/LDAP solutions do not need to do this as RADIUS servers, LDAP replication etc take care of both reliability and scalability. However, I think ser support more than one RADIUS server. A defined secondary server would be useful. With SQL-based scenarios however, I see three natural solutions: a) Rely on sql-based replication. Without checking this, I believe ser always write such FIFO commands directly to the DB, so sql-level replication should work b) Extend ser's FIFO to also have a replication configuration, i.e. in ser.cfg you define the peer servers that need replication. If the extension to t_replicate uses TCP/IP based FIFO, the code can be re-used. c) Implement provisioning systems so that each ser server is updated through the TCP/IP-based FIFO
To be honest, I'm not sure if I see the value of such an effort (b). Also, as usage of sql for storage is just one of several modes, it is probably not right to integrate such code into FIFO. a) and b) are more natural choices.
My summary and conclusions:
- I believe a TCP/IP-based FIFO (#3) is a core feature that we all can
agree would be useful and natural to implement;
- I don't know the details of how t_replicate functions, but Juha's
opinion is that it takes care of all the issues Andreas points out except one: The amount of traffic SIP messages create. I will not interfere with this discussion, of course, if t_replicate can handle unavailable servers etc, that would be great. Anyway, a reliable replication of usrloc is essential to a carrier-grade architecture
- After this discussion, I now believe we should keep provisioning
(#3) and the two types of replication (#2 and #4) separate also in implementation.
Well, my attempt at sorting out issues. Any succes, you think? ;-)
g-)
Andreas Granig wrote:
Juha Heinanen wrote:
you can have any number of proxies participating in replication.
What method are you thinking of? t_replicate() reports
ERROR: t_newtran: transaction already in process 0x4054d5ec
if you call it twice, like
t_replicate("foohost", "5060"); t_replicate("barhost", "5060");
Or do you mean something like
forward_tcp("foohost", "5060"); forward_tcp("barhost", "5060");
and on the receiving hosts
if(/* register from replicating host */) save_noreply("location");
which would be a possibility, indeed...
Beside that the domain tables (location etc) get out of synch if
one of > the SERs is down for a moment, because retransmission is only tried a > few times.
i don't see why this needs to be the case with db mode 2. when ser comes back up, it updates its location table from database.
I think mode 1 (Write-Through) should be used because the SER could start up while some of the contacts aren't flushed to DB yet.
However, how would you set up your database connections here? Using a common usrloc database for all hosts (-> single point of failure)? This is the main point. _How_ do you share the contacts as reliable as possible so that a host can go down for a while without getting out of synch regarding the contacts?
Andy
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Hi Dragos!
- Actually 1. responded to this. Load balancing is controled in
interogating sip-proxies and in AAA servers and not by "patches" like DNS SRV (please don't flame me on this one ;-) ).
How should a SIP client detect and choose one of the SIP-proxies without using SRV?
regards, klaus
Hi Klaus,
If you refer to origination messages, then by doing a UAR (User-Authorization-Request/Answer)for the originating user to the AAA server. This will offer a way to ballance user registrations between serving sers. If you refer to termination messages, by doing a LIR (Location-Information-Request/Answer) for the terminating user to the AAA server. This will find where the destination users are located.
UA->interog.ser->serv.ser->inter.ser->serv.ser->UA UAR^ ^LIR | | +----> AAA server <---+
If you refer to forwarding to new networks or entering the home network, you are right, SRV should be used. Sorry, I have not made it clear enough - I was talking about load balancing and stuff JUST inside the home network. So finding an interogating node, when no AAA server for that domain could be contacted, remains the same. But it should be more comfortable, because interogating nodes are fully stateless and should work very fast.
Dragos
Klaus Darilion wrote:
Hi Dragos!
- Actually 1. responded to this. Load balancing is controled in
interogating sip-proxies and in AAA servers and not by "patches" like DNS SRV (please don't flame me on this one ;-) ).
How should a SIP client detect and choose one of the SIP-proxies without using SRV?
regards, klaus
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Dragos,
I completely agree with you. When building architectures for scalability and redundancy, the correct match of technologies will often solve many of your problems. And it does make sense to store as much subscriber information back-end as possible and let proven and standardized(?) systems handle replication etc. In my summary of issues I tried to separate the various issues from eachother without going as far as you do. Dependent on where you come from, various combinations will look appealing.
I have followed Diameter from a distance for a while, and while it does seem to take hold in the mobile world, I haven't seen many non-mobile implementations (it may be my limited perspective...). Most legacy subcriber databases have a RADIUS interface, but rarely Diameter. In fact, I know that Siemens have implemented a Mobile RADIUS with location-awareness in RADIUS. I think that is an example of the (still) appeal of RADIUS. I would love to have Diameter capabilities, but deploying Diameter in my organization is not really feasible at this point. Hence, we try to do as much as possible with Juha's RADIUS modules. And here is the point, I think: One thing is what we would like to have, another thing is the context we work in and thus what we have. I have RADIUS and do all provisioning in an LDAP back-end, other people use mysql also for the subscriber database (which BTW is simple as serweb has been developed and it works).
The fact that you (at Fraunhofer) spend time on Diameter and SIP is only a confirmation of how "cutting-edge" such an approach is. (Well, with legacy systems, things do not have to be very cutting-edge to be very far away from implementation...) Secondly, Diameter is the successor of RADIUS, which is a very "simple" technology compared to Diameter. Just as with our xmlrpc vs soap discussion, you gain complexity when you get functionality and for installation without the need for a certain functionality will only get complexity; and complexity = $$$.
And do I think keeping location is a natural function of AAA? Hm, traditionally subscriber databases kept all the static information about the subscriber and the subscriber's services. Location was application specific. In the mobile world, location is key to all services, so HLR (Home Location Registers) had location as a natural part of its functionality. For old/traditional dial-up services, location was more an attribute of the AAA request, so that authorization could be determined based on when and where the subscriber tried to get access. Location keeps getting more and more important in TCP/IP based services as well, so one may (successfully) argue that location is a part of the AAA domain. However, where do you stop? Is the capabilities of your end-user device also information that should be in the AAA domain? You need to know these capabilities in order to provide quality services... One inherent problem of using AAA for dynamic data is that back-end technologies that are very well adapted to AAA like LDAP (made for search speed) have a real problem with frequent writes. Of course, if you use Oracle or some big SQL server as a back-end, you have solved that problem, but you cannot utilize from LDAP capabilities...
Enough of this ranting... What I'm trying to say is that we need that "piece-meal" approach to scalability and redundancy: What can we do with existing installations today that gives us what we need and still leads us in the right direction? So, in order to better understand where we should be going, I would love to look at what you are doing 3GPP IMS. ;-) If you have some more information, why don't you post a link?
g-)
Dragos Vingarzan wrote:
Hi all!
Well, maybe I don't know what I am talking about, I just saw now this message, don't know the entire discussion, so anyway, here it goes.
I am working on implementing 3GPP's IMS using ser and by doing this I am implementing the 3GPP AAA scenarios, which, as I understood, will evolve into IETF's "Diameter SIP authentication" (now is a Draft) http://www.potaroo.net/ietf/idref/draft-ietf-aaa-diameter-sip-app/ .
Usrloc, although I use it locally is not anymore needed on a large distributed scale because the AAA servers keep track of where the users are registered and you can query them by standard request (Location Information Request, User Authorization Request) to find out where to forward messages.
- more users, more servers. I think this should be the way to do it.
I never liked replication because it's kind of a too complicated solution to work reliably. I really think that the Diameter SIP auth will scale very well. Keep in mind that it was designed to work in large mobile networks (3G, right?). The network should be splitted into different functional sip proxies, and in this case:
- interogating(finds where a user is/should be registered and forwards
the msg there)
- serving (actually services the user).
2.the replication problem has just moved from ser to the AAA server. But the AAA server can be distributed so that each one will hold a limited number of users, and a special AAA routing node can be used to route requests toward the right AAA, which will respond to them. The AAA server holding users could be a point of failure so maybe you will still need to duplicate it's functionality but I think it's better to do a replication just to ensure availability/fail-safety then to do it in order to implement functionality. At SIP level the distribution can be done using the interogating nodes, first to redirect REGISTERs in load balancing if you want, to different ser servers; and then the users can be found by interogating the AAA servers which will track the users registration status. 3. the provisioning needs to be done in the AAA server only and all the ser servers in the farm will obey. I would suggest a "farm" architecture for ser, where all nodes are configured the same and then you could dinamically introduce or take back some such servers depending on load. All this would be done by provisioning on the AAA.
- Actually 1. responded to this. Load balancing is controled in
interogating sip-proxies and in AAA servers and not by "patches" like DNS SRV (please don't flame me on this one ;-) ).
I, personaly consider that ser should be limited to what it does best
- SIP - and other components should do provisioning, replication and
user tracking. Anyway, user location is more of a AAA job, don't you think? This is my personal opinion and I might be wrong :) . It is based on the patterns of the 3GPP and diameter-sip-app. Although I have reached the testing phase and it works, I have not yet performed large stress tests to see how it works because my target is to build a working 3GPP IMS, not a diameter-sip application as the draft says. So if there is interest for this, please let me know.
Dragos
Greger V. Teigre wrote:
Let me try to sort out the issues we are discussing here, so we at least can see if we agree to the goals:
- Reliability and scalability issues
Scenario: Tens of thousands or hundreds of thousands of users require a reliable and scalable infrastructure Goal: Find a good reference scenario for building a reliable and scalable infrastructure of ser servers. Problems: Everybody tries to solve this their own way and most keep their solutions as a secret because it is a competitive advantage not to tell anybody.
**** I think that your solution to #1 will dominate the discussions on the issues below. Using RADIUS (and possibly LDAP back-ends) for everything but usrloc is one solution that seems to be Juha's scenario (and mine). Andreas uses mysql for subscriber info as well. Do you have one server center with load balancing or geographically-distributed server centers? It will influence your needs. So, let's sort out our scenarios before we discuss what is the "best " solution.
- Usrloc replication across standalone ser servers.
Scenario: Independent servers with independent databases run either with some sort of load balancing or DNS SRV. Goal: Make sure that all ser servers have updated usrloc information, so each can handle any SIP message. Problems: Distribute REGISTER messages to all servers; Make sure that server unavailability does not corrupt the usrloc DB state
*** We all have this issue. It is my understanding that t_replicate: a) uses SIP messages b) uses a best-effort algorithm (haven't looked at the code...) c) can be used between several servers, but when you introduce a new server, you need to change each server's ser.cfg My suggestion for a simple solution based on the discussion so far: Extend t_replicate with a guaranteed mode of replication. mysql can be used as a queue with replication states (or even a text-file for that sake). Whether SIP messages are used or TCP/IP-based FIFO is really based on an estimation of network traffic. Result: The least work and the code is an integrated part of ser.
- Network-based provisioning of new users, aliases, etc
Scenario: One server need to be provisioned from a web server or process running on a remote server Goal: Allow ser to receive TCP/IP based provisioning messages Problems: ser's FIFO does not have a TCP/IP interface
*** I think this is an extension to ser that would benefit many people. I also believe that a provisioning interface should be SOAP based due to share number of projects that probably will use the interface for provisioning.
- Replication of user database, aliases, etc across standalone ser
servers.
Scenario: Independent servers with independent databases run either with some sort of load balancing or DNS SRV and subscriber information is stored in sql tables Goal: Make sure that each server recognizes all subscribers, aliases, etc Problems: Make sure that all servers have updated database tables *** RADIUS/LDAP solutions do not need to do this as RADIUS servers, LDAP replication etc take care of both reliability and scalability. However, I think ser support more than one RADIUS server. A defined secondary server would be useful. With SQL-based scenarios however, I see three natural solutions: a) Rely on sql-based replication. Without checking this, I believe ser always write such FIFO commands directly to the DB, so sql-level replication should work b) Extend ser's FIFO to also have a replication configuration, i.e. in ser.cfg you define the peer servers that need replication. If the extension to t_replicate uses TCP/IP based FIFO, the code can be re-used. c) Implement provisioning systems so that each ser server is updated through the TCP/IP-based FIFO
To be honest, I'm not sure if I see the value of such an effort (b). Also, as usage of sql for storage is just one of several modes, it is probably not right to integrate such code into FIFO. a) and b) are more natural choices.
My summary and conclusions:
- I believe a TCP/IP-based FIFO (#3) is a core feature that we all
can agree would be useful and natural to implement;
- I don't know the details of how t_replicate functions, but Juha's
opinion is that it takes care of all the issues Andreas points out except one: The amount of traffic SIP messages create. I will not interfere with this discussion, of course, if t_replicate can handle unavailable servers etc, that would be great. Anyway, a reliable replication of usrloc is essential to a carrier-grade architecture
- After this discussion, I now believe we should keep provisioning
(#3) and the two types of replication (#2 and #4) separate also in implementation.
Well, my attempt at sorting out issues. Any succes, you think? ;-)
g-)
Andreas Granig wrote:
Juha Heinanen wrote:
you can have any number of proxies participating in replication.
What method are you thinking of? t_replicate() reports
ERROR: t_newtran: transaction already in process 0x4054d5ec
if you call it twice, like
t_replicate("foohost", "5060"); t_replicate("barhost", "5060");
Or do you mean something like
forward_tcp("foohost", "5060"); forward_tcp("barhost", "5060");
and on the receiving hosts
if(/* register from replicating host */) save_noreply("location");
which would be a possibility, indeed...
Beside that the domain tables (location etc) get out of synch if
one of > the SERs is down for a moment, because retransmission is only tried a > few times.
i don't see why this needs to be the case with db mode 2. when ser comes back up, it updates its location table from database.
I think mode 1 (Write-Through) should be used because the SER could start up while some of the contacts aren't flushed to DB yet.
However, how would you set up your database connections here? Using a common usrloc database for all hosts (-> single point of failure)? This is the main point. _How_ do you share the contacts as reliable as possible so that a host can go down for a while without getting out of synch regarding the contacts?
Andy
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Hi all! Fortunately, for now, I don't have to deal with an already deployed network on wich to adapt in order to cope with growth. I was in that situation but with other technologies and it sure wasn't "funny". You will have to use proven technologies, that would not let you down or have hidden bugs. I will continue my reply inlined...
Greger V. Teigre wrote:
I have followed Diameter from a distance for a while, and while it does seem to take hold in the mobile world, I haven't seen many non-mobile implementations (it may be my limited perspective...). Most legacy subcriber databases have a RADIUS interface, but rarely Diameter. In fact, I know that Siemens have implemented a Mobile RADIUS with location-awareness in RADIUS. I think that is an example of the (still) appeal of RADIUS. I would love to have Diameter capabilities, but deploying Diameter in my organization is not really feasible at this point. Hence, we try to do as much as possible with Juha's RADIUS modules. And here is the point, I think: One thing is what we would like to have, another thing is the context we work in and thus what we have. I have RADIUS and do all provisioning in an LDAP back-end, other people use mysql also for the subscriber database (which BTW is simple as serweb has been developed and it works).
Diameter deployment will take a while: you will need tested diameter support in ser and then a diameter server that will run your application logic. The good news is that maybe you could do it gradually, with radius/diameter conversion tools. But the fact that you will still have to implement all your application logic remains and it is a hard task. And of course you will need something of industrial strength...
The fact that you (at Fraunhofer) spend time on Diameter and SIP is only a confirmation of how "cutting-edge" such an approach is. (Well, with legacy systems, things do not have to be very cutting-edge to be very far away from implementation...) Secondly, Diameter is the successor of RADIUS, which is a very "simple" technology compared to Diameter. Just as with our xmlrpc vs soap discussion, you gain complexity when you get functionality and for installation without the need for a certain functionality will only get complexity; and complexity = $$$.
What I am working on is/was part of my Diploma Thesis. I was exploring new grounds, while trying to ensure a high performance also. A lot of this technologies don't have final specifications and tend to change very fast so that a lot of the time I just rewrite obsolete code. The 3GPP's IMS is focusing on moving the mobile phones world to the IP infrastructure. This is not a "must" for providers of UMTS but more of a good thing to have for the future and for integration with the IP world. Indeed the complexity is high as you try to get functionality. But sometimes, trying to get an old technologies to perform better that it was designed to, is a bigger effort that adopting a new one. I am sure that none of you will use this very soon, or maybe this will just remain in the IMS world. But you should know that it exists and maybe then, building big "patching" solutions will not look so feasible. From my experience into this and into trying to make it work, I would say that it would take some effort for any provider to jump into it, because there are so many features that are "provider independent".
And do I think keeping location is a natural function of AAA? Hm, traditionally subscriber databases kept all the static information about the subscriber and the subscriber's services. Location was application specific. In the mobile world, location is key to all services, so HLR (Home Location Registers) had location as a natural part of its functionality. For old/traditional dial-up services, location was more an attribute of the AAA request, so that authorization could be determined based on when and where the subscriber tried to get access. Location keeps getting more and more important in TCP/IP based services as well, so one may (successfully) argue that location is a part of the AAA domain. However, where do you stop? Is the capabilities of your end-user device also information that should be in the AAA domain? You need to know these capabilities in order to provide quality services...
In IMS capabilites are stored on the AAA server and message routing is also based on that.
Enough of this ranting... What I'm trying to say is that we need that "piece-meal" approach to scalability and redundancy: What can we do with existing installations today that gives us what we need and still leads us in the right direction? So, in order to better understand where we should be going, I would love to look at what you are doing 3GPP IMS. ;-) If you have some more information, why don't you post a link?
If you would like to know more about it I would recommend to read this draft: http://www.potaroo.net/ietf/idref/draft-ietf-aaa-diameter-sip-app/ It will make you an ideea of how the architecture looks like, although it is more about diameter communication. But the architecture is described very well here. For 3GPP's IMS see http://www.3gpp.org/specs/numbering.htm into Technical Specifaction 29.228 and 29.229.
Even if the diameter thing looks a little out of reach for now, the suggested architecture of the sip servers could be usefull into destressing things a little bit.
Regards,
Greger V. Teigre writes:
*** We all have this issue. It is my understanding that t_replicate: a) uses SIP messages b) uses a best-effort algorithm (haven't looked at the code...) c) can be used between several servers, but when you introduce a new server, you need to change each server's ser.cfg
no you don't if you keep your server ip addresses in database and have a fifo command to reload them each time there is a change. i have started coding this, but is not ready yet.
My suggestion for a simple solution based on the discussion so far: Extend t_replicate with a guaranteed mode of replication.
this indeed is a good thing to do.
- Network-based provisioning of new users, aliases, etc
Scenario: One server need to be provisioned from a web server or process running on a remote server Goal: Allow ser to receive TCP/IP based provisioning messages Problems: ser's FIFO does not have a TCP/IP interface
*** I think this is an extension to ser that would benefit many people. I also believe that a provisioning interface should be SOAP based due to share number of projects that probably will use the interface for provisioning.
somebody mentioned that since ser fifo now has unix domain support, it would not be difficult to extend it to tcp support. that may be true atleast for the person who did unix domain support and thus knows the code. i guess fifo tcp support already is on iptel's long to-do list.
what comes to soap, i would prefer a solution that has an abstraction layer and backends for both soap and xmlrpc, since not everyone of fond of microsoft solutions.
-- juha
Juha Heinanen wrote:
*** I think this is an extension to ser that would benefit
many people. I
also believe that a provisioning interface should be SOAP based due to share number of projects that probably will use the interface for provisioning.
what comes to soap, i would prefer a solution that has an abstraction layer and backends for both soap and xmlrpc, since not everyone of fond of microsoft solutions.
-- juha
I think that using soap is a good idea and it DOES NOT necessarily mean that you have to be fond of microsoft. Successfuly deployed many nice things using SER & open source soap, I don't think it is slow or complicated.
Arek,
Juha Heinanen wrote:
Greger V. Teigre writes:
*** We all have this issue. It is my understanding that t_replicate: a) uses SIP messages b) uses a best-effort algorithm (haven't looked at the code...) c) can be used between several servers, but when you introduce a new server, you need to change each server's ser.cfg
no you don't if you keep your server ip addresses in database and have a fifo command to reload them each time there is a change. i have started coding this, but is not ready yet.
Excellent! Will this code be available through CVS in the near future?
My suggestion for a simple solution based on the discussion so far: Extend t_replicate with a guaranteed mode of replication.
this indeed is a good thing to do.
Who is code maintainer for this part??
somebody mentioned that since ser fifo now has unix domain support, it would not be difficult to extend it to tcp support. that may be true atleast for the person who did unix domain support and thus knows the code. i guess fifo tcp support already is on iptel's long to-do list.
what comes to soap, i would prefer a solution that has an abstraction layer and backends for both soap and xmlrpc, since not everyone of fond of microsoft solutions.
I don't see soap as a microsoft solution, it's a bit bigger than that... ;-) but I agree with you. It depends on your needs. g-)
Andreas Granig writes:
Or do you mean something like
forward_tcp("foohost", "5060"); forward_tcp("barhost", "5060");
and on the receiving hosts
if(/* register from replicating host */) save_noreply("location");
which would be a possibility, indeed...
this is what i meant.
I think mode 1 (Write-Through) should be used because the SER could start up while some of the contacts aren't flushed to DB yet.
yes, my error. i didn't remember which mode number was write-through.
However, how would you set up your database connections here? Using a common usrloc database for all hosts (-> single point of failure)? This is the main point. _How_ do you share the contacts as reliable as possible so that a host can go down for a while without getting out of synch regarding the contacts?
that is database replication problem. i'm pretty sure that mysql ab has a solution for it if it is not included in the freely available 4.1 release.
-- juha
Andreas Granig wrote:
Juha Heinanen wrote:
you can have any number of proxies participating in replication.
What method are you thinking of? t_replicate() reports
ERROR: t_newtran: transaction already in process 0x4054d5ec
if you call it twice, like
t_replicate("foohost", "5060"); t_replicate("barhost", "5060");
Or do you mean something like
forward_tcp("foohost", "5060"); forward_tcp("barhost", "5060");
and on the receiving hosts
if(/* register from replicating host */) save_noreply("location");
Have you tried append_branch?
append_branch("uri1"); append_branch("uri2"); t_replicate("barhost", "5060");
maybe this works. (never tried it myself)
regards, klaus
Do you plan to send the xmlrpc messages for each REGISTER or will you do some buffering and send several location update in one message? I think this will cause a lott of traffic between the ser's.
Do you know the diploma thesis of Nils? (http://www.ohlmeier.de/tuberlin.html you will find it at the bottom of the page) There are several ideas how to distribute the location data.
regards, klaus
Andreas Granig wrote:
Juha Heinanen wrote:
i don't know about andrea's efforts, but i used xmlrpc for registering ser users' phone numbers with tier-1 enum database in finland. finnish authorities had specified xmlrpc interface for that purpose.
In short, I plan to write an xmlrpc-based provisioning system for distributing FIFO commands. The main focus is sharing registration contacts and aliases among different SERs (all the data that can't be replicated on a lower (database) layer due to memory caching).
There's an external application on each SER host receiving contacts from a SER module via a local socket. These contacts are wrapped into xmlrpc (in fact the contact is converted into a FIFO command which is wrapped into xmlrpc) and are distributed to the same application on other SER hosts, which writes them into the SER FIFO. If an application is unreachable or reports an error, the distributed FIFO command is queued.
The application can receive arbitrary FIFO commands, and a flag indicates if the command should be redistributed to the other hosts.
Andy
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Greger and everyone else that is interested,
Please consider before ruling out SOAP, that SOAP has more off the shelf libraries to support it then XMLRPC. Please consider the folks that use Microsoft platforms for their back end processing and databases, and keep in mind the following:
Yukon is just around the corner. It has SOAP services built in, as well as the ability to call SOAP services directly from T-SQL.
Also we implement a carrier grade platform using SER, which is in use by a number of VoIP providers here, with the following extensions:
1) An extra module which allows for RADIUS URI translation, extended AVP lookup, via extra string parameter which lets you identify what AVP query you wish to do, and an extra flag in the registration database "FOREIGN" registration, to identify a contact which has been replicated from another SER server.
2) A service which speaks SOAP to he outside world, (it's own http server on non-standard port) to allow an external interface to the SER FIFO interface. It use is for external Voicemail MWI Notifies, and to send refresh, reboot and report notify messages to SIP devices.
3) A generic provisioning server for almost any SIP device, which can be provisioned via TFTP, HTTP, or HTTPS. This server dynamically builds configuration files in memory on the fly for any device based on RE pattern matching of the filename, mapped to SQL statement, which returns device parameters.
With just these above three things, we can implement a full carrier grade system, with full automated device provisioning, all CLASS 5 features, such as unlimited level hunting, recursive call forwarding (even when each device in the forwarding has a different dial plan) and just about anything else you can think of. To accomplish this, we depend on SOAP as a method of component communication because we consider any platform, including Microsoft and the ".NET framework" as things we need to interact with.
If your goal is to provide a framework for integrating with other platforms, SOAP bring a lot more flexibility to the game, and make it more compatible with more platforms.
Remember, this is just an opinion, however it needed to be expressed, just so you know what other folks are doing with SER.
--Dave
-----Original Message----- From: serusers-bounces@iptel.org [mailto:serusers-bounces@lists.iptel.org] On Behalf Of Greger V. Teigre Sent: Monday, January 24, 2005 11:29 PM To: Juha Heinanen Cc: serusers@lists.iptel.org Subject: Re: [Serusers] Carrier-grade framework for SER
Juha, Yes, I completely agree with you. However, I don't need to read the spec and far from understand it before I use it... ;-) So I did start to look at
SOAP and have very good experiences, both in terms of usability and scalability. But, I don't have strong opinions, if the people who are going to use the interface are all against SOAP, XMLRPC is the right choice.
The xmlrpc-provisioning work you have done, can it be coordinated with Andreas' effort? g-)
Juha Heinanen wrote:
Greger V. Teigre writes:
As I indicated in an earlier email, I would be interested in taking part in a joint effort to further develop ser's high-availability and scalability (HAS). I would probably have to do some development anyway, and I would prefer to see such support in the public domain. In Nov/Dec I called for responses on a SOAP-based provisioning interface, but heard nothing, so here is an overlap of interests.
greger,
we have done some work on xmlprc based provisioning and it looks promising. xmlrpc spec is three pages long and even i can understand it. soap spec, on the other hand, is far too thick and goes way above my head.
-- juha
_______________________________________________ Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
I'm digging through old archives and I am just wondering how people feel about 18 month later about the discussion about SOAP, XMl-RPC, etc. Any feedback would be appreciated -- what you think about it now and more importantly what's your experience if any. All in all, many are asking for a roadmap and input to that is most welcome.
-jiri
I have an opinion on this topic too but didn't want to begin egoistically with mine :-)
At 12:48 25/01/2005, David R. Kompel wrote:
Greger and everyone else that is interested,
Please consider before ruling out SOAP, that SOAP has more off the shelf libraries to support it then XMLRPC. Please consider the folks that use Microsoft platforms for their back end processing and databases, and keep in mind the following:
Yukon is just around the corner. It has SOAP services built in, as well as the ability to call SOAP services directly from T-SQL.
Also we implement a carrier grade platform using SER, which is in use by a number of VoIP providers here, with the following extensions:
- An extra module which allows for RADIUS URI translation, extended AVP
lookup, via extra string parameter which lets you identify what AVP query you wish to do, and an extra flag in the registration database "FOREIGN" registration, to identify a contact which has been replicated from another SER server.
- A service which speaks SOAP to he outside world, (it's own http
server on non-standard port) to allow an external interface to the SER FIFO interface. It use is for external Voicemail MWI Notifies, and to send refresh, reboot and report notify messages to SIP devices.
- A generic provisioning server for almost any SIP device, which can be
provisioned via TFTP, HTTP, or HTTPS. This server dynamically builds configuration files in memory on the fly for any device based on RE pattern matching of the filename, mapped to SQL statement, which returns device parameters.
With just these above three things, we can implement a full carrier grade system, with full automated device provisioning, all CLASS 5 features, such as unlimited level hunting, recursive call forwarding (even when each device in the forwarding has a different dial plan) and just about anything else you can think of. To accomplish this, we depend on SOAP as a method of component communication because we consider any platform, including Microsoft and the ".NET framework" as things we need to interact with.
If your goal is to provide a framework for integrating with other platforms, SOAP bring a lot more flexibility to the game, and make it more compatible with more platforms.
Remember, this is just an opinion, however it needed to be expressed, just so you know what other folks are doing with SER.
--Dave
-----Original Message----- From: serusers-bounces@iptel.org [mailto:serusers-bounces@iptel.org] On Behalf Of Greger V. Teigre Sent: Monday, January 24, 2005 11:29 PM To: Juha Heinanen Cc: serusers@iptel.org Subject: Re: [Serusers] Carrier-grade framework for SER
Juha, Yes, I completely agree with you. However, I don't need to read the spec and far from understand it before I use it... ;-) So I did start to look at
SOAP and have very good experiences, both in terms of usability and scalability. But, I don't have strong opinions, if the people who are going to use the interface are all against SOAP, XMLRPC is the right choice.
The xmlrpc-provisioning work you have done, can it be coordinated with Andreas' effort? g-)
Juha Heinanen wrote:
Greger V. Teigre writes:
As I indicated in an earlier email, I would be interested in taking part in a joint effort to further develop ser's high-availability and scalability (HAS). I would probably have to do some development anyway, and I would prefer to see such support in the public domain. In Nov/Dec I called for responses on a SOAP-based provisioning interface, but heard nothing, so here is an overlap of interests.
greger,
we have done some work on xmlprc based provisioning and it looks promising. xmlrpc spec is three pages long and even i can understand it. soap spec, on the other hand, is far too thick and goes way above my head.
-- juha
Serusers mailing list Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
Serusers mailing list Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
-- Jiri Kuthan http://iptel.org/~jiri/
Well, I think that integration around Web services in inevitable. And I think not about writing-just-another-SER-module, but some global policy towards Web Services.
If not SER globally, then at least some of SER user installations out there will have more or less proprietary SOAP speaking modules. Because all of us folks need sometimes to connect to billing systems, ERPs, Web portals and so on. And some of us already do it with Web services, but they just don't have time to speak about it ;-)
Our intention should be to spare all those people the need to develop their own Web services (SOAP) modules and do a consistent review of SER policy towards Web Services:
a) I think that we should at LEAST introduce SOAP for accounting. In emails sometime ago I proposed to build into SER a set of Web Services compliant to IPDR standard. Then we could at least facilitate simple requests for list of calls per user/domain/group, together with relevant call times. I have nothing against other billing information exchange standards of course, but I still think that IPDR is sexy.
b) Next we could build something around dialog module. Web methods to report ongoing calls, to terminate a call, to insert voice prompt into an ongoing call.. Options are endless here.
c) subscriber management - I think it is self descriptive
d) device provisioning, as I think Dave mentioned it - it is another good idea
e) ParlayX Web Services v.2.0 - for building all of those sexy call services and advanced call routing, reaching further that current dialog module capabilities
What is more, having SOAP (and Web Services in general) in mind, the next generation Web interface is just around the corner. SER with built-in Web Services, together with nice interface build in PHP/XSLT, OpenLaszlo or whatever-is-the-sexiest-now Web technology, could easily complement or even compete with Asterisk.
-- Regards, Arek Bekiersz
Jiri Kuthan wrote:
I'm digging through old archives and I am just wondering how people feel about 18 month later about the discussion about SOAP, XMl-RPC, etc. Any feedback would be appreciated -- what you think about it now and more importantly what's your experience if any. All in all, many are asking for a roadmap and input to that is most welcome.
-jiri
I have an opinion on this topic too but didn't want to begin egoistically with mine :-)
To some extent I agree, I am not sure it is most beneficial to embedd this logic directly in SER. -jiri
At 14:58 29/07/2006, Arek Bekiersz wrote:
Well, I think that integration around Web services in inevitable. And I think not about writing-just-another-SER-module, but some global policy towards Web Services.
If not SER globally, then at least some of SER user installations out there will have more or less proprietary SOAP speaking modules. Because all of us folks need sometimes to connect to billing systems, ERPs, Web portals and so on. And some of us already do it with Web services, but they just don't have time to speak about it ;-)
Our intention should be to spare all those people the need to develop their own Web services (SOAP) modules and do a consistent review of SER policy towards Web Services:
a) I think that we should at LEAST introduce SOAP for accounting. In emails sometime ago I proposed to build into SER a set of Web Services compliant to IPDR standard. Then we could at least facilitate simple requests for list of calls per user/domain/group, together with relevant call times. I have nothing against other billing information exchange standards of course, but I still think that IPDR is sexy.
b) Next we could build something around dialog module. Web methods to report ongoing calls, to terminate a call, to insert voice prompt into an ongoing call.. Options are endless here.
c) subscriber management - I think it is self descriptive
d) device provisioning, as I think Dave mentioned it - it is another good idea
e) ParlayX Web Services v.2.0 - for building all of those sexy call services and advanced call routing, reaching further that current dialog module capabilities
What is more, having SOAP (and Web Services in general) in mind, the next generation Web interface is just around the corner. SER with built-in Web Services, together with nice interface build in PHP/XSLT, OpenLaszlo or whatever-is-the-sexiest-now Web technology, could easily complement or even compete with Asterisk.
-- Regards, Arek Bekiersz
Jiri Kuthan wrote:
I'm digging through old archives and I am just wondering how people feel about 18 month later about the discussion about SOAP, XMl-RPC, etc. Any feedback would be appreciated -- what you think about it now and more importantly what's your experience if any. All in all, many are asking for a roadmap and input to that is most welcome.
-jiri I have an opinion on this topic too but didn't want to begin egoistically with mine :-)
Serusers mailing list Serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
-- Jiri Kuthan http://iptel.org/~jiri/
I'll weigh in.
I'd prefer SOAP. I know you're gung-ho on XML-RPC, and I can understand that its simplicity would make it useful in a lot of ways, but for the most part, the XML-RPC libraries I've seen for Java, C++, and PHP have all been a bit of a mess (Marquee/Redstone isn't TOO bad, but it's still rather poorly documented overall). SOAP, while being more complex, is a bit more powerful as well, allowing me to define the data types I'm using and accepting, and how to use them, as well as allowing a great deal more flexibility when it comes to specifying the characters sets I'm looking for and the like.
On the flip side, SOAP is still a growing standard. Some people will be turned off by the fact that it is almost certain to change as time goes on, but for any of us who got in on the ground floor of the Java spec, or PHP, or half a dozen other languages out there that change their APIs on a semi-regular basis, a spec or protocol that changes is hardly daunting.
We've used SOAP for a lot of things to connect to various services, and most of the telco services we interconnect with use SOAP and not XML-RPC. That would mean we'd end up having to do things two ways for all our apps and server side code, but if it has to be done, it has to be done.
It reall boils down to why do you like XML-RPC and what do you see its strengths are in terms of SER? If there's good reason to use one over another, even given the limitations, then I don't see a problem with it. As long as it's all well-documented with example code and such, it should be just fine.
Again, it boils down to the fact that good documentation goes a long way toward making even the most intractable projects into workable solutions.
N.
On Sat, 29 Jul 2006 11:05:24 +0200, Jiri Kuthan wrote
I'm digging through old archives and I am just wondering how people feel about 18 month later about the discussion about SOAP, XMl-RPC, etc. Any feedback would be appreciated -- what you think about it now and more importantly what's your experience if any. All in all, many are asking for a roadmap and input to that is most welcome.
-jiri
I have an opinion on this topic too but didn't want to begin egoistically with mine :-)
At 12:48 25/01/2005, David R. Kompel wrote:
Greger and everyone else that is interested,
Please consider before ruling out SOAP, that SOAP has more off the shelf libraries to support it then XMLRPC. Please consider the folks that use Microsoft platforms for their back end processing and databases, and keep in mind the following:
Yukon is just around the corner. It has SOAP services built in, as well as the ability to call SOAP services directly from T-SQL.
Also we implement a carrier grade platform using SER, which is in use by a number of VoIP providers here, with the following extensions:
- An extra module which allows for RADIUS URI translation, extended AVP
lookup, via extra string parameter which lets you identify what AVP query you wish to do, and an extra flag in the registration database "FOREIGN" registration, to identify a contact which has been replicated from another SER server.
- A service which speaks SOAP to he outside world, (it's own http
server on non-standard port) to allow an external interface to the SER FIFO interface. It use is for external Voicemail MWI Notifies, and to send refresh, reboot and report notify messages to SIP devices.
- A generic provisioning server for almost any SIP device, which can be
provisioned via TFTP, HTTP, or HTTPS. This server dynamically builds configuration files in memory on the fly for any device based on RE pattern matching of the filename, mapped to SQL statement, which returns device parameters.
With just these above three things, we can implement a full carrier grade system, with full automated device provisioning, all CLASS 5 features, such as unlimited level hunting, recursive call forwarding (even when each device in the forwarding has a different dial plan) and just about anything else you can think of. To accomplish this, we depend on SOAP as a method of component communication because we consider any platform, including Microsoft and the ".NET framework" as things we need to interact with.
If your goal is to provide a framework for integrating with other platforms, SOAP bring a lot more flexibility to the game, and make it more compatible with more platforms.
Remember, this is just an opinion, however it needed to be expressed, just so you know what other folks are doing with SER.
--Dave
-----Original Message----- From: serusers-bounces@iptel.org [mailto:serusers-bounces@iptel.org] On Behalf Of Greger V. Teigre Sent: Monday, January 24, 2005 11:29 PM To: Juha Heinanen Cc: serusers@iptel.org Subject: Re: [Serusers] Carrier-grade framework for SER
Juha, Yes, I completely agree with you. However, I don't need to read the spec and far from understand it before I use it... ;-) So I did start to look at
SOAP and have very good experiences, both in terms of usability and scalability. But, I don't have strong opinions, if the people who are going to use the interface are all against SOAP, XMLRPC is the right choice.
The xmlrpc-provisioning work you have done, can it be coordinated with Andreas' effort? g-)
Juha Heinanen wrote:
Greger V. Teigre writes:
As I indicated in an earlier email, I would be interested in taking part in a joint effort to further develop ser's high-availability and scalability (HAS). I would probably have to do some development anyway, and I would prefer to see such support in the public domain. In Nov/Dec I called for responses on a SOAP-based provisioning interface, but heard nothing, so here is an overlap of interests.
greger,
we have done some work on xmlprc based provisioning and it looks promising. xmlrpc spec is three pages long and even i can understand it. soap spec, on the other hand, is far too thick and goes way above my head.
-- juha
Serusers mailing list Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
Serusers mailing list Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
-- Jiri Kuthan http://iptel.org/~jiri/
Serusers mailing list Serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
I'll weigh in.
I'd prefer SOAP. I know you're gung-ho on XML-RPC, and I can understand that its simplicity would make it useful in a lot of ways, but for the most part, the XML-RPC libraries I've seen for Java, C++, and PHP have all been a bit of a mess (Marquee/Redstone isn't TOO bad, but it's still rather poorly documented overall). SOAP, while being more complex, is a bit more powerful as well, allowing me to define the data types I'm using and accepting, and how to use them, as well as allowing a great deal more flexibility when it comes to specifying the characters sets I'm looking for and the like.
On the flip side, SOAP is still a growing standard. Some people will be turned off by the fact that it is almost certain to change as time goes on, but for any of us who got in on the ground floor of the Java spec, or PHP, or half a dozen other languages out there that change their APIs on a semi-regular basis, a spec or protocol that changes is hardly daunting.
We've used SOAP for a lot of things to connect to various services, and most of the telco services we interconnect with use SOAP and not XML-RPC. That would mean we'd end up having to do things two ways for all our apps and server side code, but if it has to be done, it has to be done.
It reall boils down to why do you like XML-RPC and what do you see its strengths are in terms of SER? If there's good reason to use one over another, even given the limitations, then I don't see a problem with it. As long as it's all well-documented with example code and such, it should be just fine.
Again, it boils down to the fact that good documentation goes a long way toward making even the most intractable projects into workable solutions.
N.
On Sat, 29 Jul 2006 11:05:24 +0200, Jiri Kuthan wrote
I'm digging through old archives and I am just wondering how people feel about 18 month later about the discussion about SOAP, XMl-RPC, etc. Any feedback would be appreciated -- what you think about it now and more importantly what's your experience if any. All in all, many are asking for a roadmap and input to that is most welcome.
-jiri
I have an opinion on this topic too but didn't want to begin egoistically with mine :-)
At 12:48 25/01/2005, David R. Kompel wrote:
Greger and everyone else that is interested,
Please consider before ruling out SOAP, that SOAP has more off the shelf libraries to support it then XMLRPC. Please consider the folks that use Microsoft platforms for their back end processing and databases, and keep in mind the following:
Yukon is just around the corner. It has SOAP services built in, as well as the ability to call SOAP services directly from T-SQL.
Also we implement a carrier grade platform using SER, which is in use by a number of VoIP providers here, with the following extensions:
- An extra module which allows for RADIUS URI translation, extended AVP
lookup, via extra string parameter which lets you identify what AVP query you wish to do, and an extra flag in the registration database "FOREIGN" registration, to identify a contact which has been replicated from another SER server.
- A service which speaks SOAP to he outside world, (it's own http
server on non-standard port) to allow an external interface to the SER FIFO interface. It use is for external Voicemail MWI Notifies, and to send refresh, reboot and report notify messages to SIP devices.
- A generic provisioning server for almost any SIP device, which can be
provisioned via TFTP, HTTP, or HTTPS. This server dynamically builds configuration files in memory on the fly for any device based on RE pattern matching of the filename, mapped to SQL statement, which returns device parameters.
With just these above three things, we can implement a full carrier grade system, with full automated device provisioning, all CLASS 5 features, such as unlimited level hunting, recursive call forwarding (even when each device in the forwarding has a different dial plan) and just about anything else you can think of. To accomplish this, we depend on SOAP as a method of component communication because we consider any platform, including Microsoft and the ".NET framework" as things we need to interact with.
If your goal is to provide a framework for integrating with other platforms, SOAP bring a lot more flexibility to the game, and make it more compatible with more platforms.
Remember, this is just an opinion, however it needed to be expressed, just so you know what other folks are doing with SER.
--Dave
-----Original Message----- From: serusers-bounces@iptel.org [mailto:serusers-bounces@iptel.org] On Behalf Of Greger V. Teigre Sent: Monday, January 24, 2005 11:29 PM To: Juha Heinanen Cc: serusers@iptel.org Subject: Re: [Serusers] Carrier-grade framework for SER
Juha, Yes, I completely agree with you. However, I don't need to read the spec and far from understand it before I use it... ;-) So I did start to look at
SOAP and have very good experiences, both in terms of usability and scalability. But, I don't have strong opinions, if the people who are going to use the interface are all against SOAP, XMLRPC is the right choice.
The xmlrpc-provisioning work you have done, can it be coordinated with Andreas' effort? g-)
Juha Heinanen wrote:
Greger V. Teigre writes:
As I indicated in an earlier email, I would be interested in taking part in a joint effort to further develop ser's high-availability and scalability (HAS). I would probably have to do some development anyway, and I would prefer to see such support in the public domain. In Nov/Dec I called for responses on a SOAP-based provisioning interface, but heard nothing, so here is an overlap of interests.
greger,
we have done some work on xmlprc based provisioning and it looks promising. xmlrpc spec is three pages long and even i can understand it. soap spec, on the other hand, is far too thick and goes way above my head.
-- juha
Serusers mailing list Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
Serusers mailing list Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
-- Jiri Kuthan http://iptel.org/~jiri/
Serusers mailing list Serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Just by looking at it without too much practical experience, I would say xmlrpc module is *nice*, but no more: to me it seems to complex for easy tasks (like usrloc operations or some counter inspecting) and too simple for complex tasks (SOAP indeed offers much more flexibility).
On the other hand, the module API extension for RPC seems a *very* good idea and leaves room for other protocol front ends (sercmd just as an example); maybe some work to facilitate/'standardise' implementing such front ends might be welcomed (like forking a number of children dedicated for the job or binding on some communication channels aso, which would have to be done every time). I think this is what should be made more popular and concentrate our attention on.
Although I'm a tad against web services (I haven't seen yet any big deployment using them, so, I am skeptical), so much more against using them in SER, I would not oppose the idea of having an official SOAP module: if for some small guy with 600+ subscribers it makes life easier, why not? However, I haven't seen any post with such an offer... Since we now have modules which allow use of SER as a pocket calculator, how would you reject an enterprise *looking* like extension? :-)
WL.
On 7/29/06, Jiri Kuthan jiri@iptel.org wrote:
I'm digging through old archives and I am just wondering how people feel about 18 month later about the discussion about SOAP, XMl-RPC, etc. Any feedback would be appreciated -- what you think about it now and more importantly what's your experience if any. All in all, many are asking for a roadmap and input to that is most welcome.
-jiri
I have an opinion on this topic too but didn't want to begin egoistically with mine :-)
At 12:48 25/01/2005, David R. Kompel wrote:
Greger and everyone else that is interested,
Please consider before ruling out SOAP, that SOAP has more off the shelf libraries to support it then XMLRPC. Please consider the folks that use Microsoft platforms for their back end processing and databases, and keep in mind the following:
Yukon is just around the corner. It has SOAP services built in, as well as the ability to call SOAP services directly from T-SQL.
Also we implement a carrier grade platform using SER, which is in use by a number of VoIP providers here, with the following extensions:
- An extra module which allows for RADIUS URI translation, extended AVP
lookup, via extra string parameter which lets you identify what AVP query you wish to do, and an extra flag in the registration database "FOREIGN" registration, to identify a contact which has been replicated from another SER server.
- A service which speaks SOAP to he outside world, (it's own http
server on non-standard port) to allow an external interface to the SER FIFO interface. It use is for external Voicemail MWI Notifies, and to send refresh, reboot and report notify messages to SIP devices.
- A generic provisioning server for almost any SIP device, which can be
provisioned via TFTP, HTTP, or HTTPS. This server dynamically builds configuration files in memory on the fly for any device based on RE pattern matching of the filename, mapped to SQL statement, which returns device parameters.
With just these above three things, we can implement a full carrier grade system, with full automated device provisioning, all CLASS 5 features, such as unlimited level hunting, recursive call forwarding (even when each device in the forwarding has a different dial plan) and just about anything else you can think of. To accomplish this, we depend on SOAP as a method of component communication because we consider any platform, including Microsoft and the ".NET framework" as things we need to interact with.
If your goal is to provide a framework for integrating with other platforms, SOAP bring a lot more flexibility to the game, and make it more compatible with more platforms.
Remember, this is just an opinion, however it needed to be expressed, just so you know what other folks are doing with SER.
--Dave
-----Original Message----- From: serusers-bounces@iptel.org [mailto:serusers-bounces@iptel.org] On Behalf Of Greger V. Teigre Sent: Monday, January 24, 2005 11:29 PM To: Juha Heinanen Cc: serusers@iptel.org Subject: Re: [Serusers] Carrier-grade framework for SER
Juha, Yes, I completely agree with you. However, I don't need to read the spec and far from understand it before I use it... ;-) So I did start to look at
SOAP and have very good experiences, both in terms of usability and scalability. But, I don't have strong opinions, if the people who are going to use the interface are all against SOAP, XMLRPC is the right choice.
The xmlrpc-provisioning work you have done, can it be coordinated with Andreas' effort? g-)
Juha Heinanen wrote:
Greger V. Teigre writes:
As I indicated in an earlier email, I would be interested in taking part in a joint effort to further develop ser's high-availability and scalability (HAS). I would probably have to do some development anyway, and I would prefer to see such support in the public domain. In Nov/Dec I called for responses on a SOAP-based provisioning interface, but heard nothing, so here is an overlap of interests.
greger,
we have done some work on xmlprc based provisioning and it looks promising. xmlrpc spec is three pages long and even i can understand it. soap spec, on the other hand, is far too thick and goes way above my head.
-- juha
Serusers mailing list Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
Serusers mailing list Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
-- Jiri Kuthan http://iptel.org/~jiri/ http://iptel.org/%7Ejiri/
Serusers mailing list Serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Thank you for your feedback.
I think it is to large extent matter of use-cases too. I mean where we have encountered SOAP so far was merely data provisioning, which was just about turning SOAP into SER's database, something that didn't necessiate direct support of SOAP inside SER.
I agree though that for those, who think they have a use-case, the current interface allows for a SOAP module easily.
-jiri
At 10:38 30/07/2006, Weiter Leiter wrote:
Just by looking at it without too much practical experience, I would say xmlrpc module is *nice*, but no more: to me it seems to complex for easy tasks (like usrloc operations or some counter inspecting) and too simple for complex tasks (SOAP indeed offers much more flexibility).
On the other hand, the module API extension for RPC seems a *very* good idea and leaves room for other protocol front ends (sercmd just as an example); maybe some work to facilitate/'standardise' implementing such front ends might be welcomed (like forking a number of children dedicated for the job or binding on some communication channels aso, which would have to be done every time). I think this is what should be made more popular and concentrate our attention on.
Although I'm a tad against web services (I haven't seen yet any big deployment using them, so, I am skeptical), so much more against using them in SER, I would not oppose the idea of having an official SOAP module: if for some small guy with 600+ subscribers it makes life easier, why not? However, I haven't seen any post with such an offer... Since we now have modules which allow use of SER as a pocket calculator, how would you reject an enterprise *looking* like extension? :-)
WL.
On 7/29/06, Jiri Kuthan <mailto:jiri@iptel.orgjiri@iptel.org> wrote: I'm digging through old archives and I am just wondering how people feel about 18 month later about the discussion about SOAP, XMl-RPC, etc. Any feedback would be appreciated -- what you think about it now and more importantly what's your experience if any. All in all, many are asking for a roadmap and input to that is most welcome.
-jiri
I have an opinion on this topic too but didn't want to begin egoistically with mine :-)
At 12:48 25/01/2005, David R. Kompel wrote:
Greger and everyone else that is interested,
Please consider before ruling out SOAP, that SOAP has more off the shelf libraries to support it then XMLRPC. Please consider the folks that use Microsoft platforms for their back end processing and databases, and keep in mind the following:
Yukon is just around the corner. It has SOAP services built in, as well as the ability to call SOAP services directly from T-SQL.
Also we implement a carrier grade platform using SER, which is in use by a number of VoIP providers here, with the following extensions:
- An extra module which allows for RADIUS URI translation, extended AVP
lookup, via extra string parameter which lets you identify what AVP query you wish to do, and an extra flag in the registration database "FOREIGN" registration, to identify a contact which has been replicated from another SER server.
- A service which speaks SOAP to he outside world, (it's own http
server on non-standard port) to allow an external interface to the SER FIFO interface. It use is for external Voicemail MWI Notifies, and to send refresh, reboot and report notify messages to SIP devices.
- A generic provisioning server for almost any SIP device, which can be
provisioned via TFTP, HTTP, or HTTPS. This server dynamically builds configuration files in memory on the fly for any device based on RE pattern matching of the filename, mapped to SQL statement, which returns device parameters.
With just these above three things, we can implement a full carrier grade system, with full automated device provisioning, all CLASS 5 features, such as unlimited level hunting, recursive call forwarding (even when each device in the forwarding has a different dial plan) and just about anything else you can think of. To accomplish this, we depend on SOAP as a method of component communication because we consider any platform, including Microsoft and the ".NET framework" as things we need to interact with.
If your goal is to provide a framework for integrating with other platforms, SOAP bring a lot more flexibility to the game, and make it more compatible with more platforms.
Remember, this is just an opinion, however it needed to be expressed, just so you know what other folks are doing with SER.
--Dave
-----Original Message----- From: mailto:serusers-bounces@iptel.orgserusers-bounces@iptel.org [mailto: serusers-bounces@iptel.org] On Behalf Of Greger V. Teigre Sent: Monday, January 24, 2005 11:29 PM To: Juha Heinanen Cc: mailto:serusers@iptel.orgserusers@iptel.org Subject: Re: [Serusers] Carrier-grade framework for SER
Juha, Yes, I completely agree with you. However, I don't need to read the spec and far from understand it before I use it... ;-) So I did start to look at
SOAP and have very good experiences, both in terms of usability and scalability. But, I don't have strong opinions, if the people who are going to use the interface are all against SOAP, XMLRPC is the right choice.
The xmlrpc-provisioning work you have done, can it be coordinated with Andreas' effort? g-)
Juha Heinanen wrote:
Greger V. Teigre writes:
As I indicated in an earlier email, I would be interested in taking part in a joint effort to further develop ser's high-availability and scalability (HAS). I would probably have to do some development anyway, and I would prefer to see such support in the public domain. In Nov/Dec I called for responses on a SOAP-based provisioning interface, but heard nothing, so here is an overlap of interests.
greger,
we have done some work on xmlprc based provisioning and it looks promising. xmlrpc spec is three pages long and even i can understand it. soap spec, on the other hand, is far too thick and goes way above my head.
-- juha
Serusers mailing list mailto:Serusers@iptel.org Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
Serusers mailing list mailto:Serusers@iptel.org Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
-- Jiri Kuthan http://iptel.org/%7Ejiri/http://iptel.org/~jiri/
Serusers mailing list mailto:Serusers@lists.iptel.orgSerusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
-- Jiri Kuthan http://iptel.org/~jiri/
This thread is old, but I have kept it because I wanted to comment once I had some time :-) Hope it's okay...
I think we need to look at what SER is and should be from an architecture point of view.
Perspective 1, enterprise: If SER is a standalone server, eg. an enterprise server, SER runs on a box with mysql and a PSTN connection to some service provider. The interfaces (in classic component thinking) needed are then the following: a. Authentication to a corporate LDAP server (did I hear ActiveDirectory) or in local mysql database b. SIP data (user location, etc) c. Accounting for checking the bill from the PSTN provider d. Some simple management e. Provisioning of accounts if they are found in mysql database f. Change user settings
If you pull the accounts out of SER, a), e), and f) will probably be handled by RADIUS or LDAP. b) SIP data and c) accounting will be fine to do in SER's mysql, while d) management would probably be SNMP. However, if accounts stay in SER's mysql database, you need a way to do e) account provisioning and f) user settings. To be honest, SOAP and XMLRPC both fit the bill, but there are more tools for SOAP. In fact, a simple REST-based (http://en.wikipedia.org/wiki/REST) http protocol would be the easiest...
Perspective 2, carrier infrastructure: If SER is part of an infrastructure, you have more requirements. SER runs as a transaction server, you have multiple servers, you need to do replication, and you may have accounts outside the SER database. You also have provisioning systems, CRM, helpdesk tools, operations (OSS/BSS).
Looking at the interfaces described above, you will probably use RADIUS, LDAP, or DIAMETER for accessing a user database and settings (that is a), e), and f)). If your SIP infrastructure is standalone, you will probably use SER's mysql with some kind of replication or usrloc-cl + mysql cluster. In the first scenario, you don't need provisioning of accounts, in the second you do. Again, SOAP is probably the most likely to fit the bill as SOAP is more common among OSS/BSS and CRM systems.
For c) accounting, you need to interface with either a real-time billing system or periodically dump records readable for a billing system. For real-time billing, DIAMETER is defined as the IMS/3GPP protocol of choice. DIAMETER is based on the principles of RADIUS, an accounting protocol implemented by many.
Then to IPDR as Arek suggests. IPDR is many things in one. It comes out of the traditional Call Data Records, the file-based records used for encoding calls. It has turned into a more complex set of transport protocols, encoding (XML and the sucessor of CDRs: XDR), and shemas. I haven't looked at the details, but it probably supports some form of real-time billing (and thus authorization). In this respect it overlaps with Open Settlement Protocol (OSP, http://www.transnexus.com/White%20Papers/What%20is%20OSP.htm), which we already have a module for. OSP comes out of TISPAN, the ISP's standardization effort to adapt 3GPP IMS architecture to the ISP world. However, I see IPDR more as a back-end accounting specification, than a real-time settlement protocol. Here I agree with Jiri, any real-time elements of IPDR is natural to have as a SER module, however, the majority of the IPDR specification is concerned about a step that is outside SER, namely acocunting start/stop correlation, cleaning, and CDR/XDR/IPDR record generation. An IPDR accounting module would be possible,and probably needed if one wants to enable SER to send live IPDR data to an IPDR compliant CDR collector. So, anyone, feel free to implement an IPDR module ;-)
Finally, to the d) interface, an SNMP module would probably be nice, but not enough. The trend is towards actual service probing where the full user experience is monitored (ex. automated calls measuring MOS). sipp can used for this (not MOS score), but at least doing a full call.
In addition, you have another interface for carrier infrastructures: g) Application servers (AS)
The standard interface for AS is SIP. ParlayX is used for this in the old telco world, but AFAIK it has a SIP interface as well. I cannot see why SER should implement ParlayX in a module?! Maybe you could enlighten me, Arek?! ParlayX is a big thing and if SER is to be extended towards more of an application server, we need to think through what SER should be. SIP CGI is interesting for tinkerers, as well as many applications. JSLEE integration would be interesting, and of course ParlayX. (I here intentionally mix up actual application capabilities with interfaces towards applications servers...) --------------------------------
My conclusions so far: - The XMLRPC front-end is good for tinkering, but it does not really match what is available of tools and connecting systems in the two scenarios described above. A SOAP module, maybe even based on a standard(!), would simplify provisioning and user account settings - I feel an IPDR module could be a good addition to SER, as long as it does not try to do something a SIP server is not meant to be - SNMP would be a good addition - I'm all for extending SER's capabilities as an application engine. But I'm not sure yet in which direction it should go. I believe there is way to open up creativity in SIP applications, ex. through combining SER and SEMS into a platform for application development (it already is, but it's not really well known...) As for extending SER with all kinds of interface, why not? If somebody has a use case and wants to develop a module, I think that is a good thing.
My 2 cents... finally signing off... g-)
Jiri Kuthan wrote:
Thank you for your feedback.
I think it is to large extent matter of use-cases too. I mean where we have encountered SOAP so far was merely data provisioning, which was just about turning SOAP into SER's database, something that didn't necessiate direct support of SOAP inside SER.
I agree though that for those, who think they have a use-case, the current interface allows for a SOAP module easily.
-jiri
At 10:38 30/07/2006, Weiter Leiter wrote:
Just by looking at it without too much practical experience, I would say xmlrpc module is *nice*, but no more: to me it seems to complex for easy tasks (like usrloc operations or some counter inspecting) and too simple for complex tasks (SOAP indeed offers much more flexibility).
On the other hand, the module API extension for RPC seems a *very* good idea and leaves room for other protocol front ends (sercmd just as an example); maybe some work to facilitate/'standardise' implementing such front ends might be welcomed (like forking a number of children dedicated for the job or binding on some communication channels aso, which would have to be done every time). I think this is what should be made more popular and concentrate our attention on.
Although I'm a tad against web services (I haven't seen yet any big deployment using them, so, I am skeptical), so much more against using them in SER, I would not oppose the idea of having an official SOAP module: if for some small guy with 600+ subscribers it makes life easier, why not? However, I haven't seen any post with such an offer... Since we now have modules which allow use of SER as a pocket calculator, how would you reject an enterprise *looking* like extension? :-)
WL.
On 7/29/06, Jiri Kuthan <mailto:jiri@iptel.orgjiri@iptel.org> wrote: I'm digging through old archives and I am just wondering how people feel about 18 month later about the discussion about SOAP, XMl-RPC, etc. Any feedback would be appreciated -- what you think about it now and more importantly what's your experience if any. All in all, many are asking for a roadmap and input to that is most welcome.
-jiri
I have an opinion on this topic too but didn't want to begin egoistically with mine :-)
At 12:48 25/01/2005, David R. Kompel wrote:
Greger and everyone else that is interested,
Please consider before ruling out SOAP, that SOAP has more off the shelf libraries to support it then XMLRPC. Please consider the folks that use Microsoft platforms for their back end processing and databases, and keep in mind the following:
Yukon is just around the corner. It has SOAP services built in, as well as the ability to call SOAP services directly from T-SQL.
Also we implement a carrier grade platform using SER, which is in use by a number of VoIP providers here, with the following extensions:
- An extra module which allows for RADIUS URI translation, extended AVP
lookup, via extra string parameter which lets you identify what AVP query you wish to do, and an extra flag in the registration database "FOREIGN" registration, to identify a contact which has been replicated
from another SER server.
- A service which speaks SOAP to he outside world, (it's own http
server on non-standard port) to allow an external interface to the SER FIFO interface. It use is for external Voicemail MWI Notifies, and to send refresh, reboot and report notify messages to SIP devices.
- A generic provisioning server for almost any SIP device, which can be
provisioned via TFTP, HTTP, or HTTPS. This server dynamically builds configuration files in memory on the fly for any device based on RE pattern matching of the filename, mapped to SQL statement, which returns device parameters.
With just these above three things, we can implement a full carrier grade system, with full automated device provisioning, all CLASS 5 features, such as unlimited level hunting, recursive call forwarding (even when each device in the forwarding has a different dial plan) and just about anything else you can think of. To accomplish this, we depend on SOAP as a method of component communication because we consider any platform, including Microsoft and the ".NET framework" as things we need to interact with.
If your goal is to provide a framework for integrating with other platforms, SOAP bring a lot more flexibility to the game, and make it more compatible with more platforms.
Remember, this is just an opinion, however it needed to be expressed, just so you know what other folks are doing with SER.
--Dave
-----Original Message----- From: mailto:serusers-bounces@iptel.orgserusers-bounces@iptel.org [mailto: serusers-bounces@iptel.org] On Behalf Of Greger V. Teigre Sent: Monday, January 24, 2005 11:29 PM To: Juha Heinanen Cc: mailto:serusers@iptel.orgserusers@iptel.org Subject: Re: [Serusers] Carrier-grade framework for SER
Juha, Yes, I completely agree with you. However, I don't need to read the spec and far from understand it before I use it... ;-) So I did start to look at
SOAP and have very good experiences, both in terms of usability and scalability. But, I don't have strong opinions, if the people who are going to use the interface are all against SOAP, XMLRPC is the right choice.
The xmlrpc-provisioning work you have done, can it be coordinated with Andreas' effort? g-)
Juha Heinanen wrote:
Greger V. Teigre writes:
As I indicated in an earlier email, I would be interested in taking part in a joint effort to further develop ser's high-availability and scalability (HAS). I would probably have to do some development anyway, and I would prefer to see such support in the public domain. In Nov/Dec I called for responses on a SOAP-based provisioning interface, but heard nothing, so here is an overlap of interests.
greger,
we have done some work on xmlprc based provisioning and it looks promising. xmlrpc spec is three pages long and even i can understand it. soap spec, on the other hand, is far too thick and goes way above my head.
-- juha
Serusers mailing list mailto:Serusers@iptel.org Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
Serusers mailing list mailto:Serusers@iptel.org Serusers@iptel.org http://mail.iptel.org/mailman/listinfo/serusers
-- Jiri Kuthan http://iptel.org/%7Ejiri/http://iptel.org/~jiri/
Serusers mailing list mailto:Serusers@lists.iptel.orgSerusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
-- Jiri Kuthan http://iptel.org/~jiri/
Hello Greger, all,
Please find my answers inline:
----- Original Message ----- From: Greger V. Teigre To: Jiri Kuthan Cc: Weiter Leiter ; David R. Kompel ; serusers@iptel.org ; Arek Bekiersz Sent: Friday, September 22, 2006 9:44 AM Subject: Re: SOAP/XML-RPC experience, etc. (was RE: [Serusers] What do you have against SOAP? (was Carrier-grade framework for SER)
Greger: ---------- This thread is old, but I have kept it because I wanted to comment once I had some time :-) Hope it's okay...
Arek: -------- No, it is not old. It is alive and kicking ;-) I don't have much time, but also wanted to rejoin my favourite thread ;-)
So, in fact we at our company are in the middle of european IST project, related to integration of business and technology, by various means related to workflow technologies and Web services. I invite you to visit our Website at http://www.visp-project.org/
It just happened that VoIP services based on SER were chosen as first proof-of-concept in VISP. Reason is that I was playing with Web services in SER for a long time, in production systems. I'm working hardly now with workflow related technologies to finally integrate VoIP services (and their technical workflows) with corporate business workfows. I use Web services for this (SOAP/WSDL in particular) and other sexy XML and workflow stuff.
Believe me, I couldn't have done what I have done without SOAP. I will keep developing SOAP related stuff for SER. First because I have to (for the project) and second because I believe this is the best way to achieve seamless integration of SER into corporate workflow based OSS systems. I will try to publish results of that work, if possible.
Greger: ---------- I think we need to look at what SER is and should be from an architecture point of view.
Perspective 1, enterprise: If SER is a standalone server, eg. an enterprise server, SER runs on a box with mysql and a PSTN connection to some service provider. The interfaces (in classic component thinking) needed are then the following: a. Authentication to a corporate LDAP server (did I hear ActiveDirectory) or in local mysql database b. SIP data (user location, etc) c. Accounting for checking the bill from the PSTN provider d. Some simple management e. Provisioning of accounts if they are found in mysql database f. Change user settings
If you pull the accounts out of SER, a), e), and f) will probably be handled by RADIUS or LDAP. b) SIP data and c) accounting will be fine to do in SER's mysql, while d) management would probably be SNMP. However, if accounts stay in SER's mysql database, you need a way to do e) account provisioning and f) user settings. To be honest, SOAP and XMLRPC both fit the bill, but there are more tools for SOAP. In fact, a simple REST-based (http://en.wikipedia.org/wiki/REST) http protocol would be the easiest...
Arek: -------- For e) I use Web services based on SOAP. The end. I will not try to convince anybody to use it, but as I said above - some huge and really nice stuff would be virtually impossible without e) being done in SOAP.
For the rest ( points a) and f) personally I stop to care anymore about storage behind SER. It is hard to convince people to use directories (LDAP, ActiveDirectory), for sake of the real value they bring in terms of scalability, integration with the rest of enterprise intranet, friendliness with XML, etc. I could not succeed on wider scale. Why? With the MySQL clustering and redundancy people do exactly the same, reinventing things that have been done long ago. Reinventing the wheel, I mean.
I will keep developing for LDAP for my company use and try to publish that as open source later. I treat it as a task separate from Web service stuff. If there is aybody who likes Ldap, we must provide him a possibility do use it with SER directly.
Greger: ----------- Perspective 2, carrier infrastructure: If SER is part of an infrastructure, you have more requirements. SER runs as a transaction server, you have multiple servers, you need to do replication, and you may have accounts outside the SER database. You also have provisioning systems, CRM, helpdesk tools, operations (OSS/BSS).
Looking at the interfaces described above, you will probably use RADIUS, LDAP, or DIAMETER for accessing a user database and settings (that is a), e), and f)). If your SIP infrastructure is standalone, you will probably use SER's mysql with some kind of replication or usrloc-cl + mysql cluster. In the first scenario, you don't need provisioning of accounts, in the second you do. Again, SOAP is probably the most likely to fit the bill as SOAP is more common among OSS/BSS and CRM systems.
For c) accounting, you need to interface with either a real-time billing system or periodically dump records readable for a billing system. For real-time billing, DIAMETER is defined as the IMS/3GPP protocol of choice. DIAMETER is based on the principles of RADIUS, an accounting protocol implemented by many.
Then to IPDR as Arek suggests. IPDR is many things in one. It comes out of the traditional Call Data Records, the file-based records used for encoding calls. It has turned into a more complex set of transport protocols, encoding (XML and the sucessor of CDRs: XDR), and schemas. I haven't looked at the details, but it probably supports some form of real-time billing (and thus authorization). In this respect it overlaps with Open Settlement Protocol (OSP, http://www.transnexus.com/White%20Papers/What%20is%20OSP.htm), which we already have a module for. OSP comes out of TISPAN, the ISP's standardization effort to adapt 3GPP IMS architecture to the ISP world. However, I see IPDR more as a back-end accounting specification, than a real-time settlement protocol. Here I agree with Jiri, any real-time elements of IPDR is natural to have as a SER module, however, the majority of the IPDR specification is concerned about a step that is outside SER, namely acocunting start/stop correlation, cleaning, and CDR/XDR/IPDR record generation. An IPDR accounting module would be possible,and probably needed if one wants to enable SER to send live IPDR data to an IPDR compliant CDR collector. So, anyone, feel free to implement an IPDR module ;-)
Arek: -------- Read, agreed. Years are passing and we still do not have proper IPDR compliant CDR collector. Every poor SER newbie have to write those little scripts to extract those date, time and duration of his calls. Let's give them proper CDR collector (IPDR recorder I mean) in real-time. The rest of IPDR, non-real time stuff - keep inside module as well of throw away from SER - to be decided.
Greger: ---------- Finally, to the d) interface, an SNMP module would probably be nice, but not enough. The trend is towards actual service probing where the full user experience is monitored (ex. automated calls measuring MOS). sipp can used for this (not MOS score), but at least doing a full call.
In addition, you have another interface for carrier infrastructures: g) Application servers (AS)
The standard interface for AS is SIP. ParlayX is used for this in the old telco world, but AFAIK it has a SIP interface as well. I cannot see why SER should implement ParlayX in a module?! Maybe you could enlighten me, Arek?!
Arek: ------- And let me enlighten you, Greger ;-) I've mentioned ParlayX Web services, to make it easy for Parlay and Web service geeks to build custom application servers. I found it to be a nice alternative, to have well documented standard interface from SER server towards Application Server. I gave an example of SER receiving a SIP call to a number, then preparing and sending Web service message to Application Server.
Upon receiving answer from Application server, SER servce could act on SIP message. Action taken could be one like: forward here, forward there or send him a '486 Busy Here'. I proposed that because I had a nice application server (with Web service interfaces) in my company and I was looking for some standard way to talk to this server. I though that: writing ParlayX Web service module for SER + modifying Application Server to have ParlayX compliant interfaces would be a really sexy solution to build applications like prepaid, prepaid with flat fee, multimedia services, etc.
Feel free to correct me if I'm wrong.
Greger: ----------- ParlayX is a big thing and if SER is to be extended towards more of an application server, we need to think through what SER should be. SIP CGI is interesting for tinkerers, as well as many applications. JSLEE integration would be interesting, and of course ParlayX. (I here intentionally mix up actual application capabilities with interfaces towards applications servers...)
My conclusions so far: - The XMLRPC front-end is good for tinkering, but it does not really match what is available of tools and connecting systems in the two scenarios described above. A SOAP module, maybe even based on a standard(!), would simplify provisioning and user account settings
Arek: ------- YES it would. I really could send you today an WSDL of such service running at our premises for 2 consequent years. I just cannot do it because of confidentiality reasons. But will try to go with it to open source as soon as possible.
Greger: ----------- - I feel an IPDR module could be a good addition to SER, as long as it does not try to do something a SIP server is not meant to be
Arek: ------- I Agree. Believe me, I don't want to make SER a washing machine with dryer either. I just want to give people a compliant CDR recorder, to have a nice basis to build their billing applications.
Greger: ----------- - SNMP would be a good addition
Arek: -------- Yes it would be. We are ready to work on MIB for that in our company (I got a really good specialist for SNMP here, next to my desk).
Greger: ------------ - I'm all for extending SER's capabilities as an application engine. But I'm not sure yet in which direction it should go. I believe there is way to open up creativity in SIP applications, ex. through combining SER and SEMS into a platform for application development (it already is, but it's not really well known...) As for extending SER with all kinds of interface, why not? If somebody has a use case and wants to develop a module, I think that is a good thing.
Arek: ------- Again, agree. I don't want SER to be washing machine and foot massager in one. But if something is needed, there is someone willing to provide and everyone agrees - then it should be done
Greger: ------------ My 2 cents... finally signing off... g-)
Arek: -------- My 2 PLN's signing off too ;-)
-- Regards, Arek Bekiersz
... At 23:58 24/09/2006, Arek Bekiersz wrote:
Believe me, I couldn't have done what I have done without SOAP. I will keep developing SOAP related stuff for SER. First because I have to (for the project) and second because I believe this is the best way to achieve seamless integration of SER into corporate workflow based OSS systems. I will try to publish results of that work, if possible.
Just became curious -- my favorite question, did you do it as a standalone piece of software which "feeds" SER database, or as a part of SER? Is that the piece of work which is falling under your non-disclosable development?
...
Arek:
Read, agreed. Years are passing and we still do not have proper IPDR compliant CDR collector. Every poor SER newbie have to write those little scripts to extract those date, time and duration of his calls. Let's give them proper CDR collector (IPDR recorder I mean) in real-time. The rest of IPDR, non-real time stuff - keep inside module as well of throw away from SER - to be decided.
what do you call "proper CDR collector"? (I intuitivelly guess I agree but am not 100% sure we are speaking about the same).
Arek:
And let me enlighten you, Greger ;-) I've mentioned ParlayX Web services, to make it easy for Parlay and Web service geeks to build custom application servers. I found it to be a nice alternative, to have well documented standard interface from SER server towards Application Server. I gave an example of SER receiving a SIP call to a number, then preparing and sending Web service message to Application Server.
Upon receiving answer from Application server, SER servce could act on SIP message. Action taken could be one like: forward here, forward there or send him a '486 Busy Here'. I proposed that because I had a nice application server (with Web service interfaces) in my company and I was looking for some standard way to talk to this server. I though that: writing ParlayX Web service module for SER + modifying Application Server to have ParlayX compliant interfaces would be a really sexy solution to build applications like prepaid, prepaid with flat fee, multimedia services, etc.
Feel free to correct me if I'm wrong.
I am not sure -- some ideas have grown too big for a successful birth and parlay may be one of those, but that's of course my own guess. (Hope I don't sound discouraging, I'm just wondering what people think about that opinion.)
Arek:
YES it would. I really could send you today an WSDL of such service running at our premises for 2 consequent years. I just cannot do it because of confidentiality reasons. But will try to go with it to open source as soon as possible.
Greger:
- I feel an IPDR module could be a good addition to SER, as long as it does
not try to do something a SIP server is not meant to be
Arek:
I Agree. Believe me, I don't want to make SER a washing machine with dryer either. I just want to give people a compliant CDR recorder, to have a nice basis to build their billing applications.
Greger:
- SNMP would be a good addition
Arek:
Yes it would be. We are ready to work on MIB for that in our company (I got a really good specialist for SNMP here, next to my desk).
Just my 2 cents -- if people are interested in my feedback, the way I would build it is an "agent" which gathers the data from SER using existing interface and exports it to the world using SNMP. (Using your vocabulary, that's generally my favorite approach to avoid embedded washing machines in SER :-))
Other curiosity question: is this something we are discussing within the roadmap debate, or do people have some running code to share?
-jiri
-- Jiri Kuthan http://iptel.org/~jiri/
Hi Jiri,
Please see inline:
----- Original Message ----- From: "Jiri Kuthan" jiri@iptel.org To: "Arek Bekiersz" sip@perceval.net; "Greger V. Teigre" greger@teigre.com Cc: "Serusers" serusers@iptel.org; "David R. Kompel" drk@drkngs.net; "Weiter Leiter" bp4mls@googlemail.com Sent: Tuesday, September 26, 2006 12:49 AM Subject: Re: SOAP/XML-RPC experience, etc. (was RE: [Serusers] What do you have against SOAP? (was Carrier-grade framework for SER)
Just became curious -- my favorite question, did you do it as a standalone piece of software which "feeds" SER database, or as a part of SER? Is that the piece of work which is falling under your non-disclosable development?
Arek: The answer is twofold: a) outside of SER, a stand-alone application+web server that feeds SER dbase (mysql and ldap) and remote Asterisks dbs. b) partially inside of SER (some modifications + ldap module, that by the way still exists in cvs exp. directory). I've begun moving it to latest devel version some time ago, but lack of time...I keep the faith ;-)
For this 'non-disclosing stuff' I mean the SER-outside stuff. If you wish, I can share with you WSDLs of these Web services and a paper manual, on private email. You could poke that material and see if it is of any help?
Anyway, I would love to move the 'outside stuff' inside of SER, to become one or more new modules. Such module(s) would have built-in Web server. Still thinking of gSoap with its small footprint... (www.cs.fsu.edu/~engelen/soap.html).
Years are passing and we still do not have proper IPDR compliant CDR collector. Every poor SER newbie have to write those little scripts to extract those date, time and duration of his calls. Let's give them proper CDR collector (IPDR recorder I mean) in real-time. The rest of IPDR, non-real time stuff - keep inside module as well of throw away from SER - to be decided.
what do you call "proper CDR collector"? (I intuitivelly guess I agree but am not 100% sure we are speaking about the same).
Arek: Well, it depends. I will say straight: I'm still supporting the idea of Web service IPDR inteface towards the acc module, or a new module. On every request for IPDR data they would do query in SER's native acc dbase, find matching INVITE-BYE pairs, calculate duration, disconnecting party etc. and return the IPDR data in web service response.
Advantage: No need to have separate dbase for CDR, storing complete call information (time of start, end & duration), as everything is calculated on-the-fly, per web service request basis. Disadvantage: Possible slowliness or freezing of SER module, when there are tens of thousands of entries in SER acc db.
It might be combined with dialog module, to show calls in-progress as well (establishing, ringing, diverting). It is all all as described in IPDR VoIP profile (see callProgressState in www.ipdr.org/service_specs/VoIP/VoIP3.1-A.0.2.pdf).
But I'm not sure anymore. Maybe it is realy better to keep it outside, have separate IPDR db and just run a cron job to update it every minute or two? I do it like that now, by the way. To be discussed.
Arek:
And let me enlighten you, Greger ;-) I've mentioned ParlayX Web services, to make it easy for Parlay and Web service geeks to build custom application servers. I found it to be a nice alternative, to have well documented standard interface from SER server towards Application Server. I gave an example of SER receiving a SIP call to a number, then preparing and sending Web service message to Application Server.
Upon receiving answer from Application server, SER servce could act on SIP message. Action taken could be one like: forward here, forward there or send him a '486 Busy Here'. I proposed that because I had a nice application server (with Web service interfaces) in my company and I was looking for some standard way to talk to this server. I though that: writing ParlayX Web service module for SER + modifying Application Server to have ParlayX compliant interfaces would be a really sexy solution to build applications like prepaid, prepaid with flat fee, multimedia services, etc.
Feel free to correct me if I'm wrong.
I am not sure -- some ideas have grown too big for a successful birth and parlay may be one of those, but that's of course my own guess. (Hope I don't sound discouraging, I'm just wondering what people think about that opinion.)
Arek: Well,... I share the same opinion about some heavyweight ideas. But there are people tinkering with Parlay, take Ericpol company, from my home city, as an example: (http://www.ericpol.pl/www/en/).
Arek:
YES it would. I really could send you today an WSDL of such service running at our premises for 2 consequent years. I just cannot do it because of confidentiality reasons. But will try to go with it to open source as soon as possible.
Greger:
- I feel an IPDR module could be a good addition to SER, as long as it
does not try to do something a SIP server is not meant to be
Arek:
I Agree. Believe me, I don't want to make SER a washing machine with dryer either. I just want to give people a compliant CDR recorder, to have a nice basis to build their billing applications.
Greger:
- SNMP would be a good addition
Arek:
Yes it would be. We are ready to work on MIB for that in our company (I got a really good specialist for SNMP here, next to my desk).
Just my 2 cents -- if people are interested in my feedback, the way I would build it is an "agent" which gathers the data from SER using existing interface and exports it to the world using SNMP. (Using your vocabulary, that's generally my favorite approach to avoid embedded washing machines in SER :-))
Arek: I'm generally supporting idea of extending SER itself, in form of modules. It is for people, to make their installation easier. You know - download, make, install and become ISP in one day, no questions asked on mailing lists.
From the other side we cannot put new washing machine inside SER every other
day. Otherwise we would overload it with features so much, that we would have to remove the 'E' (Express) letter from SER name :-) All to be discussed.
Other curiosity question: is this something we are discussing within the roadmap debate, or do people have some running code to share?
Arek: I would say it is rather a roadmap discussion. Sorry for that - it was Greger who began with this thread again ;-)))
-- Regards, Arek Bekiersz
-jiri
-- Jiri Kuthan http://iptel.org/~jiri/
Just became curious -- my favorite question, did you do it as a standalone piece of software which "feeds" SER database, or as a part of SER? Is that the piece of work which is falling under your non-disclosable development?
Arek: The answer is twofold: a) outside of SER, a stand-alone application+web server that feeds SER dbase (mysql and ldap) and remote Asterisks dbs. b) partially inside of SER (some modifications + ldap module, that by the way still exists in cvs exp. directory). I've begun moving it to latest devel version some time ago, but lack of time...I keep the faith ;-)
:-) Good. For 0.10 we will most likely get rid of experimental and move the modules into the standard tree.
what do you call "proper CDR collector"? (I intuitivelly guess I agree but am not 100% sure we are speaking about the same).
Arek: Well, it depends. I will say straight: I'm still supporting the idea of Web service IPDR inteface towards the acc module, or a new module. On every request for IPDR data they would do query in SER's native acc dbase, find matching INVITE-BYE pairs, calculate duration, disconnecting party etc. and return the IPDR data in web service response.
This is exactly where I have a problem: a) SER is a server for SIP messages. Period. Not http, not SOAP over TCP, nothing. Even the management interface can be invoked through SIP and the SIP code is used to handle incoming requests even for non-SIP requests b) Doing costly DB lookups only drags down SER and why should it?
Advantage: No need to have separate dbase for CDR, storing complete call information (time of start, end & duration), as everything is calculated on-the-fly, per web service request basis. Disadvantage: Possible slowliness or freezing of SER module, when there are tens of thousands of entries in SER acc db.
Exactly.
It might be combined with dialog module, to show calls in-progress as well (establishing, ringing, diverting). It is all all as described in IPDR VoIP profile (see callProgressState in www.ipdr.org/service_specs/VoIP/VoIP3.1-A.0.2.pdf).
But I'm not sure anymore. Maybe it is realy better to keep it outside, have separate IPDR db and just run a cron job to update it every minute or two? I do it like that now, by the way. To be discussed.
An IPDR provider can easily be built as a gsoap service (either running under apache or standalone). An ipdr SER module can communicate with the IPDR provider using soap. Benefits: - Your IPDR provider can real time do queries into one or more SER accounting databases, locally or over a private management network - You can do accounting to a mysql cluster from many SER instances - You only need one IPDR provider for your SER installation - You get the scalability you want for each SER server - You can isolate the IPDR soap-based provider on a separate server (as it has very different security and scalability requirements)
g-)
Greger:
This thread is old, but I have kept it because I wanted to comment once I had some time :-) Hope it's okay...
Arek:
No, it is not old. It is alive and kicking ;-)
:-) well, now it is...
I don't have much time, but also wanted to rejoin my favourite thread ;-)
So, in fact we at our company are in the middle of european IST project, related to integration of business and technology, by various means related to workflow technologies and Web services. I invite you to visit our Website at http://www.visp-project.org/
Yes, I have also visited it before. The project goals sounds good. Enabling smaller providers to join forces and "operate as one" may help them stay afloat. Hopefully, it also enables them to be more innovative and compete better...?! It just puzzles me how little info is available. I assume the participants see the development as a strategic asset. Your participation in the SER community has been mainly talking about it (except on the LDAP module). If you come with code, the community process would improve it and create some dynamics. A thread on serusers does not bring us anywhere. I realize that you have confidentiality requirements, it's just a frustration to hear, but not see...
It just happened that VoIP services based on SER were chosen as first proof-of-concept in VISP. Reason is that I was playing with Web services in SER for a long time, in production systems. I'm working hardly now with workflow related technologies to finally integrate VoIP services (and their technical workflows) with corporate business workfows. I use Web services for this (SOAP/WSDL in particular) and other sexy XML and workflow stuff.
I hope you don't get lost in the sexiness of all the web service workflow stuff. ;-) KISS is a good principle.
Believe me, I couldn't have done what I have done without SOAP. I will keep developing SOAP related stuff for SER. First because I have to (for the project) and second because I believe this is the best way to achieve seamless integration of SER into corporate workflow based OSS systems. I will try to publish results of that work, if possible.
Greger:
I think we need to look at what SER is and should be from an architecture point of view.
Perspective 1, enterprise: If SER is a standalone server, eg. an enterprise server, SER runs on a box with mysql and a PSTN connection to some service provider. The interfaces (in classic component thinking) needed are then the following: a. Authentication to a corporate LDAP server (did I hear ActiveDirectory) or in local mysql database b. SIP data (user location, etc) c. Accounting for checking the bill from the PSTN provider d. Some simple management e. Provisioning of accounts if they are found in mysql database f. Change user settings
If you pull the accounts out of SER, a), e), and f) will probably be handled by RADIUS or LDAP. b) SIP data and c) accounting will be fine to do in SER's mysql, while d) management would probably be SNMP. However, if accounts stay in SER's mysql database, you need a way to do e) account provisioning and f) user settings. To be honest, SOAP and XMLRPC both fit the bill, but there are more tools for SOAP. In fact, a simple REST-based (http://en.wikipedia.org/wiki/REST) http protocol would be the easiest...
Arek:
For e) I use Web services based on SOAP. The end. I will not try to convince anybody to use it, but as I said above - some huge and really nice stuff would be virtually impossible without e) being done in SOAP.
In 0.10.x you can easily add SOAP as a front-end to the new management interface.
For the rest ( points a) and f) personally I stop to care anymore about storage behind SER. It is hard to convince people to use directories (LDAP, ActiveDirectory), for sake of the real value they bring in terms of scalability, integration with the rest of enterprise intranet, friendliness with XML, etc. I could not succeed on wider scale. Why? With the MySQL clustering and redundancy people do exactly the same, reinventing things that have been done long ago. Reinventing the wheel, I mean.
Well, I think it has something to do with competence and complexity. The more components, the more competence required, the higher the complexity, and thus people stay within their domain (which obviously is SIP and mysql). mysql is where you start out, then you get lots of subscribers and suddenly need redundancy and scalability and you look for the easiest solution (which eventually may not be the best in terms of cost or robustness). So, a full how-to with LDAP and SOAP with all code available would make more people start using it from day 1.
I will keep developing for LDAP for my company use and try to publish that as open source later. I treat it as a task separate from Web service stuff. If there is aybody who likes Ldap, we must provide him a possibility do use it with SER directly.
Yes, I agree. LDAP is good. However, for large scale setups, LDAP is also a costly protocol, so using freeRADIUS in front of your LDAP database would actually do you good (but yes, another component).
Greger:
Perspective 2, carrier infrastructure: If SER is part of an infrastructure, you have more requirements. SER runs as a transaction server, you have multiple servers, you need to do replication, and you may have accounts outside the SER database. You also have provisioning systems, CRM, helpdesk tools, operations (OSS/BSS).
Looking at the interfaces described above, you will probably use RADIUS, LDAP, or DIAMETER for accessing a user database and settings (that is a), e), and f)). If your SIP infrastructure is standalone, you will probably use SER's mysql with some kind of replication or usrloc-cl + mysql cluster. In the first scenario, you don't need provisioning of accounts, in the second you do. Again, SOAP is probably the most likely to fit the bill as SOAP is more common among OSS/BSS and CRM systems.
For c) accounting, you need to interface with either a real-time billing system or periodically dump records readable for a billing system. For real-time billing, DIAMETER is defined as the IMS/3GPP protocol of choice. DIAMETER is based on the principles of RADIUS, an accounting protocol implemented by many.
Then to IPDR as Arek suggests. IPDR is many things in one. It comes out of the traditional Call Data Records, the file-based records used for encoding calls. It has turned into a more complex set of transport protocols, encoding (XML and the sucessor of CDRs: XDR), and schemas. I haven't looked at the details, but it probably supports some form of real-time billing (and thus authorization). In this respect it overlaps with Open Settlement Protocol (OSP, http://www.transnexus.com/White%20Papers/What%20is%20OSP.htm), which we already have a module for. OSP comes out of TISPAN, the ISP's standardization effort to adapt 3GPP IMS architecture to the ISP world. However, I see IPDR more as a back-end accounting specification, than a real-time settlement protocol. Here I agree with Jiri, any real-time elements of IPDR is natural to have as a SER module, however, the majority of the IPDR specification is concerned about a step that is outside SER, namely acocunting start/stop correlation, cleaning, and CDR/XDR/IPDR record generation. An IPDR accounting module would be possible,and probably needed if one wants to enable SER to send live IPDR data to an IPDR compliant CDR collector. So, anyone, feel free to implement an IPDR module ;-)
Arek:
Read, agreed. Years are passing and we still do not have proper IPDR compliant CDR collector. Every poor SER newbie have to write those little scripts to extract those date, time and duration of his calls. Let's give them proper CDR collector (IPDR recorder I mean) in real-time. The rest of IPDR, non-real time stuff - keep inside module as well of throw away from SER - to be decided.
Yep.
Greger:
Finally, to the d) interface, an SNMP module would probably be nice, but not enough. The trend is towards actual service probing where the full user experience is monitored (ex. automated calls measuring MOS). sipp can used for this (not MOS score), but at least doing a full call.
In addition, you have another interface for carrier infrastructures: g) Application servers (AS)
The standard interface for AS is SIP. ParlayX is used for this in the old telco world, but AFAIK it has a SIP interface as well. I cannot see why SER should implement ParlayX in a module?! Maybe you could enlighten me, Arek?!
Arek:
And let me enlighten you, Greger ;-) I've mentioned ParlayX Web services, to make it easy for Parlay and Web service geeks to build custom application servers. I found it to be a nice alternative, to have well documented standard interface from SER server towards Application Server. I gave an example of SER receiving a SIP call to a number, then preparing and sending Web service message to Application Server.
Hm, how many SER users have an application server that talks ParlayX?! What does ParlayX give that SIP does not (of course if the application server supports SIP)?
Upon receiving answer from Application server, SER servce could act on SIP message. Action taken could be one like: forward here, forward there or send him a '486 Busy Here'. I proposed that because I had a nice application server (with Web service interfaces) in my company and I was looking for some standard way to talk to this server. I though that: writing ParlayX Web service module for SER + modifying Application Server to have ParlayX compliant interfaces would be a really sexy solution to build applications like prepaid, prepaid with flat fee, multimedia services, etc.
I see why you need it if you have a legacy AS that does not support SIP. But that is backwards looking, if you ask me. How many telco application servers can be bought today that does not have SIP support? (or rather: would you like to buy...?)
Feel free to correct me if I'm wrong.
Greger:
ParlayX is a big thing and if SER is to be extended towards more of an application server, we need to think through what SER should be. SIP CGI is interesting for tinkerers, as well as many applications. JSLEE integration would be interesting, and of course ParlayX. (I here intentionally mix up actual application capabilities with interfaces towards applications servers...)
My conclusions so far:
- The XMLRPC front-end is good for tinkering, but it does not really
match what is available of tools and connecting systems in the two scenarios described above. A SOAP module, maybe even based on a standard(!), would simplify provisioning and user account settings
Arek:
YES it would. I really could send you today an WSDL of such service running at our premises for 2 consequent years. I just cannot do it because of confidentiality reasons. But will try to go with it to open source as soon as possible.
That's great.
Greger:
- I feel an IPDR module could be a good addition to SER, as long as it
does not try to do something a SIP server is not meant to be
Arek:
I Agree. Believe me, I don't want to make SER a washing machine with dryer either. I just want to give people a compliant CDR recorder, to have a nice basis to build their billing applications.
I agree, that is good.
Greger:
- SNMP would be a good addition
Arek:
Yes it would be. We are ready to work on MIB for that in our company (I got a really good specialist for SNMP here, next to my desk).
Bring him or her on! g-)
-
Andreas Granig -
Arek Bekiersz -
Arek Bekiersz -
David R. Kompel -
Dragos Vingarzan -
Erik Versaevel -
Greger V. Teigre -
Jiri Kuthan -
Juha Heinanen -
Klaus Darilion -
sip -
sip -
Weiter Leiter