Hi devs,
so, after many months, finally I found the time (and courage ;-) ) to do the necessary changes so that the Open IMS Core modules would also be runnable under sip-router. And well, kudos for the work that you've done so far! Adapting was really much more easy then I expected, due to the backwards compatibility that is still there. I was actually able so far to modify my modules so that they would be compilable under both sip-router/master and our ser_ims.
Until now I have managed to have the Diameter module and the I-CSCF working, with the rest hopefully coming in the next days. These fixes are already commited on our svn tree, but they are not really usable without some sip-router patches that I will talk about a bit later.
But first of all, let me tell you the overall picture of what we have changed and added to SER 2.1.0 (I know it's an ancient version, but it was more than enough for what we wanted to get).
Modules: * cdp - The CDiameterPeer * icscf - I-CSCF functions * isc - IMS Service Control - AS triggering and forwarding * mgcf - some helpers for making some MGCF nodes attached to softswitches (this is just an experimental collection of hacks) * pcscf - P-CSCF functions * scscf - S-CSCF functions
Other patches: * dialog - small typos * enum - support for query on the originating side, useful in PSTN inbound processing * ratelimit - a much extended ratelimit module, with multiple queues, dynamic limiting capabilities based on internal/external indicators, random retry-after capabilities, etc; which although sent towards the SER trunk, never made it * tls - some hashing functions * tm - not really sure for now, but I hope we can reconcile without changes on the sip-router side * Core - support for emergency URI, some mem logging improvements, etc.
I hope that we could integrate as much as possible from the changes that we did as really we would like to converge rather than go our own separate path. Sure, many parts are crazy-experimental and they would probably stay like that unless someone would like to do a serious re-implementation, but others I think would benefit outside of the IMS targets that we had.
And here comes my first issue then. Whether you guys would be open to host this in the same git repository or should we still maintain our own separate svn tree? I started by adding a modules_osims directory and putting there the differences. And so far it seems to be working for me locally. Sure that the crazy-experimental parts won't make it to master, but hopefully the Open IMS Core could leave entirely as just one modules directory inside sip-router tree.
If we'd stay separated, I would then try and find a way to link the git repository as external in our svn one and overwrite with our changes.
Then I am attaching some small patches for now, with more to come in the next days as I will progress:
- sip-router_modules_s_dialog.diff - just some typos in logging - sip-router_pt.diff - added a drop_my_process() function - in the cdp module (Diameter) we do have dynamic processes, which fork and exit distinctly from the ser ones, so we need this to clean-up. Without it, such usages would not be possible as the process table would fill and then new forks would be denied - I have commented the pt.c mailbox:///root/private/_mail/Mail/common/Sent?number=587016624#pt.c line 210. I am not sure if this is a bug or I just used the fork_process wrongly, but my process which was forked from a mod_init() not a mod_child_init() opens some sockets, which are mistakenly closed on fork_process() by the close_extra_socks(), which uses the pt[].unix_sock values from other processes, which overlap. So please advise on what would the best way be to still allow for the forked processes to open some sockets without them being closed on fork.
Both these patches are required in order to get the Diameter stack working.
Then there is this one that we use quite a lot for tracking memory leaks: http://tracker.iptel.org/browse/SER-224 . It's not there yet I guess, so should I resubmit it or forget about it? ;-)
I am looking forward to your feedback on how to proceed and on the attached patches.
Cheers, -Dragos
Hello Dragos,
On 23.07.2009 19:59 Uhr, Dragos Vingarzan wrote:
Hi devs,
so, after many months, finally I found the time (and courage ;-) ) to do the necessary changes so that the Open IMS Core modules would also be runnable under sip-router. And well, kudos for the work that you've done so far! Adapting was really much more easy then I expected, due to the backwards compatibility that is still there. I was actually able so far to modify my modules so that they would be compilable under both sip-router/master and our ser_ims.
Until now I have managed to have the Diameter module and the I-CSCF working, with the rest hopefully coming in the next days. These fixes are already commited on our svn tree, but they are not really usable without some sip-router patches that I will talk about a bit later.
But first of all, let me tell you the overall picture of what we have changed and added to SER 2.1.0 (I know it's an ancient version, but it was more than enough for what we wanted to get).
Modules:
- cdp - The CDiameterPeer
- icscf - I-CSCF functions
- isc - IMS Service Control - AS triggering and forwarding
- mgcf - some helpers for making some MGCF nodes attached to softswitches (this is just an experimental collection of hacks)
- pcscf - P-CSCF functions
- scscf - S-CSCF functions
quite a bunch of work, I would say. Can cdp be used for generic diameter AAA or is specific for ims?
Other patches:
- dialog - small typos
- enum - support for query on the originating side, useful in PSTN inbound processing
- ratelimit - a much extended ratelimit module, with multiple queues, dynamic limiting capabilities based on internal/external indicators, random retry-after capabilities, etc; which although sent towards the SER trunk, never made it
IIRC, openser imported the ratelimit from openimscore, can you check if includes the features you have? http://sip-router.org/docbook/sip-router/branch/master/modules_k/ratelimit/r...
- tls - some hashing functions
- tm - not really sure for now, but I hope we can reconcile without changes on the sip-router side
- Core - support for emergency URI,
What is this exactly? Any reference to ietf/itu specs?
some mem logging improvements, etc.
I hope that we could integrate as much as possible from the changes that we did as really we would like to converge rather than go our own separate path. Sure, many parts are crazy-experimental and they would probably stay like that unless someone would like to do a serious re-implementation, but others I think would benefit outside of the IMS targets that we had.
And here comes my first issue then. Whether you guys would be open to host this in the same git repository or should we still maintain our own separate svn tree?
I see no problem to include modules in the git, in a new directory, as you say below (maybe modules_i to follow the tradition :-) : k - kamailio, s - ser, i - ims).
I started by adding a modules_osims directory and putting there the differences. And so far it seems to be working for me locally. Sure that the crazy-experimental parts won't make it to master, but hopefully the Open IMS Core could leave entirely as just one modules directory inside sip-router tree.
If we'd stay separated, I would then try and find a way to link the git repository as external in our svn one and overwrite with our changes.
Then I am attaching some small patches for now, with more to come in the next days as I will progress:
- sip-router_modules_s_dialog.diff - just some typos in logging
This can be applied right away. I do not know who is in charge of the SER dialog module, if no one volunteers for it and you want to jump in, send me your ssh key and desired uername for git account.
- sip-router_pt.diff
- added a drop_my_process() function - in the cdp module (Diameter)
we do have dynamic processes, which fork and exit distinctly from the ser ones, so we need this to clean-up. Without it, such usages would not be possible as the process table would fill and then new forks would be denied
- I have commented the pt.c
mailbox:///root/private/_mail/Mail/common/Sent?number=587016624#pt.c line 210. I am not sure if this is a bug or I just used the fork_process wrongly, but my process which was forked from a mod_init() not a mod_child_init() opens some sockets, which are mistakenly closed on fork_process() by the close_extra_socks(), which uses the pt[].unix_sock values from other processes, which overlap. So please advise on what would the best way be to still allow for the forked processes to open some sockets without them being closed on fork.
I am not yet familiar with the new way SER manages the process table. Andrei, Jan or other ser developer may have more insights and guide here.
Both these patches are required in order to get the Diameter stack working.
Then there is this one that we use quite a lot for tracking memory leaks: http://tracker.iptel.org/browse/SER-224 . It's not there yet I guess, so should I resubmit it or forget about it? ;-)
I was looking to build such summary for kamailio, since current version is hard to track and creates quite big logs, which slows down the shut down. Can you create a new patch against sip-router core and attach to tracker: http://sip-router.org/tracker/
Thanks, Daniel
I am looking forward to your feedback on how to proceed and on the attached patches.
Cheers, -Dragos
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Daniel-Constantin Mierla wrote:
Hello Dragos,
On 23.07.2009 19:59 Uhr, Dragos Vingarzan wrote:
- tls - some hashing functions
- tm - not really sure for now, but I hope we can reconcile without changes on the sip-router side
- Core - support for emergency URI,
What is this exactly? Any reference to ietf/itu specs?
I guess he means support for service URNs, like urn:service:sos.fire, ... http://tools.ietf.org/html/rfc5031
regards klaus
El Jueves, 23 de Julio de 2009, Klaus Darilion escribió:
I guess he means support for service URNs, like urn:service:sos.fire, ... http://tools.ietf.org/html/rfc5031
Yeah, that kind of hyper-exotic and "cool" features coming from IETF who nobody will use/implement.
On 24.07.2009 0:02 Uhr, Iñaki Baz Castillo wrote:
El Jueves, 23 de Julio de 2009, Klaus Darilion escribió:
I guess he means support for service URNs, like urn:service:sos.fire, ... http://tools.ietf.org/html/rfc5031
Yeah, that kind of hyper-exotic and "cool" features coming from IETF who nobody will use/implement.
:-) -- now seems to be an implementation, if it is that.
Daniel
El Viernes, 24 de Julio de 2009, Daniel-Constantin Mierla escribió:
On 24.07.2009 0:02 Uhr, Iñaki Baz Castillo wrote:
El Jueves, 23 de Julio de 2009, Klaus Darilion escribió:
I guess he means support for service URNs, like urn:service:sos.fire, ... http://tools.ietf.org/html/rfc5031
Yeah, that kind of hyper-exotic and "cool" features coming from IETF who nobody will use/implement.
:-) -- now seems to be an implementation, if it is that.
AFAIK (but I could be wrong) URN are used by endpoints. Real usage example:
1) The user is in danger since the fridge is empty.
2) He takes his "IMS-ready" phone and looks for "Emergency" menu.
3) There he selects: "urn:ietf:params:urn:emergeny:human:no-food"
4) The hyper-intelligent phone sends automatically a PUBLISH to all its watchers containing a header "Accept: Burger" and body:
<dm:person id="hyper-cool-ietf-ims-oma-phone"> rpid:activities rpid:hungry/ </rpid:activities> dm:noteFood please</dm:note> </dm:person>
5) The phone also makes a call to all the proxies close to it with URI: "urn:ietf:params:urn:emergeny:human:no-food"
6) The proxy translates that URI into "sip:112@LEGACY_GW_IP" and routes the call.
7) When the emergency service answers the call, a voice + video + MSRP session is started. Nothing works due to NAT.
8) At the same time some IETF member is writting a draft about "urns" uri called: "draft-ietf-i-am-cool-because-i-write-drafts-secure-urn.txt"
LOL!
agree with you and for sure a lot of people don't even make the difference between 110 and 112. From what I can tell, it's more of a use for inter-emergency-services calls.
Yet it would be embarrassing if your SIP proxy would crash when it gets such a crazy URN from some emancipated client.
Cheers, -Dragos
On Fri, Jul 24, 2009 at 12:27 AM, Iñaki Baz Castillo <ibc@aliax.net mailto:ibc@aliax.net> wrote:
El Viernes, 24 de Julio de 2009, Daniel-Constantin Mierla escribió: > On 24.07.2009 0:02 Uhr, Iñaki Baz Castillo wrote: > > El Jueves, 23 de Julio de 2009, Klaus Darilion escribió: > >> I guess he means support for service URNs, like > >> urn:service:sos.fire, ... > >> http://tools.ietf.org/html/rfc5031 > > > > Yeah, that kind of hyper-exotic and "cool" features coming from IETF who > > nobody will use/implement. > > > :-) -- now seems to be an implementation, if it is that.
AFAIK (but I could be wrong) URN are used by endpoints. Real usage example:
1) The user is in danger since the fridge is empty.
2) He takes his "IMS-ready" phone and looks for "Emergency" menu.
3) There he selects: "urn:ietf:params:urn:emergeny:human:no-food"
4) The hyper-intelligent phone sends automatically a PUBLISH to all its watchers containing a header "Accept: Burger" and body:
<dm:person id="hyper-cool-ietf-ims-oma-phone"> rpid:activities rpid:hungry/ </rpid:activities> dm:noteFood please</dm:note> </dm:person>
5) The phone also makes a call to all the proxies close to it with URI: "urn:ietf:params:urn:emergeny:human:no-food"
6) The proxy translates that URI into "sip:112@LEGACY_GW_IP" and routes the call.
7) When the emergency service answers the call, a voice + video + MSRP session is started. Nothing works due to NAT.
8) At the same time some IETF member is writting a draft about "urns" uri called: "draft-ietf-i-am-cool-because-i-write-drafts-secure-urn.txt"
-- Iñaki Baz Castillo <ibc@aliax.net mailto:ibc@aliax.net>
_______________________________________________ sr-dev mailing list sr-dev@lists.sip-router.org mailto:sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
On Donnerstag, 23. Juli 2009, Daniel-Constantin Mierla wrote:
[..]
And here comes my first issue then. Whether you guys would be open to host this in the same git repository or should we still maintain our own separate svn tree?
I see no problem to include modules in the git, in a new directory, as you say below (maybe modules_i to follow the tradition :-) : k - kamailio, s - ser, i - ims).
Hi Dragos,
thanks for the update. :) Including this in our tree sounds ok to me as well. The (core) parts that can't be merged at the moment because they need some adaptions or are really a bit too experimental could be added to a separate branch, for example.
Cheers,
Henning
- sip-router_modules_s_dialog.diff - just some typos in logging
This can be applied right away. I do not know who is in charge of the SER dialog module, if no one volunteers for it and you want to jump in, send me your ssh key and desired uername for git account.
I guess nobody is really responsible for it, so go ahead.
Jan.
Hi Daniel,
On Thu, Jul 23, 2009 at 9:00 PM, Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> wrote:
quite a bunch of work, I would say. Can cdp be used for generic diameter AAA or is specific for ims?
No, the CDiameterPeer module is not specific and it can be used for anything that talks Diameter. The only IMS specific parts are a bunch of constants defined for the IMS interfaces in a header file, but of course there is space for other applications, which are of course welcome to it.
The cdp is a bit of a "server" itself that is forked from ser and can benefit from all the advantages that a ser process has. But it has it's own timer, workers, etc processes. The advantage of this is that it is quite easy (and we actually do provide an example here: http://svn.berlios.de/wsvn/openimscore/CDiameterPeer/trunk/#_CDiameterPeer_t... ) to get a standalone Diameter node on the same platform. This is useful for implementing the other ends of the AAA communication when you like the ser memory, locking, etc routines ;-) but don't need the SIP related stuff.
I have quite a high confidence in the stability of this module as we seldomly had issues with it. On the things to improve would be though: * replace the sempahores with something better - they tend to accumulate over crashes of ser * merge a branch that adds support for the authentication state machine and add the charging state machine - although I don't know if anyone would actually use this part and it would maybe just be overkill.
IIRC, openser imported the ratelimit from openimscore, can you check if includes the features you have? http://sip-router.org/docbook/sip-router/branch/master/modules_k/ratelimit/r...
Right, now I remember. So then one less thing to worry about. Sorry, I was only monitoring the ser tree.
* Core - support for emergency URI,
What is this exactly? Any reference to ietf/itu specs?
I am not really sure myself as somebody else did the implementation. But all the specs should be listed here: http://www.openimscore.org/emergency . And even for us this is still partly a branch with work in progress. The thing was that ser was even crashing when it got some of these, so in any case, even if not used, it would be a good patch.
I'll have to do a diff and find out what was exactly changes as there were to many things in the last 3-4 years. And we probably have some garbage through that. I'll send the patch after some cleanup.
- sip-router_modules_s_dialog.diff - just some typos in logging
This can be applied right away. I do not know who is in charge of the SER dialog module, if no one volunteers for it and you want to jump in, send me your ssh key and desired uername for git account.
Will do shortly. Will still have to get up to speed with git before pushing anything though.
- sip-router_pt.diff - added a drop_my_process() function - in the cdp module (Diameter) we do have dynamic processes, which fork and exit distinctly from the ser ones, so we need this to clean-up. Without it, such usages would not be possible as the process table would fill and then new forks would be denied - I have commented the pt.c mailbox:///root/private/_mail/Mail/common/Sent?number=587016624#pt.c line 210. I am not sure if this is a bug or I just used the fork_process wrongly, but my process which was forked from a mod_init() not a mod_child_init() opens some sockets, which are mistakenly closed on fork_process() by the close_extra_socks(), which uses the pt[].unix_sock values from other processes, which overlap. So please advise on what would the best way be to still allow for the forked processes to open some sockets without them being closed on fork.
I am not yet familiar with the new way SER manages the process table. Andrei, Jan or other ser developer may have more insights and guide here.
Partly the new process table management was sparked also by me, from the same requirements as now. Unfortunately for the first part not all of my patch made it and the other part was actually greatly improved by Andrei. If the case is that it won't be accepted that a process can dynamically terminate, then I'll have to redesign cdp. It's not a huge deal, but I'd rather not as it will just complicate things and I think that the process drop is not such a big deal anyway.
For the 2nd part, this was added later and I'm not sure if I did something wrong and the fork should've been done differently. But it looks like a potential bug anyway.
I was looking to build such summary for kamailio, since current version is hard to track and creates quite big logs, which slows down the shut down. Can you create a new patch against sip-router core and attach to tracker: http://sip-router.org/tracker/
Will do. I also added to our ser_ims some clean-up code that frees-up a lot of the core's memory on shutdown. Unfortunately I got stuck at the fixed parameters, which you can't really make sense of in the core in order to free them. So I guess there's need for some "unfix" functions in the modules.
Cheers, -Dragos
Dragos Vingarzan schrieb:
* Core - support for emergency URI,
What is this exactly? Any reference to ietf/itu specs?
I am not really sure myself as somebody else did the implementation. But all the specs should be listed here: http://www.openimscore.org/emergency . And even for us this is still partly a branch with work in progress. The thing was that ser was even crashing when it got some of these, so in any case, even if not used, it would be a good patch.
I'll have to do a diff and find out what was exactly changes as there were to many things in the last 3-4 years. And we probably have some garbage through that. I'll send the patch after some cleanup.
IIRC openser/ser can not parse service URNs. Usually, if a UAC initiates an emergency call, it looks like:
INVITE urn:service:sos SIP/2.0 Route: sip:uri-of-the-psap@example.com;lr From: sip:user@domain.com;tag=123 To: urn:service:sos ...
Thus, the proxy does not even have to route based on the service URN, but just on the pre-loaded route-set. Nevertheless, the proxy must not crash or generate errors if the RURI and From/To URI does contain a service URN. Actually From/To/RURI are allowed to contain any URIs (also http URIs) - thus sr should supports parsing of these.
regards klaus
You are right, but Ancuta, which developed the attached patch, tells me that the actual issue was when ser tried to generate an ACK on >299 responses for calls to such URNs.
Anyway, if we talk so much about this, it's not such a big deal, so here it is attached and here: http://svn.berlios.de/viewcvs/openimscore?view=rev&rev=631
Cheers, -Dragos
On Fri, Jul 24, 2009 at 2:17 PM, Klaus Darilion < klaus.mailinglists@pernau.at> wrote:
Dragos Vingarzan schrieb:
* Core - support for emergency URI,
What is this exactly? Any reference to ietf/itu specs?
I am not really sure myself as somebody else did the implementation. But all the specs should be listed here: http://www.openimscore.org/emergency. And even for us this is still partly a branch with work in progress. The thing was that ser was even crashing when it got some of these, so in any case, even if not used, it would be a good patch.
I'll have to do a diff and find out what was exactly changes as there were to many things in the last 3-4 years. And we probably have some garbage through that. I'll send the patch after some cleanup.
IIRC openser/ser can not parse service URNs. Usually, if a UAC initiates an emergency call, it looks like:
INVITE urn:service:sos SIP/2.0 Route: sip:uri-of-the-psap@example.com sip%3Auri-of-the-psap@example.com ;lr From: sip:user@domain.com sip%3Auser@domain.com;tag=123 To: urn:service:sos ...
Thus, the proxy does not even have to route based on the service URN, but just on the pre-loaded route-set. Nevertheless, the proxy must not crash or generate errors if the RURI and From/To URI does contain a service URN. Actually From/To/RURI are allowed to contain any URIs (also http URIs) - thus sr should supports parsing of these.
regards klaus
On Thu, Jul 23, 2009 at 7:59 PM, Dragos Vingarzandragos.vingarzan@gmail.com wrote:
Hi devs,
so, after many months, finally I found the time (and courage ;-) ) to do the necessary changes so that the Open IMS Core modules would also be runnable under sip-router. And well, kudos for the work that you've done so far! Adapting was really much more easy then I expected, due to the backwards compatibility that is still there. I was actually able so far to modify my modules so that they would be compilable under both sip-router/master and our ser_ims.
Wow, that's a bold move! Knowing that it is relatively easy to migrate such stuff to sip-router is very encouraging, thanks.
Until now I have managed to have the Diameter module and the I-CSCF working, with the rest hopefully coming in the next days. These fixes are already commited on our svn tree, but they are not really usable without some sip-router patches that I will talk about a bit later.
But first of all, let me tell you the overall picture of what we have changed and added to SER 2.1.0 (I know it's an ancient version, but it was more than enough for what we wanted to get).
Modules: * cdp - The CDiameterPeer * icscf - I-CSCF functions * isc - IMS Service Control - AS triggering and forwarding * mgcf - some helpers for making some MGCF nodes attached to softswitches (this is just an experimental collection of hacks) * pcscf - P-CSCF functions * scscf - S-CSCF functions
Other patches: * dialog - small typos
I guess this one should go in right away.
* enum - support for query on the originating side, useful in PSTN inbound processing
Could you send this patch to the mailing list and cc Juha?
* ratelimit - a much extended ratelimit module, with multiple queues, dynamic limiting capabilities based on internal/external indicators, random retry-after capabilities, etc; which although sent towards the SER trunk, never made it
I think your improvements made it to the kamailio version which now resides in sip-router/modules_k/ratelimit. Please try to see if it contains everything that you need. If yes then we can adopt the kamailio version of the module and junk the ser version (and move the module from modules_k to modules directory).
* tls - some hashing functions
I'd be happy to see how we can integrate it into the tls module if you send me a patch.
* tm - not really sure for now, but I hope we can reconcile without changes on the sip-router side * Core - support for emergency URI, some mem logging improvements, etc.
OK, I guess we would need more information on these two.
I hope that we could integrate as much as possible from the changes that we did as really we would like to converge rather than go our own separate path. Sure, many parts are crazy-experimental and they would probably stay like that unless someone would like to do a serious re-implementation, but others I think would benefit outside of the IMS targets that we had.
And here comes my first issue then. Whether you guys would be open to host this in the same git repository or should we still maintain our own separate svn tree? I started by adding a modules_osims directory and putting there the differences. And so far it seems to be working for me locally. Sure that the crazy-experimental parts won't make it to master, but hopefully the Open IMS Core could leave entirely as just one modules directory inside sip-router tree.
If we'd stay separated, I would then try and find a way to link the git repository as external in our svn one and overwrite with our changes.
Yeah, we would be happy to host your code in our git repository on sip-router, I think that this is a good idea, having all the stuff available through git would make it much easier to integrate stuff.
First of all I suggest that we start by importing the whole openims svn tree into a standalone project in our git repository. We can setup scripts that will track the svn tree for a while and import any changes done by your developers there until you announce that the project migrated. Having the whole SVN repository imported this way would allow us to merge selected parts (i.e. openims modules) into the main repository with full history. That would be the first step. I volunteer to set this up if you send me berlios usernames, full names and email addresses (needed to convert svn commit usernames to git commit logs with full names and emails) of all developers.
After that we can decide how to integrate it with sip-router. Let me just enumerate all the options that come to my mind: 1) We keep openims as a standalone project within our git repository 2) We import all the patches and extra modules on a separate branch within the main repository. 3) We add all extra openims modules into to the master branch into a new module directory such as modules_o and commit patches that those modules needs. 4) We drop new openims modules into the directory modules and integrate patches.
I think that there is no single best option, what would work best depends on the nature of the changes that we need to do to existing code (i.e. patches to sip-router core and modules) and also on your long term goals with the openims project.
Option 1) would work well if you plan to keep openims as a standalone project that draws code from sip-router. This option would give you considerable freedom to do changes to the code (even extensive ones) and yet remain synchronized with the main sip-router tree. We would setup a new project for openims in our git repository and set it up in such a way that it becomes easy to pull changes from the sip-router repository into the openims repository. Whenever you have something that is ready to be integrated into the main sip-router tree, you could simply let us know that there is something we should pull and once we do, the change would eventually propagate from sip-router to openims next time you pull. Git makes this mode of operation relatively painless, this is also roughly the model that the Linux kernel uses.
Option 2) would give you the same amount of freedom as 1), but it would probably be difficult to maintain a large number of changes on a separate branch for extended periods of time (my own experience). In my opinion this would only be suitable if the long term goal is to fully integrate openims with sip-router, but we are unsure how long it will take to integrate all necessary changes and patches.
Option 3) (a new module directory), I am not sure whether we really need to put openims modules into a separate module directory. Having two module directories is a temporary measure because we have two versions of most modules, but our long term goal was to migrate them where possible. I suppose that all openims modules are new, so we can either put them in "modules" directory right away, or in "modules_s" or "modules_k", depending on which version of existing modules their require.
Option 3) and 4) are suitable if your long term goals is to merge openims modules into sip-router and make them part of the sip-router project. Note that this means that openims developers would have to follow the sip-router way o development (releases, testing, code integration), no matter how bad or good it is :-).
I would personally favor 1) if you want to continue on your own or 4) if you want to integrate your stuff with sip-router, but let's see what other have to say. In any case we can start with importing svn into git, that's something we will need anyway.
Jan.
Hi Jan,
On Fri, Jul 24, 2009 at 11:07 AM, Jan Janak jan@ryngle.com wrote:
Wow, that's a bold move! Knowing that it is relatively easy to migrate such stuff to sip-router is very encouraging, thanks.
This in the modules' Makefile:
ifeq ($(CFG_NAME),sip-router) DEFS+=-DSER_MOD_INTERFACE endif
and then changes in the code based on the #ifdef SER_MOD_INTERFACE did it quite nicely. Also love the new modules partitioning as I got into a big mess before with just one directory.
- enum - support for query on the originating side, useful in PSTN inbound processing
Could you send this patch to the mailing list and cc Juha?
Attached.... but... this is one of our crazy-experimental stuff. I mean I haven't yet figured it our what to do with multiple response or what is the final list of things to change in the message, so it looks a bit messy to be merged in the master now.
As a bit of background info, this thing is useful when you have some sort of PSTN access gateway that would proxy all calls towards a SIP network. The node in charge of deciding the what-next part is called in IMS related terminology a TRansit Control Function (TRC or TRCF) and I have also attached a sample cfg for such a node. The advantage of doing also enum_orig() is that this node can decide, based on ENUM configuration, whether to trigger or not an originating leg processing in an originating domain proxy. Or putting it in another way, this provides the glue to have originating SIP services for PSTN UEs.
- ratelimit - a much extended ratelimit module, with multiple queues, dynamic limiting capabilities based on internal/external indicators, random retry-after capabilities, etc; which although sent towards the SER trunk, never made it
I think your improvements made it to the kamailio version which now resides in sip-router/modules_k/ratelimit. Please try to see if it contains everything that you need. If yes then we can adopt the kamailio version of the module and junk the ser version (and move the module from modules_k to modules directory).
yes, the kamailio version includes our changes and actually more. It should be then pretty safe to drop the ser version if the original owner won't complain. I guess the original functionality is still usable from there anyway.
- tls - some hashing functions
I'd be happy to see how we can integrate it into the tls module if you send me a patch.
Will come soon, but first I want to check it one more time as they are about 2 years old.
Yeah, we would be happy to host your code in our git repository on
sip-router, I think that this is a good idea, having all the stuff available through git would make it much easier to integrate stuff.
First of all I suggest that we start by importing the whole openims svn tree into a standalone project in our git repository. We can setup scripts that will track the svn tree for a while and import any changes done by your developers there until you announce that the project migrated. Having the whole SVN repository imported this way would allow us to merge selected parts (i.e. openims modules) into the main repository with full history. That would be the first step. I volunteer to set this up if you send me berlios usernames, full names and email addresses (needed to convert svn commit usernames to git commit logs with full names and emails) of all developers.
This sounds great. Will do. We also have some branches, but I think that those would be too much to import.
Then I hope that we could keep all the openimscore changes localized just to modules as really the core does not need to be different for IMS.
After that we can decide how to integrate it with sip-router. Let me just enumerate all the options that come to my mind:
- We keep openims as a standalone project within our git repository
- We import all the patches and extra modules on a separate branch
within the main repository. 3) We add all extra openims modules into to the master branch into a new module directory such as modules_o and commit patches that those modules needs. 4) We drop new openims modules into the directory modules and integrate patches.
I think that there is no single best option, what would work best depends on the nature of the changes that we need to do to existing code (i.e. patches to sip-router core and modules) and also on your long term goals with the openims project.
Option 1) would work well if you plan to keep openims as a standalone project that draws code from sip-router. This option would give you considerable freedom to do changes to the code (even extensive ones) and yet remain synchronized with the main sip-router tree. We would setup a new project for openims in our git repository and set it up in such a way that it becomes easy to pull changes from the sip-router repository into the openims repository. Whenever you have something that is ready to be integrated into the main sip-router tree, you could simply let us know that there is something we should pull and once we do, the change would eventually propagate from sip-router to openims next time you pull. Git makes this mode of operation relatively painless, this is also roughly the model that the Linux kernel uses.
I guess we don't really need this much liberty. Plus that the past experience with ser/ser_ims shows that we only once pulled the ser over our changes. Yes, it was probably harder to pull it with cvs/svn, but we've done our changes at the core already, mainly to have Diameter support, so the differences are not that big and hopefully reconcilable.
Option 2) would give you the same amount of freedom as 1), but it would probably be difficult to maintain a large number of changes on a separate branch for extended periods of time (my own experience). In my opinion this would only be suitable if the long term goal is to fully integrate openims with sip-router, but we are unsure how long it will take to integrate all necessary changes and patches.
Agree. Then from our side we'd just like to maintain a bunch of experimental modules, which hopefully would donate code to the base sr modules if necessary. We're not really building a commercially usable IMS thingy, so we also don't have the resource to maintain it properly in such scenarios.
Option 3) (a new module directory), I am not sure whether we really need to put openims modules into a separate module directory. Having two module directories is a temporary measure because we have two versions of most modules, but our long term goal was to migrate them where possible. I suppose that all openims modules are new, so we can either put them in "modules" directory right away, or in "modules_s" or "modules_k", depending on which version of existing modules their require.
But... it's so nice :-p. I really liked this because it really lets me have a separate experimental directory, which also adds some sense to what is for what. And then I just added my crazy-enum module there too and it did not interfere with the normal stable one. It looks and feels like a branch, but it's easier for development.
Option 3) and 4) are suitable if your long term goals is to merge openims modules into sip-router and make them part of the sip-router project. Note that this means that openims developers would have to follow the sip-router way o development (releases, testing, code integration), no matter how bad or good it is :-).
errrm ... can't really do unfortunately. So I would rather stick with the experimental flag and keep it separated so that you could also just ignore any complaints on the lists about crashes or issues when using this. We don't have any releases now as we can't really aford them. Our trunk is supposed to be "stable" and then we have branches for which there is absolutely no waranty.
Our long term goals are: - to merge back with the SER from which we diverged long time ago. We had some changes that we needed quickly and could not wait for them to be stabilized, accepted and pushed to SER. Now we know that some were useless, some not, some were accepted already and some are still on the waiting list. Hopefully we'll reconcile on the core changes and then just have all the IMS stuff in the additional modules. - to merge with Kamailio, because it's quite dumb some time that you need to keep and maintain several different ser binaries and configuration files for doing very similar tasks. For example, most of our users use also the Kamailio presence server, so they also have to checkout that and care about. - to upgrade to the latest interfaces for modules, databases and so on as to take advantage of the new features - to eliminate the sr sync issues so that our users could immediately benefit from the latest sr changes, without having to go through syncs every few months. - to keep being experimental and to try crazy stuff like the RFC5031 ;-) and hopefully provide some code for future stable modules.
I would personally favor 1) if you want to continue on your own or 4) if you want to integrate your stuff with sip-router, but let's see what other have to say. In any case we can start with importing svn into git, that's something we will need anyway.
So I guess then that we'd need something like 1, which would be great if the sync would be done automatically. Sure it would be lighter on our support efforts if we'd just pull regularly with the sr releases, but this did not work so well before.
I was thinking about some sort of reversed svn:externals solution, where we could have a branch like in 1, but all the parts around the modules_ims would be pulled on checkout from the master or from some link to whatever last release that was good enough for our purposes.
Thanx&Cheers, -Dragos
> * enum - support for query on the originating side, useful in PSTN > inbound processing Could you send this patch to the mailing list and cc Juha?
Attached.... but... this is one of our crazy-experimental stuff. I mean I haven't yet figured it our what to do with multiple response or what is the final list of things to change in the message, so it looks a bit messy to be merged in the master now.
As a bit of background info, this thing is useful when you have some sort of PSTN access gateway that would proxy all calls towards a SIP network. The node in charge of deciding the what-next part is called in IMS related terminology a TRansit Control Function (TRC or TRCF) and I have also attached a sample cfg for such a node. The advantage of doing also enum_orig() is that this node can decide, based on ENUM configuration, whether to trigger or not an originating leg processing in an originating domain proxy. Or putting it in another way, this provides the glue to have originating SIP services for PSTN UEs.
If you just want to make ENUM lookups on the caller's number you can also use K's enum module: http://www.kamailio.org/docs/modules/1.5.x/enum.html#id2467740
It is very flexible and allows you to make ENUM lookup with a pseudo variable - which can be anything.
regards klaus
Thanx for the tip! This would work then just fine and move everything to the script to do then the actual message modifications. But you know, I don't play well with AVPs and would rather write the module code ;-).
Cheers, -Dragos
Klaus Darilion wrote:
> * enum - support for query on the originating side, useful in
PSTN > inbound processing
Could you send this patch to the mailing list and cc Juha?
Attached.... but... this is one of our crazy-experimental stuff. I mean I haven't yet figured it our what to do with multiple response or what is the final list of things to change in the message, so it looks a bit messy to be merged in the master now.
As a bit of background info, this thing is useful when you have some sort of PSTN access gateway that would proxy all calls towards a SIP network. The node in charge of deciding the what-next part is called in IMS related terminology a TRansit Control Function (TRC or TRCF) and I have also attached a sample cfg for such a node. The advantage of doing also enum_orig() is that this node can decide, based on ENUM configuration, whether to trigger or not an originating leg processing in an originating domain proxy. Or putting it in another way, this provides the glue to have originating SIP services for PSTN UEs.
If you just want to make ENUM lookups on the caller's number you can also use K's enum module: http://www.kamailio.org/docs/modules/1.5.x/enum.html#id2467740
It is very flexible and allows you to make ENUM lookup with a pseudo variable - which can be anything.
regards klaus
Dragos Vingarzan schrieb:
> * ratelimit - a much extended ratelimit module, with multiple > queues, dynamic limiting capabilities based on internal/external > indicators, random retry-after capabilities, etc; which although > sent towards the SER trunk, never made it I think your improvements made it to the kamailio version which now resides in sip-router/modules_k/ratelimit. Please try to see if it contains everything that you need. If yes then we can adopt the kamailio version of the module and junk the ser version (and move the module from modules_k to modules directory).
yes, the kamailio version includes our changes and actually more. It should be then pretty safe to drop the ser version if the original owner won't complain. I guess the original functionality is still usable from there anyway.
There might be problems if the ratelimit module sends replies internally - because then it either have to use K's sl or ser's sl module.
regards klaus
On Fri, Jul 24, 2009 at 8:24 AM, Klaus Darilionklaus.mailinglists@pernau.at wrote:
Dragos Vingarzan schrieb:
> * ratelimit - a much extended ratelimit module, with multiple > queues, dynamic limiting capabilities based on internal/external > indicators, random retry-after capabilities, etc; which although > sent towards the SER trunk, never made it
I think your improvements made it to the kamailio version which now resides in sip-router/modules_k/ratelimit. Please try to see if it contains everything that you need. If yes then we can adopt the kamailio version of the module and junk the ser version (and move the module from modules_k to modules directory).
yes, the kamailio version includes our changes and actually more. It should be then pretty safe to drop the ser version if the original owner won't complain. I guess the original functionality is still usable from there anyway.
There might be problems if the ratelimit module sends replies internally - because then it either have to use K's sl or ser's sl module.
ratelimit sends replies internally. It would make sense to have a single sl module (just like tm). There are many modules that requires sl and all of them would benefit from having a single sl module.
Regards, Ovidiu Sas
Hi,
Am 24.07.09 13:04, schrieb Dragos Vingarzan:
Hi Jan,
On Fri, Jul 24, 2009 at 11:07 AM, Jan Janak <jan@ryngle.com mailto:jan@ryngle.com> wrote: > * enum - support for query on the originating side, useful in PSTN > inbound processing
Could you send this patch to the mailing list and cc Juha?
Attached.... but... this is one of our crazy-experimental stuff. I mean I haven't yet figured it our what to do with multiple response or what is the final list of things to change in the message, so it looks a bit messy to be merged in the master now.
As a bit of background info, this thing is useful when you have some sort of PSTN access gateway that would proxy all calls towards a SIP network. The node in charge of deciding the what-next part is called in IMS related terminology a TRansit Control Function (TRC or TRCF) and I have also attached a sample cfg for such a node. The advantage of doing also enum_orig() is that this node can decide, based on ENUM configuration, whether to trigger or not an originating leg processing in an originating domain proxy. Or putting it in another way, this provides the glue to have originating SIP services for PSTN UEs.
thanks for this interesting explanation. It seems to me that it might make sense to allow the enum module to use the select functions. This way the script writer could make enum queries for all URIs in a SIP message. The only "issue" would be that after querying the RURI you most likely would like to overwrite the existing RURI with the result. But if the result of the enum query would be stored in one or more AVPs it could even be manipulated in the script before it overwrites the existing RURI.
Juha what do you think? If you dont want to do it would you accept a patch from me which provides a new function for you enum module?
Cheers Nils
On Jul 24, 2009 at 11:07, Jan Janak jan@ryngle.com wrote:
[...]
Option 3) (a new module directory), I am not sure whether we really need to put openims modules into a separate module directory. Having two module directories is a temporary measure because we have two versions of most modules, but our long term goal was to migrate them where possible. I suppose that all openims modules are new, so we can either put them in "modules" directory right away, or in "modules_s" or "modules_k", depending on which version of existing modules their require.
I like the separate directory better and I think that after merging ser and kamailio modules we should split the modules in more directories (based on functionality and/or implementation status). It might make easier to manage the huge number of modules we have or at least to create different "distributions" or packages.
[...]
Andrei
On Jul 23, 2009 at 19:59, Dragos Vingarzan dragos.vingarzan@gmail.com wrote:
Hi devs,
[...]
- cdp - The CDiameterPeer
- icscf - I-CSCF functions
- isc - IMS Service Control - AS triggering and forwarding
- mgcf - some helpers for making some MGCF nodes attached to softswitches (this is just an experimental collection of hacks)
- pcscf - P-CSCF functions
- scscf - S-CSCF functions
Other patches:
- dialog - small typos
- enum - support for query on the originating side, useful in PSTN inbound processing
- ratelimit - a much extended ratelimit module, with multiple queues, dynamic limiting capabilities based on internal/external indicators, random retry-after capabilities, etc; which although sent towards the SER trunk, never made it
IIRC initially the author thought it was not ready for ser. After that it probably got lost in the noise (up until now I was sure it _was_ in ser :-)).
- tls - some hashing functions
- tm - not really sure for now, but I hope we can reconcile without changes on the sip-router side
- Core - support for emergency URI, some mem logging improvements, etc.
I remember the mem logging stuff. IIRC an extra config option for turning it on/off is all that's missing (there are people for which the restart speed matters a lot and in some corner cases the mem logging summary can eat some time). I'm not sure about he emergency URI, but we could add a compile def for it (or even support it if it doesn't affect anything else).
I hope that we could integrate as much as possible from the changes that we did as really we would like to converge rather than go our own separate path. Sure, many parts are crazy-experimental and they would probably stay like that unless someone would like to do a serious re-implementation, but others I think would benefit outside of the IMS targets that we had.
And here comes my first issue then. Whether you guys would be open to host this in the same git repository or should we still maintain our own separate svn tree? I started by adding a modules_osims directory and putting there the differences. And so far it seems to be working for me locally. Sure that the crazy-experimental parts won't make it to master, but hopefully the Open IMS Core could leave entirely as just one modules directory inside sip-router tree.
As Jan said there are several options. From my point of view the separate branch doesn't make much sense, it's equivalent to a separate git repo from the point of view of everyday work complexity, but a separate repo would be more clean. So (again from my point of view), it's either a separate git repo or a new modules_* directory. I like the modules_osims directory idea, but if you choose to go this path you should be aware that if you need to make changes to something not in that directory and in particular core+tm there might be some "conflicts". In particular you should take into account _slow_ response times and/or integration for new features in core or tm.
If we'd stay separated, I would then try and find a way to link the git repository as external in our svn one and overwrite with our changes.
Then I am attaching some small patches for now, with more to come in the next days as I will progress:
- sip-router_modules_s_dialog.diff - just some typos in logging
- sip-router_pt.diff
- added a drop_my_process() function - in the cdp module (Diameter)
we do have dynamic processes, which fork and exit distinctly from the ser ones, so we need this to clean-up. Without it, such usages would not be possible as the process table would fill and then new forks would be denied
I haven't looked at the patch yet, but why do you use ser fork_process() for these dynamic processes and not normal fork()?
- I have commented the pt.c
mailbox:///root/private/_mail/Mail/common/Sent?number=587016624#pt.c line 210. I am not sure if this is a bug or I just used the fork_process wrongly, but my process which was forked from a mod_init() not a mod_child_init() opens some sockets, which are mistakenly closed on fork_process() by the close_extra_socks(), which uses the pt[].unix_sock values from other processes, which overlap. So please advise on what would the best way be to still allow for the forked processes to open some sockets without them being closed on fork.
I don't see how it could happen. All the new forked processes inherit the unix socket used for communicating with tcp_main from all the processes forked before. close_extra_socks() closes all this unneeded fd in the new forked process. If you use fork_process from mod_init, the most likely the process_count is 0 at the time of forking => close_extra_socks() won't do anything. Could you provide me with more info on how exactly you fork the processes and what fds overlap?
Both these patches are required in order to get the Diameter stack working.
Then there is this one that we use quite a lot for tracking memory leaks: http://tracker.iptel.org/browse/SER-224 . It's not there yet I guess, so should I resubmit it or forget about it? ;-)
I am looking forward to your feedback on how to proceed and on the attached patches.
I'm for the separate module directory, with a suggestive name (like modules_osims, modules_ims, osims a.s.o.), but if you need to make a lot of experimental or bad-for-noims-use changes, the separate git repo it's probably better.
Andrei
Hi Andrei,
Andrei Pelinescu-Onciul wrote:
- ratelimit - a much extended ratelimit module, with multiple queues, dynamic limiting capabilities based on internal/external indicators, random retry-after capabilities, etc; which although sent towards the SER trunk, never made it
IIRC initially the author thought it was not ready for ser. After that it probably got lost in the noise (up until now I was sure it _was_ in ser :-)).
it seems that it made it though here and even with improvements as Ovidiu Sas imported in in Kamailio, so the functionality, although not code-line identical should all be in modules_k/ratelimit
- Core - support for emergency URI, some mem logging improvements, etc.
I remember the mem logging stuff. IIRC an extra config option for turning it on/off is all that's missing (there are people for which the restart speed matters a lot and in some corner cases the mem logging summary can eat some time).
I see. OK, then I'll come back with the added flag ;-). Do you think that it would be useful if we'd demand some unfix parameter functions for the memory that the module fixes do? AFAICT, this is still one of the main roadblocks (if not the only) in having an empty mem list on ser nice exit...
I'm not sure about he emergency URI, but we could add a compile def for it (or even support it if it doesn't affect anything else).
I already posted the patch once, so here it is again. Not a big deal, just some sensible support for parsing and then some more cases in some switches before declaring it as invalid. I guess it won't hurt to have it in even without flags.
As Jan said there are several options. From my point of view the separate branch doesn't make much sense, it's equivalent to a separate git repo from the point of view of everyday work complexity, but a separate repo would be more clean. So (again from my point of view), it's either a separate git repo or a new modules_* directory. I like the modules_osims directory idea, but if you choose to go this path you should be aware that if you need to make changes to something not in that directory and in particular core+tm there might be some "conflicts". In particular you should take into account _slow_ response times and/or integration for new features in core or tm.
I really hope that we won't make changes to core+tm... We did some for the core as we needed this for the diameter integration and there are some other stuff that we might be somehow in front, like the emergency URN. But this are small occurrences for which, like you said, I would not sacrifice the advantages of not being on a different branch of sr.
And after all... OSIMS is not much more than some other way of configuring and running sr... really, IMS not reinvent the wheel or anything :-p.
- sip-router_modules_s_dialog.diff - just some typos in logging
- sip-router_pt.diff
- added a drop_my_process() function - in the cdp module (Diameter)
we do have dynamic processes, which fork and exit distinctly from the ser ones, so we need this to clean-up. Without it, such usages would not be possible as the process table would fill and then new forks would be denied
I haven't looked at the patch yet, but why do you use ser fork_process() for these dynamic processes and not normal fork()?
well... I like to have all the advantages that a ser process has, like being able to process SIP messages, without having a big interface back to ser for pushing back events and actions.
- I have commented the pt.c
mailbox:///root/private/_mail/Mail/common/Sent?number=587016624#pt.c line 210. I am not sure if this is a bug or I just used the fork_process wrongly, but my process which was forked from a mod_init() not a mod_child_init() opens some sockets, which are mistakenly closed on fork_process() by the close_extra_socks(), which uses the pt[].unix_sock values from other processes, which overlap. So please advise on what would the best way be to still allow for the forked processes to open some sockets without them being closed on fork.
I don't see how it could happen. All the new forked processes inherit the unix socket used for communicating with tcp_main from all the processes forked before. close_extra_socks() closes all this unneeded fd in the new forked process. If you use fork_process from mod_init, the most likely the process_count is 0 at the time of forking => close_extra_socks() won't do anything. Could you provide me with more info on how exactly you fork the processes and what fds overlap?
Actually, now that I looked again, I see that I am only initializing some variables from mod_init() and then forking from mod_child_init(), but only from one from PROC_MAIN.
/** * Child init function. * - starts the DiameterPeer by forking the processes * @param rank - id of the child */ static int cdp_child_init( int rank ) { if (rank == PROC_MAIN) { LOG(L_INFO,"INFO:"M_NAME":cdp_child_init(): CDiameterPeer starting ...\n"); diameter_peer_start(0); LOG(L_INFO,"INFO:"M_NAME":cdp_child_init(): ... CDiameterPeer started\n"); } return 0; }
Then one of the processes will later on get a fd on a newly opened socket and then fork, which fd looks like already in pt[].unix_sock for another process. So these are the sockets for communicating with the "owner" of the SIP TCP sockets? So there are a lot of assumptions there about what you do with the forked processes... more than I actually took into consideration.
I'll try and debug this further too and come-up with a better description of the issue. In case you'd like to try it for yourself, the module cdp is here: http://svn.berlios.de/svnroot/repos/openimscore/ser_ims/trunk/modules/cdp/ . It should be compilable under sr with the previous sip-router_pt.diff which adds the drop_my_process() function.
I'm for the separate module directory, with a suggestive name (like modules_osims, modules_ims, osims a.s.o.), but if you need to make a lot of experimental or bad-for-noims-use changes, the separate git repo it's probably better.
I am leaning also towards the modules_osims. Then if you have an experimental flag that you could put on the entire directory, it would be great. For some parts, like the cdp, if the users like it we could just as well get it stabilized and moved out. But some other parts would probably be just too crazy to be used in a real exploitation environment and I hope that this won't cause support confusions. Ultimately, we just want that our users could pull the osims together with the latest sr and have cross benefits.
Cheers, -Dragos
On Jul 23, 2009 at 19:59, Dragos Vingarzan dragos.vingarzan@gmail.com wrote: [...]
- sip-router_pt.diff
- added a drop_my_process() function - in the cdp module (Diameter)
we do have dynamic processes, which fork and exit distinctly from the ser ones, so we need this to clean-up. Without it, such usages would not be possible as the process table would fill and then new forks would be denied
That's very problematic. It breaks process_no, my_pid() and the assumption that the process number does not change. These assumptions are used when doing statistics (e.g. tm): a shared mem array is created with one "entry" for each process. Each process updates its own entry (e.g. tm_stats[process_no].s.t_created++ ) without needing any locking or atomic op (which scale very badly on multi-cpus due to cacheline ping-pongs). The same assumptions are used in the shm malloc ng (only testing prototypes for now in ll_malloc) and might be used in the future for implementing a RCU like mechanism.
Andrei
I see... so it seems quite complicated to add all the required locks and to redesign the process_no and my_pid() for not much of a benefit. I did not see this before.
Well, if this is final and the conclusion is that the restrictions should be in place there on dynamically forked processes, then I'll start redesigning my module. It's a not a huge deal, but now the code is much clear, easier to manage and also potentially faster if each Diameter TCP connection has it's own process. But this is not a must and one universal acceptor/receiver forked at the beginning could do all the ops, much like the TCP structure from ser, right? Where there any performance issues due to some bottlenecks or something like that?
(I'll probably still keep also my original design too for when you use this CDiameterPeer standalone, outside of ser...)
Cheers, -Dragos
Andrei Pelinescu-Onciul wrote:
On Jul 23, 2009 at 19:59, Dragos Vingarzan dragos.vingarzan@gmail.com wrote: [...]
- sip-router_pt.diff
- added a drop_my_process() function - in the cdp module (Diameter)
we do have dynamic processes, which fork and exit distinctly from the ser ones, so we need this to clean-up. Without it, such usages would not be possible as the process table would fill and then new forks would be denied
That's very problematic. It breaks process_no, my_pid() and the assumption that the process number does not change. These assumptions are used when doing statistics (e.g. tm): a shared mem array is created with one "entry" for each process. Each process updates its own entry (e.g. tm_stats[process_no].s.t_created++ ) without needing any locking or atomic op (which scale very badly on multi-cpus due to cacheline ping-pongs). The same assumptions are used in the shm malloc ng (only testing prototypes for now in ll_malloc) and might be used in the future for implementing a RCU like mechanism.
Andrei
... and I have a feeling that this will probably also fix the bad-assumptions that I had on the close_extra_socks(), if I only fork on mod_child_init() and not later.
Cheers, -Dragos
On Wed, Jul 29, 2009 at 2:58 PM, Dragos Vingarzan < dragos.vingarzan@gmail.com> wrote:
I see... so it seems quite complicated to add all the required locks and to redesign the process_no and my_pid() for not much of a benefit. I did not see this before.
Well, if this is final and the conclusion is that the restrictions should be in place there on dynamically forked processes, then I'll start redesigning my module. It's a not a huge deal, but now the code is much clear, easier to manage and also potentially faster if each Diameter TCP connection has it's own process. But this is not a must and one universal acceptor/receiver forked at the beginning could do all the ops, much like the TCP structure from ser, right? Where there any performance issues due to some bottlenecks or something like that?
(I'll probably still keep also my original design too for when you use this CDiameterPeer standalone, outside of ser...)
Cheers, -Dragos
Andrei Pelinescu-Onciul wrote:
On Jul 23, 2009 at 19:59, Dragos Vingarzan dragos.vingarzan@gmail.com wrote: [...]
- sip-router_pt.diff
- added a drop_my_process() function - in the cdp module (Diameter) we
do have dynamic processes, which fork and exit distinctly from the ser ones, so we need this to clean-up. Without it, such usages would not be possible as the process table would fill and then new forks would be denied
That's very problematic. It breaks process_no, my_pid() and the assumption that the process number does not change. These assumptions are used when doing statistics (e.g. tm): a shared mem array is created with one "entry" for each process. Each process updates its own entry (e.g. tm_stats[process_no].s.t_created++ ) without needing any locking or atomic op (which scale very badly on multi-cpus due to cacheline ping-pongs). The same assumptions are used in the shm malloc ng (only testing prototypes for now in ll_malloc) and might be used in the future for implementing a RCU like mechanism.
Andrei
On Jul 29, 2009 at 14:58, Dragos Vingarzan dragos.vingarzan@gmail.com wrote:
I see... so it seems quite complicated to add all the required locks and to redesign the process_no and my_pid() for not much of a benefit. I did not see this before.
Well, if this is final and the conclusion is that the restrictions should be in place there on dynamically forked processes, then I'll start redesigning my module. It's a not a huge deal, but now the code is much clear, easier to manage and also potentially faster if each Diameter TCP connection has it's own process.
We might be able to add some limited dynamic processes support. It depends a lot on when you fork and what do you want to be able to do from the dynamically forked processes. For example we could make drop_my_process() only mark an entry in the process table as empty and make fork_process() search the table for empty entries. This would keep process_no valid, but it won't allow tcp use for processes forked after startup (you probably already have this problem) and it might have some strange side effects with statistics (a dyn. forked process might "inherit" the stats of another terminated dyn. forked process).
But this is not a must and one universal acceptor/receiver forked at the beginning could do all the ops, much like the TCP structure from ser, right? Where there any performance issues due to some bottlenecks or something like that?
There are 2 possibilities: - 1 process that handles all the I/O (based on epoll/kqueue/poll/sigio). This is fast but does not scale well with the number of cpus. - 1 process that only accept()s new connections and then sends them to some workers (similar to ser tcp). This is fast and scales well and doesn't have the disadvantage of running one process or thread for each connection. The main disadvantage is much more complex code.
(I'll probably still keep also my original design too for when you use this CDiameterPeer standalone, outside of ser...)
Andrei
Andrei Pelinescu-Onciul wrote:
On Jul 29, 2009 at 14:58, Dragos Vingarzan dragos.vingarzan@gmail.com wrote:
I see... so it seems quite complicated to add all the required locks and to redesign the process_no and my_pid() for not much of a benefit. I did not see this before.
Well, if this is final and the conclusion is that the restrictions should be in place there on dynamically forked processes, then I'll start redesigning my module. It's a not a huge deal, but now the code is much clear, easier to manage and also potentially faster if each Diameter TCP connection has it's own process.
We might be able to add some limited dynamic processes support. It depends a lot on when you fork and what do you want to be able to do from the dynamically forked processes. For example we could make drop_my_process() only mark an entry in the process table as empty and make fork_process() search the table for empty entries. This would keep process_no valid, but it won't allow tcp use for processes forked after startup (you probably already have this problem) and it might have some strange side effects with statistics (a dyn. forked process might "inherit" the stats of another terminated dyn. forked process).
I don't really want to cause some nasty hack just for this, but if it works and the performance is better then maybe it's worth.
In short the cdp module works like this: 1. Initialize on module init, but don't fork 2. On module child init, for rank==PROC_MAIN, fork_process() multiple times for the following: - 1x acceptor, for all the accepting TCP sockets - Nx workers, which will actually process the incoming Diameter messages, after they've been received. Being ser processes, they can do all that a ser process can. - 1x timer 3. Later on, when connections are established with other Diameter peers, fork_process() is called from the acceptor process on incoming connection or from the timer process on outgoing ones: - 1xreceiver per each peer, which will receive all incoming messages and pass them immediately to a task queue for the workers. Also has a named pipe, where pointers to shm alloced Diameters to be sent out are signaled. On disconnection, the process is terminated
Generally the receivers are pretty light and don't do too much: - watching the tcp socket - receiving diameter messaging, doing the quick binary decode and putting the message in the task queue for the workers - in case the message is part of the base protocol, run it through a simple state machine - watch a named pipe for signaling messages to be sent out. With this in place, any ser process can send out Diameter messages.
The trouble that we have now is in these receiver processes, which fork later from the acceptor or timer and also could terminate on disconnection, thus being now a bit dynamic during the execution time. Only marking I don't know if it's a clean and safe procedure...
But this is not a must and one universal acceptor/receiver forked at the beginning could do all the ops, much like the TCP structure from ser, right? Where there any performance issues due to some bottlenecks or something like that?
There are 2 possibilities:
- 1 process that handles all the I/O (based on epoll/kqueue/poll/sigio). This is fast but does not scale well with the number of cpus.
- 1 process that only accept()s new connections and then sends them to some workers (similar to ser tcp). This is fast and scales well and doesn't have the disadvantage of running one process or thread for each connection.
The main disadvantage is much more complex code.
The first fix solution would then be to make a single receiver process forked at startup, or a combine acceptor/receiver. My initial reasoning for forking was that each connection would've got a dedicated process and as such multiple connections won't be bottlenecked by busy interfaces.
The Diameter connections are then more stable in time than those from SIP (they are always kept alive between peers). So I don't know if I understand exactly how your 2nd suggestion would work... I have now the workers and I could make a static pool of receivers or reuse the workers. Then I should pass the descriptors from accept() between the two processes. This does not seem to me as being very standard over different kernels, so how does ser do it? (could you please just point me to the lines of code that do the descriptor exchanges?)
This won't be that bad in the end, as anyway in a well provisioned environment, each peer should have pre-configured all it's possible and trusted peers. Then I would pre-fork a receiver process on startup for each, plus one for all the other "unknown" eventual peers.
For the close_extra_socks() issue, now the acceptor/timer, before forking, opens the named pipe, which had a fd the same as another unix_sock from another process and thus got closed on fork. I could maybe open it after the fork, although this won't be just as safe and I'll need to double check it... But without the assumption that all forks are only done at start-up, this looked a bit like a bug.
Cheers, -Dragos