Hello all,
I’m using two Kamailio Presence Servers that are also XCAP Servers for the presence and resources management and the Kamailio Dispatcher in order to balance the SIP requestes coming from SIP Client.
Then I’m trying to configure a HTTP load–balancer for the xcap/http traffic.
I’m testing both Apache and INginx HTTP proxy but I’m facing with a problem related with the Xcap authentication:
when a client sends the first xcap request to the http load-balancer, it forwards the request to one of the configured Xcap Server that replies with the 401 response in order to let the client authenticates. So, the client sends the second request with the Digest authentication to the http load-balancer. But, sometimes, the http load-balancer forwards the authenticate request to the second Xcap Server that is not able to authenticate the request. Then the client receive the “Unauthorized” response…
Could you please give us some hints about this problem?
Thanks in advance,
laura
Hello,
On 6/28/11 6:33 PM, laura testi wrote:
Hello all,
I’m using two Kamailio Presence Servers that are also XCAP Servers for the presence and resources management and the Kamailio Dispatcher in order to balance the SIP requestes coming from SIP Client.
Then I’m trying to configure a HTTP load–balancer for the xcap/http traffic.
I’m testing both Apache and INginx HTTP proxy but I’m facing with a problem related with the Xcap authentication:
when a client sends the first xcap request to the http load-balancer, it forwards the request to one of the configured Xcap Server that replies with the 401 response in order to let the client authenticates. So, the client sends the second request with the Digest authentication to the http load-balancer. But, sometimes, the http load-balancer forwards the authenticate request to the second Xcap Server that is not able to authenticate the request. Then the client receive the “Unauthorized” response…
Could you please give us some hints about this problem?
you have to set the same value to secret parameter of auth module: http://kamailio.org/docs/modules/stable/modules/auth.html#auth.secret
Also be sure the two servers have the same time (use ntp or so) and the one_time_nonce parameter to auth module is off (not enabled).
Then it should work.
Cheers, Daniel
Thank you very much Daniel! It works!
Following your suggestion, I have set the following parameters in the kamailio.cfg in both servers:
modparam("auth", "secret", "samesecretparam") modparam("auth", "nonce_expire", 1800) modparam("auth", "nonce_auth_max_drift", 300) modparam("auth", "one_time_nonce", 0)
Now all xcap requests are authenticated successfully and works! Thanks again!
After some tests, now we are facing another problem with multiple presence servers integrated with XCAP servers in integrated mode. We have configured 2 kamailio servers as both presence server and xcap server with the same backend DB in another machine (thank you for your great guide on http://kb.asipto.com/kamailio:presence:k31-made-simple). In front we have setup a dispatcher for SIP load balancing and apache for http (xcap) load balancing.
It seems the presence server handle the publish/subscribe/notify events with the following tables: - active_watchers (using local cache and update the local cache to db periodically) - presentity (using local cache and update to db immediately) - watchers (db) - xcap (db)
we have meet 2 problems:
1. when a subscription is created in one server (in active_watchers in local cache) and the publish related to the same subscription goes to another server (presentity), with fallback to db, only the first publish works (it generates a notify with C-SEQ increased by one). When it receives another publish, it always generate the notify with the same C-SEQ, and the SIP Client ignore it. The problem is that the subscription is in the cache of another server and it periodically update it to db. When the publish related to this subscription goes to another server, the C-SEQ in the local cache is not increased. To workaround it, we have configure the Dispatcher to use hashing over URI for all presence related messages. In this way, it seems all messages related to the same subscription go to the same presence server, so the c-seq is increased correctly. Is it correct way to work with multiple presence server? 2. pres_refresh_watchers triggered by xcap message in another presence server: with the hashing over to uri in dispacther workaround, it seems solve the first problem for SIP/SIMPLE messages, but we have the same kind of problem for xcap message. For example, a subscription is in a local cache of one server, and the incomming xcap message related to the same subscription goes to another server, and this message trigger the pres_update_watchers pres_refresh_watchers presence functions from the configuration script in the server where there is no subscription in the local cache, then it send the wrong notify message. This can happen when a user add/remove a contact, and the SUBSCRIBE goes to one server and XCAP PUT goes to another server. Unfortunately there is no DB mode only in PRESENCE module like REGISTRAR module. The fallback to db can't help either for point 1 or for point 2. Can you help please?
Thank you in advanced!
Best Regards, Laura
On Tue, Jun 28, 2011 at 8:43 PM, Daniel-Constantin Mierla <miconda@gmail.com
wrote:
Hello,
On 6/28/11 6:33 PM, laura testi wrote:
Hello all,
I’m using two Kamailio Presence Servers that are also XCAP Servers for the presence and resources management and the Kamailio Dispatcher in order to balance the SIP requestes coming from SIP Client.
Then I’m trying to configure a HTTP load–balancer for the xcap/http traffic.
I’m testing both Apache and INginx HTTP proxy but I’m facing with a problem related with the Xcap authentication:
when a client sends the first xcap request to the http load-balancer, it forwards the request to one of the configured Xcap Server that replies with the 401 response in order to let the client authenticates. So, the client sends the second request with the Digest authentication to the http load-balancer. But, sometimes, the http load-balancer forwards the authenticate request to the second Xcap Server that is not able to authenticate the request. Then the client receive the “Unauthorized” response…
Could you please give us some hints about this problem?
you have to set the same value to secret parameter of auth module: http://kamailio.org/docs/modules/stable/modules/auth.html#auth.secret
Also be sure the two servers have the same time (use ntp or so) and the one_time_nonce parameter to auth module is off (not enabled).
Then it should work.
Cheers, Daniel
-- Daniel-Constantin Mierla -- http://www.asipto.comhttp://linkedin.com/in/miconda -- http://twitter.com/miconda
On Thursday 30 June 2011, laura testi wrote:
[...] pres_refresh_watchers triggered by xcap message in another presence server: with the hashing over to uri in dispacther workaround, it seems solve the first problem for SIP/SIMPLE messages, but we have the same kind of problem for xcap message. For example, a subscription is in a local cache of one server, and the incomming xcap message related to the same subscription goes to another server, and this message trigger the pres_update_watchers pres_refresh_watchers presence functions from the configuration script in the server where there is no subscription in the local cache, then it send the wrong notify message. This can happen when a user add/remove a contact, and the SUBSCRIBE goes to one server and XCAP PUT goes to another server. Unfortunately there is no DB mode only in PRESENCE module like REGISTRAR module. The fallback to db can't help either for point 1 or for point 2. Can you help please?
Hi laura,
i can't comment on the other question, but Marius B did some presence extensions in master branch a few month ago, that the presence module now supports a db_mode module parameter including DB_ONLY. It will be in kamailio 3.2, maybe you can have a look to the master branch and backport it, if you need it faster.
Best regards,
Henning
Hi Henning, Thank you very much for your reply. I'll check it.
Best Regards, Laura
On Fri, Jul 1, 2011 at 5:02 PM, Henning Westerholt < henning.westerholt@1und1.de> wrote:
On Thursday 30 June 2011, laura testi wrote:
[...] pres_refresh_watchers triggered by xcap message in another presence
server:
with the hashing over to uri in dispacther workaround, it seems solve the first problem for SIP/SIMPLE messages, but we have the same kind of problem for xcap message. For example, a subscription is in a local cache of one server, and the incomming xcap message related to the same subscription goes to another server, and this message trigger the pres_update_watchers pres_refresh_watchers presence functions from the configuration script in the server where there is no subscription in the local cache, then it send the wrong notify message. This can happen when
a
user add/remove a contact, and the SUBSCRIBE goes to one server and XCAP PUT goes to another server. Unfortunately there is no DB mode only in PRESENCE module like REGISTRAR module. The fallback to db can't help either for point 1 or for point 2. Can you help please?
Hi laura,
i can't comment on the other question, but Marius B did some presence extensions in master branch a few month ago, that the presence module now supports a db_mode module parameter including DB_ONLY. It will be in kamailio 3.2, maybe you can have a look to the master branch and backport it, if you need it faster.
Best regards,
Henning
Hi, we have checked the presence extension for DB_MODE only in the master branch. But it's not complete DB_ONLY mode, it seems that it still uses the cache for active_watchers in write through mode. Is there any reason for it? That can still make problem for the scalability for the multiple presence servers, because if an presence event (either SIP or XCAP) of an active subscription in the cache of one presence server, goes to another presence server, can still cause the problem even the DB is sync with the cache.
Best Regards, Laura
On Fri, Jul 1, 2011 at 5:02 PM, Henning Westerholt < henning.westerholt@1und1.de> wrote:
On Thursday 30 June 2011, laura testi wrote:
[...] pres_refresh_watchers triggered by xcap message in another presence
server:
with the hashing over to uri in dispacther workaround, it seems solve the first problem for SIP/SIMPLE messages, but we have the same kind of problem for xcap message. For example, a subscription is in a local cache of one server, and the incomming xcap message related to the same subscription goes to another server, and this message trigger the pres_update_watchers pres_refresh_watchers presence functions from the configuration script in the server where there is no subscription in the local cache, then it send the wrong notify message. This can happen when
a
user add/remove a contact, and the SUBSCRIBE goes to one server and XCAP PUT goes to another server. Unfortunately there is no DB mode only in PRESENCE module like REGISTRAR module. The fallback to db can't help either for point 1 or for point 2. Can you help please?
Hi laura,
i can't comment on the other question, but Marius B did some presence extensions in master branch a few month ago, that the presence module now supports a db_mode module parameter including DB_ONLY. It will be in kamailio 3.2, maybe you can have a look to the master branch and backport it, if you need it faster.
Best regards,
Henning
On Friday 15 July 2011, laura testi wrote:
we have checked the presence extension for DB_MODE only in the master branch. But it's not complete DB_ONLY mode, it seems that it still uses the cache for active_watchers in write through mode. Is there any reason for it? That can still make problem for the scalability for the multiple presence servers, because if an presence event (either SIP or XCAP) of an active subscription in the cache of one presence server, goes to another presence server, can still cause the problem even the DB is sync with the cache.
Hello Marius,
maybe you can comment a bit on this?
Thanks and best regards,
Henning
Hello,
In the DB_ONLY mode, every SUBSCRIBE request will trigger a database operation immediately (either insert or update). Also, the SUBSCRIPTION will always be matched from the database. It behaves exactly as the name implies (write-through).
If it doesn't then that is a bug. Can you please tell me more about the scenario you are using? Is there any usecase in which you can see the subscription in memory but not in the active_watchers table?
Regards, Marius
________________________________ From: Henning Westerholt henning.westerholt@1und1.de To: "Bucur, Marius Ovidiu" marius.bucur@1and1.ro Cc: "sr-users@lists.sip-router.org" sr-users@lists.sip-router.org; laura testi lau.testi@gmail.com Sent: Friday, July 15, 2011 5:58 PM Subject: Re: [SR-Users] xcap server - http load–balancer: xcap authentication problem
On Friday 15 July 2011, laura testi wrote:
we have checked the presence extension for DB_MODE only in the master branch. But it's not complete DB_ONLY mode, it seems that it still uses the cache for active_watchers in write through mode. Is there any reason for it? That can still make problem for the scalability for the multiple presence servers, because if an presence event (either SIP or XCAP) of an active subscription in the cache of one presence server, goes to another presence server, can still cause the problem even the DB is sync with the cache.
Hello Marius,
maybe you can comment a bit on this?
Thanks and best regards,
Henning
_______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi Marius, Thank you very much for your explanations. I see your answer only in the list. Here I copy to here for the convenience: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- Hello,
In the DB_ONLY mode, every SUBSCRIBE request will trigger a database operation immediately (either insert or update). Also, the SUBSCRIPTION will always be matched from the database. It behaves exactly as the name implies (write-through).
If it doesn't then that is a bug. Can you please tell me more about the scenario you are using? Is there any usecase in which you can see the subscription in memory but not in the active_watchers table?
Regards, Marius ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The scenario wehave tested is like that we have multiple presence servers and a share DB in the separate DB server. The incoming SIP/SIMPLE traffic are dispatched with the Dispatcher. The problem we found is that the write-through mechanism guarantee the active_watchers cache is synched to DB immediately when it change like your explanation; but not vice versa. For example, when the active_watchers table in DB is changed by one presence server via the same write-through mechanism, it seems there is not possible to populate the changes in DB back to cache of another presence server. That means the cache in the different presence servers are not synchronized. It will required complex replications to have cache in synched.
I'll try to explain it by the following use case we have tested: Test Environments: - SIP clients: (jitsi) - user1 (has user2 as a contact, user 2 is watcher of user1 with the state=1) - user2 (has user1 as a contact, user 1 is watcher of user2 withe the state=1) - SIP server (kamailio3.1.4) - 1 x dispatcher (routing based call id) - 2 x presence servers (PS1 and PS2) - DB server: - 1 x backend DB server for SIP servers
Test Case (user2 is already login and the subscription to user1 is in one of the 2 PS cache and in DB): 1) when user1 login, it send SUBSCRIBE presence event of user2 to the Dispatcher, and Dispatcher forward it to PS1 - PS1 write the subscription (user1->user2 with CSEQ=1) to active_watchers cache then active_wacthers table in DB immediately (write-through) - PS1 send the NOTIFY of the user2 presence status to user1 and increase the CSEQ by 1 (becomes 2) to both cache and DB (write-through) 2) user1 receive the NOTIFY and check the CSEQ (the first time), and update the user2's status 3) user2 change it's presence status and send the PUBLISH to the dispatcher, and Dispatcher forward it to PS2 - PS2 call handle_publish, it look-ups to the local cache for the active subscription and not found, then fallback to DB and found it, it increase the CSEQ by 1 (becomes 3 in DB) and send the NOTIFY to user1 with CSEQ=3 (here is the problem, the CSEQ of the same subscription in the PS1's cache is still equal 2) 4) the user1's client receive the NOTIFY and check the CSEQ if it's greater than the previous one, it's, then update the Presence Status of user2 5) user2 change it's presence status again, this time the Dispatcher forward the PUBLISH to PS1 - PS1 call handle_publish, it look-ups to the local cache for the active subscription and found it with CSEQ=2! It increase the CSEQ by 1 (becomes 3 in the cache and update the value to DB), and send the NOTIFY to user1 with CSEQ=3 again. 6) user1's client receive the NOTIFY and check the CSEQ if it's greater than the previous one, it's not, then it ignore it, the presen status of user2 is not updated in the user1's client!
This problem can be solved if we have pure DB mode only. In that case, the CSEQ is always increased for the successive NOTIFY of the same subscription.
It would be nice to keep the new Write-Through DB Mode in the presence module, and add a Pure DB mode, just like the usrloc module?
Just in case, in usrloc module, it miss the fallback to DB mode but has write-through and pure db mode ;-)
Best Regards, Laura
On Fri, Jul 15, 2011 at 4:58 PM, Henning Westerholt henning.westerholt@1und1.de wrote:
On Friday 15 July 2011, laura testi wrote:
we have checked the presence extension for DB_MODE only in the master branch. But it's not complete DB_ONLY mode, it seems that it still uses the cache for active_watchers in write through mode. Is there any reason for it? That can still make problem for the scalability for the multiple presence servers, because if an presence event (either SIP or XCAP) of an active subscription in the cache of one presence server, goes to another presence server, can still cause the problem even the DB is sync with the cache.
Hello Marius,
maybe you can comment a bit on this?
Thanks and best regards,
Henning
Hello Laura,
Thank you for your email and providing the full test-case. You are right, there is a certain issue (that is also mentioned in another thread - "DB_ONLY mode in presence module"). The thing is, that there are two caches
- a subscription hashtable, maintaining information about active subscripions - a presentity hashtable, maintaining information about presence entities (presentities, entities that send PUBLISH requests)
The db_only mode acts only on the first type of cache - subscriptions. It really makes sense having the db_only mode act also on the second cache (the presentity cache), hence your usecase is very common. There are some cases when you only need a db_only mode only for subscriptions (like when PUBLISH requests are generated precisely on each presence server machines) but of course I think it is a good compromise to disable caching also for the "presentity" table when running in db_only mode.
I will make a patch this evening with this change, and commit it on the master branch. If anyone has other suggestions, please let me know.
Cheers, Marius
________________________________ From: laura testi lau.testi@gmail.com To: Henning Westerholt henning.westerholt@1und1.de; "Bucur, Marius Ovidiu" marius.bucur@1and1.ro Cc: "sr-users@lists.sip-router.org" sr-users@lists.sip-router.org Sent: Wednesday, July 20, 2011 2:11 PM Subject: Re: [SR-Users] xcap server - http load–balancer: xcap authentication problem
Hi Marius, Thank you very much for your explanations. I see your answer only in the list. Here I copy to here for the convenience: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- Hello,
In the DB_ONLY mode, every SUBSCRIBE request will trigger a database operation immediately (either insert or update). Also, the SUBSCRIPTION will always be matched from the database. It behaves exactly as the name implies (write-through).
If it doesn't then that is a bug. Can you please tell me more about the scenario you are using? Is there any usecase in which you can see the subscription in memory but not in the active_watchers table?
Regards, Marius ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The scenario wehave tested is like that we have multiple presence servers and a share DB in the separate DB server. The incoming SIP/SIMPLE traffic are dispatched with the Dispatcher. The problem we found is that the write-through mechanism guarantee the active_watchers cache is synched to DB immediately when it change like your explanation; but not vice versa. For example, when the active_watchers table in DB is changed by one presence server via the same write-through mechanism, it seems there is not possible to populate the changes in DB back to cache of another presence server. That means the cache in the different presence servers are not synchronized. It will required complex replications to have cache in synched.
I'll try to explain it by the following use case we have tested: Test Environments: - SIP clients: (jitsi) - user1 (has user2 as a contact, user 2 is watcher of user1 with the state=1) - user2 (has user1 as a contact, user 1 is watcher of user2 withe the state=1) - SIP server (kamailio3.1.4) - 1 x dispatcher (routing based call id) - 2 x presence servers (PS1 and PS2) - DB server: - 1 x backend DB server for SIP servers
Test Case (user2 is already login and the subscription to user1 is in one of the 2 PS cache and in DB): 1) when user1 login, it send SUBSCRIBE presence event of user2 to the Dispatcher, and Dispatcher forward it to PS1 - PS1 write the subscription (user1->user2 with CSEQ=1) to active_watchers cache then active_wacthers table in DB immediately (write-through) - PS1 send the NOTIFY of the user2 presence status to user1 and increase the CSEQ by 1 (becomes 2) to both cache and DB (write-through) 2) user1 receive the NOTIFY and check the CSEQ (the first time), and update the user2's status 3) user2 change it's presence status and send the PUBLISH to the dispatcher, and Dispatcher forward it to PS2 - PS2 call handle_publish, it look-ups to the local cache for the active subscription and not found, then fallback to DB and found it, it increase the CSEQ by 1 (becomes 3 in DB) and send the NOTIFY to user1 with CSEQ=3 (here is the problem, the CSEQ of the same subscription in the PS1's cache is still equal 2) 4) the user1's client receive the NOTIFY and check the CSEQ if it's greater than the previous one, it's, then update the Presence Status of user2 5) user2 change it's presence status again, this time the Dispatcher forward the PUBLISH to PS1 - PS1 call handle_publish, it look-ups to the local cache for the active subscription and found it with CSEQ=2! It increase the CSEQ by 1 (becomes 3 in the cache and update the value to DB), and send the NOTIFY to user1 with CSEQ=3 again. 6) user1's client receive the NOTIFY and check the CSEQ if it's greater than the previous one, it's not, then it ignore it, the presen status of user2 is not updated in the user1's client!
This problem can be solved if we have pure DB mode only. In that case, the CSEQ is always increased for the successive NOTIFY of the same subscription.
It would be nice to keep the new Write-Through DB Mode in the presence module, and add a Pure DB mode, just like the usrloc module?
Just in case, in usrloc module, it miss the fallback to DB mode but has write-through and pure db mode ;-)
Best Regards, Laura
On Fri, Jul 15, 2011 at 4:58 PM, Henning Westerholt henning.westerholt@1und1.de wrote:
On Friday 15 July 2011, laura testi wrote:
we have checked the presence extension for DB_MODE only in the master branch. But it's not complete DB_ONLY mode, it seems that it still uses the cache for active_watchers in write through mode. Is there any reason for it? That can still make problem for the scalability for the multiple presence servers, because if an presence event (either SIP or XCAP) of an active subscription in the cache of one presence server, goes to another presence server, can still cause the problem even the DB is sync with the cache.
Hello Marius,
maybe you can comment a bit on this?
Thanks and best regards,
Henning
_______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi Marius, thank you for your answer. It would e nice if you can provide a pure DB mode that disable completely subscription hashtable and presentity hashtable in cache but only working with DB. Otherwise it can be still problematic with our test case. Because when the one presence server receives a PUBLISH from one presentity, it will looks-up the local subscription hastable then DB for the active watchers's contact to send notify and update the CSEQ in the subscription hastable/DB. But if there is any subscription of this presentity is in the remote subscription hashtable (another presence server) , the CSEQ of this subscription in hashtable will not be updated.
Thanks!
Best Regards, Laura
Hello Laura,
Thank you for your email and providing the full test-case. You are right, there is a certain issue (that is also mentioned in another thread - "DB_ONLY mode in presence module"). The thing is, that there are two caches
- a subscription hashtable, maintaining information about active subscripions
- a presentity hashtable, maintaining information about presence
entities (presentities, entities that send PUBLISH requests)
The db_only mode acts only on the first type of cache - subscriptions. It really makes sense having the db_only mode act also on the second cache (the presentity cache), hence your usecase is >very common. There are some cases when you only need a db_only mode only for subscriptions (like when PUBLISH requests are >generated precisely on each presence server machines) but of course I think it is a good compromise to disable caching >also for the "presentity" table when running in db_only mode.
I will make a patch this evening with this change, and commit it on the master branch. If anyone has other suggestions, please let me know.
Cheers, Marius
On Wed, Jul 20, 2011 at 1:11 PM, laura testi lau.testi@gmail.com wrote:
Hi Marius, Thank you very much for your explanations. I see your answer only in the list. Here I copy to here for the convenience:
Hello,
In the DB_ONLY mode, every SUBSCRIBE request will trigger a database operation immediately (either insert or update). Also, the SUBSCRIPTION will always be matched from the database. It behaves exactly as the name implies (write-through).
If it doesn't then that is a bug. Can you please tell me more about the scenario you are using? Is there any usecase in which you can see the subscription in memory but not in the active_watchers table?
Regards, Marius
The scenario wehave tested is like that we have multiple presence servers and a share DB in the separate DB server. The incoming SIP/SIMPLE traffic are dispatched with the Dispatcher. The problem we found is that the write-through mechanism guarantee the active_watchers cache is synched to DB immediately when it change like your explanation; but not vice versa. For example, when the active_watchers table in DB is changed by one presence server via the same write-through mechanism, it seems there is not possible to populate the changes in DB back to cache of another presence server. That means the cache in the different presence servers are not synchronized. It will required complex replications to have cache in synched.
I'll try to explain it by the following use case we have tested: Test Environments:
- SIP clients: (jitsi)
- user1 (has user2 as a contact, user 2 is watcher of user1 with the state=1) - user2 (has user1 as a contact, user 1 is watcher of user2 withe the state=1)
- SIP server (kamailio3.1.4)
- 1 x dispatcher (routing based call id) - 2 x presence servers (PS1 and PS2)
- DB server:
- 1 x backend DB server for SIP servers
Test Case (user2 is already login and the subscription to user1 is in one of the 2 PS cache and in DB):
- when user1 login, it send SUBSCRIBE presence event of user2 to the
Dispatcher, and Dispatcher forward it to PS1 - PS1 write the subscription (user1->user2 with CSEQ=1) to active_watchers cache then active_wacthers table in DB immediately (write-through) - PS1 send the NOTIFY of the user2 presence status to user1 and increase the CSEQ by 1 (becomes 2) to both cache and DB (write-through) 2) user1 receive the NOTIFY and check the CSEQ (the first time), and update the user2's status 3) user2 change it's presence status and send the PUBLISH to the dispatcher, and Dispatcher forward it to PS2 - PS2 call handle_publish, it look-ups to the local cache for the active subscription and not found, then fallback to DB and found it, it increase the CSEQ by 1 (becomes 3 in DB) and send the NOTIFY to user1 with CSEQ=3 (here is the problem, the CSEQ of the same subscription in the PS1's cache is still equal 2) 4) the user1's client receive the NOTIFY and check the CSEQ if it's greater than the previous one, it's, then update the Presence Status of user2 5) user2 change it's presence status again, this time the Dispatcher forward the PUBLISH to PS1 - PS1 call handle_publish, it look-ups to the local cache for the active subscription and found it with CSEQ=2! It increase the CSEQ by 1 (becomes 3 in the cache and update the value to DB), and send the NOTIFY to user1 with CSEQ=3 again. 6) user1's client receive the NOTIFY and check the CSEQ if it's greater than the previous one, it's not, then it ignore it, the presen status of user2 is not updated in the user1's client!
This problem can be solved if we have pure DB mode only. In that case, the CSEQ is always increased for the successive NOTIFY of the same subscription.
It would be nice to keep the new Write-Through DB Mode in the presence module, and add a Pure DB mode, just like the usrloc module?
Just in case, in usrloc module, it miss the fallback to DB mode but has write-through and pure db mode ;-)
Best Regards, Laura
On Fri, Jul 15, 2011 at 4:58 PM, Henning Westerholt henning.westerholt@1und1.de wrote:
On Friday 15 July 2011, laura testi wrote:
we have checked the presence extension for DB_MODE only in the master branch. But it's not complete DB_ONLY mode, it seems that it still uses the cache for active_watchers in write through mode. Is there any reason for it? That can still make problem for the scalability for the multiple presence servers, because if an presence event (either SIP or XCAP) of an active subscription in the cache of one presence server, goes to another presence server, can still cause the problem even the DB is sync with the cache.
Hello Marius,
maybe you can comment a bit on this?
Thanks and best regards,
Henning
Hello Laura,
I made a patch with the changes required for a pure DB_ONLY mode. You can find it on master branch. Any feedback is very welcomed.
Regards, Marius
________________________________ From: laura testi lau.testi@gmail.com To: Henning Westerholt henning.westerholt@1und1.de; "Bucur, Marius Ovidiu" marius.bucur@1and1.ro Cc: "sr-users@lists.sip-router.org" sr-users@lists.sip-router.org Sent: Thursday, July 21, 2011 11:08 AM Subject: Re: [SR-Users] xcap server - http load–balancer: xcap authentication problem
Hi Marius, thank you for your answer. It would e nice if you can provide a pure DB mode that disable completely subscription hashtable and presentity hashtable in cache but only working with DB. Otherwise it can be still problematic with our test case. Because when the one presence server receives a PUBLISH from one presentity, it will looks-up the local subscription hastable then DB for the active watchers's contact to send notify and update the CSEQ in the subscription hastable/DB. But if there is any subscription of this presentity is in the remote subscription hashtable (another presence server) , the CSEQ of this subscription in hashtable will not be updated.
Thanks!
Best Regards, Laura
Hello Laura,
Thank you for your email and providing the full test-case. You are right, there is a certain issue (that is also mentioned in another thread - "DB_ONLY mode in presence module"). The thing is, that there are two caches
- a subscription hashtable, maintaining information about active subscripions
- a presentity hashtable, maintaining information about presence
entities (presentities, entities that send PUBLISH requests)
The db_only mode acts only on the first type of cache - subscriptions. It really makes sense having the db_only mode act also on the second cache (the presentity cache), hence your usecase is >very common. There are some cases when you only need a db_only mode only for subscriptions (like when PUBLISH requests are >generated precisely on each presence server machines) but of course I think it is a good compromise to disable caching >also for the "presentity" table when running in db_only mode.
I will make a patch this evening with this change, and commit it on the master branch. If anyone has other suggestions, please let me know.
Cheers, Marius
On Wed, Jul 20, 2011 at 1:11 PM, laura testi lau.testi@gmail.com wrote:
Hi Marius, Thank you very much for your explanations. I see your answer only in the list. Here I copy to here for the convenience:
Hello,
In the DB_ONLY mode, every SUBSCRIBE request will trigger a database operation immediately (either insert or update). Also, the SUBSCRIPTION will always be matched from the database. It behaves exactly as the name implies (write-through).
If it doesn't then that is a bug. Can you please tell me more about the scenario you are using? Is there any usecase in which you can see the subscription in memory but not in the active_watchers table?
Regards, Marius
The scenario wehave tested is like that we have multiple presence servers and a share DB in the separate DB server. The incoming SIP/SIMPLE traffic are dispatched with the Dispatcher. The problem we found is that the write-through mechanism guarantee the active_watchers cache is synched to DB immediately when it change like your explanation; but not vice versa. For example, when the active_watchers table in DB is changed by one presence server via the same write-through mechanism, it seems there is not possible to populate the changes in DB back to cache of another presence server. That means the cache in the different presence servers are not synchronized. It will required complex replications to have cache in synched.
I'll try to explain it by the following use case we have tested: Test Environments:
- SIP clients: (jitsi)
- user1 (has user2 as a contact, user 2 is watcher of user1 with the state=1) - user2 (has user1 as a contact, user 1 is watcher of user2 withe the state=1)
- SIP server (kamailio3.1.4)
- 1 x dispatcher (routing based call id) - 2 x presence servers (PS1 and PS2)
- DB server:
- 1 x backend DB server for SIP servers
Test Case (user2 is already login and the subscription to user1 is in one of the 2 PS cache and in DB):
- when user1 login, it send SUBSCRIBE presence event of user2 to the
Dispatcher, and Dispatcher forward it to PS1 - PS1 write the subscription (user1->user2 with CSEQ=1) to active_watchers cache then active_wacthers table in DB immediately (write-through) - PS1 send the NOTIFY of the user2 presence status to user1 and increase the CSEQ by 1 (becomes 2) to both cache and DB (write-through) 2) user1 receive the NOTIFY and check the CSEQ (the first time), and update the user2's status 3) user2 change it's presence status and send the PUBLISH to the dispatcher, and Dispatcher forward it to PS2 - PS2 call handle_publish, it look-ups to the local cache for the active subscription and not found, then fallback to DB and found it, it increase the CSEQ by 1 (becomes 3 in DB) and send the NOTIFY to user1 with CSEQ=3 (here is the problem, the CSEQ of the same subscription in the PS1's cache is still equal 2) 4) the user1's client receive the NOTIFY and check the CSEQ if it's greater than the previous one, it's, then update the Presence Status of user2 5) user2 change it's presence status again, this time the Dispatcher forward the PUBLISH to PS1 - PS1 call handle_publish, it look-ups to the local cache for the active subscription and found it with CSEQ=2! It increase the CSEQ by 1 (becomes 3 in the cache and update the value to DB), and send the NOTIFY to user1 with CSEQ=3 again. 6) user1's client receive the NOTIFY and check the CSEQ if it's greater than the previous one, it's not, then it ignore it, the presen status of user2 is not updated in the user1's client!
This problem can be solved if we have pure DB mode only. In that case, the CSEQ is always increased for the successive NOTIFY of the same subscription.
It would be nice to keep the new Write-Through DB Mode in the presence module, and add a Pure DB mode, just like the usrloc module?
Just in case, in usrloc module, it miss the fallback to DB mode but has write-through and pure db mode ;-)
Best Regards, Laura
On Fri, Jul 15, 2011 at 4:58 PM, Henning Westerholt henning.westerholt@1und1.de wrote:
On Friday 15 July 2011, laura testi wrote:
we have checked the presence extension for DB_MODE only in the master branch. But it's not complete DB_ONLY mode, it seems that it still uses the cache for active_watchers in write through mode. Is there any reason for it? That can still make problem for the scalability for the multiple presence servers, because if an presence event (either SIP or XCAP) of an active subscription in the cache of one presence server, goes to another presence server, can still cause the problem even the DB is sync with the cache.
Hello Marius,
maybe you can comment a bit on this?
Thanks and best regards,
Henning
_______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-
Bucur Marius -
Daniel-Constantin Mierla -
Henning Westerholt -
laura testi