Although it's difficult to know the exact cause of your "protocol" error, it does appear from your tests that the graceful handling of errors and subsequent reconnects is better in libmemcached than in libmemcache. So if you do encounter the same conditions again, I would imagine that the library would handle it and attempt to reconnect transparently.
If not, however, then at least you are more likely to find support as the library is still under active development :)
Cheers,
Charles On 6 Jun 2013 18:12, "Dragos Oancea" droancea@yahoo.com wrote:
Hi Charles,
I tried first to trigger again the bug using some iptables rules (to simulate a network problem) , so Kamailio could not communicate with the memcache anymore:
WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_readable():2582: timeout: select(2) call timed out for read(2)able fds WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_writable():3173: timeout: write select(2) call timed out WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_connect():2290: select(2) failed: select(2) timed out on establishing conn ection NOTICE: memcached [memcached.c:194]: NOTICE: memcached: mcm_server_connect():2297: connect(2) failed NOTICE: memcached [memcached.c:194]: NOTICE: memcached: mcm_server_connect_next_avail():2323: unable to find a server to connect to
This looks very much like what i've seen in the logs just before it started to report the "protocol error" problem.
But in the simulated case , after I remove the iptables rules that block the access to memcache, Kamailo does not recover from the situation, it just keeps on printing messages like the above, when it should connect to memcache instead. It looks like another libmemcache-related bug to me.
Then I tried the same thing with the latest memcached module from git: With the memcached firewalled after starting Kamailio it prints this to the logs:
ERROR: memcached [mcd_var.c:154]: pv_get_mcd_value_helper(): could not get result for key 29515_HA1 - error was 'SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY'
When I flush the firewall rules, it reconnects to memcache, which makes it better than the old one. But how can I be sure that the new one will not have the same "protocol error" issue, since I could not reproduce exactly the problem so far ?
Regards, Dragos
------------------------------ *From:* Charles Chance charles.chance@sipcentric.com *To:* Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Users Mailing List sr-users@lists.sip-router.org *Sent:* Thursday, June 6, 2013 6:27 PM *Subject:* Re: [SR-Users] memcache problem - reporting protocol error
Hi Dragos,
The memcached module has indeed been updated in the master to use libmemcached, as the old libmemcache library is no longer under active development. Have you tried using the latest version of the module from git? Do you still get the same errors?
Cheers,
Charles
On 6 June 2013 16:18, Dragos Oancea droancea@yahoo.com wrote:
Hello
We are having issues with the memcached module (Kamailio 4.0.0) .
All of a sudden we are getting this kind of messages in the logs, and the values for the requested keys are not retrieved.
ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1305: memcache(4) protocol error: STORED#015#012VALUE 29515_HA1 0 32#015#012b52299247f16c6adfa4c8cc53b34e74a#015#012END#015#012STORED#015#012 [...] ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1176: memcache(4) protocol error: server sent data for key not in request
I suspect that at some point there was a network problem between the memcached server and Kamailio that triggered this abnormal behavior and I also suspect that it is connected with the (old) libmemcache library (libmemcache-1.4.0.rc2) .
This is fixed only with a Kamailio restart.
Has anyone experience this ? Is there a fix ? I noticed that the Kamailio from the trunk compiles the memcached module with the more new libmemcached library from here: https://launchpad.net/libmemcached/+download . Maybe this is one of the reasons libmemcache is going to be dropped ?
Thank you very much.
Regards, Dragos
_______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
www.sipcentric.com
Follow us on twitter @sipcentric http://twitter.com/sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office: Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.
Hi Charles,
Thank you for your answer. I've notice that your name shows in the headers of the files of the new (patched) memcached module. Can you please detail a little bit on what does your patch bring new apart from libmemcached support ? Or what does it still miss, since you said it's under heavy development.
Many thanks!
Regards, Dragos
________________________________ From: Charles Chance charles.chance@sipcentric.com To: Dragos Oancea droancea@yahoo.com Cc: SIP Router - Kamailio (OpenSER) and SIP Express Router (SER) - Users Mailing List sr-users@lists.sip-router.org Sent: Thursday, June 6, 2013 7:37 PM Subject: Re: [SR-Users] memcache problem - reporting protocol error
Although it's difficult to know the exact cause of your "protocol" error, it does appear from your tests that the graceful handling of errors and subsequent reconnects is better in libmemcached than in libmemcache. So if you do encounter the same conditions again, I would imagine that the library would handle it and attempt to reconnect transparently. If not, however, then at least you are more likely to find support as the library is still under active development :) Cheers, Charles
On 6 Jun 2013 18:12, "Dragos Oancea" droancea@yahoo.com wrote:
Hi Charles,
I tried first to trigger again the bug using some iptables rules (to simulate a network problem) , so Kamailio could not communicate with the memcache anymore:
WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_readable():2582: timeout: select(2) call timed out for read(2)able fds WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_writable():3173: timeout: write select(2) call timed out
WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_connect():2290: select(2) failed: select(2) timed out on establishing conn ection NOTICE: memcached [memcached.c:194]: NOTICE: memcached: mcm_server_connect():2297: connect(2) failed NOTICE: memcached [memcached.c:194]: NOTICE: memcached: mcm_server_connect_next_avail():2323: unable to find a server to connect to
This looks very much like what i've seen in the logs just before it started to report the "protocol error" problem.
But in the simulated case , after I remove the iptables rules that block the access to memcache, Kamailo does not recover from the situation, it just keeps on printing messages like the above, when it should connect to memcache instead. It looks like another libmemcache-related bug to me.
Then I tried the same thing with the latest memcached module from git: With the memcached firewalled after starting Kamailio it prints this to the logs:
ERROR: memcached [mcd_var.c:154]: pv_get_mcd_value_helper(): could not get result for key 29515_HA1 - error was 'SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY'
When I flush the firewall rules, it reconnects to memcache, which makes it better than the old one. But how can I be sure that the new one will not have the same "protocol error" issue, since I could not reproduce exactly the problem so far ?
Regards, Dragos
From: Charles Chance charles.chance@sipcentric.com To: Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Users Mailing List sr-users@lists.sip-router.org Sent: Thursday, June 6, 2013 6:27 PM Subject: Re: [SR-Users] memcache problem - reporting protocol error
Hi Dragos,
The memcached module has indeed been updated in the master to use libmemcached, as the old libmemcache library is no longer under active development. Have you tried using the latest version of the module from git? Do you still get the same errors?
Cheers,
Charles
On 6 June 2013 16:18, Dragos Oancea droancea@yahoo.com wrote:
Hello
We are having issues with the memcached module (Kamailio 4.0.0) .
All of a sudden we are getting this kind of messages in the logs, and the values for the requested keys are not retrieved.
ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1305: memcache(4) protocol error: STORED#015#012VALUE 29515_HA1 0 32#015#012b52299247f16c6adfa4c8cc53b34e74a#015#012END#015#012STORED#015#012
[...] ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1176: memcache(4) protocol error: server sent data for key not in request
I suspect that at some point there was a network problem between the memcached server and Kamailio that triggered this abnormal behavior and I also suspect that it is connected with the (old) libmemcache library (libmemcache-1.4.0.rc2) .
This is fixed only with a Kamailio restart.
Has anyone experience this ? Is there a fix ? I noticed that the Kamailio from the trunk compiles the memcached module with the more new libmemcached library from here: https://launchpad.net/libmemcached/+download%C2%A0.
Maybe this is one of the reasons libmemcache is going to be dropped ?
Thank you very much.
Regards, Dragos
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
www.sipcentric.com
Follow us on twitter @sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office: Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.
www.sipcentric.com
Follow us on twitter @sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office: Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.
The main difference, other than the new library, is the ability to specify an expiry for each stored key/value at the time of setting it:
So now there are two ways to set a value:
$mct(key) = value; (default expiry will be used as per module param)
or the new way, $mct(key=>expiry) = value;
Currently, the module still only supports a single memcache server, although if you are looking for a distributed solution, we have tested Couchbase/Moxi and it works really well.
All the best,
Charles
On 6 June 2013 20:40, Dragos Oancea droancea@yahoo.com wrote:
Hi Charles,
Thank you for your answer. I've notice that your name shows in the headers of the files of the new (patched) memcached module. Can you please detail a little bit on what does your patch bring new apart from libmemcached support ? Or what does it still miss, since you said it's under heavy development.
Many thanks!
Regards, Dragos
*From:* Charles Chance charles.chance@sipcentric.com *To:* Dragos Oancea droancea@yahoo.com *Cc:* SIP Router - Kamailio (OpenSER) and SIP Express Router (SER) - Users Mailing List sr-users@lists.sip-router.org *Sent:* Thursday, June 6, 2013 7:37 PM
*Subject:* Re: [SR-Users] memcache problem - reporting protocol error
Although it's difficult to know the exact cause of your "protocol" error, it does appear from your tests that the graceful handling of errors and subsequent reconnects is better in libmemcached than in libmemcache. So if you do encounter the same conditions again, I would imagine that the library would handle it and attempt to reconnect transparently. If not, however, then at least you are more likely to find support as the library is still under active development :) Cheers, Charles On 6 Jun 2013 18:12, "Dragos Oancea" droancea@yahoo.com wrote:
Hi Charles,
I tried first to trigger again the bug using some iptables rules (to simulate a network problem) , so Kamailio could not communicate with the memcache anymore:
WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_readable():2582: timeout: select(2) call timed out for read(2)able fds WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_writable():3173: timeout: write select(2) call timed out WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_connect():2290: select(2) failed: select(2) timed out on establishing conn ection NOTICE: memcached [memcached.c:194]: NOTICE: memcached: mcm_server_connect():2297: connect(2) failed NOTICE: memcached [memcached.c:194]: NOTICE: memcached: mcm_server_connect_next_avail():2323: unable to find a server to connect to
This looks very much like what i've seen in the logs just before it started to report the "protocol error" problem.
But in the simulated case , after I remove the iptables rules that block the access to memcache, Kamailo does not recover from the situation, it just keeps on printing messages like the above, when it should connect to memcache instead. It looks like another libmemcache-related bug to me.
Then I tried the same thing with the latest memcached module from git: With the memcached firewalled after starting Kamailio it prints this to the logs:
ERROR: memcached [mcd_var.c:154]: pv_get_mcd_value_helper(): could not get result for key 29515_HA1 - error was 'SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY'
When I flush the firewall rules, it reconnects to memcache, which makes it better than the old one. But how can I be sure that the new one will not have the same "protocol error" issue, since I could not reproduce exactly the problem so far ?
Regards, Dragos
*From:* Charles Chance charles.chance@sipcentric.com *To:* Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Users Mailing List sr-users@lists.sip-router.org *Sent:* Thursday, June 6, 2013 6:27 PM *Subject:* Re: [SR-Users] memcache problem - reporting protocol error
Hi Dragos,
The memcached module has indeed been updated in the master to use libmemcached, as the old libmemcache library is no longer under active development. Have you tried using the latest version of the module from git? Do you still get the same errors?
Cheers,
Charles
On 6 June 2013 16:18, Dragos Oancea droancea@yahoo.com wrote:
Hello
We are having issues with the memcached module (Kamailio 4.0.0) .
All of a sudden we are getting this kind of messages in the logs, and the values for the requested keys are not retrieved.
ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1305: memcache(4) protocol error: STORED#015#012VALUE 29515_HA1 0 32#015#012b52299247f16c6adfa4c8cc53b34e74a#015#012END#015#012STORED#015#012 [...] ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1176: memcache(4) protocol error: server sent data for key not in request
I suspect that at some point there was a network problem between the memcached server and Kamailio that triggered this abnormal behavior and I also suspect that it is connected with the (old) libmemcache library (libmemcache-1.4.0.rc2) .
This is fixed only with a Kamailio restart.
Has anyone experience this ? Is there a fix ? I noticed that the Kamailio from the trunk compiles the memcached module with the more new libmemcached library from here: https://launchpad.net/libmemcached/+download . Maybe this is one of the reasons libmemcache is going to be dropped ?
Thank you very much.
Regards, Dragos
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
www.sipcentric.com
Follow us on twitter @sipcentric http://twitter.com/sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office: Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.
www.sipcentric.com
Follow us on twitter @sipcentric http://twitter.com/sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office: Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.
Hi Charles,
Thank you for the information. These details are good to know. Do you think it's safe to use the new module from git with kamailio-4.0.0 ? I mean just copying it in the kamailio-4.0.0 source tree, compiling it and then putting it in the modules directory to be loaded at runtime.
Regards, Dragos
________________________________ From: Charles Chance charles.chance@sipcentric.com To: Dragos Oancea droancea@yahoo.com Cc: SIP Router - Kamailio (OpenSER) and SIP Express Router (SER) - Users Mailing List sr-users@lists.sip-router.org Sent: Friday, June 7, 2013 11:06 AM Subject: Re: [SR-Users] memcache problem - reporting protocol error
The main difference, other than the new library, is the ability to specify an expiry for each stored key/value at the time of setting it:
So now there are two ways to set a value:
$mct(key) = value; (default expiry will be used as per module param)
or the new way, $mct(key=>expiry) = value;
Currently, the module still only supports a single memcache server, although if you are looking for a distributed solution, we have tested Couchbase/Moxi and it works really well.
All the best,
Charles
On 6 June 2013 20:40, Dragos Oancea droancea@yahoo.com wrote:
Hi Charles,
Thank you for your answer. I've notice that your name shows in the headers of the files of the new (patched) memcached module. Can you please detail a little bit on what does your patch bring new apart from libmemcached support ? Or what does it still miss, since you said it's under heavy development.
Many thanks!
Regards, Dragos
From: Charles Chance charles.chance@sipcentric.com To: Dragos Oancea droancea@yahoo.com Cc: SIP Router - Kamailio (OpenSER) and SIP Express Router (SER) - Users Mailing List sr-users@lists.sip-router.org Sent: Thursday, June 6, 2013 7:37 PM
Subject: Re: [SR-Users] memcache problem - reporting protocol error
Although it's difficult to know the exact cause of your "protocol" error, it does appear from your tests that the graceful handling of errors and subsequent reconnects is better in libmemcached than in libmemcache. So if you do encounter the same conditions again, I would imagine that the library would handle it and attempt to reconnect transparently. If not, however, then at least you are more likely to find support as the library is still under active development :) Cheers, Charles
On 6 Jun 2013 18:12, "Dragos Oancea" droancea@yahoo.com wrote:
Hi Charles,
I tried first to trigger again the bug using some iptables rules (to simulate a network problem) , so Kamailio could not communicate with the memcache anymore:
WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_readable():2582: timeout: select(2) call timed out for read(2)able fds WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_writable():3173: timeout: write select(2) call timed out
WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_connect():2290: select(2) failed: select(2) timed out on establishing conn ection NOTICE: memcached [memcached.c:194]: NOTICE: memcached: mcm_server_connect():2297: connect(2) failed NOTICE: memcached [memcached.c:194]: NOTICE: memcached: mcm_server_connect_next_avail():2323: unable to find a server to connect to
This looks very much like what i've seen in the logs just before it started to report the "protocol error" problem.
But in the simulated case , after I remove the iptables rules that block the access to memcache, Kamailo does not recover from the situation, it just keeps on printing messages like the above, when it should connect to memcache instead. It looks like another libmemcache-related bug to me.
Then I tried the same thing with the latest memcached module from git: With the memcached firewalled after starting Kamailio it prints this to the logs:
ERROR: memcached [mcd_var.c:154]: pv_get_mcd_value_helper(): could not get result for key 29515_HA1 - error was 'SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY'
When I flush the firewall rules, it reconnects to memcache, which makes it better than the old one. But how can I be sure that the new one will not have the same "protocol error" issue, since I could not reproduce exactly the problem so far ?
Regards, Dragos
From: Charles Chance charles.chance@sipcentric.com To: Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Users Mailing List sr-users@lists.sip-router.org Sent: Thursday, June 6, 2013 6:27 PM Subject: Re: [SR-Users] memcache problem - reporting protocol error
Hi Dragos,
The memcached module has indeed been updated in the master to use libmemcached, as the old libmemcache library is no longer under active development. Have you tried using the latest version of the module from git? Do you still get the same errors?
Cheers,
Charles
On 6 June 2013 16:18, Dragos Oancea droancea@yahoo.com wrote:
Hello
We are having issues with the memcached module (Kamailio 4.0.0) .
All of a sudden we are getting this kind of messages in the logs, and the values for the requested keys are not retrieved.
ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1305: memcache(4) protocol error: STORED#015#012VALUE 29515_HA1 0 32#015#012b52299247f16c6adfa4c8cc53b34e74a#015#012END#015#012STORED#015#012
[...] ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1176: memcache(4) protocol error: server sent data for key not in request
I suspect that at some point there was a network problem between the memcached server and Kamailio that triggered this abnormal behavior and I also suspect that it is connected with the (old) libmemcache library (libmemcache-1.4.0.rc2) .
This is fixed only with a Kamailio restart.
Has anyone experience this ? Is there a fix ? I noticed that the Kamailio from the trunk compiles the memcached module with the more new libmemcached library from here: https://launchpad.net/libmemcached/+download%C2%A0.
Maybe this is one of the reasons libmemcache is going to be dropped ?
Thank you very much.
Regards, Dragos
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
www.sipcentric.com
Follow us on twitter @sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office: Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.
www.sipcentric.com
Follow us on twitter @sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office: Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.
www.sipcentric.com
Follow us on twitter @sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office: Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.
Hi Dragos,
Yes, that should work fine.
Cheers,
Charles On 7 Jun 2013 12:00, "Dragos Oancea" droancea@yahoo.com wrote:
Hi Charles,
Thank you for the information. These details are good to know. Do you think it's safe to use the new module from git with kamailio-4.0.0 ? I mean just copying it in the kamailio-4.0.0 source tree, compiling it and then putting it in the modules directory to be loaded at runtime.
Regards, Dragos
*From:* Charles Chance charles.chance@sipcentric.com *To:* Dragos Oancea droancea@yahoo.com *Cc:* SIP Router - Kamailio (OpenSER) and SIP Express Router (SER) - Users Mailing List sr-users@lists.sip-router.org *Sent:* Friday, June 7, 2013 11:06 AM *Subject:* Re: [SR-Users] memcache problem - reporting protocol error
The main difference, other than the new library, is the ability to specify an expiry for each stored key/value at the time of setting it:
So now there are two ways to set a value:
$mct(key) = value; (default expiry will be used as per module param)
or the new way, $mct(key=>expiry) = value;
Currently, the module still only supports a single memcache server, although if you are looking for a distributed solution, we have tested Couchbase/Moxi and it works really well.
All the best,
Charles
On 6 June 2013 20:40, Dragos Oancea droancea@yahoo.com wrote:
Hi Charles,
Thank you for your answer. I've notice that your name shows in the headers of the files of the new (patched) memcached module. Can you please detail a little bit on what does your patch bring new apart from libmemcached support ? Or what does it still miss, since you said it's under heavy development.
Many thanks!
Regards, Dragos
*From:* Charles Chance charles.chance@sipcentric.com *To:* Dragos Oancea droancea@yahoo.com *Cc:* SIP Router - Kamailio (OpenSER) and SIP Express Router (SER) - Users Mailing List sr-users@lists.sip-router.org *Sent:* Thursday, June 6, 2013 7:37 PM
*Subject:* Re: [SR-Users] memcache problem - reporting protocol error
Although it's difficult to know the exact cause of your "protocol" error, it does appear from your tests that the graceful handling of errors and subsequent reconnects is better in libmemcached than in libmemcache. So if you do encounter the same conditions again, I would imagine that the library would handle it and attempt to reconnect transparently. If not, however, then at least you are more likely to find support as the library is still under active development :) Cheers, Charles On 6 Jun 2013 18:12, "Dragos Oancea" droancea@yahoo.com wrote:
Hi Charles,
I tried first to trigger again the bug using some iptables rules (to simulate a network problem) , so Kamailio could not communicate with the memcache anymore:
WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_readable():2582: timeout: select(2) call timed out for read(2)able fds WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_writable():3173: timeout: write select(2) call timed out WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_connect():2290: select(2) failed: select(2) timed out on establishing conn ection NOTICE: memcached [memcached.c:194]: NOTICE: memcached: mcm_server_connect():2297: connect(2) failed NOTICE: memcached [memcached.c:194]: NOTICE: memcached: mcm_server_connect_next_avail():2323: unable to find a server to connect to
This looks very much like what i've seen in the logs just before it started to report the "protocol error" problem.
But in the simulated case , after I remove the iptables rules that block the access to memcache, Kamailo does not recover from the situation, it just keeps on printing messages like the above, when it should connect to memcache instead. It looks like another libmemcache-related bug to me.
Then I tried the same thing with the latest memcached module from git: With the memcached firewalled after starting Kamailio it prints this to the logs:
ERROR: memcached [mcd_var.c:154]: pv_get_mcd_value_helper(): could not get result for key 29515_HA1 - error was 'SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY'
When I flush the firewall rules, it reconnects to memcache, which makes it better than the old one. But how can I be sure that the new one will not have the same "protocol error" issue, since I could not reproduce exactly the problem so far ?
Regards, Dragos
*From:* Charles Chance charles.chance@sipcentric.com *To:* Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Users Mailing List sr-users@lists.sip-router.org *Sent:* Thursday, June 6, 2013 6:27 PM *Subject:* Re: [SR-Users] memcache problem - reporting protocol error
Hi Dragos,
The memcached module has indeed been updated in the master to use libmemcached, as the old libmemcache library is no longer under active development. Have you tried using the latest version of the module from git? Do you still get the same errors?
Cheers,
Charles
On 6 June 2013 16:18, Dragos Oancea droancea@yahoo.com wrote:
Hello
We are having issues with the memcached module (Kamailio 4.0.0) .
All of a sudden we are getting this kind of messages in the logs, and the values for the requested keys are not retrieved.
ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1305: memcache(4) protocol error: STORED#015#012VALUE 29515_HA1 0 32#015#012b52299247f16c6adfa4c8cc53b34e74a#015#012END#015#012STORED#015#012 [...] ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1176: memcache(4) protocol error: server sent data for key not in request
I suspect that at some point there was a network problem between the memcached server and Kamailio that triggered this abnormal behavior and I also suspect that it is connected with the (old) libmemcache library (libmemcache-1.4.0.rc2) .
This is fixed only with a Kamailio restart.
Has anyone experience this ? Is there a fix ? I noticed that the Kamailio from the trunk compiles the memcached module with the more new libmemcached library from here: https://launchpad.net/libmemcached/+download . Maybe this is one of the reasons libmemcache is going to be dropped ?
Thank you very much.
Regards, Dragos
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
www.sipcentric.com
Follow us on twitter @sipcentric http://twitter.com/sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office: Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.
www.sipcentric.com
Follow us on twitter @sipcentric http://twitter.com/sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office: Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.
www.sipcentric.com
Follow us on twitter @sipcentric http://twitter.com/sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office: Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.
Am Freitag, 7. Juni 2013, 12:07:00 schrieb Charles Chance:
Yes, that should work fine.
Cheers,
Hello Dragos,
just one thing to add - if you find any issues with the master version of the module, please report to our developer list that we could take a look.
Best regards,
Henning Westerholt
-
Charles Chance -
Dragos Oancea -
Henning Westerholt