Hi Charles,
I tried first to trigger again the bug using some iptables rules (to simulate a network
problem) , so Kamailio could not communicate with the memcache anymore:
WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_readable():2582:
timeout: select(2) call timed out for read(2)able fds
WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_writable():3173:
timeout: write select(2) call timed out
WARNING: memcached [memcached.c:189]: WARNING: memcached: mcm_server_connect():2290:
select(2) failed: select(2) timed out on establishing conn
ection
NOTICE: memcached [memcached.c:194]: NOTICE: memcached: mcm_server_connect():2297:
connect(2) failed
NOTICE: memcached [memcached.c:194]: NOTICE: memcached:
mcm_server_connect_next_avail():2323: unable to find a server to connect to
This looks very much like what i've seen in the logs just before it started to report
the "protocol error" problem.
But in the simulated case , after I remove the iptables rules that block the access to
memcache, Kamailo does not recover from the situation, it just keeps on printing messages
like the above, when it should
connect to memcache instead. It looks like another libmemcache-related bug to me.
Then I tried the same thing with the latest memcached module from git:
With the memcached firewalled after starting Kamailio it prints this to the logs:
ERROR: memcached [mcd_var.c:154]: pv_get_mcd_value_helper(): could not get result for
key 29515_HA1 - error was 'SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY'
When I flush the firewall rules, it reconnects to memcache, which makes it better than the
old one.
But how can I be sure that the new one will not have the same "protocol error"
issue, since I could not reproduce exactly the problem so far ?
Regards,
Dragos
________________________________
From: Charles Chance <charles.chance(a)sipcentric.com>
To: Dragos Oancea <droancea(a)yahoo.com>om>; Kamailio (SER) - Users Mailing List
<sr-users(a)lists.sip-router.org>
Sent: Thursday, June 6, 2013 6:27 PM
Subject: Re: [SR-Users] memcache problem - reporting protocol error
Hi Dragos,
The memcached module has indeed been updated in the master to use libmemcached, as the old
libmemcache library is no longer under active development. Have you tried using the latest
version of the module from git? Do you still get the same errors?
Cheers,
Charles
On 6 June 2013 16:18, Dragos Oancea <droancea(a)yahoo.com> wrote:
Hello
We are having issues with the memcached module (Kamailio 4.0.0) .
All of a sudden we are getting this kind of messages in the logs, and the values for the
requested keys are not retrieved.
ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1305: memcache(4)
protocol error: STORED#015#012VALUE 29515_HA1 0
32#015#012b52299247f16c6adfa4c8cc53b34e74a#015#012END#015#012STORED#015#012
[...]
ALERT: memcached [memcached.c:189]: ALERT: memcached: mcm_fetch_cmd():1176: memcache(4)
protocol error: server sent data for key not in request
I suspect that at some point there was a network problem between the memcached server and
Kamailio that triggered this abnormal behavior and I also suspect that it is
connected with the (old) libmemcache library (libmemcache-1.4.0.rc2) .
This is fixed only with a Kamailio restart.
Has anyone experience this ? Is there a fix ?
I noticed that the Kamailio from the trunk compiles the memcached module with the more new
libmemcached library from here:
https://launchpad.net/libmemcached/+download .
Maybe this is one of the reasons libmemcache is going to be dropped ?
Thank you very much.
Regards,
Dragos
_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users(a)lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
www.sipcentric.com
Follow us on twitter @sipcentric
Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered office:
Unit 10 iBIC, Birmingham Science Park, Holt Court South, Birmingham B7 4EJ.