Hi

Here is the output.

1st core file:
(gdb) p *(struct qm_frag_end*)((char*)frag-sizeof(struct qm_frag_end))
$1 = {size = 64, prev_free = 0x7fd961885e90}

2nd core file:
(gdb) p *(struct qm_frag_end*)((char*)frag-sizeof(struct qm_frag_end))
$1 = {size = 64, prev_free = 0x7f563a34e0d0}


Regards,
Dragos


On Thursday, November 21, 2013 7:58 PM, Daniel-Constantin Mierla <miconda@gmail.com> wrote:
Hello,

the values are invalid, probably the previous fragment was writing more.

For now, can you give:

p *(struct qm_frag_end*)((char*)frag-sizeof(struct qm_frag_end))

I expect to be some invalid values as well.

Later I will try to come up with a gdb script to spot the previous fragment.

MEMDBG=1 will make it slightly slower and increases a bit the overhead. But I guess you don't run at the limits of CPU. In the past we used to have it for couple of releases on and nobody complained about performances.

Cheers,
Daniel

On 11/21/13 7:39 PM, Dragos Oancea wrote:
Hi

Here is the output:

gdb) 
(gdb) frame 0
#0  qm_detach_free (qm=0x7fd96175e010, size=112) at mem/q_malloc.c:266
266 in mem/q_malloc.c
(gdb) 
#0  qm_detach_free (qm=0x7fd96175e010, size=112) at mem/q_malloc.c:266
266 in mem/q_malloc.c
(gdb) p *frag
$1 = {size = 7599108840079127868, u = {nxt_free = 0x3965663931343a64, is_free = 4135824228634344036}}
(gdb) 
$2 = {size = 7599108840079127868, u = {nxt_free = 0x3965663931343a64, is_free = 4135824228634344036}}
(gdb) 
$3 = {size = 7599108840079127868, u = {nxt_free = 0x3965663931343a64, is_free = 4135824228634344036}}
(gdb) p *((char*)frag + sizeof(struct qm_frag))
$4 = 99 'c'

Is it okay to run with MEMDBG=1 in production ? Wouldn't it make it a little slow ?

Just let me know if u want so see something else with gdb. Unfortunally  I do not have SIP traces, but I have the core file and some logs.


Regards,
Dragos


On Thursday, November 21, 2013 6:50 PM, Daniel-Constantin Mierla <miconda@gmail.com> wrote:
Hello,

can you provde the output in gdb for:

frame 0

p *frag


p *((char*)frag + sizeof(struct qm_frag))
There were similar reports, so apparently there is a buffer overflow somewhere.

You should update to latest git branch 4.0, because there were some other fixes from 4.0.3. With this occasion, you should set MEMDBG=1 in Makefile.defs before recompiling the new version to be able to catch easier the overwrites of memory.

Cheers,
Daniel

On 11/21/13 6:36 PM, Dragos Oancea wrote:
Hello

We had this crash today with kamailio 4.0.3 .
It ran stable for few weeks until this crash.

GDB here:



Regards,
Dragos