Hi Bogdan,
Bad news, I am afraid. I got another "out of memory". I did another memory dump on process 10714 this time. Actually there is one dump that I did earlier to see if there were any segments allocated by subst_str. And I did a second dump after the "out of memory" happened. The log file is: http://www.real.gr/files/openser.20070622.log.gz
thank you
George
-----Original Message----- From: Bogdan-Andrei Iancu [mailto:bogdan@voice-system.ro] Sent: Friday, June 22, 2007 1:44 PM To: Papadopoulos Georgios Cc: users@openser.org Subject: Re: [Users] out of memory - please help
Hi George,
Papadopoulos Georgios wrote:
Hi Bogdan,
The server has been running for about 18 hours without any "out of memory" messages. Usually these appeared after a few hours.
I think it
is fixed, but I will keep you posted if it reappears. Thanks a lot!
That is super! I will commit the patch on SVN.
I guess the same fix needs to go in subst_uri() and maybe
also add a
check if(result->s!=0) ?
well, in subst_uri() there is no need because the result->s chunk is transferred to msg->new_uri (and freed later from there); in subst_user(), the result->s chunk was copied by the SET_USER_T action, so we need to free it also.
Why did this only appear on the first receiver child? I
would expect
that all children receive an equal amount of traffic so they should all run out of memory more or less at the same time. Except
if when a
message is received, Openser finds the first available
child and does
this by trying child 1,2,3... Is this the case?
this scheduling is done by kernel and there is no guarantee of a uniform distribution in case of low traffic.
Last (related?) question: There is a thread that the first receiver child starts, what is its purpose?
you mean a process (there are no threads in openser). it might be a listener of mi_fifo.
Regards, Bogdan
Disclaimer The information in this e-mail and any attachments is confidential. It is intended solely for the attention and use of the named addressee(s). If you are not the intended recipient, or person responsible for delivering this information to the intended recipient, please notify the sender immediately. Unless you are the intended recipient or his/her representative you are not authorized to, and must not, read, copy, distribute, use or retain this message or any part of it. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses.
Hi George,
looking over the log, it does seams to be a leak. My guess is that there is too few PKG memory configured. You have probably a large script and some intensive message processing which may result in mem outage at a point. Also memory fragmentation may increase the memory usage.
Try to increase the PKG memory and see if you get rid of the problem.
regards, bogdan
Papadopoulos Georgios wrote:
Hi Bogdan,
Bad news, I am afraid. I got another "out of memory". I did another memory dump on process 10714 this time. Actually there is one dump that I did earlier to see if there were any segments allocated by subst_str. And I did a second dump after the "out of memory" happened. The log file is: http://www.real.gr/files/openser.20070622.log.gz
thank you
George
-----Original Message----- From: Bogdan-Andrei Iancu [mailto:bogdan@voice-system.ro] Sent: Friday, June 22, 2007 1:44 PM To: Papadopoulos Georgios Cc: users@openser.org Subject: Re: [Users] out of memory - please help
Hi George,
Papadopoulos Georgios wrote:
Hi Bogdan,
The server has been running for about 18 hours without any "out of memory" messages. Usually these appeared after a few hours.
I think it
is fixed, but I will keep you posted if it reappears. Thanks a lot!
That is super! I will commit the patch on SVN.
I guess the same fix needs to go in subst_uri() and maybe
also add a
check if(result->s!=0) ?
well, in subst_uri() there is no need because the result->s chunk is transferred to msg->new_uri (and freed later from there); in subst_user(), the result->s chunk was copied by the SET_USER_T action, so we need to free it also.
Why did this only appear on the first receiver child? I
would expect
that all children receive an equal amount of traffic so they should all run out of memory more or less at the same time. Except
if when a
message is received, Openser finds the first available
child and does
this by trying child 1,2,3... Is this the case?
this scheduling is done by kernel and there is no guarantee of a uniform distribution in case of low traffic.
Last (related?) question: There is a thread that the first receiver child starts, what is its purpose?
you mean a process (there are no threads in openser). it might be a listener of mi_fifo.
Regards, Bogdan
Disclaimer The information in this e-mail and any attachments is confidential. It is intended solely for the attention and use of the named addressee(s). If you are not the intended recipient, or person responsible for delivering this information to the intended recipient, please notify the sender immediately. Unless you are the intended recipient or his/her representative you are not authorized to, and must not, read, copy, distribute, use or retain this message or any part of it. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses.
small errata :)
Bogdan-Andrei Iancu wrote:
Hi George,
looking over the log, it does seams to be a leak. My guess is that there is too few PKG memory configured.
^^^^^ doesn't !
You have probably a large script and some intensive message processing which may result in mem outage at a point. Also memory fragmentation may increase the memory usage.
Try to increase the PKG memory and see if you get rid of the problem.
regards, bogdan
Papadopoulos Georgios wrote:
Hi Bogdan,
Bad news, I am afraid. I got another "out of memory". I did another memory dump on process 10714 this time. Actually there is one dump that I did earlier to see if there were any segments allocated by subst_str. And I did a second dump after the "out of memory" happened. The log file is: http://www.real.gr/files/openser.20070622.log.gz
thank you
George
-----Original Message----- From: Bogdan-Andrei Iancu [mailto:bogdan@voice-system.ro] Sent: Friday, June 22, 2007 1:44 PM To: Papadopoulos Georgios Cc: users@openser.org Subject: Re: [Users] out of memory - please help
Hi George,
Papadopoulos Georgios wrote:
Hi Bogdan,
The server has been running for about 18 hours without any "out of memory" messages. Usually these appeared after a few hours.
I think it
is fixed, but I will keep you posted if it reappears. Thanks a lot!
That is super! I will commit the patch on SVN.
I guess the same fix needs to go in subst_uri() and maybe
also add a
check if(result->s!=0) ?
well, in subst_uri() there is no need because the result->s chunk is transferred to msg->new_uri (and freed later from there); in subst_user(), the result->s chunk was copied by the SET_USER_T action, so we need to free it also.
Why did this only appear on the first receiver child? I
would expect
that all children receive an equal amount of traffic so they should all run out of memory more or less at the same time. Except
if when a
message is received, Openser finds the first available
child and does
this by trying child 1,2,3... Is this the case?
this scheduling is done by kernel and there is no guarantee of a uniform distribution in case of low traffic.
Last (related?) question: There is a thread that the first receiver child starts, what is its purpose?
you mean a process (there are no threads in openser). it might be a listener of mi_fifo.
Regards, Bogdan
-
Bogdan-Andrei Iancu -
Papadopoulos Georgios