Hi!
I suspect openser to have TCP problems under high load: Scenario: sipp sends INVITE to openser, which replies with 404 I have 4x sipp with 1000 INVITEs/s and 1x sipp with 3000INVITEs/s, thus 5 TCP connections
After some time, I get error on the TCP connection with 3000INVITEs/s: -->syslog.txt
Looks like the first part of the request is missing, but sniffing revealed that the request is proper formated as you see in the TCP packet -->ngrep.txt
Are there known high-load problems?
Any ideas how I can debug this?
regards klaus
Sorry - forgotten to attach the files :-)
klaus
Klaus Darilion wrote:
Hi!
I suspect openser to have TCP problems under high load: Scenario: sipp sends INVITE to openser, which replies with 404 I have 4x sipp with 1000 INVITEs/s and 1x sipp with 3000INVITEs/s, thus 5 TCP connections
After some time, I get error on the TCP connection with 3000INVITEs/s: -->syslog.txt
Looks like the first part of the request is missing, but sniffing revealed that the request is proper formated as you see in the TCP packet -->ngrep.txt
Are there known high-load problems?
Any ideas how I can debug this?
regards klaus
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
graham:/home/darilion# ngrep -I test6.cap -q 106789 input: test6.cap
T 83.136.32.91:32904 -> 83.136.32.132:5060 [A] INVITE sip:service@83.136.32.132:5060 SIP/2.0..Via: SIP/2.0/TCP 83.136.32.91:5062;branch=z9hG4bK-106789-0..From: sipp sip:sipp@83.136.32.91:5062;tag=106789..To: sut sip:service@83.136.32.132:5060..Call-ID: 106789-10754@83.136.32.91..CSeq: 1 INVITE..Contact: sip:sipp@83.136.32.91:5062..Max-Forwards: 70..Subject: Perform ance Test..Content-Type: application/sdp..Content-Length: 135....v=0..o=user1 53655765 2353687637 IN IP4 83.136.32.91..s=-..c=IN IP4 83.136.32.91..t=0 0..m=audio 6002 RTP/AVP 0..a=rtpmap:0 PCMU/8000..INVITE sip:service@83.136.32.132:5060 SIP/2.0..Via: SIP/2.0/TCP 83.136.32.91:5062;branch=z9hG4bK-106790-0..From: sipp <sip: sipp@83.136.32.91:5062>;tag=106790..To: sut sip:service@83.136.32.132:5060..Call-ID: 106790-10754@83.136.32.91..CSeq: 1 INVITE..Contact: sip:sipp@83.136.32.91:5 062..Max-Forwards: 70..Subject: Performance Test..Content-Type: application/sdp..Content-Length: 135....v=0..o=user1 53655765 2353687637 IN IP4 83.136.32.91..s=- ..c=IN IP4 83.136.32.91..t=0 0..m=audio 6002 RTP/AVP 0..a=rtpmap:0 PCMU/8000..INVITE sip:service@83.136.32.132:5060 SIP/2.0..Via: SIP/2.0/TCP 83.136.32.91:5062;br anch=z9hG4bK-106791-0..From: sipp sip:sipp@83.136.32.91:5062;tag=106791..To: sut sip:service@83.136.32.132:5060..Call-ID: 106791-10754@83.136.32.91..CSeq: 1 I NVITE..Contact: sip:sipp@83.136.32.91:5062..Max-Forwards: 70..Subject: Performance Test..Content-Type: application/sdp..Content-Length: 135....v=0..o=u
graham:/home/darilion# Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR:parse_first_line: method not followed by SP Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR:parse_first_line: bad message Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR: parse_msg: message=<-106789-0^M From: sipp sip:sipp@83.136.32.91:5062;tag=106789^M To: sut sip:service@83.136.32.132:5060^M Call-ID: 106789-10754@83.136.32.91^M CSeq: 1 INVITE^M Contact: sip:sipp@83.136.32.91:5062^M Max-Forwards: 70^M Subject: Performance Test^M Content-Type: application/sdp^M Content-Length: 135^M ^M v=0^M o=user1 53655765 2353687637 IN IP4 83.136.32.91^M s=-^M c=IN IP4 83.136.32.91^M t=0 0^M m=audio 6002 RTP/AVP 0^M a=rtpmap:0 PCMU/8000^M > Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR: receive_msg: parse_msg failed
Hello,
this looks like the TCP kernel buffer is overflows. How many OpenSER TCP processes do you have?
Cheers, Daniel
On 07/26/06 15:44, Klaus Darilion wrote:
Sorry - forgotten to attach the files :-)
klaus
Klaus Darilion wrote:
Hi!
I suspect openser to have TCP problems under high load: Scenario: sipp sends INVITE to openser, which replies with 404 I have 4x sipp with 1000 INVITEs/s and 1x sipp with 3000INVITEs/s, thus 5 TCP connections
After some time, I get error on the TCP connection with 3000INVITEs/s: -->syslog.txt
Looks like the first part of the request is missing, but sniffing revealed that the request is proper formated as you see in the TCP packet -->ngrep.txt
Are there known high-load problems?
Any ideas how I can debug this?
regards klaus
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
graham:/home/darilion# ngrep -I test6.cap -q 106789 input: test6.cap
T 83.136.32.91:32904 -> 83.136.32.132:5060 [A] INVITE sip:service@83.136.32.132:5060 SIP/2.0..Via: SIP/2.0/TCP 83.136.32.91:5062;branch=z9hG4bK-106789-0..From: sipp sip:sipp@83.136.32.91:5062;tag=106789..To: sut sip:service@83.136.32.132:5060..Call-ID: 106789-10754@83.136.32.91..CSeq: 1 INVITE..Contact: sip:sipp@83.136.32.91:5062..Max-Forwards: 70..Subject: Perform ance Test..Content-Type: application/sdp..Content-Length: 135....v=0..o=user1 53655765 2353687637 IN IP4 83.136.32.91..s=-..c=IN IP4 83.136.32.91..t=0 0..m=audio 6002 RTP/AVP 0..a=rtpmap:0 PCMU/8000..INVITE sip:service@83.136.32.132:5060 SIP/2.0..Via: SIP/2.0/TCP 83.136.32.91:5062;branch=z9hG4bK-106790-0..From: sipp <sip: sipp@83.136.32.91:5062>;tag=106790..To: sut sip:service@83.136.32.132:5060..Call-ID: 106790-10754@83.136.32.91..CSeq: 1 INVITE..Contact: sip:sipp@83.136.32.91:5 062..Max-Forwards: 70..Subject: Performance Test..Content-Type: application/sdp..Content-Length: 135....v=0..o=user1 53655765 2353687637 IN IP4 83.136.32.91..s=- ..c=IN IP4 83.136.32.91..t=0 0..m=audio 6002 RTP/AVP 0..a=rtpmap:0 PCMU/8000..INVITE sip:service@83.136.32.132:5060 SIP/2.0..Via: SIP/2.0/TCP 83.136.32.91:5062;br anch=z9hG4bK-106791-0..From: sipp sip:sipp@83.136.32.91:5062;tag=106791..To: sut sip:service@83.136.32.132:5060..Call-ID: 106791-10754@83.136.32.91..CSeq: 1 I NVITE..Contact: sip:sipp@83.136.32.91:5062..Max-Forwards: 70..Subject: Performance Test..Content-Type: application/sdp..Content-Length: 135....v=0..o=u
graham:/home/darilion# Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR:parse_first_line: method not followed by SP Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR:parse_first_line: bad message Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR: parse_msg: message=<-106789-0^M From: sipp sip:sipp@83.136.32.91:5062;tag=106789^M To: sut sip:service@83.136.32.132:5060^M Call-ID: 106789-10754@83.136.32.91^M CSeq: 1 INVITE^M Contact: sip:sipp@83.136.32.91:5062^M Max-Forwards: 70^M Subject: Performance Test^M Content-Type: application/sdp^M Content-Length: 135^M ^M v=0^M o=user1 53655765 2353687637 IN IP4 83.136.32.91^M s=-^M c=IN IP4 83.136.32.91^M t=0 0^M m=audio 6002 RTP/AVP 0^M a=rtpmap:0 PCMU/8000^M > Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR: receive_msg: parse_msg failed
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
Daniel-Constantin Mierla wrote:
Hello,
this looks like the TCP kernel buffer is overflows. How many OpenSER TCP processes do you have?
children=100, --> ~208 processes. How can a TCP kernel buffer overflow? Shouldn't TCP flow control take care of this?
regards klaus
Cheers, Daniel
On 07/26/06 15:44, Klaus Darilion wrote:
Sorry - forgotten to attach the files :-)
klaus
Klaus Darilion wrote:
Hi!
I suspect openser to have TCP problems under high load: Scenario: sipp sends INVITE to openser, which replies with 404 I have 4x sipp with 1000 INVITEs/s and 1x sipp with 3000INVITEs/s, thus 5 TCP connections
After some time, I get error on the TCP connection with 3000INVITEs/s: -->syslog.txt
Looks like the first part of the request is missing, but sniffing revealed that the request is proper formated as you see in the TCP packet -->ngrep.txt
Are there known high-load problems?
Any ideas how I can debug this?
regards klaus
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
graham:/home/darilion# ngrep -I test6.cap -q 106789 input: test6.cap
T 83.136.32.91:32904 -> 83.136.32.132:5060 [A] INVITE sip:service@83.136.32.132:5060 SIP/2.0..Via: SIP/2.0/TCP 83.136.32.91:5062;branch=z9hG4bK-106789-0..From: sipp sip:sipp@83.136.32.91:5062;tag=106789..To: sut sip:service@83.136.32.132:5060..Call-ID: 106789-10754@83.136.32.91..CSeq: 1 INVITE..Contact: sip:sipp@83.136.32.91:5062..Max-Forwards: 70..Subject: Perform ance Test..Content-Type: application/sdp..Content-Length: 135....v=0..o=user1 53655765 2353687637 IN IP4 83.136.32.91..s=-..c=IN IP4 83.136.32.91..t=0 0..m=audio 6002 RTP/AVP 0..a=rtpmap:0 PCMU/8000..INVITE sip:service@83.136.32.132:5060 SIP/2.0..Via: SIP/2.0/TCP 83.136.32.91:5062;branch=z9hG4bK-106790-0..From: sipp <sip: sipp@83.136.32.91:5062>;tag=106790..To: sut sip:service@83.136.32.132:5060..Call-ID: 106790-10754@83.136.32.91..CSeq: 1 INVITE..Contact: sip:sipp@83.136.32.91:5 062..Max-Forwards: 70..Subject: Performance Test..Content-Type: application/sdp..Content-Length: 135....v=0..o=user1 53655765 2353687637 IN IP4 83.136.32.91..s=- ..c=IN IP4 83.136.32.91..t=0 0..m=audio 6002 RTP/AVP 0..a=rtpmap:0 PCMU/8000..INVITE sip:service@83.136.32.132:5060 SIP/2.0..Via: SIP/2.0/TCP 83.136.32.91:5062;br anch=z9hG4bK-106791-0..From: sipp sip:sipp@83.136.32.91:5062;tag=106791..To: sut sip:service@83.136.32.132:5060..Call-ID: 106791-10754@83.136.32.91..CSeq: 1 I NVITE..Contact: sip:sipp@83.136.32.91:5062..Max-Forwards: 70..Subject: Performance Test..Content-Type: application/sdp..Content-Length: 135....v=0..o=u
graham:/home/darilion# Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR:parse_first_line: method not followed by SP Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR:parse_first_line: bad message Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR: parse_msg: message=<-106789-0^M From: sipp sip:sipp@83.136.32.91:5062;tag=106789^M To: sut sip:service@83.136.32.132:5060^M Call-ID: 106789-10754@83.136.32.91^M CSeq: 1 INVITE^M Contact: sip:sipp@83.136.32.91:5062^M Max-Forwards: 70^M Subject: Performance Test^M Content-Type: application/sdp^M Content-Length: 135^M ^M v=0^M o=user1 53655765 2353687637 IN IP4 83.136.32.91^M s=-^M c=IN IP4 83.136.32.91^M t=0 0^M m=audio 6002 RTP/AVP 0^M a=rtpmap:0 PCMU/8000^M > Jul 26 14:29:08 graham /usr/sbin/openser[19736]: ERROR: receive_msg: parse_msg failed
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users