Description

S-CSCF stops processing REGISTER and INVITE requests.

Troubleshooting

Gdb stack traces revealed several processes trying to do lock_udmain() unsuccessfully.
Adding more debug pointed to the direction of ims_registrar_scscf [lookup.c:107]: lookup().
This function does lock_udomain() with one "slot" and releases it with another.

The lookup() locks domain with one value in the aor but before unlocking it at lockup.c:209 the said
aor is changed. Aor points to the URI that is rewritten and therefore the wrong slot is unlocked. In the log below the locked slot is 461 while unlocked is 427.

9(791) DEBUG: ims_registrar_scscf [lookup.c:90]: lookup(): looking for any type of terminal
9(791) DEBUG: ims_registrar_scscf [lookup.c:103]: lookup(): Looking for tel:+46xxxxxxxxxx
9(791) DEBUG: ims_usrloc_scscf [udomain.c:445]: lock_ulslot(): LOCKING UDOMAIN SLOT [461]
9(791) DEBUG: ims_registrar_scscf [lookup.c:119]: lookup(): Found a valid contact [sip:10.110.2.199:5060;alias=10.110.2.19950602]
9(791) DEBUG: [core/parser/parse_rr.c:464]: get_path_dst_uri(): path for branch: 'sip:term@pcscf.yyy.xxx.3gppnetwork.org;lr'
9(791) DEBUG: ims_usrloc_scscf [udomain.c:465]: unlock_ulslot(): UN-LOCKING UDOMAIN SLOT [427]

Reproduction

Debugging Data

(gdb) bt #0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 
#1 0x00007fecd3f61dbd in futex_get (lock=0x7feccf6a2214) at ../../core/mem/../futexlock.h:108 #2 0x00007fecd3f6832e in lock_ulslot (_d=0x7feccf70a718, i=461) at udomain.c:450 
#3 0x00007fecd3f68273 in lock_udomain (_d=0x7feccf70a718, _aor=0x7feccf72e5a8) at udomain.c:424 #4 0x00007fecd3ccd375 in update_contacts (msg=0x7fecd3f1b6c0 <_pv_treq>, _d=0x7feccf70a718, public_identity=0x7feccf73e0c8, assignment_type=2, s=0x7fff31beac58, ccf1=0x7fff31beac80, ccf2=0x7fff31beac90, ecf1=0x7fff31beaca0, ecf2=0x7fff31beacb0, contact_header=0x7feccf73e0e0) at save.c:886 #5 0x00007fecd3c6cbe5 in async_cdp_callback (is_timeout=0, param=0x7feccf73e0a0, saa=0x7feccf7b9430, elapsed_msecs=59) at cxdx_sar.c:252 
#6 0x00007fecd460fb18 in api_callback (p=0x7feccf69cd18, msg=0x7feccf7b9430, ptr=0x0) at api_process.c:120 #7 0x00007fecd46868aa in worker_process (id=0) at worker.c:346 
#8 0x00007fecd464636b in diameter_peer_start (blocking=0) at diameter_peer.c:242 
#9 0x00007fecd4622666 in cdp_child_init (rank=0) at cdp_mod.c:243 #10 0x00000000005d2fbc in init_mod_child (m=0x7fecd940f8a0, rank=0) at core/sr_module.c:943 #11 0x00000000005d2c5e in init_mod_child (m=0x7fecd9410550, rank=0) at core/sr_module.c:939 #12 0x00000000005d2c5e in init_mod_child (m=0x7fecd9410958, rank=0) at core/sr_module.c:939 #13 0x00000000005d2c5e in init_mod_child (m=0x7fecd9410d68, rank=0) at core/sr_module.c:939 #14 0x00000000005d2c5e in init_mod_child (m=0x7fecd9411560, rank=0) at core/sr_module.c:939 #15 0x00000000005d2c5e in init_mod_child (m=0x7fecd9411aa0, rank=0) at core/sr_module.c:939 #16 0x00000000005d2c5e in init_mod_child (m=0x7fecd9411f00, rank=0) at core/sr_module.c:939 #17 0x00000000005d3390 in init_child (rank=0) at core/sr_module.c:970 #18 0x000000000042539c in main_loop () at main.c:1701 #19 0x000000000042bdd5 in main (argc=6, argv=0x7fff31beb998) at main.c:2638

Log Messages

9(791) DEBUG: ims_registrar_scscf [lookup.c:90]: lookup(): looking for any type of terminal
9(791) DEBUG: ims_registrar_scscf [lookup.c:103]: lookup(): Looking for tel:+46xxxxxxxxxx
9(791) DEBUG: ims_usrloc_scscf [udomain.c:445]: lock_ulslot(): LOCKING UDOMAIN SLOT [461]
9(791) DEBUG: ims_registrar_scscf [lookup.c:119]: lookup(): Found a valid contact [sip:10.110.2.199:5060;alias=10.110.2.19950602]
9(791) DEBUG: [core/parser/parse_rr.c:464]: get_path_dst_uri(): path for branch: 'sip:term@pcscf.yyy.xxx.3gppnetwork.org;lr'
9(791) DEBUG: ims_usrloc_scscf [udomain.c:465]: unlock_ulslot(): UN-LOCKING UDOMAIN SLOT [427]

SIP Traffic

(paste your sip traffic here)

Possible Solutions

Make a copy of the aor string.

Additional Information

version: kamailio 5.1.5 (x86_64/linux) 
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144 MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown 
compiled with gcc 4.8.2
Linux scscf 4.14.67-coreos #1 SMP Mon Sep 10 23:14:26 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.