6 Mar
2020
6 Mar
'20
8:58 p.m.
Hi,
On 4.4.7, since switching to db_mode 2 for usrloc with the following settings:
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "timer_procs", 2)
modparam("usrloc", "skip_remote_socket", 1)
modparam("usrloc", "timer_interval", 30)
modparam("usrloc", "matching_mode", 0)
I seem to run into a situation where, despite starting from zero local
registrations and an empty DB `location` table, out of ~1700
registrations visible in the runtime usrloc table, about 60-80 never
make it into the database.
This is because they're being continuously UPDATE'd rather than
INSERT'd, so there is some presumption that they should already be in
the Postgres database. But they're not.
This gap seems to very slowly grow over time.
I understand that this option is intended to deal with a scenario like
that:
https://kamailio.org/docs/modules/5.3.x/modules/usrloc.html#usrloc.p.db_che…
but the documentation doesn't say it needs to be on for db_mode 2 to
work correctly, and it's off by default.
I'm wondering what causes certain registrants to get into this state in
the first place, where the registrar presumes that they've already been
synced to the DB and only need to be update'd on renewal, where in fact
they have not been synced.
Causes I've ruled out:
1) The contacts being ignored due to non-local socket or other
invalidation;
2) Contacts being explicitly deleted by an outside process.
Any ideas appreciated!
-- Alex
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
6 Mar
6 Mar
9:31 p.m.
And another question that goes along with this:
Do I need `db_timer_clean` to make db_mode 2 work as expected?
A reasonable person would conclude from the documentation for
`timer_interval` and `db_mode 2` ...
"2 - Write-Back scheme. This is a combination of previous two
schemes. All changes are made to memory and database synchronization
is done in the timer. The timer deletes all expired contacts and
flushes all modified or new contacts to database."
"The module uses a timer to delete expired contacts, synchronize with
database and other tasks, that need to be run periodically."
... that deletion of expired contacts is propagated to underlying DB
storage (passed through the logic of whichever db_mode) in the form of
periodic deletions of expired contacts.
It appears not to be enabled by default. DELETE statements appear to be
issued periodically from usrloc even though it's not enabled.
So, what's that setting for? When and why do I need it?
-- Alex
On Fri, Mar 06, 2020 at 12:58:16PM -0500, Alex Balashov wrote:
Hi,
On 4.4.7, since switching to db_mode 2 for usrloc with the following settings:
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "timer_procs", 2)
modparam("usrloc", "skip_remote_socket", 1)
modparam("usrloc", "timer_interval", 30)
modparam("usrloc", "matching_mode", 0)
I seem to run into a situation where, despite starting from zero local
registrations and an empty DB `location` table, out of ~1700
registrations visible in the runtime usrloc table, about 60-80 never
make it into the database.
This is because they're being continuously UPDATE'd rather than
INSERT'd, so there is some presumption that they should already be in
the Postgres database. But they're not.
This gap seems to very slowly grow over time.
I understand that this option is intended to deal with a scenario like
that:
https://kamailio.org/docs/modules/5.3.x/modules/usrloc.html#usrloc.p.db_che…
but the documentation doesn't say it needs to be on for db_mode 2 to
work correctly, and it's off by default.
I'm wondering what causes certain registrants to get into this state in
the first place, where the registrar presumes that they've already been
synced to the DB and only need to be update'd on renewal, where in fact
they have not been synced.
Causes I've ruled out:
1) The contacts being ignored due to non-local socket or other
invalidation;
2) Contacts being explicitly deleted by an outside process.
Any ideas appreciated!
-- Alex
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
7 Mar
7 Mar
9:26 a.m.
Hi,
There are multiple scenarios and possibilities for this behavior, please
check,
1. What kind of sip endpoints you have? Any TCP based transport e.g. TLS or
WSS endpoint? Are they mobile apps (especially iOS devices) with push
notification and "backgrounding" support?
2. How many distinct SIP re-registers (SQL Updates) you have at a time?
Please also check if they are lower then actual endpoints.
Thank you.
On Fri, Mar 6, 2020 at 7:32 PM Alex Balashov <abalashov(a)evaristesys.com>
wrote:
And another question that goes along with this:
Do I need `db_timer_clean` to make db_mode 2 work as expected?
A reasonable person would conclude from the documentation for
`timer_interval` and `db_mode 2` ...
"2 - Write-Back scheme. This is a combination of previous two
schemes. All changes are made to memory and database synchronization
is done in the timer. The timer deletes all expired contacts and
flushes all modified or new contacts to database."
"The module uses a timer to delete expired contacts, synchronize with
database and other tasks, that need to be run periodically."
... that deletion of expired contacts is propagated to underlying DB
storage (passed through the logic of whichever db_mode) in the form of
periodic deletions of expired contacts.
It appears not to be enabled by default. DELETE statements appear to be
issued periodically from usrloc even though it's not enabled.
So, what's that setting for? When and why do I need it?
-- Alex
On Fri, Mar 06, 2020 at 12:58:16PM -0500, Alex Balashov wrote:
Hi,
On 4.4.7, since switching to db_mode 2 for usrloc with the following
settings:
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "timer_procs", 2)
modparam("usrloc", "skip_remote_socket", 1)
modparam("usrloc", "timer_interval", 30)
modparam("usrloc", "matching_mode", 0)
I seem to run into a situation where, despite starting from zero local
registrations and an empty DB `location` table, out of ~1700
registrations visible in the runtime usrloc table, about 60-80 never
make it into the database.
This is because they're being continuously UPDATE'd rather than
INSERT'd, so there is some presumption that they should already be in
the Postgres database. But they're not.
This gap seems to very slowly grow over time.
I understand that this option is intended to deal with a scenario like
that:
https://kamailio.org/docs/modules/5.3.x/modules/usrloc.html#usrloc.p.db_che…
but the documentation doesn't say it needs to be on for db_mode 2 to
work correctly, and it's off by default.
I'm wondering what causes certain registrants to get into this state in
the first place, where the registrar presumes that they've already been
synced to the DB and only need to be update'd on renewal, where in fact
they have not been synced.
Causes I've ruled out:
1) The contacts being ignored due to non-local socket or other
invalidation;
2) Contacts being explicitly deleted by an outside process.
Any ideas appreciated!
-- Alex
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
9:35 a.m.
Hi,
The endpoints are just plain UDP.
I don’t have an answer on #2, except to say that the majority of endpoints are behind NAT
and therefore that their re-registration interval is quite low—perhaps 60-120 seconds.
I don’t have a great knowledge of Kamailio modules’ startup sequence, but my suspicion is
that a certain number of registrations might be flooding into SIP workers that are already
available and responsive while the database connections are still being initialised, but
before they are available. Or perhaps some other race condition along those general
lines.
That still leaves some questions surrounding the purpose and necessity of various
settings, however.
— Alex
—
Sent from mobile, with due apologies for brevity and errors.
On Mar 7, 2020, at 1:28 AM, M S
<shaheryarkh(a)gmail.com> wrote:
Hi,
There are multiple scenarios and possibilities for this behavior, please check,
1. What kind of sip endpoints you have? Any TCP based transport e.g. TLS or WSS endpoint?
Are they mobile apps (especially iOS devices) with push notification and
"backgrounding" support?
2. How many distinct SIP re-registers (SQL Updates) you have at a time? Please also check
if they are lower then actual endpoints.
Thank you.
On Fri, Mar 6, 2020 at 7:32 PM Alex Balashov
<abalashov(a)evaristesys.com> wrote:
And another question that goes along with this:
Do I need `db_timer_clean` to make db_mode 2 work as expected?
A reasonable person would conclude from the documentation for
`timer_interval` and `db_mode 2` ...
"2 - Write-Back scheme. This is a combination of previous two
schemes. All changes are made to memory and database synchronization
is done in the timer. The timer deletes all expired contacts and
flushes all modified or new contacts to database."
"The module uses a timer to delete expired contacts, synchronize with
database and other tasks, that need to be run periodically."
... that deletion of expired contacts is propagated to underlying DB
storage (passed through the logic of whichever db_mode) in the form of
periodic deletions of expired contacts.
It appears not to be enabled by default. DELETE statements appear to be
issued periodically from usrloc even though it's not enabled.
So, what's that setting for? When and why do I need it?
-- Alex
On Fri, Mar 06, 2020 at 12:58:16PM -0500, Alex Balashov wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users Hi,
On 4.4.7, since switching to db_mode 2 for usrloc with the following settings:
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "timer_procs", 2)
modparam("usrloc", "skip_remote_socket", 1)
modparam("usrloc", "timer_interval", 30)
modparam("usrloc", "matching_mode", 0)
I seem to run into a situation where, despite starting from zero local
registrations and an empty DB `location` table, out of ~1700
registrations visible in the runtime usrloc table, about 60-80 never
make it into the database.
This is because they're being continuously UPDATE'd rather than
INSERT'd, so there is some presumption that they should already be in
the Postgres database. But they're not.
This gap seems to very slowly grow over time.
I understand that this option is intended to deal with a scenario like
that:
https://kamailio.org/docs/modules/5.3.x/modules/usrloc.html#usrloc.p.db_che…
but the documentation doesn't say it needs to be on for db_mode 2 to
work correctly, and it's off by default.
I'm wondering what causes certain registrants to get into this state in
the first place, where the registrar presumes that they've already been
synced to the DB and only need to be update'd on renewal, where in fact
they have not been synced.
Causes I've ruled out:
1) The contacts being ignored due to non-local socket or other
invalidation;
2) Contacts being explicitly deleted by an outside process.
Any ideas appreciated!
-- Alex
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
11:25 a.m.
Well, since you have UDP endpoints, so most likely they are fixed endpoint
e.g. desk phones which almost never have network change. I know i am
assuming a lot here but this seems like a corporate network with hardware
router/firewall. The hardware routers usually have UDP NAT mapping
allocation interval of 25 seconds (e.g. CISCO ASA 55xx). It means that if
there is no UDP traffic on NATted connection for 25 seconds then router
will close the NAT mapping and SIP client will have to request for another
one when it sends next register after 60 seconds.
Also, if sip endpoint registers for 60 seconds then LCM(60, 25) = 300
seconds i.e. there will be one sip re-register (SQL UPDATE) after every 5
registers (SQL INSERT) for same endpoint due to matching_mode 0 (Contact
Only). If the SIP register interval is 120 seconds then LCM(120, 25) = 600
seconds, i.e. 1 in 12 will be SQL UPDATE. Now, since you have
timer_interval of 30 seconds [i.e. LCM(300, 30) = 300 and LCM(600, 30) =
600] so depending on sorting order of usrloc records fetched from db, the
SQL UPDATE is likely to fail (i.e. have no matching db record) and kept in
memory till next clean up.
Here are a few recommendations.
1. Enable UDP keepalives with very short intervals, e.g. 10 seconds to keep
NAT mapping at client side router open indefinitely. Set more then one
dedicated NAT ping processes if you have high number of active users
(roughly 1 process per 1K active users at peak times).
https://kamailio.org/docs/modules/4.4.x/modules/nathelper.html#nathelper.p.…
2. Enable db_check_update module option, to force insert the record in db,
this will convert orphan SQL Updates to SQL Inserts.
https://kamailio.org/docs/modules/4.4.x/modules/usrloc.html#usrloc.p.db_che…
Or alternatively, use db_update_as_insert (somewhat more efficient),
https://kamailio.org/docs/modules/4.4.x/modules/usrloc.html#usrloc.p.db_upd…
3. Enable db_clean_timer, it is kind of required option for db mode 2,
especially if sip endpoint disconnects from network and its network changes
or kamailio restarts, all so often.
https://kamailio.org/docs/modules/4.4.x/modules/usrloc.html#usrloc.p.db_tim…
4. Also optionally, consider using matching_mode 3 (Call-ID Only), this
solves a lot of duplication problems in SIP re-registration and SIP forking
over UDP transport.
https://kamailio.org/docs/modules/4.4.x/modules/usrloc.html#usrloc.p.matchi…
5. Optionally, for fixed sip endpoints, it is better to set larger SIP
expiry when possible. I recommend using minimum 300 seconds and maximum 600
seconds.
Hope this help.
On Sat, Mar 7, 2020 at 7:37 AM Alex Balashov <abalashov(a)evaristesys.com>
wrote:
Hi,
The endpoints are just plain UDP.
I don’t have an answer on #2, except to say that the majority of endpoints
are behind NAT and therefore that their re-registration interval is quite
low—perhaps 60-120 seconds.
I don’t have a great knowledge of Kamailio modules’ startup sequence, but
my suspicion is that a certain number of registrations might be flooding
into SIP workers that are already available and responsive while the
database connections are still being initialised, but before they are
available. Or perhaps some other race condition along those general lines.
That still leaves some questions surrounding the purpose and necessity of
various settings, however.
— Alex
—
Sent from mobile, with due apologies for brevity and errors.
On Mar 7, 2020, at 1:28 AM, M S <shaheryarkh(a)gmail.com> wrote:
Hi,
There are multiple scenarios and possibilities for this behavior, please
check,
1. What kind of sip endpoints you have? Any TCP based transport e.g. TLS
or WSS endpoint? Are they mobile apps (especially iOS devices) with push
notification and "backgrounding" support?
2. How many distinct SIP re-registers (SQL Updates) you have at a time?
Please also check if they are lower then actual endpoints.
Thank you.
On Fri, Mar 6, 2020 at 7:32 PM Alex Balashov <abalashov(a)evaristesys.com>
wrote:
And another question that goes along with this:
Do I need `db_timer_clean` to make db_mode 2 work as expected?
A reasonable person would conclude from the documentation for
`timer_interval` and `db_mode 2` ...
"2 - Write-Back scheme. This is a combination of previous two
schemes. All changes are made to memory and database synchronization
is done in the timer. The timer deletes all expired contacts and
flushes all modified or new contacts to database."
"The module uses a timer to delete expired contacts, synchronize with
database and other tasks, that need to be run periodically."
... that deletion of expired contacts is propagated to underlying DB
storage (passed through the logic of whichever db_mode) in the form of
periodic deletions of expired contacts.
It appears not to be enabled by default. DELETE statements appear to be
issued periodically from usrloc even though it's not enabled.
So, what's that setting for? When and why do I need it?
-- Alex
On Fri, Mar 06, 2020 at 12:58:16PM -0500, Alex Balashov wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi,
On 4.4.7, since switching to db_mode 2 for usrloc with the following
settings:
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "timer_procs", 2)
modparam("usrloc", "skip_remote_socket", 1)
modparam("usrloc", "timer_interval", 30)
modparam("usrloc", "matching_mode", 0)
I seem to run into a situation where, despite starting from zero local
registrations and an empty DB `location` table, out of ~1700
registrations visible in the runtime usrloc table, about 60-80 never
make it into the database.
This is because they're being continuously UPDATE'd rather than
INSERT'd, so there is some presumption that they should already be in
the Postgres database. But they're not.
This gap seems to very slowly grow over time.
I understand that this option is intended to deal with a scenario like
that:
https://kamailio.org/docs/modules/5.3.x/modules/usrloc.html#usrloc.p.db_che…
but the documentation doesn't say it needs to be on for db_mode 2 to
work correctly, and it's off by default.
I'm wondering what causes certain registrants to get into this state in
the first place, where the registrar presumes that they've already been
synced to the DB and only need to be update'd on renewal, where in fact
they have not been synced.
Causes I've ruled out:
1) The contacts being ignored due to non-local socket or other
invalidation;
2) Contacts being explicitly deleted by an outside process.
Any ideas appreciated!
-- Alex
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
11:19 p.m.
Hi Alex,
the db_timer_clean is just an additional mechanism to clean-up expired records. Like if
you get some errors from time to time in the DB connection, and some deletes are not
correctly processed on the DB. Then some stale records should be left in the DB, and you
can use this one to clean up.
Normally it should be not necessary, as Kamailio will synchronize it just by the normal
timer. From the source:
/*!
* \brief Write-back timer, used for WRITE_BACK db_mode
*
* Write-back timer, used for WRITE_BACK db_mode. Process
* all contacts from the record, delete expired ones from the DB.
* Furthermore it updates changed contacts, and also insert new
* ones in the DB.
* \param _r processed record
*/
static inline void wb_timer(urecord_t* _r) {..}
About the first question, why some registrations are missing in the DB - this needs to be
investigated more closely. As a workaround you could try (as already suggested) to use the
INSERT instead of UPDATE module parameter, e.g. with db_check_update.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
-----Original Message-----
From: sr-users <sr-users-bounces(a)lists.kamailio.org> On Behalf Of Alex Balashov
Sent: Friday, March 6, 2020 7:31 PM
To: sr-users(a)lists.kamailio.org
Subject: Re: [SR-Users] usrloc db_mode 2 and dropped contacts
And another question that goes along with this:
Do I need `db_timer_clean` to make db_mode 2 work as expected?
A reasonable person would conclude from the documentation for `timer_interval` and
`db_mode 2` ...
"2 - Write-Back scheme. This is a combination of previous two
schemes. All changes are made to memory and database synchronization
is done in the timer. The timer deletes all expired contacts and
flushes all modified or new contacts to database."
"The module uses a timer to delete expired contacts, synchronize with
database and other tasks, that need to be run periodically."
... that deletion of expired contacts is propagated to underlying DB storage (passed
through the logic of whichever db_mode) in the form of periodic deletions of expired
contacts.
It appears not to be enabled by default. DELETE statements appear to be issued
periodically from usrloc even though it's not enabled.
So, what's that setting for? When and why do I need it?
-- Alex
On Fri, Mar 06, 2020 at 12:58:16PM -0500, Alex Balashov wrote:
Hi,
On 4.4.7, since switching to db_mode 2 for usrloc with the following settings:
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "timer_procs", 2)
modparam("usrloc", "skip_remote_socket", 1)
modparam("usrloc", "timer_interval", 30)
modparam("usrloc", "matching_mode", 0)
I seem to run into a situation where, despite starting from zero local
registrations and an empty DB `location` table, out of ~1700
registrations visible in the runtime usrloc table, about 60-80 never
make it into the database.
This is because they're being continuously UPDATE'd rather than
INSERT'd, so there is some presumption that they should already be in
the Postgres database. But they're not.
This gap seems to very slowly grow over time.
I understand that this option is intended to deal with a scenario like
that:
https://kamailio.org/docs/modules/5.3.x/modules/usrloc.html#usrloc.p.d
b_check_update
but the documentation doesn't say it needs to be on for db_mode 2 to
work correctly, and it's off by default.
I'm wondering what causes certain registrants to get into this state
in the first place, where the registrar presumes that they've already
been synced to the DB and only need to be update'd on renewal, where
in fact they have not been synced.
Causes I've ruled out:
1) The contacts being ignored due to non-local socket or other
invalidation;
2) Contacts being explicitly deleted by an outside process.
Any ideas appreciated!
-- Alex
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
11:29 p.m.
Hi Henning,
Yes, the db_check_update approach provides an adequate solution. I already explored that
before making this post.
My interest in that issue is insofar as:
1) How do registrants come to go missing from the DB sync, if assured to be starting from
0?
2) If this is a quality that is innate to how db_mode 2 works, shouldn’t db_check_update
be on by default, and/or shouldn’t the documentation reflect that db_check_update is a
necessary or best practice alongside db_mode 2?
— Alex
—
Sent from my iPad
On Mar 7, 2020, at 3:19 PM, Henning Westerholt
<hw(a)skalatan.de> wrote:
Hi Alex,
the db_timer_clean is just an additional mechanism to clean-up expired records. Like if
you get some errors from time to time in the DB connection, and some deletes are not
correctly processed on the DB. Then some stale records should be left in the DB, and you
can use this one to clean up.
Normally it should be not necessary, as Kamailio will synchronize it just by the normal
timer. From the source:
/*!
* \brief Write-back timer, used for WRITE_BACK db_mode
*
* Write-back timer, used for WRITE_BACK db_mode. Process
* all contacts from the record, delete expired ones from the DB.
* Furthermore it updates changed contacts, and also insert new
* ones in the DB.
* \param _r processed record
*/
static inline void wb_timer(urecord_t* _r) {..}
About the first question, why some registrations are missing in the DB - this needs to be
investigated more closely. As a workaround you could try (as already suggested) to use the
INSERT instead of UPDATE module parameter, e.g. with db_check_update.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
-----Original Message-----
From: sr-users <sr-users-bounces(a)lists.kamailio.org> On Behalf Of Alex Balashov
Sent: Friday, March 6, 2020 7:31 PM
To: sr-users(a)lists.kamailio.org
Subject: Re: [SR-Users] usrloc db_mode 2 and dropped contacts
And another question that goes along with this:
Do I need `db_timer_clean` to make db_mode 2 work as expected?
A reasonable person would conclude from the documentation for `timer_interval` and
`db_mode 2` ...
"2 - Write-Back scheme. This is a combination of previous two
schemes. All changes are made to memory and database synchronization
is done in the timer. The timer deletes all expired contacts and
flushes all modified or new contacts to database."
"The module uses a timer to delete expired contacts, synchronize with
database and other tasks, that need to be run periodically."
... that deletion of expired contacts is propagated to underlying DB storage (passed
through the logic of whichever db_mode) in the form of periodic deletions of expired
contacts.
It appears not to be enabled by default. DELETE statements appear to be issued
periodically from usrloc even though it's not enabled.
So, what's that setting for? When and why do I need it?
-- Alex
On Fri, Mar 06, 2020 at 12:58:16PM -0500, Alex
Balashov wrote:
Hi,
On 4.4.7, since switching to db_mode 2 for usrloc with the following settings:
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "timer_procs", 2)
modparam("usrloc", "skip_remote_socket", 1)
modparam("usrloc", "timer_interval", 30)
modparam("usrloc", "matching_mode", 0)
I seem to run into a situation where, despite starting from zero local
registrations and an empty DB `location` table, out of ~1700
registrations visible in the runtime usrloc table, about 60-80 never
make it into the database.
This is because they're being continuously UPDATE'd rather than
INSERT'd, so there is some presumption that they should already be in
the Postgres database. But they're not.
This gap seems to very slowly grow over time.
I understand that this option is intended to deal with a scenario like
that:
https://kamailio.org/docs/modules/5.3.x/modules/usrloc.html#usrloc.p.d
b_check_update
but the documentation doesn't say it needs to be on for db_mode 2 to
work correctly, and it's off by default.
I'm wondering what causes certain registrants to get into this state
in the first place, where the registrar presumes that they've already
been synced to the DB and only need to be update'd on renewal, where
in fact they have not been synced.
Causes I've ruled out:
1) The contacts being ignored due to non-local socket or other
invalidation;
2) Contacts being explicitly deleted by an outside process.
Any ideas appreciated!
-- Alex
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users 
9 Mar
9 Mar
10:54 a.m.
Hello,
On 07.03.20 21:29, Alex Balashov wrote:
Hi Henning,
Yes, the db_check_update approach provides an adequate solution. I already explored that
before making this post.
My interest in that issue is insofar as:
1) How do registrants come to go missing from the DB sync, if assured to be starting from
0?
this can happen if there is some interruption on db connection, like db
server restart or freeze due to some locking done by external lengthy
sync/search operations. Normally there should be some error messages in
the syslog, either for kamailio or for db server.
As you run a rather old version there (I understand 4.4.x), it may also
be one of the old issues related to generation of ruid, iirc, at some
point there were some conflicts, but I do not recall when (what specific
cases ) and for what versions. Lately, for quite a few years, all was
smooth from code point of view, only disruptions to db connectivity or
external factors could end up here.
One more thing to take in consideration is firewall (+conntrack kernel
mods)/enterprise limits systems (like selinux) -- if you end up in high
rates of traffic, I notices that in some cases, without spotting a clear
pattern, those systems fire up and close connections or drop traffic.
Cheers,
Daniel
2) If this is a quality that is innate to how db_mode 2 works, shouldn’t db_check_update
be on by default, and/or shouldn’t the documentation reflect that db_check_update is a
necessary or best practice alongside db_mode 2?
— Alex
—
Sent from my iPad
--
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - March 9-11, 2020, Berlin - www.asipto.com
Kamailio World Conference - April 27-29, 2020, in Berlin -- www.kamailioworld.com
On Mar 7, 2020, at 3:19 PM, Henning Westerholt
<hw(a)skalatan.de> wrote:
Hi Alex,
the db_timer_clean is just an additional mechanism to clean-up expired records. Like if
you get some errors from time to time in the DB connection, and some deletes are not
correctly processed on the DB. Then some stale records should be left in the DB, and you
can use this one to clean up.
Normally it should be not necessary, as Kamailio will synchronize it just by the normal
timer. From the source:
/*!
* \brief Write-back timer, used for WRITE_BACK db_mode
*
* Write-back timer, used for WRITE_BACK db_mode. Process
* all contacts from the record, delete expired ones from the DB.
* Furthermore it updates changed contacts, and also insert new
* ones in the DB.
* \param _r processed record
*/
static inline void wb_timer(urecord_t* _r) {..}
About the first question, why some registrations are missing in the DB - this needs to be
investigated more closely. As a workaround you could try (as already suggested) to use the
INSERT instead of UPDATE module parameter, e.g. with db_check_update.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
-----Original Message-----
From: sr-users <sr-users-bounces(a)lists.kamailio.org> On Behalf Of Alex Balashov
Sent: Friday, March 6, 2020 7:31 PM
To: sr-users(a)lists.kamailio.org
Subject: Re: [SR-Users] usrloc db_mode 2 and dropped contacts
And another question that goes along with this:
Do I need `db_timer_clean` to make db_mode 2 work as expected?
A reasonable person would conclude from the documentation for `timer_interval` and
`db_mode 2` ...
"2 - Write-Back scheme. This is a combination of previous two
schemes. All changes are made to memory and database synchronization
is done in the timer. The timer deletes all expired contacts and
flushes all modified or new contacts to database."
"The module uses a timer to delete expired contacts, synchronize with
database and other tasks, that need to be run periodically."
... that deletion of expired contacts is propagated to underlying DB storage (passed
through the logic of whichever db_mode) in the form of periodic deletions of expired
contacts.
It appears not to be enabled by default. DELETE statements appear to be issued
periodically from usrloc even though it's not enabled.
So, what's that setting for? When and why do I need it?
-- Alex
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users On Fri, Mar 06, 2020 at 12:58:16PM -0500, Alex
Balashov wrote:
Hi,
On 4.4.7, since switching to db_mode 2 for usrloc with the following settings:
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "timer_procs", 2)
modparam("usrloc", "skip_remote_socket", 1)
modparam("usrloc", "timer_interval", 30)
modparam("usrloc", "matching_mode", 0)
I seem to run into a situation where, despite starting from zero local
registrations and an empty DB `location` table, out of ~1700
registrations visible in the runtime usrloc table, about 60-80 never
make it into the database.
This is because they're being continuously UPDATE'd rather than
INSERT'd, so there is some presumption that they should already be in
the Postgres database. But they're not.
This gap seems to very slowly grow over time.
I understand that this option is intended to deal with a scenario like
that:
https://kamailio.org/docs/modules/5.3.x/modules/usrloc.html#usrloc.p.d
b_check_update
but the documentation doesn't say it needs to be on for db_mode 2 to
work correctly, and it's off by default.
I'm wondering what causes certain registrants to get into this state
in the first place, where the registrar presumes that they've already
been synced to the DB and only need to be update'd on renewal, where
in fact they have not been synced.
Causes I've ruled out:
1) The contacts being ignored due to non-local socket or other
invalidation;
2) Contacts being explicitly deleted by an outside process.
Any ideas appreciated!
-- Alex
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
11:47 a.m.
Hi Alex,
about the second question - a bit hard to tell. I remember that some big deployments used
patches with a similar effect already many years ago. On the other hand, there have been
not much reports on list or tracker on these problems in the past.
The documentation can be always improved, of course. Maybe something in the way to support
against interruptions or similar events.
About to make db_check_update the default - it has no effect on DBs that do not support
the necessary affected_rows API function. But the functionality is rather small, and the
performance impact negligible, so it sounds like a good idea. Would be probably good to
ask again briefly on sr-users, if there are other opinion on this step.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
-----Original Message-----
From: sr-users <sr-users-bounces(a)lists.kamailio.org> On Behalf Of Alex Balashov
Sent: Saturday, March 7, 2020 9:30 PM
To: sr-users(a)lists.kamailio.org
Subject: Re: [SR-Users] usrloc db_mode 2 and dropped contacts
Hi Henning,
Yes, the db_check_update approach provides an adequate solution. I already explored that
before making this post.
My interest in that issue is insofar as:
1) How do registrants come to go missing from the DB sync, if assured to be starting from
0?
2) If this is a quality that is innate to how db_mode 2 works, shouldn’t db_check_update
be on by default, and/or shouldn’t the documentation reflect that db_check_update is a
necessary or best practice alongside db_mode 2?
— Alex
—
Sent from my iPad
On Mar 7, 2020, at 3:19 PM, Henning Westerholt
<hw(a)skalatan.de> wrote:
Hi Alex,
the db_timer_clean is just an additional mechanism to clean-up expired records. Like if
you get some errors from time to time in the DB connection, and some deletes are not
correctly processed on the DB. Then some stale records should be left in the DB, and you
can use this one to clean up.
Normally it should be not necessary, as Kamailio will synchronize it just by the normal
timer. From the source:
/*!
* \brief Write-back timer, used for WRITE_BACK db_mode
*
* Write-back timer, used for WRITE_BACK db_mode. Process
* all contacts from the record, delete expired ones from the DB.
* Furthermore it updates changed contacts, and also insert new
* ones in the DB.
* \param _r processed record
*/
static inline void wb_timer(urecord_t* _r) {..}
About the first question, why some registrations are missing in the DB - this needs to be
investigated more closely. As a workaround you could try (as already suggested) to use the
INSERT instead of UPDATE module parameter, e.g. with db_check_update.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/ Kamailio services –
https://gilawa.com
-----Original Message-----
From: sr-users <sr-users-bounces(a)lists.kamailio.org> On Behalf Of Alex
Balashov
Sent: Friday, March 6, 2020 7:31 PM
To: sr-users(a)lists.kamailio.org
Subject: Re: [SR-Users] usrloc db_mode 2 and dropped contacts
And another question that goes along with this:
Do I need `db_timer_clean` to make db_mode 2 work as expected?
A reasonable person would conclude from the documentation for `timer_interval` and
`db_mode 2` ...
"2 - Write-Back scheme. This is a combination of previous two
schemes. All changes are made to memory and database synchronization
is done in the timer. The timer deletes all expired contacts and
flushes all modified or new contacts to database."
"The module uses a timer to delete expired contacts, synchronize with
database and other tasks, that need to be run periodically."
... that deletion of expired contacts is propagated to underlying DB storage (passed
through the logic of whichever db_mode) in the form of periodic deletions of expired
contacts.
It appears not to be enabled by default. DELETE statements appear to be issued
periodically from usrloc even though it's not enabled.
So, what's that setting for? When and why do I need it?
-- Alex
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
On Fri, Mar 06, 2020 at 12:58:16PM -0500, Alex
Balashov wrote:
Hi,
On 4.4.7, since switching to db_mode 2 for usrloc with the following settings:
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "timer_procs", 2)
modparam("usrloc", "skip_remote_socket", 1)
modparam("usrloc", "timer_interval", 30)
modparam("usrloc", "matching_mode", 0)
I seem to run into a situation where, despite starting from zero
local registrations and an empty DB `location` table, out of ~1700
registrations visible in the runtime usrloc table, about 60-80 never
make it into the database.
This is because they're being continuously UPDATE'd rather than
INSERT'd, so there is some presumption that they should already be in
the Postgres database. But they're not.
This gap seems to very slowly grow over time.
I understand that this option is intended to deal with a scenario
like
that:
https://kamailio.org/docs/modules/5.3.x/modules/usrloc.html#usrloc.p.
d
b_check_update
but the documentation doesn't say it needs to be on for db_mode 2 to
work correctly, and it's off by default.
I'm wondering what causes certain registrants to get into this state
in the first place, where the registrar presumes that they've already
been synced to the DB and only need to be update'd on renewal, where
in fact they have not been synced.
Causes I've ruled out:
1) The contacts being ignored due to non-local socket or other
invalidation;
2) Contacts being explicitly deleted by an outside process.
Any ideas appreciated!
-- Alex
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
1719
days inactive
1722
days old
8 comments
4 participants
participants (4)
-
Alex Balashov -
Daniel-Constantin Mierla -
Henning Westerholt -
M S