Hello,
I need to query a database for every SIP request coming into Kamailio, but I want this to be handled as fast as possible, so I was thinking of loading the data I need in memory using the HTABLE or MTREE modules.
When the SIP request is coming from one of our carriers, the called number ($rU) must be used to get the data for this called number. Normally, I would query the database, using sqlops, and pass the value of $rU as the parameter and get the column values using the $dbr variable. Can this be accomplished with HTABLE or MTREE?
Also, the autoexpire parameter in HTABLE; once expired, will the data be reloaded again from the database?
Thanks,
Grant
Sure, you can do it that way. However, htable and mtree accommodate relatively primitive data sets.
Given sufficient table complexity and/or size, your best bet is to use an in-memory storage backend on the database side itself.
In MySQL, this is called:
http://dev.mysql.com/doc/refman/5.7/en/memory-storage-engine.html
-- Alex
On 07/03/2013 05:41 AM, Grant Bagdasarian wrote:
Hello,
I need to query a database for every SIP request coming into Kamailio, but I want this to be handled as fast as possible, so I was thinking of loading the data I need in memory using the HTABLE or MTREE modules.
When the SIP request is coming from one of our carriers, the called number ($rU) must be used to get the data for this called number. Normally, I would query the database, using sqlops, and pass the value of $rU as the parameter and get the column values using the $dbr variable. Can this be accomplished with HTABLE or MTREE?
Also, the autoexpire parameter in HTABLE; once expired, will the data be reloaded again from the database?
Thanks,
Grant
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
I am using mtree for the same purpose and it works great. It never expire. I had loaded about 7 millions records in mtree table and there is no performance issue at all.
Gary
On Wed, Jul 3, 2013 at 7:57 AM, Alex Balashov abalashov@evaristesys.comwrote:
Sure, you can do it that way. However, htable and mtree accommodate relatively primitive data sets.
Given sufficient table complexity and/or size, your best bet is to use an in-memory storage backend on the database side itself.
In MySQL, this is called:
http://dev.mysql.com/doc/**refman/5.7/en/memory-storage-**engine.htmlhttp://dev.mysql.com/doc/refman/5.7/en/memory-storage-engine.html
-- Alex
On 07/03/2013 05:41 AM, Grant Bagdasarian wrote:
Hello,
I need to query a database for every SIP request coming into Kamailio, but I want this to be handled as fast as possible, so I was thinking of loading the data I need in memory using the HTABLE or MTREE modules.
When the SIP request is coming from one of our carriers, the called number ($rU) must be used to get the data for this called number. Normally, I would query the database, using sqlops, and pass the value of $rU as the parameter and get the column values using the $dbr variable. Can this be accomplished with HTABLE or MTREE?
Also, the autoexpire parameter in HTABLE; once expired, will the data be reloaded again from the database?
Thanks,
Grant
______________________________**_________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/**cgi-bin/mailman/listinfo/sr-**usershttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Alex Balashov - Principal Evariste Systems LLC 235 E Ponce de Leon Ave Suite 106 Decatur, GA 30030 United States Tel: +1-678-954-0670 Web: http://www.evaristesys.com/, http://www.alexbalashov.com/
______________________________**_________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/**cgi-bin/mailman/listinfo/sr-**usershttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Alex, we're primarily using MSSQL as our database server and would like to keep it that way. But since Gary has accomplished this with MTREE, I'll give that a go. 7 million records is very impressive. Our tables won't have near that much records, just a few hundred.
I'd like to reload the data every x interval, but I don't see the MTREE having support for this, except a MI command mt_reload. I can't call this from within the configuration script right? If not, how does the TIMER module work? Does it run independently from requests, and gets triggered every set interval?
From: sr-users-bounces@lists.sip-router.org [mailto:sr-users-bounces@lists.sip-router.org] On Behalf Of Gary Chen Sent: Thursday, July 4, 2013 4:09 AM To: Kamailio (SER) - Users Mailing List Subject: Re: [SR-Users] Load database table in memory
I am using mtree for the same purpose and it works great. It never expire. I had loaded about 7 millions records in mtree table and there is no performance issue at all. Gary
On Wed, Jul 3, 2013 at 7:57 AM, Alex Balashov <abalashov@evaristesys.commailto:abalashov@evaristesys.com> wrote: Sure, you can do it that way. However, htable and mtree accommodate relatively primitive data sets.
Given sufficient table complexity and/or size, your best bet is to use an in-memory storage backend on the database side itself.
In MySQL, this is called:
http://dev.mysql.com/doc/refman/5.7/en/memory-storage-engine.html
-- Alex
On 07/03/2013 05:41 AM, Grant Bagdasarian wrote: Hello,
I need to query a database for every SIP request coming into Kamailio, but I want this to be handled as fast as possible, so I was thinking of loading the data I need in memory using the HTABLE or MTREE modules.
When the SIP request is coming from one of our carriers, the called number ($rU) must be used to get the data for this called number. Normally, I would query the database, using sqlops, and pass the value of $rU as the parameter and get the column values using the $dbr variable. Can this be accomplished with HTABLE or MTREE?
Also, the autoexpire parameter in HTABLE; once expired, will the data be reloaded again from the database?
Thanks,
Grant
_______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.orgmailto:sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Alex Balashov - Principal Evariste Systems LLC 235 E Ponce de Leon Ave Suite 106 Decatur, GA 30030 United States Tel: +1-678-954-0670tel:%2B1-678-954-0670 Web: http://www.evaristesys.com/, http://www.alexbalashov.com/
_______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.orgmailto:sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
On 07/04/2013 03:10 AM, Grant Bagdasarian wrote:
Alex, we’re primarily using MSSQL as our database server and would like to keep it that way. But since Gary has accomplished this with MTREE, I’ll give that a go. 7 million records is very impressive. Our tables won’t have near that much records, just a few hundred.
With a few hundred records, honestly, it makes no difference which data structure or algorithm you use.
On 07/04/2013 03:10 AM, Grant Bagdasarian wrote:
Alex, we’re primarily using MSSQL as our database server and would like to keep it that way. But since Gary has accomplished this with MTREE, I’ll give that a go. 7 million records is very impressive. Our tables won’t have near that much records, just a few hundred.
I’d like to reload the data every x interval, but I don’t see the MTREE having support for this, except a MI command mt_reload. I can’t call this from within the configuration script right? If not, how does the TIMER module work? Does it run independently from requests, and gets triggered every set interval?
I think what you want is rtimer:
http://kamailio.org/docs/modules/4.0.x/modules/rtimer.html
And indeed, you can think of it as a kind of cron daemon that runs outside the SIP message processor thread pool.
While there is no way to trigger an mtree reload from inside route script, you can use rtimer + the 'exec' module to trigger one from the outside, using the 'mt_reload' MI command that you mentioned above:
http://kamailio.org/docs/modules/4.0.x/modules/exec.html
-- Alex