Hi, I wonder how feasible is the folowing:
I have a regular expresion like:
REGEXP = "^((1|20)|(13|3|4)|(5|6))"
There are 3 matching groups:
1) (1|20) 2) (13|3|4) 3) (5|6)
I want to match the RURI user, let's suppose $rU = "138787686223". In this case the above regexp matches groups 1 and 2, but since group 2 matching is longest ("13" is longest than "1") I want to get value 2.
In case $rU = "999999" I need to get "null", in case $rU = "1000022323" I need to get "1".
AFAIK there is no way to do it with current regular expression utils in Kamailio, but maybe I miss something. Thanks a lot.
Hello,
On 6/6/11 10:25 AM, Iñaki Baz Castillo wrote:
Hi, I wonder how feasible is the folowing:
I have a regular expresion like:
REGEXP = "^((1|20)|(13|3|4)|(5|6))"
There are 3 matching groups:
- (1|20)
- (13|3|4)
- (5|6)
I want to match the RURI user, let's suppose $rU = "138787686223". In this case the above regexp matches groups 1 and 2, but since group 2 matching is longest ("13" is longest than "1") I want to get value 2.
In case $rU = "999999" I need to get "null", in case $rU = "1000022323" I need to get "1".
AFAIK there is no way to do it with current regular expression utils in Kamailio, but maybe I miss something. Thanks a lot.
one option that comes in my mind is to break it in several if conditions, matching one group at a time, written in the order of preferences.
Cheers, Daniel
2011/6/6 Daniel-Constantin Mierla miconda@gmail.com:
one option that comes in my mind is to break it in several if conditions, matching one group at a time, written in the order of preferences.
The problem is that it would require perform N regular expression matching and such expressions will be very long (all the country prefixes in the world). The main problem is that some groups would contain prefixes starting with, for example, 12, while in other group there is the prefix 1 (an the very same in multiple cases, so doing it manually would be too much manual work and any change in the world would require hard modifications).
I'm evaluating building a custom function for this purpose.
Thanks.
Iñaki Baz Castillo writes:
The problem is that it would require perform N regular expression matching and such expressions will be very long (all the country prefixes in the world). The main problem is that some groups would contain prefixes starting with, for example, 12, while in other group there is the prefix 1 (an the very same in multiple cases, so doing it manually would be too much manual work and any change in the world would require hard modifications).
perhaps you could use dialplan module. it there is overlapping regular expressions, one is selected based on its priority.
-- juha
2011/6/6 Juha Heinanen jh@tutpro.com:
perhaps you could use dialplan module. it there is overlapping regular expressions, one is selected based on its priority.
Thanks. I don't need different priorities, I just need that, in case of multiple matching (i.e. ^1 , ^10) the longest matching wins. Does the dialplan module allow it? By reading the doc it seems that it's not possible, so I should need to assign lower priority to those shorter country prefixes. It could be a solution however.
Thanks to both.
Iñaki Baz Castillo writes:
Thanks. I don't need different priorities, I just need that, in case of multiple matching (i.e. ^1 , ^10) the longest matching wins. Does the dialplan module allow it?
try by giving rule ^1 lower priority than ^10.
-- juha
2011/6/6 Juha Heinanen jh@tutpro.com:
Thanks. I don't need different priorities, I just need that, in case of multiple matching (i.e. ^1 , ^10) the longest matching wins. Does the dialplan module allow it?
try by giving rule ^1 lower priority than ^10.
Yes, the trick will be:
priority = 20 - strlen(match_exp)
so longest expressions would have higher priority.
Hello,
On 6/6/11 10:39 AM, Iñaki Baz Castillo wrote:
2011/6/6 Daniel-Constantin Mierlamiconda@gmail.com:
one option that comes in my mind is to break it in several if conditions, matching one group at a time, written in the order of preferences.
The problem is that it would require perform N regular expression matching and such expressions will be very long (all the country prefixes in the world). The main problem is that some groups would contain prefixes starting with, for example, 12, while in other group there is the prefix 1 (an the very same in multiple cases, so doing it manually would be too much manual work and any change in the world would require hard modifications).
I'm evaluating building a custom function for this purpose.
maybe dialplan module could give you a solution -- you can play with priorities for matching rules.
Cheers, Daniel
On Monday 06 June 2011, Iñaki Baz Castillo wrote:
[..] I want to match the RURI user, let's suppose $rU = "138787686223". In this case the above regexp matches groups 1 and 2, but since group 2 matching is longest ("13" is longest than "1") I want to get value 2.
Hi Iñaki,
if you just want to have the longest match on a prefix tree, this is what carrierroute was build for.
In case $rU = "999999" I need to get "null", in case $rU = "1000022323" I need to get "1".
Its works a bit differently, but this is similar to the logic that we use. You've a bunch of specific prefixes with special routing, normal country prefixes that you want to route differently and then a default routing for the rest.
Best regards,
Henning
2011/6/6 Henning Westerholt henning.westerholt@1und1.de:
if you just want to have the longest match on a prefix tree, this is what carrierroute was build for.
In case $rU = "999999" I need to get "null", in case $rU = "1000022323" I need to get "1".
Its works a bit differently, but this is similar to the logic that we use. You've a bunch of specific prefixes with special routing, normal country prefixes that you want to route differently and then a default routing for the rest
Thanks Henning. I do know that CR and LCR operate with longest prefix matching, but I don't want this feature now for routing purposes, but for blacklist purposes. So using dialplan module with higher priorities in longer prefixes would be valid in my case.
Thanks a lot.
On Monday 06 June 2011, Iñaki Baz Castillo wrote:
Its works a bit differently, but this is similar to the logic that we use. You've a bunch of specific prefixes with special routing, normal country prefixes that you want to route differently and then a default routing for the rest
Thanks Henning. I do know that CR and LCR operate with longest prefix matching, but I don't want this feature now for routing purposes, but for blacklist purposes. So using dialplan module with higher priorities in longer prefixes would be valid in my case.
Hi Iñaki,
ah, for blacklisting purposes, i see. Then dialplan should be indeed a good choice then. We use the userblacklist module for this, it works with the same longest prefix match logic as cr.
Best regards,
Henning
2011/6/6 Henning Westerholt henning.westerholt@1und1.de:
We use the userblacklist module for this, it works with the same longest prefix match logic as cr.
Ops, I missed that module :) I'll take a look to it. Thanks.
2011/6/6 Iñaki Baz Castillo ibc@aliax.net:
2011/6/6 Henning Westerholt henning.westerholt@1und1.de:
We use the userblacklist module for this, it works with the same longest prefix match logic as cr.
Ops, I missed that module :) I'll take a look to it. Thanks.
It's not valid for my custom case as I need a function returning an id of the matched groups. This is, I have various geo-zone groups:
1: Europe (prefixes of Europe) 2: North America (prefixes of North America) ...
Each client has a boolean field to set permission for calls to each geo-zones.
So I think it's better for me to use dialplan module and return the geo-zone id in the "attrs" AVP.
Thanks.
On Monday 06 June 2011, Iñaki Baz Castillo wrote:
It's not valid for my custom case as I need a function returning an id of the matched groups. This is, I have various geo-zone groups:
1: Europe (prefixes of Europe) 2: North America (prefixes of North America) ...
Each client has a boolean field to set permission for calls to each geo-zones.
So I think it's better for me to use dialplan module and return the geo-zone id in the "attrs" AVP.
Yes - the dialplan is much more flexible of course. :-)
Henning
Iñaki Baz Castillo writes:
We use the userblacklist module for this, it works with the same longest prefix match logic as cr.
do both of them also support regular expressions, what inaki was asking for?
-- juha
2011/6/6 Juha Heinanen jh@tutpro.com:
do both of them also support regular expressions, what inaki was asking for?
Dialplan module allows setting a regular expression per table entry (by setting the column match_op to 1 (regexp).
Iñaki Baz Castillo writes:
Dialplan module allows setting a regular expression per table entry (by setting the column match_op to 1 (regexp).
sure, but i was asking about cr and the other module that henning was recommending.
-- juha
2011/6/6 Juha Heinanen jh@tutpro.com:
sure, but i was asking about cr and the other module that henning was recommending.
Well, for sure cr and userblacklist implements regular expression matching by taking the prefix from the DB and internally using "^PREFIX" (I expect). Same as LCR.
The fact is that I don't need more regular expressions than that. This is, I don't need to set a prefix like "(1|2[0123]{2})". :)
Hello Iñaki,
I did this a long time ago using a db backend, i can't remember exactly how.
the table would be:
areacode, route 1 route1 13 route2
I connected to mysql and ran something like:
select * from routes where '$rU' like concat(areacode,'%') order by len(areacode) desc limit 1;
that works perfectly
hope it helps
David On Mon, Jun 6, 2011 at 10:25 AM, Iñaki Baz Castillo ibc@aliax.net wrote:
Hi, I wonder how feasible is the folowing:
I have a regular expresion like:
REGEXP = "^((1|20)|(13|3|4)|(5|6))"
There are 3 matching groups:
- (1|20)
- (13|3|4)
- (5|6)
I want to match the RURI user, let's suppose $rU = "138787686223". In this case the above regexp matches groups 1 and 2, but since group 2 matching is longest ("13" is longest than "1") I want to get value 2.
In case $rU = "999999" I need to get "null", in case $rU = "1000022323" I need to get "1".
AFAIK there is no way to do it with current regular expression utils in Kamailio, but maybe I miss something. Thanks a lot.
-- Iñaki Baz Castillo ibc@aliax.net
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
2011/6/6 David Villasmil david.villasmil.work@gmail.com:
I connected to mysql and ran something like: select * from routes where '$rU' like concat(areacode,'%') order by len(areacode) desc limit 1;
Hi David, that would work indeed, but I prefer not to saturate the DB with such a query which cannot use table indexes. My kamailio handles 2000-3000 concurrent calls and such calls come from callcenters so they are very "aggressive" (maybe 200-300 calls in the same second), so querying the database with a low performance query is not an option in my case.
Thanks.
Hello Iñaki,
I had much the same traffic, and with a good index that wouldn't be a problem, IMHO. You could also load the table in memory, can't be faster than that.
Of course, doing it in the config script is faster, but it limits your flexibility... just a thought... I even got the rate and created a "custom" cdr for each call. we had around 4k concurrent calls at that time (like 5 years ago)
Anyway, good luck!
David
On Mon, Jun 6, 2011 at 2:52 PM, Iñaki Baz Castillo ibc@aliax.net wrote:
2011/6/6 David Villasmil david.villasmil.work@gmail.com:
I connected to mysql and ran something like: select * from routes where '$rU' like concat(areacode,'%') order by len(areacode) desc limit 1;
Hi David, that would work indeed, but I prefer not to saturate the DB with such a query which cannot use table indexes. My kamailio handles 2000-3000 concurrent calls and such calls come from callcenters so they are very "aggressive" (maybe 200-300 calls in the same second), so querying the database with a low performance query is not an option in my case.
Thanks.
-- Iñaki Baz Castillo ibc@aliax.net
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
2011/6/6 David Villasmil david.villasmil.work@gmail.com:
I had much the same traffic, and with a good index that wouldn't be a problem, IMHO.
Hi David, your query cannot make usage of table indexes:
select * from routes where '$rU' like concat(areacode,'%') order by len(areacode) desc limit 1;
You are using "LIKE" and concat(TABLE_COLUMN...) so you are forcing the database server to read *all* the values of "areacode" (all the rows), create a new string by concatening "%" at the end, and then performing a regular expressión (LIKE "XXX%") for *every* resulting values. This can never use an index.
You can check it by yourself by entering into mysql console:
DESCRIBE select * from routes where '999999999' like concat(areacode,'%') order by len(areacode) desc limit 1;
You will realize that no index is being used.
You could also load the table in memory, can't be faster than that.
Yes, but for that a custom module is required :)
Of course, doing it in the config script is faster, but it limits your flexibility... just a thought...
Humm, in fact not. I can manage the table content via a web interface (or whatever) and running a MI "reload" command for the used module so the table content is read again into memory. This is the same as I do with LCR module and others.
Regards.
On Jun 6, 2011, at 2:52 PM, Iñaki Baz Castillo ibc@aliax.net wrote:
2011/6/6 David Villasmil david.villasmil.work@gmail.com:
I connected to mysql and ran something like: select * from routes where '$rU' like concat(areacode,'%') order by len(areacode) desc limit 1;
Hi David, that would work indeed, but I prefer not to saturate the DB with such a query which cannot use table indexes. My kamailio handles 2000-3000 concurrent calls and such calls come from callcenters so they are very "aggressive" (maybe 200-300 calls in the same second), so querying the database with a low performance query is not an option in my case.
Btw, very lightweight module for longest prefix matching is mtree. It has option to return a value in a cfg variable, that can be the group id for example. It does caching in memory trees, so it is really fast in matching.
Cheers, Daniel
Thanks.
-- Iñaki Baz Castillo ibc@aliax.net
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
2011/6/6 Daniel-Constantine Mierla miconda@gmail.com:
Btw, very lightweight module for longest prefix matching is mtree. It has option to return a value in a cfg variable, that can be the group id for example. It does caching in memory trees, so it is really fast in matching.
I also missed that module. Let's take a look to it.
Thanks.
2011/6/6 Iñaki Baz Castillo ibc@aliax.net:
2011/6/6 Daniel-Constantine Mierla miconda@gmail.com:
Btw, very lightweight module for longest prefix matching is mtree. It has option to return a value in a cfg variable, that can be the group id for example. It does caching in memory trees, so it is really fast in matching.
I also missed that module. Let's take a look to it.
Hi Daniel, sounds really specific for which I need :) Let me ask some doubts:
- You say "It has option to return a value in a cfg variable, that can be the group id for example". Do you mean pv_value AVP?: http://kamailio.org/docs/modules/3.1.x/modules/mtree.html#id2845134 Would that AVP be filled with the content of the "value" column for the matched row?
- Which is the meaning of "payload" here?: http://kamailio.org/docs/modules/3.1.x/modules/mtree.html#id2807543
- What does mt_match() return in case there is match? and when there is not?
Thanks a lot.
Hello,
On 6/6/11 3:29 PM, Iñaki Baz Castillo wrote:
2011/6/6 Iñaki Baz Castilloibc@aliax.net:
2011/6/6 Daniel-Constantine Mierlamiconda@gmail.com:
Btw, very lightweight module for longest prefix matching is mtree. It has option to return a value in a cfg variable, that can be the group id for example. It does caching in memory trees, so it is really fast in matching.
I also missed that module. Let's take a look to it.
Hi Daniel, sounds really specific for which I need :) Let me ask some doubts:
- You say "It has option to return a value in a cfg variable, that can
be the group id for example". Do you mean pv_value AVP?: http://kamailio.org/docs/modules/3.1.x/modules/mtree.html#id2845134
yes, but it can be any kind of writable variable, not restricted to AVP -- see the example in the readme where it is a $var()
Would that AVP be filled with the content of the "value" column for the matched row?
Exactly
- Which is the meaning of "payload" here?: http://kamailio.org/docs/modules/3.1.x/modules/mtree.html#id2807543
when it is 0, will return the value as string in varibale. The goal is to parse in cache the value into some internal structure. Right now there is also the option of 1, expecting some 'id=value', both being integers, with the purpose of returning all matches for the prefix ordered by value. It hasn't be tested extensively and thus not documented - initially work was on hold to get first xavps so the structure can be returned nicely to config, then lack of time kept it posponed.
- What does mt_match() return in case there is match? and when there is not?
True and false.
Cheers, Daniel
Thanks a lot.
2011/6/6 Daniel-Constantin Mierla miconda@gmail.com:
- You say "It has option to return a value in a cfg variable, that can
be the group id for example". Do you mean pv_value AVP?: http://kamailio.org/docs/modules/3.1.x/modules/mtree.html#id2845134
yes, but it can be any kind of writable variable, not restricted to AVP -- see the example in the readme where it is a $var()
Would that AVP be filled with the content of the "value" column for the matched row?
Exactly
- Which is the meaning of "payload" here?:
http://kamailio.org/docs/modules/3.1.x/modules/mtree.html#id2807543
when it is 0, will return the value as string in varibale. The goal is to parse in cache the value into some internal structure. Right now there is also the option of 1, expecting some 'id=value', both being integers, with the purpose of returning all matches for the prefix ordered by value. It hasn't be tested extensively and thus not documented - initially work was on hold to get first xavps so the structure can be returned nicely to config, then lack of time kept it posponed.
- What does mt_match() return in case there is match? and when there is
not?
True and false.
Thanks Daniel, please let me one question more: does the mtree module allow setting two different tables (with maybe also different columns names)? The function mt_match(mtree, pv, mode) seems to allow, as first argument, the "model/table" to use, but I see no way of defining different models, is it?
Thanks a lot.
2011/6/6 Iñaki Baz Castillo ibc@aliax.net:
Thanks Daniel, please let me one question more: does the mtree module allow setting two different tables (with maybe also different columns names)? The function mt_match(mtree, pv, mode) seems to allow, as first argument, the "model/table" to use, but I see no way of defining different models, is it?
Ok, it seems that using "mtree" parameter does the job (but columns must have same name in all the tables):
modparam("mtree", "mtree", "name=mytable;dbtable=routes;type=0;")
2011/6/6 Iñaki Baz Castillo ibc@aliax.net:
Thanks Daniel, please let me one question more: does the mtree module allow setting two different tables (with maybe also different columns names)? The function mt_match(mtree, pv, mode) seems to allow, as first argument, the "model/table" to use, but I see no way of defining different models, is it?
Ok, it seems that using "mtree" parameter does the job (but columns must have same name in all the tables):
modparam("mtree", "mtree", "name=mytable;dbtable=routes;type=0;")
Sorry for so many questions but, what is "tname_column"?:
3.4. tname_column (string) Name of 'tname' column.
The table scheme clearly shows id, tprefix and tvalue columns, tprefix_column and tvalue_column parameters can be used, as usual, to rename such columns but, what is "tname_column"?
Thanks.
On 6/6/11 5:26 PM, Iñaki Baz Castillo wrote:
2011/6/6 Iñaki Baz Castilloibc@aliax.net:
Thanks Daniel, please let me one question more: does the mtree module allow setting two different tables (with maybe also different columns names)? The function mt_match(mtree, pv, mode) seems to allow, as first argument, the "model/table" to use, but I see no way of defining different models, is it?
Ok, it seems that using "mtree" parameter does the job (but columns must have same name in all the tables):
modparam("mtree", "mtree", "name=mytable;dbtable=routes;type=0;")
Sorry for so many questions but, what is "tname_column"?:
3.4. tname_column (string) Name of 'tname' column.
The table scheme clearly shows id, tprefix and tvalue columns, tprefix_column and tvalue_column parameters can be used, as usual, to rename such columns but, what is "tname_column"?
The module can load on db table in a memory tree or one db table in many trees. For the second case, the db table name is a module parameter and in addition it must have an extra column to specify the memory tree name.
Cheers, Daniel
2011/6/6 Daniel-Constantin Mierla miconda@gmail.com:
The module can load on db table in a memory tree or one db table in many trees. For the second case, the db table name is a module parameter and in addition it must have an extra column to specify the memory tree name.
Great. So we can extract different mtrees from the same table, just by filtering by tname column (and then using the assigned name for the module function). Just great :)
Thanks a lot.
On 6/6/11 5:23 PM, Iñaki Baz Castillo wrote:
2011/6/6 Iñaki Baz Castilloibc@aliax.net:
Thanks Daniel, please let me one question more: does the mtree module allow setting two different tables (with maybe also different columns names)? The function mt_match(mtree, pv, mode) seems to allow, as first argument, the "model/table" to use, but I see no way of defining different models, is it?
Ok, it seems that using "mtree" parameter does the job (but columns must have same name in all the tables):
modparam("mtree", "mtree", "name=mytable;dbtable=routes;type=0;")
yes columns names are configurable per module -- maybe in the future will be per mtree.
Meanwhile maybe you can use mysql views to get to the column names expected by mtree.
Cheers, Daniel