Hello,
the access control list in openser is based on group membership checking which does a database query each time when user's ACL is verified. We are considering to optimize this operations since they are very often used and propose the following solutions:
1. Load all groups to whom an user belongs once per request processing (one database query) and then store some bitmap flags to be used further when doing group checking. At start up time, some fixups will be made to replace the names with positions in bitmap - advantages: the old group table structure is used and changes in script should be minimal - disadvantages: after loading all group names, string comparisons are required to set the bitmap
2. Introduce a new column in the subscriber table that stores the ACL bitmap and load it once per request processing - advantages: very fast load and checking -- old version of group membership checking is kept - disadvantages: more complex provisioning system
What do you think? Any other idea?
Daniel
Hi Daniel,
The major drawback of the current ACL implementation is the number of DB queries required for checking the membership of a user to certain groups. In most real life scenarios, there are more than 4-5 groups (ex: disabled, voicemail, GW access, other service access ) - which means for each INVITE you will have like 2-3 DB queries; and that's only for ACLs!!
So, a new design which will reduce the number of queries is welcomed (even required).
Even if it's more radical, I will go for option nr 2: groups will be kept all together as a bit mask (32 groups should be ok); either in a grp table, either in subscribers table the mask may be load (in an AVP??) at auth (no extra query) or on request (only one query for all of them); during all script processing, any group may be test as many time as wanted without any penalties; also bitwise checkings will be more fast than string one. for provisioning and script fixing purposes, a second table will keep the association between each group name and it's bit mask; at OpenSER startup, the group name will be converted to bit mask.
Ex: grp_definition voicemail , 0x01 PSTN , 0x02 conference, 0x04
grp (subscriber) userx, 0x03 (voicemail and PSTN) usery, 0x05 (voicemail and conference)
regards, bogdan
Daniel-Constantin Mierla wrote:
Hello,
the access control list in openser is based on group membership checking which does a database query each time when user's ACL is verified. We are considering to optimize this operations since they are very often used and propose the following solutions:
- Load all groups to whom an user belongs once per request processing
(one database query) and then store some bitmap flags to be used further when doing group checking. At start up time, some fixups will be made to replace the names with positions in bitmap
- advantages: the old group table structure is used and changes in
script should be minimal
- disadvantages: after loading all group names, string comparisons are
required to set the bitmap
- Introduce a new column in the subscriber table that stores the ACL
bitmap and load it once per request processing
- advantages: very fast load and checking -- old version of group
membership checking is kept
- disadvantages: more complex provisioning system
What do you think? Any other idea?
Daniel
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
Hi,
Why stay with limiting stuff? This 32 limit I have already seen with the number of available flags ... and also with the sip_header_types defined (ever tried to define a new HDR_xxx type? there is no space available! ) ... why don't you try to find a way so that all this limitations are overcome? I know it is faster a bitmap check ... but maybe assigning plain index numbers and checking for equality would be almost as fast. This way, instead of 32 ... we can have 2^32! :)
Just my comment ...
Cesc
On 7/16/05, Bogdan-Andrei Iancu bogdan@voice-system.ro wrote:
Hi Daniel,
The major drawback of the current ACL implementation is the number of DB queries required for checking the membership of a user to certain groups. In most real life scenarios, there are more than 4-5 groups (ex: disabled, voicemail, GW access, other service access ) - which means for each INVITE you will have like 2-3 DB queries; and that's only for ACLs!!
So, a new design which will reduce the number of queries is welcomed (even required).
Even if it's more radical, I will go for option nr 2: groups will be kept all together as a bit mask (32 groups should be ok); either in a grp table, either in subscribers table the mask may be load (in an AVP??) at auth (no extra query) or on request (only one query for all of them); during all script processing, any group may be test as many time as wanted without any penalties; also bitwise checkings will be more fast than string one. for provisioning and script fixing purposes, a second table will keep the association between each group name and it's bit mask; at OpenSER startup, the group name will be converted to bit mask.
Ex: grp_definition voicemail , 0x01 PSTN , 0x02 conference, 0x04
grp (subscriber) userx, 0x03 (voicemail and PSTN) usery, 0x05 (voicemail and conference)regards, bogdan
Daniel-Constantin Mierla wrote:
Hello,
the access control list in openser is based on group membership checking which does a database query each time when user's ACL is verified. We are considering to optimize this operations since they are very often used and propose the following solutions:
- Load all groups to whom an user belongs once per request processing
(one database query) and then store some bitmap flags to be used further when doing group checking. At start up time, some fixups will be made to replace the names with positions in bitmap
- advantages: the old group table structure is used and changes in
script should be minimal
- disadvantages: after loading all group names, string comparisons are
required to set the bitmap
- Introduce a new column in the subscriber table that stores the ACL
bitmap and load it once per request processing
- advantages: very fast load and checking -- old version of group
membership checking is kept
- disadvantages: more complex provisioning system
What do you think? Any other idea?
Daniel
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
Hi Cesc,
that's a good thinking - let;s learn from history.
if we assign a group ID, the group IDs a user belongs to will be kept in a separate table (like grp now) and all IDs will be loaded together in avps with same ID/name; once loaded, only checking the AVP values will be enough for ACL membership.
In this case we will not be able to load the ACL at auth time (at least not from DB) and a separate query will be needed. But is a good compromise comparing with current approach.
Also I still consider keeping a table with group Id definitions names -> ID for an easy manipulation.
regards, Bogdan
PS: what I really like to this possible approach is that nothing more must be done :) - avpops will do it
Cesc wrote:
Hi,
Why stay with limiting stuff? This 32 limit I have already seen with the number of available flags ... and also with the sip_header_types defined (ever tried to define a new HDR_xxx type? there is no space available! ) ... why don't you try to find a way so that all this limitations are overcome? I know it is faster a bitmap check ... but maybe assigning plain index numbers and checking for equality would be almost as fast. This way, instead of 32 ... we can have 2^32! :)
Just my comment ...
Cesc
On 7/16/05, Bogdan-Andrei Iancu bogdan@voice-system.ro wrote:
Hi Daniel,
The major drawback of the current ACL implementation is the number of DB queries required for checking the membership of a user to certain groups. In most real life scenarios, there are more than 4-5 groups (ex: disabled, voicemail, GW access, other service access ) - which means for each INVITE you will have like 2-3 DB queries; and that's only for ACLs!!
So, a new design which will reduce the number of queries is welcomed (even required).
Even if it's more radical, I will go for option nr 2: groups will be kept all together as a bit mask (32 groups should be ok); either in a grp table, either in subscribers table the mask may be load (in an AVP??) at auth (no extra query) or on request (only one query for all of them); during all script processing, any group may be test as many time as wanted without any penalties; also bitwise checkings will be more fast than string one. for provisioning and script fixing purposes, a second table will keep the association between each group name and it's bit mask; at OpenSER startup, the group name will be converted to bit mask.
Ex: grp_definition voicemail , 0x01 PSTN , 0x02 conference, 0x04
grp (subscriber) userx, 0x03 (voicemail and PSTN) usery, 0x05 (voicemail and conference)
regards, bogdan
Daniel-Constantin Mierla wrote:
Hello,
the access control list in openser is based on group membership checking which does a database query each time when user's ACL is verified. We are considering to optimize this operations since they are very often used and propose the following solutions:
- Load all groups to whom an user belongs once per request processing
(one database query) and then store some bitmap flags to be used further when doing group checking. At start up time, some fixups will be made to replace the names with positions in bitmap
- advantages: the old group table structure is used and changes in
script should be minimal
- disadvantages: after loading all group names, string comparisons are
required to set the bitmap
- Introduce a new column in the subscriber table that stores the ACL
bitmap and load it once per request processing
- advantages: very fast load and checking -- old version of group
membership checking is kept
- disadvantages: more complex provisioning system
What do you think? Any other idea?
Daniel
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
Hello,
On 07/18/05 19:35, Bogdan-Andrei Iancu wrote:
Hi Cesc,
that's a good thinking - let;s learn from history.
if we assign a group ID, the group IDs a user belongs to will be kept in a separate table (like grp now) and all IDs will be loaded together in avps with same ID/name; once loaded, only checking the AVP values will be enough for ACL membership.
In this case we will not be able to load the ACL at auth time (at least not from DB) and a separate query will be needed. But is a good compromise comparing with current approach.
Also I still consider keeping a table with group Id definitions names -> ID for an easy manipulation.
regards, Bogdan
PS: what I really like to this possible approach is that nothing more must be done :) - avpops will do it
indeed, for this kind of approach nothing has to be developed in openser, but for sure there are provisioning interfaces that have to be upgraded.
The option 1) I proposed requires no changes in the database structure or in the provisioning interface -- it uses old structure but does some optimization for loading/testing and compatibility will be kept, too. It is not that fast as keeping just the bitmask in the subscriber table, but faster than current version.
So, the question is now whether it worth to optimize the old group module or drop it and use avpops for ACL (either for storing directly the bitmask or the id/name -- both requires no development)?
I would rather prefer to develop C code than changing my provisioning application :).
Daniel
Cesc wrote:
Hi,
Why stay with limiting stuff? This 32 limit I have already seen with the number of available flags ... and also with the sip_header_types defined (ever tried to define a new HDR_xxx type? there is no space available! ) ... why don't you try to find a way so that all this limitations are overcome? I know it is faster a bitmap check ... but maybe assigning plain index numbers and checking for equality would be almost as fast. This way, instead of 32 ... we can have 2^32! :)
Just my comment ... Cesc
On 7/16/05, Bogdan-Andrei Iancu bogdan@voice-system.ro wrote:
Hi Daniel,
The major drawback of the current ACL implementation is the number of DB queries required for checking the membership of a user to certain groups. In most real life scenarios, there are more than 4-5 groups (ex: disabled, voicemail, GW access, other service access ) - which means for each INVITE you will have like 2-3 DB queries; and that's only for ACLs!!
So, a new design which will reduce the number of queries is welcomed (even required).
Even if it's more radical, I will go for option nr 2: groups will be kept all together as a bit mask (32 groups should be ok); either in a grp table, either in subscribers table the mask may be load (in an AVP??) at auth (no extra query) or on request (only one query for all of them); during all script processing, any group may be test as many time as wanted without any penalties; also bitwise checkings will be more fast than string one. for provisioning and script fixing purposes, a second table will keep the association between each group name and it's bit mask; at OpenSER startup, the group name will be converted to bit mask.
Ex: grp_definition voicemail , 0x01 PSTN , 0x02 conference, 0x04
grp (subscriber) userx, 0x03 (voicemail and PSTN) usery, 0x05 (voicemail and conference)
regards, bogdan
Daniel-Constantin Mierla wrote:
Hello,
the access control list in openser is based on group membership checking which does a database query each time when user's ACL is verified. We are considering to optimize this operations since they are very often used and propose the following solutions:
- Load all groups to whom an user belongs once per request processing
(one database query) and then store some bitmap flags to be used further when doing group checking. At start up time, some fixups will be made to replace the names with positions in bitmap
- advantages: the old group table structure is used and changes in
script should be minimal
- disadvantages: after loading all group names, string comparisons are
required to set the bitmap
- Introduce a new column in the subscriber table that stores the ACL
bitmap and load it once per request processing
- advantages: very fast load and checking -- old version of group
membership checking is kept
- disadvantages: more complex provisioning system
What do you think? Any other idea?
Daniel
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
Daniel-Constantin Mierla writes:
What do you think? Any other idea?
i haven't used group module at all, because it really does not implement ACL capabilit, it just checks if a user belongs to a group. which group a request belongs to, is not addressed by group module.
group module is useful for simple group membership checking, for example, if a user is allowed to make a pstn call. but if we (actually telecom law in finland) complicate things a bit by introducing different types of pstn destinations, for example, national local, national long distance, national cellular, national long distance, international, different kinds of service numbers, etc. we would first need to figure out the group of the request by some module function AND then find out if caller belongs to that group.
an efficient solution would combine the two actions into one.
-- juha
Hi Juha,
Just correct me if I get it wrong - you are talking about the possibility of checking a dynamically selected group against a user; so far only fix group may be tested.
like, after some logic to figure out you want to test ACL "acl_test", name stored in an AVP - and later test if the user belongs to the group name stored in the AVP?
Is this right?
regards, Bogdan
Juha Heinanen wrote:
Daniel-Constantin Mierla writes:
What do you think? Any other idea?
i haven't used group module at all, because it really does not implement ACL capabilit, it just checks if a user belongs to a group. which group a request belongs to, is not addressed by group module.
group module is useful for simple group membership checking, for example, if a user is allowed to make a pstn call. but if we (actually telecom law in finland) complicate things a bit by introducing different types of pstn destinations, for example, national local, national long distance, national cellular, national long distance, international, different kinds of service numbers, etc. we would first need to figure out the group of the request by some module function AND then find out if caller belongs to that group.
an efficient solution would combine the two actions into one.
-- juha
Users mailing list Users@openser.org http://openser.org/cgi-bin/mailman/listinfo/users
Bogdan-Andrei Iancu writes:
Just correct me if I get it wrong - you are talking about the possibility of checking a dynamically selected group against a user; so far only fix group may be tested.
in this particular acl application there would be too many groups to be tested separately in ser.cfg.
like, after some logic to figure out you want to test ACL "acl_test", name stored in an AVP - and later test if the user belongs to the group name stored in the AVP?
Is this right?
yes. first group of request is determined based on r-uri and then a test is done if user belongs to that group.
-- juha
-
Bogdan-Andrei Iancu -
Cesc -
Daniel-Constantin Mierla -
Juha Heinanen