@miconda thanks for reviewing this. I chose to go with replacement since I believe this better represents the original intention of the weighted / relative weighted algorithms.

Now to be honest, I could be completely wrong in thinking that. At n>100 calls; or simply when scaling, the new algorithm distributes the same percentage of calls to each destination. The only difference being the ordering the destinations are selected.

If you think that is incorrect then I can just move it to a new dispatching algorithm.

And i'll update the formatting on those comments as well.

@devopsec pushed 1 commit.

13351fab04743058a30212f39ec39cc0dca81c08 dispatcher: fix weighted sort for n<100 calls

@henningw: the weight being expressed in percentage, you can't ensure proper accuracy for less than 100 calls, maybe in some particular cases, but not for most of the possible variants. Think about a simple to exemplify case of 100 destinations each with weight 1, if you have 2 calls, then two destinations get 50% of the traffic and the rest 0%. So try to fix that, I would love to see a solution.

@devopsec: I haven't checked the code, but I guess it is more like getting to a round robin with some destinations being many times in the group, like for example when there are two addresses, first with weight 75 and second with weight 25: first is added 3 times then the second (... and repeat). Is it randomised within these 4? Or just going to next one for each new call? If my guess is confirmed, then it is a new algorithm, the previous behaviour is lost, it is no longer a randomised distribution across 100 calls, but an ordered distribution (or randomised in group lower than 100).

@devopsec: looking at the updated commit, some fields of pointers are used before checking that the pointer is NULL: declaration of arrays using a field of a structure pointer -- we tried to avoid variable sized arrays in the past for compatibility with old compilers/C standards, I am not really against them, iirc they are allowed by C99, they have to bee relocated after the null check The old code does `pkg_malloc/free()` for `ds_dests_flags` for that reason.

@miconda yes the new code is not using randomized selection for the destination. It is selecting based on probability throughout (recalculate the percentage of calls to that destination each iteration, out of the total calls made). Think of the sorting loop as if it were "processing a new call" per each iteration of `i`. Maybe an explanation based on your above example; weight1=75, weight2=25, would clarify.

First iteration of `i` no calls have been processed, so the call goes to dst1. On the second iteration of `i` dst1 has 1/1 calls, 100% is greater than the intended weight, so we distribute the call to dst2. for `i=3` both dsts have 50% of the calls, but dst1 is intended to have 75% of the calls, so the call goes to dst1. This repeats until 100 "calls" have been processed.

The effect is the exact same in terms of distribution of calls as a percentage, i.e. the intention of the algorithm, but the "bug fix" is that distribution is equivalent at high call volumes and low call volumes. Thus, it is true that the new algorithm does not use random sorting under 100 calls like the last, but I believe that is a bug fix, not a new feature. I think that was a decent explanation, but if not please ask for more clarification.

On the topic of memory allocation, thanks for the reference, I am always happy to learn! In my live tests the compiler happily accepted my code and gdb gave me good output back... I don't tout to be a master of memory management so I will defer to you on whether `pkg_malloc/free()` is required here to be compliant with all the intended C specs.

@miconda Hmmm, I see your point there. No worries, I'll resubmit as a new algorithm.

@devopsec pushed 1 commit.

972723b29c20ae05be22285ca759414987424a68 htable add column packing features

Closed #3531.