Now that K v3.1.0 provides send_reply(), what is the preferred ideology about whether to send stateful or stateless negative error replies?
I mean in general, not in specific cases like digest authentication in the new 'auth' module, where, according to the docs, a transaction and stateful replies are required in order for enhanced nonce replay protection.
The traditional wisdom in past bodies of scripts and knowledge in the public realm has been, in my interpretation, that stateless replies are acceptable for final negative responses from script[1]. After all, a reply like "404 Not Found" from the proxy is basically fire-and-forget; why devote additional transactional memory to it while the transaction hold-down timer expires? We do not care if we receive an ACK, nor want anything further to do with this branch.
On the other hand, there is a point of view out there that mixing stateless and stateful replies is not good, despite the somewhat increased overhead and resource usage associated with making all replies stateful. This perspective emphasises that if the proxy is fundamentally operating in a stateful manner, it should do so with respect to all messages, rather than allowing a large category of replies to slip under the radar of TM and have possibly inconsistent results.
For now, I have changed my code to use send_reply() in all cases where sl_send_reply() was used in the past; it seems easier to let Kamailio/SR make the decision. However, in practice, most requests will result in the creation of a transaction, so it is like replacing sl_send_reply() with t_reply() in the majority of cases.
Am I missing anything? Any salient considerations I missed?
I am curious as to the consensus about the best practice here from the vantage point of the developers.
Thanks,
-- Alex
[1] Except in FAILURE-ROUTEs, from which only stateful replies may be issued.
On 10/7/10 10:42 AM, Alex Balashov wrote:
Now that K v3.1.0 provides send_reply(), what is the preferred ideology about whether to send stateful or stateless negative error replies?
I mean in general, not in specific cases like digest authentication in the new 'auth' module, where, according to the docs, a transaction and stateful replies are required in order for enhanced nonce replay protection.
That's a fair point -- transaction processing could be used as "nonce memory" to mitigate replay attacks. For that however transaction processing would have to be in script earlier before anything else (sending a positive reply, doing anything that's subject to authentication), and the nonce verification would have to be bound to transaction id. Otherwise a replay attack could occur just with a different transaction id, so that TM would not filter it out.
One could see it as a trade-off between a bit better replay-attack resilience and a bit higher vulnerability to memory exhaustion. Both kind attacks are possibly either way, so it is not a really either-or type of choice.
The traditional wisdom in past bodies of scripts and knowledge in the public realm has been, in my interpretation, that stateless replies are acceptable for final negative responses from script[1]. After all, a reply like "404 Not Found" from the proxy is basically fire-and-forget; why devote additional transactional memory to it while the transaction hold-down timer expires? We do not care if we receive an ACK, nor want anything further to do with this branch.
Yes, that has been the purpose.
On the other hand, there is a point of view out there that mixing stateless and stateful replies is not good, despite the somewhat increased overhead and resource usage associated with making all replies stateful.
The problem is that "somewhat increased" can be A LOT. transaction context is in order of kilobytes and creating transaction context on every single request can exhaust memory very very quickly. It is not just about evil attacks but also about resilience against different sorts of misconfigurations which flood the server with traffic. Which happens.
This perspective emphasises that if the proxy is fundamentally operating in a stateful manner, it should do so with respect to all messages, rather than allowing a large category of replies to slip under the radar of TM and have possibly inconsistent results.
For now, I have changed my code to use send_reply() in all cases where sl_send_reply() was used in the past; it seems easier to let Kamailio/SR make the decision. However, in practice, most requests will result in the creation of a transaction, so it is like replacing sl_send_reply() with t_reply() in the majority of cases.
Am I missing anything? Any salient considerations I missed?
I think the argumentation is perfectly right, it is just the operational trade-offs to be made.
I am curious as to the consensus about the best practice here from the vantage point of the developers.
I'm personally most worried about memory exhaustion, but that's really my choice which I would not advertise as generally applicable/best/whatsoever. Eventually that's why we haven't cemented one way or other.
-jiri
Thanks,
-- Alex
[1] Except in FAILURE-ROUTEs, from which only stateful replies may be issued.
On 10/07/2010 04:57 AM, Jiri Kuthan wrote:
The problem is that "somewhat increased" can be A LOT. transaction context is in order of kilobytes and creating transaction context on every single request can exhaust memory very very quickly. It is not just about evil attacks but also about resilience against different sorts of misconfigurations which flood the server with traffic. Which happens.
Thanks to you and Marius for taking the time to formulate thorough replies. I read them with interest!
Is the problem you mention above not mitigated to some extent by the fact that a transaction is not created until either t_newtran() or t_relay() is called? For example:
route {
...
if(is_method("INVITE")) {
...
if(!check_from()) { xlog("L_ERR", "[...] From URI user part does not " "match authorisation username!\n");
send_reply("403", "Forbidden"); exit; }
...
if(!t_relay()) sl_reply_error(); }
As far as I know, the 403 Forbidden above would go out statelessly, right?
So, I assume that the misconfigurations and memory exhaust vulnerabilities under discussion are -- if it is INVITEs we are talking about -- ones in which the request is relayed and the negative replies come end-to-end from upstream, right? But in that case, by virtue of having called t_relay(), the reply is already processed statefully.
Now, of course, there are other types of request handling where a transaction is implicitly created, but am I correct in my interpretation of INVITE handling vis-a-vis transaction creation above?
Thanks,
inline
On 10/7/10 11:11 AM, Alex Balashov wrote:
On 10/07/2010 04:57 AM, Jiri Kuthan wrote:
The problem is that "somewhat increased" can be A LOT. transaction context is in order of kilobytes and creating transaction context on every single request can exhaust memory very very quickly. It is not just about evil attacks but also about resilience against different sorts of misconfigurations which flood the server with traffic. Which happens.
Thanks to you and Marius for taking the time to formulate thorough replies. I read them with interest!
Is the problem you mention above not mitigated to some extent by the fact that a transaction is not created until either t_newtran() or t_relay() is called? For example:
route {
...
if(is_method("INVITE")) {
...
if(!check_from()) { xlog("L_ERR", "[...] From URI user part does not " "match authorisation username!\n");
send_reply("403", "Forbidden"); exit; }
...
if(!t_relay()) sl_reply_error(); }
As far as I know, the 403 Forbidden above would go out statelessly, right?
yes
So, I assume that the misconfigurations and memory exhaust vulnerabilities under discussion are -- if it is INVITEs we are talking about -- ones in which the request is relayed and the negative replies come end-to-end from upstream, right?
I was refering to all other checks you may want to do before you establish transaction context and forward. Particularly if you do authenticate. Generally the "default" way of scripting was based on the assumption "don't establish transaction/forward until it is clear you want to forward".
But in that case, by virtue of having called t_relay(), the reply is already processed statefully.
Yes, but I was making a point to a different case. Again -- I think the question is "when does the script writer choose that a request is good enough to be processed". Before all sort of tests the writer chooses to deploy are positive, stateless reply seems safer to me.
Now, of course, there are other types of request handling where a transaction is implicitly created, but am I correct in my interpretation of INVITE handling vis-a-vis transaction creation above?
Yes. Until you do some t_command, there is no transaction contet. Once you do, there is. The point is really in the script that incorporates some request filtering policy -- what is good enough to spend memory on...
-jiri
Thanks,
On Oct 07, 2010 at 10:57, Jiri Kuthan jiri@iptel.org wrote:
On 10/7/10 10:42 AM, Alex Balashov wrote:
Now that K v3.1.0 provides send_reply(), what is the preferred ideology about whether to send stateful or stateless negative error replies?
I mean in general, not in specific cases like digest authentication in the new 'auth' module, where, according to the docs, a transaction and stateful replies are required in order for enhanced nonce replay protection.
That's a fair point -- transaction processing could be used as "nonce memory" to mitigate replay attacks.
There's a bit of a misunderstanding here. Stateful mode when using nounce_count or one-time-nonces is required to avoid challenging a possible retransmission (and not to enhance protection). Consider the one-time-nonce example: each nonce is allowed only once, if seen more then once the message will be challenged. Now consider an authenticated message that is retransmitted: the first message will pass authentication, but it's retransmission will fail => the retransmission will be challenged. This can cause problems when the UA gets back an OK reply and a challenge reply for the same message (theoretically it should work if they arrive in order, but in practice...).
tm is used only to make sure retransmission are identified an not challenged again.
[...]
In rest I agree with Jiri: I would use sl_send_reply() wherever I could to send negative replies. The biggest problem right now is memory consumption and not creating transactions helps a lot (one has to remember that even a replied transaction is kept in memory for at least 5s and that is ignoring negative replies to INVITE when tm has to wait for the ACK first).
OTOH if you start ser/sr/kamailio with 10Gb of RAM, you don't have more then 1000-3000 cps and you are in a controlled environment (no DOS, crazy device floods a.s.o), then it doesn't matter.
Andrei
On 10/7/10 12:16 PM, Andrei Pelinescu-Onciul wrote:
On Oct 07, 2010 at 10:57, Jiri Kuthanjiri@iptel.org wrote:
There's a bit of a misunderstanding here.
...
Now consider an authenticated message that is retransmitted: the first message will pass authentication, but it's retransmission will fail =>
I admit I haven't thought it through but would it really fail? in both cases (retransmission and replay attack) it will resend initial answer (100 if forwarded, challenge if failed to authenticate) and do nothing downstream --> attacker won't gain an unfair advantage, won't it?
the retransmission will be challenged.
why if the original request passed authentication?
I see the point TM can't differentiate between retransmissions and replay attacks easily (unless we do more of nonce-based protection). But does it really matter?
-jiri
On Oct 11, 2010 at 19:01, Jiri Kuthan jiri@iptel.org wrote:
On 10/7/10 12:16 PM, Andrei Pelinescu-Onciul wrote:
On Oct 07, 2010 at 10:57, Jiri Kuthanjiri@iptel.org wrote:
There's a bit of a misunderstanding here.
...
Now consider an authenticated message that is retransmitted: the first message will pass authentication, but it's retransmission will fail =>
I admit I haven't thought it through but would it really fail? in both cases (retransmission and replay attack) it will resend initial answer (100 if forwarded, challenge if failed to authenticate) and do nothing downstream --> attacker won't gain an unfair advantage, won't it?
If you set the one-time-nonce option (which causes the auth module to keep state and remember seen nonces) or qop+nc (nonce-count), then the retr. will be challenged (because the auth module has already seen the nonce in the original request and the auth module cannot tell retransmitted requests by itself).
the retransmission will be challenged.
why if the original request passed authentication?
I see the point TM can't differentiate between retransmissions and replay attacks easily (unless we do more of nonce-based protection). But does it really matter?
It's not about any kind of attacks. The auth module if configured with one-time-nonces or nonce_count, needs tm for taking care of the possible retransmissions. Without tm, the auth module will challenge retransmissions. With tm, tm will take care of the retransmissions and send back the same reply the original request got.
Consider the following scenario: (auth configured with one-time-nonce, no tm)
0. INVITE w/o auth-> <- challenge
1. INVITE w/auth => ok passes through
2. retr. INVITE w/auth (1) <- challenge (because auth has already seen this nonce)
With tm, (2) will be identified as retr. by tm, and the auth functions will not be called on it => ok.
Andrei
-
Alex Balashov -
Andrei Pelinescu-Onciul -
Jiri Kuthan