Hi,
What is the relationship between the `ctl` modparams `binrpc_buffer_size`, `binrpc_max_body_size`, and `binrpc_struct_max_body_size`?
More especially, how interdependent are they? Is there a set of circumstances in which one would wish for the structure buffer to be smaller than the reply body buffer? What about the internal buffer? Is it used only for some kind of intermediate formatting, or does it also hold the entire reply in some aspect?
The reasons for asking are predictable: RPC command 'dlg.list' throws out "reply too big" on some occasions, and rather than simply increasing all of these drastically, I would like to learn more about which ones it makes sense to increase, and in what proportion with respect to each other. I also think this idea should be more clearly developed in the documentation.
Thanks,
-- Alex
Hello,
the ctl module was developed by SER developers during 2005-2008, so I do not have a first hand knowledge about its design, but based on past experiences:
- binrpc_max_body_size - the max size expected for the entire rpc response
- binrpc_struct_max_body_size - the max size expected for one structure in the rpc response
- binrpc_buffer_size - seems to be used for printf-like functions used to build the rpc response
A structure in rpc response is enclosed between { ... } and can also contain structures.
The ctl module uses system malloc by default for its internal buffers related to rpc response, so it doesn't impact the allocated pkg memory.
It might be better to switch to jsonrpcs, which does dynamic allocation for building rpc response (also using system malloc).
Cheers, Daniel
On 29.10.19 01:32, Alex Balashov wrote:
Hi,
What is the relationship between the `ctl` modparams `binrpc_buffer_size`, `binrpc_max_body_size`, and `binrpc_struct_max_body_size`?
More especially, how interdependent are they? Is there a set of circumstances in which one would wish for the structure buffer to be smaller than the reply body buffer? What about the internal buffer? Is it used only for some kind of intermediate formatting, or does it also hold the entire reply in some aspect?
The reasons for asking are predictable: RPC command 'dlg.list' throws out "reply too big" on some occasions, and rather than simply increasing all of these drastically, I would like to learn more about which ones it makes sense to increase, and in what proportion with respect to each other. I also think this idea should be more clearly developed in the documentation.
Thanks,
-- Alex
On Wed, Oct 30, 2019 at 09:05:46AM +0100, Daniel-Constantin Mierla wrote:
The ctl module uses system malloc by default for its internal buffers related to rpc response, so it doesn't impact the allocated pkg memory.
It might be better to switch to jsonrpcs, which does dynamic allocation for building rpc response (also using system malloc).
That's an interesting thought I hadn't considered. Kamcmd does support JSONRPC as an underlying transport in place of the binrpc protocol used by CTL, right?
On Wed, Oct 30, 2019 at 01:46:21PM -0400, Alex Balashov wrote:
On Wed, Oct 30, 2019 at 09:05:46AM +0100, Daniel-Constantin Mierla wrote:
The ctl module uses system malloc by default for its internal buffers related to rpc response, so it doesn't impact the allocated pkg memory.
It might be better to switch to jsonrpcs, which does dynamic allocation for building rpc response (also using system malloc).
That's an interesting thought I hadn't considered. Kamcmd does support JSONRPC as an underlying transport in place of the binrpc protocol used by CTL, right?
Oh, I see. kamcmd is binrpc-only... the main reason I used 'ctl' is because I became so accustomed to it, but really, interacting with JSONRPCS over FIFO is no worse, and yields actual JSON in the response. Thanks for the suggestion.
Hello,
On 30.10.19 19:00, Alex Balashov wrote:
On Wed, Oct 30, 2019 at 01:46:21PM -0400, Alex Balashov wrote:
On Wed, Oct 30, 2019 at 09:05:46AM +0100, Daniel-Constantin Mierla wrote:
The ctl module uses system malloc by default for its internal buffers related to rpc response, so it doesn't impact the allocated pkg memory.
It might be better to switch to jsonrpcs, which does dynamic allocation for building rpc response (also using system malloc).
That's an interesting thought I hadn't considered. Kamcmd does support JSONRPC as an underlying transport in place of the binrpc protocol used by CTL, right?
Oh, I see. kamcmd is binrpc-only... the main reason I used 'ctl' is because I became so accustomed to it, but really, interacting with JSONRPCS over FIFO is no worse, and yields actual JSON in the response. Thanks for the suggestion.
indeed the ctl is binrpc only.
The kamctl (bash script) or kamcli (python app) can be used with jsonrpcs module.
Cheers, Daniel
-
Alex Balashov -
Daniel-Constantin Mierla