18 Jan
2016
18 Jan
'16
4:46 p.m.
Hi all,
with two colleagues (Camille Oudot and Giacomo Vacca) we have been working
to an asynchronous HTTP module.
The module uses libevent (http://libevent.org/) and cURL's multi inteface (
http://curl.haxx.se/libcurl/c/libcurl-multi.html). It instantiates N
workers at startup, each able to generate multiple requests in parallel and
to resume the execution of the routing script upon reception of the HTTP
reply (or on error); if the query is executed in a transaction context, the
transaction itself will be suspended and resumed before continuing the
routing logic execution.
The module, apart from implementing non blocking HTTP queries, also
provides the ability to specify some HTTPS options per request, to
customize HTTP headers and to use several HTTP methods
(GET,POST,PUT,DELETE) making it very suitable for interacting with RESTful
APIs.
We hope that this module can be useful to many.
As usual any feedback and improvement proposals are more than welcome!
Regards,
Federico
18 Jan
18 Jan
4:56 p.m.
On 18 Jan 2016, at 15:46, Federico Cabiddu
<federico.cabiddu(a)gmail.com> wrote:
Hi all,
with two colleagues (Camille Oudot and Giacomo Vacca) we have been working to an
asynchronous HTTP module.
The module uses libevent (http://libevent.org/ <http://libevent.org/>) and
cURL's multi inteface (http://curl.haxx.se/libcurl/c/libcurl-multi.html
<http://curl.haxx.se/libcurl/c/libcurl-multi.html>). It instantiates N workers at
startup, each able to generate multiple requests in parallel and to resume the execution
of the routing script upon reception of the HTTP reply (or on error); if the query is
executed in a transaction context, the transaction itself will be suspended and resumed
before continuing the routing logic execution.
The module, apart from implementing non blocking HTTP queries, also provides the ability
to specify some HTTPS options per request, to customize HTTP headers and to use several
HTTP methods (GET,POST,PUT,DELETE) making it very suitable for interacting with RESTful
APIs.
We hope that this module can be useful to many.
As usual any feedback and improvement proposals are more than welcome!
I would like to see if we can merge this work with my curl module instead of
having yet another HTTP interface (we already have too many). Daniel Stenberg, the author
of Curl, has indicated that having multiple libCurl bindings in the same process is not a
good thing, especially if we use HTTPS and have multiple OpenSSL initialisations.
If it’s too far from the CURL module maybe it can use the CURL API that Hugh created and
depend on curl.
If it’s impossible, we should propably make a note that one should use one, not both
modules, in the same configuration. If that’s the way forward, please steal the
pseudovariable I implemented to catch the return codes from curl and possibly some of the
modparams. :-)
/O
5:40 p.m.
Hi,
On Mon, Jan 18, 2016 at 3:56 PM, Olle E. Johansson <oej(a)edvina.net> wrote:
> I would like to see if we can merge this work with my curl module instead
> of having yet another HTTP interface (we already have too many). Daniel
> Stenberg, the author of Curl, has indicated that having multiple libCurl
> bindings in the same process is not a good thing, especially if we use
> HTTPS and have multiple OpenSSL initialisations.
> If it’s too far from the CURL module maybe
it can use the CURL API that
> Hugh created and depend on curl.
I agree that it would be good to have a single HTTP interface but the
curl's handlers management done with the multi interface is really
different and even the API is not enough. Maybe we could have a module
offering two way of working: one using classic (single) easy handlers and
the other using multi interface and async processing. It would be a good
piece of work anyway :)
If it’s impossible, we should propably make a note that one should use one,
> not both modules, in the same configuration. If that’s the way forward,
> please steal the
> pseudovariable I implemented to catch the return codes from curl and
> possibly some of the modparams. :-)
I agree also on updating the documentation (at least for the moment until
we find if there's a way to have a single module) and borrowing pv and
modparams from the CURL module :)
Thanks for the feedback!
Cheers,
Federico
3232
days inactive
3232
days old
2 comments
2 participants
participants (2)
-
Federico Cabiddu -
Olle E. Johansson