Speed comparisons of HTTP clients in Perl

Back in 2011 I wrote Faster HTTP usage. At the time Curl was the winner for the way I use HTTP. Since then a few more HTTP client modules have appeared on the CPAN and I was nudged by Tim Bunce to repeat my testing with the newer Hijk and HTTP::Tiny. Initially, I added comments to the original blog entry but I ran into a problem with HTTP::Tiny and Clinton Gormley's patch that Tim had pointed me at.

If you just want to skip to the results click here

Clinton's results were that HTTP::Tiny with his keep_alive patch were significantly faster than the base HTTP::Tiny. His patch works by not closing the socket (and not sending Connection: close in the request headers) so long as the server, port, scheme and timeout have not changed. When I ran this version with keep_alive = 1 it was really slow (I mean like 79 seconds compared with 5s).

Out came Devel::NYTProf and I found the patched HTTP::Tiny was spending an inordinate amount of time in select() compared with the unpatched one or without keep_alive=1. However, examination of the changes for keep_alive only highlighted one possibly significant difference in the code paths when keep_alive was set and that was on each new request it called select(with 0 timeout) to check that there was nothing to read. How could a select that does not wait make so much difference - it doesn't, and I proved that.

When something is doing a lot of networking and it is mysteriously slow I always think of the nagle algorithmn. Nagle intervenes if you do consecutive writes and the second write is small and done before the ACK from the other end comes back. Nagle holds on to the data briefly in the hope you are going to write more. The normal way to disable nagle is to set TCP_NODELAY on the socket and I tried that but it made no difference. Then I went off on various wild goose chases and realised I hadn't set TCP_NODELAY correctly and it did in fact appear to fix the problem. So why? Surely HTTP::Tiny was not writing the get request in 2 writes - oh yes it is:

sub write_request_header {
    @_ == 4 || die(q/Usage: $handle->write_request_header(method, request_uri, headers)/ . "\n");
    my ($self, $method, $request_uri, $headers) = @_;

    return $self->write("$method $request_uri HTTP/1.1\x0D\x0A")
         + $self->write_header_lines($headers);

So, my simple change is to avoid 2 sequential writes and is:

diff --git a/lib/HTTP/Tiny.pm b/lib/HTTP/Tiny.pm
index 46d0604..40fc826 100644
--- a/lib/HTTP/Tiny.pm
+++ b/lib/HTTP/Tiny.pm
@@ -1104,11 +1104,13 @@ my %HeaderCase = (
     'x-xss-protection' => 'X-XSS-Protection',
 );
 
+# to avoid multiple small writes and hence nagle, you can pass the method line
+# combine writes.
 sub write_header_lines {
-    (@_ == 2 && ref $_[1] eq 'HASH') || die(q/Usage: $handle->write_header_line
-    my($self, $headers) = @_;
+    (@_ == 2 || @_ == 3 && ref $_[1] eq 'HASH') || die(q/Usage: $handle->write_
+    my($self, $headers, $prefix_data) = @_;
 
-    my $buf = '';
+    my $buf = (defined $prefix_data ? $prefix_data : '');
     while (my ($k, $v) = each %$headers) {
         my $field_name = lc $k;
         if (exists $HeaderCase{$field_name}) {
@@ -1279,8 +1281,7 @@ sub write_request_header {
     @_ == 4 || die(q/Usage: $handle->write_request_header(method, request_uri,
     my ($self, $method, $request_uri, $headers) = @_;
 
-    return $self->write("$method $request_uri HTTP/1.1\x0D\x0A")
-         + $self->write_header_lines($headers);
+    return $self->write_header_lines($headers, "$method $request_uri HTTP/1.1\x
 }
 
 sub _do_timeout {

It is possible, the POST side of HTTP::Tiny suffers from this too but I haven't looked as yet.

The results

So I've revised my benchmark and due to comments made in the past I've changed the GETed file to one which is 202 bytes in size instead of one 300K in size and do 5000 GETs instead of 2000 before. These are using Perl 5.16.0 on Ubuntu Linux and to a modified nginx server which returns microsecond file modification times and has various other changes I doubt are of any significance here.

Versions:

LWP::UserAgent 6.04
HTTP::GHTTP 1.07
WWW::Curl::Easy 4.15
Furl 3.01
Hijk 0.12
HTTP::Tiny Clinton's clone on git + patch above

HTTP::Tiny keep_alive=1:  6 wallclock secs ( 2.95 usr +  0.59 sys =  3.54 CPU) @ 1412.43/s (n=5000)
HTTP::Tiny keep_alive=0: 13 wallclock secs ( 7.04 usr +  2.51 sys =  9.55 CPU) @ 523.56/s (n=5000)
       LWP: 20 wallclock secs (14.35 usr +  2.71 sys = 17.06 CPU) @ 293.08/s (n=5000)
  LWPGHHTP: 13 wallclock secs ( 8.11 usr +  2.12 sys = 10.23 CPU) @ 488.76/s (n=5000)
     GHTTP:  6 wallclock secs ( 1.13 usr +  1.64 sys =  2.77 CPU) @ 1805.05/s (n=5000)
      curl:  4 wallclock secs ( 0.88 usr +  0.40 sys =  1.28 CPU) @ 3906.25/s (n=5000)
      furl:  6 wallclock secs ( 2.95 usr +  0.64 sys =  3.59 CPU) @ 1392.76/s (n=5000)
      hijk:  3 wallclock secs ( 0.75 usr +  0.20 sys =  0.95 CPU) @ 5263.16/s (n=5000)

and because Tim asked for it, here it is again with Dumbbench (and without LWP + GHTTP):

LWP: Ran 7031 iterations (2004 outliers).
LWP: Rounded run time per iteration: 3.92333e-03 +/- 4.1e-07 (0.0%)
GHTTP: Ran 6379 iterations (1196 outliers).
GHTTP: Rounded run time per iteration: 1.07975e-03 +/- 2.0e-07 (0.0%)
WWW::Curl::Easy: Ran 6379 iterations (1337 outliers).
WWW::Curl::Easy: Rounded run time per iteration: 8.1370e-04 +/- 1.9e-07 (0.0%)
Furl: Ran 5787 iterations (691 outliers).
Furl: Rounded run time per iteration: 1.21889e-03 +/- 2.4e-07 (0.0%)
hijk: Ran 6697 iterations (180 outliers).
hijk: Rounded run time per iteration: 7.498e-04 +/- 1.1e-06 (0.1%)
HTTP::Tiny keep_alive: Ran 7031 iterations (1905 outliers).
HTTP::Tiny keep_alive: Rounded run time per iteration: 2.61473e-03 +/- 3.6e-07 (0.0%)
HTTP::Tiny: Ran 6379 iterations (1246 outliers).
HTTP::Tiny: Rounded run time per iteration: 2.62002e-03 +/- 3.4e-07 (0.0%)

As you can see HTTP::Tiny + keep_alive is not at all bad for a pure perl module but Hijk seems quite amazing when you think Curl is mostly C. For us, Hijk is probably not an option as we use cookies and SSL which I don't think Hijk supports.

Comments

Speed comparisons of HTTP clients in Perl - Hijk

I got this reply from Ævar Arnfjörð Bjarmason via email as the openid logon on this site did not work for him.

Hijk does "support" cookies, it just doesn't support cookie jars. But you can set a Cookie header like any other header, read it back by reading the headers, and either include the cookie on the next request
or not.

However Hijk doesn't handle a bunch of other things for you like redirects etc. So if you want cookies you're probably communicating with something that expects a general UA, which Hijk really isn't, although it can be mostly used as such by writing some custom logic on top of it.

As for SSL we haven't needed it yet, but from looking at it I'm pretty sure that if I need it I'll just use nginx running on localhost as an ssl proxy. There's so much you can screw up in doing SSL correctly, and it seems much easier just to outsource it to an external process so Hijk can speak plain-old HTTP.

In our case if someone's compromised root on the box so they can read IPC communication like that we're screwed anyway, so it doesn't really reduce security to not do SSL entirely inside one process.