Monday, January 13, 2014

Geolocation detection with haproxy

A useful feature for a web application is the ability to detect the user's country of origin based on their source IP address. This used not to be possible in haproxy unless you applied Cyril Bonté's geolocation patches (see the end of this blog post for how exactly to do that if you don't want to live on the bleeding edge of haproxy). However, the latest development version of haproxy (which is 1.5-dev21 at this time) contains geolocation detection functionality.

Here's how to use the geolocation detection feature of haproxy:

1) Generate text file which maps IP address ranges to ISO country codes

This is done using Cyril's haproxy-geoip utility, which is available in his geolocation patches. Here's how to locate and run this utility:
  • clone patch git repo: git clone https://github.com/cbonte/haproxy-patches.git
  • the haproxy-geoip script is now available in haproxy-patches/geolocation/tools
    • for the script to run, you need to have the funzip utility available on your system (it's part of the unzip package in Ubuntu)
    • you also need the iprange binary, which you can 'make' from its source file available in the haproxy-1.5-dev21/contrib/iprange directory; once you generate the binary, copy it somewhere in your PATH so that haproxy-geoip can locate it
  • run haproxy-geoip, which prints its output (IP ranges associated to ISO country codes) to stdout, and capture stdout to a file: haproxy-geoip > geolocation.txt
  • copy geolocation.txt to /etc/haproxy
2) Set custom HTTP header based on geolocation

For this, haproxy provides the map_ip function, which locates the source IP (the predefined 'src' variable in the line below) in the IP range in geolocation.txt and returns the ISO country code. We assign this country code to the custom X-Country HTTP header:

http-request set-header X-Country %[src, map_ip(/etc/haproxy/geolocation.txt)]

If you didn't want to map the source IP to a country code, but instead wanted to inspect the value of an HTTP header such as X-Forwarded-For, you could do this:

http-request set-header X-Country %[req.hdr_ip(X-Forwarded-For,-1), map_ip(/etc/haproxy/geolocation.txt)]

3) Use geolocation in ACLs

Let's assume that if the country detected via geolocation is not US, then you want to send the user to a different backend. You can do that with an ACL. Note that we compare the HTTP header X-Country which we already set above to the string 'US' using the '-m str' string matching functionality of haproxy, and we also specify that we want a case insensitive comparison with '-i US':

acl acl_geoloc_us req.hdr(X-Country) -m str -i US
use_backend www-backend-non-us if !acl_geoloc_us

If you didn't want to set the custom HTTP header, you could use the map_ip function directly in the definition of the ACL, like this:

acl acl_geoloc_us %[src, map_ip(/etc/haproxy/geolocation.txt)] -m str -i US
use_backend www-backend-non-us if !acl_geoloc_us

Speaking of ACLs, here's an example of defining ACLs based on the existence of a cookie and based on the value of the cookie then choosing a backend based on those ACLs:

acl acl_cookie_country req.cook_cnt(country_code) eq 1
acl acl_cookie_country_us req.cook(country_code) -m str -i US
use_backend www-backend-non-us if acl_cookie_country !acl_cookie_country_us

And now for something completely different...which is what I mentioned in the beginning of this post: 

How to use the haproxy geolocation patches with the current stable (1.4) version of haproxy

a) Patch haproxy source code with gelocation patches, compile and install haproxy:
  • clone patch git repo: git clone https://github.com/cbonte/haproxy-patches.git
  • change to haproxy-1.4.24 directory
  • copy haproxy-1.4-geolocation.patc to the root of haproxy-1.4.24 
  • apply the patch: patch -p1 < haproxy-1.4-geolocation.patch
  • make clean
  • make TARGET=linux26
  • make install
b) Generate text file which maps IP address ranges to ISO country codes
  • install funzip: apt-get install unzip
  • create iprange binary
    • cd haproxy-1.4.24/contrib/iprange
    • make
    • the iprange binary will be created in the same folder. copy that to /usr/local/sbin
  • haproxy-geoip is located here: haproxy-patches/geolocation/tools
  • haproxy-geoip > geolocation.txt
  • copy geolocation.txt to /etc/haproxy 
c) Obtain country code based on source IP and use it in ACL

This is done via the special 'geolocate' statement and the 'geoloc' variable added to the haproxy configuration syntax by the geolocation patch:

geolocate src /etc/haproxy/geolocation.txt
acl acl-au geoloc eq AU
use_backend www-backend-au if acl-au


If instead of the source IP you want to map the value of the X-Forwarded-For header to a country, use:

geolocate hdr_ip(X-Forwarded-For,-1) /etc/haproxy/geolocation.txt

If you wanted to redirect to another location instead of using an ACL, use:

redirect location http://some.location.example.com:4567 if { geoloc AU }

That's it for now. I want to thank Cyril Bonté, the author of the geolocation patches, and Willy Tarreau, the author of haproxy, for their invaluable help and their amazingly fast responses to my emails. It's a pleasure to deal with such open source developers passionate about the software they produce.  Also thanks to my colleagues Zmer Andranigian for working on getting version 1.4 of haproxy to work with geolocation, and Jeff Roberts for working on getting 1.5-dev21 to work.

One last thing: haproxy-1.5-dev21 has been very stable in production for us, but of course test it thoroughly before deploying it in your environment.

Modifying EC2 security groups via AWS Lambda functions

One task that comes up again and again is adding, removing or updating source CIDR blocks in various security groups in an EC2 infrastructur...