cPanel & nginx

Well, its about time I posted something new and interesting. I have come to love nginx, its the best reverse proxy I have used, and it has some very nice features, but most importantly, its FAST.

Now, I know there are other sites that show you how to setup nginx on a cPanel server, but they don't go all the way, they still rely on the admin running a script to update the nginx configuration, and also manual configuration of mod_rpaf for Apache.

The problem is say admin X adds a new domain to the server, it wont work until nginx has been updated. Or, if a new IP is added to the server, it wont get logged correctly until the Apache configuration has been updated.

So here is how you do it properly

Install nginx

# yum install nginx

/etc/nginx/nginx.conf:

user                    nginx;
worker_processes        3;
error_log               /var/log/nginx/error.log;
pid                     /var/run/nginx.pid;

events {
        worker_connections      1024;
        use                     epoll;
}

http {
        server_names_hash_max_size      2048;
        include                         /etc/nginx/mime.types;
        default_type                    application/octet-stream;
        server_tokens                   off;

        sendfile                        on;
        tcp_nopush                      on;
        tcp_nodelay                     on;
        keepalive_timeout               10;
        ignore_invalid_headers          on;

        gzip                            on;
        gzip_http_version               1.0;
        gzip_comp_level                 1;
        gzip_min_length                 1100;
        gzip_buffers                    4 32k;
        gzip_types                      text/plain application/x-javascript text/xml text/php text/css application/javascript text/js;
        gzip_proxied                    any;
        gzip_disable                    msie6;

        client_header_timeout           3m;
        client_body_timeout             3m;
        send_timeout                    3m;
        connection_pool_size            256;
        client_header_buffer_size       4k;
        large_client_header_buffers     4 32k;
        request_pool_size               4k;
        output_buffers                  4 32k;
        postpone_output                 1460;

        server {
                listen          80;
                server_name     _;
                rewrite         ^(.*)$ http://primary_website.com$1 permanent;
        }

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/vhosts.d/*.conf;
}

You must update the bolded parts...

• worker_processes should be set to the same number of CPUs you have for best performance.
• primary_website.com should be changed to your primary website address, this will re-write direct IP address access to your website, instead of presenting the cPanel default page.

Make the vhosts.d path:

# mkdir -p /etc/nginx/vhosts.d

The build script (/usr/local/sbin/build-nginx-config):

#!/bin/bash

rm -rf /etc/nginx/vhosts.d
mkdir -p /etc/nginx/vhosts.d

pushd /var/cpanel/users &>/dev/null
for USER in *; do
 IP="`grep '^IP=' $USER | cut -d'=' -f2`"
 DNS=""
 NAMES=""
 while [ 1 ]; do
  RECORD=`grep "^DNS$DNS=" $USER`
  if [ $? -ne 0 ]; then break; fi
  FQDN=`echo $RECORD | cut -d'=' -f2`
  if [ -z "$FQDN" ] && [ -z "$DNS" ] ; then
   DNS=1;
   continue;
  fi

  if [ -z "$DNS" ]; then
   NAMES=".$FQDN"
   DNS=1;
  else
   if [ -z "$NAMES" ]; then
    NAMES=".$FQDN"
   else
    NAMES="$NAMES .$FQDN"
   fi
   let DNS=$DNS+1
  fi
 done

cat > "/etc/nginx/vhosts.d/$USER.conf" <
server {
 access_log off;
 error_log off;
 listen  80;
 server_name $NAMES;
 location / {
  proxy_pass  http://$IP:81/;
  proxy_set_header Host   \$host;
  proxy_set_header X-Real-IP  \$remote_addr;
  proxy_set_header X-Forwarded-For  \$proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Port \$server_port;
  proxy_hide_header X-Powered-By;
 }
}
EOF

 if [ -z "$IPLIST" ]; then
  IPLIST="$IP"
 else
  echo $IPLIST | grep -q $IP
  if [ $? -ne 0 ]; then
   IPLIST="$IPLIST $IP"
  fi
 fi
done
popd &>/dev/null

cat > "/usr/local/apache/conf/includes/pre_main_global.conf" <
LoadModule        rpaf_module modules/mod_rpaf.so
RPAF_Enable       On
RPAF_ProxyIPs     127.0.0.1 $IPLIST
RPAF_SetHostName  On
RPAF_SetHTTPS     On
RPAF_SetPort      On
EOF

service nginx reload
service httpd restart

WARNING: this will overwrite pre_main_global.conf, be sure that you copy anything that is in that file into this script just before the last bold EOF.

Make sure this file is executable:

chmod +x /usr/local/sbin/build-nginx-config

mod_rpaf

Please note that the settings and configuration here relies on a new version of mod_rpaf you can obtain from my git repository here: https://github.com/gnif/mod_rpaf. Click the top right "Download" button.

Now we just have to build and install it:

# tar -xvzf master
# cd gnif-mod_rpaf-*
# make
# make install

This version of mod_rpaf has been endorsed by the original author and accepted as the next version, it now has a few fixes for ipv6 and security, fixes the port that apache sees to the one the proxy is using, and sets the HTTPS environment variable if it is set in the headers.

Final steps:

• In cPanel under "Tweak Settings" change the HTTP port to 80 (not the HTTPS port!) and save.
• run: /usr/local/sbin/build-nginx-config
• run: chkconfig nginx on
• run: service nginx start

Now all thats left to do is make apache run the build-nginx-config script each time a domain is added or removed, this is through a script hook:

/scripts/postupdateuserdomains:

#!/bin/bash
/usr/local/sbin/build-nginx-config

Create the file if it does not exist, cPanel provides this file for exactly this kind of thing. Make sure that it has the execute bit set (chmod +x /scripts/postupdateuserdomains)