Showing posts with label cPanel. Show all posts
Showing posts with label cPanel. Show all posts

Tuesday, August 30, 2011

cPanel & nginx

Well, its about time I posted something new and interesting. I have come to love nginx, its the best reverse proxy I have used, and it has some very nice features, but most importantly, its FAST.

Now, I know there are other sites that show you how to setup nginx on a cPanel server, but they don't go all the way, they still rely on the admin running a script to update the nginx configuration, and also manual configuration of mod_rpaf for Apache.

The problem is say admin X adds a new domain to the server, it wont work until nginx has been updated. Or, if a new IP is added to the server, it wont get logged correctly until the Apache configuration has been updated.

So here is how you do it properly

Install nginx
# yum install nginx

/etc/nginx/nginx.conf:
user                    nginx;
worker_processes        3;
error_log               /var/log/nginx/error.log;
pid                     /var/run/nginx.pid;

events {
        worker_connections      1024;
        use                     epoll;
}

http {
        server_names_hash_max_size      2048;
        include                         /etc/nginx/mime.types;
        default_type                    application/octet-stream;
        server_tokens                   off;

        sendfile                        on;
        tcp_nopush                      on;
        tcp_nodelay                     on;
        keepalive_timeout               10;
        ignore_invalid_headers          on;

        gzip                            on;
        gzip_http_version               1.0;
        gzip_comp_level                 1;
        gzip_min_length                 1100;
        gzip_buffers                    4 32k;
        gzip_types                      text/plain application/x-javascript text/xml text/php text/css application/javascript text/js;
        gzip_proxied                    any;
        gzip_disable                    msie6;

        client_header_timeout           3m;
        client_body_timeout             3m;
        send_timeout                    3m;
        connection_pool_size            256;
        client_header_buffer_size       4k;
        large_client_header_buffers     4 32k;
        request_pool_size               4k;
        output_buffers                  4 32k;
        postpone_output                 1460;

        server {
                listen          80;
                server_name     _;
                rewrite         ^(.*)$ http://primary_website.com$1 permanent;
        }

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/vhosts.d/*.conf;
}

You must update the bolded parts...
  • worker_processes should be set to the same number of CPUs you have for best performance.
  • primary_website.com should be changed to your primary website address, this will re-write direct IP address access to your website, instead of presenting the cPanel default page.

Make the vhosts.d path:
# mkdir -p /etc/nginx/vhosts.d

The build script (/usr/local/sbin/build-nginx-config):
#!/bin/bash

rm -rf /etc/nginx/vhosts.d
mkdir -p /etc/nginx/vhosts.d

pushd /var/cpanel/users &>/dev/null
for USER in *; do
 IP="`grep '^IP=' $USER | cut -d'=' -f2`"
 DNS=""
 NAMES=""
 while [ 1 ]; do
  RECORD=`grep "^DNS$DNS=" $USER`
  if [ $? -ne 0 ]; then break; fi
  FQDN=`echo $RECORD | cut -d'=' -f2`
  if [ -z "$FQDN" ] && [ -z "$DNS" ] ; then
   DNS=1;
   continue;
  fi

  if [ -z "$DNS" ]; then
   NAMES=".$FQDN"
   DNS=1;
  else
   if [ -z "$NAMES" ]; then
    NAMES=".$FQDN"
   else
    NAMES="$NAMES .$FQDN"
   fi
   let DNS=$DNS+1
  fi
 done

cat > "/etc/nginx/vhosts.d/$USER.conf" <
server {
 access_log off;
 error_log off;
 listen  80;
 server_name $NAMES;
 location / {
  proxy_pass  http://$IP:81/;
  proxy_set_header Host   \$host;
  proxy_set_header X-Real-IP  \$remote_addr;
  proxy_set_header X-Forwarded-For  \$proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Port \$server_port;
  proxy_hide_header X-Powered-By;
 }
}
EOF

 if [ -z "$IPLIST" ]; then
  IPLIST="$IP"
 else
  echo $IPLIST | grep -q $IP
  if [ $? -ne 0 ]; then
   IPLIST="$IPLIST $IP"
  fi
 fi
done
popd &>/dev/null

cat > "/usr/local/apache/conf/includes/pre_main_global.conf" <
LoadModule        rpaf_module modules/mod_rpaf.so
RPAF_Enable       On
RPAF_ProxyIPs     127.0.0.1 $IPLIST
RPAF_SetHostName  On
RPAF_SetHTTPS     On
RPAF_SetPort      On
EOF

service nginx reload
service httpd restart

WARNING: this will overwrite pre_main_global.conf, be sure that you copy anything that is in that file into this script just before the last bold EOF.

Make sure this file is executable:

chmod +x /usr/local/sbin/build-nginx-config

mod_rpaf

Please note that the settings and configuration here relies on a new version of mod_rpaf you can obtain from my git repository here: https://github.com/gnif/mod_rpaf. Click the top right "Download" button.

Now we just have to build and install it:

# tar -xvzf master
# cd gnif-mod_rpaf-*
# make
# make install

This version of mod_rpaf has been endorsed by the original author and accepted as the next version, it now has a few fixes for ipv6 and security, fixes the port that apache sees to the one the proxy is using, and sets the HTTPS environment variable if it is set in the headers.


Final steps:

  • In cPanel under "Tweak Settings" change the HTTP port to 80 (not the HTTPS port!) and save.
  • run: /usr/local/sbin/build-nginx-config
  • run: chkconfig nginx on
  • run: service nginx start

Now all thats left to do is make apache run the build-nginx-config script each time a domain is added or removed, this is through a script hook:


/scripts/postupdateuserdomains:

#!/bin/bash
/usr/local/sbin/build-nginx-config

Create the file if it does not exist, cPanel provides this file for exactly this kind of thing. Make sure that it has the execute bit set (chmod +x /scripts/postupdateuserdomains)

Monday, May 3, 2010

cPanel secondary master DNS without cPanel

At work I manage a cPanel server for over 200 clients that we needed to setup a secondary DNS server for. Since we wanted to use the machine for more then just backup DNS, and we wanted to run Debian, the cPanel DNS Only solution is no solution for us.

Here is how to slave cPanel to a Bind9 server without breaking cPanel or having cPanel undo the sync changes.

On the slave server I installed bind (apt-get install bind9) and made a directory "/Scripts" where I write two scripts, one is a class to wrap up cPanel API functions, the other is the actual DNS zone sync script. Both were written in PHP since it is my language of choice and is very easy to interface to the cPanel API with.

The class
=================================
<?PHP
        class cPanel {
                private $host;
                private $user;
                private $hash;

                private function __construct($host, $user, $hash) {
                        $this->host = $host;
                        $this->user = $user;
                        $this->hash = preg_replace("/[\t\r\n ]/", "", $hash);
                }

                private static $instance;
                public static function getInstance($host, $user, $hash) {
                        if (!(cPanel::$instance instanceof cPanel))
                                cPanel::$instance = new cPanel($host, $user, $hash);
                        return cPanel::$instance;
                }

                private function getJSON($request) {
                        $query = "https://{$this->host}:2087/json-api/" . $request;
                        $curl = curl_init();
                        curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 1);
                        curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, 1);
                        curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
                        $header = array("Authorization: WHM {$this->user}:{$this->hash}");
                        curl_setopt($curl,CURLOPT_HTTPHEADER,$header);
                        curl_setopt($curl, CURLOPT_URL, $query);
                        $result = curl_exec($curl);
                        curl_close($curl);
                        if ($result == false)
                                throw new Exception("curl_exec threw error \"" . curl_error($curl) . "\" for $query");
                        return json_decode($result);
                }

                public function listzones() {
                        $info = $this->getJSON('listzones');
                        $zones = array();
                        foreach($info->zone as $zone)
                                $zones[$zone->domain] = $zone->zonefile;
                        return $zones;
                }
        }
?>

=================================
 

And the script:
=================================
#!/usr/bin/php
<?PHP ob_start(); ?
>
PUT YOUR cPanel ACCESS HASH HERE 
<?PHP
        $hash   = ob_get_clean();
        $master = '123.123.123.123'; //YOUR MASTER SERVER IP
        require('/Scripts/cPanel.php');
        $cpanel = cPanel::getInstance('www.YOUR_HOST.com', 'root', $hash);
        $zones = $cpanel->listzones();
        if (count($zones) == 0)
                throw new Exception('Unable to retrieve the server\'s zone list');

        $fp = fopen('/etc/bind/cpanel.zones', 'w');
        foreach($zones as $zone => $file) {
                $zone =
                        "zone \"{$zone}\" {\n" .
                        "       type slave;\n" .
                        "       file \"{$file}\";\n" .
                        "       masters { {$master}; };\n" .
                        "};\n";

                fwrite($fp, $zone);
        }
        fclose($fp);
        shell_exec('/usr/sbin/rndc reload');
?>

================================= 

I then ran the script to create the zone file, and then created a symlink in /etc/cron.hourly to the sync script, so every hour the zones are fetched from the server, the configuration is re-built and bind is reloaded.

Then I modified /etc/bind/named.conf.options with the following two options, the first for security, the second to accept notify events from the master server to tell us to update our records.

options {
        ........ default options ........

    recursion no;
    allow-notify { 123.123.123.123; };
};

replace 123.123.123.123 with the master server's ip address. Then I just had to make a change to /etc/bind/name.conf.local to include the zone list built from the master server.

include "/etc/bind/cpanel.zones"

And thats it for the slave... now the master needs a minor modification to tell the client to update and to allow zone transfers to the slave. Add the following two lines to the options section in /etc/named.conf

allow-transfer { 123.123.123.123; };
also-notify { 123.123.123.123; };

Again, replacing the 123.123.123.123 with the IP address of the slave server.

Thursday, March 4, 2010

Reduce your server's bandwidth consumption with cPanel

For those that are interested, I manage a server running cPanel with over 200 accounts, on average we are using around 220GiB per month traffic server wide.

Turns out, that cPanel does not enable any form of HTTP compression for general requests, even though the WHM supports it, pretty dumb IMHO.

Anyway, here is how to turn it on... go to "Apache Configuration" and choose "Include Editor"
Select "All Versions" under "Pre Main Include"
Paste the following in:
AddType application/vnd.ms-word.document.macroEnabled.12 .docm
AddType application/vnd.openxmlformats-officedocument.wordprocessingml.document .docx
AddType application/vnd.ms-word.template.macroEnabled.12 .dotm
AddType application/vnd.openxmlformats-officedocument.wordprocessingml.template .dotx
AddType application/vnd.ms-powerpoint.slideshow.macroEnabled.12 .ppsm
AddType application/vnd.openxmlformats-officedocument.presentationml.slideshow .ppsx
AddType application/vnd.ms-powerpoint.presentation.macroEnabled.12 .pptm
AddType application/vnd.openxmlformats-officedocument.presentationml.presentation .pptx
AddType application/vnd.ms-excel.sheet.binary.macroEnabled.12 .xlsb
AddType application/vnd.ms-excel.sheet.macroEnabled.12 .xlsm
AddType application/vnd.openxmlformats-officedocument.spreadsheetml.sheet .xlsx
AddType application/vnd.ms-xpsdocument .xps

DeflateCompressionLevel 9
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javascript text/vbscript text/x-html text/iuls text/richtext text/scriptlet text/tab-separated-values text/webviewhtml text/x-component text/x-setext text/x-vcard application/x-javascript application/javascript image/bmp

<IfModule mod_expires.c>
ExpiresActive on
ExpiresDefault A600
ExpiresByType image/gif "access plus 1 day"
ExpiresByType image/jpeg "access plus 1 day"
ExpiresByType image/png "access plus 1 day"
ExpiresByType image/x-icon "access plus 1 day"
<FilesMatch "\.(php|php4)$">
ExpiresByType text/html "now"
</FilesMatch>
</IfModule>

This will also add mime type for the Microsoft formats that Apache does not know about by default, and set any image content to expire +1 day, so that they are not refreshed over and over.

This has reduced the servers consumption to around 80GiB per month, less then half.