Wsgi on cPanel improved
There is a long time now from my original post on running Django with python 2.6 on cPanel based servers, and as time passed there was some issues while deploying Django in such a way. So I decided to post another post with some updates regarding deployment of any python based scripts trough mod_wsgi and cPanel.
What's changed since last post?
There is a fix for easy_apache rebuilds, retaining the mod_wsgi and not braking rebuild process
There was some additions in virtualhost include files, enabling python app to run as a user, not nobody
A fix for cpanel's shell fork bomb protection while running python aps as a user and not nobody
Some minor changes regarding media handling, so it get's included in virtualhost includes and not by eating up user's subdomains
And of course python and Django version updates
Disclaimer at the beginning, this setup works for me on latest RELESE cPanel build, running on top of CentOS 5.5. I'm posting this as a way how I managed to get it working, so this shouldn't be considered as a official way of running Django or any other python app on cPanell servers. Although, as per my last post <a href="/blog/2010/django-on-cpanel-with-python2-6-virtualenv-and-mod_wsgi/">Django on cpanel with python2.6, virtualenv and mod_wsgi</a> lot's of people are reporting this is working for them also, and this version of deployment guide will bring some more enhancements.
Installing python and dependencies
mkdir /usr/src/python2.7 && cd /usr/src/python2.7
again, if you need sqlite, now is the time to install it.
wget http://www.sqlite.org/sqlite-autoconf-3070500.tar.gz
tar zxvf sqlite-autoconf-3070500.tar.gz
cd sqlite-autoconf-3070500
./configure
make
make install
Since CentOS 5.5 is still running python 2.4 and if you don't wish to brake your system you should install your python 2.7 in alternate location:
cd /usr/src/python2.7/ wget http://www.python.org/ftp/python/2.7.1/Python-2.7.1.tgz tar zxvf Python-2.7.1.tgz cd Python-2.7.1 ./configure --prefix=/opt/python2.7 --with-threads --enable-shared ./configure --help make make install
This will install python 2.7 in /opt/python2.7 directory. To make any use of this alternate install we must do the following:
ln -s /opt/python2.7/bin/python /usr/bin/python2.7 echo '/opt/python2.7/lib'>> /etc/ld.so.conf.d/opt-python2.7.conf ldconfig
It will make a symbolic link in your path and instruct the system where to find libs for this alternate python install.
Now is a good time to check if everything is working as it should
root@toy2 [/usr/src/python2.7]# /usr/bin/python2.7 Python 2.7.1 (r271:86832, Mar 26 2011, 22:31:33) [GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sqlite3 >>>
Type CTRL+D to exit.
Next thing to do is installation of python-setup tools:
/usr/bin/python2.7 cd /usr/src/python2.7/ wget http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg sh setuptools-0.6c11-py2.7.egg --prefix=/opt/python2.7
In this tutorial, I'm skipping the installation of mysql for python since you will have to install it per user basis in their virtual environments. All we need to do next is to install virtualenv to our python 2.7 so we can distribute them to our users
cd /opt/python2.7/bin/ ./easy_install virtualenv
mod_wsgi installation
cd /opt/python2.7/lib/python2.7/config/ ln -s ../../libpython2.7.so . cd /usr/src/python2.7/ wget http://modwsgi.googlecode.com/files/mod_wsgi-3.3.tar.gz tar zxvf mod_wsgi-3.3.tar.gz cd mod_wsgi-3.3 ./configure --with-python=/opt/python2.7/bin/python make make install
Now this will install mod_wsgi.so file in /usr/local/apache/modules folder, as I experienced during cPanel easy_apache rebuilds it will completely empty this folder, thus any virtualhost includes relying on mod_wsgi will fail and easy_apache will not be able to properly rebuild your httpd.conf file and whole apache or php upgrade you were running will fail and revert back to last good state.
To avoid this we will now copy the mod_wsgi to different folder:
mkdir /usr/local/apache/extramodules mv /usr/local/apache/modules/mod_wsgi.so /usr/local/apache/extramodules/
Note that this wsgi module is built with python 2.7 and will not work with any other version. If for some strange reason later on you try to run any given python application with system's default python2.4 it will not work.
Anyways, all we need to do next is to include mod_wsgi into the apache configuration:
nano /usr/local/apache/conf/includes/pre_virtualhost_global.conf
and paste:
LoadModule wsgi_module /usr/local/apache/extramodules/mod_wsgi.so AddHandler wsgi-script .wsgi
Now do configtest:
root@toy2 [~]# service httpd configtest Syntax OK
If you get Syntax OK at the end, you can safely restart your apache:
/scripts/restartsrv httpd
This is it, our apache can now serve python apps like Django, pylons, etc...
Setting up Django projects
Just like in my previous post I'll use '[username]' for cpanel username reference and '[domain]' for that user's domain. To match your needs you will have to replace those where appropriate.
We will quickly setup user's virtualenv, be aware though some of the python packages requires compile rights, and most of the cPanel setups I've seen so far are forbidding compiler access to their users. You can either make an permanent exception for this user, or you can instruct your user to contact you whenever he needs to install some package that requires compile rights (PIL, mysql...).
To enable compile rights for that specific user you can find that user in WHM WHM->Compiler Access->Allow specific users to use the compilers or if you don't like to exit the shell you can do it like this:
gpasswd -a [username] compiler
to remove it, use the same path in WHM and in shell:
gpasswd -d [username] compiler
For now, lets add this user to the compiler group since we will be installing some python packages for him in the start. Also you will have to enable your user a normal login shell, let's first check what shell this user has.
cat /etc/passwd |grep [username]
it should return something like this:
[username]:x:928:923::/home/[username]:/usr/local/cpanel/bin/jailshell
This user has jailshell enabled to change that into normal shell do this:
usermod -s /bin/bash [username]
If it's already bash, don't change it. You can also do this via WHM-> Manage Shell Access.
So let's finaly create a virtualenv for our user:
cd /home/[username] /opt/python2.7/bin/virtualenv --no-site-packages --distribute virtualenv chown -R [username].[username] virtualenv
this will create a new directory /home/[username]/virtualenv with our new virtual python environment for our user. From this point on we will do everything as the user and not root, until told otherwise.
su [username] source virtualenv/bin/activate
And the promt will change to something like this:
(virtualenv)[username]@[domain] [~]#
that way we know we are loged in as user and we are using his virtualenv. Now let's install Django in this users environment, simply enter:
easy_install django
Now is also a good time to install additional python site-packages like mysql and PIL.
easy_install pil easy_install mysql-python
So we have the django installed inside the users virtualenv, all we have to do now is start one of the instance. It's probably the best practice to keep django sites outside of public_html folders on cpanel servers. So we will do (still as a user)
cd mkdir djangosites cd djangosites django-admin.py startproject [projectname]
So by now we should have a directory structure like this:
/home/[username]/virtualenv < - python virtual environment /home/[username]/djangosites <- django sites folder /home/[username]/djangosites/[projectname] <- django project
Create the wsgi script for that project, usualy [projectname].wsgi in django sites folder
nano /home/[username]/djangosites/[projectname].wsgi
and paste the following code:
import sys import site import os vepath = '/home/[username]/virtualenv/lib/python2.7/site-packages' prev_sys_path = list(sys.path) site.addsitedir(vepath) sys.path.append('/home/[username]/djangosites') new_sys_path = [p for p in sys.path if p not in prev_sys_path] for item in new_sys_path: sys.path.remove(item) sys.path[:0] = new_sys_path from django.core.handlers.wsgi import WSGIHandler os.environ['DJANGO_SETTINGS_MODULE'] = '[projectname].settings' application = WSGIHandler()
give the scripts execute permissions:
chmod +x /home/[username]/djangosites/[projectname].wsgi
and exit user login with:
exit
now you will have your root propmt at the shell:
root@servername [/home/username]#
If you have decided so, now is the good time to disable the compiler access for this user.
create the apache include folder for that virtualhost:
mkdir -p /usr/local/apache/conf/userdata/std/2/[username]/[domain]/ nano /usr/local/apache/conf/userdata/std/2/[username]/[domain]/django.conf
and add:
<ifmodule mod_wsgi.c> WSGIScriptAlias / /home/[username]/djangosites/[projectname].wsgi WSGIDaemonProcess [projectname] user=[username] group=[username] processes=5 threads=15 display-name=%{GROUP} WSGIProcessGroup [projectname] WSGIApplicationGroup %{GLOBAL} </ifmodule>
Ok, so this will create a processgroup for this specific django projects with 5 processes each 15 threads with our user's uid and gid. This is great since we have isolated this user to his own application pool and his own home path, also we are enforcing user quotas since if we omit "user=[username] group=[username]" from WSGIDaemonProcess the deamon process will run ass apache user nobody, and all files made/uploaded via this application will not count into this user's quota. They will also make lot's of problems if the user don't grant global write permissions on some folders. Just like mod_php and php_suexec.
Shell fork bomb exception
On the other hand, running 5 processes each with 15 threads (you can customize this ofcourse), will hit the defaults ulimits enforced by cPanels shell fork bomb protection (if enabled). You could either disable the protection, or you can make a little adjustments to its logic so that you can make exceptions per user allowing them a bit more freedom but still not unlimited.
I have found this method somewhere on cPanel forums (I can't seem to find the exact post right now, but I will update this post when I find it), anyways with little patching of profile, limits and bashrc you can create a list of users that have a slightly higher limits, so our django users won't run into any memory or fork limits normally enforced by fork bomb protection.
I've created a little tarball with installer scripts for patched fork bomb logic, I advise you to check the files prior to installing them. Script will make a copies of your original files, to the same destination with .orig suffixes.
cd /usr/src/ wget https://toic.org/files/django/forkbomb.tar.gz tar zxvf forkbomb.tar.gz cd forkbomb ./install.sh
Now all you have to do is put one username per line in /etc/profile.exclude
Each user in that line will have slightly bigger ulimits
Back to the Django
Let's also make admin media paths and media folder paths
mkdir /home/[username]/djangosites/[projectname]/media nano /usr/local/apache/conf/userdata/std/2/[username]/[domain]/django-media.conf
and add.
Alias /admin_media/ /home/[username]/virtualenv/lib/python2.7/site-packages/Django-1.2.4-py2.7.egg/django/contrib/admin/media/ <Directory /home/[username]/virtualenv/lib/python2.7/site-packages/Django-1.2.4-py2.7.egg/django/contrib/admin/media> Order deny,allow Allow from all </Directory> Alias /media/ /home/[username]/djangosites/[projectname]/media/ <Directory /home/[username]/djangosites/[projectname]/media> Order deny,allow Allow from all </Directory>
Just replace your python version, django version and username and projectname in paths. This will create [domain]/media - for site media and [domain]/admin_media/ for admin media
Now verify those apache include files and rebuild apache conf:
/scripts/verify_vhost_includes /scripts/rebuildhttpdconf
As always you can make subdomains for your media files, you will just need to update your [projectname]/settings.py
And that's it, hope you enjoy your django installation