toic.org - Latest entrieshttps://toic.org/blog/The latest entries on the site toic.orgen-usZinniaFri, 08 Jan 2016 14:19:45 +0000IPv4 country code IP ranges database https://toic.org/blog/2013/ipv4-country-code-ip-ranges-database/<p>For a while now, I was using some third party IP ranges file lists for some restrictions lists based on country codes.</p> <p>Long story short those third party sites either died or are not properly detecting IP ranges from certain country codes of my interest. I'm trying to put up my own list. In the process my IP ranges zone files will be available for public use on <a href="http://toic.org/network/cc_zones/">this site</a>. I'll try to announce any changes to the background logic.</p> <p>For now, the script in the background will be running every 6h rebuilding those files with new info. Current source is <a class="reference external" href="ftp://ftp.ripe.net/ripe/stats/RIR-Statistics-Exchange-Format.txt">RIR-Statistics-Exchange-Format</a> gathered from <a class="reference external" href="http://www.ripe.net/lir-services/member-support/info/list-of-members/list-of-country-codes-and-rirs">RIR members</a></p> branko@toic.org (branko)Sun, 23 Jun 2013 00:24:58 +0000https://toic.org/blog/2013/ipv4-country-code-ip-ranges-database/CodingApache monitoring tool ApTop beta released https://toic.org/blog/2012/apache-monitoring-tool-aptop-beta-released/<p>This is my first public publish to PyPi, without further delays, here is the full project description</p> <p><h1>ApTop</h1></p> <p>ApTop ia a top(1) alike monitoring utility for Apache.</p> <p>Being daily involved in monitoring system load in shared environment I find it lacking proper tool for monitoring Apache behavior in real time. Granted there are some tools for this job, they are usually of limited options for some monitoring needs.</p> <p>ApTop was written as a hobby project in python for debugging purposes while investigating several system load situations caused by web applications in shared hosting environment.</p> <p>ApTop is still an early beta release with some known bugs, nevertheless it still provides a great starting point in monitoring Apache. Every feedback and feature request at this point is more than welcomed.</p> <div class="section" id="description"> <h1>Description</h1> <p>ApTop will show near real time Apache active working proceses and virtual host connections. By default sleeping / waiting for connections slots will be filtered out.</p> <p>ApTop is also displaying several screens</p> <div class="section" id="dashboard-or-home-screen"> <h2>Dashboard or Home screen</h2> <p>This screen displays some relevant informations gathered from <a class="reference external" href="http://httpd.apache.org/docs/2.2/mod/mod_status.html">apache-status</a> page. It is by default sorted by shortest last apache access time &quot;SS&quot; and it displays only active connections. Displayed fields are the same as in server-status page, so you can find detailed explanation for their meaning at the bottom of server-status page.</p> <p>Displayed fields are:</p> <p>PID - OS process ID M - Mode of operation CPU - CPU usage, number of seconds SS - Seconds since beginning of most recent request&lt; Req - Milliseconds required to process most recent request Conn - Kilobytes transferred this connection Acc - Number of accesses this connection / this child / this slot Client - Remote client VHost - Local virtualhost Request - Request method and partial URL</p> </div> <div class="section" id="count-by-vhosts-screen"> <h2>Count by vhosts screen</h2> <p>This screen displays virtualhosts in order by highest count of active connections along with connection count.</p> <p>Very useful for detecting high traffic sites.</p> </div> <div class="section" id="count-by-clients-screen"> <h2>Count by clients screen</h2> <p>This screen displays clients in order by highest count of active connections along with connection count.</p> <p>Very useful for detecting resource exhausters.</p> </div> </div> <div class="section" id="requirements"> <h1>Requirements</h1> <p>ApTop should work just fine on any unix based system with python standard packages and lxml.</p> <p>Make sure you have libxml2 libxslt libxslt-devel libxml2-devel packages instaleld prior to building lxml:</p> <p>For RHEL based distro you can install those with</p> <pre class="literal-block"> yum install libxml2 libxslt libxslt-devel libxml2-devel </pre> <p>After that you can install lxml python package with <strong>easy_install lxml</strong> or <strong>pip install lxml</strong></p> <p>ApTop is using Apaches <strong>mod_status</strong> with <strong>ExtendedStatus On</strong> for gathering data. Please ensure that <strong>mod_status</strong> is enabled and proper status url is defined in <strong>aptop.conf</strong> There are some concernes regarding enabled ExtendedStatus On, but I think <a class="reference external" href="http://www.philchen.com/2008/06/02/apache-20-mod_status-effects-on-performance-server-resources">this links</a> covers this nicely.</p> </div> <div class="section" id="instalation"> <h1>Instalation</h1> <p>These installation options are for RHEL based distros, adjust accordingly.</p> <blockquote> yum install libxml2 libxslt libxslt-devel libxml2-devel python-setuptools</blockquote> <p>Now you can do</p> <blockquote> easy_install ApTop</blockquote> <p>or</p> <blockquote> easy_install <a class="reference external" href="https://bitbucket.org/btoic/aptop/get/master.tar.gz">https://bitbucket.org/btoic/aptop/get/master.tar.gz</a></blockquote> <p>You can now run aptop.py</p> </div> <div class="section" id="configuration"> <h1>Configuration</h1> <p>ApTop has some built in default configuration directives, but it will also look first in ~/.aptop.conf and than /etc/aptop.conf for overrides.</p> <p>ApTop uses .ini style configuration files and main cofniguration options should be placed after <strong>[aptop]</strong> section.</p> <p>By default ApTop will use <a class="reference external" href="http://localhost/server-status">http://localhost/server-status</a> url for grabbing data to display, and with default refresh rate of 5 seconds.</p> <div class="section" id="available-configuration-options"> <h2>Available configuration options:</h2> <blockquote> status_url = URL</blockquote> <p>This configuration option will require you to enter valid mod_status status url for server you wish to monitor. This defaults to <a class="reference external" href="http://localhost/server-status">http://localhost/server-status</a> Note that Apache should have ExtendedStatus conf option set to On</p> <blockquote> refresh = seconds</blockquote> <p>This option will control ApTop default refresh interval. It defaults to 5 seconds and can't be lower than 1 second.</p> <p>You can even monitor the remote systems if they granted you access rights to theirs server-status URL. In the future ApTop will provide options to save remote monitored servers to its configuration.</p> </div> </div> <div class="section" id="runtime-options"> <h1>Runtime options</h1> <p>While running ApTop responds to following options:</p> <p><strong>H</strong> - display main screen</p> <p><strong>V</strong> - display vhosts by conn. count</p> <p><strong>c</strong> - display clients by conn. count</p> <p><strong>R</strong> - reverse the current sort order</p> <p><strong>I</strong> - toggle active/inactive slots</p> </div> <div class="section" id="aditional-links"> <h1>Aditional links</h1> <p><a class="reference external" href="http://toic.org">http://toic.org</a></p> <p><a class="reference external" href="https://bitbucket.org/btoic/aptop">https://bitbucket.org/btoic/aptop</a></p> </div> branko@toic.org (branko)Wed, 12 Dec 2012 23:34:19 +0000https://toic.org/blog/2012/apache-monitoring-tool-aptop-beta-released/CodingMonitoringReclaiming InnoDB ibdata unused space. https://toic.org/blog/2012/reducing-innodb-ibdata-unused-space/<p><img align='left' src="/media/filer_public/9f/70/9f70cbfc-1a4f-4885-97e1-66196ea4e805/crystal_128_kcmpartitions.png" alt="ssh" width="120" height="120" style="margin-right:20px; margin-bottom:20px;" /></p> <p>As you well know InnoDB will store its data either in one big system tablespace called ibdata or you can store it in multiple tablespaces (<a href="http://dev.mysql.com/doc/refman/5.5/en/innodb-parameters.html#sysvar_innodb_file_per_table">innodb_file_per_table</a>). In either case InnoDB by default will not reclaim unused space gained when deleting data. Using innodb_file_per_table will separate each InnoDB table in its own .ibd file and it can be easily reclaimed by doing optimize table, using system tablespace will get you stuck since there is no real supported method for reducing you ibdata file and reclaiming unused disk space.</p> <p>One could wonder why would anyone want to have its ibdata in system tablespace? Well there are some &quot;<a class="reference external" href="http://bugs.mysql.com/bug.php?id=51325">bugs</a>&quot; (in versions prior to 5.5) and performance issues that cause mysql lock-ups when using innodb_file_per_table and doing truncate table or drop table or even drop database queries with large buffer pools. In a nutshell MySQL will do a whole buffer pool lock and scan on each of the above mentioned commands effectively stoping the entire MySQL server until it scans over the entire buffer pool for references to those dropped/truncated tables.</p> <p>When given choice and when I do know the pattern of data input to my tablespace i will chose system tablespace.</p> <p>Recently I was a victim of my own system and uncontrolled input of data to my InnoDB table space which leaved me with almost 36Gb of garbage data over one night. Naturally i truncated the data, but it left me with 37Gb ibdata file.</p> <p>So here are the steps I've taken to reduce this system tablespace.</p> <p><strong>First of all backup!</strong></p> <p>I highly recommend <a class="reference external" href="http://www.percona.com/software/percona-xtrabackup">Xtrabackup</a> from Percona for this task, yes it will require additional 37Gb of space for backup but this is by far best point in time backup for MySQL.</p> <pre class="literal-block"> innobackupex $backupdir </pre> <p>And replace the backupdir with your preferred location</p> <p>Let's create a working environment, say you have a lots of space on /home</p> <pre class="literal-block"> mkdir /home/inno-reduce cd /home/inno-reduce </pre> <p>Ok, next stop dumping all the databases that have innodb tables inside, for this I've created a small bash script a while ago..</p> <pre class="code bash literal-block"> <span class="comment">#!/bin/bash </span><span class="name variable">CWD</span><span class="operator">=</span><span class="literal string backtick">`</span><span class="name builtin">pwd</span><span class="literal string backtick">`</span> mysql -e <span class="literal string double">&quot;SELECT distinct(table_schema) FROM information_schema.TABLES where ENGINE = 'InnoDB'&quot;</span> &gt; <span class="name variable">$CWD</span>/innodb-databases.list sed <span class="literal string double">&quot;/table_schema/d&quot;</span> <span class="name variable">$CWD</span>/innodb-databases.list &gt; <span class="name variable">$CWD</span>/tmp mv <span class="name variable">$CWD</span>/tmp <span class="name variable">$CWD</span>/innodb-databases.list <span class="keyword">for</span> database in <span class="literal string backtick">`</span>cat <span class="name variable">$CWD</span>/innodb-databases.list<span class="literal string backtick">`</span><span class="punctuation">;</span> <span class="keyword">do</span> <span class="name builtin">echo</span> <span class="literal string double">&quot;dumping data from </span><span class="name variable">$database</span><span class="literal string double"> to </span><span class="name variable">$CWD</span><span class="literal string double">/</span><span class="name variable">$database</span><span class="literal string double">.sql&quot;</span> /usr/bin/mysqldump --triggers --routines <span class="name variable">$database</span> &gt; <span class="name variable">$CWD</span>/<span class="name variable">$database</span>.sql<span class="punctuation">;</span> <span class="keyword">done</span> </pre> <p>Now would be a good time to cut off any application using MySQL (for example turn off Apache, or drop the packets in firewall). Just copy and paste this code in <strong>step1-dump.sh</strong> file, make it executable and run it. It will produce a .sql dump files in your current directory (hopefully we are still on /home/inno-reduce)</p> <p>Step2 script will iterate trough that dumped data files and drop all the databases that have innodb tables inside.</p> <pre class="code bash literal-block"> <span class="comment">#!/bin/bash </span><span class="name variable">CWD</span><span class="operator">=</span><span class="literal string backtick">`</span><span class="name builtin">pwd</span><span class="literal string backtick">`</span> <span class="keyword">for</span> database in <span class="literal string backtick">`</span>ls <span class="name variable">$CWD</span>/*.sql<span class="punctuation">|</span>awk -F<span class="literal string single">'/'</span> <span class="literal string single">'{print $NF}'</span><span class="punctuation">|</span>cut -d. -f1<span class="literal string backtick">`</span><span class="punctuation">;</span> <span class="keyword">do</span> <span class="name builtin">echo</span> <span class="literal string double">&quot;droping database </span><span class="name variable">$database</span><span class="literal string double">&quot;</span> mysqladmin drop <span class="name variable">$database</span> <span class="keyword">done</span> </pre> <p>Again just copy and paste this code in <strong>step2-drop.sh</strong>, make it executable and run it. At this point There shouldn't be any InnoDB tables in MySQL server.</p> <p>Now, it is a good time to shutdown your MySQL service and rm ibdata file. On typical CentOS system this would be done with</p> <pre class="literal-block"> service mysqld stop rm -f /var/lib/mysql/ibdata1 </pre> <p>If you are not running CentOS and default MySQL install or if you have ibdata file on some other path… please adjust the above. Starting MySQL service at this point would &quot;regenerate&quot; ibdata file with its smallest size increment as defined in my.cnf lets do so:</p> <pre class="literal-block"> service mysqld start </pre> <p>And now for the third script</p> <pre class="code bash literal-block"> <span class="comment">#!/bin/bash </span><span class="name variable">CWD</span><span class="operator">=</span><span class="literal string backtick">`</span><span class="name builtin">pwd</span><span class="literal string backtick">`</span> <span class="keyword">for</span> database in <span class="literal string backtick">`</span>ls <span class="name variable">$CWD</span>/*.sql<span class="punctuation">|</span>awk -F<span class="literal string single">'/'</span> <span class="literal string single">'{print $NF}'</span><span class="punctuation">|</span>cut -d. -f1<span class="literal string backtick">`</span><span class="punctuation">;</span> <span class="keyword">do</span> mysqladmin create <span class="name variable">$database</span> mysql <span class="name variable">$database</span> &lt; <span class="name variable">$CWD</span>/<span class="name variable">$database</span>.sql <span class="keyword">done</span> </pre> <p>As before copy and paste this script into <strong>step3-import.sh</strong> (remember to place this file at the same directory as other two, and dumps), make it executable and run it.</p> <p>Voila! your data is back in, ibdata reduced and MySQL up&amp;running.</p> <p>Remember to restore any access to your MySQL service (starting Apache, clearing firewall drops, etc..)</p> branko@toic.org (branko)Tue, 04 Dec 2012 23:05:56 +0000https://toic.org/blog/2012/reducing-innodb-ibdata-unused-space/ServicesWsgi on cPanel improved https://toic.org/blog/2011/wsgi-on-cpanel-improved/<p>There is a long time now from my original post on running Django with python 2.6 on cPanel based servers, and as time passed there was some issues while deploying Django in such a way. So I decided to post another post with some updates regarding deployment of any python based scripts trough mod_wsgi and cPanel.</p> <p><strong>What's changed since last post?</strong></p> <ul> <li><p class="first">There is a fix for easy_apache rebuilds, retaining the mod_wsgi and not braking rebuild process</p> </li> <li><p class="first">There was some additions in virtualhost include files, enabling python app to run as a user, not nobody</p> </li> <li><p class="first">A fix for cpanel's shell fork bomb protection while running python aps as a user and not nobody</p> </li> <li><p class="first">Some minor changes regarding media handling, so it get's included in virtualhost includes and not by eating up user's subdomains</p> </li> <li><p class="first">And of course python and Django version updates</p> <blockquote> <p>Disclaimer at the beginning, this setup works for me on latest RELESE cPanel build, running on top of CentOS 5.5. I'm posting this as a way how I managed to get it working, so this shouldn't be considered as a official way of running Django or any other python app on cPanell servers. Although, as per my last post &lt;a href=&quot;/blog/2010/django-on-cpanel-with-python2-6-virtualenv-and-mod_wsgi/&quot;&gt;Django on cpanel with python2.6, virtualenv and mod_wsgi&lt;/a&gt; lot's of people are reporting this is working for them also, and this version of deployment guide will bring some more enhancements.</p> </blockquote> </li> </ul> <div class="section" id="installing-python-and-dependencies"> <h1>Installing python and dependencies</h1> <pre class="literal-block"> mkdir /usr/src/python2.7 &amp;&amp; cd /usr/src/python2.7 </pre> <p>again, if you need sqlite, now is the time to install it.</p> <pre class="code bash literal-block"> wget http://www.sqlite.org/sqlite-autoconf-3070500.tar.gz tar zxvf sqlite-autoconf-3070500.tar.gz <span class="name builtin">cd </span>sqlite-autoconf-3070500 ./configure make make install </pre> <p>Since CentOS 5.5 is still running python 2.4 and if you don't wish to brake your system you should install your python 2.7 in alternate location:</p> <pre class="code bash literal-block"> <span class="name builtin">cd</span> /usr/src/python2.7/ wget http://www.python.org/ftp/python/2.7.1/Python-2.7.1.tgz tar zxvf Python-2.7.1.tgz <span class="name builtin">cd </span>Python-2.7.1 ./configure --prefix<span class="operator">=</span>/opt/python2.7 --with-threads --enable-shared ./configure --help make make install </pre> <p>This will install python 2.7 in <strong>/opt/python2.7</strong> directory. To make any use of this alternate install we must do the following:</p> <pre class="code bash literal-block"> ln -s /opt/python2.7/bin/python /usr/bin/python2.7 <span class="name builtin">echo</span> <span class="literal string single">'/opt/python2.7/lib'</span>&gt;&gt; /etc/ld.so.conf.d/opt-python2.7.conf ldconfig </pre> <p>It will make a symbolic link in your path and instruct the system where to find libs for this alternate python install.</p> <p>Now is a good time to check if everything is working as it should</p> <pre class="code python literal-block"> <span class="name">root</span><span class="name decorator">&#64;toy2</span> <span class="punctuation">[</span><span class="operator">/</span><span class="name">usr</span><span class="operator">/</span><span class="name">src</span><span class="operator">/</span><span class="name">python2</span><span class="operator">.</span><span class="literal number integer">7</span><span class="punctuation">]</span><span class="comment"># /usr/bin/python2.7</span> <span class="name">Python</span> <span class="literal number float">2.7</span><span class="operator">.</span><span class="literal number integer">1</span> <span class="punctuation">(</span><span class="name">r271</span><span class="punctuation">:</span><span class="literal number integer">86832</span><span class="punctuation">,</span> <span class="name">Mar</span> <span class="literal number integer">26</span> <span class="literal number integer">2011</span><span class="punctuation">,</span> <span class="literal number integer">22</span><span class="punctuation">:</span><span class="literal number integer">31</span><span class="punctuation">:</span><span class="literal number integer">33</span><span class="punctuation">)</span> <span class="punctuation">[</span><span class="name">GCC</span> <span class="literal number float">4.1</span><span class="operator">.</span><span class="literal number integer">2</span> <span class="literal number integer">20080704</span> <span class="punctuation">(</span><span class="name">Red</span> <span class="name">Hat</span> <span class="literal number float">4.1</span><span class="operator">.</span><span class="literal number integer">2</span><span class="operator">-</span><span class="literal number integer">48</span><span class="punctuation">)]</span> <span class="name">on</span> <span class="name">linux2</span> <span class="name">Type</span> <span class="literal string">&quot;help&quot;</span><span class="punctuation">,</span> <span class="literal string">&quot;copyright&quot;</span><span class="punctuation">,</span> <span class="literal string">&quot;credits&quot;</span> <span class="operator word">or</span> <span class="literal string">&quot;license&quot;</span> <span class="keyword">for</span> <span class="name">more</span> <span class="name">information</span><span class="operator">.</span> <span class="operator">&gt;&gt;&gt;</span> <span class="keyword namespace">import</span> <span class="name namespace">sqlite3</span> <span class="operator">&gt;&gt;&gt;</span> </pre> <p><strong>Type CTRL+D to exit.</strong></p> <p>Next thing to do is installation of python-setup tools:</p> <pre class="code bash literal-block"> /usr/bin/python2.7 <span class="name builtin">cd</span> /usr/src/python2.7/ wget http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg sh setuptools-0.6c11-py2.7.egg --prefix<span class="operator">=</span>/opt/python2.7 </pre> <p>In this tutorial, I'm skipping the installation of mysql for python since you will have to install it per user basis in their virtual environments. All we need to do next is to install virtualenv to our python 2.7 so we can distribute them to our users</p> <pre class="literal-block"> cd /opt/python2.7/bin/ ./easy_install virtualenv </pre> </div> <div class="section" id="mod-wsgi-installation"> <h1>mod_wsgi installation</h1> <pre class="code bash literal-block"> <span class="name builtin">cd</span> /opt/python2.7/lib/python2.7/config/ ln -s ../../libpython2.7.so . <span class="name builtin">cd</span> /usr/src/python2.7/ wget http://modwsgi.googlecode.com/files/mod_wsgi-3.3.tar.gz tar zxvf mod_wsgi-3.3.tar.gz <span class="name builtin">cd </span>mod_wsgi-3.3 ./configure --with-python<span class="operator">=</span>/opt/python2.7/bin/python make make install </pre> <p>Now this will install mod_wsgi.so file in /usr/local/apache/modules folder, as I experienced during cPanel easy_apache rebuilds it will completely empty this folder, thus any virtualhost includes relying on mod_wsgi will fail and easy_apache will not be able to properly rebuild your httpd.conf file and whole apache or php upgrade you were running will fail and revert back to last good state.</p> <p>To avoid this we will now copy the mod_wsgi to different folder:</p> <pre class="literal-block"> mkdir /usr/local/apache/extramodules mv /usr/local/apache/modules/mod_wsgi.so /usr/local/apache/extramodules/ </pre> <p><strong>Note that this wsgi module is built with python 2.7 and will not work with any other version.</strong> If for some strange reason later on you try to run any given python application with system's default python2.4 it will not work.</p> <p>Anyways, all we need to do next is to include mod_wsgi into the apache configuration:</p> <pre class="literal-block"> nano /usr/local/apache/conf/includes/pre_virtualhost_global.conf </pre> <p>and paste:</p> <pre class="literal-block"> LoadModule wsgi_module /usr/local/apache/extramodules/mod_wsgi.so AddHandler wsgi-script .wsgi </pre> <p>Now do configtest:</p> <pre class="literal-block"> root&#64;toy2 [~]# service httpd configtest Syntax OK </pre> <p>If you get <strong>Syntax OK</strong> at the end, you can safely restart your apache:</p> <pre class="literal-block"> /scripts/restartsrv httpd </pre> <p>This is it, our apache can now serve python apps like Django, pylons, etc...</p> </div> <div class="section" id="setting-up-django-projects"> <h1>Setting up Django projects</h1> <p>Just like in my <a class="reference external" href="/blog/2010/django-on-cpanel-with-python2-6-virtualenv-and-mod_wsgi/">previous post</a> I'll use '<strong>[username]</strong>' for cpanel username reference and '<strong>[domain]</strong>' for that user's domain. To match your needs you will have to replace those where appropriate.</p> <p>We will quickly setup user's virtualenv, be aware though some of the python packages requires compile rights, and most of the cPanel setups I've seen so far are forbidding compiler access to their users. You can either make an permanent exception for this user, or you can instruct your user to contact you whenever he needs to install some package that requires compile rights (PIL, mysql...).</p> <p>To enable compile rights for that specific user you can find that user in WHM <strong>WHM-&gt;Compiler Access-&gt;Allow specific users to use the compilers</strong> or if you don't like to exit the shell you can do it like this:</p> <pre class="literal-block"> gpasswd -a [username] compiler </pre> <p>to remove it, use the same path in WHM and in shell:</p> <pre class="literal-block"> gpasswd -d [username] compiler </pre> <p>For now, lets add this user to the compiler group since we will be installing some python packages for him in the start. Also you will have to enable your user a normal login shell, let's first check what shell this user has.</p> <pre class="literal-block"> cat /etc/passwd |grep [username] </pre> <p>it should return something like this:</p> <pre class="literal-block"> [username]:x:928:923::/home/[username]:/usr/local/cpanel/bin/jailshell </pre> <p>This user has jailshell enabled to change that into normal shell do this:</p> <pre class="literal-block"> usermod -s /bin/bash [username] </pre> <p>If it's already bash, don't change it. You can also do this via <strong>WHM-&gt; Manage Shell Access.</strong></p> <p>So let's finaly create a virtualenv for our user:</p> <pre class="code bash literal-block"> <span class="name builtin">cd</span> /home/<span class="operator">[</span>username<span class="operator">]</span> /opt/python2.7/bin/virtualenv --no-site-packages --distribute virtualenv chown -R <span class="operator">[</span>username<span class="operator">]</span>.<span class="operator">[</span>username<span class="operator">]</span> virtualenv </pre> <p>this will create a new directory <strong>/home/[username]/virtualenv</strong> with our new virtual python environment for our user. From this point on we will do everything as the user and not root, until told otherwise.</p> <pre class="literal-block"> su [username] source virtualenv/bin/activate </pre> <p>And the promt will change to something like this:</p> <pre class="literal-block"> (virtualenv)[username]&#64;[domain] [~]# </pre> <p>that way we know we are loged in as user and we are using his virtualenv. Now let's install Django in this users environment, simply enter:</p> <pre class="literal-block"> easy_install django </pre> <p>Now is also a good time to install additional python site-packages like mysql and PIL.</p> <pre class="literal-block"> easy_install pil easy_install mysql-python </pre> <p>So we have the django installed inside the users virtualenv, all we have to do now is start one of the instance. It's probably the best practice to keep django sites outside of public_html folders on cpanel servers. So we will do (<strong>still as a user</strong>)</p> <pre class="code bash literal-block"> <span class="name builtin">cd </span>mkdir djangosites <span class="name builtin">cd </span>djangosites django-admin.py startproject <span class="operator">[</span>projectname<span class="operator">]</span> </pre> <p>So by now we should have a directory structure like this:</p> <pre class="literal-block"> /home/[username]/virtualenv &lt; - python virtual environment /home/[username]/djangosites &lt;- django sites folder /home/[username]/djangosites/[projectname] &lt;- django project </pre> <p>Create the wsgi script for that project, usualy <strong>[projectname].wsgi</strong> in django sites folder</p> <pre class="literal-block"> nano /home/[username]/djangosites/[projectname].wsgi </pre> <p>and paste the following code:</p> <pre class="code python literal-block"> <span class="keyword namespace">import</span> <span class="name namespace">sys</span> <span class="keyword namespace">import</span> <span class="name namespace">site</span> <span class="keyword namespace">import</span> <span class="name namespace">os</span> <span class="name">vepath</span> <span class="operator">=</span> <span class="literal string">'/home/[username]/virtualenv/lib/python2.7/site-packages'</span> <span class="name">prev_sys_path</span> <span class="operator">=</span> <span class="name builtin">list</span><span class="punctuation">(</span><span class="name">sys</span><span class="operator">.</span><span class="name">path</span><span class="punctuation">)</span> <span class="name">site</span><span class="operator">.</span><span class="name">addsitedir</span><span class="punctuation">(</span><span class="name">vepath</span><span class="punctuation">)</span> <span class="name">sys</span><span class="operator">.</span><span class="name">path</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">(</span><span class="literal string">'/home/[username]/djangosites'</span><span class="punctuation">)</span> <span class="name">new_sys_path</span> <span class="operator">=</span> <span class="punctuation">[</span><span class="name">p</span> <span class="keyword">for</span> <span class="name">p</span> <span class="operator word">in</span> <span class="name">sys</span><span class="operator">.</span><span class="name">path</span> <span class="keyword">if</span> <span class="name">p</span> <span class="operator word">not</span> <span class="operator word">in</span> <span class="name">prev_sys_path</span><span class="punctuation">]</span> <span class="keyword">for</span> <span class="name">item</span> <span class="operator word">in</span> <span class="name">new_sys_path</span><span class="punctuation">:</span> <span class="name">sys</span><span class="operator">.</span><span class="name">path</span><span class="operator">.</span><span class="name">remove</span><span class="punctuation">(</span><span class="name">item</span><span class="punctuation">)</span> <span class="name">sys</span><span class="operator">.</span><span class="name">path</span><span class="punctuation">[:</span><span class="literal number integer">0</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="name">new_sys_path</span> <span class="keyword namespace">from</span> <span class="name namespace">django.core.handlers.wsgi</span> <span class="keyword namespace">import</span> <span class="name">WSGIHandler</span> <span class="name">os</span><span class="operator">.</span><span class="name">environ</span><span class="punctuation">[</span><span class="literal string">'DJANGO_SETTINGS_MODULE'</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal string">'[projectname].settings'</span> <span class="name">application</span> <span class="operator">=</span> <span class="name">WSGIHandler</span><span class="punctuation">()</span> </pre> <p>give the scripts execute permissions:</p> <pre class="literal-block"> chmod +x /home/[username]/djangosites/[projectname].wsgi </pre> <p>and exit user login with:</p> <pre class="literal-block"> exit </pre> <p>now you will have your <strong>root propmt</strong> at the shell:</p> <pre class="literal-block"> root&#64;servername [/home/username]# </pre> <p><strong>If you have decided so, now is the good time to disable the compiler access for this user.</strong></p> <p>create the apache include folder for that virtualhost:</p> <pre class="literal-block"> mkdir -p /usr/local/apache/conf/userdata/std/2/[username]/[domain]/ nano /usr/local/apache/conf/userdata/std/2/[username]/[domain]/django.conf </pre> <p>and add:</p> <pre class="code apacheconf literal-block"> <span class="name tag">&lt;ifmodule</span> <span class="literal string">mod_wsgi.c</span><span class="name tag">&gt;</span> <span class="name builtin">WSGIScriptAlias</span> / <span class="literal string other">/home/</span>[username]/djangosites/[projectname].wsgi <span class="name builtin">WSGIDaemonProcess</span> [projectname] <span class="keyword">user</span>=[username] <span class="keyword">group</span>=[username] processes=5 threads=15 display-name=%{GROUP} <span class="name builtin">WSGIProcessGroup</span> [projectname] <span class="name builtin">WSGIApplicationGroup</span> %{GLOBAL} <span class="name tag">&lt;/ifmodule&gt;</span> </pre> <p>Ok, so this will create a processgroup for this specific django projects with 5 processes each 15 threads with our user's uid and gid. This is great since we have isolated this user to his own application pool and his own home path, also we are enforcing user quotas since if we omit &quot;user=[username] group=[username]&quot; from WSGIDaemonProcess the deamon process will run ass apache user nobody, and all files made/uploaded via this application will not count into this user's quota. They will also make lot's of problems if the user don't grant global write permissions on some folders. Just like mod_php and php_suexec.</p> </div> <div class="section" id="shell-fork-bomb-exception"> <h1>Shell fork bomb exception</h1> <p>On the other hand, running 5 processes each with 15 threads (you can customize this ofcourse), will hit the defaults ulimits enforced by cPanels shell fork bomb protection (if enabled). You could either disable the protection, or you can make a little adjustments to its logic so that you can make exceptions per user allowing them a bit more freedom but still not unlimited.</p> <p>I have found this method somewhere on cPanel forums (I can't seem to find the exact post right now, but I will update this post when I find it), anyways with little patching of profile, limits and bashrc you can create a list of users that have a slightly higher limits, so our django users won't run into any memory or fork limits normally enforced by <strong>fork bomb protection</strong>.</p> <p>I've created a little tarball with installer scripts for patched fork bomb logic, I advise you to check the files prior to installing them. Script will make a copies of your original files, to the same destination with .orig suffixes.</p> <pre class="code bash literal-block"> <span class="name builtin">cd</span> /usr/src/ wget https://toic.org/files/django/forkbomb.tar.gz tar zxvf forkbomb.tar.gz <span class="name builtin">cd </span>forkbomb ./install.sh </pre> <p>Now all you have to do is put one username per line in <strong>/etc/profile.exclude</strong></p> <p>Each user in that line will have slightly bigger ulimits</p> </div> <div class="section" id="back-to-the-django"> <h1>Back to the Django</h1> <p>Let's also make admin media paths and media folder paths</p> <div class="system-message"> <p class="system-message-title">System Message: WARNING/2 (<tt class="docutils">&lt;string&gt;</tt>, line 323)</p> Literal block expected; none found.</div> <p>mkdir /home/[username]/djangosites/[projectname]/media nano /usr/local/apache/conf/userdata/std/2/[username]/[domain]/django-media.conf</p> <p>and add.</p> <pre class="code apacheconf literal-block"> <span class="name builtin">Alias</span> <span class="literal string other">/admin_media/</span> <span class="literal string other">/home/</span>[username]/virtualenv/lib/python2.7/site-packages/Django-1.2.4-py2.7.egg/django/contrib/admin/media/ <span class="name tag">&lt;Directory</span> <span class="literal string">/home/[username]/virtualenv/lib/python2.7/site-packages/Django-1.2.4-py2.7.egg/django/contrib/admin/media</span><span class="name tag">&gt;</span> <span class="name builtin">Order</span> deny,allow <span class="name builtin">Allow</span> from <span class="keyword">all</span> <span class="name tag">&lt;/Directory&gt;</span> <span class="name builtin">Alias</span> <span class="literal string other">/media/</span> <span class="literal string other">/home/</span>[username]/djangosites/[projectname]/media/ <span class="name tag">&lt;Directory</span> <span class="literal string">/home/[username]/djangosites/[projectname]/media</span><span class="name tag">&gt;</span> <span class="name builtin">Order</span> deny,allow <span class="name builtin">Allow</span> from <span class="keyword">all</span> <span class="name tag">&lt;/Directory&gt;</span> </pre> <p>Just replace your python version, django version and username and projectname in paths. This will create [domain]/media - for site media and [domain]/admin_media/ for admin media</p> <p>Now verify those apache include files and rebuild apache conf:</p> <pre class="literal-block"> /scripts/verify_vhost_includes /scripts/rebuildhttpdconf </pre> <p>As always you can make subdomains for your media files, you will just need to update your [projectname]/settings.py</p> <p>And that's it, hope you enjoy your django installation</p> </div> branko@toic.org (branko)Sun, 27 Mar 2011 20:01:02 +0000https://toic.org/blog/2011/wsgi-on-cpanel-improved/CodingLinuxTutorialsDjango on cpanel with python2.6, virtualenv and mod_wsgi https://toic.org/blog/2010/django-on-cpanel-with-python2-6-virtualenv-and-mod_wsgi/<p><img align='left' src="/media/filer_public/47/a4/47a4abec-d7e5-4beb-b61e-cb2688d0dc02/umltopython.png" alt="ssh" style="padding-top: 20px; margin-right:20px; margin-bottom: 20px;" /><br />Here is the simple to follow step by step tutorial on howto run latest Django on cpanel powered servers with python 2.6, virtual env and mod_wsgi.<br />If you are running cPanel on your server it's most probably a rhel or centos distro, and those will have by default python2.4 installed. You can't just overwrite your python2.4 as many of the system utilities are depended on the python2.4 So what we will do is install an alternate python 2.6 and setup our server to run Django instances with python2.6</p> <p>First of all let's make a folder where we will download all those source packages</p> <div class="section" id="installing-python-and-dependencies"> <h1>Installing python and dependencies</h1> <pre class="literal-block"> mkdir -p /usr/src/python26 &amp;&amp; cd /usr/src/python26 </pre> <p>Next, we have to download and install sqlite3 dependency. It's not needed if your Django instances will not run sqlite databases, but on a safe side it's better to install it than not.</p> <pre class="code bash literal-block"> wget http://www.sqlite.org/sqlite-amalgamation-3.6.4.tar.gz tar zxvf sqlite-amalgamation-3.6.4.tar.gz <span class="name builtin">cd </span>sqlite-3.6.4 ./configure make make install </pre> <p>If all went well it's time to install python 2.6 as alternate install.</p> <pre class="code bash literal-block"> <span class="name builtin">cd</span> /usr/src/python26 wget http://www.python.org/ftp/python/2.6.5/Python-2.6.5.tgz tar zxvf Python-2.6.5.tgz <span class="name builtin">cd </span>Python-2.6.5 ./configure --prefix<span class="operator">=</span>/opt/python2.6 --with-threads --enable-shared make make install </pre> <p>This will install python 2.6 in <strong>/opt/python2.6</strong> and will not interfere with system python install. Let's make a symbolic link to our newly installed python:</p> <pre class="literal-block"> ln -s /opt/python2.6/bin/python /usr/bin/python2.6&lt; </pre> <p>Also we must instruct our system where it should find the libs for new python.</p> <pre class="code bash literal-block"> <span class="name builtin">echo</span> <span class="literal string single">'/opt/python2.6/lib'</span> &gt;&gt; /etc/ld.so.conf.d/opt-python2.6.conf ldconfig </pre> <p>You shoud check if your new python install is working as it should:</p> <pre class="code python literal-block"> <span class="operator">/</span><span class="name">usr</span><span class="operator">/</span><span class="name builtin">bin</span><span class="operator">/</span><span class="name">python2</span><span class="operator">.</span><span class="literal number integer">6</span> <span class="name">Python</span> <span class="literal number float">2.6</span><span class="operator">.</span><span class="literal number integer">5</span> <span class="punctuation">(</span><span class="name">r265</span><span class="punctuation">:</span><span class="literal number integer">79063</span><span class="punctuation">,</span> <span class="name">Jul</span> <span class="literal number integer">15</span> <span class="literal number integer">2010</span><span class="punctuation">,</span> <span class="literal number integer">16</span><span class="punctuation">:</span><span class="literal number integer">55</span><span class="punctuation">:</span><span class="literal number integer">23</span><span class="punctuation">)</span> <span class="punctuation">[</span><span class="name">GCC</span> <span class="literal number float">4.1</span><span class="operator">.</span><span class="literal number integer">2</span> <span class="literal number integer">20080704</span> <span class="punctuation">(</span><span class="name">Red</span> <span class="name">Hat</span> <span class="literal number float">4.1</span><span class="operator">.</span><span class="literal number integer">2</span><span class="operator">-</span><span class="literal number integer">48</span><span class="punctuation">)]</span> <span class="name">on</span> <span class="name">linux2</span> <span class="name">Type</span> <span class="literal string">&quot;help&quot;</span><span class="punctuation">,</span> <span class="literal string">&quot;copyright&quot;</span><span class="punctuation">,</span> <span class="literal string">&quot;credits&quot;</span> <span class="operator word">or</span> <span class="literal string">&quot;license&quot;</span> <span class="keyword">for</span> <span class="name">more</span> <span class="name">information</span><span class="operator">.</span> <span class="operator">&gt;&gt;&gt;</span> </pre> <p>Type <strong>CTRL+D</strong> to exit. Next thing to do is installation of python-setup tools:</p> <pre class="code bash literal-block"> <span class="name builtin">cd</span> /usr/src/python26 wget http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg sh setuptools-0.6c11-py2.6.egg --prefix<span class="operator">=</span>/opt/python2.6 </pre> <p>Make sure to define the prefix on where your new python 2.6 installation is residing. We should make a temporary alias for new python as it will be needed for installation of python mysql package:</p> <pre class="literal-block"> alias python=&quot;/opt/python2.6/bin/python&quot; </pre> <p>Let's check if everything is working ok:</p> <pre class="code python literal-block"> <span class="name">python</span> <span class="name">Python</span> <span class="literal number float">2.6</span><span class="operator">.</span><span class="literal number integer">5</span> <span class="punctuation">(</span><span class="name">r265</span><span class="punctuation">:</span><span class="literal number integer">79063</span><span class="punctuation">,</span> <span class="name">Jul</span> <span class="literal number integer">15</span> <span class="literal number integer">2010</span><span class="punctuation">,</span> <span class="literal number integer">16</span><span class="punctuation">:</span><span class="literal number integer">55</span><span class="punctuation">:</span><span class="literal number integer">23</span><span class="punctuation">)</span> <span class="punctuation">[</span><span class="name">GCC</span> <span class="literal number float">4.1</span><span class="operator">.</span><span class="literal number integer">2</span> <span class="literal number integer">20080704</span> <span class="punctuation">(</span><span class="name">Red</span> <span class="name">Hat</span> <span class="literal number float">4.1</span><span class="operator">.</span><span class="literal number integer">2</span><span class="operator">-</span><span class="literal number integer">48</span><span class="punctuation">)]</span> <span class="name">on</span> <span class="name">linux2</span> <span class="name">Type</span> <span class="literal string">&quot;help&quot;</span><span class="punctuation">,</span> <span class="literal string">&quot;copyright&quot;</span><span class="punctuation">,</span> <span class="literal string">&quot;credits&quot;</span> <span class="operator word">or</span> <span class="literal string">&quot;license&quot;</span> <span class="keyword">for</span> <span class="name">more</span> <span class="name">information</span><span class="operator">.</span> <span class="operator">&gt;&gt;&gt;</span> <span class="name">CTRL</span> <span class="operator">+</span> <span class="name">D</span> </pre> <p>So next in line is mysql for python:</p> <pre class="code bash literal-block"> <span class="name builtin">cd</span> /usr/src/python26 wget http://netcologne.dl.sourceforge.net/project/mysql-python/mysql-python/1.2.3/MySQL-python-1.2.3.tar.gz tar zxvf MySQL-python-1.2.3.tar.gz <span class="name builtin">cd </span>MySQL-python-1.2.3 python setup.py build python setup.py install </pre> <p>Again let's verify the install:</p> <pre class="code python literal-block"> <span class="name">cd</span> <span class="operator">/</span><span class="name">usr</span><span class="operator">/</span><span class="name">src</span><span class="operator">/</span><span class="name">python26</span> <span class="name">python</span> <span class="name">Python</span> <span class="literal number float">2.6</span><span class="operator">.</span><span class="literal number integer">5</span> <span class="punctuation">(</span><span class="name">r265</span><span class="punctuation">:</span><span class="literal number integer">79063</span><span class="punctuation">,</span> <span class="name">Jul</span> <span class="literal number integer">15</span> <span class="literal number integer">2010</span><span class="punctuation">,</span> <span class="literal number integer">17</span><span class="punctuation">:</span><span class="literal number integer">57</span><span class="punctuation">:</span><span class="literal number integer">29</span><span class="punctuation">)</span> <span class="punctuation">[</span><span class="name">GCC</span> <span class="literal number float">4.1</span><span class="operator">.</span><span class="literal number integer">2</span> <span class="literal number integer">20080704</span> <span class="punctuation">(</span><span class="name">Red</span> <span class="name">Hat</span> <span class="literal number float">4.1</span><span class="operator">.</span><span class="literal number integer">2</span><span class="operator">-</span><span class="literal number integer">48</span><span class="punctuation">)]</span> <span class="name">on</span> <span class="name">linux2</span> <span class="name">Type</span> <span class="literal string">&quot;help&quot;</span><span class="punctuation">,</span> <span class="literal string">&quot;copyright&quot;</span><span class="punctuation">,</span> <span class="literal string">&quot;credits&quot;</span> <span class="operator word">or</span> <span class="literal string">&quot;license&quot;</span> <span class="keyword">for</span> <span class="name">more</span> <span class="name">information</span><span class="operator">.</span> <span class="operator">&gt;&gt;&gt;</span> <span class="keyword namespace">import</span> <span class="name namespace">sqlite3</span> <span class="operator">&gt;&gt;&gt;</span> <span class="keyword namespace">import</span> <span class="name namespace">MySQLdb</span> <span class="operator">&gt;&gt;&gt;</span> <span class="name">CTRL</span> <span class="operator">+</span> <span class="name">D</span> </pre> <p>It shouldn't return any errors while issuing import statements. Ok for the end let's install the virtualenv so we can create a virtual python enviroment for each user and his Django instances</p> <pre class="code bash literal-block"> <span class="name builtin">cd</span> /opt/python2.6/bin ./easy_install virtualenv </pre> <p>And here we conclude the python instalation. Next order of business is mod_wsgi</p> </div> <div class="section" id="mod-wsgi-installation"> <h1>mod_wsgi installation</h1> <pre class="code bash literal-block"> <span class="name builtin">cd</span> /opt/python2.6/lib/python2.6/config/ ln -s ../../libpython2.6.so . <span class="name builtin">cd</span> /usr/src/python26 wget http://modwsgi.googlecode.com/files/mod_wsgi-3.2.tar.gz tar zxvf mod_wsgi-3.2.tar.gz <span class="name builtin">cd </span>mod_wsgi-3.2 ./configure –with-python<span class="operator">=</span>/opt/python2.6/bin/python make make install </pre> <p>This will install mod_wsgi in <strong>/usr/local/apache/modules/mod_wsgi.so</strong> Note that this wsgi module is built with python 2.6 and will not work with any other version. All we have to do now is add the mod_wsgi in our apache conf. You can do that on cpanel servers via WHM. Log into <strong>WHM</strong> , click on <strong>Apache Configuration</strong> then <strong>Include Editor</strong> then <strong>Pre-Virtualhost</strong> Include and select <strong>All versions</strong> Paste this:</p> <pre class="literal-block"> LoadModule wsgi_module /usr/local/apache/modules/mod_wsgi.so AddHandler wsgi-script .wsgi </pre> <p>** Save and restart apache.** And that's it. Our apache is now ready to serve python scripts via mod_wsgi Now all we have to do is install some Django instances for our user.</p> </div> <div class="section" id="setting-up-django-projects"> <h1>Setting up Django projects</h1> <p>I'll use '<strong>[username]</strong> ' for cpanel username reference and '<strong>[domain]</strong> ' for that user's domain. To match your needs you will have to replace those where appropriate. First of all you will have to decide if you will do empty virtual environments so that the user can decide what python site-packages will he use or will you preinstall some packages so that user can only update his virtualenv with packages that are missing. If you are choosing the first method user will have to have compile rights under security setting of cpanel WHM otherwise he will not be able to install packages as python mysql, PIL and some others. If you choose the former method you can preinstall those packages systemwide for our alternate python install, but whenever you update those packages all virtual environments will be updated with them. It's a matter of personal preference. I for myself rather like the clean virtualenv, and before delivering it to the client you as root preinstall in his virtualenv those packages requiring gcc rights. So let's create a virtualenv for our user:</p> <pre class="code bash literal-block"> <span class="name builtin">cd</span> /home/<span class="operator">[</span>username<span class="operator">]</span> /opt/python2.6/bin/virtualenv --no-site-packages --distribute virtualenv chown -R <span class="operator">[</span>username<span class="operator">]</span>.<span class="operator">[</span>username<span class="operator">]</span> virtualenv </pre> <p>this will create a new directory __ /home/[username]/virtualenv __ with our new virtual python environment for our user. You will have to enable normal shell for this user, let's first check what shell this user has.</p> <pre class="literal-block"> cat /etc/passwd |grep [username] </pre> <p>it should return something like this:</p> <pre class="literal-block"> [username]:x:928:923::/home/[username]:/usr/local/cpanel/bin/jailshell </pre> <p>This user has jailshell enabled to change that into normal shell do this:</p> <pre class="literal-block"> usermod -s /bin/bash [username] </pre> <p>You can also do this via <strong>WHM-&gt; Manage Shell Access.</strong> From this point on we will do everything as the user and not root, until told otherwise.</p> <pre class="literal-block"> su [username] source virtualenv/bin/activate </pre> <p>And the promt will change to something like this:</p> <pre class="literal-block"> (virtualenv)[username]&#64;[domain] [~]# </pre> <p>that way we know we are loged in as user and we are using his virtualenv. Now let's install Django in this users environment, simply enter:</p> <pre class="literal-block"> easy_install django </pre> <p>Now is also a good time to install additional python site-packages like mysql and PIL. First temporarily enable compilers for this user via <strong>WHM-&gt;Compiler Access-&gt;Allow specific users to use the compilers</strong> do the modules install with:</p> <pre class="literal-block"> easy_install [module_name] </pre> <p><strong>disable the compiler access afterwards</strong></p> <p>So we have the django installed inside the users virtualenv, all we have to do now is start one of the instance. It's probably the best practice to keep django sites outside of public_html folders on cpanel servers. So we will do (<strong>still as a user</strong>)</p> <pre class="literal-block"> cd mkdir djangosites cd djangosites django-admin.py startproject [projectname] </pre> <p>So by now we should have a directory structure like this:</p> <pre class="literal-block"> /home/[username]/virtualenv &lt; - python virtual environment /home/[username]/djangosites &lt;- django sites folder /home/[username]/djangosites/[projectname] &lt;- django project </pre> <p>Create the wsgi script for that project, usualy <strong>[projectname].wsgi</strong> in django sites folder</p> <pre class="literal-block"> nano /home/[username]/djangosites/[projectname].wsgi </pre> <p>and paste the following code:</p> <pre class="code python literal-block"> <span class="keyword namespace">import</span> <span class="name namespace">sys</span> <span class="keyword namespace">import</span> <span class="name namespace">site</span> <span class="keyword namespace">import</span> <span class="name namespace">os</span> <span class="name">vepath</span> <span class="operator">=</span> <span class="literal string">'/home/[username]/virtualenv/lib/python2.6/site-packages'</span> <span class="name">prev_sys_path</span> <span class="operator">=</span> <span class="name builtin">list</span><span class="punctuation">(</span><span class="name">sys</span><span class="operator">.</span><span class="name">path</span><span class="punctuation">)</span> <span class="name">site</span><span class="operator">.</span><span class="name">addsitedir</span><span class="punctuation">(</span><span class="name">vepath</span><span class="punctuation">)</span> <span class="name">sys</span><span class="operator">.</span><span class="name">path</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">(</span><span class="literal string">'/home/[username]/djangosites'</span><span class="punctuation">)</span> <span class="name">new_sys_path</span> <span class="operator">=</span> <span class="punctuation">[</span><span class="name">p</span> <span class="keyword">for</span> <span class="name">p</span> <span class="operator word">in</span> <span class="name">sys</span><span class="operator">.</span><span class="name">path</span> <span class="keyword">if</span> <span class="name">p</span> <span class="operator word">not</span> <span class="operator word">in</span> <span class="name">prev_sys_path</span><span class="punctuation">]</span> <span class="keyword">for</span> <span class="name">item</span> <span class="operator word">in</span> <span class="name">new_sys_path</span><span class="punctuation">:</span> <span class="name">sys</span><span class="operator">.</span><span class="name">path</span><span class="operator">.</span><span class="name">remove</span><span class="punctuation">(</span><span class="name">item</span><span class="punctuation">)</span> <span class="name">sys</span><span class="operator">.</span><span class="name">path</span><span class="punctuation">[:</span><span class="literal number integer">0</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="name">new_sys_path</span> <span class="keyword namespace">from</span> <span class="name namespace">django.core.handlers.wsgi</span> <span class="keyword namespace">import</span> <span class="name">WSGIHandler</span> <span class="name">os</span><span class="operator">.</span><span class="name">environ</span><span class="punctuation">[</span><span class="literal string">'DJANGO_SETTINGS_MODULE'</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal string">'[projectname].settings'</span> <span class="name">application</span> <span class="operator">=</span> <span class="name">WSGIHandler</span><span class="punctuation">()</span> </pre> <p>give the scripts execute permissions:</p> <pre class="literal-block"> chmod +x /home/[username]/djangosites/[projectname].wsgi </pre> <p>and exit user login with:</p> <pre class="literal-block"> exit </pre> <p>now you will have your <strong>root propmt</strong> at the shell:</p> <pre class="literal-block"> root&#64;servername [/home/username]# </pre> <p>create the apache include folder for that virtualhost:</p> <pre class="literal-block"> mkdir -p /usr/local/apache/conf/userdata/std/2/[username]/[domain]/ nano /usr/local/apache/conf/userdata/std/2/[username]/[domain]/django.conf </pre> <p>and add:</p> <pre class="code apacheconf literal-block"> <span class="name tag">&lt;ifmodule</span> <span class="literal string">mod_wsgi.c</span><span class="name tag">&gt;</span> <span class="name builtin">WSGIScriptAlias</span> / <span class="literal string other">/home/</span>[username]/djangosites/[projectname].wsgi <span class="name builtin">WSGIDaemonProcess</span> [username] processes=7 threads=1 display-name=%{GROUP} <span class="name builtin">WSGIProcessGroup</span> [username] <span class="name builtin">WSGIApplicationGroup</span> %{GLOBAL} <span class="name tag">&lt;/ifmodule&gt;</span> </pre> <p>edit the virtualhost entry for that domain:</p> <pre class="literal-block"> nano /usr/local/apache/conf/httpd.conf </pre> <p>and in that virtualhost add:</p> <pre class="literal-block"> Include &quot;/usr/local/apache/conf/userdata/std/2/[username]/[domain]/*.conf&quot; </pre> <p>save &amp; exit</p> <p><strong>test the apache configuration prior to reload:</strong></p> <pre class="literal-block"> service httpd configtest </pre> <p>Should say syntax OK at the end, if so reload the apache:</p> <pre class="literal-block"> /scripts/restartsrv_httpd </pre> <p>And voila, you have your django instance up&amp; running You can now add via User's cpanel media subdomain for django media files, and define the path to the subdomain root in setings.py media root can live inside public_html folder. You can also create additional subdomain for admin media and copy the admin media content to that subdoman. For example: admin-media.[domain]:</p> <pre class="literal-block"> cp -r /home/[username]/virtualenv/lib/python2.6/site-packages/Django-1.2.1-py2.6.egg/django/contrib/admin/media/* /home/[username]/public_html/admin-media/ </pre> </div> branko@toic.org (branko)Sat, 14 Aug 2010 19:59:44 +0000https://toic.org/blog/2010/django-on-cpanel-with-python2-6-virtualenv-and-mod_wsgi/CodingLinuxTutorialsSSH port forwarding https://toic.org/blog/2010/ssh-port-forwarding/<p>In one of my previous post I made a tutorial <a href="/blog/2009/reverse-ssh-port-forwarding/">how to bypass corporate firewalls</a> and gain access into your office computer. It work well if you are at your home and you need ssh access (or any other service) to your office computer. However if the situation is reversed, and you need to access some outside service which your firewall is blocking then you would use this little tutorial with explanations. Although all this is covered in the ssh man pages, one always learn best by real life examples, so here I'll try to cover few of them.</p> <p>To better explain our first problem look at the picture below:</p> <div class="section" id="first-problem"> <h1>First problem</h1> <img alt="/media/filer_public/0e/81/0e819bee-0668-49a6-88f9-0159dfecd287/problem.png" src="/media/filer_public/0e/81/0e819bee-0668-49a6-88f9-0159dfecd287/problem.png" /> <p>We are located at our office computer which is behind very restrictive firewall, and we want to get to the non-standard service running on remote server.</p> <p>Normally I'll use Mysql Administrator for example, to connect on my MySql database on a remote server. That communication would happen on port 3306, and for this to work Mysql Administrator must have appropriate rules set in our firewall to allow that traffic.</p> <p><strong>But what if traffic on that port is blocked?</strong></p> <p>Here is where we come to <strong>ssh port forwarding</strong>. If we have ssh access on any outside computer we can route our traffic through the tunnel and gain access to the service via standard ports.</p> </div> <div class="section" id="second-problem"> <h1>Second problem</h1> <p>Ssh tunnel can be also used to establish connection from insecure networks to standard and non-standard services inside secured firewalled network as shown in picture bellow:</p> <img alt="/media/filer_public/29/44/2944d5e5-2bf1-4d3f-a267-ead351c8c8ae/problem2.png" src="/media/filer_public/29/44/2944d5e5-2bf1-4d3f-a267-ead351c8c8ae/problem2.png" /> <p>As you can see here, server is behind firewall and all the standard ports on that server (like http, pop, imap...) are allowed. Server is also running MySQL service which is listening on port 3306. To connect to it server's firewall should allow this incoming connection.</p> <p><strong>But what if it doesn't?</strong></p> <p>What if the network from which we are connecting is insecure and we wish to maintain our data private while communicating with our trusted, firewalled network?</p> <p>Again we can solve this problem with ssh port forwarding. If we have clear and working ssh access on server or any other machine in the firewalled network, we can route our traffic through that ssh connection.</p> </div> <div class="section" id="solving-the-first-problem"> <h1>Solving the first problem</h1> <p>If our firewall is very restrictive (inbound and outbound), and you don't have control over it, you can use ssh port forwarding to your advantage. Essential things you will need are:</p> <blockquote> <ul class="simple"> <li>allowed ssh traffic on your restrictive firewall, port 22 by default</li> <li>a remote server to which you can connect via ssh, and preferably control over that remote server's firewall.</li> </ul> </blockquote> <p>If even port 22 is blocked in your restrictive firewall you can setup your outside ssh server to listen on some other port that is allowed through your firewall. You can than use <strong>-p</strong> switch in your ssh command to connect on your server. If you don't really know anything about ssh, you can always read <a class="reference external" href="/blog/2008/ssh-basics">ssh basics</a> and than come back here.</p> <p>So in our real example we want to connect to mysql service running on remote server and our firewall won't allow it. We have ssh access on that server so we will use it to tunnel this traffic.</p> <img alt="/media/filer_public/22/4b/224b7310-4743-4e27-bc7a-574dcafc11d7/solution1-1.png" src="/media/filer_public/22/4b/224b7310-4743-4e27-bc7a-574dcafc11d7/solution1-1.png" /> <p>To start up this tunnel this command will be used:</p> <pre class="literal-block"> ssh -L 3306:localhost:3306 username&#64;server </pre> <p>This will actually open up a port 3306 on our local computer listening on loopback interface through established ssh connection on to server's port 3306. If we already have mysql server running on local machine then the port 3306 is already in use, so we need to use another port on our loopback interface, so the command would look like:</p> <pre class="literal-block"> ssh -L 3307:localhost:3306 username&#64;server </pre> <p>Then we use our service client, in this case Mysql administrator and instruct it to connect to <strong>127.0.0.1</strong> at our specified port. We can use the same command to tunnel any other port and or service this way.</p> <div class="section" id="extending-this-example"> <h2>Extending this example</h2> <p>We can also use this connection method to our remote server for routing traffic to some other servers. Of course our remote server must to be able to connect to that remote service. This is usually very popular to forward traffic to some online games running on non standard ports, EvE, Warcraft, and any other game (this might produce additional lag on FPS games).</p> <img alt="/media/filer_public/1b/b4/1bb407b5-0aee-4081-8511-d2d66eb75abb/solution1-2.png" src="/media/filer_public/1b/b4/1bb407b5-0aee-4081-8511-d2d66eb75abb/solution1-2.png" /> <p>As shown on picture, for example we have outside gaming server running at port 66732, and that port is blocked in our firewall. We can use our remote server with ssh connection to establish a ssh tunnel and then route that traffic to our local computer.</p> <p>To do so we would use this command:</p> <pre class="literal-block"> ssh -L 66732:remote.gameserver:66732 username&#64;our.server </pre> <p>On our loopback interface (127.0.0.1), this will create a listening port <strong>66732</strong> which will then be forwarded to <strong>remote.gameserver</strong>'s port <strong>66732</strong> through our ssh connection on port 22. All you need to do is instruct your game client to connect on localhost. This can also be used in constructive purposes, like using your remote shell server to route traffic this way to remote mysql server on which you don't have ssh access but is available to your remote server via 3306 port. Bare in mind that your remote ssh server will have to be able to connect to remote.gameserver/mysql server on appropriate port, it your remote server have any outbound firewall rules filtering this traffic, this example will not work until you open that port.</p> <p>As with <a class="reference external" href="/blog/2009/reverse-ssh-port-forwarding/">reverse ssh port forwarding</a> we can make this connection available to other computers on our LAN by specifying listening interface while establishing the tunnel. I'll go with remote gameserver example and enable our co-workers to connect via same ssh tunnel to remote gameserver without them needing to create their own tunnels.</p> <p>If you have multiple network interfaces on your computer you will specify the one whit which you are connected to your co workers, but you can also enable it on all interfaces like this:</p> <img alt="/media/filer_public/98/28/98282ce6-94d4-4e0a-af3e-23031da34173/solution1-3.png" src="/media/filer_public/98/28/98282ce6-94d4-4e0a-af3e-23031da34173/solution1-3.png" /> <pre class="literal-block"> ssh -L 0.0.0.0:66732:remote.gameserver:66732 username&#64;our.server </pre> <p>when you do <strong>netstat -ntl</strong> on your machine you will see it's listening on <strong>0.0.0.0:66732</strong>. Your co-workers can now connect to your's office pc ip on that port and their connections will be also tunneled via this established ssh connection. Bare in mind that the <strong>remote.gameserver</strong> will see all the connections comming from our.server so if the remote.gameserver have any per ip connection count limit this will obviously be a problem.</p> </div> </div> <div class="section" id="solving-the-second-problem"> <h1>Solving the second problem</h1> <p>How is this problem different form the first one. In essence it's not, only differences is that our firewall is permitting the non standard traffic, or we don't even have a firewall to worry about, but the server's firewalled network is very restrictive and its blocking our non standard ports. Ssh tunnel commands used in this example will be the same. However I can use this example for demonstrating why ssh tunnel can be useful for.</p> <p>Say we don't have firewall and port limitations, but we are temporary on insecure network and server is on secured/trusted network. If we were to use any of the old plain test services like ftp, pop3, imap, synergy, etc... malicious hosts/users can sniff out that traffic and find out any usernames and passwords sent via plain text. Even the contents of the connection. So here's were ssh tunnel steps in again.</p> <p>We can start an ssh tunnel from our insecure network to our secured server's network and tunnel any plain text/insecure traffic through it. By default ssh tunnel binds its self to loopback interface on our computer, which malicious network user doesn't have the access to, and as the ssh is encrypted, all the traffic passing through this tunnel will be plain gibberish to any malicious user sniffing our traffic on this insecure network.</p> <p>Just like in one of the previous posts (<a class="reference external" href="/blog/2009/secure-synergy-setup/">secure synergy setup</a>) we will use this tunnel to secure our traffic from eavesdropping. Ftp actually has sftp (ftp over ssh) so if you already have ssh you will want to use that instead of tunneling ftp traffic, but if for any reason you are not able to use it then this should work as well.</p> <p>I'll take my example on pop3:</p> <img alt="/media/filer_public/50/3e/503e2672-c102-4a04-9889-09a0e71365c4/problem2-1.png" src="/media/filer_public/50/3e/503e2672-c102-4a04-9889-09a0e71365c4/problem2-1.png" /> <p>We establish ssh connection to secured remote server and tunnel the port 110 on our loopback interface to server's 110 through established ssh session. Then we connect with our mail client to localhost and preform plain text pop3 authentication through secure ssh tunnel.</p> <pre class="literal-block"> ssh -L 110:localhost:110 user&#64;secure.server </pre> <p>If you don't have access to the mail server's ssh then you can use another ssh host on secured network to route traffic with this:</p> <pre class="literal-block"> ssh -L 110:mailserver:110 user&#64;secure.server </pre> <div class="section" id="few-useful-tips"> <h2>Few useful tips</h2> <p>As mentioned few times before, perhaps the firewall will not allow standard ssh ports through, or your ssh server is running on different port in which case you should use <strong>-p</strong> switch with ssh. For example your ssh server is running on port 2210 then to forward pop3 traffic you would use:</p> <pre class="literal-block"> ssh -p 2210 -L 110:mailserver:110 user&#64;secure.server </pre> <p>You can also speed things up by using <a class="reference external" href="/blog/2008/ssh-basics/#ssh-key">ssh keys</a> and <a class="reference external" href="/blog/2008/ssh-basics/#ssh-agent">ssh-agent</a></p> <p>And of course if you wish to use your ssh connection only for port forwarding an wish to put it into background you would use:</p> <pre class="literal-block"> ssh -N -f -L 110:mailserver.110 user&#64;secure.server </pre> <p>And from here it's all combinations of above used commands.</p> <p>Have fun!</p> </div> </div> branko@toic.org (branko)Tue, 16 Feb 2010 20:58:36 +0000https://toic.org/blog/2010/ssh-port-forwarding/LinuxTutorialsHowto create rsync server https://toic.org/blog/2009/howto-create-rsync-server/<p>There are tons of reasons why would one want to create a rsync server. For example you wish to backup your data to a remote server but you don't want to backup everything every time. <br /><blockquote>rsync is an open source utility that provides fast incremental file transfer. rsync is freely available under the GNU General Public License and is currently being maintained by Wayne Davison. </blockquote></p> <p>As you can see rsync is ideal for this. You can use it within ssh protocol, rsh and rsync itself. Creating a rsync server will allow you to create easily accessible storage server, update server for your scripts, etc.</p> <p>Anyway let's get started on configuring rsync server which will serve as remote backup server.</p> <p>First make sure you have tcp and udp port <strong>873</strong> open in your firewall. Next install rsync on your machine (if you don't have it yet), and xinetd as well.</p> <pre class="literal-block"> yum install -y rsync xinetd </pre> <p>We will make rsync available trouh xinetd so you must enable it by editing its conf file</p> <pre class="literal-block"> nano /etc/xinetd.d/rsync </pre> <p>edit the line saying:</p> <pre class="literal-block"> disable = yes </pre> <p>to:</p> <pre class="literal-block"> disable = no </pre> <p>so the entire file should look something like this:</p> <pre class="literal-block"> service rsync { disable = no socket_type = stream wait = no user = root server = /usr/bin/rsync server_args = --daemon log_on_failure += USERID } </pre> <p>Next we want to create rsync client username and password:</p> <pre class="literal-block"> nano /etc/rsyncd.secrets </pre> <p>and enter a username and password in format:</p> <pre class="literal-block"> username:password </pre> <p>yes it's plain text.</p> <p>Let's create a rsync server conf file:</p> <pre class="literal-block"> nano /etc/rsyncd.conf </pre> <p>now here enter:</p> <pre class="literal-block"> #maximum allowed connections max connections = 10 #where to log log file = /var/log/rsync.log timeout = 300 </pre> <p>To create a share using a password and being able to send files to rsync server, we shoulwd add to our /etc/rsyncd.conf:</p> <pre class="literal-block"> [backup] comment = Backup place for my office computers path = /backup/ read only = false list = yes uid = backup gid = backup hosts allow = 192.168.0.0/24 # i want to limit the rsnyc server only to this group of hosts secrets file = /etc/rsyncd.secrets auth users = username #enter username specified in secrets file </pre> <p>Now we have a rsync server module at path /backup which will allow only hosts within 192.168.0.0/24 network and users authenticated by username specified in secrets file.</p> <p>To make sure this will be somewhat secure let's change permissions on rsync config files</p> <pre class="literal-block"> chown root.root /etc/rsyncd.* chmod 600 /etc/rsyncd.* </pre> <p>Restart the xinetd:</p> <pre class="literal-block"> service xinetd restart </pre> <p>and voila.</p> <p>Let's go test it out from one of our client hosts:</p> <pre class="literal-block"> rsync rsync.server.com:: backup Backup place for my office computers </pre> <p>To actually backup something onto this host we would use:</p> <pre class="literal-block"> rsync -avz ./ username&#64;rsync.server.com::backup </pre> <p>the command would ask us for a password specified in secrets file. After successful login rsync will start to transfer files to remote machine. Next time we start it it will only transfer the differences since last time.</p> <p>If you would like to script this, entering a password could be a problem. Luckily rsync offers a solution as password file.</p> <pre class="literal-block"> nano /home/branko/.rsync.pass </pre> <p>enter your password here and chmod this file to 600 so it's only readable by you. start the rsync with following command:</p> <pre class="literal-block"> rsync -avz --password-file=/home/branko/.rsync.passw ./ username&#64;rsync.server.com::backup </pre> <p>To setup another share for download only we would create a read-only share without passwords. just append this to your /etc/rsyncd.conf file:</p> <pre class="literal-block"> [update] comment = update downloads path = /home/branko/update read only = true list = yes uid = branko gid = branko hosts allow = 192.168.0.0/24 </pre> <p>Now you may see there is no auth user or secrets password. So when we issue the rsync command on our server again:</p> <pre class="literal-block"> rsync rsync.server.com:: </pre> <p>you will se another module available by the name update.</p> <p>to rsync content from this module just use:</p> <pre class="literal-block"> rsync -avz rsync.server.com::update ./ </pre> branko@toic.org (branko)Wed, 21 Oct 2009 19:57:21 +0000https://toic.org/blog/2009/howto-create-rsync-server/TutorialsMunin centralized monitoring on Centos https://toic.org/blog/2009/munin-centralized-monitoring-on-centos/<p>Munin is a great tool for performance graphing your servers, by default it will graph resources on localhost, however if you wish to monitor multiple servers from single location, then you must deploy a central Munin server.</p> <p>Having central graphs mean you can have central hub of data about performance of your systems. This can later be better presented, viewed and analyzed, </p> <p>Let's begin setting up a central server</p> <p>To get things started we will need one server for centralized graphs. For start this can be a low budget dedicated server or a small vps, if you have large amount of monitored nodes, and large amout of metrics on them, then you will probably want to invest in better disks. Updating huge amounts of rrd files and regenerating html an graphs can be disk IO intensive with large instalations.</p> <div class="section" id="installing-munin"> <h1>Installing munin</h1> <p>This will be a minimal install for a central munin server. I'm using a small vps with minimal centos 5 install.</p> <p>First let's setup elrepo:</p> <pre class="literal-block"> rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm </pre> <p>Now we need to install munin and munin-node (if you wish to monitor this host as well):</p> <pre class="literal-block"> yum install -y munin munin-node </pre> <p>by default munin will put its html files into <strong>/var/www/html/munin</strong> folder If you wish to move that to another place, now is your time. For the sake of simplicity I'll just leave it where it is. Of course we will need apache to access munin html files, so if you don't have apache installed do:</p> <pre class="literal-block"> yum install -y httpd </pre> <p>now start the apache:</p> <pre class="literal-block"> service httpd start </pre> <p>If you left everything as it is munin html should be available at: <a class="reference external" href="http://yourhostname.com/munin/">http://yourhostname.com/munin/</a> You may notice that there is nothing there yet, just wait until we configure all other hosts. start the munin-node on this host (if you installed it):</p> <pre class="literal-block"> service munin-node start </pre> <p>Make sure your cron is runing:</p> <pre class="literal-block"> service crond status </pre> <p>and let's go configure those other hosts.</p> </div> <div class="section" id="installing-munin-node"> <h1>Installing munin-node</h1> <div class="section" id="installing-on-cpanel"> <h2>Installing on cPanel</h2> <p>Since lot's of my servers to monitor are with cPanel installed there is an easy way to install munin.</p> <p>Login to your <strong>whm</strong> go to: <strong>Manage plugins</strong>, now find <strong>Munin</strong>, click a check box, scroll down and click save. After the munin is installed it should appear in your <strong>whm</strong> at the bottom of the navigation. Go and check up if the munin is installed correctly.</p> <p>Installing trough cPanel will install munin-node and munin, you can disable the munin graphing later if you like.</p> </div> <div class="section" id="installing-on-non-cpanel"> <h2>Installing on non cPanel</h2> <p>We can install munin on Centos trough Elrepo. first we will setup elrepo:</p> <pre class="literal-block"> rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm </pre> <p>and then install a munin-node:</p> <pre class="literal-block"> yum install -y munin-node </pre> <p>voila... let's configure nodes on remote servers now.</p> </div> </div> <div class="section" id="configuring-munin-node-on-remote-hosts"> <h1>Configuring munin-node on remote hosts</h1> <p>For both cPanel and non-cPanel servers all we need to do is add allowed host in munin-node.conf:</p> <pre class="literal-block"> nano /etc/munin/munin-node.conf </pre> <p>add at the end of the file:</p> <pre class="literal-block"> allow ^192\.168\.0\.20$ </pre> <p>where 192.168.0.20 is the IP address of you central munin server. restart the munin-node:</p> <pre class="literal-block"> service munin-node restart </pre> <p>If you have firewall installed on that host (and I hope you do), allow the incoming tcp port 4949 for the IP of the central node.</p> <p>In <a class="reference external" href="http://www.configserver.com/cp/csf.html">csf</a> add the following line:</p> <pre class="literal-block"> tcp:in:d=4949:s=192.168.0.20 </pre> <p>to your <strong>/etc/csf/csf.allow</strong> file</p> <p>or just run:</p> <pre class="literal-block"> iptables -A INPUT -p tcp -s 192.168.0.2 --dport 4949 -m state --state NEW,ESTABLISHED -j ACCEPT </pre> <p>Modify this to your firewall, and don't forget to replace 192.168.0.20 with your munin server IP. Now everything should be ready for data collection from central server</p> </div> <div class="section" id="configuring-munin-server"> <h1>Configuring munin server</h1> <p>We need to configure munin conf file on our central server to collect data from remote servers:</p> <pre class="literal-block"> nano /etc/munin/munin.conf </pre> <p>If you didn't change any locations of html files and munin datastore you realy don't need to change that in the conf file.</p> <p>What we are interested with are the host sections. You will notice there is configuration for our localhost You can change its name now, leave the address field as it is.</p> <p>To add up a new host just add:</p> <pre class="literal-block"> [myhost.mydomain.com] address 192.168.0.10 use_host_name yes </pre> <p>change the 192.168.0.10 with the IP of the server you wish to monitor. you can now add as many host you like. Make sure that you have enabled outgoing connections on tcp port 4949 on your central munin server. After a while the first results should start to appear.</p> </div> <div class="section" id="configuring-multi-host-display-graphs"> <h1>Configuring multi host display graphs</h1> <p>The real benefit of having all the host graphs and data on one place is you can easily make multi host graphs and compare the loads on the servers. This could help you grasp a bigger picture of individual server workloads and give you an idea what to improve and how to load balance between the machines.</p> <p>Here is one of the example graphs, showing apache request per second. If the machines were the same hardware configuration that would give indications that some of the machines have higher hit rate and we would need to rewrite our load balancing.</p> <p>We could do the same thing with load graphs and see which servers have the spikes, and distribute the workload on some less loaded servers.</p> <img alt="/media/filer_public/00/62/00627fe5-be95-4a9f-bb98-fc5979a0b320/munin1.png" src="/media/filer_public/00/62/00627fe5-be95-4a9f-bb98-fc5979a0b320/munin1.png" /> <img alt="/media/filer_public/37/24/37244bfb-ece6-4bb3-b152-b2d4bc0d6d6d/munin2.png" src="/media/filer_public/37/24/37244bfb-ece6-4bb3-b152-b2d4bc0d6d6d/munin2.png" /> <p><strong>So how do we configure this?</strong></p> <p>First you need to find out rrd's name of the data you wish to put on the graph.</p> <p>for example apache accesses per second:</p> <pre class="literal-block"> cd /var/lib/munin/yourdomain.com </pre> <p><strong>ls -lh</strong> in the directory and you will find out what data is available to munin. in case of the apache accesses data we will have few files named:</p> <blockquote>hostname.domainname.com-<span style="color: red;">apache_accesses-accesses80</span>-d.rrd hostname2.domainname.com-<span style="color: red;">apache_accesses-accesses80</span>-d.rrd</blockquote><p>what we are interested with are those fields (marked in red) after the domain name separated by dash. Ok let's write a conf in munin.conf for this two hosts:</p> <pre class="literal-block"> nano /etc/munin/munin.conf </pre> <p>Go under the host definitions in your conf file and add:</p> <blockquote><strong>[domainname.com;Totals] <span style="color: green;">update no</span> <span style="color: blue;">apacheaccess</span>.graph_title Apache access side by side <span style="color: blue;">apacheaccess</span>.graph_order hostname=hostname.domainname.com:<span style="color: red;">apache_accesses.accesses80</span> hostname1=hostname1.domainname.com.com:<span style="color: red;">apache_accesses.accesses80</span></strong></blockquote><p>Notice the red lines, they are the same as rrd filenames red parts we saw earlier, just replace dash with dot. Green text is to disable updates for this domain declaration since updates are already done at the host declaration in the conf file. Blue is the graph representation name, followed by title in the first line and data in second.</p> <p>This way you can make all the side by side graphs for all the data munin collected in rrd files. After the changes wait for a next munin update and enjoy the graphs</p> </div> branko@toic.org (branko)Fri, 09 Oct 2009 19:56:24 +0000https://toic.org/blog/2009/munin-centralized-monitoring-on-centos/MonitoringTail -f in python, truncate aware https://toic.org/blog/2009/tail-f-in-python-truncate-aware/<p>While doing a little coding I tried to find some tail -f class in python that will recognize when file that we tailing is been truncated. All I found was some tail -f classes that brakes on file truncate or rotate. </p> <p></p> <p>Eventually I came up with this</p> <pre class="code python literal-block"> <span class="keyword namespace">import</span> <span class="name namespace">time</span> <span class="keyword namespace">from</span> <span class="name namespace">os</span> <span class="keyword namespace">import</span> <span class="name">stat</span> <span class="keyword namespace">from</span> <span class="name namespace">os.path</span> <span class="keyword namespace">import</span> <span class="name">abspath</span> <span class="keyword namespace">from</span> <span class="name namespace">stat</span> <span class="keyword namespace">import</span> <span class="name">ST_SIZE</span> <span class="keyword">class</span> <span class="name class">LogTail</span><span class="punctuation">:</span> <span class="keyword">def</span> <span class="name function">__init__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">logfile</span><span class="punctuation">):</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">logfile</span> <span class="operator">=</span> <span class="name">abspath</span><span class="punctuation">(</span><span class="name">logfile</span><span class="punctuation">)</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">f</span> <span class="operator">=</span> <span class="name builtin">open</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">logfile</span><span class="punctuation">,</span><span class="literal string">&quot;r&quot;</span><span class="punctuation">)</span> <span class="name">file_len</span> <span class="operator">=</span> <span class="name">stat</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">logfile</span><span class="punctuation">)[</span><span class="name">ST_SIZE</span><span class="punctuation">]</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">f</span><span class="operator">.</span><span class="name">seek</span><span class="punctuation">(</span><span class="name">file_len</span><span class="punctuation">)</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">pos</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">f</span><span class="operator">.</span><span class="name">tell</span><span class="punctuation">()</span> <span class="keyword">def</span> <span class="name function">_reset</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">):</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">f</span><span class="operator">.</span><span class="name">close</span><span class="punctuation">()</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">f</span> <span class="operator">=</span> <span class="name builtin">open</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">logfile</span><span class="punctuation">,</span> <span class="literal string">&quot;r&quot;</span><span class="punctuation">)</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">pos</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">f</span><span class="operator">.</span><span class="name">tell</span><span class="punctuation">()</span> <span class="keyword">def</span> <span class="name function">tail</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">):</span> <span class="keyword">while</span> <span class="literal number integer">1</span><span class="punctuation">:</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">pos</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">f</span><span class="operator">.</span><span class="name">tell</span><span class="punctuation">()</span> <span class="name">line</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">f</span><span class="operator">.</span><span class="name">readline</span><span class="punctuation">()</span> <span class="keyword">if</span> <span class="operator word">not</span> <span class="name">line</span><span class="punctuation">:</span> <span class="keyword">if</span> <span class="name">stat</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">logfile</span><span class="punctuation">)[</span><span class="name">ST_SIZE</span><span class="punctuation">]</span> <span class="operator">&lt;</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">pos</span><span class="punctuation">:</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_reset</span><span class="punctuation">()</span> <span class="keyword">else</span><span class="punctuation">:</span> <span class="name">time</span><span class="operator">.</span><span class="name">sleep</span><span class="punctuation">(</span><span class="literal number integer">1</span><span class="punctuation">)</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">f</span><span class="operator">.</span><span class="name">seek</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">pos</span><span class="punctuation">)</span> <span class="keyword">else</span><span class="punctuation">:</span> <span class="literal string doc">&quot;&quot;&quot;print, return or otherwise manipulate the tailed line&quot;&quot;&quot;</span> <span class="keyword">print</span> <span class="name">line</span> </pre> <p>Maybe it's a too much overhead to check each time for file size, but you get the general idea.</p> <p>So anyways here's a usage example</p> <pre class="code python literal-block"> <span class="name">tail</span> <span class="operator">=</span> <span class="name">LogTail</span><span class="punctuation">(</span><span class="literal string">&quot;/var/log/messages&quot;</span><span class="punctuation">)</span> <span class="name">tail</span><span class="operator">.</span><span class="name">tail</span><span class="punctuation">()</span> </pre> <p>This will print out any new line appended to /var/log/messages file. If the file gets truncated or log rotated, class will detect it and will return to the start.</p> branko@toic.org (branko)Tue, 11 Aug 2009 19:55:19 +0000https://toic.org/blog/2009/tail-f-in-python-truncate-aware/CodingFirewalling xen bridge https://toic.org/blog/2009/firewalling-xen-bridge/<p>Occasionally you will wish to block certain ports to your DomUs from Dom0. By default you wish to allow any traffic from and to DomU but for some security considerations, I found it to be wise to block some ports to and from my clients DomUs. One such port range is for example IRC. Although it can be routed trough alternate ports, most of automated malicious scripts use default ones. It's quite handy to block them so they ain't able to contact home.</p> <p>As said by default Xen bridge is open for all traffic from and to DomUs. It's up to DomU admin to firewall their own virtual machine. Unfortunately some just forget to do the proper securing of the system, and as a result you get compromised DomU contacting various botnets, and executing all kind of nasty stuff.</p> <p>To prevent this we can make a firewall rules in DomU that will by default block some traffic. Since I'm using bridged network, firwalling must be done on bridge. I found <a class="reference external" href="http://www.shorewall.net/Xen.html">this</a> great article on shorewall manuals explaining how to setup bridged network firewall. I installed it and tested it on 32bit Centos 5.2, but it should work on any system.</p> <p>Fist of all you will need to <a class="reference external" href="http://shorewall.net/download.htm">download</a> and install latest shorewall.</p> <p>As stated in documentation link above, you must enable bridge support in shorewall.conf:</p> <pre class="literal-block"> nano /etc/shorewall/shorewall.conf </pre> <p>Set:</p> <pre class="literal-block"> BRIDGING=Yes </pre> <p>Now we have to edit our firewall zones:</p> <pre class="literal-block"> nano /etc/shorewall/zones </pre> <p>It should look something like this:</p> <pre class="literal-block"> #ZONE TYPE OPTIONS IN OUT # OPTIONS OPTIONS fw &nbsp; &nbsp;firewall dom0 ipv4 domU ipv4 net ipv4 #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE </pre> <p>Next thing to do is to define network interfaces, we will be dealing with two network interfaces: virtualized eth0 and bridge:</p> <pre class="literal-block"> nano /etc/shorewall/interfaces </pre> <p>And the file should look like this:</p> <pre class="literal-block"> #ZONE INTERFACE &nbsp; &nbsp;BROADCAST &nbsp; &nbsp;OPTIONS - &nbsp; &nbsp;xenbr0 - dhcp net eth0 detect dhcp #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE </pre> <p>Next stop, hosts file:</p> <pre class="literal-block"> nano /etc/shorewall/hosts </pre> <p>And the file should look like this:</p> <pre class="literal-block"> #ZONE HOST(S) OPTIONS dom0 xenbr0:vif0.0 domU xenbr0:vif+ &nbsp; &nbsp; routeback net xenbr0:peth0 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS LINE -- DO NOT REMOVE </pre> <p>Now let's make some policies in our firewall:</p> <pre class="literal-block"> nano /etc/shorewall/policy </pre> <p>And the file should look like this:</p> <pre class="literal-block"> #SOURCE DEST POLICY LOG LIMIT: CONNLIMIT: # LEVEL BURST MASK fw &nbsp; &nbsp;all ACCEPT all fw &nbsp; &nbsp;ACCEPT info dom0 all ACCEPT all dom0 ACCEPT info domU all ACCEPT all domU &nbsp; &nbsp; ACCEPT net net NONE all all REJECT info #LAST LINE -- DO NOT REMOVE </pre> <p>This will by default allow any traffic through the bridge. You can also specify DROP policy for your Dom0 and then open necessary ports in rules file. Note that the fw and dom0 are the same but they both need to be declared in policy and rules file. So... for now, this does not block IRC traffic as we started to do, all we need to do now is to setup the rules file:</p> <pre class="literal-block"> nano /etc/shorewall/rules </pre> <p>And the file should look like this:</p> <pre class="literal-block"> #ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ MARK CONNLIMIT &nbsp; &nbsp;TIME PORT PORT(S) DEST LIMIT GROUP #irc REJECT net domU tcp 6660:6669 REJECT domU net tcp 6660:6669 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE </pre> <p>After adding this, it will block all incoming and outgoing traffic from port range 6660 to 6669 for all DomUs. If you wish to add an exception to one DomU you can simply edit the rules file and insert the exception above the REJECT:</p> <pre class="literal-block"> #ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ MARK CONNLIMIT &nbsp; &nbsp;TIME PORT PORT(S) DEST LIMIT GROUP #DomU exceptions ACCEPT net domU:192.168.0.10 &nbsp; &nbsp;tcp 6660:6669 ACCEPT domU:192.168.0.10 &nbsp; &nbsp;net tcp 6660:6669 #DomU restrictions #irc REJECT net domU tcp 6660:6669 REJECT domU net tcp 6660:6669 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE </pre> <p>This way only the DomU with ip 192.168.0.10 will have unblocked IRC ports. Although the above config should work it didn't for me. Centos 5.2 by default comes with:</p> <pre class="literal-block"> net.bridge.bridge-nf-call-iptables = 0 </pre> <p>so no bridge firewalling is actually done. To enable this edit your sysctl.conf file:</p> <pre class="literal-block"> nano /etc/sysctl.conf </pre> <p>and append:</p> <pre class="literal-block"> net.bridge.bridge-nf-call-iptables = 1 </pre> <p>now run:</p> <pre class="literal-block"> sysctl -p </pre> <p>And the bridged firewall for your DomUs should work now.</p> branko@toic.org (branko)Sun, 19 Apr 2009 19:52:29 +0000https://toic.org/blog/2009/firewalling-xen-bridge/VirtualizationSecure synergy setup https://toic.org/blog/2009/secure-synergy-setup/<p>Synergy is a nifty tool for cross platform clipboard, keyboard and mouse sharing. It's reasonably easy to configure synergy server for use with multiple synergy clients. </p> <p>Doing so will spare you some time while working on multiple computers at your desk at once. I use it at office to connect my laptop's and office computer mouse, keyboard and clipboard and thus reducing or completely eliminating need to lean over my laptop every time I need to use it. Anyway, most of the people use it with quicksynergy wrapper allowing even easier setup, but what the synergy lack is a means of authentication and security in data transfers. I'll try to guide you how to make a secure synergy setup on untrusted networks.</p> <p>So for a starter you will need to setup a synergy config file to use it with your synergy server. While using a quicksynergy may be easier we won't use it since it lacks some flexibility.</p> <p>I'm using my laptop named blap and my office computer named kex. Blap is located to the left of kex so I will need a conf file looking like this:</p> <pre class="literal-block"> section: screens blap: kex: end section: links kex: left = blap blap: right = kex end </pre> <p>at the first section we define two screens, one for laptop and one for office computer each named by their host name. At the second section we define links between two screens which states that left from computer kex is computer blap. And for blap right edge of screen is linked with computer kex. We can define as many hosts we like in relative positions. You can consult manual page of synergys f or all available options.</p> <p>When done configuring screens and links save that file as <strong>synergy.conf</strong> in your home directory.</p> <p>Starting a server with:</p> <pre class="literal-block"> synergys -c /home/branko/synergy.conf </pre> <p>will allow us to connect to our office computer using our laptop and merging screens.</p> <p>Like stated earlier, synergy server have no means of authentication so any client within our network can connect to. Naturally if I'm on busy or untrusted network this isn't very appealing thought. O n top of that, all traffic between synergy server and client is unencrypted so anyone on local network can eavesdrop with tcpdump, wireshark or any other network capturing program. Anything that gets</p> <div class="system-message"> <p class="system-message-title">System Message: ERROR/3 (<tt class="docutils">&lt;string&gt;</tt>, line 32)</p> Unexpected indentation.</div> <blockquote> to clipboard is available to our malicious user on our local network.</blockquote> <div class="system-message"> <p class="system-message-title">System Message: WARNING/2 (<tt class="docutils">&lt;string&gt;</tt>, line 33)</p> Block quote ends without a blank line; unexpected unindent.</div> <p>So how can we implement some sort of encryption and authentication on our synergy server.</p> <p>First we will add additional parameter to our synergy server startup line:</p> <pre class="literal-block"> synergys -a 127.0.0.1 -c /home/branko/synergy.conf </pre> <p>this way synergy server will start listening on loopback network interface only, instead on all network interfaces. This way we are only allowing access to synergy server to locally authenticated use rs. You can now put this command in session startup.</p> <p>Since server is now not available on any outside interface we must first login and authenticate our self to the office computer. While doing so we will also open a ssh tunnel to our laptop.</p> <p>Prior to executing our synergy client on laptop I will need to execute:</p> <pre class="literal-block"> ssh -N -f -L 24800:localhost:24800 branko&#64;192.168.0.100 </pre> <p>this will open up ssh connection to my office computer (192.168.0.100) for which I will need to login as user <strong>branko</strong> an when I do so port <strong>24800</strong> on <strong>192.168.0.100</strong> will be tunneled to my loc alhost's port <strong>24800</strong>.</p> <p>Now I can simply start up my synergy client on my laptop by executing:</p> <pre class="literal-block"> synergyc localhost </pre> <p>Now all the traffic between my laptop and office computer is encrypted and as such information traveling trough the ssh tunnel are unavailable to possible eavesdropping, and since we started the serv er on a loopback interface no malicious client can be connected from outside. For the ease of use you can combine the above comands in single shell script and saving it in users private bin folder:</p> <pre class="literal-block"> vim ~/bin/synergy </pre> <p>paste the text inside:</p> <pre class="literal-block"> #!/bin/sh ssh -N -f -L 24800:localhost:24800 username&#64;synergyserver synergyc localhost </pre> <p>Make it executable:</p> <pre class="literal-block"> chmod +x ~/bin/synergy </pre> <p>And now you can simply type <strong>synergy</strong> at your terminal or run command prompt after pressing <strong>ALT + F2</strong></p> branko@toic.org (branko)Sun, 01 Feb 2009 20:50:21 +0000https://toic.org/blog/2009/secure-synergy-setup/LinuxTutorialsBypassing corporate firewall with reverse ssh port forwarding https://toic.org/blog/2009/reverse-ssh-port-forwarding/<p>Probably lots of you are behind some sort of very restrictive corporate firewall. Unable to access your office pc from home because of firewall policies. In normal cases this scenario is more than welcomed. No outsiders should be allowed to access internal parts of secure network! Ideally companies will setup secure VPN access allowing its employees to access their work computers and do some work remotely. What if you aren't one of the lucky ones having such option? You desperately need to access your office pc?</p> <div class="section" id="the-problem"> <h1>The problem</h1> <img alt="/media/filer_public/e8/e7/e8e7562f-b4ed-4b6b-aab7-a1aa89c8c4dc/current.png" src="/media/filer_public/e8/e7/e8e7562f-b4ed-4b6b-aab7-a1aa89c8c4dc/current.png" /> <p>As shown on the picture above, we have our office PC behind very restrictive corporate firewall connected to Internet. Firewall will not allow any traffic originating from Internet to internal networ k except previously initiated traffic. Meaning you can contact remote hosts on Internet from your office PC and they can respond, but remote computers can't initiate connection to your office PC. Thi s is of course huge problem if you have to access your work materials on office PC from your home. Additionally corporate firewall will only allow certain traffic from your office PC to remote hosts. Meaning you can only establish FTP, SSH, HTTP, POP3... communications, all other ports are blocked.</p> <p><strong>How can you access your office PC then?</strong></p> <p>One way is to setup corporate VPN access allowing secure connections to internal network. Another method is to setup a port forwarding on corporate firewall so it redirects certain ports to your offi ce PC. But if you don't have the means to accomplish any of this then the only way to do it is to use ssh tunnels and reverse port forwarding.</p> </div> <div class="section" id="the-solution"> <h1>The solution</h1> <p>If we can only contact remote hosts on certain ports, the solution would be to contact remote hosts via allowed port and piggyback the connection on already established link.</p> <img alt="/media/filer_public/0a/cb/0acb7032-91d1-4b2d-b79f-5044c503385b/reverese-ssh.png" src="/media/filer_public/0a/cb/0acb7032-91d1-4b2d-b79f-5044c503385b/reverese-ssh.png" /> <p>Something like shown on the picture above. Fortunately we can do this with ssh.</p> </div> <div class="section" id="real-life-example"> <h1>Real life example</h1> <style> .red {color:red} .blue {color:blue} .green {color:green} .yellow {color:yellow} .brown {color:brown} .purple {color:purple} </style><p>I will assume that home PC is connected via dynamically assigned IP address. First thing you will need to make sure you have ssh server installed on your home PC and it should be accessible from Inte rnet. If you have some NAT routers, be sure to forward port 22 to your home PC. Secondly you will need to setup a dyndns account so you can connect to your home PC regardless of IP address changes. N ow the goal will be to connect to that home ssh server from our office PC.</p> <blockquote> For the purpose of this example i will name my home PC: bhome.dyndns.com Office computer name will be bwork.office.com bwork computer uses private IP range of 192.168.0.0/24 with address 192.168.0.100</blockquote> <p>So if the firewall is preventing outside connections to our bwork computer we must initiate connection from it.</p> <p>We can do this with simple ssh command:</p> <pre class="literal-block"> ssh -R 2210:localhost:22 bhome.dyndns.com </pre> <p><strong>So what just happened here?</strong> We are initiating ssh connection <strong>ssh</strong> with reverse port forwarding option <strong>-R</strong> which will then open listening port <strong>2210:</strong> who is going to be forwarded back to <strong>localhost</strong>'s port <strong>:22</strong> an d all this will happen on remote computer <strong>bhome.dyndns.com</strong>.</p> <p>This connection represents the <span class="green">green</span> line in the diagram above, and it's a legit connection as far as corporate firewall is concerned.</p> <p>Now if weopen up a terminal on <strong>bhome</strong> computer, and type in:</p> <pre class="literal-block"> ssh -p 2210 localhost </pre> <p>we will try to connect to <strong>localhost</strong> (bhome.dyndns.com) on port <strong>2210</strong>. Since that port is setuped by remote ssh connection it will tunnel the request back via that link to the <strong>bwork.office.co m</strong> computer. This is the <span class="red">red</span> line on the diagram above. Looking from firewall's perspective it's a legit traffic, since it is responding traffic on already initiated link from <strong>bwork</strong> c omputer.</p> </div> <div class="section" id="real-life-example-2"> <h1>Real life example 2</h1> <p>What if your home computer is not always on-line? Or perhaps you wish to access your office computer from multiple locations? For this you will have to have some dedicated server or VPS outside the c orporate firewall.</p> <img alt="/media/filer_public/2e/0d/2e0d1abf-c331-4625-ba0b-efed1c800254/reverese-ssh2.png" src="/media/filer_public/2e/0d/2e0d1abf-c331-4625-ba0b-efed1c800254/reverese-ssh2.png" /> <p>To accomplish this we will use the same command as previously, only this time we will open up a reverse ssh tunnel to remote server or VPS.</p> <blockquote> For the purpose of this example we will name the server bserver.outside.com with IP 89.xxx.xx.4</blockquote> <pre class="literal-block"> ssh -R 2210:localhost:22 bserver.outside.com </pre> <p>again this will open up reverse ssh tunnel to the machine 89.xxx.xx.4 (bserver.outside.com). So when we login to the server and issue the command:</p> <pre class="literal-block"> ssh -p 2210 localhost </pre> <p>we will end up with bwork computer's ssh login prompt.</p> <p><strong>Can I use this previously established reverse ssh tunnel to the server to directly connect to my office computer?</strong></p> <p>Of course, but some slight modifications are required. By default ssh tunnels only bind to local address, and can be accessible only locally. Meaning, in the example above, you can't just type:</p> <pre class="literal-block"> ssh -p 2210 bserver.outside.com </pre> <p>on your home PC and be connected to your office PC!</p> <p>If you run:</p> <pre class="literal-block"> netstat -ntl </pre> <p>on bserver you will see that the port 2210 is only listening on 127.0.0.1 IP address. To get it listen on interface connected to Internet we must enable <strong>GatewayPorts</strong> option in ssh server's config uration. By default GatewayPorts are disabled in sshd, but we can simply enable them:</p> <pre class="literal-block"> vim /etc/ssh/sshd_config </pre> <p>then add:</p> <pre class="literal-block"> GatewayPorts clientspecified </pre> <p>save the file and restart sshd:</p> <pre class="literal-block"> /etc/init.d/ssh restart </pre> <p>We could have just enable GatewayPorts by typing <strong>On</strong> instead of <strong>clientspecified</strong>, that would route any ssh tunnel to network interface. This way we can control which tunnel will be accessible f rom outside, and on which interface.</p> <p>So if we initiate reverse ssh tunnel like this:</p> <pre class="literal-block"> ssh -R 89.xxx.xx.4:2210:localhost:22 bserver.outside.com </pre> <p>we will have bserver listening on port 2210 on network interface bound to ip 89.xxx.xx.4 and forwarding all traffic via established tunnel to bwork computer. If you omit the 89.xxx.xx.4 address from the command above server will again listen on port 2210 only on local loopback interface. If you have multiple network interfaces on server be sure to select the one you can connect to. To enable listening port on all interfaces, just use IP 0.0.0.0 in the command above</p> <img alt="/media/filer_public/a2/16/a2167b08-ef7a-47fb-b200-7259e8c8e1ad/reverese-ssh3.png" src="/media/filer_public/a2/16/a2167b08-ef7a-47fb-b200-7259e8c8e1ad/reverese-ssh3.png" /> <p>Now when we run:</p> <pre class="literal-block"> ssh -p 2210 bserver.outside.com </pre> <p>from our home PC we will initiate ssh connection on port <strong>2210</strong> towards server <strong>bserver.outside.com</strong> (<span class="blue">blue line</span>). Server will then forward that traffic to office PC (<span class="red">red line</span>) via the previously established reverse ssh tunnel (<span class="green">green line</span>).</p> <blockquote> Of course you will have to open up port 2210 on server's firewall to be able to connect.</blockquote> </div> <div class="section" id="some-more-fun-with-reverse-tunnels"> <h1>Some more fun with reverse tunnels</h1> <p>Fun doesn't stops there. Say I have a printer behind that corporate firewall. How can i connect to it? Easy... remember the first example? the command ssh -R is taking 5 arguments of which 4 are mandatory:</p> <pre class="literal-block"> ssh -R [bind_address:]port:host:hostport </pre> <p><strong>bind_address</strong> is the network address on which <strong>port</strong> will be listening, and forwarded to <strong>host</strong> (connected to network from which reverse tunnel originated) on <strong>hostport</strong>.</p> <p>so if we issue the command like this on our bwork pc:</p> <pre class="literal-block"> ssh -R 89.xxx.xx.4:2211:192.168.0.10:631 bserver.outside.com </pre> <p>we will get something like this:</p> <img alt="/media/filer_public/2c/ce/2ccea4dc-0a3b-4d0e-b36e-8484ad6e6262/reverese-ssh4.png" src="/media/filer_public/2c/ce/2ccea4dc-0a3b-4d0e-b36e-8484ad6e6262/reverese-ssh4.png" /> <p>so again we have previously established reverse ssh tunnel listening on port <strong>2210</strong> to channel the ssh connection towards office PC. Now with this new command we established the reverse ssh tunnel (<span class="yellow">yellow line</span>) towards bserver which will listen for incoming connections on port <strong>2211</strong>. When the home pc makes a data connection to port <strong>2211</strong> on bserver (<span class="brown">brown line</span>) it is t hen forwarded to office PC (<strong>black line</strong>) which is then redirected towards office printer at address <strong>192.168.0.10</strong> on port <strong>631</strong> (<span class="purple">purple line</span>). Remember, all this traffic is passing trough corporate firewall as legit traffic, even if the illustration perhaps shows otherwise.</p> </div> <div class="section" id="automating-the-task"> <h1>Automating the task</h1> <p>By now we should have covered the basics on how to bypass corporate firewall in order to get to your office computer and network equipment. Now, ssh -R isn't really practical, it consumes one terminal, and as soon as it shuts down there is no tunnel and no outside connectivity for that matter. The easiest thing to do is putting a cron jo b that will connect to remote server if the connection fails, office computer reboots etc.</p> <p>First of all <a class="reference external" href="/blog/2008/ssh-basics/">generate ssh keys</a>, and add them to ssh-agent so that script won't ask you for remote server's password or local key phassphrase all the time.</p> <p>Next we will add two extra parameters to our command <strong>-N</strong> and <strong>-f</strong> so that the connection goes into the background.</p> <p>the command will look like:</p> <pre class="literal-block"> ssh -N -f -R [bind_address:]port:host:hostport </pre> <p>next we need a shell script that will be triggered by the cron. For this example we will use the Real life example 2:</p> <pre class="literal-block"> #!/bin/sh COMMAND=&quot;ssh -N -f -R 89.xxx.xx.4:2210:localhost:22 bserver.outside.com&quot; pgrep -f -x &quot;$COMMAND&quot; &gt; /dev/null 2&gt;&amp;1 || $COMMAND </pre> <p>Edit this code so it suits your needs, and save it in your home dir as <strong>reverse_ssh_tunnel.sh</strong></p> <p>Now we need to add a crontab entry which will trigger this script every 5 minutes:</p> <pre class="literal-block"> crontab -e </pre> <p>and add:</p> <pre class="literal-block"> */5 * * * * /bin/sh /home/username/reverse_ssh_tunnel.sh </pre> <p>If you are connecting to different user name on remote server you can edit your commands so they look like:</p> <pre class="literal-block"> ssh -R [bind_address]:port:host:host_port username&#64;remote_host </pre> <p>Have fun and be safe!</p> </div> branko@toic.org (branko)Sun, 18 Jan 2009 20:49:02 +0000https://toic.org/blog/2009/reverse-ssh-port-forwarding/LinuxSSH basics https://toic.org/blog/2008/ssh-basics/<p><img align='left' src="/media/filer_public/15/e3/15e30315-8620-4f84-b235-984eee853e3e/120px-gnome-terminalsvg.png" alt="ssh" width="120" height="120" style="margin-right:20px" /></p> <p>Using ssh and its functionality should be a second nature to all Linux sysadmins, for some users some aspects of ssh are still a mystery, so let's shed some light on it. Here you will not learn basics shell commands and usage, I will simply try to explain usage of ssh protocol itself, and it benefits. How to use ssh, how to setup keys, how to use ssh agent that will enable some extra functionality for programs like scp, rsync, etc.</p> <div class="section" id="first-of-all-what-is-ssh"> <h1>First of all what is ssh?</h1> <p>SSH is client-server secure shell network protocol usually used for remote host or network device administration. Other than that it allows secure tunneling, TCP port and X11 forwarding, file transfers.</p> <p>For now, let's stick to remote host administration. I'll be focusing on ssh client rather than configuring ssh server, suffice to say ssh server is a daemon, by default running on remote host and listening on tcp port 22.</p> <p>SSH client is a program used to connect to sshd on remote host.</p> <p>Simple usage would be:</p> <pre class="literal-block"> bash# ssh username&#64;hostname </pre> <p>If you're contacting remote hostname for the first time it will ask you to review and save remote host's fingerprint. It will be used in future connections to determine host's authenticity. If for some reason remote host changes its IP, hardware, location, key... you will be warned and denied access to remote host because it might be fraud, or some kind of man in the middle attack.</p> <p>If you are sure that fingerprint change is valid you will have to remove previously learned fingerprint and save a new one.</p> <p>If for some reason remote sshd is not listening on TCP port 22 you should give additional <strong>-p</strong> option to ssh client like this:</p> <pre class="literal-block"> bash# ssh -p 2210 username&#64;hostname </pre> <p>this way you will instruct your ssh client to connect to remote TCP port 2210.</p> <p>By default sshd has the password authentication enabled so using examples above will prompt you for remote user's password. Sometimes you will find remote hosts have the password authentication disabled for security reasons. To &quot;circumvent&quot; this restriction ssh has an ability to authenticate with pre-generated keys.</p> </div> <div class="section" id="how-to-generate-ssh-keys"> <h1>How to generate ssh keys?</h1> <blockquote> <strong>Note:</strong> if you are using some old Debian or Ubuntu distribution, before generating keypair verify that you have patched your libssl as noted here: <a class="reference external" href="http://www.ubuntu.com/usn/usn-612-1">http://www.ubuntu.com/usn/usn-612-1</a></blockquote> <p>To enable key authentication we must generate a new key pair. By default command:</p> <pre class="literal-block"> bash# ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: 28:ce:58:0a:61:7b:b4:d6:7b:b2:e0:11:9e:d9:b2:77 root&#64;branko.toic.org </pre> <p>will generate new default 2048bit rsa key, it will prompt you to enter a path where key will be saved and enter a passphrase. I strongly recommend that you use strong password on your key. I</p> <p>If you saved your key in default location (<strong>~/.ssh/id_rsa</strong> or <strong>~/.ssh/id_dsa</strong>) when contacting remote host ssh agent will try to use your key file as first method of authentication even if password auth is enabled on remote server. If you have saved your key file on some other location you will have to supply your ssh client with additional -i parameter followed by your private key location (later on you will learn how to overcome this parameter so don't panic just yet):</p> <pre class="literal-block"> How this thing works anyway? </pre> <p>For now all we have is two keys generated. I will asume you have used default rsa key in default location <strong>~/.ssh/id_rsa</strong> if you have made it somewhat different you will have to edit your commands accordingly.</p> <p>You should by now have your keypair located at <strong>~/.ssh/id_rsa</strong> and <strong>~/.ssh/id_rsa.pub</strong> first key (id_rsa) is your private key that you shouldn't distribute around, second key (id_rsa.pub) is public key which you will set up on remote hosts as authorized key for certain user(s).</p> <p>When connecting to remote host it will try to locate <strong>~/.ssh/authorized_keys</strong> and <strong>~/.ssh/authorized_keys2</strong> files for that user to which you are connecting. If one of those file have your public key (content of your id_rsa.pub) that is matching your private key you will be authenticated and will be allowed to login, if not ssh will try other means of authentication such as password based if enabled by the server.</p> </div> <div class="section" id="distributing-your-public-key"> <h1>Distributing your public key</h1> <p>You now have to distribute content of id_rsa.pub to authorized_keys file on remote servers for each user name that we will use. To do so you must upload your id_rsa.pub file to remote server (upload it to your home directory for now) and issue command:</p> <pre class="literal-block"> bash# cat ~/id_rsa.pub &amp;gt;&amp;gt; ~/.ssh/authorized_keys </pre> <p>This way we will append our key to authorized keys for that user.</p> <p>If you yet have no means to connect via ssh to remote host because password auth is disabled when uploading make sure remote filename will be called authorized_keys. This way we will have content of our public key in authorized keys and will enable us to connect to remote host.</p> <blockquote> <strong>Caution:</strong> If authorized_keys file already exists on remote server, uploading our id_rsa.pub file over it will disable all other authorized keys, try other means of adding content of your id_rsa.pub to authorized_keys</blockquote> </div> <div class="section" id="connecting-to-remote-hosts"> <h1>Connecting to remote hosts</h1> <p>Now we can connect to remote host with same commands as before:</p> <pre class="literal-block"> bash# ssh root&#64;branko.toic.org Enter passphrase for key '/home/branko/.ssh/id_rsa': </pre> <p>Here you will enter passphrase defined when we were creating keyfile.</p> <blockquote> <p><strong>Note:</strong> to change passphrase for current private key use command:</p> <pre class="literal-block"> bash# ssh-keygen -f [private key location] -p </pre> </blockquote> <p>Now you maybe wondering how is this any different from password authentication? For one you can now distribute your public key file to any number of hosts and use single private key file to contact all off those hosts. You can even use same private key on more than one workstation so it's not needed to recreate steps above for each workstation. Other than that, it will reduce possibility of remote password brute force attacks on your servers, enables you to use applications that use your key files for contacting remote hosts like scp, rsync, etc.</p> <p>Besides that, using ssh keys enables you some more functionality by using ssh-agent</p> </div> <div class="section" id="ssh-agent-what-is-that"> <h1>Ssh-agent? What is that?</h1> <p>Ssh agent is simple shell tool that comes bundled with ssh client. It enables you to &quot;put your keyfile and your passphrase in its hands&quot;.</p> <p>Sounds awfully! Now why would I do that?</p> <p>Consider this situation, you are working on dedicated workstation and you have 8h work time. In that period you will be contacting multiple remote hosts on multiple occasions. Entering passphrase one time after another may be acceptable at first, but later it becomes waste of time.</p> <p>Ssh-agent allows you to unlock you private key so you won't be prompted for your passphrase each time you try to connect to remote host.</p> <p>Also it allows you to add multiple private keys so you can us them all when connecting to remote hosts, if you have created keyfile that isn't in its default location, adding it to ssh-agent will not require you enter additional -i parameter to ssh client.</p> <p>After finishing with your work you can simply remove your key from agent, and it will again prompt you for your passphrase when trying to connect to remote host.</p> <blockquote> <strong>Note:</strong> for Gnome or KDE users (or any DE for that matter) ssh-agent is integrated within DE keyring for the duration of your session</blockquote> </div> <div class="section" id="how-do-i-use-this-ssh-agent"> <h1>How do I use this ssh-agent?</h1> <p>First of all we need to start the ssh agent. One would say easy way of doing it is typing ssh-agent to terminal, that may be true but it will require you to set extra environment variables afterwards. Trying to run ssh-agent will start the process and output those variables, so even easiest way of doing it is to invoke ssh agent command with eval:</p> <pre class="literal-block"> bash# eval 'ssh-agent' </pre> <p>that way we will have ssh agent and variables set at once.</p> <blockquote> <strong>Note:</strong> ssh-agent supports key lifetime, you can limit the time key can be in ssh agent by adding -t parameter followed by number of seconds or in other time format specified in sshd_config.</blockquote> <p>Now when the agent is running all we need to do is add our keyfile to agent with:</p> <pre class="literal-block"> bash# ssh-add </pre> <p>It will then add your default private key located in <strong>~/.ssh/id_rsa</strong> or id_dsa to agent. It will ask you for it's passphrase only this time. If you wish to add another private key from different location you should use:</p> <pre class="literal-block"> bash# ssh-add [location to private key] **Note:** adding **-t** parameter followed by lifetime in seconds will overwrite ssh-agents **-t** parameter. </pre> <p>To remove private key from ssh agent simply type:</p> <pre class="literal-block"> bash# ssh-add -d </pre> <p>again this will only remove default key if you have additional key inserted you should use:</p> <pre class="literal-block"> bash# ssh-add -d [location to private key] </pre> <p>to remove all private keys from ssh-agent simply type:</p> <pre class="literal-block"> bash# ssh-add -D </pre> </div> <div class="section" id="practical-use-of-all-above"> <h1>Practical use of all above?</h1> <p>Using ssh keys in combination with ssh agent will save you a lot of time. Consider example above when you frequently use ssh client to connect to multiple hosts, this will save you troubles, with password managers, time consuming password finding and retyping etc.</p> </div> branko@toic.org (branko)Mon, 17 Nov 2008 20:45:56 +0000https://toic.org/blog/2008/ssh-basics/LinuxTutorialsMultiple network interfaces in Xen https://toic.org/blog/2008/multiple-network-interfaces-in-xen/<p>By default xen tools comes with only one network interface enabled for your dom0 and domU machines. So what if you want to add some more? It's actually very simple.</p> <p>All you need to do is run:</p> <p>/etc/xen/scripts/network-bridge start vifnum=1 netdev=eth1 bridge=xenbr1</p> <p>This will effectively create one extra xenbr attached to eth1 interface. You can repeat the above command for all your interfaces, and you can stop them in the same manner, just replace start with stop.</p> <p></p> <p>To enable this automatically you can create file named, multi-network-bridge:</p> <pre class="literal-block"> vim /etc/xen/scripts/multi-network-bridge </pre> <p>paste this:</p> <pre class="literal-block"> #!/bin/sh /etc/xen/scripts/network-bridge $&#64; vifnum=0 netdev=eth0 bridge=xenbr0 /etc/xen/scripts/network-bridge $&#64; vifnum=1 netdev=eth1 bridge=xenbr1 </pre> <p>Of course you can add up as many interfaces you like in this script. After you have added your interfaces, you need to edit xend-config.spx file:</p> <pre class="literal-block"> vim /etc/xen/xend-config.spx </pre> <p>Find a line defining network script, it should by default look like this:</p> <pre class="literal-block"> (network-script network-bridge) </pre> <p>Edit it so it contains your newly created multi network bridge script. In my case it should look like:</p> <pre class="literal-block"> (network-script multi-network-bridge) </pre> <p>All you need to do now is restart xend service:</p> <pre class="literal-block"> /etc/init.d/xend restart </pre> <p>New network bridge named xenbr1 should be available now.</p> <p><strong>How can I add up another network bridge to my domU machine?</strong></p> <p>It's pretty simple actually, you already have defined <strong>vif</strong> statements for current network interface, all you need to do now is edit that domU config file, precisely <strong>vif</strong> line and add up another xenbr interface.</p> <p>For example, if my domU vif line looks like this:</p> <pre class="literal-block"> vif = ['ip=xx.xxx.167.4, vifname=vifbran0, rate = 10000KB/s, bridge=xenbr0'] </pre> <p>with another xenbr interface it should look like this:</p> <pre class="literal-block"> vif = ['ip=89.201.167.4, vifname=vifbran0, rate = 10000KB/s, bridge=xenbr0', 'ip=192.168.1.10, vifname=vifbran1, rate = 10000KB/s, bridge=xenbr1'] </pre> <p>Also if you wish to use <a class="reference external" href="/blog/2008/09/22/preventing-ip-conflicts-in-xen/">ip conflict prevention</a> you must add additional mac section in vif configuration. Restart your domU and voila, another network interface is present</p> branko@toic.org (branko)Mon, 06 Oct 2008 19:44:41 +0000https://toic.org/blog/2008/multiple-network-interfaces-in-xen/LinuxVirtualizationPreventing ip conflicts in xen https://toic.org/blog/2008/preventing-ip-conflicts-xen/<p>Lately I was playing with stock xen kernel and virtualization, and I came across one relatively big problem. Let’s say I want to share my guest machines to, let’s say clients. You must give them root… because that’s whats VPS-es all all about, having root access to OS without having to purchase expensive hardware. Having that in mind they are by default untrusted and unpredictable, they can do god knows what in there!</p> <p>So what caught my eye?</p> <p>By default xen, and available management tools, don’t really have a way of sorting out IP conflicts in bridged mode. Basically you have bunch of scripts that will provision VPS alongside with IP address. Looking at the conf files you have vif and IP declarations in vm_xen.conf file. <br /></p> <p>But what is really preventing clients from entering:</p> <pre class="literal-block"> ifconfig eth0 xxx.xxx.xxx.xxx </pre> <p>where xxx is ip of some super important server in same netmask?</p> <p>Luckily, I came across this problem while still in testing, here’s what I found and came up with after 3 days intensive googling.</p> <p>Xen supports IP declaration in <strong>vif</strong> statment of domU config file like this:</p> <pre class="literal-block"> vif = ['ip=xxx.xxx.xxx.xxx, more parametars here....'] </pre> <p>Also you can declare multiple IP's by simply putting space between them, like this:</p> <pre class="literal-block"> vif = ['ip=xxx.xxx.xxx.xx1 xxx.xxx.xxx.xx2, more parametars here....'] </pre> <p>For the purpose of IP conflict prevention make sure you declare unique mac address in vif section to.</p> <p>So what does this IP thingy in <strong>vif</strong> do? Absolutely nothing (at least not yet)!</p> <p>Next step is to install ebtables (<a class="reference external" href="http://ebtables.sourceforge.net/">http://ebtables.sourceforge.net/</a>) on your distro. Then all we need to do is patch up a vif-bridge script located in /etc/xen/scripts/</p> <p>So here’s the diff:</p> <pre class="literal-block"> --- vif-bridge-org 2008-07-30 21:26:16.000000000 +0200 +++ vif-bridge 2008-07-30 21:30:59.000000000 +0200 &#64;&#64; -57,15 +57,35 &#64;&#64; online) setup_bridge_port &quot;$vif&quot; add_to_bridge &quot;$bridge&quot; &quot;$vif&quot; - ;; - + ebtables -N $vif + ebtables -P $vif DROP + ebtables -A INPUT -i $vif -j $vif + ebtables -A FORWARD -i $vif -j $vif + ebtables -A $vif -p ARP --arp-opcode 1 -j ACCEPT + + if [ ! -z &quot;$ip&quot; ] + then + for oneip in $ip + do + ebtables -A $vif -p IPv4 --ip-src $oneip -j ACCEPT + ebtables -A $vif -p IPv4 --ip-dst $oneip -j ACCEPT + ebtables -A $vif -p ARP --arp-opcode 2 --arp-ip-src $oneip -j ACCEPT + done + ebtables -A $vif --log-prefix=&quot;arp-drop&quot; --log-arp -j DROP + fi + ;; + offline) do_without_error brctl delif &quot;$bridge&quot; &quot;$vif&quot; do_without_error ifconfig &quot;$vif&quot; down - ;; + do_without_error ebtables -D INPUT -i $vif -j $vif + do_without_error ebtables -D FORWARD -i $vif -j $vif + do_without_error ebtables -F $vif + do_without_error ebtables -X $vif + ;; esac-handle_iptable +#handle_iptable log debug &quot;Successful vif-bridge $command for $vif, bridge $bridge.&quot; if [ &quot;$command&quot; == &quot;online&quot; ] </pre> <p>Asuming you use bridging scripts this effectively restricts IP address(es) from &quot;vif = ['ip=xxx.xxx.xxx.xxx']&quot; list to <strong>mac</strong> addresses in <strong>vif</strong> list. Restrictoins are done while booting up VPS and removed when powering it off. This way untrusted user is limited only to IP addresses defined in xen guest conf file. Trying to change existing IP address into another IP on network will only render that machine unresponsive.</p> branko@toic.org (branko)Mon, 22 Sep 2008 19:07:40 +0000https://toic.org/blog/2008/preventing-ip-conflicts-xen/LinuxVirtualization