Tag Archives: fab

How to Deploy hgwebdir.fcgi behind Nginx with Fab

If you’re managing multiple mercurial repositories, it’s nice to see them all in one place, using a simple web-based repository browser. There’s various ways to publish mercurial repositories, but hgwebdir is the only method that supports multiple repos. Since I prefer fastcgi and nginx, I decided to use hgwebdir.fcgi, which unfortunately isn’t documented on the mercurial wiki.

hgweb.config

Let’s start by creating hgweb.config, which tells hgwebdir where the repos are and what the web UI should look like.

[paths]
/REPO1NAME = /PATH/TO/REPO1
/REPO2NAME = /PATH/TO/REPO2

[web]
base =
style = monoblue

There’s a few different included themes you can choose from, I like the monoblue style. The empty base= line is apparently required to make everything work.

hgwebdir.fcgi

Next, make a copy of hgwebdir.fcgi, which in Ubuntu can be found in /usr/share/doc/mercurial/examples. Below is a simplified version with all comments removed. The one line you may want to change is the path to hgweb.config on the server, but I’ll assume you’ll want it in /etc/mercurial.

from mercurial import demandimport; demandimport.enable()
from mercurial.hgweb.hgwebdir_mod import hgwebdir
from mercurial.hgweb.request import wsgiapplication
from flup.server.fcgi import WSGIServer

def make_web_app():
    return hgwebdir("/etc/mercurial/hgweb.config")

WSGIServer(wsgiapplication(make_web_app)).run()

hg_server.conf

This is a simple nginx fastcgi config you can modify for your own purposes. It forwards all requests for hg.DOMAIN.COM to the hgwebdir.fcgi socket we’ll be starting below.

server {
	listen 80;
	server_name hg;
	server_name hg.DOMAIN.COM;

	access_log /var/log/hg_access.log;
	error_log /var/log/hg_error.log;

	location / {
		fastcgi_pass	unix:/var/run/hgwebdir.sock;
		fastcgi_param	PATH_INFO	$fastcgi_script_name;
		fastcgi_param	QUERY_STRING	$query_string;
		fastcgi_param	REQUEST_METHOD	$request_method;
		fastcgi_param	CONTENT_TYPE	$content_type;
		fastcgi_param	CONTENT_LENGTH	$content_length;
		fastcgi_param	SERVER_PROTOCOL	$server_protocol;
		fastcgi_param	SERVER_PORT	$server_port;
		fastcgi_param	SERVER_NAME	$server_name;
	}
}

The hg_server.conf file will need a link from /etc/nginx/sites-enabled to its location in /etc/nginx/sites-available, assuming that you’re using the default nginx config which includes every server conf found in /etc/nginx/sites-available.

fab hgweb restart_nginx

To make deployment easy, I use fab, so that if I make any changes to hgweb.config or hg_server.conf, I can simply run fab hgweb restart_nginx. For starting hgwebdir.fcgi, we can use spawn-fcgi, which usually comes with lighttpd, so you’ll need that installed too.

hgweb copies hgweb.config and hgwebdir.fcgi to appropriate locations on the server, then starts the fastcgi process with a socket at /var/run/hgwebdir.sock.

restart_nginx copies hg_server.conf to the server and tells nginx to reload its config.

def hgweb():
    env.runpath = '/var/run'
    put('hgweb.config', '/tmp')
    put('hgwebdir.fcgi', '/tmp')
    sudo('mv /tmp/hgwebdir.fcgi /usr/local/bin/')
    sudo('chmod +x /usr/local/bin/hgwebdir.fcgi')
    sudo('mv /tmp/hgweb.config /etc/mercurial/hgweb.config')
    sudo('kill `cat %s/hgwebdir.pid`' % env.runpath)
    sudo('spawn-fcgi -f /usr/local/bin/hgwebdir.fcgi -s %s/hgwebdir.sock -P %s/hgwebdir.pid' % (env.runpath, env.runpath), user='www-data')

def restart_nginx():
    put('hg_server.conf', '/tmp/')
    sudo('mv /tmp/hg_server.conf /etc/nginx/sites-available/')
    sudo('killall -HUP nginx')

Once you’ve got these commands in your fabfile.py, you can run fab hgweb restart_nginx to deploy.

hgrc

Now that you’ve got hgwebdir.fcgi running (you can make sure it works by going to http://hg.DOMAIN.COM), you’ll probably want to customize the info about each repo by editing .hg/hgrc.

[web]
description = All about my repo
contacts = Me

And that’s it, you should now have a fast web-based browser for multiple repos 🙂

Erlang Release Handling with Fab and Reltools

You’ve already got a first target system installed, and now you’ve written some new code and want to deploy it. This article will show you how to setup make and fab commands that use reltools to build & install new releases.

Appup

Your code should be part of an OTP application structure. Additionally, you will need an appup file in the ebin/ directory for each application you want to upgrade. There’s a lot you can do in an appup file:

Once you’ve updated app files with the newest version and configuration and created appup files with all the necessary commands, you’re ready to create a new release.

Note: The app configuration will always be updated to the newest version, even if you have no appup commands.

Release

To create a new release, you’ll need a new rel file, which I’ll refer to as NAME-VSN.rel. VSN should be greater than your previous release version. My usual technique is to copy my latest rel file to NAME-VSN.rel, then update the release VSN and all the application versions.

Note: reltools assumes that the rel file will be in $ROOTDIR/releases/, where $ROOTDIR defaults to code:root_dir(). This path is also used below in the make and fab commands. You can pass a different value for $ROOTDIR, but releases/ is hard coded. This may change in the future, but for now your rel files must be in $ROOTDIR/releases/ if you want to use reltools.

Reltools

Before you finalize the new release, make sure reltools is in your code path. There 2 ways to do this:

  1. Make a copy of reltools and add it to your application.
  2. Clone elib and add it to your code path with erl -pa PATH/TO/elib/ebin.

If you choose option 1, be sure to include reltools in your app modules, and add it to your appup file with {add_module, reltools}.

But I’ll assume you want option 2 because it provides cleaner code separation and easier release handling. Keeping elib external means you can easily pull new code, and only need to add the elib application to your rel file with the latest vsn.

Make Upgrade

Now that you have a new release defined, and elib is in your code path, you’re ready to build release upgrade packages. Below is the make command I use to call reltools:make_upgrade("NAME-VSN"). Be sure to update PATH/TO/ to your particular code paths.

ERL=erl # use default erl command, but can override path on command line

src: FORCE
    @$(ERL) -pa lib/*/ebin -make # requires an Emakefile

upgrade: src
    @$(ERL) -noshell \                          # run erlang with no shell
        -pa lib/*/ebin \                        # include your local code repo
        -pa PATH/TO/elib/ebin \                 # include elib
        -pa PATH/TO/erlang/lib/*/ebin \         # include local erlang libs
        -run reltools make_upgrade $(RELEASE) \ # run reltools:make_upgrade
        -s init stop                            # stop the emulator when finished

FORCE: # empty rule to force run of erl -make

Using the above make rules, you can do make upgrade RELEASE=PATH/TO/releases/NAME-VSN to build a release upgrade package. Once you can do this locally, you can use fab to do remote release builds and installs. But in order to build a release remotely, you need to get the code onto the server. There are various ways to do this, the simplest being to clone your repo on the remote server(s), and push your updates to each one.

fab release build install

Below is an example fabfile.py for building and installing releases remotely using fab. Add your own hosts and roles as needed.

PATH/TO/TARGET should be the path to your first target system.

release is a separate command so that it you are only asked for NAME-VSN once, no matter how many hosts you build and install on.

build will run make upgrade RELEASE=releases/NAME-VSN on the remote system, using the target system’s copy of erl. Theoretically, you could build a release package once, then distribute it to each target system’s releases/ directory. But that requires each target system being exactly the same, with all the same releases and applications installed. If that’s the case, modify the above recipe to run build on a single build server, have it put the release package into all the other node’s releases/ directory, then run install on each node.

install uses _rpcall to run rpc:call(NODE@HOST, reltools, install_release, ["NAME-VSN"]). I’ve kept _rpcall separate so you can see how to define your own fab commands by setting env.mfa.

from fabric.api import env, prompt, require, run

env.erl = 'PATH/TO/TARGET/bin/erl'

def release():
    '''Prompt for release NAME-VSN. rel file must be in releases/.'''
    prompt('Specify release as NAME-VERSION:', 'release',
        validate=r'^\w+-\d+(\.\d+)*$')

def build():
    '''Build upgrade release package.'''
    require('release')
    run('cd PATH/TO/REPO && hg up && make upgrade ERL=%s RELEASE=releases/%s' % (env.erl, env.release))

def install():
    '''Install release to running node.'''
    require('release')
    env.mfa = 'reltools,install_release,["%s"]' % env.release
    _rpccall()

def _rpccall():
    require('mfa')
    evalstr = 'io:format(\"~p~n\", [rpc:call(NODE@%s, %s)])' % (env.host, env.mfa)
    # NOTE: local user must have same ~/.erlang.cookie as running nodes
    run("%s -noshell -sname fab -eval '%s' -s init stop" % (env.erl, evalstr))

Workflow

Once you’ve updated your Makefile and created fabfile.py, your workflow can be something like this:

  1. Write new application code.
  2. Update the app and appup files for each application to upgrade.
  3. Create a new rel file as releases/NAME-VSN.rel.
  4. Commit and push your changes.
  5. Run fab release build install.
  6. Enter NAME-VSN for your new release.
  7. Watch your system hot upgrade in real-time 🙂

Troubleshooting

Sometimes reltools:install_release(NAME-VSN) can fail, usually when the release_handler can’t find an older version of your code. In this case, your new release will be unpacked but not installed. You can see the state of all the known releases using release_handler:which_releases().. This can usually be fixed by removing old releases and trying again. Shell into your target system and do something like this (where OLDVSN is the VSN of a release marked as old):

See the release_handler manual for more information.

release_handler:remove_release("OLDVSN"). % repeat as necessary
release_handler:install_release("VSN").
release_handler:make_permanent("VSN").

Deploying Django with Mercurial, Fab and Nginx

Writing web apps with Django can be a lot of fun, but deploying them can be a chore, even if you’re using Apache. Here’s a setup I’ve been using that makes deployment fast and easy. This all assumes you’ve got sudo access on a remote server running Ubuntu or something similar.

Mercurial

This setup assumes you’ve got 2 mercurial repositories: 1 on your local machine, and 1 on the remote server you’re deploying to. In the remote repository, add the following to .hg/hgrc

[hooks]
changegroup = hg up

This makes mercurial run hg up whenever you push new code. Then in your local repo’s .hg/hgrc, make sure the default path is to your remote repo. Here’s an example

[paths]
default = ssh://user@domain.com/repo

Now when you run hg push, you don’t need to include the path to the repo, and your code will be updated immediately.

Django FastCGI Deployment

Since I’m using nginx instead of Apache, we’ll be deploying Django with FastCGI. Here’s an example script you can use to start and restart your Django FastCGI server. Add this script to your mercurial repo as run_fcgi.sh.

#!/bin/bash
PIDFILE="/tmp/django.pid"
SOCKET="/tmp/django.sock"

# kill current fcgi process if it exists
if [ -f $PIDFILE ]; then
    kill `cat -- $PIDFILE`
    rm -f -- $PIDFILE
fi

python manage.py runfcgi socket=$SOCKET pidfile=$PIDFILE method=prefork

Important note: the FastCGI socket file will need to be readable & writable by nginx worker processes, which run as the www-data user in Ubuntu. This will be handled by the fab restart command below, or you could add chmod a+w $SOCKET to the end of the above script.

Nginx FastCGI Proxy

Nginx is a great high performance web server with simple configuration. Here’s a simple example server config for proxying to your Django FastCGI process. Add this config to your mercurial repo as django.nginx.

server {
    listen 80;
    # change to your FQDN
    server_name YOUR.DOMAIN.COM;

    location / {
        # must be the same socket file as in the above fcgi script
        fastcgi_pass unix:/tmp/django.sock;
    }
}

On the remote server, make sure the following lines are in the http section of /etc/nginx/nginx.conf

include /etc/nginx/sites-enabled/*;
# fastcgi_params should contain a lot of fastcgi_param variables
include /etc/nginx/fastcgi_params;

You must also make sure there is a link in /etc/nginx/sites-enabled to your django.nginx config. Don’t worry if django.nginx doesn’t exist yet, it will once you run fab nginx the first time.

you@remote.ubuntu$ cd /etc/nginx/sites-enabled
you@remote.ubuntu$ sudo ln -s ../sites-available/django.nginx django.nginx

Python Fabric

Fab, or properly Fabric, is my favorite new tool. It’s designed specifically for making remote deployment simple and easy. You create a fabfile where each function is a fab command that can run remote and sudo commands on one or more remote hosts. So let’s deploy Django using fab. Here’s an example fabfile with 2 commands: restart and nginx. These commands should only be run after you’ve done a hg push.

config.fab_hosts = ['YOUR.DOMAIN.COM']
config.projdir = '/PATH/TO/YOUR/REMOTE/HG/REPO'

def restart():
    sudo('cd %(projdir)s; run_fcgi.sh', user='www-data', fail='abort')

def nginx():
    sudo('cp %(projdir)s/django.nginx /etc/nginx/sites-available/', fail='abort')
    sudo('killall -HUP nginx', fail='abort')

fab restart

You only need to run fab restart if you’ve changed the actual Django python code. Changes to templates or static files don’t require a restart and will be used automatically (because of the hg up changegroup hook). Executing run_fcgi.sh as the www-data user ensures that nginx can read & write the socket.

fab nginx

If you’ve changed your nginx server config, you can run fab nginx to install and reload the new server config without restarting the nginx server.

Wrap Up

Now that everything is setup, the next time you want to deploy some new code, it’s as simple as hg push && fab restart. And if you’ve only changed templates, all you need to do is hg push. I hope this helps make your Django development life easier. It has certainly done so for me 🙂